Method and device for detecting the motion state of a target

Abstract

The application provides a method and device for detecting a motion state of a target, comprising: acquiring poses of a vehicle at a plurality of collection time points and point clouds of the target at the plurality of collection time points; sampling the point clouds at the plurality of collection time points to obtain a plurality of data sets; acquiring a reference pose of each data set from the poses at the plurality of collection time points according to a collection time point corresponding to the point cloud in each data set; performing coordinate conversion on each data set by using the reference pose of each data set; identifying each converted data set by using a motion state detection model to obtain a motion state of the target at a collected collection time point. The method can eliminate errors between frames of point clouds and improve the accuracy of target motion state identification.

Description

Technical Field

[0001] This application relates to the field of autonomous driving, and in particular to a method and device for detecting the motion state of a target. Background Technology

[0002] Point cloud detection of targets is one of the key technologies in the field of autonomous driving perception. Whether it is the autonomous driving function on the vehicle or the automated labeling in the cloud, it is necessary to estimate the motion state (moving or stationary) of the target in order to perform further processing on the target's point cloud.

[0003] Determining the motion state of a target based on point clouds requires combining it with temporal data. This means collecting point clouds at continuous intervals for judgment. However, the errors between point cloud frames monitored at continuous intervals can be significant. For example, the position of the vehicle or target may be jittery, the orientation of the sensors on the vehicle may be incorrect, and the target tracking may be wrong. These factors can cause errors between point cloud frames, which can easily lead to misclassification of motion state. Summary of the Invention

[0004] This application provides a method and device for detecting the motion state of a target, in order to solve the technical problem that the motion state recognition of a target using continuous multi-frame point clouds is prone to misidentification due to errors between points in each frame.

[0005] One embodiment of this application provides a method for detecting the motion state of a target, comprising:

[0006] Acquire the pose of the data acquisition vehicle at multiple acquisition times and the point cloud of the target at multiple acquisition times;

[0007] Multiple datasets are obtained by sampling point clouds at multiple acquisition times; the reference pose of each dataset is obtained from the poses at multiple acquisition times.

[0008] Perform coordinate transformation on each dataset using the reference pose of each dataset;

[0009] The motion state detection model is used to identify each transformed dataset to obtain the motion state of the target at the sampling time.

[0010] In one embodiment, point clouds at multiple acquisition times are sampled to obtain multiple datasets, specifically including:

[0011] The sampling window is used to sample point clouds at multiple acquisition times according to the sampling step size to obtain multiple datasets; wherein the sampling step size is smaller than the length of the sampling window, so that the point clouds at the acquisition times being sampled are located in multiple datasets.

[0012] Accordingly, a motion state detection model is used to identify each transformed dataset to obtain the motion state of the target at the sampling time, specifically including:

[0013] The motion state detection model is used to identify each transformed dataset to obtain at least one identification result of the target at the sampling acquisition time.

[0014] Process at least one recognition result of the target at the sampling acquisition time and output the detected motion state of the target at the sampling acquisition time.

[0015] In one embodiment, acquiring the point cloud of the target at multiple acquisition times specifically includes:

[0016] Acquire raw point clouds and target contour data from multiple acquisition times collected by the acquisition vehicle.

[0017] The target's contour data at each acquisition time is used to process the original point cloud at the corresponding time to obtain the target's point cloud at each acquisition time.

[0018] In one embodiment, the method includes:

[0019] Clustering is performed on each transformed dataset to remove noisy data outside the clustered regions;

[0020] Accordingly, a motion state detection model is used to identify each transformed dataset to obtain the motion state of the target at the sampling time, specifically including:

[0021] The motion state detection model is used to identify each denoised sample set to obtain at least one identification result of the target at the sampling time.

[0022] In one embodiment, obtaining the reference pose of each dataset from poses at multiple acquisition times specifically includes:

[0023] The pose at the earliest acquisition time in the dataset is selected as the reference pose of the dataset.

[0024] In one embodiment, at least one identification result of the target at each acquisition time is processed to output the detected motion state of the target at each time, specifically including:

[0025] When there is only one recognition result at a certain acquisition time, the detected motion state of the target at the acquisition time is the recognition result;

[0026] When there are multiple recognition results at a certain acquisition time, the state value of the target's recognition results at the acquisition time is counted, and the state value with the most occurrences is taken as the target's detected motion state at the acquisition time.

[0027] One embodiment of this application provides a device for detecting the motion state of a target, comprising:

[0028] The acquisition module is used to acquire the pose of the acquisition vehicle at multiple acquisition times and the point cloud of the target at multiple acquisition times;

[0029] The processing module is used to sample point clouds at multiple acquisition times to obtain multiple datasets; obtain the reference pose of each dataset from the poses at multiple acquisition times; perform coordinate transformation on each dataset using the reference pose of each dataset; and use a motion state detection model to identify each transformed dataset to obtain the motion state of the target at the sampled acquisition time.

[0030] One embodiment of this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0031] The memory stores instructions that the computer executes;

[0032] The processor executes computer execution instructions stored in memory to implement the methods described in the above embodiments.

[0033] One embodiment of this application provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, are used to implement the methods described in the above embodiments.

[0034] One embodiment of this application provides a computer program product, including a computer program that, when executed by a processor, implements the methods described in the above embodiments.

[0035] The target motion state detection method and device provided in this application, after obtaining the point cloud of the target, sample the point cloud at multiple acquisition times to obtain multiple datasets, and then use the reference pose of each dataset to perform coordinate transformation on the point cloud in the dataset, and then use the coordinate transformed dataset to perform motion state recognition, which can eliminate the error between point clouds in each frame and improve the accuracy of target motion state recognition. Attached Figure Description

[0036] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0037] Figure 1 A flowchart of a target detection method provided in an embodiment of this application;

[0038] Figure 2 A schematic diagram illustrating the principle of sliding window sampling provided in this application;

[0039] Figure 3A schematic diagram of a target detection device provided in an embodiment of this application;

[0040] Figure 4 This is a schematic diagram of an electronic device provided in an embodiment of this application.

[0041] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0042] The exemplary embodiments will now be described in detail herein, and these embodiments are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0043] like Figure 1 As shown, Figure 1 An embodiment of this application provides a method for detecting the motion state of a target, the method comprising the following steps:

[0044] S101. Acquire the pose of the vehicle at multiple acquisition times and the point cloud of the target at multiple acquisition times.

[0045] The process involves using a data acquisition vehicle to continuously acquire data about the target, obtaining the vehicle's pose and the target's point cloud at multiple acquisition times. The pose includes the vehicle's position and attitude. The vehicle's position is represented by coordinates in a three-dimensional coordinate system, and its attitude is represented by pitch, roll, and yaw angles.

[0046] Taking the acquisition of data from a vehicle at n consecutive time points as an example, the target point cloud includes point clouds at n acquisition time points, namely point cloud P1, point cloud P2, ..., point cloud Pn. The pose of the acquisition vehicle includes poses at n acquisition time points, namely pose E1, pose E2, ..., pose En.

[0047] S102. Sample the point cloud at multiple acquisition times to obtain multiple datasets; obtain the reference pose of each dataset from the poses at multiple acquisition times.

[0048] In this process, conventional sampling methods can be used to sample point clouds at multiple acquisition times to obtain multiple datasets.

[0049] Taking the sampling to obtain m datasets as an example, the m datasets are divided into dataset Q1, dataset Q2, ..., dataset Qm. Each dataset contains multiple point clouds.

[0050] Taking a dataset with L point clouds as an example, the dataset contains L sampled acquisition times and L point clouds at the sampled acquisition times. According to the order of acquisition times, they are the first sampled acquisition time sp1, the first sampled acquisition time sp2, ..., the Lth sampled acquisition time spL.

[0051] As one example, a reference pose for each dataset is obtained from the poses of multiple acquisition times based on the acquisition times sampled in the dataset.

[0052] As one example, the pose at the earliest acquisition time in each dataset can be selected as the reference pose of the dataset.

[0053] Taking the dataset Q1 with 3 sampled acquisition times and the point cloud at the 3 sampled acquisition times as an example, from the first sampled acquisition time t1, the first sampled acquisition time t3, and the third sampled acquisition time t3, acquisition time t1 is earlier than acquisition time t2, and acquisition time t2 is earlier than acquisition time t3. Therefore, the pose at acquisition time t1 is selected as the reference pose of dataset Q1.

[0054] S103. Perform coordinate transformation on each dataset using the reference pose of each dataset.

[0055] The data collection is based on the coordinate system on the collection vehicle. The reference pose of each dataset is used to perform coordinate transformation on each dataset to obtain the dataset in the reference coordinate system.

[0056] In some embodiments, when performing coordinate transformation on the point cloud of the i-th sampled acquisition time spi in the dataset, the pose of the i-th sampled acquisition time spi and the reference pose are used to perform coordinate transformation on the point cloud of the i-th sampled acquisition time spi. i is a positive integer, 1≤i≤L.

[0057] Using the vehicle pose at acquisition time t1 in dataset Q1 as an example of the reference pose for dataset Q1, which includes point clouds at acquisition time t1, t3, and t5, we perform coordinate transformation on the point cloud at acquisition time t3 using the vehicle pose at acquisition time t3 and the reference pose of dataset Q1 to obtain the point cloud data at acquisition time t3 in the reference coordinate system.

[0058] S104. Use the motion state detection model to identify each transformed dataset to obtain the motion state of the target at the sampling time.

[0059] The motion state detection model can use a conventional machine learning model. Each point cloud in the dataset after coordinate transformation is input into the motion state detection model to obtain the motion state of the target at the sampling time.

[0060] The sampling time is the time corresponding to the sampled data when sampling the point cloud at each sampling time.

[0061] In the above technical solution, after obtaining the point cloud of the target, multiple datasets are obtained by sampling the point cloud at multiple acquisition times. Then, the reference pose of each dataset is used to perform coordinate transformation on the point cloud in the dataset. Finally, the coordinate-transformed dataset is used to perform motion state recognition, which can eliminate the error between point clouds in each frame and improve the accuracy of target motion state recognition.

[0062] In some embodiments, to obtain point clouds of a target at multiple acquisition times, the original point clouds collected by the acquisition vehicle at multiple acquisition times and the target contour data at multiple acquisition times are acquired. The original point cloud at the corresponding acquisition time is processed using the target contour data at each acquisition time to finally obtain the point cloud of the target at each acquisition time.

[0063] Existing methods can be used to acquire target contour data at each acquisition moment. The target contour data includes the center point pose, contour length, contour width, and contour height, and can be expressed by the following formula:

[0064]

[0065] in, These represent the coordinates of the center point, These represent the pitch angle, roll angle, and yaw angle at the center point, respectively. i ,w i ,h i These represent the length, width, and height of the outline, respectively.

[0066] Furthermore, based on the target contour data and the original point cloud acquired at the same acquisition time, the target contour and the original point cloud are displayed, and the data located within the target contour are extracted from the original point cloud as the target point cloud. Taking the acquisition of original point clouds at n acquisition times as an example, they are respectively labeled as original point cloud O1, original point cloud O2, ..., original point cloud On. The target contours at the n acquisition times are target contour C1, target contour C2, ..., target contour Cn.

[0067] Extract data located within the target contour C1 from the original point cloud O1, and use this as the target's point cloud at acquisition time 1. Extract data located within the target contour C2 from the original point cloud Q2, and use this as the target's point cloud at acquisition time 2. Continue this process, extracting data located within the target contour Cn from the original point cloud Qn, and using this as the target's point cloud at acquisition time n.

[0068] In this technical solution, the raw point cloud at each acquisition moment is processed using the target contour data at the corresponding moment, which can accurately obtain the target point cloud.

[0069] In some embodiments, when sampling point clouds at multiple acquisition times, a sampling window is used to sample point clouds at multiple acquisition times according to the sampling step size to obtain multiple datasets.

[0070] Assuming the time sampling window size is w and the sampling step size is s, after sampling the point cloud data at n acquisition times, the number of data sets obtained is:

[0071]

[0072] Where n is the number of data collection times.

[0073] Taking a dataset containing s acquisition times as an example, the j-th dataset Qj is:

[0074] {t (j-1)s+1 ,t (j-1)s+2 ,...,t js}

[0075] Among them, t (j-1)s+1 Let t be the first sampling time in the dataset Qj. (j-1)s+2 This refers to the first sampling time in the dataset Qj, and so on, t js Let s be the sampling time of the s-th sampled data in dataset Qj.

[0076] The last dataset is:

[0077] {t (m-1)s+1 ,...,t n}

[0078] Among them, t (m-1)s+1 Let t be the first sampling time in the dataset Qm. (m-1)s+2 This refers to the second sampling time in the dataset Qm, and so on, t n Let s be the sampling time of the s-th sampled data in dataset Qm.

[0079] In one embodiment, the sampling step size is smaller than the length of the sampling window, so that the point cloud at the sampling time is located in multiple datasets. For example... Figure 2As shown, Figure 2 The schematic diagram of the sliding window sampling principle provided in this application is as follows: taking the length of the sampling window as three times the sampling step size as an example, the point cloud at the black filling area will appear in the data set of the (i-1)th sampling, the i-th sampling, and the (i+1)th sampling after the (i-1)th sampling, the i-th sampling, and the (i+1)th sampling.

[0080] In some embodiments, after obtaining multiple datasets through sampling using the above method, coordinate transformation is performed on each dataset, and clustering is performed on each transformed dataset to remove noisy data outside the clustering regions. Conventional clustering algorithms can be used to perform clustering on each transformed dataset.

[0081] During the sampling process, because the sampling window size is larger than the sampling step size, a single acquisition moment may be sampled multiple times, appearing in multiple datasets. By using a motion state detection model to identify each denoised dataset, at least one identification result of the target at the same sampled acquisition moment can be obtained. The at least one identification result of the target at the sampled acquisition moment is processed to output the detected motion state of the target at the sampled acquisition moment.

[0082] In some embodiments, when there is only one identification result at a certain acquisition time, the detected motion state of the target at the acquisition time is the identification result.

[0083] In some embodiments, when there are multiple recognition results at a certain acquisition time, the state values ​​of the recognition results of the target at the acquisition time are counted, and the state value with the most occurrences is taken as the detection motion state of the target at the acquisition time.

[0084] The data is sampled only once at acquisition time t1 and is located in dataset Q1. When processing dataset Q1 using the motion state recognition model, a recognition result is obtained at acquisition time t5. If the state value of the recognition result is "motion", then the motion state is detected as motion. If the state value of the recognition result is "stationary", then the motion state is detected as stationary.

[0085] Data was sampled multiple times at time t5, falling within datasets Q3, Q4, and Q5. When a motion state recognition model was used to process datasets Q3, Q4, and Q5 respectively, three recognition results were obtained at time t5. Each recognition result was either motion or stationary. If the three results were categorized as motion, motion, or stationary, then the motion state was detected as stationary. If the number of occurrences of motion was the highest, then the motion state was detected as motion. Conversely, if the number of occurrences of stationary was the highest, then the motion state was detected as stationary.

[0086] In the above technical solution, point clouds at multiple acquisition times are sampled using a sliding window, with the sampling window being larger than the sampling step size. This allows the point cloud at the same acquisition time to be sampled multiple times and appear in multiple datasets. Then, using each dataset as a unit, the reference pose of each dataset is used to perform coordinate transformation on the point cloud in the dataset. The coordinate-transformed dataset is then used for motion state recognition. At least one recognition result at the same acquisition time is used. After statistical analysis of the recognition results, the detection motion state of the target is determined. This can eliminate errors between point clouds in different frames and improve the accuracy of target motion state recognition.

[0087] like Figure 3 As shown, Figure 3 This is a schematic diagram of a target detection device according to an embodiment of this application. An embodiment of this application provides a target motion state detection device 200, comprising:

[0088] The acquisition module 201 is used to acquire the pose of the acquisition vehicle at multiple acquisition times and the point cloud of the target at multiple acquisition times;

[0089] The processing module 202 is used to sample point clouds at multiple acquisition times to obtain multiple datasets; obtain the reference pose of each dataset from the poses at multiple acquisition times; perform coordinate transformation on each dataset using the reference pose of each dataset; and use a motion state detection model to identify each transformed dataset to obtain the motion state of the target at the sampled acquisition time.

[0090] In some embodiments, the processing module 202 is specifically used for:

[0091] The sampling window is used to sample point clouds at multiple acquisition times according to the sampling step size to obtain multiple datasets; wherein the sampling step size is smaller than the length of the sampling window, so that the point clouds at the acquisition times being sampled are located in multiple datasets.

[0092] Accordingly, a motion state detection model is used to identify each transformed dataset to obtain the motion state of the target at the sampling time, specifically including:

[0093] The motion state detection model is used to identify each transformed dataset to obtain at least one identification result of the target at the sampling acquisition time.

[0094] Process at least one recognition result of the target at the sampling acquisition time and output the detected motion state of the target at the sampling acquisition time.

[0095] In one embodiment, the acquisition module 201 is specifically used for:

[0096] Acquire raw point clouds and target contour data from multiple acquisition times collected by the acquisition vehicle.

[0097] The target's contour data at each acquisition time is used to process the original point cloud at the corresponding time to obtain the target's point cloud at each acquisition time.

[0098] In one embodiment, the processing module 202 further includes:

[0099] Clustering is performed on each transformed dataset to remove noisy data outside the clustered regions;

[0100] The motion state detection model is used to identify each denoised sample set to obtain at least one identification result of the target at the sampling time.

[0101] In one embodiment, the processing module 202 is specifically used for:

[0102] The pose at the earliest acquisition time in the dataset is selected as the reference pose of the dataset.

[0103] In one embodiment, the processing module 202 is specifically used for:

[0104] When there is only one recognition result at a certain acquisition time, the detected motion state of the target at the acquisition time is the recognition result;

[0105] When there are multiple recognition results at a certain acquisition time, the state value of the target's recognition results at the acquisition time is counted, and the state value with the most occurrences is taken as the target's detected motion state at the acquisition time.

[0106] like Figure 4 As shown, one embodiment of this application provides an electronic device 300, which includes a memory 301 and a processor 302.

[0107] Among them, memory 301 is used to store computer instructions that can be executed by the processor;

[0108] The processor 302 implements the various steps of the method in the above embodiments when executing computer instructions. For details, please refer to the relevant descriptions in the foregoing method embodiments.

[0109] Optionally, the memory 301 can be either independent or integrated with the processor 302. When the memory 301 is configured independently, the electronic device also includes a bus for connecting the memory 301 and the processor 302.

[0110] This application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the methods described above.

[0111] This application also provides a computer program product, including computer instructions that, when executed by a processor, implement the various steps in the methods described above.

[0112] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered illustrative only, and the true scope and spirit of this application are indicated by the following claims.

[0113] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for detecting the motion state of a target, characterized in that, include: Acquire the pose of the data acquisition vehicle at multiple data acquisition times and the point cloud of the target at multiple data acquisition times; The point clouds at the multiple acquisition times are sampled using a sampling window with a sampling step size to obtain multiple datasets; wherein the sampling step size is less than the length of the sampling window, so that the point clouds at the sampled acquisition times are located in the multiple datasets; the pose at the earliest acquisition time in the dataset is selected as the reference pose of the dataset. Coordinate transformation is performed on each dataset using the reference pose of each dataset; The motion state detection model is used to identify each transformed dataset to obtain the motion state of the target at the sampling time.

2. The detection method according to claim 1, characterized in that, The motion state detection model is used to identify each transformed dataset to obtain the motion state of the target at the sampled acquisition time, specifically including: The motion state detection model is used to identify each transformed dataset to obtain at least one identification result of the target at the sampling acquisition time. The system processes at least one identification result of the target at the sampling time and outputs the detected motion state of the target at the sampling time.

3. The detection method according to claim 1 or 2, characterized in that, Obtaining the point cloud of the target at multiple acquisition times specifically includes: Obtain the original point cloud data collected by the acquisition vehicle at multiple acquisition times and the contour data of the target at multiple acquisition times; The original point cloud at each acquisition time is processed using the contour data of the target at each acquisition time to obtain the point cloud of the target at each acquisition time.

4. The detection method according to claim 2, characterized in that, The method includes: Clustering is performed on each transformed dataset to remove noisy data outside the clustered regions; Accordingly, a motion state detection model is used to identify each transformed dataset to obtain the motion state of the target at the sampling time, specifically including: The motion state detection model is used to identify each denoised sample set to obtain at least one identification result of the target at the sampling time.

5. The method according to claim 2, characterized in that, Process at least one recognition result of the target at each acquisition time, and output the detected motion state of the target at each time, specifically including: When there is only one identification result at a certain acquisition time, the detected motion state of the target at that acquisition time is the identification result; When there are multiple recognition results at a certain acquisition time, the state value of the recognition result of the target at that acquisition time is counted, and the state value with the most occurrences is taken as the detection motion state of the target at that acquisition time.

6. A device for detecting the motion state of a target, characterized in that, include: The acquisition module is used to acquire the pose of the acquisition vehicle at multiple acquisition times and the point cloud of the target at multiple acquisition times; The processing module is configured to sample point clouds at multiple acquisition times using a sampling window with a sampling step size to obtain multiple datasets; wherein the sampling step size is less than the length of the sampling window, so that the point clouds at the sampled acquisition times are located in multiple datasets; select the pose at the earliest acquisition time in the dataset as the reference pose of the dataset; perform coordinate transformation on each dataset using the reference pose of each dataset; and use a motion state detection model to identify each transformed dataset to obtain the motion state of the target at the sampled acquisition time.

7. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 5.

9. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 5.

Images