Motion recognition method of an object and related apparatus

Abstract

The application discloses a motion recognition method of an object and a related device. The method comprises the following steps: merging first point cloud data and second point cloud data to obtain third point cloud data; determining a projection direction corresponding to the third point cloud data; projecting the third point cloud data onto the projection direction to obtain fourth point cloud data; and determining whether a target object corresponding to the point cloud data moves based on the fourth point cloud data. In this way, the calculation complexity of determining a moving target can be simplified, and the robustness is high.

Description

Technical Field

[0001] This application relates to the field of point cloud processing technology, and in particular to a method and apparatus for motion recognition of objects. Background Technology

[0002] Point cloud data is typically acquired by radar sensors and can be used to identify moving targets. For example, in the field of autonomous driving, the primary task of the perception system is to use radar sensors to collect point cloud data to perceive the target's position, speed, category, and behavior prediction, with the detection and tracking of moving targets being paramount. If the perception of a moving target is flawed—for instance, misidentifying a stationary target A as a moving one—the subsequent trajectory planning module of the autonomous driving system might plan a trajectory at position A, potentially leading to a traffic accident. Similarly, misidentifying a moving target as static could trigger sudden braking, impacting the passenger experience. Summary of the Invention

[0003] To address the aforementioned issues, this application provides a motion recognition method and related apparatus for objects, which can simplify the computational complexity of determining moving targets and has high robustness.

[0004] To solve the above-mentioned technical problems, one technical solution adopted in this application is: to provide a motion recognition method for an object, the method comprising: merging a first point cloud data and a second point cloud data to obtain a third point cloud data; determining the projection direction corresponding to the third point cloud data; projecting the third point cloud data onto the projection direction to obtain a fourth point cloud data; and determining whether the target object corresponding to the point cloud data is moving based on the fourth point cloud data.

[0005] Determining the projection direction corresponding to the third point cloud data includes: determining the covariance matrix corresponding to the third point cloud data; determining the eigenvalues ​​and corresponding eigenvectors of the covariance matrix; determining the eigenvector matrix based on the eigenvectors, and using the eigenvector matrix as the projection direction.

[0006] The step of determining whether the target object corresponding to the point cloud data is moving based on the fourth point cloud data includes: determining a first vector of the first point cloud data in the projection direction and a second vector of the second point cloud data in the projection direction based on the fourth point cloud data; and determining whether the target object corresponding to the point cloud data is moving based on the first vector and the second vector.

[0007] The process of determining whether the target object corresponding to the point cloud data is moving based on the first vector and the second vector includes: determining the intersection-union ratio (IU / U) of the first vector and the second vector; and determining whether the target object is moving based on the IU / U.

[0008] The process of determining the first vector of the first point cloud data in the projection direction and the second vector of the second point cloud data in the projection direction includes: obtaining the time sequence number of each data point in the fourth point cloud data; and determining the corresponding first vector and second vector based on data points with the same time sequence number.

[0009] The process of merging the first point cloud data and the second point cloud data to obtain the third point cloud data includes: projecting the first point cloud data onto the second point cloud data to obtain the third point cloud data.

[0010] Before determining the projection direction corresponding to the third point cloud data, the process includes: downsampling the third point cloud data; determining the projection direction corresponding to the third point cloud data includes: determining the projection direction corresponding to the downsampled third point cloud data.

[0011] To solve the above-mentioned technical problems, another technical solution adopted in this application is: to provide an on-board control system, which includes: a radar sensor for collecting first point cloud data and second point cloud data; and a processor connected to the radar sensor for implementing the method provided by the above technical solution.

[0012] To solve the above-mentioned technical problems, another technical solution adopted in this application is to provide an unmanned vehicle, which includes an on-board control system as provided by the above technical solution.

[0013] To solve the above-mentioned technical problems, another technical solution adopted in this application is to provide a computer-readable storage medium for storing a computer program, which, when executed by a processor, is used to implement the method provided by the above-mentioned technical solution.

[0014] The beneficial effects of the embodiments of this application are as follows: Unlike existing technologies, the motion recognition method for objects provided in this application includes: merging first point cloud data and second point cloud data to obtain third point cloud data; determining the projection direction corresponding to the third point cloud data; projecting the third point cloud data onto the projection direction to obtain fourth point cloud data; and determining whether the target object corresponding to the point cloud data is moving based on the fourth point cloud data. By performing dimensionality reduction processing on the three-dimensional point cloud data in the above manner, the computational complexity of determining moving targets is simplified, resulting in higher robustness. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0016] Figure 1 This is a flowchart illustrating the first embodiment of the motion recognition method for objects provided in this application;

[0017] Figure 2 This is a flowchart illustrating the second embodiment of the motion recognition method for objects provided in this application;

[0018] Figure 3 This is a flowchart illustrating an embodiment of step 25 provided in this application;

[0019] Figure 4 This is a diagram illustrating the intersection-union ratio provided in this application;

[0020] Figure 5 This is a flowchart illustrating the third embodiment of the motion recognition method for objects provided in this application;

[0021] Figure 6 This is a schematic diagram of the structure of an embodiment of the vehicle control system provided in this application;

[0022] Figure 7 This is a schematic diagram of the structure of an embodiment of the driverless vehicle provided in this application;

[0023] Figure 8 This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It is understood that the specific embodiments described herein are only for explaining this application and not for limiting it. Furthermore, it should be noted that, for ease of description, only the parts related to this application are shown in the accompanying drawings, not all structures. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0025] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0026] See Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the motion recognition method for objects provided in this application. The method includes:

[0027] Step 11: Merge the first point cloud data and the second point cloud data to obtain the third point cloud data.

[0028] The first and second point cloud data are collected from the same object at different times. For example, the second point cloud data is collected at the next moment after the first point cloud data. The target object in the point cloud data could be a human body, a tree, a lane, other vehicles, a building, etc.

[0029] In some embodiments, point cloud data is acquired using a radar sensor. The radar sensor can be a lidar, such as a mechanical lidar or a solid-state lidar. Therefore, the radar sensor can acquire corresponding point cloud data at different times. For example, the second point cloud data can be acquired at time t-1, and the first point cloud data can be acquired at time t.

[0030] When the radar sensor's acquisition frequency is high, the interval between each moment is small. Therefore, point cloud data acquired at adjacent moments can be used to determine whether the target object corresponding to these point cloud data is in a moving state. Target objects that are in a moving state are identified as moving targets.

[0031] In some embodiments, since the first point cloud data and the second point cloud data are collected at different times, there may be positional differences. Therefore, the first position information of the autonomous vehicle corresponding to the first point cloud data and the second position information of the autonomous vehicle corresponding to the second point cloud data can be obtained. Coordinate system transformation can then be performed on the first or second point cloud data using the second and first position information. Because the autonomous vehicle itself is also moving during operation, coordinate transformation based on the autonomous vehicle's pose is necessary to eliminate its own errors.

[0032] At this point, the second location information can be used as a reference, and the coordinates of the first point cloud data can be transformed based on the first location information.

[0033] First, determine the transformation relationship between the first and second positional information, such as by determining the transformation matrix. Then, use the transformation matrix to perform coordinate transformation on the first point cloud data to obtain the coordinates of the second positional information corresponding to the first point cloud data.

[0034] Since the second point cloud data is collected under the second position information, the first point cloud data after coordinate system transformation can be projected into the second point cloud data, and the coordinate system is the same.

[0035] Step 12: Determine the projection direction corresponding to the third point cloud data.

[0036] Before step 12, the third point cloud data can be downsampled. Downsampling reduces the number of data points in the third point cloud data, reduces the amount of subsequent data computation, and can also filter out noise in the point cloud data.

[0037] Then step 12 can determine the projection direction corresponding to the downsampled third point cloud data.

[0038] In some embodiments, the covariance matrix corresponding to the third point cloud data can be determined; then the eigenvalues ​​and corresponding eigenvectors of the covariance matrix can be determined; the eigenvector matrix can be determined based on the eigenvectors, and the eigenvector matrix can be used as the projection direction.

[0039] Specifically, the point cloud data is organized into an n x m matrix X according to the number of data points and the dimension. Here, n represents the dimension, and m represents the number of data points. Since the point cloud data is three-dimensional, n equals 3, and each row can represent one-dimensional data of the point cloud. For example, the first row represents the x-axis coordinates, the second row represents the y-axis coordinates, and the third row represents the z-axis coordinates. The three rows of data in each column form the coordinates of a single data point.

[0040] To obtain matrix X', we perform zero-mean subtraction on each row of matrix X, which involves subtracting the mean of that row.

[0041] Calculate the covariance matrix C using matrix X'. Where,

[0042] Then, the eigenvalues ​​λ and the corresponding eigenvectors c of the covariance matrix C are calculated.

[0043] Arrange the eigenvectors c into a matrix from top to bottom according to the magnitude of their corresponding eigenvalues ​​λ, and take the first k rows to form a matrix p. If the dimension is reduced to one dimension, then k = 1.

[0044] The fourth point cloud data is determined using matrices p and X. That is, Y = PX, where Y represents the fourth point cloud data.

[0045] Step 13: Project the third point cloud data onto the projection direction to obtain the fourth point cloud data.

[0046] By projecting the third point cloud data onto the projection direction, the 3D point cloud data can be reduced to 1D point cloud data. That is, the fourth point cloud data at this time is 1D data.

[0047] Step 14: Determine whether the target object corresponding to the point cloud data is moving based on the fourth point cloud data.

[0048] In some embodiments, the fourth point cloud data can be clustered, and then the clustered fourth point cloud data can be classified into point cloud clusters corresponding to the first point cloud data and point cloud clusters corresponding to the second point cloud data. Then, based on the two types of point cloud clusters, it can be determined whether the corresponding target object is moving.

[0049] In this embodiment, by merging the first point cloud data and the second point cloud data, a third point cloud data is obtained; the projection direction corresponding to the third point cloud data is determined; the third point cloud data is projected onto the projection direction to obtain a fourth point cloud data; and the method of determining whether the target object corresponding to the point cloud data is moving based on the fourth point cloud data can reduce the dimensionality of the three-dimensional point cloud data, thereby simplifying the computational complexity of determining the moving target and exhibiting high robustness.

[0050] See Figure 2 , Figure 2 This is a flowchart illustrating a second embodiment of the motion recognition method for objects provided in this application. The method includes:

[0051] Step 21: Merge the first point cloud data and the second point cloud data to obtain the third point cloud data; the second point cloud data is collected at the next moment after the first point cloud data.

[0052] In some embodiments, the first point cloud data and the second point cloud data may have positional differences because they were collected at different times. Therefore, the first point cloud data is projected onto the second point cloud data to obtain the third point cloud data. This method eliminates the error between the point cloud data.

[0053] Step 22: Determine the projection direction corresponding to the third point cloud data.

[0054] Steps 21 and 22 have the same or similar technical solutions as any of the above embodiments, and will not be described in detail here.

[0055] In some embodiments, the PCA (Principal Component Analysis) algorithm can be used to determine the projection direction.

[0056] Step 23: Project the third point cloud data onto the projection direction to obtain the fourth point cloud data.

[0057] By projecting the third point cloud data onto the projection direction, the 3D point cloud data can be reduced to 1D point cloud data. That is, the fourth point cloud data at this time is 1D data.

[0058] Step 24: Based on the fourth point cloud data, determine the first vector of the first point cloud data in the projection direction, and determine the second vector of the second point cloud data in the projection direction.

[0059] The first and second point cloud data each contain many data points. Each data point, when projected onto the projection direction, will also form a corresponding data point. These data points can then form a vector.

[0060] In some embodiments, after obtaining the fourth point cloud data, the time sequence number of each data point in the fourth point cloud data can be obtained. Based on data points with the same time sequence number, the corresponding first vector and second vector are determined.

[0061] In some embodiments, all data points with the same time sequence number are sorted numerically, and noise at both ends of the sorting is removed to obtain the maximum and minimum values ​​of the data points corresponding to that time sequence number. The maximum and minimum values ​​are then used to determine the corresponding vector.

[0062] It is understandable that the acquisition time can be used as identification information for each point cloud data. After projection transformation, the identification information can be used for corresponding classification to distinguish the first vector corresponding to the first point cloud data and the second vector corresponding to the second point cloud data.

[0063] Step 25: Determine whether the target object corresponding to the point cloud data is moving based on the first vector and the second vector.

[0064] Since vectors have magnitudes, we can determine whether the target object corresponding to the vector is moving based on the magnitudes of the first and second vectors and their overlap.

[0065] In some embodiments, see Figure 3 Step 25 can be the following process:

[0066] Step 251: Determine the intersection-union ratio of the first vector and the second vector.

[0067] Step 252: Determine whether the corresponding target object is moving based on the intersection-union ratio.

[0068] Combination Figure 4 Explanation:

[0069] For example, the size of the first vector E is the line segment x1x2, and the size of the second vector F is the line segment y1y2.

[0070] Therefore, the intersection of the first vector E and the second vector F is E∩F, which is the line segment y1x2. The union of the first vector E and the second vector F is E∪F, which is the line segment x1y2. Then the intersection-union ratio can be calculated.

[0071] A higher intersection-to-union (IoU) ratio indicates a higher degree of overlap, suggesting a greater probability that the target object is not moving. Conversely, a lower IoU ratio indicates a lower degree of overlap, suggesting a greater probability that the target object is moving. Therefore, an IoU threshold can be set. If the IoU ratio obtained from determining overlap is greater than the threshold, the target object is determined to be a fixed object and has not moved. If the IoU ratio obtained from determining overlap is less than or equal to the threshold, the target object is determined to be moving.

[0072] In this embodiment, by reducing the dimensionality of the three-dimensional point cloud data and then determining the moving target using a one-dimensional vector at adjacent time points, the computational complexity of determining the moving target can be simplified, and the system exhibits high robustness.

[0073] See Figure 5 , Figure 5 This is a flowchart illustrating a third embodiment of the motion recognition method for objects provided in this application. The method includes:

[0074] Step 51: Merge the first point cloud data and the second point cloud data to obtain the third point cloud data; the second point cloud data is collected at the next moment after the first point cloud data.

[0075] The first point cloud data and the second point cloud data were collected from the same object at different times.

[0076] Step 52: Determine the projection direction corresponding to the third point cloud data.

[0077] Step 53: Project the third point cloud data onto the projection direction to obtain the fourth point cloud data.

[0078] Steps 51-53 have the same or similar technical solutions as any of the above embodiments, and will not be described in detail here.

[0079] Step 54: Obtain the time sequence number of each data point in the fourth cloud data.

[0080] When collecting point cloud data, each data point in the point cloud has a corresponding time sequence number. For example, if the second point cloud data is collected at time t-1, then the time sequence number of the data point in the second point cloud data can be set to t-1. If the first point cloud data is collected at time t, then the time sequence number of the data point in the first point cloud data can be set to t.

[0081] Therefore, the time sequence number of each data point in the fourth cloud data can be obtained.

[0082] Step 55: Determine the corresponding first vector and second vector based on data points with the same time sequence number.

[0083] Data points with the same time sequence number are grouped into one category. For example, they can be classified according to time sequence number t-1 and time sequence number t, resulting in a first vector and a second vector. For instance, if data points A, B, C, and D have a time sequence number of t-1, then they belong to the same category, corresponding to the first vector. Similarly, if data points E, F, G, and H have a time sequence number of t, then they belong to the same category, corresponding to the second vector.

[0084] In some embodiments, all data points with the same time sequence number are sorted numerically, and noise at both ends of the sorting is removed to obtain the maximum and minimum values ​​of the data points corresponding to that time sequence number. The maximum and minimum values ​​are then used to determine the corresponding vector.

[0085] It is understandable that the acquisition time can be used as identification information for each point cloud data. After projection transformation, the identification information can be used for corresponding classification to distinguish the first vector corresponding to the first point cloud data and the second vector corresponding to the second point cloud data.

[0086] Step 56: Determine whether the target object corresponding to the point cloud data is moving based on the first vector and the second vector.

[0087] Step 56 has the same or similar technical solution as any of the above embodiments, and will not be described in detail here.

[0088] In some embodiments, PCA-IOU can be used to determine whether the target object corresponding to the point cloud data is moving.

[0089] In this embodiment, by reducing the dimensionality of the three-dimensional point cloud data and then determining the moving target using a one-dimensional vector at adjacent time points, the computational complexity of determining the moving target can be simplified, and the system exhibits high robustness.

[0090] See Figure 6 , Figure 6 This is a schematic diagram of an embodiment of the vehicle control system provided in this application. The vehicle control system 60 includes a radar sensor 61 and a processor 62.

[0091] The radar sensor 61 is used to collect first point cloud data and second point cloud data. The processor 82 is connected to the radar sensor 61 and is used to execute the following steps:

[0092] The first and second point cloud data are merged to obtain the third point cloud data; the projection direction corresponding to the third point cloud data is determined; the third point cloud data is projected onto the projection direction to obtain the fourth point cloud data; based on the fourth point cloud data, it is determined whether the target object corresponding to the point cloud data is moving.

[0093] It is understood that the processor 62 is also used to implement the technical solutions of any of the above embodiments, which will not be elaborated here.

[0094] The vehicle control system 60 is able to determine the driving route and speed for the next moment based on the determined moving target.

[0095] In other embodiments, the processor 62 is also connected to a memory (not shown). The memory is used to store computer programs, which, when executed by the processor 62, are used to implement the technical solutions of any of the above embodiments, and will not be elaborated here.

[0096] See Figure 7 , Figure 7 This is a schematic diagram of an embodiment of the driverless vehicle provided in this application. The driverless vehicle 70 includes an onboard control system 60.

[0097] See Figure 8 , Figure 8 This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. The computer-readable storage medium 80 is used to store a computer program 81, which, when executed by a processor, implements the following methods:

[0098] The first and second point cloud data are merged to obtain the third point cloud data; the second point cloud data is acquired at the next moment after the first point cloud data; the projection direction corresponding to the third point cloud data is determined; the third point cloud data is projected onto the projection direction to obtain the fourth point cloud data; based on the fourth point cloud data, it is determined whether the target object corresponding to the point cloud data is moving.

[0099] It is understood that when computer program 81 is executed by the processor, it is also used to implement the technical solutions of any of the above embodiments, which will not be elaborated here.

[0100] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of circuits or units is merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0101] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0102] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0103] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural changes made based on the description and drawings of this application, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for motion recognition of an object, characterized in that, The method includes: The first and second point cloud data are merged to obtain the third point cloud data; Determine the covariance matrix corresponding to the third point cloud data; Determine the eigenvalues ​​and corresponding eigenvectors of the covariance matrix; Based on the feature vectors, a feature vector matrix is ​​determined, and the feature vector matrix is ​​used as the projection direction of the third point cloud data; The third point cloud data is projected onto the projection direction to obtain the fourth point cloud data; Based on the fourth point cloud data, a first vector of the first point cloud data in the projection direction is determined, and a second vector of the second point cloud data in the projection direction is determined. Determine the intersection-union ratio of the first vector and the second vector; Based on the intersection-union ratio, it is determined whether the corresponding target object is moving.

2. The method according to claim 1, characterized in that, Determining the first vector of the first point cloud data in the projection direction and determining the second vector of the second point cloud data in the projection direction includes: Obtain the time sequence number of each data point in the fourth point cloud data; The first vector and the second vector are determined based on data points with the same time sequence number.

3. The method according to claim 1, characterized in that, The process of merging the first point cloud data and the second point cloud data to obtain the third point cloud data includes: The first point cloud data is projected onto the second point cloud data to obtain the third point cloud data.

4. The method according to claim 1, characterized in that, Before determining the projection direction corresponding to the third point cloud data, the following steps are included: The third point cloud data is downsampled; Determining the projection direction corresponding to the third point cloud data includes: Determine the projection direction corresponding to the downsampled third point cloud data.

5. A vehicle-mounted control system, characterized in that, The vehicle control system includes: Radar sensor, used to collect first point cloud data and second point cloud data; A processor, connected to the radar sensor, is configured to implement the method as described in any one of claims 1-4.

6. An unmanned vehicle, characterized in that, The unmanned vehicle includes the onboard control system as described in claim 5.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program, which, when executed by a processor, is used to implement the method as described in any one of claims 1-4.

Images