A deep learning-based millimeter wave radar point cloud data iterative registration method

Abstract

The application discloses a kind of based on deep learning's millimeter wave radar point cloud data iterative registration method, this method includes: obtaining first to be matched point cloud data and second to be matched point cloud data;Determine the n different neighborhood of each point in these two point cloud data;According to the spatial coordinates of each point, polar coordinates, the Doppler velocity of each neighborhood point of each point, the PPF vector between each point and each neighborhood point, and the point feature extraction network trained, determine the fusion point feature of each point;The spatial coordinates, Doppler velocity and point cloud mark of each point in these two point cloud data are input into the registration parameter prediction network trained, and the point cloud registration parameter is predicted;The point cloud mark of each point is used to indicate the point cloud data that the point belongs to;Based on point cloud registration parameter, the fusion point feature of each point in these two point cloud data, determine transformation parameter;Based on transformation parameter, these two point cloud data are registered.

Description

Technical Field

[0001] This invention belongs to the field of millimeter-wave radar technology, specifically relating to an iterative registration method for millimeter-wave radar point cloud data based on deep learning. Background Technology

[0002] Radar point cloud registration is used to convert data collected by frequency-modulated continuous wave (FMCW) scanning radars at different locations into a common coordinate system for optimal attitude estimation. The registration process includes key point detection of radar data, conversion of key points into radar point clouds, translation of point clouds obtained from different locations to the common coordinate system, attitude estimation, and superposition of multiple point clouds to make the point cloud denser and cover a wider area.

[0003] The current approach involves having a person walk in a straight line within the coverage area of ​​multiple millimeter-wave radars. Using the perspective of one radar as a reference, the equation y1 of the walking trajectory is obtained. In another radar, the equation y2 is obtained in the same way. The parameters needed to transform y2 to y1 are then obtained, and the rotation and translation equations of the two point clouds are derived using these parameters. However, this approach requires the person to walk in a straight line within the radar coverage area.

[0004] To address the issue of limited walking trajectories, current solutions utilize the movement trajectories of people in the environment to automatically estimate the position and orientation of multiple radars relative to a reference coordinate system. Standard Kalman filter-based tracking procedures are used, employing singular value decomposition (SVD) and least squares (LS) methods to obtain the relative position and orientation of any number of millimeter-wave radars.

[0005] Existing techniques that fuse radar point clouds based on human movement trajectories calculate the relative position and orientation of two radars. While accurate, these techniques have limitations, supporting only straight-line trajectories and only functioning correctly when tracking a single person. These limitations necessitate a dedicated calibration phase within the multi-radar network under controlled conditions, which is impractical and time-consuming in real-world applications. Although automated tracking programs address the issue of supporting only straight-line trajectories, calibration still relies on human movement along the radar's trajectory. In scenarios with multiple people, the accuracy and performance of this approach deteriorate drastically. In other words, traditional registration schemes are unsuitable for millimeter-wave radar point cloud registration in multi-target detection scenarios. Summary of the Invention

[0006] To address the aforementioned problems in the existing technology, this invention provides an iterative registration method for millimeter-wave radar point cloud data based on deep learning.

[0007] The technical problem to be solved by this invention is achieved through the following technical solution:

[0008] This invention provides an iterative registration method for millimeter-wave radar point cloud data based on deep learning, comprising:

[0009] Acquire the first and second point cloud data to be matched;

[0010] Determine n distinct neighborhoods for each point in the first and second point cloud data to be matched; n is an integer greater than or equal to 2; each neighborhood contains multiple neighboring points;

[0011] Based on the spatial coordinates and polar coordinates of each point, the Doppler velocity of each neighboring point of each point, the PPF vector between each point and each neighboring point, and the trained point feature extraction network, the fused point features of each point are determined.

[0012] The spatial coordinates, Doppler velocity, and point cloud markers of each point in the first and second point cloud data to be matched are input into a trained registration parameter prediction network to predict the point cloud registration parameters; the point cloud marker of each point is used to represent the point cloud data to which the point belongs;

[0013] Based on the point cloud registration parameters, the fusion point features of each point in the first point cloud data to be matched and the second point cloud data to be matched, the transformation parameters are determined.

[0014] Based on the transformation parameters, the first point cloud data to be matched and the second point cloud data to be matched are registered.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0016] This invention introduces a point feature extraction network and uses the polar coordinates, polar axis length, angle, vertical height, and Doppler velocity of points in point cloud data to extract point features for each point in the point cloud data. It also introduces multiple different neighborhoods for each point in the point cloud data and combines the point feature extraction network with these neighborhoods to extract the fused point features of that point. This introduces a richer description of point cloud features, allowing for more descriptive information about each point. This helps to more accurately capture the relationships between point clouds, especially in the presence of deformation and noise, thereby improving the registration accuracy and robustness during subsequent registration. Therefore, it is suitable for millimeter-wave radar point cloud registration in multi-target detection scenarios.

[0017] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating an iterative registration method for millimeter-wave radar point cloud data based on deep learning, provided in an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram illustrating the principle of generating a point feature for each point using a trained point feature extraction network, as provided in an embodiment of the present invention.

[0020] Figure 3 This is a schematic diagram illustrating the principle of generating point cloud registration parameters using a trained registration parameter prediction network, as provided in an embodiment of the present invention.

[0021] Figure 4 This is a training principle diagram of the point feature extraction network and the registration parameter prediction network provided in the embodiments of the present invention;

[0022] Figure 5 This is a schematic diagram of two point clouds obtained using millimeter-wave radar according to an embodiment of the present invention;

[0023] Figure 6 This is a visualization diagram of the registration process using the ICP algorithm provided in an embodiment of the present invention;

[0024] Figure 7 This is a visualization diagram of the registration process using the RPM algorithm provided in an embodiment of the present invention;

[0025] Figure 8 This is a visualization diagram of the registration process performed using the method provided by this invention, as provided in an embodiment of the invention. Detailed Implementation

[0026] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0027] Figure 1 This is a flowchart illustrating an iterative registration method for millimeter-wave radar point cloud data based on deep learning, as provided in an embodiment of the present invention. Figure 1 As shown, the method includes:

[0028] S101. Obtain the first point cloud data to be matched and the second point cloud data to be matched.

[0029] S102. Determine n distinct neighborhoods for each point in the first and second point cloud data to be matched; n is an integer greater than or equal to 2; each neighborhood contains multiple neighborhood points.

[0030] S103. Based on the spatial coordinates, polar coordinates, Doppler velocity of each neighboring point of each point, PPF vector between each point and each neighboring point, and the trained point feature extraction network, determine the fused point features of each point.

[0031] S104. Input the spatial coordinates, Doppler velocity, and point cloud labels of each point in the first and second point cloud data to be matched into the trained registration parameter prediction network to predict the point cloud registration parameters; the point cloud label of each point is used to represent the point cloud data to which the point belongs.

[0032] S105. Based on the point cloud registration parameters, the fusion point features of each point in the first and second point cloud data to be matched, determine the transformation parameters.

[0033] S106. Based on the transformation parameters, register the first point cloud data to be matched and the second point cloud data to be matched.

[0034] In some embodiments, n is 3. For example, for each point in the first point cloud data to be matched, three different radii can be used to find three different neighborhoods of that point in the first point cloud data, and each neighborhood includes multiple neighboring points of that point. Similarly, for each point in the second point cloud data to be matched, three different radii can be used to find three different neighborhoods of that point in the second point cloud data.

[0035] In some embodiments, S103 is achieved through the following steps:

[0036] S1031. Take the spatial coordinates, polar coordinates, Doppler velocity of each neighboring point in the m-th neighborhood of each point, and PPF vector between each point and each neighboring point as the m-th set of information values ​​for each point; where m = 1, 2, ..., n.

[0037] Here, when the m-th neighborhood contains K neighboring points, the m-th information value of each point is a K-row vector, where the k-th row vector contains: the spatial coordinates of each point, the difference between the spatial coordinates of each point and the spatial coordinates of the k-th neighboring point in the m-th neighborhood, the polar coordinates of each point, the PPF vector between each point and the k-th neighboring point, and the Doppler velocity of the k-th neighboring point; where k = 1, 2, ..., K.

[0038] For example, when each point p in the first and second point cloud data to be matched c If the first neighborhood contains two neighboring points, then p c The first set of information values ​​for a point consists of two row vectors, and then p c The first set of information values ​​for the point is as follows: Figure 2 As shown, it is obvious that p c The first row vector of the first set of information values ​​of a point includes: p c Δp c , PPF(p c p1) and d1, and, Figure 2 In this context, pc is used to represent p. c Spatial coordinates of a point, Δp c Used to represent p c The difference between the spatial coordinates of a point and the spatial coordinates of the first neighboring point in the first neighborhood. Let represent the polar coordinates of point pc, where ρ represents the polar axis length and θ represents the angle. PPF represents the vertical height. c p1) represents p c The PPF vector between point p1 and its first neighboring point p1, where d1 represents the Doppler velocity of p1. The second row vector of the first set of information values ​​is similarly represented, as shown below. Figure 2 As shown, p2 represents p c The second neighboring point in the first neighborhood of point p2, d2 represents the Doppler velocity of p2, PPF(p c p2) represents p c The PPF vector between point p2 and point p2.

[0039] For example, p c Point and its k-th neighbor point p k The PPF vector between PPF(p) c p k The expression for PPF is as follows: PPF(p c p k )=(||d|| 2 , <(n c ,d),<(n k ,d),<(n c n k ), where d represents the result of p c Point to p k The vector, n c Represents the center point p c The normal vector, n k For the neighborhood point p k The normal vector, <(n) c d) represents vector n c The angle between the vector and the vector d, and < (n c ,d)∈[0,π],<(n k ,d) and <(n c n k Similarly, PPF is rotationally invariant, meaning that the PPF value remains consistent even if the point cloud is rotated.

[0040] S1032. Input the m-th group of information values ​​of each point into the trained point feature extraction network, and output the m-th point feature of each point.

[0041] Here, the network structure of the trained point feature extraction network is an improvement on the structure of the original PointNet network. Specifically, in this invention, an additional MLP is stacked on top of the first MLP of the multiple MLPs stacked sequentially in the original PointNet network. This expands the dimension of the three-dimensional input of the original PointNet network, thereby supporting the input of n sets of information values ​​for each point in this invention.

[0042] Continue to refer to the above. Figure 2 , will p c The first set of information values ​​of a point is input into the trained point feature extraction network to obtain p. c The first point feature F1 of the point, thus, when p c When a point has n distinct neighborhoods, it can have n sets of information values, thus allowing us to obtain p. c Point features F1 to F n .

[0043] S1033. Add the n point features of each point to obtain the fused point feature of each point.

[0044] Continuing with the example above, when p c Point features F1 to F1 are obtained. n At that time, you can switch from F1 to F n Add them together to get p c The fusion feature of points F.

[0045] In some embodiments, the structure of the registration parameter prediction network in S104 is the structure of the original PointNet network. For example, the point cloud label of each point in S104 can be 0 or 1. For instance, when the point cloud label of a point is 1, it indicates that the point belongs to the first point cloud data to be matched; when the point cloud label of a point is 0, it indicates that the point belongs to the second point cloud data to be matched, and vice versa.

[0046] For example, when both the first point cloud data D1 and the second point cloud data D2 contain n′ points, the feature data that needs to be input into the trained registration parameter prediction network for both the first and second point cloud data D1 and D2 are n′ row vectors. Each row vector contains the spatial coordinates (x, y, z) of each point and the Doppler velocity and point cloud label of that point. For example, as... Figure 3As shown, the first point cloud data D1 to be matched needs to be input into the first row vector of the feature data in the trained registration parameter prediction network. This row vector contains the spatial coordinates (x1, y1, z1) of the first point in the first point cloud data D1, the Doppler velocity d′1 of that point, and the point cloud label 1 of that point. The remaining row vectors are processed similarly. Similarly, the second point cloud data D2 to be matched needs to be input into the first row vector of the feature data in the trained registration parameter prediction network. This row vector contains the spatial coordinates (x1, y1, z1) of the first point in the first point cloud data D2, the Doppler velocity d′1 of that point, and the point cloud label 0 of that point. The remaining row vectors are processed similarly.

[0047] like Figure 3 As shown, when the feature data of the first point cloud data D1 and the second point cloud data D2 to be matched are input into the trained registration parameter prediction network, the trained registration parameter prediction network outputs the predicted point cloud registration parameters α and β.

[0048] In some embodiments, S105 can be implemented through the following steps:

[0049] S1051. Based on the point cloud registration parameters and the fusion point features of each point, determine the matching degree between each point in the first point cloud data to be matched and each point in the second point cloud data to be matched, and obtain the matching degree matrix.

[0050] Here, the point cloud registration parameters include parameter α and parameter β. In the matching degree matrix, the matching degree m between the j-th point in the first point cloud data to be matched and the i-th point in the second point cloud data to be matched is... jk The expression is:

[0051]

[0052] Where j = 1, 2, ..., A, A represents the number of points in the first point cloud data to be matched, i = 1, 2, ..., B, B represents the number of points in the second point cloud data to be matched, and Fx j Fy represents the fusion point feature of the j-th point. i This represents the fusion point feature of the i-th point.

[0053] S1052. Based on the matching degree matrix and the first point cloud data to be matched, determine the transformed spatial coordinates of each point in the first point cloud data to be matched, and obtain the transformed first point cloud data to be matched.

[0054] Here, the expression for the transformed spatial coordinates of the j-th point in the first point cloud data to be matched is as follows:

[0055]

[0056] in, Let m represent the transformed spatial coordinates of the j-th point, j = 1, 2, ..., A, where A represents the number of points in the first point cloud data to be matched, i = 1, 2, ..., B, where B represents the number of points in the second point cloud data to be matched, and m... ji y represents the matching degree between the j-th point in the matching degree matrix and the i-th point in the second point cloud data to be matched. i This represents the spatial coordinates of the i-th point.

[0057] S1053. Calculate the weight matrix based on the matching degree matrix.

[0058] Specifically, the matching degrees between the j-th point in the first point cloud data to be matched and all points in the second point cloud data are summed in the matching degree matrix to obtain the j-th element in the weight matrix W, resulting in a weight matrix containing A elements; where j = 1, 2, ..., A, and A represents the number of points in the first point cloud data to be matched. The j-th element w in the weight matrix W... j The expression is:

[0059]

[0060] Where i = 1, 2, ..., B, and B represents the number of points in the second point cloud data to be matched.

[0061] S1054. Based on the weight matrix, perform weighted singular value decomposition on the first point cloud data to be matched to the transformed first point cloud data to be matched, and obtain the transformation parameters.

[0062] For example, the transformation parameters are the rotation matrix and the translation vector.

[0063] Specifically, in S106 above, transformation parameters are used to transform the first point cloud data to be matched, thereby achieving registration between the first point cloud data to be matched and the second point cloud data to be matched.

[0064] In some embodiments, prior to S103, the method further includes S201 to S209, which are described below in conjunction with... Figure 4 The explanations of S201 to S209 are provided, in which... Figure 4 The midpoint cloud X represents the following P′ c-1 The point cloud Y represents Q as described below, M represents the matching degree matrix, and Fx represents P′. c-1 Let Fy represent the fusion point feature of each point in Q. Represents P′ c Singular Value Decomposition (SVD) is also known as Singular Value Decomposition. S201 to S209 are detailed below:

[0065] S201, Initialization point feature extraction network and registration parameter prediction network.

[0066] Here, the network parameters of the feature extraction network and the registration parameter prediction network are initialized to obtain the initialized point feature extraction network and the initialized registration parameter prediction network.

[0067] S202. Obtain the raw point cloud data, and then sample, segment, and add noise to the raw point cloud data to obtain point cloud data P and point cloud data Q.

[0068] Here, the original point cloud data S is obtained, and the original point cloud data S is uniformly resampled to obtain point cloud data S′ containing preset points. Then, a clipping factor and noise are generated. The clipping factor is used to divide the point cloud data S′ into two parts, and noise is added to the divided part of the point cloud data. Thus, point cloud data P and Q are obtained, and noise is added to the point cloud data P.

[0069] S203. Using preset transformation parameters, perform a rigid transformation on the point cloud data P to obtain the transformed point cloud data P′, and calculate the inverse transformation parameters of the preset transformation parameters.

[0070] Here, preset transformation parameters can be generated, and the point cloud data P can be rigidly transformed using the preset transformation parameters.

[0071] For example, a rotation matrix R and a translation vector T can be randomly generated as preset transformation parameters; correspondingly, the calculated inverse transformation parameters are R. -1 and -T, where P′R -1 =PRR -1 =P, where P represents point cloud data P, P′ represents transformed point cloud data P′, and R represents rotation matrix R.

[0072] S204. During the c-th training iteration, acquire the point cloud data P′ from the (c-1)-th iteration. c-1 The point feature extraction network T at the c-1th iteration c-1 And the registration parameter prediction network Y for the (c-1)th time c-1 ; c is an integer greater than or equal to 1; when c is 1, P′ c-1 Let P′, T c-1 For the initial point feature extraction network, Y c-1 Predict the registration parameters for initialization.

[0073] Here, during the first training, it is necessary to obtain the initialized point feature extraction network, the initialized registration parameter prediction network, and the transformed point cloud data P′ for the first training, and to use the point cloud data P′ obtained from the first training to the first training. 1 The point feature extraction network T trained for the first time 1 The registration parameter prediction network Y trained in the first training session1 .

[0074] S205, Determine P′ c-1 The fusion point features of each point in Q.

[0075] Specifically, determine P′ c-1 and n distinct neighborhoods of each point in Q; according to P′ c-1 Spatial coordinates, polar coordinates, and P′ of each point in Q c-1 And the Doppler velocity of each neighboring point of each point in Q, P′ c-1 The PPF vector between each point in Q and each neighboring point, and T c-1 Determine P′ c-1 The fusion point features of each point in Q are used. The specific processing principle is the same as that of S1031 to S1033 above, and will not be repeated here.

[0076] S206, P′ c-1 Input Y with the spatial coordinates, Doppler velocity, and point cloud markers of each point in Q. c-1 The point cloud registration parameters for the c-th iteration are predicted.

[0077] Here, the point cloud registration parameters for the c-th iteration are: α c and β c .

[0078] S207, Based on the point cloud registration parameters of the c-th iteration, and P′ c-1 By combining the fusion point features of each point in Q, the parameters of the inverse transform at the c-th time and the point cloud data P′ at the c-th time are determined. c To conduct the (c+1)th training session.

[0079] Specifically, based on the point cloud registration parameters of the c-th iteration, P′ c-1 By combining the fusion point features of each point in Q, P′ is determined. c-1 The matching degree between each point in Q and each point in Q is used to obtain the matching degree matrix M for the c-th iteration. c According to M c and P′ c-1 Determine P′ c-1 The transformed spatial coordinates of each point in the equation are used to obtain the transformed P′. c-1 Transformed P′ c-1 As the point cloud data P′ of the cth iteration c Next, according to M c Calculate the weight matrix Wc for the c-th iteration; based on Wc, use P′ c-1 To P′ c Perform weighted singular value decomposition to obtain the inverse transform parameters. The specific principle of this part is the same as that of S1051 to S1054 above.

[0080] Here, the inverse transformation parameter is the inverse matrix R of a rotation matrix obtained by solving. pred And the inverse vector t of a translation vector obtained by solving. pred .

[0081] S208, Based on the calculated inverse transform parameters, the c-th inverse transform parameters, and P′ c-1 The spatial coordinates of each point are used to determine the loss value for the c-th iteration.

[0082] Specifically, the expression for the loss value at the c-th iteration is:

[0083]

[0084] Where L is the loss value at the c-th iteration, and J is the loss value of P′. c-1 The number of points in the matrix, R and t are the calculated inverse transform parameters, R pred and t pred Let x be the inverse transform parameter of the c-th order. j For P′ c-1 The spatial coordinates of the j′-th point in the array are given, where j′ = 1, 2, ..., J.

[0085] S209. Adjust T based on the loss value of the cth iteration. c-1 and Y c-1 The point feature extraction network T at the c-th iteration is obtained. c And the registration parameter prediction network Y for the cth time c The training is performed for the (c+1)th time, and this process is repeated until the preset conditions are met. Then the training is stopped, and the trained registration parameter prediction network is obtained.

[0086] Here, through the first training, we can obtain the point cloud data P′ of the first training. 1 The point feature extraction network T trained for the first time 1 The registration parameter prediction network Y trained in the first training session 1 Next, the point cloud data P′ from the first training iteration was used. 1 The point feature extraction network T trained for the first time 1 The registration parameter prediction network Y trained in the first training session 1 The same principle is applied for a second training iteration, resulting in point cloud data P′ for the second training iteration. 2 The point feature extraction network T trained in the second training iteration 2 The registration parameter prediction network Y trained in the second training session 2 This process is repeated iteratively until the training is complete, thus obtaining the trained registration parameter prediction network.

[0087] Here, training can end after a preset training time, after a preset number of training sessions, or when the obtained loss value is less than a set threshold. This invention does not limit the specific training time.

[0088] To verify the effectiveness of this solution, simulation and comparative experiments were conducted. The datasets used in the comparative experiments were point cloud data collected by a millimeter-wave radar device (TI-IWR6843) and an acquisition device (DCA1000), as well as the Coloradar dataset collected using a TI-AWR1843.

[0089] Evaluation metrics for point cloud registration include: Root Mean Square Error (RMSE): Calculates the average square root error of the distance between corresponding points in two point clouds. A smaller RMSE indicates better registration. Mean Distance Error (MDE): Calculates the average distance between corresponding points in two point clouds. MDE is also used to measure registration accuracy, and a smaller MDE is better. Normalized Distribution of Symmetric Distances (NDS): This metric is used to evaluate the symmetry of the alignment. It measures the symmetry between two point clouds, ensuring that the distances between corresponding points are distributed within a reasonable range. Least Squares (LS): Utilizes the least squares method to optimize the parameters of the rigid body transformation to minimize the overall squared distance between point clouds. In the following experiments, the registration results of the algorithm designed in this patent, along with the ICP and RPM algorithms, will be used, and RMSE will be used to evaluate the registration effect.

[0090] Figure 5 Figures (a) and (b) show two frames of point clouds obtained using millimeter-wave radar. Next, the proposed method will be compared with the registration results obtained using the ICP algorithm and the RMP algorithm. To verify the effectiveness of this scheme, comparative experiments were conducted using the ICP algorithm, the RPM algorithm, and the proposed method on randomly generated point clouds and millimeter-wave radar datasets, respectively. The dataset is the Colordar 2_24_2021_aspen_run0 sequence.

[0091] The time taken to calculate the registration loss is less than τ is shown in Table 1 below. Obviously, the time taken by the method proposed in this invention is less than that of other schemes in all three sets of tests.

[0092] Table 1

[0093] ICP 0.71 0.32 0.41 RPM 0.67 0.26 0.37 This plan 0.54 0.21 0.33

[0094] The ICP algorithm, RPM algorithm, and the method proposed in this invention were used to register the generated point cloud, IWR-6843 point cloud, and TI-AWR1843 point cloud, respectively. The resulting visualizations are as follows: Figures 6-8 The root mean square error of the registered point cloud was calculated using RMSE, and the specific results are shown in Table 2. As shown in Table 2, the method proposed in this invention is superior to the other two algorithms in terms of the accuracy of the registered point cloud. For point clouds acquired using IWR-6843, the accuracy is higher than that of the RPM algorithm. For data acquired using AWR-1843, the method proposed in this invention is superior to the other two algorithms.

[0095] ICP 5.204 4.514 5.350 RPM 8.327 5.907 6.216 This plan 4.849 5.071 4.792

[0096] It should be noted that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0097] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0098] In this specification, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. While different embodiments may describe certain measures, this does not mean that these measures cannot be combined to produce a good effect.

[0099] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A deep learning-based iterative registration method for millimeter-wave radar point cloud data, characterized in that, include: Acquire the first and second point cloud data to be matched; Determine n distinct neighborhoods for each point in the first and second point cloud data to be matched; n is an integer greater than or equal to 2; each neighborhood contains multiple neighboring points; Based on the spatial coordinates and polar coordinates of each point, the Doppler velocity of each neighboring point of each point, the PPF vector between each point and each neighboring point, and the trained point feature extraction network, the fused point features of each point are determined. The spatial coordinates, Doppler velocity, and point cloud markers of each point in the first and second point cloud data to be matched are input into a trained registration parameter prediction network to predict the point cloud registration parameters; the point cloud marker of each point is used to represent the point cloud data to which the point belongs. Based on the point cloud registration parameters, the fusion point features of each point in the first point cloud data to be matched and the second point cloud data to be matched, the transformation parameters are determined. Based on the transformation parameters, the first point cloud data to be matched and the second point cloud data to be matched are registered. The step of determining the fused point features of each point based on its spatial coordinates, polar coordinates, Doppler velocities of each neighboring point, PPF vectors between each point and its neighbors, and a trained point feature extraction network includes: The spatial coordinates, polar coordinates, Doppler velocity of each neighboring point in the m-th neighborhood of each point, and PPF vector between each point and each neighboring point are used as the m-th set of information values ​​for each point; where m = 1, 2, ..., n; Input the m-th group of information values ​​of each point into the trained point feature extraction network, and output the m-th point feature of each point; The n-point features of each point are added together to obtain the fused point features of each point.

2. The iterative registration method for millimeter-wave radar point cloud data based on deep learning according to claim 1, characterized in that, When the m-th neighborhood contains K neighboring points, the m-th information value of each point is a K-row vector, where the k-th row vector contains: the spatial coordinates of each point, the difference between the spatial coordinates of each point and the spatial coordinates of the k-th neighboring point in the m-th neighborhood, the polar coordinates of each point, the PPF vector between each point and the k-th neighboring point, and the Doppler velocity of the k-th neighboring point; where k = 1, 2, ..., K.

3. The iterative registration method for millimeter-wave radar point cloud data based on deep learning according to claim 1, characterized in that, The step of determining transformation parameters based on the point cloud registration parameters, the fusion point features of each point in the first and second point cloud data to be matched, includes: Based on the point cloud registration parameters and the fusion point features of each point, the matching degree between each point in the first point cloud data to be matched and each point in the second point cloud data to be matched is determined, and a matching degree matrix is ​​obtained. Based on the matching degree matrix and the first point cloud data to be matched, the transformed spatial coordinates of each point in the first point cloud data to be matched are determined to obtain the transformed first point cloud data to be matched. Calculate the weight matrix based on the matching degree matrix; Based on the weight matrix, weighted singular value decomposition is performed on the first point cloud data to be matched to the transformed first point cloud data to be matched, and the transformation parameters are obtained.

4. The iterative registration method for millimeter-wave radar point cloud data based on deep learning according to claim 3, characterized in that, The point cloud registration parameters include: parameters and parameters In the matching degree matrix, the matching degree between the j-th point in the first point cloud data to be matched and the i-th point in the second point cloud data to be matched is... The expression is: ； Where j = 1, 2, ..., A, A represents the number of points in the first point cloud data to be matched, and i = 1, 2, ..., B, B represents the number of points in the second point cloud data to be matched. The fusion point feature of the j-th point is represented. The fusion point feature represents the i-th point.

5. The iterative registration method for millimeter-wave radar point cloud data based on deep learning according to claim 3, characterized in that, The expression for the transformed spatial coordinates of the j-th point in the first point cloud data to be matched is as follows: ； in, Let represent the transformed spatial coordinates of the j-th point, j = 1, 2, ..., A, where A represents the number of points in the first point cloud data to be matched, and i = 1, 2, ..., B, where B represents the number of points in the second point cloud data to be matched. This represents the matching degree between the j-th point in the matching degree matrix and the i-th point in the second point cloud data to be matched. This represents the spatial coordinates of the i-th point.

6. The iterative registration method for millimeter-wave radar point cloud data based on deep learning according to claim 3, characterized in that, The step of calculating the weight matrix based on the matching degree matrix includes: The matching degrees between the j-th point of the first point cloud data to be matched and each point in the second point cloud data to be matched are summed in the matching degree matrix to obtain the j-th element in the weight matrix, thus obtaining the weight matrix containing A elements; where j=1,2,…,A, and A represents the number of points in the first point cloud data to be matched.

7. The iterative registration method for millimeter-wave radar point cloud data based on deep learning according to claim 1, characterized in that, Before determining the fused point features of each point based on the spatial coordinates, polar coordinates, Doppler velocities of each neighboring point, PPF vectors between each point and each neighboring point, and the trained point feature extraction network, the method further includes: Initialize the point feature extraction network and the registration parameter prediction network; The raw point cloud data is acquired, and then sampled, segmented, and noise-added to obtain the final point cloud data. And point cloud data Q; Using preset transformation parameters, a rigid transformation is performed on the point cloud data P to obtain the transformed point cloud data. And calculate the inverse transformation parameters of the preset transformation parameters; During the c-th training iteration, obtain the point cloud data from the (c-1)-th iteration. Point feature extraction network at the (c-1)th iteration Registration parameter prediction network for the (c-1)th iteration ; c is an integer greater than or equal to 1; when c is 1, the... For the The For the initial point feature extraction network, the Predict the registration parameters for initialization; Determine the and the fusion point features of each point in Q; The Input the spatial coordinates, Doppler velocity, and point cloud markers of each point in Q. The point cloud registration parameters for the c-th time are predicted. Based on the point cloud registration parameters of the c-th time, and the Based on the fusion point features of each point in Q, determine the inverse transform parameters and the point cloud data of the c-th transformation. To perform the (c+1)th training iteration; Based on the calculated inverse transform parameters, the inverse transform parameters of the c-th iteration, and the... Determine the loss value for the c-th iteration by using the spatial coordinates of each point in the equation. Adjust the loss value based on the c-th time. and stated The point feature extraction network of the c-th iteration is obtained. And the registration parameter prediction network of the cth time The training is performed for the (c+1)th time, and this process is repeated until the preset conditions are met. Then the training is stopped, and the trained registration parameter prediction network is obtained.

8. The iterative registration method for millimeter-wave radar point cloud data based on deep learning according to claim 7, characterized in that, The expression for the loss value at the c-th iteration is: ； in, Let c be the loss value. For the The number of points in and The calculated inverse transform parameters are... and Let be the parameters of the c-th inverse transform. For the The first in The spatial coordinates of the points =1,2,…, .

Images