Point cloud registration method for 3D map reconstruction in autonomous driving

Abstract

The present invention provides a point cloud registration method for 3D map reconstruction in autonomous driving. By using a pre-built dual-branch feature extraction network and acquiring real-time 3D point cloud data, local and global features of the 3D point cloud data can be extracted, and then feature interaction is used to correlate the data of the two point clouds. To generate more refined features for generating transformation parameters, the present invention uses a channel-level feature fusion method. This method correlates the data between the features of the two point clouds. The channel-level feature fusion method performs feature interaction between the point clouds to be registered, making it possible to find the correlation between the deep features of a pair of point clouds and allowing the network to perform feature enhancement through feature difference learning, further improving the accuracy of the point cloud registration task and the robustness of the network itself. Therefore, the present invention can improve the accuracy of the reconstructed map generated by autonomous driving.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to a point cloud registration method for reconstructing three-dimensional maps for autonomous driving. Background Art

[0002] Building 3D maps is a crucial step in robotics and autonomous driving. Point cloud registration reconstructs these maps, enabling the vehicle to update its driving path in real time. During 3D map reconstruction, point cloud registration first extracts features from the image surrounding the driving path and then uses these features to estimate the rigid transformation. The quality of these features directly impacts the point cloud registration results.

[0003] The classic method in point cloud registration is Iterative Close Point (ICP), which aims to compute a rigid transformation by establishing correspondences between a pair of point clouds and iteratively transforming them to minimize the error between the corresponding points. Due to its non-convexity and local minima, ICP algorithms are more prone to falling into local optima. Compared to traditional methods, deep learning point cloud registration algorithms demonstrate superior performance and robustness. Mainstream deep learning point cloud registration methods are correspondence-based point cloud registration methods, aiming to find corresponding points for transformation prediction. However, in the presence of multiple anomalies and significant noise, the correspondences found can be unreliable, making accurate identification extremely difficult. In contrast, correspondence-free point cloud registration methods eliminate the process of establishing correspondences between a pair of point clouds, significantly saving time and cost. PointNetLK pioneered a global feature-based approach to address rigid transformations; PCRNet uses data-driven techniques based on PointNetLK to improve the network's noise resistance; and DirectNet builds on the previous two approaches, predicting transformations by regressing Euler angles. A review of previous methods revealed the following issues:

[0004] When using PointNet as a feature extractor, global features are obtained but local domain features are lost. The basic idea of ​​PointNet is to aggregate all individual point features into a global code. Because of this, PointNet cannot capture the local structure of the three-dimensional point cloud, nor can it extract accurate local features from the neighborhood of the point, resulting in poor reconstructed three-dimensional maps. The quality of the three-dimensional map determines the safety of autonomous driving. Both manufacturers and consumers are more concerned about this performance of the vehicle. Summary of the Invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides a point cloud registration method for 3D map reconstruction for autonomous driving. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0006] The present invention provides a point cloud registration method for autonomous driving three-dimensional map reconstruction, comprising:

[0007] S100, acquiring a standard data set from a database and three-dimensional data of an external environment collected in real time from a sensor of a vehicle, and reading point cloud data of each point cloud sample from the standard data set;

[0008] S200, randomly sampling each point cloud sample to obtain a sampling point of each point cloud sample;

[0009] S300, constructing source point cloud and template point cloud based on point cloud data;

[0010] S400, inputting the source point cloud and the corresponding template point cloud into a pre-built point cloud registration network based on dual-branch feature difference learning, using the point cloud registration network to obtain the transformation parameters from the source point cloud to the corresponding template point cloud, and obtaining the final point cloud registration model by optimizing the network parameters of the point cloud registration network;

[0011] S500, inputting 3D data from the same sensor at adjacent moments or from different sensors at the same moment into a point cloud registration model to obtain transformation parameters at adjacent moments;

[0012] S600: align the corresponding three-dimensional data to the same reference frame using the transformation parameters, thereby completing the three-dimensional map reconstruction.

[0013] The present invention provides a point cloud registration method for 3D map reconstruction in autonomous driving. By using a pre-built dual-branch feature extraction network and acquiring real-time 3D point cloud data, local and global features of the 3D point cloud data can be extracted, and then feature interaction is used to correlate the data of the two point clouds. To generate more refined features for generating transformation parameters, the present invention uses a channel-level feature fusion method. This method correlates the data between the features of the two point clouds. The channel-level feature fusion method performs feature interaction between the point clouds to be registered, making it possible to find the correlation between the deep features of a pair of point clouds and allowing the network to perform feature enhancement through feature difference learning, further improving the accuracy of the point cloud registration task and the robustness of the network itself. Therefore, the present invention can improve the accuracy of the reconstructed map generated by autonomous driving.

[0014] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 1 is a flow chart of a point cloud registration method for autonomous driving three-dimensional map reconstruction provided by the present invention;

[0016] Figure 2Schematic diagram of the process of the point cloud registration method for autonomous driving three-dimensional map reconstruction provided by the present invention;

[0017] Figure 3 It is a schematic diagram of local feature selection provided by the present invention;

[0018] Figure 4 This is a visual comparison diagram of the registration results provided by the present invention. DETAILED DESCRIPTION

[0019] The present invention will be further described in detail below with reference to specific examples, but the embodiments of the present invention are not limited thereto.

[0020] The present invention provides a point cloud registration method for autonomous driving three-dimensional map reconstruction, comprising:

[0021] S100, acquiring a standard data set from a database and three-dimensional data of an external environment collected in real time from a sensor of a vehicle, and reading point cloud data of each point cloud sample from the standard data set;

[0022] The present invention reads point cloud data from a dataset. Each point cloud contains 2048 points. The three-dimensional coordinates and category labels of each point in the point cloud are read. Each point cloud data read from the dataset contains 2048 points. Each point cloud sample is randomly sampled to obtain 1024 points.

[0023] S200, randomly sampling each point cloud sample to obtain a sampling point of each point cloud sample;

[0024] S300, constructing source point cloud and template point cloud based on point cloud data;

[0025] S400, inputting the source point cloud and the corresponding template point cloud into a pre-built point cloud registration network based on dual-branch feature difference learning, using the point cloud registration network to obtain the transformation parameters from the source point cloud to the corresponding template point cloud, and obtaining the final point cloud registration model by optimizing the network parameters of the point cloud registration network;

[0026] S500, inputting 3D data from the same sensor at adjacent moments or from different sensors at the same moment into a point cloud registration model to obtain transformation parameters at adjacent moments;

[0027] S600: align the corresponding three-dimensional data to the same reference frame using the transformation parameters, thereby completing the three-dimensional map reconstruction.

[0028] S300 of the present invention includes:

[0029] S310: taking each point cloud data as a source point cloud;

[0030] S320: Performing random transformation on each source point cloud to obtain a rigidly transformed template point cloud;

[0031] The random rotation angle range of the random transformation is [-45°, 45°], and the random translation distance range is [-1, 1].

[0032] This step includes: S321: obtaining a random rotation matrix and a random translation vector; S322: using the random rotation matrix and the random translation vector to perform a random transformation on each source point cloud to obtain a template point cloud.

[0033] The process of randomly rotating the matrix is ​​as follows

[0034]

[0035]

[0036]

[0037] R=R x ·R y ·R z

[0038] Where x, y, z∈[-45°, 45°].

[0039] The process of randomly translating the vector is as follows

[0040] t=(t1,t2,t3)

[0041] Among them, t1, t2, t3∈[-1, 1].

[0042] This step sets the random transformation applied on each source point cloud to the true value of the rigid transformation.

[0043] Before S400, that is, before constructing the point cloud registration network based on dual-branch feature difference learning, the point cloud registration method further includes:

[0044] (a1): Calculate the distance matrix D of the point cloud X :

[0045]

[0046] Among them, x i , x j represents a point in the point cloud, (k) represents the index of the point coordinate;

[0047] (a2): According to the distance matrix D X Select the 20 nearest neighbor points with the smallest distance to obtain the nearest neighbor relationship of the initial point cloud;

[0048]

[0049] Among them, argmin means according to the distance matrix D X The operation of obtaining the k nearest neighbor position indexes row by row, (i) represents D x The i-th row of .

[0050] refer to Figure 2 ,The point cloud registration network based on dual-branch feature difference learning,includes a local feature extraction module, a global feature extraction,module, similarity fusion module and output layer;

[0051] Among them, the local feature extraction module and the global feature extraction module are in parallel to extract the point cloud features of the input point cloud at the same time. The local feature extraction module extracts local features, and the global feature extraction module extracts global features; the similarity fusion module calculates the similarity between the global features and the local features to output the fused features, and outputs them through the output layer.

[0052] refer to Figure 3 As shown, the local feature extraction module is composed of four multilayer perceptrons (MLPs) with sizes of 64, 64, 64, and 128 connected in series. The process of extracting local features by the local feature extraction module includes:

[0053] (b1): Input the source point cloud into the first layer of MLP to obtain the initial features output by each layer of MLP;

[0054] Input the point cloud into the MLP,

[0055] L′1=Φ1(X)

[0056] L′ l =Φ l (L′ l-1 )

[0057] where Φ represents MLP.

[0058] (b2): Index the initial features output by each layer of MLP according to the index matrix to obtain the neighboring point features of the source point cloud;

[0059]

[0060]

[0061] Where cat represents the splicing operation, L′ l [ID X ] means taking the feature at the corresponding position according to the index value.

[0062] (b3): ​​All neighbor point features obtained by the initial feature index of each layer of MLP are concatenated and max-pooled to obtain the local features of the source point cloud.

[0063] The global feature extraction module is composed of 5 layers of MLP connected in series. The process of extracting global features by the global feature extraction module includes:

[0064] (c1): Embed the source point cloud into a high-dimensional space through each layer of MLP to obtain the initial global features of each layer;

[0065] (c2): The initial global features of the 5th layer MLP are used as value features, and the initial global features of the 3rd layer MLP are used as query features and key features;

[0066] (c3): Calculate the similarity between the query feature and the key feature to obtain the attention value;

[0067] (c4): Perform weighted summation of the attention value and the value feature to obtain the final enhanced feature;

[0068] (c5): Perform maximum pooling on the final enhanced features to obtain global features.

[0069] The fusion process of the similarity fusion module includes:

[0070] (d1): Fuse the global features and local features of the source point cloud, and fuse the global features and local features of the template point cloud to obtain their respective mixed features.

[0071] Finally, four layers of features are obtained, and these four layers of features are cat (splicing) and maxpooling (maximum pooling) to obtain the final local features.

[0072] L X =Max(Relu(MLP(Cat([L1, L2, L3, L4))));

[0073] F′ l =MLP(X)

[0074] In the process of extracting local point features, a 5-layer MLP is used to embed the point cloud into a high-dimensional space, where l represents the number of MLP layers.

[0075] The attention mechanism first calculates the similarity between the query feature and the key feature. The similarity generates an alignment probability value, which is the attention value. The attention value is then weighted and summed with the value feature to obtain the final enhanced feature. The formula is as follows

[0076]

[0077] Where Q represents the query feature, K represents the key feature, and V represents the value feature.

[0078] F′ X =Max(Att(F′3·W Q , F′3·W K ·F′5·W V )+F′5)

[0079] Where Max represents the maximum pooling operation. WQ, WK, and WV are projection matrices. The present invention projects F′3, F′5, and F′5 onto queries Q, K, and V, respectively.

[0080] Obtain point feature F′ through a dual-branch network X and local features L′ X After that, we can get the mixed feature F of the source point cloud X (1, C1). The formula is as follows

[0081] F X =Relu(MLP(Cat([F′ X , L X ])))

[0082] Similarly, the mixed feature F of the template point cloud is obtained Y (1, C2).

[0083] In the present invention, after step (d1), the point cloud registration method further includes:

[0084] (e1): Normalize the mixed features of the source point cloud and the template point cloud respectively;

[0085] (e2): Calculate the channel similarity matrix between the normalized mixed features of the source point cloud and the normalized mixed features of the template point cloud;

[0086] (e3): Add the channel similarity matrix row by row to obtain the similarity vector S X , and add column by column to get the similarity vector S Y ;

[0087] (e4): Calculate the similarity vector S X The similarity score and similarity vector S Y Similarity score of

[0088] (e5): Update the corresponding hybrid features according to the similarity score.

[0089] The output process of the output layer includes:

[0090] (f1): Convert the mixed features into a 7-dimensional vector q to obtain the predicted translation vector t and rotation matrix R;

[0091] (f2): Obtain the rigid transformation parameter G based on the translation vector t and the rotation matrix R;

[0092] (f3): Update the source point cloud according to the number of iterations and the rigid transformation parameter G.

[0093] For feature F X and feature F Y Perform feature normalization. The channel-level feature fusion method is used for these two features to perform feature interaction. First, the present invention calculates the channel similarity matrix S:

[0094]

[0095] where · represents matrix multiplication.

[0096] Secondly, the present invention adds the channel similarity matrix row by row to generate the vector S X , this vector represents the similarity between each channel in FX and all channels in FY.

[0097]

[0098] Where i∈[1, C1]. Similarly, the present invention adds the similarity matrix S column by column to generate a vector S Y :

[0099]

[0100] where j∈[1, C2]. Next, a softmax operation is applied to these two similarity vectors to obtain a similarity score,

[0101] α=Softmax(S X )

[0102] β=Softmax(S Y )

[0103] Among them, α(C1, 1), β(1, C2).

[0104] Finally, the present invention can update the hybrid feature based on these two scores,

[0105]

[0106]

[0107] in Represents the Hadamard product, and combining R and t can obtain the rigid transformation parameter G:

[0108]

[0109] Update the source point cloud according to the predicted rotation matrix and translation vector. The final predicted rigid body transformation formula is as follows:

[0110] G est =G(n)×G(n-1)×…×G(1)

[0111] Where n represents the number of iterations.

[0112] The present invention constructs a loss function for the point cloud registration model: the loss function adopts the classic chamfer distance loss to calculate the distance loss between corresponding points of two point clouds.

[0113]

[0114] Among them, p s represents the source point cloud, p t Represents the template point cloud.

[0115] In order to verify the effectiveness and superiority of the improved target detection algorithm in multi-scale remote sensing target detection tasks, simulation experiments and comparative experiments were set up.

[0116] The hardware platform is: HP-Z840 workstation, Intel(R) Xeon(R) E5-2630-CPU, main frequency is 2.40GHz, RTX2080TI-11GB-GPU, 64GB RAM.

[0117] The software platform is: Python, PyTorch deep learning framework.

[0118] In the simulation: The simulation experiment of the present invention uses the ModelNet40 dataset, which consists of 40 different categories of CAD data, including airplanes, cars, tables, etc. Each 3D point cloud data contains 2048 points, and each point consists of 3D coordinates (x, y, z). The three-dimensional coordinates of the points contained in each sample and the category label of the sample are obtained. The simulation experiment of the present invention predicts the transformation parameters of the source point cloud and the template point cloud, and calculates the difference with the actual transformation. The evaluation indicators are as follows:

[0119]

[0120] Where n is the total number of samples, e est and e * are the predicted and true Euler angles, t est and t * are the predicted and true translation vectors, respectively.

[0121] In addition to the experiments of the present invention, the previous methods were also compared. The experimental results are shown in Table 1. Figure 4 .

[0122] Table 1. Performance comparison of the present invention and previous registration methods

[0123] method RMSE(R) RMSE(T) ICP 21.2084 0.2874 FGR 9.5964 0.1186 DirectNet 5.0895 0.1025 PCRNet 3.0788 0.0303 PointNetLK 4.0131 0.7616 DCP-v1 7.0840 0.1003 DeepGMR 7.1320 0.7603 The present invention 2.7996 0.0285

[0124] As can be seen from Table 1, the error of the present invention is greatly reduced compared with traditional methods such as ICP and FGR, and it is also improved compared with previous deep learning methods.

[0125] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature identified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0126] Although the present application is described herein with reference to various embodiments, those skilled in the art may understand and implement other variations of the disclosed embodiments in practicing the claimed application by examining the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or S, and "a" or "an" does not exclude a plurality.

[0127] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A point cloud registration method for 3D map reconstruction in autonomous driving, characterized in that: include: S100, acquiring a standard data set from a database and three-dimensional data of an external environment collected in real time from a sensor of a vehicle, and reading point cloud data of each point cloud sample from the standard data set; S200, randomly sampling each point cloud sample to obtain a sampling point of each point cloud sample; S300, constructing source point cloud and template point cloud based on point cloud data; S400: Inputting the source point cloud and the corresponding template point cloud into a pre-built point cloud registration network based on dual-branch feature difference learning, using the point cloud registration network to obtain the transformation parameters from the source point cloud to the corresponding template point cloud, and optimizing the network parameters of the point cloud registration network to obtain the final point cloud registration model; the point cloud registration network based on dual-branch feature difference learning includes a local feature extraction module, a global feature extraction module, a similarity fusion module, and an output layer; The local feature extraction module and the global feature extraction module are operated in parallel to extract point cloud features of the input point cloud at the same time. The local feature extraction module extracts local features, and the global feature extraction module extracts global features. The similarity fusion module calculates the similarity between the global features and the local features to output the fused features, and outputs them through the output layer. S500, inputting 3D data from the same sensor at adjacent moments or from different sensors at the same moment into a point cloud registration model to obtain transformation parameters at adjacent moments; S600: align the corresponding three-dimensional data to the same reference frame using the transformation parameters, thereby completing the three-dimensional map reconstruction.

2. The point cloud registration method for autonomous driving 3D map reconstruction according to claim 1, characterized in that: S300 includes: S310: taking each point cloud data as a source point cloud; S320: Performing random transformation on each source point cloud to obtain a rigidly transformed template point cloud; The random rotation angle range of the random transformation is [-45°, 45°], and the random translation distance range is [-1, 1].

3. The point cloud registration method for autonomous driving 3D map reconstruction according to claim 2, characterized in that: The S320 includes: S321: Obtain a random rotation matrix and a random translation vector; S322: Using a random rotation matrix and a random translation vector, each source point cloud is randomly transformed to obtain a template point cloud.

4. The point cloud registration method for autonomous driving 3D map reconstruction according to claim 1, characterized in that: Before S400, the point cloud registration method further includes: (a1): Calculate the distance matrix of the point cloud : in, represents a point in the point cloud, (k) represents the index of the point coordinate; (a2): According to the distance matrix Select the 20 nearest neighbor points with the smallest distance to obtain the nearest neighbor relationship of the initial point cloud; Among them, argmin means according to the distance matrix The operation of obtaining the k nearest neighbor position indexes row by row, (i) represents The i-th row of .

5. The point cloud registration method for autonomous driving 3D map reconstruction according to claim 1, characterized in that: The local feature extraction module is composed of four multilayer perceptrons (MLPs) with sizes of 64, 64, 64, and 128 connected in series. The process of extracting local features by the local feature extraction module includes: (b1): Input the source point cloud into the first layer of MLP to obtain the initial features output by each layer of MLP; (b2): Index the initial features output by each layer of MLP according to the index matrix to obtain the neighboring point features of the source point cloud; (b3): ​​All neighbor point features obtained by the initial feature index of each layer of MLP are concatenated and max-pooled to obtain the local features of the source point cloud.

6. The point cloud registration method for autonomous driving 3D map reconstruction according to claim 1, characterized in that: The global feature extraction module is composed of 5 layers of MLP connected in series. The process of extracting global features by the global feature extraction module includes: (c1): Embed the source point cloud into a high-dimensional space through each layer of MLP to obtain the initial global features of each layer; (c2): The initial global features of the 5th layer MLP are used as value features, and the initial global features of the 3rd layer MLP are used as query features and key features; (c3): Calculate the similarity between the query feature and the key feature to obtain the attention value; (c4): Perform weighted summation of the attention value and the value feature to obtain the final enhanced feature; (c5): Perform maximum pooling on the final enhanced features to obtain global features.

7. The point cloud registration method for autonomous driving 3D map reconstruction according to claim 1, characterized in that: The fusion process of the similarity fusion module includes: (d1): Fuse the global features and local features of the source point cloud, and fuse the global features and local features of the template point cloud to obtain their respective mixed features.

8. The point cloud registration method for autonomous driving 3D map reconstruction according to claim 7, characterized in that: After (d1), the point cloud registration method further includes: (e1): Normalize the mixed features of the source point cloud and the template point cloud respectively; (e2): Calculate the channel similarity matrix between the normalized mixed features of the source point cloud and the normalized mixed features of the template point cloud; (e3): Add the channel similarity matrix row by row to obtain the similarity vector , and add column by column to get the similarity vector ; (e4): Calculate the similarity vector Similarity score and similarity vector Similarity score of (e5): Update the corresponding hybrid features according to the similarity score.

9. The point cloud registration method for autonomous driving 3D map reconstruction according to claim 1, characterized in that: The output process of the output layer includes: (f1): Convert the mixed features into a 7-dimensional vector , get the predicted translation vector and the rotation matrix R; (f2): According to the translation vector And the rotation matrix R to obtain the rigid transformation parameter G; (f3): Update the source point cloud according to the number of iterations and the rigid transformation parameter G.

Images