Low-overlap three-dimensional point cloud registration method and system based on point pair convolution

Through the method of point-to-feature convolution, rotation invariant features are extracted and geometric and semantic features are integrated, which solves the problems of performance degradation and local limitations in the existing technology at large rotation angles, and improves the accuracy of point cloud registration and rotation robustness.

CN118967756AActive Publication Date: 2024-11-15WUHAN UNIV
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202410875603.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-02
Publication Date
2024-11-15
Estimated Expiration
2044-07-02

AI Technical Summary

Technical Problem

Existing deep learning-based point cloud registration technology reduces performance when processing point clouds with larger rotations, and the locality of convolutional neural networks limits the accuracy of point cloud overlapping areas, especially when overlapping areas are small or feature distribution is not Outstanding performance when uniform.

Method used

Using a low-overlapping point cloud registration method based on point-to-feature convolution, the point-core convolution, geometric guide encoding layer, global aggregation layer and decoder layer, the rotation invariant features and fusion geometric and semantic features are extracted, combined with RANSAC Then, the post-processing method calculates the transformation parameters.

Benefits of technology

The accuracy and rotation invariance of point cloud registration are improved, the prediction ability of overlapping parts of low overlap point clouds is enhanced, and the performance of the network in large-angle rotation scenarios and the prediction accuracy of overlapping areas are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118967756A_ABST
    Figure CN118967756A_ABST
Patent Text Reader

Abstract

The invention relates to a low-overlap point cloud registration method and device based on point pair feature convolution. Firstly, an encoder capable of extracting rotation invariant features is provided, and the performance of the network under the condition of large rotation is enhanced. Secondly, a global feature aggregation method is provided, the limitation that the performance is affected by the size of a receptive field of a convolutional layer is improved, and the ability of the network to identify the overlapping part of the point clouds is stronger. Through the combination of the two, the low-overlap point cloud registration method based on the point pair feature convolution is researched and explored.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a low-overlap point cloud registration method and system based on point-pair feature convolution. Background Art

[0002] Point cloud registration refers to aligning two or more 3D point cloud data to determine the best correspondence between them. Point cloud registration achieves the integration and fusion of spatial data by identifying and matching the common features between different point cloud sets. Therefore, point cloud registration technology is of great significance for many application directions such as 3D reconstruction, autonomous driving, and cultural relics protection.

[0003] With the development of deep learning technology, deep learning-based methods have been widely used in point cloud registration research. By leveraging the powerful fitting ability of neural networks, these methods have achieved good results. Most of the existing deep learning-based point cloud registration technologies use feature learning-based methods. This method uses deep neural networks to learn robust feature correspondence searches. Then, the transformation matrix between the two point clouds is estimated based on the corresponding point pairs. This method can learn deep point cloud features and has strong robustness and interpretability. However, the current feature learning-based point cloud registration network also has the following problems.

[0004] First, most point cloud descriptors are designed for specific poses or viewpoints, which means they are sensitive to rotation changes. When there is a large rotation between point clouds, these descriptors may not be able to effectively match corresponding points because their feature representations will change significantly due to the rotation. This sensitivity to rotation will lead to reduced network performance in the case of large rotations.

[0005] In addition, when using convolutional neural networks to process point cloud data, the receptive field size of the convolutional layer has an important impact on its performance. Limited receptive field means that the network can only consider information in a limited area around each point when processing it. This locality may limit the accuracy of the network in identifying overlapping parts in the point cloud, especially when the overlapping area of ​​the point cloud is small or the features are unevenly distributed. Summary of the invention

[0006] In order to improve the accuracy of point cloud registration, especially the ability of the network to extract rotation invariant features and the ability to predict the overlapping parts of low-overlap point cloud pairs, the present invention proposes a low-overlap point cloud registration method and system based on point pair feature convolution. The invented method takes the source point cloud X and the target point cloud Y as input, outputs the point-by-point feature descriptors of X and Y, and finally matches the feature points and calculates the transformation parameters.

[0007] The low overlap point cloud registration method based on point pair feature convolution of the present invention comprises the following steps:

[0008] Step S1: construct an encoder layer based on point kernel convolution to downsample the input source point cloud and the target point cloud to obtain a point cloud with lower density and point cloud features after convolution;

[0009] Step S2: construct a geometry-guided coding layer, which includes two branches: geometry coding and semantic coding. The geometry coding branch inputs the original point cloud and the downsampled point cloud, and outputs the geometric features of the downsampled point cloud. The semantic coding branch inputs the downsampled point cloud and the convolutional point cloud features, and outputs the semantic features of the downsampled point cloud. The obtained geometry features and semantic features are input into the fully connected layer to obtain the source point cloud and target point cloud features that are finally output by the layer.

[0010] Step S3: Construct a global aggregation layer, which is used to aggregate the global features of the source point cloud and the target point cloud; the input of the global aggregation layer is the source point cloud and target point cloud features output by S2, and the output is the features of the downsampled point cloud after global aggregation;

[0011] Step S4: Construct a decoder layer based on nearest neighbor upsampling, which is used to restore the downsampled point cloud features to the dense features of the original point cloud, and pass the encoder features to the decoder through skip connection; the input of this layer is the features of the downsampled point cloud output by S1, and the output is the point-by-point feature descriptor of the original point cloud;

[0012] Step S5: feature matching is performed through the point-by-point feature descriptors of the original point cloud to obtain a matching point set between the source point cloud and the target point cloud, and the transformation parameters are calculated through the matching point set using a post-processing method such as RANSAC.

[0013] Step S6: Use the data set with the real matching point set labels to train the deep learning network constructed by S1-S5;

[0014] Step S7: Use the network trained in step S6 to perform point cloud registration;

[0015] Furthermore, the step S1 adopts the improved point kernel convolution module to extract the rotation invariant features of the point cloud; the specific process of S1 is:

[0016] S1-1, the convolution kernel is a spherical area, and each kernel point has a weight W k :

[0017]

[0018] in, represents the kernel point, r represents the radius of the convolution kernel;

[0019] S1-2, for each area that needs convolution, let the point obtained after convolution be x 0, N x is the set of points in the convolution region, f is the feature of the points in the convolution region, PPF(x i ,x 0 ) is x i and x 0 The point pair features of , the convolution process is:

[0020]

[0021]

[0022] The input original point cloud X is output through step S1 as the downsampled point cloud X' and feature F X’ , the points in X' are called super points, y i =x i -x 0 , mlp is a multi-layer perceptron layer, PPF represents point pair features, g and h are intermediate variables.

[0023] Furthermore, the specific process of step S2 is as follows:

[0024] S2-1, input the original point cloud X and the downsampled point cloud X' into the geometry encoding branch in the geometry guidance layer, and use the K nearest neighbor method to estimate the normal of the downsampled point cloud, that is, the normal direction of a point in the downsampled point cloud is the average of the normal directions of the k neighboring points of the point in the original point cloud;

[0025] S2-2, select k neighboring points for each superpoint in X' and construct PPF features, N x’ is the normal of the point:

[0026]

[0027] Combine the PPF feature with x i ' and x i '-x ij 'For concatenation, each point can obtain features with a dimension of k×10;

[0028] S2-3, input the features obtained in S2-2 into the one-dimensional convolution layer and the maximum pooling layer to obtain the geometric features G output by the geometric encoding branch X’ ;

[0029] S2-4, the downsampled point cloud X' and feature F X’ Input the semantic encoding branch in the geometric guidance layer, pass through the KNN graph neural network, and input it into the fully connected layer to obtain the semantic feature H output by the semantic encoding branch. X’ :

[0030]

[0031] S2-5, concatenate the features obtained by S2-3 and S2-4, input them into the fully connected layer, and obtain the geometric and semantically enhanced point cloud features output by the geometry-guided encoding layer

[0032]

[0033] Furthermore, the specific process of step 3 is as follows:

[0034] S3-1, respectively input the enhanced features of the source point cloud and the target point cloud output in step 2 into the global aggregation layer, and after point-by-point maximum pooling, obtain two 256-dimensional feature vectors respectively;

[0035] S3-2, repeat and concatenate the two feature vectors obtained in the previous step to obtain features with dimensions of N'×1024 and M'×1024 respectively. Input these two feature matrices into the fully connected layer to obtain super-point features after aggregating global features, with dimensions of N'×256 and M'×256.

[0036] Furthermore, the decoder in step S4 is used to generate a dense feature descriptor of the original point cloud; the super-point features obtained in step S1 are sent to the decoder, and the decoder part uses nearest neighbor upsampling to obtain the final point-by-point feature descriptor; jump connection is used to pass the encoder features to the decoder; these features are connected to the upsampled features and processed by MLP; the output of the decoder is N×256 and M×256 dense original point features.

[0037] Furthermore, the specific process of step S5 is as follows:

[0038] S5-1, calculate the Gaussian correlation between the super-point features in point clouds X' and Y':

[0039]

[0040] Then, through the bidirectional normalization operation, the ambiguous matching is further suppressed. X' is the downsampled point cloud of the original point cloud, Y' is the number of super points of the target point cloud, and H i is the super-point feature of the original point cloud, H j is the super-point feature of the target point cloud;

[0041] S5-2, according to S ij Select the most reliable P pairs of superpoints; for each pair of superpoints, perform fine-grained matching between the two point sets around the superpoints, and finally obtain the matching point set of the original point cloud;

[0042] S5-3, the P matching point sets obtained in the previous step are used to calculate the transformation parameters R,t using the random sampling consensus algorithm RANSAC, and P groups R,t can be obtained as candidates;

[0043] S5-4, use the probability sampling method to select the key points of the source point cloud and the target point cloud according to the matching factor, and use RANSAC to estimate a set of global R,t; a total of P+1 sets of R,t are obtained;

[0044] S5-5, select the optimal R,t as the final result.

[0045] Furthermore, the specific process of step S6 is as follows:

[0046] S6-1, using H samples for training, each sample contains two point clouds, the source point cloud and the target point cloud, and their corresponding real matching point sets and real transformation parameters;

[0047] S6-2, calculate the feature loss for the point cloud features output by S4; let the overlapping part of point cloud X and point cloud Y be X overlap and Y overlap ; For point x in point cloud X, ε x Y overlap The corresponding point set in is a positive sample; y Y overlap With ε x The difference between them is a negative sample; is the Euclidean distance between point features, Δ p , Δ n is the distance between positive samples and negative samples, Different weights are defined for each positive and negative sample; the feature loss is calculated using the ring loss:

[0048]

[0049]

[0050] S6-3, calculate the point matching loss for the super-point matching point set obtained in S5. i * , a soft assignment matrix Z is calculated through the optimal transport layer i * , represents the confidence of the point correspondence; random sampling N g The correspondence relationship between the real super points is set, and M i is the set of real point correspondences under a given super-point correspondence, I i and J i For the set of points that do not match in point cloud X' and point cloud Y', calculate the negative log-likelihood loss:

[0051]

[0052] S6-5, add the above two losses to calculate the total network loss:

[0053] L=L c +L p

[0054] Based on the same inventive concept, the present invention also designs an electronic device, which is special in that it includes:

[0055] one or more processors;

[0056] A storage device for storing one or more programs;

[0057] When one or more programs are executed by the one or more processors, the one or more processors implement the low overlap point cloud registration method based on point pair feature convolution as described above.

[0058] Based on the same inventive concept, the present invention also designs a computer-readable medium on which a computer program is stored, and its special feature is that when the program is executed by a processor, the low-overlap point cloud registration method based on point pair feature convolution as described above is implemented.

[0059] Advantages of the present invention:

[0060] 1. The present invention innovatively changes the convolution method of point kernel convolution, and modifies the point coordinate features that extract rotational variability into point pair features that extract rotational invariance, thereby improving the performance of the network in scenarios with large-angle rotation.

[0061] 2. The present invention innovatively proposes a geometry-guided coding layer, which integrates geometric features with semantic features and increases the proportion of internal corresponding points. In the geometry coding branch, the point pair features of K nearest neighbor points are innovatively constructed to improve the network's robustness to rotation; in the semantic coding branch, the KNN graph neural network is innovatively used. At the same time, a normal smoothing method is proposed to make the calculation of the normal direction of the super point more accurate.

[0062] 3. The present invention innovatively proposes a global aggregation layer, which increases the prediction accuracy of the overlapping parts of the point cloud. Due to the limited receptive field of convolution, the convolution features in the encoder alone may not be sufficient to accurately predict the overlapping parts. The global aggregation module effectively improves the prediction accuracy by introducing global features into the prediction process. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 1 is a diagram of the encoder-decoder layer neural network architecture of the present invention.

[0064] Figure 2 This is a diagram of the neural network architecture of the geometrically guided encoding layer of the present invention.

[0065] Figure 3 This is a diagram of the global aggregation layer neural network architecture of the present invention. DETAILED DESCRIPTION

[0066] In order to facilitate ordinary technicians in the field to understand and implement the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0067] Embodiment 1

[0068] The method provided by the present invention designs a low-overlap point cloud registration method based on point pair feature convolution, which comprises the following steps:

[0069] Step S1, first use voxel downsampling to downsample the original point cloud, and after downsampling three times, four point clouds with different densities are obtained, and the density of the point cloud decreases successively; use the improved point kernel convolution module to convolve the input original point cloud for a total of three times, and the point cloud obtained after each convolution is the same as the point cloud of the corresponding stage of voxel downsampling; the convolution radius is 2.5 times the voxel downsampling grid unit size, and the radius of the convolution kernel is 1.5 times the voxel downsampling grid unit size; the specific implementation process is described as follows:

[0070] S1-1, the convolution kernel is a spherical area, where K = 15 kernel points are preset, and each kernel point has a weight W k :

[0071]

[0072] S1-2, for each area that needs convolution, its center point is x 0 ,PPF(x i ,x 0 ) is x i and x 0 The point pair features of , the convolution process is:

[0073]

[0074] The input original point cloud X is output through step S1 as the downsampled point cloud X' and feature F X’ , and the points in X' are called super points.

[0075] Step S2, construct a geometry-guided coding layer, which includes two branches: geometry coding and semantic coding; the geometry coding branch inputs the original point cloud and the downsampled point cloud, and outputs the geometric features of the downsampled point cloud; the semantic coding branch inputs the downsampled point cloud and the convolutional point cloud features, and outputs the semantic features of the downsampled point cloud; the obtained geometric features and semantic features are input into the fully connected layer to obtain the source point cloud and target point cloud features of the final output of this layer. The specific implementation process is described as follows:

[0076] S2-1, input the original point cloud X and the downsampled point cloud X' into the geometry encoding branch in the geometry guidance layer, and use the K nearest neighbor method to estimate the normal of the downsampled point cloud, that is, the normal direction of a point in the downsampled point cloud is the average of the normal directions of the k neighboring points of the point in the original point cloud;

[0077] S2-2, select k neighboring points for each superpoint in X' and construct PPF features, N x’ is the normal of the point:

[0078]

[0079] Combine the PPF feature with x i ' and x i '-x ij 'Concatenate the two 3D coordinates of each point with a 4D point pair feature to obtain a feature with a dimension of k×10;

[0080] S2-3, input the features obtained in S2-2 into the one-dimensional convolution layer and the maximum pooling layer to obtain the geometric features G output by the geometric encoding branch X’ ;

[0081] S2-4, the downsampled point cloud X' and feature F X’ Input the semantic encoding branch in the geometric guidance layer, pass through the KNN graph neural network, and input it into the fully connected layer to obtain the semantic feature H output by the semantic encoding branch. X’ :

[0082]

[0083] S2-5, concatenate the features obtained by S2-3 and S2-4, input them into the fully connected layer, and obtain the geometric and semantically enhanced point cloud features F output by the geometry-guided encoding layer X’ gge :

[0084]

[0085] Step S3, construct a global aggregation layer, which is used to aggregate the global features of the source point cloud and the target point cloud; the global aggregation layer inputs the source point cloud and the target point cloud features output by S2, and outputs the point-by-point feature descriptor of the downsampled point cloud; the specific implementation process is described as follows:

[0086] S3-1, respectively input the enhanced features of the source point cloud and the target point cloud output in step 2 into the global aggregation layer, and after point-by-point maximum pooling, obtain two 256-dimensional feature vectors respectively;

[0087] S3-2, repeat and concatenate the two feature vectors obtained in the previous step to obtain features with dimensions of N'×1024 and M'×1024 respectively. Input these two feature matrices into the fully connected layer to obtain super-point features after aggregating global features, with dimensions of N'×256 and M'×256.

[0088] Step S4, construct the decoder layer; the decoder is used to generate dense feature descriptors of the original point cloud; the super-point features obtained in step S1 are fed into the decoder, and the decoder part uses nearest neighbor upsampling to obtain the final point-by-point feature descriptor; jump connection is used to pass the encoder features to the decoder; these features are connected to the upsampled features and processed by mlp; the output of the decoder is N×256 and M×256 dense original point features. The specific implementation process is described as follows:

[0089] S4, using the nearest neighbor upsampling, restores the downsampled point cloud to the density before downsampling, and obtains the features of the corresponding density point cloud. After three upsamplings, the dense features of the original point cloud are obtained.

[0090] Step S5, feature matching is performed through the point-by-point feature descriptors of the original point cloud to obtain the matching point set between the source point cloud and the target point cloud, and the transformation parameters are calculated using post-processing methods such as RANSAC; the input of this step is the matching point set of the original source point cloud and the target point cloud, and the output is the transformation parameter R,t from the source point cloud to the target point cloud; the specific implementation process is described as follows:

[0091] S5-1, calculate the Gaussian correlation between the super-point features in point clouds X' and Y':

[0092]

[0093] Then, through the bidirectional normalization operation, ambiguous matches are further suppressed;

[0094] S5-2, according to S ij Select the most reliable P pairs of superpoints; for each pair of superpoints, perform fine-grained matching between the two point sets around the superpoints, and finally obtain the matching point set of the original point cloud;

[0095] S5-3, the P matching point sets obtained in the previous step are used to calculate the transformation parameters R,t using the random sampling consensus algorithm RANSAC, and P groups R,t can be obtained as candidates;

[0096] S5-4, use the probability sampling method to select the key points of the source point cloud and the target point cloud according to the matching factor, and use RANSAC to estimate a set of global R,t; a total of P+1 sets of R,t are obtained;

[0097] S5-5, select the optimal R,t as the final result.

[0098] Step S6, use the point cloud registration datasets 3DMatch and KITTI to train the network constructed in steps S1-S5; the 3DMatch dataset is trained for 40 cycles, and the KITTI dataset is trained for 40 cycles; the specific implementation process is described as follows:

[0099] S6-1, using H samples for training, each sample contains two point clouds, the source point cloud and the target point cloud, and their corresponding real matching point sets and real transformation parameters;

[0100] S6-2, calculate the feature loss for the point cloud features output by S4; let the overlapping part of point cloud X and point cloud Y be X overlap and Y overlap ; For point x in point cloud X, ε x Y overlap The corresponding point set in is a positive sample; y Y overlap With ε x The difference between them is a negative sample; is the Euclidean distance between point features, Δ p , Δ n is the distance between positive samples and negative samples, Different weights are defined for each positive sample and negative sample; circle loss is used to calculate feature loss:

[0101]

[0102] S6-3, calculate the point matching loss for the super-point matching point set obtained in S5. i * , a soft assignment matrix Z is calculated through the optimal transport layer i * , represents the confidence of the point correspondence; random sampling N g The correspondence relationship between the real super points is set, and M i is the set of real point correspondences under a given super-point correspondence, I i and Ji For the set of points that do not match in point cloud X' and point cloud Y', calculate the negative log-likelihood loss:

[0103]

[0104] S6-5, add the above two losses to calculate the total network loss:

[0105] L=L c +L p

[0106] Step S7, using the model trained in step S6 to perform point cloud registration. First, the source point cloud and the target point cloud are input, and the model outputs the matching point set of the source point cloud and the target point cloud; then the RANSAC method is used for post-processing to obtain the point cloud registration result.

[0107] This method constructs and trains a point cloud registration neural network, which improves the network's robustness to rotation; and improves the accuracy of network feature matching by extracting rotation-invariant features and aggregating global features.

[0108] Embodiment 2

[0109] Based on the same inventive concept, the present invention also provides an electronic device, including one or more processors; a storage device for storing one or more programs; when one or more programs are executed by the one or more processors, the one or more processors implement the method described in Example 1.

[0110] Since the device introduced in the second embodiment of the present invention is an electronic device used to implement the low overlap point cloud registration method based on point pair feature convolution in the first embodiment of the present invention, a person skilled in the art can understand the specific structure and deformation of the electronic device based on the method introduced in the first embodiment of the present invention, so it is not repeated here. All electronic devices used in a method in the embodiment of the present invention belong to the scope of protection of the present invention.

[0111] Embodiment 3

[0112] Based on the same inventive concept, the present invention further provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processor, the method described in the first embodiment is implemented.

[0113] Since the device introduced in the third embodiment of the present invention is a computer-readable medium used to implement the low-overlap point cloud registration method based on point-pair feature convolution in the first embodiment of the present invention, a person skilled in the art can understand the specific structure and deformation of the electronic device based on the method introduced in the first embodiment of the present invention, so it is not repeated here. All electronic devices used in a method in the embodiment of the present invention belong to the scope of protection of the present invention.

[0114] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

Claims

1. Low-overlap 3D point cloud registration method based on point-to-point convolution, It is characterized in that The following steps are involved: Step S1: construct an encoder layer based on point kernel convolution to downsample the input source point cloud and the target point cloud to obtain a point cloud with lower density and point cloud features after convolution; Step S2: construct a geometry-guided coding layer, which includes two branches: geometry coding and semantic coding. The geometry coding branch inputs the source point cloud and the downsampled point cloud, and outputs the geometric features of the downsampled point cloud. The semantic coding branch inputs the downsampled point cloud and the convolutional point cloud features, and outputs the semantic features of the point cloud. The obtained geometric features and semantic features are input into the fully connected layer to obtain the fusion features of the source point cloud and the target point cloud r that are finally output by this layer. Step S3: construct a global aggregation layer, which is used to aggregate the global features of the source point cloud and the target point cloud; The input of the global aggregation layer is the fused features of step S2, and the output is the features of the point cloud after global aggregation; Step S4: Construct a decoder layer based on nearest neighbor upsampling, which is used to restore the features aggregated in step S3 to the dense features of the original point cloud, and pass the encoder features to the decoder through skip connection; the input of this layer is the features aggregated in S3, and the output is the point-by-point feature descriptor of the original point cloud; Step S5: performing feature matching through the point-by-point feature descriptors of the original point cloud to obtain a matching point set between the source point cloud and the target point cloud, and using a post-processing method to calculate the transformation parameters through the matching point set; Step S6: Use the data set with the real matching point set labels to train the deep learning network constructed by S1-S5; Step S7: Use the network trained in step S6 to perform point cloud registration.

2. The low-overlap 3D point cloud registration method based on point pair convolution according to claim 1, Features: The specific process of constructing the encoder layer based on point kernel convolution in step S1 is as follows: S1-1, the convolution kernel is a spherical area, and each kernel point has a weight W k : in, represents the kernel point, r represents the radius of the convolution kernel; S1-2, for each area that needs convolution, let the point obtained after convolution be x 0 , N x is the set of points in the convolution region, f is the feature of the points in the convolution region, PPF(x i ,x 0 ) is x i and x 0 The point pair features, x 0 is the center point of the convolution area, that is, the point obtained after convolution, x i For other points in the area, the convolution process is: The input original point cloud X is output through step S1 as the downsampled point cloud X' and feature F X’ , the points in X' are called super points, and g and h are intermediate variables.

3. The low-overlap 3D point cloud registration method based on point pair convolution according to claim 1, Features: The specific process of step S2 is: S2-1, input the original point cloud X and the downsampled point cloud X' into the geometry encoding branch in the geometry guidance layer, and use the K nearest neighbor method to estimate the normal of the downsampled point cloud; S2-2, select k neighboring points for each superpoint in X' and construct PPF features, N x’ is the normal of the point: Combine the PPF feature with x i ' and x i '-x ij 'Concatenate to obtain multi-dimensional features, x ij ' is x i 'Neighborhood points; S2-3, input the features obtained in S2-2 into the one-dimensional convolution layer and the maximum pooling layer to obtain the geometric features G output by the geometric encoding branch X’ ; S2-4, the downsampled point cloud X' and feature F X’ Input the semantic encoding branch in the geometric guidance layer, pass through the KNN graph neural network, and input it into the fully connected layer to obtain the semantic feature H output by the semantic encoding branch. X’ : S2-5, concatenate the features obtained by S2-3 and S2-4, input them into the fully connected layer, and obtain the geometric and semantically enhanced point cloud features F output by the geometry-guided encoding layer X ' gge : Among them, MLP is a multi-layer perceptron layer, and cat is feature concatenation.

4. The low-overlap 3D point cloud registration method based on point pair convolution according to claim 1, Features: The specific process of step 3 is as follows: S3-1, respectively input the enhanced features of the source point cloud and the target point cloud output in step 2 into the global aggregation layer, and obtain two feature vectors after point-by-point maximum pooling; S3-2, repeat and concatenate the two feature vectors obtained in the previous step, input the concatenated two feature matrices into the fully connected layer, and obtain the super-point features after aggregating the global features.

5. The low-overlap 3D point cloud registration method based on point pair convolution according to claim 4, Features: The specific process of step S4 is as follows: the decoder is used to generate a dense feature descriptor of the original point cloud; the super-point feature obtained in step S1 is sent to the decoder, and the decoder part uses the nearest neighbor upsampling to obtain the final point-by-point feature descriptor; the jump connection is used to pass the encoder feature to the decoder; These features are concatenated to the upsampled features and processed by the MLP; the output of the decoder is dense raw point features.

6. The low-overlap 3D point cloud registration method based on point pair convolution according to claim 1, Features: The specific process of step S5 is as follows: S5-1, calculate the Gaussian correlation between the super-point features in point clouds X' and Y': Then, through the bidirectional normalization operation, the ambiguous matching is further suppressed. X' is the downsampled point cloud of the original point cloud, Y' is the number of super points of the target point cloud, and H i is the super-point feature of the original point cloud, H j is the super-point feature of the target point cloud; S5-2, according to S ij Select the most reliable P pairs of superpoints; for each pair of superpoints, perform fine-grained matching between the two point sets around the superpoints, and finally obtain the matching point set of the original point cloud; S5-3, the P matching point sets obtained in the previous step are used to calculate the transformation parameters R,t respectively using the random sampling consensus algorithm, and P groups R,t can be obtained as candidates; S5-4, use the probability sampling method to select the key points of the source point cloud and the target point cloud according to the matching factor, and use the random sampling consensus algorithm to estimate a set of global R,t; a total of P+1 sets of R,t are obtained; S5-5, select the optimal R,t as the final result.

7. The low-overlap 3D point cloud registration method based on point pair convolution according to claim 1, Features: The specific process of step S5 is as follows: The specific process of step S6 is as follows: S6-1, using H samples for training, each sample contains two point clouds, the source point cloud and the target point cloud, and their corresponding real matching point sets and real transformation parameters; S6-2, calculate the feature loss for the point cloud features output by S4; let the overlapping part of point cloud X and point cloud Y be X overlap and Y overlap ; For point x in point cloud X, ε x Y overlap The corresponding point set in is a positive sample; y Y overlap With ε x The difference between them is a negative sample; is the Euclidean distance between point features, Δ p , Δ n is the distance between positive samples and negative samples, Different weights are defined for each positive and negative sample; the feature loss is calculated using the ring loss: S6-3, calculate the point matching loss for the super-point matching point set obtained in S5, for each super-point correspondence C i * , a soft assignment matrix Z is calculated through the optimal transmission layer i * , represents the confidence of the point correspondence; random sampling N g The correspondence relationship between the real super points is set, and M i is the set of real point correspondences under a given super-point correspondence, I i and J i For the set of points that do not match in point cloud X' and point cloud Y', calculate the negative log-likelihood loss: S6-5, add the above two losses to calculate the total network loss: L=L c +L p .

8. An electronic device, It is characterized in that include: one or more processors; A storage device for storing one or more programs; When one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

9. A computer readable medium having a computer program stored thereon, Features: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Low-overlap-ratio point cloud registration method

    CN113838109A

  • Point cloud registration method and system based on multi-scale feature voting

    CN115631341A

  • Point cloud registration method and system based on key-point-free detection

    CN115731272A

  • Local and global enhanced point cloud semantic segmentation graph convolutional network

    CN116524179A

  • Partially overlapped point cloud registration method and system based on unsupervised bidirectional completion

    CN117745774A