A three-dimensional point cloud registration method based on feature interaction and reliable correspondence estimation
By employing feature interaction and reliable correspondence estimation methods, this paper utilizes graph neural networks and Transformer networks to extract point cloud features. By combining attention mechanisms and triangular descriptors, the problems of noise and partial overlap in point cloud registration are solved, achieving fast and accurate 3D point cloud registration.
Patent Information
- Application Number
- CN202310155735.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-02-23
- Publication Date
- 2026-01-20
- Estimated Expiration
- 2043-02-23
AI Technical Summary
Existing 3D point cloud registration methods struggle to achieve fast and accurate registration when faced with a large amount of noise and outliers, especially when there is partial overlap between point clouds. Traditional methods tend to converge to local optima, while deep learning methods are less robust to outliers, leading to a decrease in registration accuracy.
A method based on feature interaction and reliable correspondence estimation is adopted. Point cloud features are extracted by graph neural network, aggregated by Transformer network with position encoding, attention mechanism is used to identify overlapping areas, key points are sampled, and point cloud transformation matrix is estimated by triangular descriptor and singular value decomposition algorithm to improve registration accuracy and efficiency.
Even when point clouds contain noise and partial overlap, point cloud registration can be completed quickly and accurately, improving the robustness and accuracy of registration and adapting to the point cloud registration needs of complex scenarios.
Smart Images
Figure CN116128944B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to a three-dimensional point cloud model registration technology, in particular to a three-dimensional point cloud registration method based on feature interaction and reliable correspondence estimation. BACKGROUND
[0002] With the rapid development of three-dimensional point clouds, rigid point cloud registration is a crucial task in many fields, such as three-dimensional reconstruction, autonomous driving, object pose estimation, and digital protection of cultural relics. The goal is to find a rigid transformation that aligns two given point clouds. However, when using a laser radar scanner to obtain point cloud data, a large amount of noise and outliers are often introduced. Noise and outliers can cause errors in calculations, severely affecting the subsequent estimation of local features of the point cloud, and thus leading to failure of point cloud registration. Therefore, how to handle noise and outliers and find a good correspondence between two point clouds is still a key task in point cloud registration.
[0003] Currently, most popular registration methods are traditional registration methods based on optimization, among which the most classic method is the Iterative Closest Point (ICP) algorithm. Iterative Closest Point (ICP) is the most widely used traditional registration method. The algorithm includes two steps of correspondence search and transformation estimation, and iteratively solves the point cloud registration problem alternately. However, ICP is sensitive to noise and is prone to converge to a local optimum. Therefore, some ICP variants are proposed to improve performance by improving matching or searching a larger pose space. In addition, some traditional registration methods with complex optimization strategies are also designed. However, the registration accuracy of the above methods will decrease sharply when dealing with scenes with large initial position difference, strong noise interference, and small overlap.
[0004] In recent years, with the rapid development of deep learning, more and more attention has been paid to point cloud registration based on deep learning methods. However, previous work mainly tried to design complex neural networks to overcome the problems of a large amount of noise, outliers, and partial overlap of point clouds. However, although these methods have better performance than traditional methods, they are less robust to outliers and cannot handle local registration problems well. Moreover, the design of these complex structures will reduce the efficiency of point cloud registration. SUMMARY
[0005] In view of the deficiencies in the prior art, the purpose of the present application is to provide a three-dimensional point cloud registration method based on feature interaction and reliable correspondence estimation, which can quickly and accurately handle a large amount of outliers and partial overlap between point clouds.
[0006] To achieve the above purpose, the present application adopts the following technical solutions:
[0007] A three-dimensional point cloud registration method based on feature interaction and reliable correspondence estimation, comprising the following steps:
[0008] Step 1, obtaining multiple view three-dimensional point clouds of the model by a three-dimensional scanner;
[0009] Step 2, extracting features of two pieces of point clouds by using a graph neural network, and then using a Transformer network with position encoding to aggregate features of the target point cloud and the source point cloud;
[0010] Step 3, identifying the overlapping area between the two pieces of point clouds by an overlapping mask module based on an attention mechanism, and further sampling key points in the point cloud according to the area significance score;
[0011] Step 4, calculating a matching matrix by using shape features and three-dimensional coordinate features of the two pieces of point clouds;
[0012] Step 5, using a triangular descriptor to further distinguish in-line points and abnormal points, and combining an SVD algorithm to estimate the transformation matrix between the source point cloud and the target point cloud, thereby completing the registration of the three-dimensional point cloud.
[0013] Preferably, the process of extracting features of the two pieces of point clouds by using the graph neural network in step 2 comprises:
[0014] The initial features are represented by three-dimensional coordinates, the K nearest neighbor search is used to find the neighboring points, and N i represents the set of K points closest to the x i point, and the vector feature F i of the nth layer of the GNN is calculated as follows:
[0015]
[0016] Where f and g represent single-layer and multi-layer perceptron and two hidden layers of MLP respectively, and BN and ReLU activation functions are used after each layer of MLP.
[0017] Further, each layer of the graph neural network is stacked 5 times to increase the network depth, and the dimension of the output feature of each layer is 64, 64, 128, 256, and 512 respectively.
[0018] Preferably, the position encoding network of step 2 is composed of a fully connected layer (FC), sigmoid and ReLU activation, and its processing process and feature representation are as follows:
[0019] P X = ReLU(FC(Sigmoid(FC(X))))
[0020]
[0021] Where and respectively represent the learning of source point cloud and target point cloud features by GNN, P X and P Y represent the learned position encoding;
[0022] The features are aggregated by the Transformerφ, and the final point cloud features can be represented as:
[0023]
[0024] Preferably, the identification process of the overlapping area between the two pieces of point cloud in step 3 includes:
[0025] The overlapping mask module based on attention mechanism uses a set of channel cross-attention blocks to exchange feature information between the two point clouds, and the output feature dimensions of the channel cross-attention blocks are 512, 256, and 512 respectively.
[0026] Two feature vectors of point clouds X and Y are used as the input of the contrastive channel attention, and then the input feature vector of X is divided into a query operation Q X and a value operation V x .
[0027] The input feature vector of Y is divided into a key operation K y and a value operation V y ; Q x and K y are multiplied, and then the cross-attention matrix A xy and A yx are obtained by the softmax function; finally, the attention-based global features of the two point clouds X and Y are calculated and
[0028]
[0029]
[0030] Preferably, the sampling process of the key points in the point cloud includes:
[0031] The output feature dimensions of a set of MLPs with sizes of 512, 256, 128, and 1 are used to estimate the mask of the two point clouds, and the deep features are converted into saliency scores by a sigmoid layer, and the top K key points are selected.
[0032] Preferably, the matching matrix calculation process of the shape features and three-dimensional coordinate features of the two pieces of point cloud in step 4 includes:
[0033] The point cloud key point features are represented as f X (i)∈FX and f Y (j)∈F Y Then the shape feature tensor is represented as:
[0034]
[0035] The coordinate feature of a 3D point cloud is represented as c X (i)∈C X and c Y (j)∈C Y Then the three-dimensional coordinate tensor is represented as:
[0036]
[0037] Where [;] represents concatenation, and [·] represents the magnitude of the vector;
[0038] Two sets of MLPs with BN and ReLU are applied to the 3D coordinate tensor and the key point feature tensor to obtain the coordinate matching matrix and the feature matching matrix.
[0039] Finally, the two are added together to obtain the fused matching matrix.
[0040] Furthermore, the feature dimensions of the MLP output used to obtain the feature matching matrix are 256, 128, 128, and 1, respectively;
[0041] The feature dimensions of the MLP output used to obtain the coordinate matching matrix are 32, 32, 32, and 1, respectively.
[0042] Furthermore, step 5, which uses triangular descriptors to further distinguish inline points from outliers and combines them with the SVD algorithm to estimate the transformation matrix between the source and target point clouds, includes:
[0043] Based on the feature matching matrix M, the top p points are selected as the final representative points. Obtain the similarity matrix M' of the features;
[0044] Using KNN to search C' X Chinese x i Sampling point Idx X and Idx X With x i Combined to obtain g X g is obtained by mapping using the corresponding matrix M'. Y , use g X and g Y Calculate the length l of each triangle. X and l Y Then the error E r (x i ,y j ) and {xi ,y i} corresponding weight can be calculated as:
[0045]
[0046] w(x i ,y j )=2×sigmoid(-λE r (x i ,y j ))
[0047] Wherein min is the operation of taking k minimum values;λ is a parameter for adjusting the confidence estimate;
[0048] A threshold τ is set to filter the correspondence, if {x i ,y i} corresponding weight is greater than τ, the weight is set to 1, indicating that the corresponding triangle is similar, the point pair is in-line point;Otherwise, the weight is set to 0, indicating that the point pair is an outlier;
[0049] Finally, the filtered correspondence is combined with singular value decomposition (SVD) algorithm to estimate the transformation matrix.
[0050] Further, the threshold τ is set to 0.6.
[0051] Compared with the prior art, the present application has the following technical effects:
[0052] The three-dimensional point cloud registration network model proposed by the present application consists of five parts: point feature extraction, key point selection, matching matrix calculation, reliable correspondence evaluation and singular value decomposition (SVD);Specifically, after the simple graph neural network (GNN) extracts the point features, the Transformer with position encoding network is used to enrich the point features;In addition, the overlapping mask based on the attention mechanism is used to identify the overlapping area, and some interesting points are further sampled according to the area's saliency score;Finally, in order to find reliable corresponding points and improve the registration accuracy, the triangular descriptor is used to further distinguish the inliers and outliers;In order to speed up the point cloud registration, the key point selection module is introduced in the first stage to detect the overlapping area and learn the discriminative features;The local features and coordinate features are used to calculate the corresponding matching matrix, and the final matching matrix is obtained by fusion;Then, in the second stage, the triangular descriptor is used to obtain higher correspondence;This makes the point cloud registration network constructed by the present application still be able to quickly and accurately complete the three-dimensional point cloud registration under the condition that the point cloud contains a large amount of noise, outliers and partial overlap between point clouds. BRIEF DESCRIPTION OF DRAWINGS
[0053] Figure 1 The flowchart of the present application;
[0054] Figure 2 Point cloud registration network model diagram
[0055] Figure 3 Structure diagram of Transformer with position encoding
[0056] Figure 4 Structure diagram of attention-based key point selection module
[0057] Figure 5 Structure of reliable correspondence estimation module
[0058] Figure 6 Teracotta point cloud registration effect diagram DETAILED DESCRIPTION
[0059] The specific content of the application will be further explained and described in detail in combination with the embodiments.
[0060] Reference Figures 1-5 The embodiment provides a three-dimensional point cloud registration method based on feature interaction and reliable correspondence estimation, including the following steps:
[0061] Step 1, obtaining a plurality of three-dimensional point clouds of a model from multiple perspectives through a three-dimensional scanner;
[0062] Step 2, extracting features of two point clouds by using a graph neural network (GNN), and then aggregating features of target point clouds and source point clouds by using a Transformer network with position encoding; in order to identify features of adjacent structures, K nearest neighbor (KNN) is used to search for adjacent points, and N i The set of K points closest to the x i point in the middle can be expressed as i The vector feature F
[0063]
[0064] Where f and g represent single-layer and two-hidden-layer MLPs respectively, and BN and ReLU activation functions are used after each MLP. The three-dimensional coordinates of the points are used to represent the initial features, and one layer of the GNN network is stacked 5 times to increase the network depth, and the dimension of the output feature of each layer is 64, 64, 128, 256, and 512 respectively.
[0065] The position encoding network is composed of a fully connected layer (FC), a sigmoid, and a ReLU activation, which is used to learn the relationship between the two clouds; its processing process and features can be expressed as:
[0066] P X = ReLU (FC (Sigmoid (FC (X))))
[0067]
[0068] where and denote the source and target point cloud features learned by GNN, P X and P Y denote the learned positional encodings.
[0069] The features are aggregated by a Transformer φ, which consists of several stacked encoder and decoder layers. The encoder adopts a multi-head self-attention mechanism, and a shared-weight MLP encodes the features into embeddings in a feature space. The decoder mainly utilizes a multi-head cross-attention layer to correlate the two point cloud feature information. The final point cloud feature can be represented as:
[0070]
[0071] Step 3: Identify the overlapping area between the two point clouds through an attention mechanism-based overlapping mask module, and further sample the key points in the point cloud according to the region's saliency score; where a set of channel cross-attention blocks are first used to exchange feature information between the two point clouds. The dimension of the channel attention output feature is 512, 256, 512 respectively. The two feature vectors of point clouds X and Y are used as the input of the CCA block. Then, we divide the input feature vector of X into a query operation Q X and a value operation V x . In addition, we also divide the input feature vector of Y into a key operation K y and a value operation V y . Multiply Q x and K y , and then get the cross-attention matrix A xy and A yx through the softmax function. Finally, the attention-based global features of the two point clouds X and Y are calculated and
[0072]
[0073]
[0074] Finally, a set of MLPs with size (512, 256, 128, 1) are used to estimate the masks of the two point clouds, and the deep features are converted into saliency scores by a sigmoid layer. The higher the saliency score is, the more likely the point is located in the overlapping area of the two point clouds. We keep the K points with the highest saliency scores and discard the rest.
[0075] Step 4, shape features and three-dimensional coordinate features of the two point clouds are used to calculate the matching matrix respectively; wherein the point cloud key point features are represented as f X (i)∈F X and f Y (j)∈F Y , and the three-dimensional point cloud coordinate feature is represented as c X (i)∈C X and c Y (j)∈C Y , then the shape feature tensor and the three-dimensional coordinate tensor can be represented as:
[0076]
[0077]
[0078] Where [;] represents splicing, and [·] represents the modulus of the vector. The shape feature tensor integrates the distance and the relative direction of the features, and the three-dimensional coordinate tensor contains the three-dimensional coordinates of the source point cloud, the distance between the points, and the direction.
[0079] Finally, in order to obtain the coordinate and feature matching matrix, two sets of MLPs with BN and ReLU are applied to the three-dimensional coordinate tensor and the key point feature tensor. Then, in order to obtain the corresponding relationship of the correctly matched points, the two are added to obtain the fusion matching matrix M. The feature dimensions of the MLP output for obtaining the feature matching matrix are 256, 128, 128, 1 respectively; and the feature dimensions of the MLP output for obtaining the coordinate matching matrix are 32, 32, 32, 1 respectively.
[0080] Step 5, in order to find the reliable corresponding relationship of the two point clouds, the triangular descriptor is used to further distinguish the inlier points and the outlier points, and the SVD algorithm is combined to estimate the transformation matrix between the source point cloud and the target point cloud, so as to complete the registration of the three-dimensional point cloud. Wherein, some points without significant features in C X can be removed, and according to the feature matching matrix M, the top p points are selected as the final representative points In addition, the feature similarity matrix M' is obtained. Then, in order to further distinguish the inlier points and the outlier points, the triangular descriptor is used to capture the geometric features. By judging the similarity of the two triangular descriptors, the relationship between the points is determined. If {x i , y iThe correspondence between {x} is an inline pair, then when {x} i ,y i When the confidence value of} is high, the triangular descriptors will be very similar. First, use KNN to search for C' X Chinese x i Sampling point Idx X and Idx X With x i Combined to obtain g X Then, g is obtained by mapping using the corresponding matrix M'. Y Then use g X and g Y Calculate the length l of each triangle. X and l Y Final error E r (x i ,y j ) and {x i ,y i The corresponding weights can be calculated as follows:
[0081]
[0082] w(x i ,y j )=2×sigmoid(-λE r (x i ,y j ))
[0083] Where min is the operation of taking k minimum values; λ is the parameter for adjusting the confidence estimate.
[0084] Set a threshold τ to filter the corresponding relationships. If {x i ,y i If the weight of the corresponding triangle is greater than τ, the weight is set to 1, indicating that the triangles are similar and the point pair is an inline point; otherwise, the weight is set to 0, indicating that the point pair is an outlier. After multiple tests, the best registration effect was achieved when the threshold τ was set to 0.6 in this experiment.
[0085] Finally, the filtered correspondences are combined with the singular value decomposition (SVD) algorithm to estimate the transformation matrix.
[0086] Figure 6 The point cloud registration result after registration using the method described in this invention is shown in the figure. As can be seen from the figure, the method described in this invention can still robustly handle partial overlap between point clouds and the presence of noise.
Claims
1. A 3D point cloud registration method based on feature interaction and reliable correspondence estimation, characterized in that, Includes the following steps: Step 1: Obtain 3D point clouds of the model from multiple perspectives using a 3D scanner; Step 2: Use graph neural networks to extract features from the two point clouds, and then use a Transformer network with position encoding to aggregate the features of the target point cloud and the source point cloud. Step 3: Identify the overlapping region between two point clouds using an attention-based overlapping masking module, and sample key points in the point cloud based on the salience score of the region. The process of identifying the overlapping region between the two point clouds in step 3 includes: The attention-based overlapping masking module uses a set of channel cross attention blocks to exchange feature information between two point clouds. The dimensions of the output features of the channel cross attention blocks are 512, 256, and 512, respectively. Point cloud and The two feature vectors are used as inputs to the contrast-aware channel attention, and then... The input feature vector is divided into query operations. Sum operation ; Will The input feature vector is divided into a key operation A value operation ,Will and Multiply them, and then use the softmax function to obtain the cross-attention matrix. and Finally, the two point clouds were calculated. and Attention-based global features and : ； Step 4: Obtain the shape feature tensor using the key point features of the point cloud; obtain the three-dimensional coordinate tensor using the coordinate features of the three-dimensional point cloud; apply two sets of MLPs with BN and ReLU to the three-dimensional coordinate tensor and the shape feature tensor to obtain the coordinate matching matrix and the feature matching matrix; add the coordinate matching matrix and the feature matching matrix to obtain the feature matching matrix. Step 5: Select the top p representative points of the feature matching matrix to obtain the corresponding matrix; use triangular descriptors to distinguish inline points and outliers, and combine the SVD algorithm to estimate the transformation matrix between the source point cloud and the target point cloud; complete the registration of the 3D point cloud.
2. The 3D point cloud registration method based on feature interaction and reliable correspondence estimation as described in claim 1, characterized in that, The process of extracting features from the two point clouds using a graph neural network in step 2 includes: The initial features are represented using three-dimensional coordinates, and neighboring points are searched using K-nearest neighbors. Indicates closest Given a set of K points, calculate the GNN's 1st generation. Layer vector features Represented as: in, MLP representing a single-layer perceptron hidden layer This represents the MLP of the hidden layers of a multilayer perceptron, and BN and ReLU activation functions are used after each MLP layer.
3. The 3D point cloud registration method based on feature interaction and reliable correspondence estimation as described in claim 2, characterized in that, Each layer of the graph neural network is stacked 5 times to increase the network depth, and the dimensions of the output features of each layer are 64, 64, 128, 256, and 512, respectively.
4. The 3D point cloud registration method based on feature interaction and reliable correspondence estimation as described in claim 1 or 3, characterized in that, The Transformer network with positional encoding described in step 2 consists of fully connected layers, sigmoid, and ReLU activation. Its processing procedure and features are represented as follows: in and These represent learning the features of the source point cloud and the target point cloud using GNN, respectively. and This represents the learned positional encoding; Through Transformer The features are aggregated, and the final feature representation of the point cloud is as follows: 。 5. The 3D point cloud registration method based on feature interaction and reliable correspondence estimation as described in claim 1, characterized in that, The sampling process for key points in a point cloud includes: The masks of two point clouds are estimated using a set of output feature dimensions of MLP with sizes of 512, 256, 128, and 1. The deep features are converted into saliency scores through a sigmoid layer, and the top K key points are selected.
6. The 3D point cloud registration method based on feature interaction and reliable correspondence estimation as described in claim 1, characterized in that, Step 4 includes: The key point features of the point cloud are respectively represented as follows: and Then the shape feature tensor is represented as: The coordinate features of a 3D point cloud are represented as follows and Then the three-dimensional coordinate tensor is represented as: [;] indicates splicing. Represents the magnitude of a vector.
7. The 3D point cloud registration method based on feature interaction and reliable correspondence estimation as described in claim 6, characterized in that, The feature dimensions of the MLP output used to obtain the feature matching matrix are 256, 128, 128, and 1, respectively. The feature dimensions of the MLP output used to obtain the coordinate matching matrix are 32, 32, 32, and 1, respectively.
8. The 3D point cloud registration method based on feature interaction and reliable correspondence estimation as described in claim 7, characterized in that, Step 5 includes: Based on the feature matching matrix The top p points are selected as the final representative points. , The corresponding matrix of the features is obtained. ; Using KNN search middle sampling points and will and Combined Using the corresponding matrix Mapping ,use and Calculate the length of each triangle and Then the error and The corresponding weights can be calculated as follows: in It is an operation that takes the k minimum values; It is a parameter for adjusting the confidence level estimate; Set a threshold To filter the corresponding relationships, if and Point pairs between corresponding weights Greater than If the points are similar, the weight is set to 1, indicating that the corresponding triangles are similar and the point pair is an inline point; otherwise, the weight is set to 0, indicating that the point pair is an outlier. Finally, the filtered correspondences are combined with the singular value decomposition algorithm to estimate the transformation matrix.
9. The 3D point cloud registration method based on feature interaction and reliable correspondence estimation as described in claim 8, characterized in that, The threshold τ is set to 0.6.