A point cloud matching method based on tree structure converter

Abstract

The present invention provides a point cloud matching method based on a tree structure converter, introduces a tree structure and proposes a point cloud matching method based on a tree structure converter, so that the model can extract rich local features and achieve linear computational complexity through the learned participation area; proposes a tree attention, which realizes dynamic attention sparsification and extracts rich local features by specifying high-correlation key points layer by layer and based on a tree-structured point cloud; the method of the present invention achieves optimal performance on the 3DMatch, ModelNet40 and KITTI datasets.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional computer vision, and in particular relates to a point cloud matching method based on a tree structure converter. Background Art

[0002] Point cloud matching, which aims to derive the optimal transformation for aligning point clouds, is a fundamental problem in computer vision and robotics. With the continuous advancement of 3D representation learning, point cloud registration is evolving from traditional methods to learning-based approaches. The most widely known traditional method is the iterative closest point (ICP) ({Besl, 1992#79}), which iterates between establishing correspondences and computing rotational and translational transformations. However, when the initial error is large, ICP can easily become trapped in a local minimum. To improve registration accuracy, learning-based methods incorporate neural networks to extract features from each point separately and establish point-to-point correspondences based on feature similarity. However, the independence between point clouds hinders the identification of common structures and the extraction of discriminative features. To address this issue, recent transformer-based point cloud matching methods leverage the advantages of transformer models in terms of order invariance and modeling dependencies. They employ transformer models to enable one point cloud to perceive the other and extract contextual information between the pair, thereby enhancing the discriminative power of the extracted features.

[0003] Existing transformer-based point cloud matching networks generally use a standard attention module, which considers many points with low correlation and has difficulty in centrally distributing attention weights, both of which lead to limited local feature extraction capabilities and quadratic computational complexity. In addition to the standard attention module, there are also many local transformer models applied to point cloud processing tasks, which design local attention mechanisms to eliminate points with low correlation. Most of them sparse the attention map by restricting the receptive field to a predefined pattern. However, the sparsity of attention that is independent of the input content creates difficulties in accurately focusing on highly correlated regions and applying cross-attention between two misaligned point clouds. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a point cloud matching method based on a tree structure converter, which can extract rich local features.

[0005] A point cloud matching method based on a tree structure converter, comprising:

[0006] Step 1: For a given source point cloud X and target point cloud Y, downsample point clouds X and Y to and And extract relevant features, and then obtain features through linear projection again and

[0007] Step 2: Downsample the point cloud and extracted features Sent to the tree converter encoder for encoding, specifically including:

[0008] Step 21: Target point clouds separately and Building a tree structure and Specifically:

[0009] Point clouds and Voxelization, then merge multiple adjacent voxels into a large voxel to get a layer, and continue to merge multiple adjacent voxels of the layer to get a layer with larger voxels. The upper layer of the two adjacent layers is defined as a dense layer, and the points in it are defined as dense nodes. The lower layer is defined as a coarse layer, and the points in it are defined as coarse nodes. And so on, we get the point clouds respectively. and With L τ Layer tree structure and

[0010] The constructed tree structure and Definition: (1) Each layer contains points and points l=1,2,...,L τ ; (2) The relationship between the coarse nodes in each voxel in the next layer and the dense nodes in all the corresponding merged voxels in the previous layer is expressed as and c=1,2,...,L τ -1, d=c+1; (3) The relationship between the dense nodes in the merged voxels in the previous layer and the coarse nodes in the merged large voxels in the next layer is expressed as and Coordinates of coarse nodes and The coordinates of the dense nodes are and Find the average obtained, where the coarse nodes Coordinates Coordinates Expressed as:

[0011]

[0012] in, Represents the index of the child node corresponding to the i-th coarse node, where the coarse node of a certain voxel in the coarse layer is taken as the parent node, and the dense nodes in the merged voxels in the dense layer are called the child nodes of the coarse node; They are sets The cardinality, that is, the number of point clouds;

[0013] Step 22, feature pooling: connect the features of the child nodes with their relative positions and project them again using a two-layer multi-layer perceptron MLP, where the MLP consists of two fully connected layers and a linear rectifier function activation; the i-th coarse node Features

[0014] Features Obtained through the following methods:

[0015]

[0016] Step 23. Tree Attention: Feature Trees for Two Different Point Clouds and Tree attention first obtains the average attention map through global attention at the coarsest layer and the extracted features The next layer merges the extracted features To guide feature extraction, and according to the attention map Specify the participating area, and then perform the attention mechanism operation in the specified participating area; after that, the tree attention uses the shared parameters to iterate to the densest layer, thereby obtaining the conditional features Output, specifically including:

[0017] Step (1), for the feature tree For any two adjacent layers, calculate the average attention map of the coarse layer and the extracted features in is the number of key points associated with each query point in the coarse layer;

[0018] Step (2) extracts the features with high semantic information from the coarse layer Merge the features of the dense layer d obtained by feature pooling in step 22 To guide the feature extraction of dense layer d; among them, the combined features of the i-th dense node in dense layer d Obtained by:

[0019]

[0020] Step (3), use the attention map of the coarse layer c The participating area for attention operation at the specified dense layer d is: For each query point in the Key points in the feature tree In the dense layer d, determine the above The key points are all the child nodes corresponding to the parent node, and are the key points for attention calculation in the dense layer d; In the dense layer d, all child nodes corresponding to the parent nodes of each query point in the coarse layer c are determined as query points for attention operation in the dense layer d; thus, based on the combined features of the i-th dense node in the dense layer d Use the key points and query points determined by this layer to perform attention operations and obtain features and average attention map

[0021] The current dense layer is used as the coarse layer c for the next iterative calculation, and the layer below the current dense layer is used as the dense layer d for the next iterative calculation. The features of the dense layer d are calculated according to the method of steps (1) to (3). and average attention map And so on until the feature tree is calculated the densest layer;

[0022] Step (4), for the feature tree First, follow the method in step (1) to obtain the average attention map of the coarse layer c and the extracted features Then follow the method in step (2) to obtain the merged features Finally, the feature is obtained according to the method of step (3) and average attention map And so on until the feature tree is calculated the densest layer;

[0023] Get the feature number at different layers l=1,2,...,L τ Features and After that, the last layer of features and Conditional features as tree attention Output;

[0024] Step 3. Decoder: Conditional Features and Generate point cloud through two layers of MLP and The corresponding point cloud and And get the overlap score through a fully connected layer prediction and Obtain the final rotation and translation transformation by solving

[0025] The present invention provides a point cloud matching method based on a tree structure converter, which has the following beneficial effects:

[0026] 1. A tree structure is introduced and a point cloud matching method based on a tree structure transformer is proposed, which enables the model to extract rich local features and achieve linear computational complexity through the learned participation areas.

[0027] 2. Tree attention is proposed, which realizes dynamic attention sparsification and extracts rich local features by specifying high-correlation key points layer by layer and structuring point clouds based on trees.

[0028] 3. Extensive experiments show that our method outperforms the comparison methods and achieves the best performance on 3DMatch, ModelNet40, and KITTI datasets. In addition, ablation studies verify the advantages of each component. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is the overall framework of the point cloud matching method based on the tree structure converter of the present invention.

[0030] Figure 2 A schematic diagram of selecting an attention operation participation area in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0032] The present invention provides a point cloud matching method based on a tree structure converter, for a given source point cloud and target point cloud The goal of point cloud registration is to predict a rotation matrix and translation vectors Align the source point cloud with the target point cloud.

[0033] The overall framework of the point cloud matching method based on the tree structure converter of the present invention is as follows: Figure 1 First, kernel point convolution is used to downsample the point cloud X, Y to a smaller point set. And extract point features Subsequently, the tree transformer encoder learns context information and extracts features This feature has rich local information. These features are used to generate the corresponding point cloud in the decoder and predict the overlap score Finally, according to the predicted correspondence and overlap fraction Calculate the rotation and translation transformation matrix.

[0034] 1. Downsampling and feature extraction

[0035] The kernel point convolution backbone consists of residual network-like blocks and strided convolution for downsampling and feature extraction. arrive And extract relevant features, which are again linearly projected to obtain features

[0036] 2. Tree Transformer Encoder

[0037] Downsampled point cloud and extracted features is fed into the tree transformer encoder, which consists of a tree construction layer for representing the point cloud and L e The encoder layer consists of two feature pooling layers and two tree attention layers.

[0038] 1) Tree construction method: In order to focus the attention weights to sparse and meaningful points and gradually structure the point cloud, a tree structure is introduced to represent the point cloud. Specifically, L τ Layer Tree and Built on point cloud and First, the point cloud is voxelized, and then The adjacent voxels are merged into a large voxel to get a layer. The adjacent voxels are merged continuously to obtain a layer with larger voxels. The upper layer of the two adjacent layers is defined as a dense layer, and the points therein are defined as dense nodes. The lower layer is defined as a coarse layer, and the points therein are defined as coarse nodes. Similarly, the layer with L τ The tree structure of the layers.

[0039] The constructed tree and It is defined as: (1) Each layer contains points and points l=1,2,...,L τ ; (2) The relationship between the coarse nodes in each voxel in the next layer and the dense nodes in all the corresponding merged voxels in the previous layer is expressed as and (3) The relationship between the dense nodes in the merged voxels in the previous layer and the coarse nodes in the merged large voxels in the next layer is expressed as and c=1,2,...,L τ-1,d=c+1. Coordinates of the coarse node and The coordinates of the dense nodes are and Find the average obtained. Among them, the coarse nodes Coordinates Coordinates Expressed as:

[0040]

[0041] in, Represents the index of the child node corresponding to the i-th coarse node, where the coarse node of a certain voxel in the coarse layer is taken as the parent node, and the dense nodes in the merged voxels in the dense layer are called the child nodes of the coarse node; They are sets The cardinality, that is, the number of point clouds;

[0042] 2) Feature pooling: To aggregate information from dense nodes to coarse nodes and build a feature tree and l=1,2,...,L τ , the features of the dense layers are aggregated to the parent node, where the features of the densest layer utilize the input features. We consider the contribution of child nodes to the parent node to be related to their relative positions. In order to adaptively recalibrate the features point by point based on their contributions, the features of the child nodes are concatenated with their relative positions and projected again using a two-layer multi-layer perceptron (MLP), where the MLP consists of two fully connected layers and a rectified linear unit (ReLU) activation. The i-th coarse node Features Features Obtained through the following methods:

[0043]

[0044]

[0045] And so on, we get the tree structure and Corresponding feature tree and where l=1,2,...,L τ .

[0046] 3) Tree Attention: In order to capture important local structures and reduce computational complexity, a tree attention module is proposed to gradually specify the participating regions and structure the point cloud. Consider a general case where the feature trees of two different point clouds are given. and Tree attention first obtains the average attention map through global attention at the coarsest layer and the extracted features The next layer merges the extracted features To guide feature extraction, and according to the attention map Specify the participating area, and then perform the attention mechanism operation in the specified participating area. After that, the tree attention uses the shared parameters to iterate to the densest layer to obtain features with mutual information and

[0047] The tree attention is explained in detail below:

[0048] Step (1), for the feature tree For any two adjacent layers (coarse layer c and dense layer d), calculate the average attention map of the coarse layer and the extracted features in is the number of keypoints associated with each query point in the coarse layer.

[0049] Step (2) In order to enhance local feature extraction and aggregate multi-scale information, the features with high semantic information extracted by the coarse layer are Merge the features of the dense layer d obtained by feature pooling in step 2) to guide the feature extraction of the dense layer d.

[0050] Among them, the combined features of the i-th dense node in the dense layer d are Obtained by:

[0051]

[0052] Step (3), use the attention map of the coarse layer c The participating area for attention operation at the specified dense layer d is: For each query point in the Key points in the feature tree In the dense layer d, determine the above The key points are all the child nodes corresponding to the parent node, and are the key points for attention calculation in the dense layer d; In the dense layer d, all child nodes corresponding to the parent nodes of each query point in the coarse layer c are determined as query points for attention operation in the dense layer d; thus, based on the combined features of the i-th dense node in the dense layer d Use the key points and query points determined by this layer to perform attention operations and obtain features and average attention map

[0053] The current dense layer is used as the coarse layer c for the next iterative calculation, and the layer below the current dense layer is used as the dense layer d for the next iterative calculation. The features of the dense layer d are calculated according to the method of steps (1) to (3). and average attention map And so on until the feature tree is calculated The densest layer.

[0054] For feature trees First, follow the method in step (1) to obtain the average attention map of the coarse layer c and the extracted features

[0055] Then follow the method in step (2) to obtain the merged features

[0056] Finally, the feature is obtained according to the method of step (3) and average attention map

[0057] All output values and At different layers l=1,2,...,L τ Get. Features and is used as the output of the tree attention.

[0058] Step (4) The tree transformer encoder uses context information to update features in a series of tree attention-based self-attention and cross-attention modules, and takes the last attention module feature output as the final output feature

[0059] 3. Decoder

[0060] In the decoder, the conditional features and Generate point cloud through two layers of MLP and The corresponding point cloud and And predict the overlap score through a fully connected layer and Obtain the final rotation and translation transformation by solving

[0061] Example:

[0062] Table 1 Performance of various methods on 3DMatch and 3DLoMatch

[0063]

[0064]

[0065] 1. Matching performance on the 3DMatch dataset

[0066] 3DMatch: To demonstrate the performance of our method for real-world point cloud matching, experiments were conducted on 3DMatch {Huang, 2021#68}. The 3DMatch dataset is a real-world pairwise registration dataset with 46 scenes designed for training and the remaining 16 scenes evenly divided between validation and testing. Comparison methods are evaluated on the 3DMatch (overlap > 30%) {Zeng, 2017#136} and 3DLoMatch (overlap 10%-30%) {Huang, 2021#68} benchmarks.

[0067] Comparison Methods: Our method is compared with the state-of-the-art methods RegTR{Yew,2022#134}, Lepard{Li,2022#64}, and SC$^2$PCR{Chen,2022#67}. In addition, the comparison methods also include representative methods on 3DMatch: 3DSN{Gojcic,2019#127}, FCGF{Choy,2019#128}, CG-SAC{Quan,2020#124}, D3Fea{Bai,2020#129}, DGR{Choy,2020#62}, PCAM{Cao,2021#133}, MNet{Xu,2021#131}, DHVR{Lee,2021#126}, Predator{Huang,2021#68}, CoFiNet{Yu,2021#113}.

[0068] Evaluation method: The performance of each method was evaluated using the following evaluation methods: (1) Relative Rotation Error (RRE) which estimates the distance between the estimated and true rotation matrices, Relative Translation Error (RTE) which evaluates the Euclidean distance between the predicted and true values, and Registration Recall (RR) which evaluates the percentage of successful matching pairs, where a successful matching pair is defined as a matching pair with a root mean square error of correspondence less than 0.2.

[0069] Comparative Results Analysis: The comparative results are shown in Table 1. The results demonstrate that our method accurately aligns real-world point clouds even at low overlap rates and outperforms other methods on both 3DMatch and 3DLoMatch. Specifically, our method achieves a match recall of 95.1% on the 3DMatch benchmark, outperforming RegTR, Lepard, and SCPCR by 3.1%, 1.6%, and 1.8%, respectively. Compared to Lepard and SCPCR on the 3DloMatch benchmark, our method achieves superior match recall, reducing both the RRE and RTE by 26.8%-37.5%. Compared to RegTR on the 3DLoMatch benchmark, our method achieves a 4.7% advantage in match recall. The results on 3DLoMatch demonstrate that our method effectively extracts global information and enhances robustness to low overlap rates. Overall, the guidance from the learned attending regions and coarse features facilitates local feature extraction. Therefore, our method accurately and robustly aligns real-world point clouds with excellent accuracy and matching recall.

[0070] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A point cloud matching method based on a tree structure converter, characterized in that: include: Step 1: For a given source point cloud X and target point cloud Y, downsample point clouds X and Y to and And extract relevant features, and then obtain features through linear projection again and Step 2: Downsample the point cloud and extracted features Sent to the tree converter encoder for encoding, specifically including: Step 21: Target point clouds separately and Building a tree structure and Specifically: Point clouds and Voxelization, then merge multiple adjacent voxels into a large voxel to get a layer, and continue to merge multiple adjacent voxels of the layer to get a layer with larger voxels. The upper layer of the two adjacent layers is defined as a dense layer, and the points in it are defined as dense nodes. The lower layer is defined as a coarse layer, and the points in it are defined as coarse nodes. And so on, we get the point clouds respectively. and With L τ Layer tree structure and The constructed tree structure and Definition: (1) Each layer contains points and points l=1,2,...,L τ ; (2) The relationship between the coarse nodes in each voxel in the next layer and the dense nodes in all the corresponding merged voxels in the previous layer is expressed as and (3) The relationship between the dense nodes in the merged voxels in the previous layer and the coarse nodes in the merged large voxels in the next layer is expressed as and Coordinates of coarse nodes and The coordinates of the dense nodes are and Find the average obtained, where the coarse nodes Coordinates Coordinates Expressed as: in, Represents the index of the child node corresponding to the i-th coarse node, where the coarse node of a certain voxel in the coarse layer is taken as the parent node, and the dense nodes in the merged voxels in the dense layer are called the child nodes of the coarse node; They are sets The cardinality, that is, the number of point clouds; Step 22, feature pooling: connect the features of the child nodes with their relative positions and project them again using a two-layer multi-layer perceptron MLP, where the MLP consists of two fully connected layers and a linear rectifier function activation; the i-th coarse node Features Features Obtained through the following methods: Step 23. Tree Attention: Feature Trees for Two Different Point Clouds and Tree attention first obtains the average attention map through global attention at the coarsest layer and the extracted features The next layer merges the extracted features To guide feature extraction, and according to the attention map Specify the participating area, and then perform the attention mechanism operation in the specified participating area; after that, the tree attention uses the shared parameters to iterate to the densest layer, thereby obtaining the conditional features Output, specifically including: Step (1), for the feature tree For any two adjacent layers, calculate the average attention map of the coarse layer and the extracted features in is the number of key points associated with each query point in the coarse layer; Step (2) extracts the features with high semantic information from the coarse layer Merge the features of the dense layer d obtained by feature pooling in step 22 To guide the feature extraction of dense layer d; among them, the combined features of the i-th dense node in dense layer d Obtained by: Step (3), use the attention map of the coarse layer c The participating area for attention operation at the specified dense layer d is: For each query point in the Key points in the feature tree In the dense layer d, determine the above The key points are all the child nodes corresponding to the parent node, and are the key points for attention calculation in the dense layer d; In the dense layer d, all child nodes corresponding to the parent nodes of each query point in the coarse layer c are determined as query points for attention operation in the dense layer d; thus, based on the combined features of the i-th dense node in the dense layer d Use the key points and query points determined by this layer to perform attention operations and obtain features and average attention map The current dense layer is used as the coarse layer c for the next iterative calculation, and the layer below the current dense layer is used as the dense layer d for the next iterative calculation. The features of the dense layer d are calculated according to the method of steps (1) to (3). and average attention map And so on until the feature tree is calculated the densest layer; Step (4), for the feature tree First, follow the method in step (1) to obtain the average attention map of the coarse layer c and the extracted features Then follow the method in step (2) to obtain the merged features Finally, the feature is obtained according to the method of step (3) and average attention map And so on until the feature tree is calculated the densest layer; Get the feature number at different layers l=1,2,...,L τ Features and After that, the last layer of features and Conditional features as tree attention Output; Step 3. Decoder: Conditional Features and Generate point cloud through two layers of MLP and The corresponding point cloud and And get the overlap score through a fully connected layer prediction and Obtain the final rotation and translation transformation by solving

Images