End-to-end point cloud registration method and system based on transformer model

Abstract

The application discloses an end-to-end point cloud registration method and system based on a Transformer model, relates to the technical field of industrial 3D vision, and aims at the low registration accuracy of the prior art under a low overlap rate. The application adopts a resample KPConv as a main body, and improves the registration performance of the model under a low overlap rate. A GNF feature fusion module capable of fusing enhanced neighborhood information of adjacent points is designed, the module is located at a bottleneck layer, and the robustness of the network when facing sparse and noisy data is enhanced. Finally, the application utilizes an 8-layer Transformer Encoder network to perform deeper interaction between source point cloud information and target point cloud information, so that more accurate conditional features can be obtained, the accuracy of point cloud registration under a low overlap rate can be improved according to simple Mlp output, and random rotation error and translation error can be effectively reduced.

Description

Technical Field

[0001] This invention relates to the field of industrial 3D vision technology, specifically to an end-to-end cloud registration method and system based on the Transformer model. Background Technology

[0002] 3D point cloud registration is a crucial component in numerous fields, including scene reconstruction, autonomous vehicle navigation, and robotics. Recently, with the development of deep learning technology, the need to construct an end-to-end registration algorithm that meets industrial requirements has become a priority. For object-level point cloud data, such as mobile phone frame data, the point density is often quite high. Because high-precision line laser scanners are used for data acquisition, a series of downsampling steps are required during preprocessing. Furthermore, the data exhibits characteristics such as symmetry and high overlap, further complicating 3D registration in industrial applications. In real-world applications, due to sensor viewing angle limitations and object occlusion, the low-overlap point cloud to be registered is often incomplete, with only partial overlap, resulting in a low overlap rate. This leads to the low accuracy of existing technologies in registering with low overlap rates. Summary of the Invention

[0003] The purpose of this invention is to address the problem of low accuracy in registration under low overlap conditions in existing technologies by providing an end-to-end cloud registration method and system based on the Transformer model.

[0004] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0005] The end-to-end cloud registration method based on the Transformer model includes the following steps:

[0006] Step 1: Targeting the source key points (0) x i and key target points (0) y i Deep feature extraction is performed separately to obtain source unconditional features and target unconditional features;

[0007] Step 2: Source Key Points (0) x i Key Targets (0) y i The source unconditional features and the target unconditional features are input into the GNF feature fusion module to obtain the fused features.

[0008] The GNF feature fusion module specifically performs the following steps:

[0009] Step 21: Use K-NN to analyze the source keypoints. (0) x i and key target points(0) y i The process is performed, and the results are compared with the source key points. (0) x i and key target points (0) y i After concatenation, the data undergoes normalization, dimensionality reduction, activation function processing, and max pooling to obtain the first-stage source feature description. (1) x i and the first stage target feature description (1) y i , (1) x i and (1) y i They are represented as follows:

[0010]

[0011] Among them, h θ This represents normalization, dimensionality reduction, and activation function processing; `max` represents max pooling; `cat` represents concatenation; and `x` is set. i ∈R b This represents the feature encoding of the source point cloud superpoint set, b represents the dimension of the feature matrix, and i,j∈ε represents the graph edge information between the source point cloud superpoint set and the target point cloud superpoint set;

[0012] Step 22: Use K-NN to describe the source features of the first stage. (1) x i and the first stage target feature description (1) y i The processed features are then compared with the source feature descriptions from the first stage. (1) x i and the first stage target feature description (1) y i After concatenation, the data undergoes normalization, dimensionality reduction, activation function processing, and max pooling to obtain the second-stage source feature description. (2) x i Second-stage target feature description (2) y i , (2) x i and (2) y i They are represented as follows:

[0013]

[0014] Steps 2 and 3: Describing the source features in the second stage using a multi-head attention mechanism (2) x i Second-stage target feature description (2) yi Enhancement is performed to obtain enhanced feature x. i GNF and enhanced features y i GNF x i GNF and y i GNF They are represented as follows:

[0015] x i GNF = (2) x i +MLP[cat(S i ,v i )]

[0016] y i GNF = (2) y i +MLP[cat(S i ,v i )]

[0017]

[0018] Where MLP stands for dimensionality reduction, S i and a i Let 'b' represent an intermediate variable, and 'b' represent the value of 'head'. Let q represent the learnable weight matrix. i Represents Query, k i Indicates Key, v i Represents Value;

[0019] Step Two Four: For x i GNF and y i GNF Normalization and activation function processing are performed separately to obtain the enhanced feature x. i 'and enhanced features y i ', x i 'and y i 'represented as:

[0020] x i '=w θ (x i GNF )

[0021] y i '=w θ (y i GNF )

[0022] Among them, w θ Represents the normalization and activation functions;

[0023] Step 25: Place x i 'and y i 'Concatenate the features to obtain fully connected features' Represented as:

[0024]

[0025] Step 26: Concatenate the source unconditional features and the target unconditional features to obtain the fully connected feature F, denoted as:

[0026]

[0027] Step 27: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.] The features are then fused with F to obtain the final fused features. Represented as:

[0028]

[0029] Among them, S θ This represents the splitting function; `add` means adding along the matrix dimensions.

[0030] Step 3: Merge the features After the source keypoints and target keypoints are cross-encoded in the Transformer Encoder, the output point cloud rigid transformation matrix and the predicted corresponding position coordinates are obtained through a three-layer MLP network.

[0031] Furthermore, the deep feature extraction in step one is performed through an improved resampling point convolutional network. The improved resampling point convolutional network includes 11 downsampling point convolutional modules and 1 resampling module. The 11 downsampling point convolutional modules are connected sequentially. The second downsampling point convolutional module and the tenth downsampling point convolutional module are connected in a skip connection every three layers. The eleventh downsampling point convolutional module is connected to the resampling module.

[0032] Furthermore, the multi-head attention mechanism is a four-head attention mechanism.

[0033] Furthermore, the predicted corresponding position coordinates are represented as follows:

[0034]

[0035] in, and These represent the predicted locations of the source and target point clouds, respectively. and These represent the actual locations of the source and target point clouds, respectively. express and The point set obtained after splicing express and The point set obtained after splicing This represents the indicator matrix.

[0036] Furthermore, the point cloud rigid transformation matrix is ​​expressed as:

[0037]

[0038] Where R and t represent the rotation matrix and translation matrix, respectively. and Let M' and N' represent the optimal rotation matrix and translation vector, respectively, and M' and N' represent the superpoint sets of the source point cloud and the target point cloud after feature extraction, respectively. and They represent and The i-th row.

[0039] An end-to-end cloud registration system based on the Transformer model, the system includes a feature extraction module, a feature fusion module, and a cross-coding module;

[0040] The feature extraction module is used to target source key points. (0) x i and key target points (0) y i Deep feature extraction is performed separately to obtain source unconditional features and target unconditional features;

[0041] The feature fusion module is used to integrate source key points. (0) x i Key Targets (0) y i The source unconditional features and the target unconditional features are input into the GNF feature fusion module to obtain the fused features.

[0042] The GNF feature fusion module specifically performs the following steps:

[0043] Step 1: Use K-NN to analyze the source keypoints. (0) x i and key target points (0) y i The process is performed, and the results are compared with the source key points. (0) x i and key target points (0) y i After concatenation, the data undergoes normalization, dimensionality reduction, activation function processing, and max pooling to obtain the first-stage source feature description.(1) x i and the first stage target feature description (1) y i , (1) x i and (1) y i They are represented as follows:

[0044]

[0045] Among them, h θ This represents normalization, dimensionality reduction, and activation function processing; `max` represents max pooling; `cat` represents concatenation; and `x` is set. i ∈R b This represents the feature encoding of the source point cloud superpoint set, b represents the dimension of the feature matrix, and i,j∈ε represents the graph edge information between the source point cloud superpoint set and the target point cloud superpoint set;

[0046] Step 2: Use K-NN to describe the source features in the first stage. (1) x i and the first stage target feature description (1) y i The processed features are then compared with the source feature descriptions from the first stage. (1) x i and the first stage target feature description (1) y i After concatenation, the data undergoes normalization, dimensionality reduction, activation function processing, and max pooling to obtain the second-stage source feature description. (2) x i Second-stage target feature description (2) y i , (2) x i and (2) y i They are represented as follows:

[0047]

[0048] Step 3: Describe the source features in the second stage using a multi-head attention mechanism. (2) x i Second-stage target feature description (2) y i Enhancement is performed to obtain enhanced feature x. i GNF and enhanced features y i GNF x i GNF and y i GNF They are represented as follows:

[0049] x i GNF = (2) x i +MLP[cat(S i ,v i )]

[0050] y i GNF = (2) y i +MLP[cat(S i ,v i )]

[0051]

[0052] Where MLP stands for dimensionality reduction, S i and a i Let 'b' represent an intermediate variable, and 'b' represent the value of 'head'. Let q represent the learnable weight matrix. i Represents Query, k i Indicates Key, v i Represents Value;

[0053] Step 4: For x i GNF and y i GNF Normalization and activation function processing are performed separately to obtain the enhanced feature x. i 'and enhanced features y i ', x i 'and y i 'represented as:

[0054] x i '=w θ (x i GNF )

[0055] y i '=w θ (y i GNF )

[0056] Among them, w θ Represents the normalization and activation functions;

[0057] Step 5: Set x i 'and y i 'Concatenate the features to obtain fully connected features' Represented as:

[0058]

[0059] Step 6: Concatenate the source unconditional features and the target unconditional features to obtain the fully connected feature F, denoted as:

[0060]

[0061] Step 7: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full The features are then fused with F to obtain the final fused features. Represented as:

[0062]

[0063] Among them, S θ This represents the splitting function; `add` means adding along the matrix dimensions.

[0064] The cross-coding module is used to process the fused features. After the source keypoints and target keypoints are cross-encoded in the Transformer Encoder, the output point cloud rigid transformation matrix and the predicted corresponding position coordinates are obtained through a three-layer MLP network.

[0065] Furthermore, the deep feature extraction in the feature extraction module is performed through an improved resampling point convolutional network. The improved resampling point convolutional network includes 11 downsampling point convolutional modules and 1 resampling module. The 11 downsampling point convolutional modules are connected sequentially. The second downsampling point convolutional module and the tenth downsampling point convolutional module are connected in a skip connection every three layers. The eleventh downsampling point convolutional module is connected to the resampling module.

[0066] Furthermore, the multi-head attention mechanism is a four-head attention mechanism.

[0067] Furthermore, the predicted corresponding position coordinates are represented as follows:

[0068]

[0069] in, and These represent the predicted locations of the source and target point clouds, respectively. and These represent the actual locations of the source and target point clouds, respectively. express and The point set obtained after splicing express and The point set obtained after splicing This represents the indicator matrix.

[0070] Furthermore, the point cloud rigid transformation matrix is ​​expressed as:

[0071]

[0072] Where R and t represent the rotation matrix and translation matrix, respectively. and Let M' and N' represent the optimal rotation matrix and translation vector, respectively, and M' and N' represent the superpoint sets of the source point cloud and the target point cloud after feature extraction, respectively. and They represent and The i-th row.

[0073] The beneficial effects of this invention are:

[0074] This application employs a resampled point convolutional network (Resample KPConv) as the backbone, improving the model's registration performance under low overlap rates. Furthermore, a GNF feature fusion module, located in the bottleneck layer, is designed to enhance the network's robustness to sparse and noisy data by fusing and enhancing neighborhood information between neighboring points. Finally, this application utilizes an 8-layer Transformer Encoder network to perform deeper interaction between the source and target point clouds, thereby obtaining more accurate conditional features. Finally, based on a simple MLP output, the accuracy of point cloud registration under low overlap rates can be improved, and random rotation and translation errors can be effectively reduced. Attached Figure Description

[0075] Figure 1 This is a flowchart of the application;

[0076] Figure 2 Here is a diagram of the GNFTR structure;

[0077] Figure 3 Here is a diagram of the Resample KPConv backbone network structure;

[0078] Figure 4 This is a structural diagram of the GNF feature fusion module;

[0079] Figure 5 This is a diagram of the Transformer encoding layer structure. Detailed Implementation

[0080] It should be noted that, where there is no conflict, the various embodiments disclosed in this application can be combined with each other.

[0081] Specific Implementation Method 1: The end-to-end point cloud registration method based on the Transformer model described in this implementation method first inputs the processed source and target point clouds into the ResampleKPConvBackbone. Through four layers of downsampling, the initial input point cloud data is sampled into a set of superpoints. Then, the set of superpoints and their corresponding features are fed into our designed GNF feature fusion module. The missing neighborhood information in the features is compensated through a self-attention mechanism and the K-NN algorithm. The dynamic weight allocation of the attention mechanism suppresses the influence of noise points and outliers. Subsequently, the features and corresponding point sets are input into the Transformer model for cross-encoding. Utilizing the powerful interaction capabilities of the Transformer, the features in the source and target point clouds undergo deep interaction. Finally, a three-layer MLP is used to predict the corresponding points in the source and target point clouds, outputting a rigid transformation matrix and the final registered point cloud data. The overall flowchart of this application is shown below. Figure 1 As shown.

[0082] Specifically:

[0083] Step 1: Create point cloud datasets of different categories, including scene-level and object-level point cloud data. Preprocess the datasets by performing random rotations, translations, and adding Gaussian noise. Divide them into training, validation, and test sets in a 7:2:1 ratio. The scene-level dataset is similar to 3DMatch, including point cloud sets for laboratory spaces and classrooms. The object-level point cloud data includes everyday items such as mobile phones, cups, and chairs. In practical applications, 3D point cloud data can be captured using a DeepVision smart camera. After random data augmentation and Gaussian noise addition, the data is converted to a PLY point cloud format.

[0084] Step 2. Using the training set obtained in Step 1, apply it to the network model for training. Input the source point cloud data and target point cloud data from the mobile phone frame dataset into the improved resampled point convolutional network for feature extraction. The network consists of 11 downsampled point convolutional layers and 1 resampled layer. Introducing the resampled module at the lowest level improves the model's robustness and registration recall in low overlap registration and effectively reduces errors. The resampled point convolutional module is used as the backbone network in the form of residual connections, increasing the diversity and effectiveness of the features obtained by the network, enabling it to acquire more in-depth features and improving its robustness and registration performance. A skip connection is required after every 3 downsampled layers, with the connection method resembling the ResNet residual form, to obtain deep features of the source and target point clouds, referred to as source unconditional features and target unconditional features, respectively. The resampled point convolutional module (Resample KPConv Block) is a point convolutional module (KPConv Block, such as...) Figure 3 A variant of (shown), it can dynamically adjust the position of feature points based on multi-scale feature extraction, thereby achieving point redistribution during the decoding process. This dynamic adjustment mechanism can not only effectively alleviate the interference of noise points on feature extraction, but also provide consistent feature representations across regions of different densities, ensuring that features in sparse regions can also be described with high quality.

[0085] In scene data matching tasks, since the interrelationships between many point cloud features often extend beyond their local neighborhoods, the attention mechanism can simultaneously consider both local details and global geometric relationships, thereby improving overall matching accuracy. Furthermore, when processing 3D point cloud data that is typically noisy and sparsely distributed, the attention mechanism can, based on the input from step 3, feed the corresponding keypoints and features obtained from downsampling into the GNF feature fusion module (structure as shown in step 3). Figure 4 As shown in the figure, this is used to balance the model's performance and utilizes an attention mechanism to simultaneously consider local details and global geometric relationships; the GNF feature fusion module is fed into the design for deep compensation of neighborhood information, where the following formula briefly describes the GNF representation of X and Y. The sparse point set generated by downsampling is selected, and then the K-NN algorithm is used to connect several sparse points into a three-dimensional spatial graph.

[0086] Step 3. The source keypoints and source unconditional features, target keypoints and target unconditional features obtained from downsampling are simultaneously fed into the GNF feature fusion module to balance model performance and utilize the attention mechanism to simultaneously consider local details and global geometric relationships. The following formula briefly describes the GNF representation of X (source point cloud) and Y (target point cloud). The source and target keypoints generated by downsampling are selected, and then the K-NN algorithm is used to connect several sparse points into a 3D spatial map. The feature encoding of the sparse point set is represented by x.i ∈R b The following steps involve normalizing the feature matrix formed by the concatenated K-NN (this application introduces self-attention into the K-NN algorithm. The main idea is to concatenate two K-NN modules, introduce three ports between the first and second K-NN layers, project the information from the three layers into the target vector through the multi-head self-attention mechanism q, k, v, and finally fuse them into the relevant information of the final GNF using the softmax algorithm) to normalize it. Then, dimensionality reduction and max pooling operations are performed on the K-NN representations of the concatenated source and target point clouds respectively. Thus, the overall formula for the first-stage feature description method can be written as:

[0087]

[0088] in (0) x i , (0) y i This represents the initial input source keypoints and target keypoints. The operation in the formula signifies that each calculation in the K-NN algorithm involves crossing the original keypoints and the target keypoints. (1) x i The first-stage K-NN representation of the source point cloud can be extended to... (1) x i This is the representation of the target point cloud in the first stage, where max is the max pooling operation, and h... θ This represents a series of operations such as normalization, dimensionality reduction, and activation functions. A second K-NN description will then be performed, and the same settings will be updated as follows:

[0089]

[0090] The symbols indicate that the various operations remain unchanged and all represent the same operational meaning. In the subsequent calculations, this application introduces a multi-head self-attention mechanism to enhance the features described by the K-NN, which can be specifically represented as:

[0091]

[0092] This formula describes a general formula for multi-head attention mechanisms, where we choose 4 heads, where W... O This represents the learnable weight matrix. Where:

[0093]

[0094] Where q i k i v iUnlike the traditional Q, K, V, we treat the K-NN descriptions at different stages as multiple projections of the multi-head self-attention mechanism, and we also need to perform the following calculations based on the above operations:

[0095]

[0096] In the above formula The formulas for this part are fixed computational steps in the attention mechanism, replacing the parameters originally used as random projection matrices with features from different stages of the k-NN description method. Therefore, this application encapsulates the above part into a novel GNF representation, which can be expressed as x. i GNF y i GNF The specific operation is completed according to the following formula:

[0097] x i GNF = (2) x i +MLP[cat(S i ,v i )]

[0098] y i GNF = (2) y i +MLP[cat(S i ,v i )]

[0099] Where x i GNF This represents the GNF representation of the source point cloud, and the GNF representation of the target point cloud is the same. After completing this part, I will perform distribution fusion on the useful feature parts.

[0100] The subsequent feature fusion process is needed to enhance the source and target keypoint features. The first step of the fusion process involves the following steps:

[0101] x i '=w θ (x i GNF )

[0102] y i '=w θ (y i GNF )

[0103] Where w θ This represents a series of operations, including normalization and activation functions. The subsequent feature fusion in the first stage is shown in the following formula:

[0104]

[0105] In the formula The process fuses the GNF descriptions of the source and target point clouds. The 'F' component fuses the features of both point clouds before they enter the GNF module, concatenating the unconditional features of the source and target point clouds into a single vector. This can be considered equivalent to a fully connected operation. The final two steps can then be expressed together using the following formula:

[0106]

[0107] Where S θ The splitting function represents the ability to split a long array matrix formed by a fully connected layer according to its original position, thus restoring the fully connected operation. `add` represents adding along the matrix dimensions, while `MLP` uses only one convolutional layer for dimensionality reduction.

[0108] This application is influenced by GNN and Transformer modules (Transformer coding layer structure diagram as shown in Figure 1). Figure 5 Inspired by the example shown, this application designs a GNF feature fusion module, which introduces a self-attention mechanism into the k-nn algorithm. It can dynamically adjust the weights of each point according to the context of the input features, focusing on the points that are crucial to the task, while suppressing the influence of noise or irrelevant points, thereby enhancing the robustness of the network when facing sparse and noisy data.

[0109] This application utilizes the GNF feature fusion module as a transitional structure for the bottleneck layer to capture multi-scale information of point clouds. This module enhances the ability to capture local and global features, which is a key factor in its excellent performance on mobile phone frame datasets. By dynamically adjusting the weights for each feature and establishing connections between points globally, this module enables the network to focus on those points that carry geometrically important information.

[0110] Step 4. Input the fused feature data and point information into the TransformerEncoder for cross-encoding.

[0111] The TransformerEncoder layer is used to perform deep cross-encode the features and keypoints corresponding to the source and target point clouds. First, each feature is enhanced by a self-attention module. At the same time, sine encoding is performed based on the keypoint matrix to encode the keypoint information and embed it into the corresponding feature information. Then, deep information interaction is performed through cross-attention. Finally, information is aggregated by a fully connected FFN. Finally, a three-layer MLP network is used to output the rigid transformation matrix of the point cloud and the predicted position coordinates.

[0112] The TransformerEncoder module in step 4 differs slightly from that in NLP tasks. It is a variant designed for point cloud data formats and can simultaneously use data as dual input channels. Feature encoding is particularly important in order to fuse global information and local feature details. It is more suitable for point cloud-related tasks and overcomes problems such as insufficient global position information and loss of absolute position information of points.

[0113] Step 5. Finally, output the rigid transformation matrix of the point cloud and predict the corresponding position coordinates through a three-layer MLP network. At the same time, use a single fully connected layer with sigmoid activation to predict the overlap confidence. This can be used to mask the impact of inaccurately predicted correspondences outside the overlapping region. The model's hyperparameters are then tuned using the validation set, and after determining the optimal hyperparameters, the model's registration performance is evaluated using the test set. The main evaluation metrics include registration recall (RR), random translation error (RTE), and random rotation error (RRE).

[0114] The various symbols in the formula are used, as shown in the following example, to connect the predicted transformation positions in two directions to obtain the final set of correspondences for M′+N′. This part is calculated using a weighted variant of the Kabsch-Umeyama algorithm to obtain a closed form:

[0115]

[0116] in, and These represent the predicted locations of the source and target point clouds, respectively. and These represent the actual locations of the source and target point clouds, respectively. express and The point set obtained after splicing express and The point set obtained after splicing This represents the indicator matrix, used to mark which points are valid corresponding points.

[0117] The rigid transformation matrix of a point cloud is represented as:

[0118]

[0119] Where R and t represent the rotation matrix and translation matrix, respectively. and Let M' and N' represent the optimal rotation matrix and translation vector, respectively, and M' and N' represent the superpoint sets of the source point cloud and the target point cloud after feature extraction, respectively. and They represent and The i-th row.

[0120] The rigid transformation matrix of a point cloud is a commonly used optimization objective function in point cloud registration, typically used in the Iterative Closest Point (ICP) algorithm or its variants. The goal of the formula is to find an optimal rotation matrix. Translation vector This minimizes the distance between the source point cloud and the target point cloud.

[0121] Objective function: The goal of the formula is to find an optimal rotation matrix. Translation vector The goal is to minimize the objective function, which is the sum of squared distances between all corresponding pairs of points.

[0122] Summation range: The summation range is from 1 to M′+N′, where M′ and N′ are the number of points in the source point cloud and the target point cloud, respectively, that participate in the registration.

[0123] Distance term: It is an indicator function used to mark points in the source point cloud. Are they related to points in the target point cloud? Match successful. otherwise,

[0124] Rotation and translation: Represents points in the source point cloud The coordinates after rotation and translation.

[0125] Target point: Points in the target point cloud and the source point cloud Corresponding points

[0126] distance: It is the square of the Euclidean distance between a point in the source point cloud and the corresponding point in the target point cloud after rotation and translation.

[0127] This application utilizes the Transformer Encoder module to perform deep cross-encodement of data. By integrating self-attention and cross-attention mechanisms, it achieves effective learning and fusion of point cloud features, improving the performance of point cloud registration tasks. Furthermore, its end-to-end design avoids the cumbersome steps of traditional methods, making the model simpler and easier to train.

[0128] In step 5, a three-layer MLP network is used to output the rigid transformation matrix of the point cloud and predict the corresponding position coordinates. The model's hyperparameters are then adjusted using a validation set to determine the optimal hyperparameters. Finally, the registration performance of the model is evaluated using a test set. This completes the registration task perfectly. To improve the aesthetics of the registration results, we provide source code that allows for easy replacement of weight files and observation of the registration visualization. The error values ​​between the source and target point clouds are also output during the process.

[0129] Experiments show that GNFTR not only demonstrates superior registration performance on mobile phone frame datasets, but also exhibits excellent performance on other scene-level and object-level datasets. This application's point cloud registration algorithm based on GNFTR achieves efficient, accurate, and rapid registration with excellent results. It provides strong support for industrial inspection.

[0130] It should be noted that the specific embodiments are merely explanations and illustrations of the technical solution of the present invention and should not be used to limit the scope of protection. Any modifications made in accordance with the claims and specification of the present invention that are only partial should still fall within the protection scope of the present invention.

Claims

1. An end-to-end cloud registration method based on the Transformer model, characterized in that... Includes the following steps: Step 1: Targeting the source key points (0) x i and key target points (0) y i Deep feature extraction is performed separately to obtain source unconditional features and target unconditional features; Step 2: Source Key Points (0) x i Key Targets (0) y i The source unconditional features and the target unconditional features are input into the GNF feature fusion module to obtain the fused features. The GNF feature fusion module specifically performs the following steps: Step 21: Use K-NN to analyze the source keypoints. (0) x i and key target points (0) y i The process is performed, and the results are compared with the source key points. (0) x i and key target points (0) y i After concatenation, the data undergoes normalization, dimensionality reduction, activation function processing, and max pooling to obtain the first-stage source feature description. (1) x i and the first stage target feature description (1) y i , (1) x i and (1) y i They are represented as follows: Among them, h θ This represents normalization, dimensionality reduction, and activation function processing; `max` represents max pooling; `cat` represents concatenation; and `x` is set. i ∈R b This represents the feature encoding of the source point cloud superpoint set, b represents the dimension of the feature matrix, and i,j∈ε represents the graph edge information between the source point cloud superpoint set and the target point cloud superpoint set; Step 22: Use K-NN to describe the source features of the first stage. (1) x i and the first stage target feature description (1) y i The processed features are then compared with the source feature descriptions from the first stage. (1) x i and the first stage target feature description (1) y i After concatenation, the data undergoes normalization, dimensionality reduction, activation function processing, and max pooling to obtain the second-stage source feature description. (2) x i Second-stage target feature description (2) y i , (2) x i and (2) y i They are represented as follows: Steps 2 and 3: Describing the source features in the second stage using a multi-head attention mechanism (2) x i Second-stage target feature description (2) y i Enhancement is performed to obtain enhanced feature x. i GNF and enhanced features y i GNF x i GNF and y i GNF They are represented as follows: x i GNF = (2) x i +MLP[cat(S i ,v i )] and i GNF = (2) and i +MLP[cat(S i ,v i )] Where MLP stands for dimensionality reduction, S i and a i Let 'b' represent an intermediate variable, and 'b' represent the value of 'head'. Let q represent the learnable weight matrix. i Represents Query, k i Indicates Key, v i Represents Value; Step Two Four: For x i GNF and y i GNF Normalization and activation function processing are performed separately to obtain the enhanced feature x. i 'and enhanced features y i ', x i 'and y i 'represented as: x i '=w θ (x i GNF ) y i '=w θ (y i GNF ) Among them, w θ Represents the normalization and activation functions; Step 25: Place x i 'and y i 'Concatenate the features to obtain fully connected features' Represented as: Step 26: Concatenate the source unconditional features and the target unconditional features to obtain the fully connected feature F, denoted as: Step 27: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.] The features are then fused with F to obtain the final fused features. Represented as: Among them, S θ This represents the splitting function; `add` means adding along the matrix dimensions. Step 3: Merge the features After the source keypoints and target keypoints are cross-encoded in the Transformer Encoder, the output point cloud rigid transformation matrix and the predicted corresponding position coordinates are obtained through a three-layer MLP network.

2. The end-to-end cloud registration method based on the Transformer model according to claim 1, characterized in that... In step one, deep feature extraction is performed using an improved resampling point convolutional network. This improved resampling point convolutional network includes 11 downsampling point convolutional modules and 1 resampling module. The 11 downsampling point convolutional modules are connected sequentially. The second downsampling point convolutional module and the tenth downsampling point convolutional module are connected every three layers. The eleventh downsampling point convolutional module is connected to the resampling module.

3. The end-to-end cloud registration method based on the Transformer model according to claim 1, characterized in that... The multi-head attention mechanism is a four-head attention mechanism.

4. The end-to-end cloud registration method based on the Transformer model according to claim 3, characterized in that... The predicted corresponding position coordinates are represented as follows: in, and These represent the predicted locations of the source and target point clouds, respectively. and These represent the actual locations of the source and target point clouds, respectively. express and The point set obtained after splicing express and The point set obtained after splicing This represents the indicator matrix.

5. The end-to-end cloud registration method based on the Transformer model according to claim 4, characterized in that... The rigid transformation matrix of the point cloud is represented as follows: Where R and t represent the rotation matrix and translation matrix, respectively. and Let M' and N' represent the optimal rotation matrix and translation vector, respectively, and M' and N' represent the superpoint sets of the source point cloud and the target point cloud after feature extraction, respectively. and They represent and The i-th row.

6. An end-to-end cloud registration system based on the Transformer model, characterized in that... The system includes a feature extraction module, a feature fusion module, and a cross-coding module; The feature extraction module is used to target source key points. (0) x i and key target points (0) y i Deep feature extraction is performed separately to obtain source unconditional features and target unconditional features; The feature fusion module is used to integrate source key points. (0) x i Key Targets (0) y i The source unconditional features and the target unconditional features are input into the GNF feature fusion module to obtain the fused features. The GNF feature fusion module specifically performs the following steps: Step 1: Use K-NN to analyze the source keypoints. (0) x i and key target points (0) y i The process is performed, and the results are compared with the source key points. (0) x i and key target points (0) y i After concatenation, the data undergoes normalization, dimensionality reduction, activation function processing, and max pooling to obtain the first-stage source feature description. (1) x i and the first stage target feature description (1) y i , (1) x i and (1) y i They are represented as follows: Among them, h θ This represents normalization, dimensionality reduction, and activation function processing; `max` represents max pooling; `cat` represents concatenation; and `x` is set. i ∈R b This represents the feature encoding of the source point cloud superpoint set, b represents the dimension of the feature matrix, and i,j∈ε represents the graph edge information between the source point cloud superpoint set and the target point cloud superpoint set; Step 2: Use K-NN to describe the source features in the first stage. (1) x i and the first stage target feature description (1) y i The processed features are then compared with the source feature descriptions from the first stage. (1) x i and the first stage target feature description (1) y i After concatenation, the data undergoes normalization, dimensionality reduction, activation function processing, and max pooling to obtain the second-stage source feature description. (2) x i Second-stage target feature description (2) y i , (2) x i and (2) y i They are represented as follows: Step 3: Describe the source features in the second stage using a multi-head attention mechanism. (2) x i Second-stage target feature description (2) y i Enhancement is performed to obtain enhanced feature x. i GNF and enhanced features y i GNF x i GNF and y i GNF They are represented as follows: x i GNF = (2) x i +MLP[cat(S i ,v i )] and i GNF = (2) and i +MLP[cat(S i ,v i )] Where MLP stands for dimensionality reduction, S i and a i Let 'b' represent an intermediate variable, and 'b' represent the value of 'head'. Let q represent the learnable weight matrix. i Represents Query, k i Indicates Key, v i Represents Value; Step 4: For x i GNF and y i GNF Normalization and activation function processing are performed separately to obtain the enhanced feature x. i 'and enhanced features y i ', x i 'and y i 'represented as: x i '=w θ (x i GNF ) y i '=w θ (y i GNF ) Among them, w θ Represents the normalization and activation functions; Step 5: Set x i 'and y i 'Concatenate the features to obtain fully connected features' Represented as: Step 6: Concatenate the source unconditional features and the target unconditional features to obtain the fully connected feature F, denoted as: Step 7: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full The features are then fused with F to obtain the final fused features. Represented as: Among them, S θ This represents the splitting function; `add` means adding along the matrix dimensions. The cross-coding module is used to process the fused features. After the source keypoints and target keypoints are cross-encoded in the TransformerEncoder, the output point cloud rigid transformation matrix and the predicted corresponding position coordinates are obtained through a three-layer MLP network.

7. The end-to-end cloud registration system based on the Transformer model according to claim 6, characterized in that... In the feature extraction module, deep feature extraction is performed through an improved resampling point convolutional network. The improved resampling point convolutional network includes 11 downsampling point convolutional modules and 1 resampling module. The 11 downsampling point convolutional modules are connected sequentially. The second downsampling point convolutional module and the tenth downsampling point convolutional module are connected in a skip connection every three layers. The eleventh downsampling point convolutional module is connected to the resampling module.

8. The end-to-end cloud registration system based on the Transformer model according to claim 6, characterized in that... The multi-head attention mechanism is a four-head attention mechanism.

9. The end-to-end cloud registration system based on the Transformer model according to claim 8, characterized in that... The predicted corresponding position coordinates are represented as follows: in, and These represent the predicted locations of the source and target point clouds, respectively. and These represent the actual locations of the source and target point clouds, respectively. express and The point set obtained after splicing express and The point set obtained after splicing This represents the indicator matrix.

10. The end-to-end cloud registration system based on the Transformer model according to claim 9, characterized in that... The rigid transformation matrix of the point cloud is represented as follows: Where R and t represent the rotation matrix and translation matrix, respectively. and Let M' and N' represent the optimal rotation matrix and translation vector, respectively, and M' and N' represent the superpoint sets of the source point cloud and the target point cloud after feature extraction, respectively. and They represent and The i-th row.

Images