A point cloud registration method, apparatus, terminal, and computer-readable storage medium

Abstract

This invention discloses a point cloud registration method, apparatus, terminal, and computer-readable storage medium. The method includes: acquiring a source point cloud and a target point cloud to be registered; inputting the source point cloud and the target point cloud into a trained point cloud registration model, and sequentially processing them through an initialization module, a transformer-based overlap score module, a self-attention mechanism-based mismatch point removal module, and a singular value decomposition module in the trained point cloud registration model to obtain a final rigid transformation matrix between the source point cloud and the target point cloud; and using the final rigid transformation matrix to transform the source point cloud to obtain a final source point cloud, thereby completing the final registration between the source point cloud and the target point cloud. This invention reduces the impact of overlapping regions and mismatch points on registration by clearing non-overlapping regions and removing mismatch points from the point cloud to be registered in the point cloud registration model, effectively improving the accuracy of point cloud registration.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and robot localization, and more particularly to a point cloud registration method, apparatus, terminal, and computer-readable storage medium. Background Technology

[0002] 3D point cloud registration technology aims to align two point cloud datasets of the same object or environment acquired from different locations, viewpoints, and sensors by performing coordinate transformation. This technology has wide applications in robotics, autonomous driving, 3D reconstruction, and many other fields.

[0003] Currently, traditional 3D point cloud registration methods are divided into geometric structure-based methods and deep learning-based methods. Among them, the Iterative Closest Point (ICP) method is the most widely used geometric structure-based method. ICP finds the closest points in two point clouds, establishes a correspondence between them, and then applies geometric transformations such as singular value transformations to gradually bring the two point clouds closer together, thereby reducing their distance. However, ICP is prone to getting trapped in local optima, failing to guarantee the accuracy of point cloud registration. Most deep learning-based registration methods do not further process non-overlapping points in the two point clouds. The negative impact of non-overlapping regions prevents the model from learning effective features. Furthermore, existing deep learning-based registration methods cannot effectively remove point clouds with incorrect correspondences, thus failing to guarantee the accuracy of point cloud registration.

[0004] Therefore, existing technologies have shortcomings and need to be improved and developed. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a point cloud registration method, apparatus, terminal and computer-readable storage medium to address the above-mentioned deficiencies of the prior art, and to solve the problem that the accuracy of point cloud registration cannot be guaranteed in the prior art.

[0006] The technical solution adopted by this invention to solve the technical problem is as follows:

[0007] In a first aspect, embodiments of the present invention provide a point cloud registration method, the method comprising:

[0008] Obtain the source point cloud and target point cloud to be registered;

[0009] The source point cloud and the target point cloud are input into a trained point cloud registration model, and then processed sequentially through the initialization module, the transformer-based overlap score module, the self-attention mechanism-based mismatch point removal module, and the singular value decomposition module in the trained point cloud registration model to obtain the final rigid transformation matrix between the source point cloud and the target point cloud.

[0010] The source point cloud is transformed using the final rigid transformation matrix to obtain the final source point cloud, thereby completing the final registration between the source point cloud and the target point cloud;

[0011] The initialization module generates an intermediate source point cloud to complete the initial registration of the source point cloud and the target point cloud. The transformer-based overlap score module calculates the overlap score of each point in the intermediate source point cloud and the target point cloud, and removes points in non-overlapping regions between the intermediate source point cloud and the target point cloud based on the overlap score. The self-attention-based mismatch point removal module identifies and removes mismatch points in the point cloud output by the overlap score module. The singular value decomposition module calculates the point cloud output by the mismatch point removal module to generate the final rigid transformation matrix between the source point cloud and the target point cloud.

[0012] In one implementation, the specific processing steps of the initialization module include:

[0013] Feature extraction is performed on the source point cloud and the target point cloud respectively to obtain global features of the source point cloud and global features of the target point cloud;

[0014] The global features of the source point cloud and the global features of the target point cloud are processed to obtain a first rotation matrix and a first translation vector.

[0015] The source point cloud is transformed using the first rotation matrix and the first translation vector to generate an intermediate source point cloud, thereby completing the initial registration of the source point cloud and the target point cloud.

[0016] In one implementation, the specific processing steps of the transformer-based overlap fraction module include:

[0017] Feature extraction is performed on the intermediate source point cloud and the target point cloud according to the preset feature dimensions to obtain high-dimensional features of the intermediate source point cloud and high-dimensional features of the target point cloud, respectively.

[0018] The high-dimensional features of the intermediate source point cloud and the high-dimensional features of the target point cloud are input into the transformer network of the overlap score module, and the first interaction feature and the second interaction feature between the intermediate source point cloud and the target point cloud are obtained based on the self-attention mechanism.

[0019] Max pooling is performed on the high-dimensional features of the intermediate source point cloud using the max pooling function to obtain the global features of the intermediate source point cloud.

[0020] The high-dimensional features of the intermediate source point cloud, the global features of the intermediate source point cloud, the global features of the target point cloud, and the first interaction feature are processed to obtain the overlap score of each point in the intermediate source point cloud; and the high-dimensional features of the target point cloud, the global features of the target point cloud, and the second interaction feature are processed to obtain the overlap score of each point in the target point cloud.

[0021] The overlap score of each point in the intermediate source point cloud and the target point cloud is compared with a preset score threshold. Points with scores below the preset score threshold are removed as points in the non-overlapping region between the intermediate source point cloud and the target point cloud to obtain the second source point cloud.

[0022] In one embodiment, the mismatch point removal module includes a dynamic graph convolutional neural network module; the specific processing steps of the mismatch point removal module based on the self-attention mechanism include:

[0023] The dynamic graph convolutional neural network module is used to process the second source point cloud and the target point cloud to obtain local features of the second source point cloud and local features of the target point cloud.

[0024] The local features of the second source point cloud and the local features of the target point cloud are processed to obtain a similarity matrix between the second source point cloud and the target point cloud. Each element in the similarity matrix represents the matching relationship between each pair of points in the second source point cloud and the target point cloud.

[0025] Each element in the similarity matrix is ​​compared with a preset threshold, and the points corresponding to elements that are lower than the preset threshold are taken as mismatch points.

[0026] Remove the mismatched points to obtain a third source point cloud, where any point in the third source point cloud has one and only one matching point in the target point cloud.

[0027] In one implementation, the specific processing steps of the singular value decomposition module include:

[0028] The point cloud to be processed is obtained by filtering the third source point cloud and the target point cloud according to a preset number of points;

[0029] The overlap score of each point in the point cloud to be processed is used as the weight of each point, and the point cloud to be processed and the weight of each point in the point cloud to be processed are used to construct the processing matrix;

[0030] The matrix to be processed is subjected to singular value decomposition to obtain the second rotation matrix and the second translation vector.

[0031] Based on the first rotation matrix and the second rotation matrix, a final rotation matrix is ​​obtained. Based on the second rotation matrix, the first translation vector, and the second translation vector, a final translation vector is obtained. The final rigid transformation matrix includes the final rotation matrix and the final translation vector.

[0032] In one implementation, the training steps of the point cloud registration model include:

[0033] Construct a training dataset, which includes a training source point cloud and a training target point cloud;

[0034] An initial point cloud registration model is constructed, comprising an initialization training module, a fine registration training module, and a singular value transform training module. The initialization training module generates intermediate training source point clouds to complete coarse registration between the training source point clouds and the training target point clouds. The fine registration training module includes a transformer-based overlap score training module and a self-attention-based mismatch point removal training module. The overlap score training module calculates the training overlap score for each point in the intermediate training source point cloud and the training target point cloud, and removes points in non-overlapping regions between the two points based on the training overlap score. The mismatch point removal training module identifies and removes mismatched points from the point cloud output by the overlap score training module. The singular value transform training module calculates the point cloud output by the mismatch point removal training module to generate the final rigid transformation training matrix between the training source point cloud and the training target point cloud.

[0035] The initial point cloud registration model is trained multiple times using the training dataset until the loss function reaches the preset target, thus obtaining the trained point cloud registration model.

[0036] In one implementation, constructing the dataset includes:

[0037] Obtain the target open-source dataset, and then perform cropping and sampling on the target open-source dataset in sequence to obtain the original training source point cloud and the original training target point cloud;

[0038] The original training source point cloud is randomly rotated and randomly translated along the x-axis, y-axis and z-axis to obtain the second training source point cloud;

[0039] Gaussian noise is added to the second training source point cloud and the original training target point cloud to obtain the third training source point cloud and the second training target point cloud.

[0040] The second training target point cloud is randomly selected according to a preset probability to obtain a third training target point cloud and a fourth training target point cloud that has not been selected. The third training target point cloud is flipped to obtain a fifth training target point cloud. The training target point cloud includes the fourth training target point cloud and the fifth training target point cloud.

[0041] The fourth training source point cloud corresponding to the fourth training target point cloud is obtained from the third training source point cloud, and the fifth training target point cloud is randomly rotated and randomly translated along the x-axis, y-axis and z-axis to obtain the fifth training source point cloud. The training source point cloud includes the fourth training source point cloud and the fifth training source point cloud.

[0042] Secondly, embodiments of the present invention also provide a point cloud registration device, comprising:

[0043] The acquisition module is used to acquire the source point cloud and target point cloud to be registered;

[0044] The point cloud processing module is used to input the source point cloud and the target point cloud into a trained point cloud registration model, and then process them sequentially through an initialization module, a transformer-based overlap score module, a self-attention mechanism-based mismatch point removal module, and a singular value decomposition module to obtain the final rigid transformation matrix between the source point cloud and the target point cloud.

[0045] The initialization module is used to generate an intermediate source point cloud to complete the initial registration of the source point cloud and the target point cloud. The overlap score module based on transformer is used to calculate the overlap score of each point in the intermediate source point cloud and the target point cloud, and remove points in the non-overlapping region between the intermediate source point cloud and the target point cloud based on the overlap score. The mismatch point removal module based on self-attention mechanism is used to identify and remove mismatch points in the point cloud output by the overlap score module. The singular value decomposition module is used to calculate the point cloud output by the mismatch point removal module to generate the final rigid transformation matrix between the source point cloud and the target point cloud.

[0046] The final registration module is used to transform the source point cloud using the final rigid transformation matrix to obtain the final source point cloud, thereby completing the final registration between the source point cloud and the target point cloud.

[0047] Thirdly, embodiments of the present invention also provide a terminal, including: a memory, a processor, and a point cloud registration program stored in the memory and executable on the processor, wherein the point cloud registration program, when executed by the processor, implements the steps of the point cloud registration method as described above.

[0048] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that can be executed to implement the steps of the point cloud registration method described above.

[0049] The beneficial effects of this invention are as follows: In this embodiment, the source and target point clouds to be registered are obtained; the source and target point clouds are input into a trained point cloud registration model, and sequentially processed by an initialization module, a transformer-based overlap score module, a self-attention-based mismatch point removal module, and a singular value decomposition module within the trained point cloud registration model to obtain the final rigid transformation matrix between the source and target point clouds; the source point cloud is then transformed using the final rigid transformation matrix to obtain the final source point cloud, thus completing the final registration between the source and target point clouds. This invention reduces the impact of overlapping regions and mismatch points on registration by clearing non-overlapping regions and removing mismatch points from the point cloud to be registered in the point cloud registration model, effectively improving the accuracy of point cloud registration. Attached Figure Description

[0050] Figure 1 This is a flowchart of a preferred embodiment of the point cloud registration method in this invention.

[0051] Figure 2 This is a schematic diagram illustrating the generation of the final rigid transformation matrix in this invention.

[0052] Figure 3 This is a functional principle block diagram of a preferred embodiment of the point cloud registration device in this invention.

[0053] Figure 4 This is a functional principle block diagram of a preferred embodiment of the terminal in this invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0055] Traditional 3D point cloud registration methods are divided into geometric structure-based methods and deep learning-based methods.

[0056] On the one hand, the Iterative Closest Point (ICP) method is the most widely adopted geometry-based approach. ICP solves the rigid transformation between two point clouds by iteratively solving two subproblems. It finds the nearest point based on the current spatial coordinates of the point cloud, establishing a correspondence between points, and uses Singular Value Decomposition (SVD) to calculate the optimal transformation. However, ICP requires a small difference between the two frames of points, i.e., a good initial position, and is easily affected by local optima, failing to guarantee the accuracy of point cloud registration and unable to solve the entire point cloud registration problem. Global Optimization ICP (GO-ICP), as an improvement on ICP, uses pruning algorithms to alleviate the local optima problem, but its time cost is high. Simultaneously, both ICP and GO-ICP struggle with partial-to-part point cloud registration problems and have low robustness to noise.

[0057] On the other hand, deep learning-based methods use multilayer perceptrons or transformer networks. These methods leverage the powerful learning capabilities of deep networks to fully learn the structural information of point clouds, thus significantly reducing point cloud registration errors. However, for partial-to-part point clouds, there are non-overlapping points between the two point clouds, which can affect the extraction of point cloud features. Many deep learning-based methods do not handle non-overlapping points, so when a large number of non-overlapping points exist, traditional deep learning-based methods struggle to achieve accurate results. Furthermore, most deep learning-based registration methods cannot effectively remove point clouds with incorrect correspondences, thus failing to guarantee the accuracy of point cloud registration.

[0058] To address the aforementioned deficiencies in existing technologies, this invention discloses a point cloud registration method, apparatus, terminal, and computer-readable storage medium. The method includes: acquiring a source point cloud and a target point cloud to be registered; inputting the source point cloud and the target point cloud into a trained point cloud registration model, and sequentially processing them through an initialization module, a transformer-based overlap score module, a self-attention mechanism-based mismatch point removal module, and a singular value decomposition module within the trained point cloud registration model to obtain a final rigid transformation matrix between the source point cloud and the target point cloud; and using the final rigid transformation matrix to transform the source point cloud to obtain a final source point cloud, thereby completing the final registration between the source point cloud and the target point cloud. This invention reduces the impact of overlapping regions and mismatch points on registration by clearing non-overlapping regions and removing mismatch points from the point cloud to be registered in the point cloud registration model, effectively improving the accuracy of point cloud registration.

[0059] Please see Figure 1The point cloud registration method described in this embodiment of the invention includes the following steps:

[0060] Step S100: Obtain the source point cloud and target point cloud to be registered.

[0061] Specifically, the source point cloud and the target point cloud to be registered are obtained, and the source point cloud is aligned with the target point cloud.

[0062] like Figure 1 As shown, the point cloud registration method further includes the following steps:

[0063] Step S200: Input the source point cloud and the target point cloud into a trained point cloud registration model. The model sequentially processes the initialization module, the transformer-based overlap score module, the self-attention-based mismatch point removal module, and the singular value decomposition module to obtain the final rigid transformation matrix between the source point cloud and the target point cloud. Specifically, the initialization module generates an intermediate source point cloud to complete the initial registration of the source and target point clouds; the overlap score module calculates the overlap score for each point in the intermediate source and target point clouds and removes points in non-overlapping regions based on the overlap score; the mismatch point removal module identifies and removes mismatched points in the point cloud output by the overlap score module; and the singular value decomposition module calculates the point cloud output by the mismatch point removal module to generate the final rigid transformation matrix between the source and target point clouds.

[0064] Specifically, this invention inputs the source point cloud and the target point cloud into a point cloud registration model. The point cloud registration model includes an initialization module, a transformer-based overlap score module, a self-attention-based mismatch point removal module, and a singular value decomposition module. Through the sequential processing of these four modules, the final rigid transformation matrix between the source point cloud and the target point cloud can be obtained. In this invention, the initialization module is used for coarse registration of the point clouds, and then the overlap score module and the mismatch point removal module are used for fine registration. By using the transformer-based overlap score module to remove points in non-overlapping regions between the intermediate source point cloud and the target point cloud, and then using the self-attention-based mismatch point removal module to remove mismatch points, the negative impact of non-overlapping regions and mismatch points can be effectively eliminated, ensuring the model's effective extraction of point cloud features, thereby improving the accuracy of point cloud registration.

[0065] In one implementation, the specific processing steps of the initialization module include:

[0066] Feature extraction is performed on the source point cloud and the target point cloud respectively to obtain global features of the source point cloud and global features of the target point cloud;

[0067] The global features of the source point cloud and the global features of the target point cloud are processed to obtain a first rotation matrix and a first translation vector.

[0068] The source point cloud is transformed using the first rotation matrix and the first translation vector to generate an intermediate source point cloud, thereby completing the initial registration of the source point cloud and the target point cloud.

[0069] Specifically, the initialization module includes a PointNet network. The PointNet network is used to extract global features of the source point cloud and global features of the target point cloud and connect them to obtain comprehensive features. Then, a decoder composed of a fully connected layer network is used to decode the comprehensive features to obtain a first rotation matrix and a first translation vector. The goal of decoding is to apply the first rotation matrix and the first translation vector to the source point cloud so that the source point cloud and the target point cloud are registered.

[0070] The specific operation process of the initialization module can be represented as follows: Where v is a seven-dimensional vector, including the quaternion corresponding to the first rotation matrix R1 and the first translation vector t1. The source point cloud global features are encoded using the PointNet network. This refers to the global features of the target point cloud encoded using the PointNet network. The `cat` function is used for feature concatenation, and `h`... θ This represents the decoding process of the decoding layer.

[0071] The intermediate source point cloud can be represented as X. * =R1·X+t1, where X represents the source point cloud, R1 is the first rotation matrix, and t1 is the first translation vector.

[0072] In point cloud registration, the conventional operation is to select a subset of points from the non-overlapping regions for registration; these points are called "dependent points." The initialization module of this invention creates a good initial position for selecting these "dependent points" without making the point cloud registration model too large. This step achieves "coarse registration."

[0073] In one implementation, the specific processing steps of the overlapping fraction module include:

[0074] Feature extraction is performed on the intermediate source point cloud and the target point cloud according to the preset feature dimensions to obtain high-dimensional features of the intermediate source point cloud and high-dimensional features of the target point cloud, respectively.

[0075] The high-dimensional features of the intermediate source point cloud and the high-dimensional features of the target point cloud are input into the transformer network of the overlap score module, and the first interaction feature and the second interaction feature between the intermediate source point cloud and the target point cloud are obtained based on the self-attention mechanism.

[0076] Max pooling is performed on the high-dimensional features of the intermediate source point cloud using the max pooling function to obtain the global features of the intermediate source point cloud.

[0077] The high-dimensional features of the intermediate source point cloud, the global features of the intermediate source point cloud, the second global features of the target point cloud, and the first interactive features are processed to obtain the overlap score of each point in the intermediate source point cloud; and the high-dimensional features of the target point cloud, the global features of the target point cloud, and the second interactive features are processed to obtain the overlap score of each point in the target point cloud.

[0078] The overlap score of each point in the intermediate source point cloud and the target point cloud is compared with a preset score threshold. Points with scores below the preset score threshold are removed as points in the non-overlapping region between the intermediate source point cloud and the target point cloud to obtain the second source point cloud.

[0079] Specifically, features are extracted from the intermediate source point cloud and the target point cloud according to a preset feature dimension to obtain high-dimensional features of the intermediate source point cloud. and the high-dimensional features F of the target point cloud Y ; to extract the high-dimensional features of the intermediate source point cloud and the high-dimensional features F of the target point cloud Y The first interaction features between the intermediate source point cloud and the target point cloud are obtained in the transformer network of the overlapping score module based on the self-attention mechanism. The second interaction feature between the intermediate source point cloud and the target point cloud High-dimensional features of the intermediate source point cloud are obtained using the max pooling function. Max pooling is then performed to obtain the global features of the intermediate source point cloud.

[0080] The overlap score module also includes an MLP network, used to obtain the overlap score of the point cloud based on various feature information of the point cloud. After the high-dimensional features of the intermediate source point cloud, the global features of the intermediate source point cloud, the first interaction feature, the second interaction feature, the high-dimensional features of the target point cloud, and the global features of the target point cloud have all been extracted, these features are processed and concatenated, and the overlap score is obtained using the MLP network.

[0081] The overlap score of each point in the intermediate source point cloud can be expressed as: Among them, S X*Let $\mathbf{h}$ be the overlap score of each point in the intermediate source point cloud, $\mathbf{h}$ be the MLP network, and $\mathbf{cat}$ be the feature concatenation function. This indicates that the global features of the intermediate source point cloud will be augmented. This indicates that the global features of the target point cloud will be augmented. This indicates that the difference features between the global features of the intermediate source point cloud and the global features of the target point cloud are expanded. This is the first interactive feature.

[0082] The overlap score of each point in the target point cloud can be represented as: Among them, S Y The overlap score for each point in the target point cloud. This indicates that the difference features between the global features of the target point cloud and the global features of the intermediate source point cloud are expanded. This is the second interactive feature.

[0083] This invention utilizes a transformer network and a self-attention mechanism to obtain the first and second interaction features of the intermediate source point cloud and the target point cloud, and performs subsequent calculations to obtain the overlap score of each point more accurately.

[0084] In one implementation, the mismatch point removal module includes a dynamic graph convolutional neural network module; the specific processing steps of the mismatch point removal module based on the self-attention mechanism include:

[0085] The dynamic graph convolutional neural network module is used to process the second source point cloud and the target point cloud to obtain local features of the second source point cloud and local features of the target point cloud.

[0086] The local features of the second source point cloud and the local features of the target point cloud are processed to obtain a similarity matrix between the second source point cloud and the target point cloud. Each element in the similarity matrix represents the matching relationship between each pair of points in the second source point cloud and the target point cloud.

[0087] Each element in the similarity matrix is ​​compared with a preset threshold, and the points corresponding to elements that are lower than the preset threshold are taken as mismatch points.

[0088] Remove the mismatched points to obtain a third source point cloud, where any point in the third source point cloud has one and only one matching point in the target point cloud.

[0089] Specifically, the dynamic graph convolutional neural network module is used to process the second source point cloud and the target point cloud to obtain local features of the second source point cloud and the target point cloud. These local features are then input into a self-attention network to obtain a similarity matrix H between the second source point cloud and the target point cloud. The similarity matrix is ​​represented as follows: in, The second source point cloud local feature vector is the output of the self-attention network, representing the local features of the second source point cloud. H is the local feature vector of the target point cloud, output by the self-attention network after processing the local features of the target point cloud. H is an N*M matrix, where N is the number of points in X' and M is the number of points in Y. The rows of this matrix represent the matching relationship between each point in X' and each point in Y.

[0090] The accuracy of point cloud registration can be further improved by removing mismatched points.

[0091] In one implementation, the preset threshold is adjustable to meet actual needs.

[0092] In one implementation, the specific processing steps of the singular value decomposition module include:

[0093] The point cloud to be processed is obtained by filtering the third source point cloud and the target point cloud according to a preset number of points;

[0094] The overlap score of each point in the point cloud to be processed is used as the weight of each point, and the point cloud to be processed and the weight of each point in the point cloud to be processed are used to construct the processing matrix;

[0095] The matrix to be processed is subjected to singular value decomposition to obtain the second rotation matrix and the second translation vector.

[0096] Based on the first rotation matrix and the second rotation matrix, a final rotation matrix is ​​obtained. Based on the second rotation matrix, the first translation vector, and the second translation vector, a final translation vector is obtained. The final rigid transformation matrix includes the final rotation matrix and the final translation vector.

[0097] Specifically, the singular value decomposition module is used to generate the final rigid transformation matrix to achieve the final registration of the source point cloud and the target point cloud, that is, to achieve "fine registration".

[0098] Since the overlap score processing module and the mismatch point removal module have removed points in non-overlapping regions and eliminated mismatch points, the point cloud to be processed, after filtering from the third source point cloud and the target point cloud according to a preset number of points, is considered a "reliable point" with high accuracy. The overlap score of each point in the point cloud to be processed is used as the weight of each point, and a processing matrix is ​​constructed using the point cloud to be processed and the weights of each point in it. The processing matrix is ​​then decomposed using Singular Value Decomposition (SVD) to obtain the second rotation matrix R2 and the second translation vector t2. The final rigid rotation matrix is ​​expressed as R = R2·R1, where R is the final rigid transformation matrix, R1 is the first rotation matrix, and R2 is the second rotation matrix. The final translation vector is expressed as t = R2·t1 + t2, where t is the final translation vector, t1 is the first translation vector, and t2 is the second translation vector.

[0099] like Figure 2 As shown, in this invention, the source point cloud and target point cloud to be registered are first input into the trained point cloud registration model. After processing by the initialization module, the first rotation matrix is ​​generated. Then, after processing by the overlap score module, the overlap score is generated. After processing by the mismatch point removal module, the mismatch points are removed. Finally, after processing by the singular value decomposition module, the final rigid transformation matrix is ​​generated.

[0100] This invention designs a registration approach combining "coarse registration" and "fine registration." A simple neural network is used for coarse registration, reducing the distance between two point clouds with significantly different spatial coordinates. Then, a "fine registration" network using "reliable points" is used to obtain an accurate rigid transformation matrix. This invention improves the accuracy of point cloud registration, achieving centimeter-level or even smaller positioning errors in industrial applications. It solves the problem of partial-to-partial point cloud registration and improves the robustness of point cloud registration to noise and missing point clouds. To implement the "fine registration" network, this invention designs a two-step method based on an attention mechanism to find "reliable points." In the first step, a transformer-based network is used to calculate the overlap score of two point clouds, removing "non-overlapping points." In the second step, a similarity matrix is ​​calculated using local features and a self-attention network to further remove points with "incorrect correspondences." These two steps ultimately yield "reliable points" for registration. Based on the overlap score, a weighted singular value decomposition (SVD) method is used to calculate the rigid transformation between the point clouds.

[0101] In one implementation, the training steps of the point cloud registration model include:

[0102] Construct a training dataset, which includes a training source point cloud and a training target point cloud;

[0103] An initial point cloud registration model is constructed, comprising an initialization training module, a fine registration training module, and a singular value transform training module. The initialization training module generates intermediate training source point clouds to complete coarse registration between the training source point clouds and the training target point clouds. The fine registration training module includes a transformer-based overlap score training module and a self-attention-based mismatch point removal training module. The overlap score training module calculates the training overlap score for each point in the intermediate training source point cloud and the training target point cloud, and removes points in non-overlapping regions between the two points based on the training overlap score. The mismatch point removal training module identifies and removes mismatched points from the point cloud output by the overlap score training module. The singular value transform training module calculates the point cloud output by the mismatch point removal training module to generate the final rigid transformation training matrix between the training source point cloud and the training target point cloud.

[0104] The initial point cloud registration model is trained multiple times using the training dataset until the loss function reaches the preset target, thus obtaining the trained point cloud registration model.

[0105] Specifically, a training dataset is first constructed, followed by an initial point cloud registration model. This initial point cloud registration model is then trained multiple times using the training dataset until the loss function reaches a preset target, resulting in a trained point cloud registration model. This invention utilizes a self-supervised model training approach. Through multiple training rounds, the model gradually learns and adjusts its parameters, improving accuracy, enhancing generalization ability, and reducing the risk of overfitting.

[0106] In one implementation, constructing the training dataset includes:

[0107] Obtain the target open-source dataset, and then perform cropping and sampling on the target open-source dataset in sequence to obtain the original training source point cloud and the original training target point cloud;

[0108] The original training source point cloud is randomly rotated and randomly translated along the x-axis, y-axis and z-axis to obtain the second training source point cloud;

[0109] Gaussian noise is added to the second training source point cloud and the original training target point cloud to obtain the third training source point cloud and the second training target point cloud.

[0110] The second training target point cloud is randomly selected according to a preset probability to obtain a third training target point cloud and a fourth training target point cloud that has not been selected. The third training target point cloud is flipped to obtain a fifth training target point cloud. The training target point cloud includes the fourth training target point cloud and the fifth training target point cloud.

[0111] The fourth training source point cloud corresponding to the fourth training target point cloud is obtained from the third training source point cloud, and the fifth training target point cloud is randomly rotated and randomly translated along the x-axis, y-axis and z-axis to obtain the fifth training source point cloud. The training source point cloud includes the fourth training source point cloud and the fifth training source point cloud.

[0112] Specifically, when constructing training data, this invention first performs data preprocessing and then data augmentation to enrich the training data and reduce the risk of overfitting. This invention proposes a training method for a dual-target point cloud registration model. During model training, a point cloud obtained by rotating and transforming the source point cloud is added as a "virtual target" in the training process. This virtual target has the same pose as the target point cloud and its spatial location is near the source point cloud. Through this virtual target, the loss function can be changed during training, improving the registration accuracy of the initialization module. This, in turn, accelerates the training process and increases the accuracy of model registration.

[0113] In the data preprocessing stage, the target open-source dataset is obtained, and it is then cropped and sampled sequentially to obtain the original training source point cloud and the original training target point cloud. The target open-source dataset is the ModelNet40 dataset. The original training source point cloud is then randomly rotated and translated along the x-axis, y-axis, and z-axis to obtain the second training source point cloud. At this point, the data preprocessing process is complete.

[0114] To avoid overfitting due to overly simplistic training data, the training data is augmented. Gaussian noise is added to the second training source point cloud and the original training target point cloud to obtain a third training source point cloud and a second training target point cloud. The second training target point cloud is randomly selected according to a preset probability to obtain a third training target point cloud and a fourth training target point cloud that was not selected. The third training target point cloud is flipped to obtain a fifth training target point cloud, which includes the fourth and fifth training target point clouds. A fourth training source point cloud corresponding to the fourth training target point cloud is obtained from the third training source point cloud. The fifth training target point cloud is then randomly rotated and translated along the x, y, and z axes to obtain a fifth training source point cloud, which includes the fourth and fifth training source point clouds.

[0115] At this point, the construction of the training dataset is complete. The training source point clouds in the training dataset now include the fourth and fifth training source point clouds, and the training target point clouds include the fourth and fifth training target point clouds. This invention uses this method to augment data, effectively addressing the "mirror registration" situation. The preset probability can be adjusted according to actual conditions.

[0116] In one implementation, the preset probability is 50%.

[0117] like Figure 1 As shown, the point cloud registration method further includes the following steps:

[0118] Step S300: Transform the source point cloud using the final rigid transformation matrix to obtain the final source point cloud, thereby completing the final registration between the source point cloud and the target point cloud.

[0119] Specifically, this invention generates the final rigid transformation matrix through a trained point cloud pairing model. Since the non-overlapping regions of the point cloud to be registered are cleared and mismatched points are removed in the point cloud registration model, the influence of overlapping regions and mismatched points on registration is reduced, thus improving the accuracy of the final rigid transformation matrix and effectively enhancing the accuracy of point cloud registration.

[0120] In one implementation, in a noisy partial point cloud registration scenario with a given random rotation angle of (0, 45) degrees and a random translation unit of (-0.5, 0.5), for registration of categories not present in the training set, the angle error of the final source point cloud calculated by this invention is 0.80 degrees, and the translation error is 0.0091. For categories present in the training set, the angle error of the final source point cloud calculated by this invention is 0.39 degrees, and the translation error reaches 0.0042. Both are within acceptable levels.

[0121] In one implementation, such as in practical industrial applications where high real-time performance is required, the point cloud registration model only includes a first initialization module, a first mismatch point removal module, and a first final rigid matrix generation module. The first initialization module completes the initial registration of the source point cloud and the target point cloud; the mismatch point removal module identifies and removes mismatch points from the point cloud output by the first initialization module; and the singular value decomposition module calculates the point cloud output by the mismatch point removal module to generate the final rigid transformation matrix between the source and target point clouds. In this scenario, for registration of categories not present in the training set, the angular error is 3.21 degrees and the translation error is 0.036, which is an acceptable level of error.

[0122] In one embodiment, such as Figure 3As shown, based on the above point cloud registration method, the present invention also provides a point cloud registration device, comprising:

[0123] The acquisition module 100 is used to acquire the source point cloud and the target point cloud to be registered;

[0124] The point cloud processing module 200 is used to input the source point cloud and the target point cloud into a trained point cloud registration model, and then process them sequentially through the initialization module, the transformer-based overlap score module, the self-attention mechanism-based mismatch point removal module, and the singular value decomposition module in the trained point cloud registration model to obtain the final rigid transformation matrix between the source point cloud and the target point cloud.

[0125] The initialization module is used to generate an intermediate source point cloud to complete the initial registration of the source point cloud and the target point cloud. The overlap score module based on transformer is used to calculate the overlap score of each point in the intermediate source point cloud and the target point cloud, and remove points in the non-overlapping region between the intermediate source point cloud and the target point cloud based on the overlap score. The mismatch point removal module based on self-attention mechanism is used to identify and remove mismatch points in the point cloud output by the overlap score module. The singular value decomposition module is used to calculate the point cloud output by the mismatch point removal module to generate the final rigid transformation matrix between the source point cloud and the target point cloud.

[0126] The final registration module 300 is used to transform the source point cloud using the final rigid transformation matrix to obtain the final source point cloud, so as to complete the final registration between the source point cloud and the target point cloud.

[0127] In one embodiment, the present invention also provides a terminal, such as... Figure 4 As shown, it includes: a memory 20, a processor 10, and a point cloud calibration program 30 stored in the memory 20 and executable on the processor 10. When the point cloud calibration program 30 is executed by the processor 10, it implements the steps of the point cloud calibration method as described above.

[0128] The present invention also provides a computer-readable storage medium storing a computer program that can be executed to implement the steps of the point cloud registration method described above.

[0129] In summary, this invention discloses a point cloud registration method, apparatus, terminal, and computer-readable storage medium. The method includes: acquiring a source point cloud and a target point cloud to be registered; inputting the source point cloud and the target point cloud into a trained point cloud registration model, and sequentially processing them through an initialization module, a transformer-based overlap score module, a self-attention mechanism-based mismatch point removal module, and a singular value decomposition module in the trained point cloud registration model to obtain a final rigid transformation matrix between the source point cloud and the target point cloud; and using the final rigid transformation matrix to transform the source point cloud to obtain a final source point cloud, thereby completing the final registration between the source point cloud and the target point cloud. This invention reduces the impact of overlapping regions and mismatch points on registration by clearing non-overlapping regions and removing mismatch points from the point cloud to be registered in the point cloud registration model, effectively improving the accuracy of point cloud registration.

[0130] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A point cloud registration method, characterized in that, The method includes: Obtain the source point cloud and target point cloud to be registered; The source point cloud and the target point cloud are input into a trained point cloud registration model, and then processed sequentially through the initialization module, the transformer-based overlap score module, the self-attention mechanism-based mismatch point removal module, and the singular value decomposition module in the trained point cloud registration model to obtain the final rigid transformation matrix between the source point cloud and the target point cloud. The source point cloud is transformed using the final rigid transformation matrix to obtain the final source point cloud, thereby completing the final registration between the source point cloud and the target point cloud; The initialization module generates an intermediate source point cloud to complete the initial registration of the source point cloud and the target point cloud. The transformer-based overlap score module calculates the overlap score of each point in the intermediate source point cloud and the target point cloud, and removes points in non-overlapping regions between the intermediate source point cloud and the target point cloud based on the overlap score. The self-attention-based mismatch point removal module identifies and removes mismatch points in the point cloud output by the overlap score module. The singular value decomposition module calculates the point cloud output by the mismatch point removal module to generate the final rigid transformation matrix between the source point cloud and the target point cloud. The specific processing steps of the overlap fraction module based on transformer include: Feature extraction is performed on the intermediate source point cloud and the target point cloud according to the preset feature dimensions to obtain high-dimensional features of the intermediate source point cloud and high-dimensional features of the target point cloud, respectively. The high-dimensional features of the intermediate source point cloud and the high-dimensional features of the target point cloud are input into the transformer network of the overlap score module, and the first interaction feature and the second interaction feature between the intermediate source point cloud and the target point cloud are obtained based on the self-attention mechanism. Max pooling is used to perform max pooling calculation on the high-dimensional features of the intermediate source point cloud to obtain the global features of the intermediate source point cloud. The high-dimensional features of the intermediate source point cloud, the global features of the intermediate source point cloud, the global features of the target point cloud, and the first interaction feature are processed to obtain the overlap score of each point in the intermediate source point cloud; and the high-dimensional features of the target point cloud, the global features of the target point cloud, and the second interaction feature are processed to obtain the overlap score of each point in the target point cloud. The overlap score of each point in the intermediate source point cloud and the target point cloud is compared with a preset score threshold. Points with scores below the preset score threshold are removed as points in the non-overlapping region between the intermediate source point cloud and the target point cloud to obtain the second source point cloud.

2. The point cloud registration method according to claim 1, characterized in that, The specific processing steps of the initialization module include: Feature extraction is performed on the source point cloud and the target point cloud respectively to obtain global features of the source point cloud and global features of the target point cloud; The global features of the source point cloud and the global features of the target point cloud are processed to obtain a first rotation matrix and a first translation vector. The source point cloud is transformed using the first rotation matrix and the first translation vector to generate an intermediate source point cloud, thereby completing the initial registration of the source point cloud and the target point cloud.

3. The point cloud registration method according to claim 2, characterized in that, The mismatch point removal module includes a dynamic graph convolutional neural network module; the specific processing steps of the mismatch point removal module based on the self-attention mechanism include: The dynamic graph convolutional neural network module is used to process the second source point cloud and the target point cloud to obtain local features of the second source point cloud and local features of the target point cloud. The local features of the second source point cloud and the local features of the target point cloud are processed to obtain a similarity matrix between the second source point cloud and the target point cloud. Each element in the similarity matrix represents the matching relationship between each pair of points in the second source point cloud and the target point cloud. Each element in the similarity matrix is ​​compared with a preset threshold, and the points corresponding to elements that are lower than the preset threshold are taken as mismatch points. Remove the mismatched points to obtain a third source point cloud, where any point in the third source point cloud has one and only one matching point in the target point cloud.

4. The point cloud registration method according to claim 3, characterized in that, The specific processing steps of the singular value decomposition module include: The point cloud to be processed is obtained by filtering the third source point cloud and the target point cloud according to a preset number of points; The overlap score of each point in the point cloud to be processed is used as the weight of each point, and the point cloud to be processed and the weight of each point in the point cloud to be processed are used to construct the processing matrix; The matrix to be processed is subjected to singular value decomposition to obtain the second rotation matrix and the second translation vector. Based on the first rotation matrix and the second rotation matrix, a final rotation matrix is ​​obtained. Based on the second rotation matrix, the first translation vector, and the second translation vector, a final translation vector is obtained. The final rigid transformation matrix includes the final rotation matrix and the final translation vector.

5. The point cloud registration method according to claim 1, characterized in that, The training steps for the point cloud registration model include: Construct a training dataset, which includes a training source point cloud and a training target point cloud; An initial point cloud registration model is constructed, comprising an initialization training module, a fine registration training module, and a singular value transform training module. The initialization training module generates intermediate training source point clouds to complete coarse registration between the training source point clouds and the training target point clouds. The fine registration training module includes a transformer-based overlap score training module and a self-attention-based mismatch point removal training module. The overlap score training module calculates the training overlap score for each point in the intermediate training source point cloud and the training target point cloud, and removes points in non-overlapping regions between the two points based on the training overlap score. The mismatch point removal training module identifies and removes mismatched points from the point cloud output by the overlap score training module. The singular value transform training module performs calculations on the point cloud output by the mismatch point removal training module to generate the final rigid transformation training matrix between the training source point cloud and the training target point cloud. The initial point cloud registration model is trained multiple times using the training dataset until the loss function reaches the preset target, thus obtaining the trained point cloud registration model.

6. The point cloud registration method according to claim 5, characterized in that, The construction of the training dataset includes: Obtain the target open-source dataset, and then perform cropping and sampling on the target open-source dataset in sequence to obtain the original training source point cloud and the original training target point cloud; The original training source point cloud is randomly rotated and randomly translated along the x-axis, y-axis and z-axis to obtain the second training source point cloud; Gaussian noise is added to the second training source point cloud and the original training target point cloud to obtain the third training source point cloud and the second training target point cloud. The second training target point cloud is randomly selected according to a preset probability to obtain a third training target point cloud and a fourth training target point cloud that has not been selected. The third training target point cloud is flipped to obtain a fifth training target point cloud. The training target point cloud includes the fourth training target point cloud and the fifth training target point cloud. The fourth training source point cloud corresponding to the fourth training target point cloud is obtained from the third training source point cloud, and the fifth training target point cloud is randomly rotated and randomly translated along the x-axis, y-axis and z-axis to obtain the fifth training source point cloud. The training source point cloud includes the fourth training source point cloud and the fifth training source point cloud.

7. A point cloud registration device, characterized in that, include: The acquisition module is used to acquire the source point cloud and target point cloud to be registered; The point cloud processing module is used to input the source point cloud and the target point cloud into a trained point cloud registration model, and then process them sequentially through the initialization module, the transformer-based overlap score module, the self-attention mechanism-based mismatch point removal module, and the singular value decomposition module in the trained point cloud registration model to obtain the final rigid transformation matrix between the source point cloud and the target point cloud. The initialization module is used to generate an intermediate source point cloud to complete the initial registration of the source point cloud and the target point cloud. The overlap score module based on transformer is used to calculate the overlap score of each point in the intermediate source point cloud and the target point cloud, and remove points in the non-overlapping region between the intermediate source point cloud and the target point cloud based on the overlap score. The mismatch point removal module based on self-attention mechanism is used to identify and remove mismatch points in the point cloud output by the overlap score module. The singular value decomposition module is used to calculate the point cloud output by the mismatch point removal module to generate the final rigid transformation matrix between the source point cloud and the target point cloud. The specific processing steps of the overlap fraction module based on transformer include: Feature extraction is performed on the intermediate source point cloud and the target point cloud according to the preset feature dimensions to obtain high-dimensional features of the intermediate source point cloud and high-dimensional features of the target point cloud, respectively. The high-dimensional features of the intermediate source point cloud and the high-dimensional features of the target point cloud are input into the transformer network of the overlap score module, and the first interaction feature and the second interaction feature between the intermediate source point cloud and the target point cloud are obtained based on the self-attention mechanism. Max pooling is used to perform max pooling calculation on the high-dimensional features of the intermediate source point cloud to obtain the global features of the intermediate source point cloud. The high-dimensional features of the intermediate source point cloud, the global features of the intermediate source point cloud, the global features of the target point cloud, and the first interaction feature are processed to obtain the overlap score of each point in the intermediate source point cloud; and the high-dimensional features of the target point cloud, the global features of the target point cloud, and the second interaction feature are processed to obtain the overlap score of each point in the target point cloud. The overlap score of each point in the intermediate source point cloud and the target point cloud is compared with a preset score threshold. Points with scores below the preset score threshold are removed as points in the non-overlapping region between the intermediate source point cloud and the target point cloud to obtain the second source point cloud. The final registration module is used to transform the source point cloud using the final rigid transformation matrix to obtain the final source point cloud, thereby completing the final registration between the source point cloud and the target point cloud.

8. A terminal, characterized in that, include: The device includes a memory, a processor, and a point cloud registration program stored in the memory and executable on the processor, wherein the point cloud registration program, when executed by the processor, implements the steps of the point cloud registration method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that can be executed to implement the steps of the point cloud registration method as described in any one of claims 1 to 6.

Images