A noise-resistant bidirectional attention point cloud registration method

Abstract

This invention relates to the field of 3D graphics computing technology, specifically to a noise-resistant bidirectional attention point cloud registration method. First, an RPMNet++ network is constructed based on the RPM-Net framework. The Copula denoising module performs feature correlation analysis on the point cloud data, removing negatively correlated noise points, outliers, and anomalies, while retaining positively correlated inliers. Next, a local bidirectional attention mechanism module enhances the sampling point features by aggregating neighborhood information, improving the network model's ability to learn local neighborhood context information, thus fully learning neighborhood correlation features from noisy and unevenly dense point cloud data. Finally, in the correspondence estimation module, the Sinkhorn normalized iterative algorithm is used to iterate the matching points. The corresponding points are then fed into the rotation matrix estimation module for singular value decomposition, calculating a new rotation transformation matrix. This rotation matrix is ​​introduced in the next iteration until the network converges, outputting the final registration result.

Description

Technical Field

[0001] This invention relates to the field of 3D graphics computing technology, specifically to a noise-resistant bidirectional attention point cloud registration method. Background Technology

[0002] Point cloud registration refers to incorporating different point clouds undergoing rigid body transformations (rotation, translation) into a unified coordinate framework to achieve a more accurate and complete description of the same target or scene. It is widely used in numerous industries and fields such as robot navigation, 3D reconstruction, and optical measurement. Based on the precise geometric features with corresponding relationships in the point clouds to be registered, traditional point cloud registration techniques can directly estimate or iteratively optimize rigid body transformation parameters. However, due to the unstructured and disordered nature of point cloud data, as well as significant differences in acquisition mechanisms, noise interference, spatial distribution, and density among different point clouds, automatic high-precision point cloud registration remains a highly challenging task.

[0003] Benefiting from the successful application of Convolutional Neural Networks (CNNs) in numerous image vision tasks and the rapid development of deep learning technology for point cloud processing, designing point cloud registration networks that can flexibly handle various scenarios (targets) and non-ideal sample conditions has become an important research direction. Based on the simulation methods of the local functions or overall process of traditional point cloud registration, existing point cloud registration networks can be divided into two categories: feature learning frameworks and parameter learning frameworks. Point cloud registration networks based on feature learning frameworks utilize the advantages of CNN deep learning to capture effective features (points) from the point cloud, achieving robust search of correspondences, replacing the same-name feature extraction module in traditional point cloud registration calculations, and improving its performance. Compared with the traditional point cloud registration process, the feature learning framework is beneficial for global context, diversified feature learning, and simplifies feature descriptor design, but it relies on high-quality sample label data for training, which is difficult to guarantee in practical applications. Point cloud registration networks under the parameter learning framework, through network structure and loss function design, realize the entire process of feature learning of the point cloud to be registered, correspondence search, and transformation parameter optimization estimation, outputting transformation parameters (or matrices) end-to-end. Compared with feature learning frameworks, point cloud registration networks under end-to-end parameter learning frameworks combine the advantages of traditional registration optimization strategies with deep learning of point cloud features. While ensuring registration accuracy, they reduce the network's requirement for the completeness of training samples and improve the network's convergence speed, making them more promising. However, they still have shortcomings in the registration of point cloud targets with many locally similar structures and rely on high-quality sample label data for training.

[0004] In existing point cloud feature extraction networks, a neighborhood system is often constructed using the target point as the sampling center to achieve convolution of irregular point cloud data. However, the weight-sharing characteristic of convolution operations sacrifices the accuracy of feature extraction in local noisy regions to ensure the effectiveness of global feature extraction. Therefore, when noisy points are distributed throughout the space, the network model struggles to effectively extract point cloud features, exacerbating the model's bias in describing matching features. Summary of the Invention

[0005] The purpose of this invention is to provide a noise-resistant bidirectional attention point cloud registration method, which aims to solve the technical problems that existing point cloud registration network models are susceptible to noise interference and have limited ability to learn local fine-grained features.

[0006] To achieve the above objectives, the present invention provides a noise-resistant bidirectional attention point cloud registration method, comprising the following steps:

[0007] Step 1: Construct the RPMNet++ network based on the RPM-Net framework. The RPMNet++ network includes a Copula denoising module, a local bidirectional attention module, an annealing parameter prediction module, a correspondence estimation module, and a rotation matrix estimation module.

[0008] Step 2: Use the initial or current rotation matrix to perform a rigid transformation on the origin set X to obtain the rotated point set, and use the RPM-Net feature extraction module to extract the mixed features of the corresponding data respectively;

[0009] Step 3: In the Copula denoising module, based on the Kendall correlation coefficient... The Clayton Copula model is used to filter interior points and store them as a new set of origin points and a new set of target points.

[0010] Step 4: In the local bidirectional attention module, use bidirectional attention mechanisms from sampling point to neighboring point and from neighboring point to sampling point;

[0011] Step 5: Estimate annealing parameters using the annealing parameter prediction module;

[0012] Step 6: In the correspondence estimation module, the Sinkhorn normalization iterative algorithm is used to estimate the corresponding points in the two point clouds;

[0013] Step 7: The corresponding points are fed into the rotation matrix estimation module for singular value decomposition, a new rotation transformation matrix is ​​calculated, and this rotation matrix is ​​introduced in the new round of iteration until the RPMNet++ network converges and the final registration result is output.

[0014] Optionally, the distribution function of the Clayton Copula model in step 3 is:

[0015]

[0016] in, and These represent data from point cloud datasets. The mixed features of the local neighborhood center point and neighboring points obtained by sampling, where λ is a parameter of the Clayton Copula model, 0 < λ < ∞, when hour, All points tend to be independent; when hour, The higher the correlation between all points, the better.

[0017] Optionally, for the point cloud U={fi|i=1,...,N} obtained by random sampling, where N is the number of point clouds, RPM-Net uses the nearest neighbor sampling algorithm to obtain the K nearest neighbors of the sampled points, and then performs feature encoding according to the following formula:

[0018]

[0019] In the formula, F ui It includes the sampling center point u i The hybrid characteristics of spatial coordinates and neighborhood topology; f θ It is based on the PointNet encoding calculation formula, and its function is to aggregate the information of all points in the neighborhood into a single feature descriptor, u i The spatial coordinates of the center point, △u i ,k is the relative displacement between the neighboring points and the center point; PPF(u i , u k The center point u is described in a rotation-invariant manner. i and its neighboring points u k The surface between, where ni and nk are the normal vectors of the center point and the neighborhood points, respectively, and ∠(·) is the surface that guarantees the range of [0, ]. The formula for calculating angles within a certain range is given; ||·|| is the formula for calculating Euclidean distance.

[0020] RPM-Net will center point u i and each of its neighboring points u k The features are concatenated into a 10-D feature vector, which is then input into the MLP layer and the max pooling layer to generate a global feature vector with a larger receptive field. Finally, the global feature vector is mapped onto the multilayer perceptron and processed. 2. Normalization yields a single mixed feature vector F ui ;

[0021] The hybrid features F extracted based on the RPM-Net feature extraction module ui Calculate the Kendall correlation coefficient. The calculation formula is:

[0022]

[0023] in, , && indicates AND logic, || indicates OR logic. This indicates the number of pairs of points that are positively correlated, and the method for determining this is as follows: , , Each element in the list is greater than , , ; This indicates the number of pairs of points that are negatively correlated, determined by the following method: , , There exists any one element less than , , ; Indicates from Combinations of selecting 2 points from 10 points without considering sorting; , , These represent the autocorrelation of the sampling point, neighboring points, neighborhood structure information, and relative displacement vector, respectively.

[0024] Optionally, in step 3, the Copula denoising module uses Kendall's rank to measure the positive and negative correlation between the features of each point cloud, namely point coordinate features, neighborhood structure features and relative displacement vector features. It filters out negatively correlated point clouds, specifically outlier points and outliers, through iterative loops, to ensure that the point clouds in the local neighborhood are all internal points to the greatest extent possible.

[0025] Optionally, in step 4, the local bidirectional attention module defines the feature encoding process as the following two steps:

[0026] Step 4.1: Bidirectional attention feature encoding of sampling centers. Through sampling point self-reinforcement, the interaction force of sampling points to neighboring points and the influence factor of neighboring points on sampling points, the reinforcement learning of the neighborhood spatial relationship of sampling center points is realized.

[0027] Step 4.2: Neighborhood spatial structure feature encoding, which realizes local feature encoding through the spatial Euclidean distance between the sampling point and the neighboring points, the direction vector and the normal vector information.

[0028] Optionally, the process of obtaining the annealing parameters of the point cloud dataset first requires feeding the features extracted by the local bidirectional attention module into a fully connected network to estimate the parameter values ​​from the feature information; secondly, the estimated annealing parameters α and β are fed into the SoftPlus activation function for positive conversion.

[0029] Optionally, the two point cloud features and annealing parameters extracted and estimated in steps 4 and 5 are used to solve for the optimal distance on the double random constraint matrix using the Sinkhorn algorithm. The initial state of this matrix is ​​shown below:

[0030] In the formula F xj and F yk These represent the characteristics of two point clouds respectively; since each point x in point cloud X... j There is not always a corresponding point y in the point cloud Y. k We need to add a row and a column to the j-th row and k-th column of matrix M respectively to satisfy the iteration conditions of the double random constraint matrix; for each point x in the point cloud X j Construct the corresponding point y. j The formula for its solution is shown below:

[0031] To obtain (x) with a one-to-one correspondence j y j Then, the rotation matrix R and translation matrix t can be solved by singular value decomposition.

[0032] This invention provides a noise-resistant bidirectional attention point cloud registration method. First, an RPMNet++ network is constructed based on the RPM-Net framework. The Copula denoising module performs feature correlation analysis on the point cloud data, removing noisy points, outliers, and negatively correlated features of abnormal values, while retaining positively correlated inliers. Then, a local bidirectional attention mechanism module enhances the sampling point features by aggregating neighborhood information, thereby improving the network model's ability to learn local neighborhood context information and fully learning neighborhood correlation features from noisy and unevenly dense point cloud data. Finally, in the correspondence estimation module, the Sinkhorn normalized iterative algorithm is used to iterate the matching points. The corresponding points are fed into the rotation matrix estimation module for singular value decomposition to calculate a new rotation transformation matrix, which is then introduced into the new round of iterations until the network converges, outputting the final registration result. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a schematic diagram of the structure of the RPMNet++ network of the present invention.

[0035] Figure 2 This is a schematic diagram illustrating the specific process of a noise-resistant bidirectional attention point cloud registration method according to the present invention.

[0036] Figure 3 This is a schematic diagram of the Copula noise reduction module of the present invention.

[0037] Figure 4 This is a schematic diagram of the local bidirectional attention module structure of the present invention.

[0038] Figure 5 This is a schematic diagram comparing the noise-free point cloud registration effect in a specific embodiment of the present invention.

[0039] Figure 6 This is a schematic diagram comparing the registration effects of point clouds with different densities of Gaussian noise according to a specific embodiment of the present invention.

[0040] Figure 7 This is a schematic diagram comparing the registration effects of Gaussian noise point clouds with structural defects in a specific embodiment of the present invention.

[0041] Figure 8 This is a schematic diagram comparing the noise retention ratio of three sets of data before and after adding the Copula noise reduction module, according to a specific embodiment of the present invention.

[0042] Figure 9 This is a schematic diagram comparing the visualization effects of denoising models under different Gaussian noise variances according to a specific embodiment of the present invention. Detailed Implementation

[0043] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0044] The following are explanations of some of the English terms used in the text:

[0045] RPM-Net: Robust point matching network;

[0046] Farthest point sampling (FPS): Farthest point sampling algorithm;

[0047] Multilayer perceptron (MLP): Multilayer perceptron;

[0048] Singular value decomposition (SVD)

[0049] This invention provides a noise-resistant bidirectional attention point cloud registration method, comprising the following steps:

[0050] Step 1: Construct the RPMNet++ network based on the RPM-Net framework. The RPMNet++ network includes a Copula denoising module, a local bidirectional attention module, an annealing parameter prediction module, a correspondence estimation module, and a rotation matrix estimation module.

[0051] Step 2: Use the initial or current rotation matrix to perform a rigid transformation on the origin set X to obtain the rotated point set, and use the RPM-Net feature extraction module to extract the mixed features of the corresponding data respectively;

[0052] Step 3: In the Copula denoising module, based on the Kendall correlation coefficient... The Clayton Copula model is used to filter interior points and store them as a new set of origin points and a new set of target points.

[0053] Step 4: In the local bidirectional attention module, use bidirectional attention mechanisms from sampling point to neighboring point and from neighboring point to sampling point;

[0054] Step 5: Estimate annealing parameters using the annealing parameter prediction module;

[0055] Step 6: In the correspondence estimation module, the Sinkhorn normalization iterative algorithm is used to estimate the corresponding points in the two point clouds;

[0056] Step 7: The corresponding points are fed into the rotation matrix estimation module for singular value decomposition, a new rotation transformation matrix is ​​calculated, and this rotation matrix is ​​introduced in the new round of iteration until the RPMNet++ network converges and the final registration result is output.

[0057] The structure of the RPMNet++ network is as follows: Figure 1 As shown, the flowchart of the noise-resistant bidirectional attention point cloud registration method of the present invention is as follows: Figure 2As shown, firstly, an RPMNet++ network is built based on the RPM-Net framework. The Copula denoising module performs feature correlation analysis on the point cloud data, removing noisy points, outliers, and negatively correlated features of abnormal values, while retaining positively correlated inliers. Then, the local bidirectional attention mechanism module enhances the sampling point features by aggregating neighborhood information, while improving the network model's ability to learn local neighborhood context information, fully learning neighborhood correlation features from noisy and unevenly dense point cloud data. Finally, in the correspondence estimation module, the Sinkhorn normalized iterative algorithm is used to iterate the matching points. The corresponding points are fed into the rotation matrix estimation module for singular value decomposition, calculating a new rotation transformation matrix, and introducing this rotation matrix in the new round of iteration until the network converges, outputting the final registration result.

[0058] The following section provides further explanation based on the principles of the main components in the execution process:

[0059] 1. Explanation of the Copula noise reduction module principle:

[0060] From a mathematical perspective, the correlation between noise points and inliers can be quantified using deterministic metrics. The Clayton Copula model is employed to quantify the correlation among multiple points, and noise points are removed based on the positive and negative correlations of data points within the K-neighborhood. The distribution function of the Clayton Copula model is defined as follows:

[0061] (1)

[0062] in, and These represent data from point cloud datasets. The mixed features of the local neighborhood center point and neighboring points obtained from mid-sampling. These are the parameters of the Clayton Copula model. When... hour, All points tend to be independent; when hour, The higher the correlation between all points, the better. The value of not only determines the range distribution of the Clayton Copula model, but also defines the correlation between points in the local neighborhood. The parameters can be obtained from Kendall's correlation coefficient. The Kendall correlation coefficient is calculated based on the spatial location of the sampling center point and neighboring points, neighborhood structure information, and relative displacement vectors. The formula is

[0063] .

[0064] in,

[0065] (2)

[0066] && indicates AND logic, || indicates OR logic. This indicates the number of pairs of points that are positively correlated, and the method for determining this is as follows: , , Each element in the list is greater than , , ; This indicates the number of pairs of points that are negatively correlated, determined by the following method: , , There exists any one element less than , , ; Indicates from The combination of selecting two points from a given set of points, without considering the order of the points, is given in equation (2). , , The autocorrelation of the sampling point, neighborhood point, neighborhood structure information, and relative displacement vector are respectively defined as follows:

[0067] (3)

[0068] in, , These represent the spatial location information of the sampling point and its neighboring points, respectively. Neighborhood structure information used to describe point-pair features This represents the spatial displacement vector from the sampling point to neighboring points. according to and The changes describe the trend of change between local neighborhood points: when hour, This indicates that the relationships between neighboring points are nearly identical; when hour, This indicates that the relationships between neighboring points are nearly completely contradictory; when hour, This indicates that the relationship between neighboring points cannot be determined. From equations (1) and (2), we can see that... The range of is (0, ∞). The domain is Furthermore, define and The relationship function is

[0069] (4)

[0070] The result obtained from equation (4) Substituting into equation (1), the Copula denoising module proposed in this invention can be constructed, such as... Figure 3 As shown, this module fully utilizes the similarity of point cloud features in the K-neighborhood system to remove noisy points, thereby improving the network model's ability to learn point cloud data features. Figure 3 middle, , and These represent the original point cloud datasets, obtained by using the farthest point sampling (FPS) algorithm. The sub-region dataset obtained from the data, and the dataset obtained from the random sampling method. The set of sampling center points obtained by extracting without replacement is given. Much greater than and ,and The degree of visual noise interference is usually... Half or a quarter of the points. Centered on the target, the neighboring points are obtained using the K-nearest neighbor (KNN) algorithm. .extract and The local point cloud data features are composed of equation (3), and the inner product is calculated using equation (3). This inner product is then substituted into equation (2) to calculate the Kendall correlation coefficient. Calculate: (1) When At that time, the corresponding Points identified as noise are removed from the original point cloud data and randomly sampled. Perform on the remaining points In addition, the sampling process is also in the form of extraction without replacement, and the operation is consistent with the original operation after the point set is updated. (2) When When that time, substitute it into equation (4) to calculate the Clayton Copula model. Parameters, and at the same time Substituting into equation (1) quantifies the correlation of each neighboring point. Then, calculate the negative correlation coefficient of each neighboring point according to equation (2). The process involves several steps: first, identifying and retaining all positively correlated neighboring points, while removing all negatively correlated neighboring points as noise from the original point cloud data; then, using the KNN algorithm to sample the remaining points in the local space until a minimum of K neighboring points are identified; finally, local blending feature extraction and correlation assessment are performed on the recombined neighboring points until... ,Right now At that time, the output is generated by the sampling points. The local spatial structure is formed. When the inliers in the local space cannot meet the sampling requirements of K neighboring points, the inlier search is performed by doubling the neighborhood radius to ensure consistency of local feature dimensions. Clearly, calculating the correlation between point pairs and features can effectively identify noisy points, reducing the interference of noisy features to some extent and improving the network's overall ability for parameter estimation and feature learning.

[0071] Copula's denoising module is based on the assumption that point cloud data within a neighborhood have similar characteristics. It first... Perform random sampling to obtain And select K neighboring points Perform feature encoding; then construct the Clayton Copula model distribution function based on the feature information. And calculate Kendall's correlation coefficient. Finally utilize and Filter out noise points and iterate. The points in the middle, until the condition is met. The specific steps of the algorithm are as follows:

[0072] 1. For the target point cloud dataset Set sub-region dataset sampling point set Neighborhood point set Neighborhood radius r, number of neighborhood points K, Kendall correlation coefficient Clayton Copula distribution function and interior point array .

[0073] 2. Utilize FPS from Medium sampling Using random sampling algorithms and KNN from Obtain And substitute the sampled point set into formula (2) for... Value calculation. If If the value is less than or equal to 0, then from Reacquired in And the initial Abandon, until A value greater than zero will meet the condition. Save the point set.

[0074] 3. Use formula (1) to process the sample obtained in the second step. The correlation between neighboring points is calculated for the point set. If Then Save and output, otherwise save and output the neighboring area. Resampling until It should be noted that traversing all points in the neighborhood cannot satisfy this condition. Then the neighborhood radius r is doubled until all satisfy .

[0075] Copula's denoising module uses Kendall's rank to measure the positive and negative correlation between various point cloud features (point coordinate features, neighborhood structure features, and relative displacement vector features). It filters out negatively correlated point clouds (outliers and outliers) through iterative loops, ensuring that point clouds in the local neighborhood are all inliers to the greatest extent possible.

[0076] 2. Explanation of the principle of local bidirectional attention module:

[0077] Although the Copula denoising module can remove noise points with significant feature differences within the neighborhood of a sampling point, when the K-neighborhood contains multiple types of targets or the noise data does not differ significantly from other data in the neighborhood, the Copula denoising result will still contain a small amount of noise with similar features to the inlier data, affecting the local neighborhood feature representation. At the same time, Copula denoising will exacerbate the unevenness of point cloud density, especially in local areas where noise points are densely distributed, which will become sparse after denoising. This will, to some extent, disrupt the adjacency relationship of the original target point cloud data, affecting the representation of the target's local geometric structure features and the CNN's ability to learn neighborhood similarity features.

[0078] To overcome the influence of a small number of noise points and effectively extract data features from sparse point clouds, and further improve the correlation of data features within the neighborhood, this invention designs, as follows: Figure 4 The local bidirectional attention module shown consists of sampling center bidirectional attention feature encoding and neighborhood spatial structure feature encoding. Here, B represents the batch size; N represents the number of sampling points in each batch. This invention only considers the 3D spatial location information of K neighborhood points and N sampling points. The sampling center bidirectional attention feature encoding enhances the learning of the neighborhood spatial relationship of the sampling center point through sampling point self-reinforcement, the interaction force between sampling points and neighborhood points, and the influence factor of neighborhood points on sampling points. The neighborhood spatial structure feature encoding aims to improve the network model's ability to learn local fine-grained features of point cloud data by using the spatial Euclidean distance, direction vector, and normal vector information between sampling points and neighborhood points to achieve local feature encoding. First, the input features are encoded using the following formula:

[0079] (5)

[0080] in, Enhanced features by sampling points and local geometric features composition, ; , and They represent linear transformation operations ( (convolution operation); C represents the number of output channels, for features The feature similarity can be obtained by performing two linear transformations, transposing the result, and taking the inner product. This feature similarity is then calculated by dividing by... Adjustments are made to make the training process have a more stable gradient. Equation (5) assigns the self-attention weights calculated by the normalized exponential function softmax to the local spatial coordinate features after linear transformation, and then adds them to itself to construct the output result of the local bidirectional self-attention mechanism, which is represented by a feature vector and reflects the fine-grained geometric structure of the neighborhood of the sampling point. middle, The aim is to further aggregate the influence of each point in the neighborhood on the sampling point, and the aggregation calculation formula is as follows:

[0081] (6)

[0082] in, , , and These represent the spatial coordinates of the sampling point and its neighboring points, respectively, based on the PointNet feature encoding method. Generate sampling point features and neighborhood point features and substitute them into Enhance the features of the sampling points. Equation (6) consists of three parts: (1) (2) Indicates self-enhancing features at sampling points; First, calculate the influence factor of each neighborhood point feature on the sampled point feature, and then project them onto a multilayer perceptron (MLP); (3) This represents the relationship between the features of each neighboring point and the features of the sampling point. Multiplying and accumulating these features with (2) allows for the learning of enhanced features of the sampling points. Equation (6) can be analyzed from the perspective of the force field, that is, each [factor] in the local space [is related to the relationship between the features of each neighboring point and the features of the sampling point]. Both are correct. There is one force, gravity, trying to pull... Zoom in The repulsive force pushes the two apart; the strength of the force is determined by... The direction of the force is determined by The decision is made that the two learn adaptively based on the difference between the two feature vectors. Therefore, It fully integrates the relationships between each neighboring point and the sampling point, enabling a better description of neighborhood correlation characteristics. Based on the sampling points... and its neighboring points The Euclidean distance between them, the direction vector, the normal vector, and the spatial position of neighboring points. The formula for spatial encoding is:

[0083] (7)

[0084] in,

[0085]

[0086] It should be noted that in equation (7) , and These represent the Euclidean distance between the sampling point and its neighboring points, the relative displacement (direction) vector, and the normal vector, respectively. These three are concatenated to generate a 13-dimensional feature vector. .

[0087] This invention also demonstrates the registration experiment results through specific embodiments to verify feasibility:

[0088] For the purpose of evaluating the registration performance of homogeneous point cloud data under different noise conditions, three data types were divided: noise-free point cloud, Gaussian noise point cloud at different densities, and Gaussian noise point cloud with structural loss. In addition, an ablation experiment was designed to evaluate the role of the proposed network functional modules. ModelNet40 contains CAD models of 40 different object categories. In order to evaluate the network's ability to generalize to different object categories, 20 categories of data were randomly selected for training and evaluation, and all data of the remaining categories were used for testing, for a total of 5112 training models, 1202 validation models, and 1266 test models. To ensure the consistency of the number of points in each model during the experiment, the PointNet++ farthest point sampling algorithm was used to downsample each model and the number of sampling points was set to 2048. The three Euler angles of the rigid transformation matrix were randomly sampled in the range of [0, 45°] and shifted randomly in the range of [-0.5, 0.5] on each axis.

[0089] Noise-free point cloud experiment:

[0090] First, a source point cloud with precise point correspondences is obtained by randomly selecting 2048 points from each ModelNet40 model. and reference point cloud Then, the randomly generated rigid transformation matrix Acting on And randomly scrambled to obtain Table 1 presents the registration results of the method of this invention and the method in the literature under the above-mentioned noise-free point cloud data conditions. The registration evaluation includes isotropic and anisotropic indices and chamfer distance. Bold text indicates the optimal performance parameters, and italics indicate the suboptimal performance parameters. The comparison methods include not only deep learning-based point cloud registration networks PointNetLK, Deep Closest Point (DCP-v2), and RPM-Net, but also traditional point cloud registration algorithms ICP and FGR. Figure 5 The visualization effect of point cloud registration of the present invention and the method in the literature is given.

[0091] Table 1. Statistics on Noise-Free Point Cloud Registration Performance

[0092]

[0093] From Table 1, the registration accuracy and Figure 5 The registration visualization results show that the source point cloud under the ICP algorithm... (Blue) and reference point cloud (Red) Deviations occurred at different angles, such as Figure 5 In (a), the blue trees show a clear leftward tilt and excessive rightward rotation compared to the red trees. Figure 5 (b) The blue chair and Figure 5 The blue bar stool in (c) leans backward to some extent. This is because the ICP algorithm is sensitive to initial parameter values ​​and easily converges to local minima, resulting in the highest R1_err and T1_err values, reaching 6.407 and 0.0506 respectively. DCP-v2's advantage lies in its ability to aggregate global and local feature information for soft matching matrix calculation, but its insufficient ability to precisely optimize correspondences leads to... Figure 5 In (a), (b), and (c), the blue point clouds still exhibit leftward, forward, and backward tilting, respectively. Therefore, although there is a significant improvement compared to ICP (R1_err and T1_err are reduced by 2.415 and 0.0214, respectively), the registration accuracy ranks second to last in Table 1. Compared to the two methods mentioned above, PointNetLK achieves better registration results in all three target types, with R1_err and T1_err reaching good results of 0.847 and 0.0054, respectively. Figure 5 In (a), the top branches and leaves of the blue tree still exhibit offset. This is because PointNetLK does not fully consider local feature information when extracting the global feature descriptor for each point, causing it to converge to a local minimum when processing targets with similar local structures. Furthermore, comparing... Figure 5The registration results of RPM-Net and the proposed method show that RPM-Net is slightly inferior to the proposed method in tree registration. This is because the proposed method adds a local feature learning module with a bidirectional attention mechanism to RPM-Net. For models with weak global feature saliency and repetitive local features, this new module can effectively improve the network's ability to distinguish similar local structures. Meanwhile, the Copula denoising module added to the proposed method increases computation time because it performs noise verification in each iteration. For point clouds with a large number of precise point correspondences, FGR can obtain highly consistent model outer contour surface information for accurate estimation of transformation parameters, thus achieving the best registration results (R1_err and T1_err are 0.022 and 0.0002, respectively). The proposed method is generally close to this, but slightly inferior in terms of isotropic rotation error (only R1_err is 0.008 lower).

[0094] Experiments on point clouds with different Gaussian noise densities:

[0095] First, 2048 points were randomly sampled from the same model to obtain source point clouds that do not have point correspondence relationships. and reference point cloud Then use the random rigid transformation matrix Source point cloud A spatial transformation was performed, and 2048 Gaussian noise points were added to the two point clouds. The Gaussian noise model is as follows: The parameters are taken respectively and Point cloud data with added Gaussian noise were fed into the network for training and testing. The registration results of the methods in this invention and those in the literature are statistically shown in Table 2. Figure 6 The visualization effect of point cloud registration for the methods of this invention and the literature is also presented.

[0096] Table 2. Statistics on registration performance of point clouds with different Gaussian noise densities

[0097] Comparing Tables 1 and 2, it can be seen that because Gaussian noise point cloud data cannot guarantee the correspondence between points, DCP-v2's estimation of the soft assignment matrix using feature differences results in a large number of mismatched points, leading to the highest R1_err and T1_err values ​​of 8.922 and 0.0707, respectively. Figure 6 In (a), (b), and (c), the source point cloud and the reference point cloud have the largest angular offsets along each coordinate axis; although the R1_err and T1_err of ICP are reduced by 1.923 and 0.0193 respectively compared to DCP-v2, good registration is still not achieved. It is worth noting that... Figure 6 The registration performance of FGR in curtains, potted plants, and aircraft is far inferior to that in [other applications]. Figure 5 The registration performance was poor because noise affected the effective extraction of object surface information and the accurate estimation of the correspondence between point clouds by FGR, resulting in a significant drop in registration accuracy. R1_err and T1_err were 2.991 and 0.0252, respectively. Similarly, the PointNetLK feature encoding method has weak noise resistance. Figure 6 The registration results of PointNetLK showed a phenomenon where the source point cloud partially deviated from the reference point cloud. By employing rotation-invariant 4D point pair features and replacing spatial coordinate differences with feature differences to describe the local spatial structure, RPM-Net can obtain a robust estimate of the point cloud correspondence and outperforms methods in other literature in terms of R1_err (0.664) and T1_err (0.0062). Compared with the registration effect of RPM-Net, although the visual difference is not significant, the method of this invention has an accuracy improvement of nearly 50% in terms of isotropic and anisotropic indicators. This is because the network of this invention uses the Copula denoising module to remove noise points, outliers, and abnormal points while supplementing unjudged point clouds as inliers, so as to minimize the interference of noise points on the annealing parameter estimation and feature extraction process, thereby achieving a more accurate registration effect than RPM-Net, which fully demonstrates the effectiveness of the Copula denoising model.

[0098] Experiment on Gaussian noise point clouds with missing structures:

[0099] To verify the registration performance of the network in this invention on partially structurally missing data, 30% of the local regions of the model were randomly discarded during point sampling. Simultaneously, Gaussian noise was added to the sampled point cloud using the method described in the previous section before being fed into the network for training and testing. Table 3 presents the statistical results of the registration performance of the methods in this invention and the literature on Gaussian noise point clouds with structural missing data. Figure 7 This will provide the corresponding point cloud registration visualization effect.

[0100] Table 3. Statistics on registration performance of Gaussian noise point clouds with missing structures

[0101]

[0102] Comparing Tables 3 and 2, it can be seen that when the model has partial structural defects, the point cloud registration performance of all algorithms except the algorithm of this invention and RPM-Net decreases significantly, especially in terms of isotropic and anisotropic rotation errors. Specifically, FGR has the worst R1_err (30.839) and T1_err (0.192) in the relatively complex guitar ( Figure 7 a) and bookshelves ( Figure 7b) The large tilt and offset indicate that FGR is not only sensitive to noise, but also has difficulty in effectively handling structurally complex missing data; the PointNetLK feature extraction module also cannot effectively cope with complex environments or missing target structures, and obvious offsets appear in all three types of target registration tasks, with R1_err (12.607) and T1_err (0.169) ranking last; the missing target structure also exacerbates the mismatch situation of ICP and DCP-v2, with obvious offsets in the former in all three types of target registration tasks, while the latter only has a smaller offset in the computer registration task. Compared to the methods mentioned above, RPM-Net demonstrates superior performance in registration of all three target classes, with R1_err and T1_err values ​​of 1.712 and 0.018, respectively. This is because its 4D point-pair feature extraction method effectively describes the features of each local structure, and its Sinkhorn iterative method facilitates matching of partially identical data under conditions of structural incompleteness. In contrast, the feature information extracted by the local bidirectional attention module of this invention not only possesses the 4D point-pair feature information of RPM-Net but also includes interaction information between the center point and the sampling point, as well as between the sampling point and the center point. This allows for the acquisition of richer feature information than RPM-Net for corresponding point search, thus achieving the best registration accuracy. This indicates that the bidirectional attention module of this invention, through adaptive local feature learning, enhances the ability to learn fine-grained local features while facilitating accurate search of interior point correspondences.

[0103] Copula noise reduction module experiment:

[0104] To further verify the effectiveness of the Copula denoising model, three sets of Gaussian noise point cloud datasets were first generated from the Modelnet40 dataset, specifically as follows: ① The model was sampled with 2048 points and duplicated into two copies, with only one copy having Gaussian white noise (2048 noise points) added; ② The model was sampled with 2048 points and duplicated into two copies, with Gaussian white noise (2048 noise points) following the same distribution added to both copies; ③ The same model was randomly sampled twice with 2048 points, and Gaussian white noise (2048 noise points) with the same distribution added to each copy. The three datasets were then fed into the original RPM-Net and an RPM-Net with only the Copula denoising module added for training, respectively. Noise point search was then performed on the test dataset based on the matching matrix obtained during training. Figure 8 The statistical results of RPM-Net's denoising capabilities before and after adding the Copula denoising module are presented for three datasets. Figure 9 The second set of data is then given under different Gaussian variance parameters. The visualization of the denoising model is shown below, where Gaussian white noise follows a distribution. And parameters Take the value 0. The parameters are set to (a) 0.02, (b) 0.03, (c) 0.04, and (d) 0.05, respectively.

[0105] Depend on Figure 8 The noise residue ratio in the first set of data shows that the interior features with corresponding relationships are easy for the network to learn and distinguish, while randomly generated Gaussian white noise does not have a clear correspondence. Therefore, RPM-Net can remove most of the noise points without corresponding relationships through the Sinkhorn normalization iteration method. Figure 9 The Copula denoising module retains far fewer noise points than RPM-Net; therefore, the noise points retained by the Copula denoising module are magnified three times for easier observation. Comparative observation is then performed. Figure 8 Group 2 and Figure 9 It can be observed that as the noise level increases, the proportion of noise points remaining in both networks gradually increases. This is because a larger σ value results in a greater diffusion of Gaussian noise, leading to the extraction of more noise point features by the two algorithms, ultimately affecting the accurate estimation of interior point relationships. However, in Figure 9 In (b), (c), and (d), the Copula denoising module leaves far fewer noise points than RPM-Net. This indicates that as Gaussian noise increases, RPM-Net's ability to remove noise points decreases significantly, while the Copula denoising module can still effectively identify and remove noise points. This is because the Copula denoising module can pre-remove some noise points and outliers by comparing and analyzing point feature information. Then, the processed data is fed into the Sinkhorn normalization layer for further noise point removal and corresponding point estimation, ultimately resulting in more inliers and fewer noise-corresponding points than RPM-Net. On the other hand, when the two sets of point clouds do not have a one-to-one correspondence, the proportion of noise points increases significantly for both RPM-Net and the Copula denoising module. This is because randomly distributed noise points have spatial location information approximating inliers, leading to mismatches in both networks, such as matching noise points to inliers or matching noise points to other noise points as corresponding points. In summary, both the Copula denoising module and the Sinkhorn normalization iteration method built into RPM-Net can handle noisy points. However, the Copula denoising module uses the differences in feature information to distinguish between noisy points and inliers, making it more effective than RPM-Net in removing discrete noise points.

[0106] In summary, the point cloud registration experiments show that, on the ModelNet40 dataset, RPMNet++ significantly improves the reduction of isotropic average rotation and translation errors compared to RPM-Net, decreasing by (0.026, 0.001), (0.267, 0.0019), and (0.560, 0.007) on the noiseless dataset, noisy dataset, and partially noisy dataset, respectively. On the Stanford 3D dataset, RPMNet++ achieves the best results compared to seven cross-source registration algorithms, demonstrating good generalization performance and application value.

[0107] The above description discloses only one preferred embodiment of the present invention, and should not be construed as limiting the scope of the present invention. Those skilled in the art will understand that all or part of the processes of the above embodiments can be implemented, and equivalent changes made in accordance with the claims of the present invention are still within the scope of the invention.

Claims

1. A noise-resistant bidirectional attention point cloud registration method, characterized in that, Includes the following steps: Step 1: Construct the RPMNet++ network based on the RPM-Net framework. The RPMNet++ network includes a Copula denoising module, a local bidirectional attention module, an annealing parameter prediction module, a correspondence estimation module, and a rotation matrix estimation module. Step 2: Use the initial or network-feedback rotation matrix to perform a rigid transformation on the origin set X to obtain the rotated point set, and extract the mixed features of the corresponding data through the RPM-Net feature extraction module respectively; Step 3: In the Copula denoising module, based on the Kendall correlation coefficient... The Clayton Copula model is used to filter interior points and store them as a new set of origin points and a new set of target points. Step 4: In the local bidirectional attention module, use bidirectional attention mechanisms from sampling point to neighboring point and from neighboring point to sampling point; Step 5: Estimate annealing parameters using the annealing parameter prediction module; Step 6: In the correspondence estimation module, the Sinkhorn normalization iterative algorithm is used to estimate the corresponding points in the two point clouds; Subsequently, the corresponding points are fed into the rotation matrix estimation module for singular value decomposition, a new rotation transformation matrix is ​​calculated, and this rotation matrix is ​​introduced in the new round of iterations until the RPMNet++ network converges and the final registration result is output.

2. The noise-resistant bidirectional attention point cloud registration method as described in claim 1, characterized in that, The distribution function of the Clayton Copula model in step 3 is: ； in, and These represent data from point cloud datasets. The mixed features of the local neighborhood center point and neighboring points obtained by sampling, where λ is a parameter of the Clayton Copula model, 0 < λ < ∞, when hour, All points tend to be independent; when hour, The higher the correlation between all points, the better.

3. The noise-resistant bidirectional attention point cloud registration method as described in claim 2, characterized in that, For a point cloud U={fi|i=1,...,N} obtained through random sampling, where N is the number of points in the point cloud, RPM-Net uses a nearest neighbor sampling algorithm to obtain the K nearest neighbors of each sampled point, and then performs feature encoding according to the following formula: ； In the formula, F ui It includes the sampling center point u i A hybrid feature of spatial coordinates and neighborhood topology; f θ It is based on the PointNet encoding calculation formula, and its function is to aggregate the information of all points in the neighborhood into a single feature descriptor, u i The spatial coordinates of the center point, △u i ,k is the relative displacement between the neighboring points and the center point; PPF(u i , u k The center point u is described in a rotation-invariant manner. i and its neighboring points u k The surface between, where ni and nk are the normal vectors of the center point and the neighborhood points, respectively, and ∠(·) is the surface that guarantees the range of [0, ]. The formula for calculating angles within a certain range is given; ||·|| is the formula for calculating Euclidean distance. RPM-Net will center point u i and each of its neighboring points u k The features are concatenated into a 10-D feature vector, which is then input into the MLP layer and the max pooling layer to generate a global feature vector with a larger receptive field. Finally, the global feature vector is mapped onto the multilayer perceptron and processed.

2. Normalization yields a single mixed feature vector F ui ; The hybrid features F extracted based on the RPM-Net feature extraction module ui Calculate the Kendall correlation coefficient. The calculation formula is: ； in, , && represents AND logic, || represents OR logic. This indicates the number of pairs of points that are positively correlated, and the method for determining this is as follows: , , Each element in the list is greater than , , ; This indicates the number of pairs of points that are negatively correlated, determined by the following method: , , There exists any one element less than , , ; Indicates from Combinations of selecting 2 points from 10 points without considering sorting; , , These represent the autocorrelation of the sampling point, neighboring points, neighborhood structure information, and relative displacement vector, respectively.

4. The noise-resistant bidirectional attention point cloud registration method as described in claim 3, characterized in that, In step 3, the Copula denoising module uses Kendall's rank to measure the positive and negative correlation between the features of each point cloud, namely point coordinate features, neighborhood structure features and relative displacement vector features. It filters out negatively correlated point clouds, specifically outlier points and isolated points, through iterative loops, to ensure that the point clouds in the local neighborhood are all internal points to the greatest extent possible.

5. The noise-resistant bidirectional attention point cloud registration method as described in claim 4, characterized in that, In step 4, the local bidirectional attention module defines the feature encoding process as the following two steps: Step 4.1: Bidirectional attention feature encoding of sampling centers. Through sampling point self-reinforcement, the interaction force of sampling points to neighboring points and the influence factor of neighboring points on sampling points, the reinforcement learning of the neighborhood spatial relationship of sampling center points is realized. Step 4.2: Neighborhood spatial structure feature encoding, which realizes local feature encoding through the spatial Euclidean distance, direction vector and normal vector information between the sampling point and the neighboring points.

6. The noise-resistant bidirectional attention point cloud registration method as described in claim 5, characterized in that, The process of obtaining the annealing parameters of the point cloud dataset first requires feeding the features extracted by the local bidirectional attention module into a fully connected network to estimate the parameter values ​​from the feature information; secondly, the estimated annealing parameters α and β are fed into the SoftPlus activation function for positive conversion.

7. The noise-resistant bidirectional attention point cloud registration method as described in claim 6, characterized in that, The two point cloud features and annealing parameters extracted and estimated in steps 4 and 5 are used to solve for the optimal distance in the double random constraint matrix using the Sinkhorn algorithm. The initial state of this matrix is ​​shown below: ； In the formula F xj and F yk These represent the characteristics of two point clouds respectively; since each point x in point cloud X... j There is not always a corresponding point y in the point cloud Y. k We need to add a row and a column to the j-th row and k-th column of matrix M respectively to satisfy the iteration conditions of the double random constraint matrix; for each point x in the point cloud X j Construct the corresponding point y. j The formula for its solution is shown below: ; To obtain (x) with a one-to-one correspondence j y j Then, the rotation matrix R and translation matrix t can be solved by singular value decomposition.

Images