Anti-noise bidirectional attention point cloud registration method

Abstract

The invention relates to the technical field of three-dimensional graphic calculation, in particular to an anti-noise two-way attention point cloud registration method, which comprises the following steps: firstly, constructing an RPMNet + + network based on an RPMNet as a basic framework, performing feature correlation analysis on point cloud data through a Copula denoising module, eliminating noise points, outliers and abnormal values which are negatively correlated, and reserving inner points which are negatively correlated; performing neighborhood information aggregation enhancement on sampling point features through a local bidirectional attention mechanism module, improving the ability of a network model to learn local neighborhood context information at the same time, and fully learning neighborhood correlation features from point cloud data which contains noise points and is non-uniform in density; and finally, in a corresponding relation estimation module, utilizing a Sinkhorn normalization iteration algorithm to iterate the matching points, sending the corresponding points to a rotation matrix estimation module to carry out singular value decomposition, calculating a new rotation transformation matrix, introducing the rotation matrix in a new round of iteration until the network converges, and outputting a final registration result.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional graphics computing technology, and in particular to a noise-resistant bidirectional attention point cloud registration method. Background Art

[0002] Point cloud registration refers to the process of incorporating different point clouds with rigid body transformations (rotation and translation) into a unified coordinate framework to achieve a more accurate and complete description of the same target or scene. It is widely used in many industries and fields such as robot navigation, 3D reconstruction, and optical measurement. Based on the precise geometric features with corresponding relationships in the point cloud to be registered, traditional point cloud registration technology can directly estimate or iteratively optimize the rigid body transformation parameters; however, due to the unstructured and disordered characteristics of point cloud data itself, as well as the significant differences in acquisition mechanism, noise interference, spatial distribution, density, etc. between different point clouds, automatic high-precision point cloud registration is still a very challenging task.

[0003] Thanks to the successful application of convolutional neural networks (CNNs) in many image vision tasks and the rapid development of deep learning technology for point cloud processing, point cloud registration networks that can be flexibly designed to cope with various scenarios (targets) under non-ideal sample conditions have become an important research direction. According to the simulation method of the local function or overall process of traditional point cloud registration, the existing point cloud registration networks can be divided into two categories: feature learning framework and parameter learning framework. The point cloud registration network based on the feature learning framework uses the advantages of CNN deep learning to capture effective features (points) from the point cloud, realize the robust search of correspondences, replace the feature extraction module of the same name in the traditional point cloud registration calculation and improve its performance. Compared with the traditional point cloud registration process, the feature learning framework is conducive to global context, diversified feature learning and can simplify the design of feature descriptors, but it relies on high-quality sample label data training, which is difficult to guarantee in practical application. The point cloud registration network under the parameter learning framework realizes the whole process of feature learning of the point cloud to be registered, corresponding relationship search and transformation parameter optimization estimation through the design of network structure and loss function, and outputs the transformation parameters (or matrix) end-to-end. Compared with the feature learning framework, the point cloud registration network under the end-to-end parameter learning framework combines the traditional registration optimization strategy with the advantages of deep learning of point cloud features. It reduces the network's requirements for the completeness of training samples while ensuring the registration accuracy, while improving the network convergence speed, and has more development prospects; however, there are still shortcomings in the registration of point cloud targets with more locally similar structures, and it relies on high-quality sample label data training.

[0004] In the existing point cloud feature extraction network, the target point is often used as the sampling center to build a neighborhood system to implement the convolution of irregular point cloud data. The weight sharing characteristics of the convolution operation will sacrifice the accuracy of feature extraction in the local noise area to ensure the effectiveness of global feature extraction. Therefore, when the noise points are distributed throughout the space, it is difficult for the network model to effectively extract point cloud features, which will aggravate the model's description deviation of matching features. Summary of the invention

[0005] The purpose of the present invention is to provide a noise-resistant bidirectional attention point cloud registration method, aiming to solve the technical problems that the existing point cloud registration network model is susceptible to noise interference and has limited ability to learn local fine-grained features.

[0006] To achieve the above object, the present invention provides a noise-resistant bidirectional attention point cloud registration method, comprising the following steps:

[0007] Step 1: Based on RPM-Net as the basic framework, an RPMNet++ network is constructed, wherein the RPMNet++ network includes a Copula denoising module, a local bidirectional attention module, an annealing parameter prediction module, a correspondence estimation module, and a rotation matrix estimation module;

[0008] Step 2: Use the initial / current rotation matrix to perform a rigid transformation on the origin set X to obtain the rotation point set, and use the RPM-Net feature extraction module to extract the mixed features of the corresponding data;

[0009] Step 3: In the Copula denoising module, the inliers are screened according to the Kendall correlation coefficient and the Clayton Copula model, and stored as a new origin point set and a new target point set;

[0010] Step 4: In the local bidirectional attention module, a bidirectional attention mechanism is used from the sampling point to the neighboring point and from the neighboring point to the sampling point;

[0011] Step 5: Estimate annealing parameters using the annealing parameter prediction module;

[0012] Step 6: In the correspondence estimation module, the Sinkhorn normalized iterative algorithm is used to estimate the corresponding points in the two point clouds;

[0013] Step 7: The corresponding points are sent to the rotation matrix estimation module for singular value decomposition, and a new rotation transformation matrix is ​​calculated. This rotation matrix is ​​introduced in a new round of iteration until the RPMNet++ network converges and outputs the final registration result.

[0014] Optionally, the Clayton Copula model distribution function in step 3 is

[0015]

[0016] Among them, h i and h i,k They represent the mixed features of the local neighborhood center point and the neighborhood points sampled from the point cloud dataset U, respectively. λ is the Clayton Copula model parameter, 0<λ<∞. When λ→0, C(·)→0, and all points tend to be independent; when λ→∞, C(·)→1, and the correlation between all points is higher.

[0017] Optionally, RMP-Net uses the nearest point sampling algorithm to obtain K neighboring points of the sampling point for the point cloud U = {fi|i = 1, ..., N} obtained by random sampling, where N is the number of point clouds, and performs feature encoding according to the following calculation formula:

[0018]

[0019] In the formula, F ui is the sampling center point u i The hybrid features of spatial coordinates and neighborhood topology; f θ It is an encoding calculation formula based on PointNet, which is used to aggregate all point information in the neighborhood into a single feature descriptor. i is the spatial coordinate of the center point, △u i , k is the relative displacement / direction vector between the neighborhood point and the center point; PPF(u i ,u k ) describes the center point u in a rotationally invariant way i and its neighboring point u k The surface between, where ni and nk and are the normal vectors of the center point and the neighborhood point respectively, and ∠(·) is to ensure that the value range is [0, ) is the angle calculation formula within the range, ||·|| is the Euclidean distance calculation formula;

[0020] RPM-Net takes the center point u i And each of its neighboring points u k The features are concatenated into a 10-D feature vector, and the feature is input into the MLP layer and the maximum pooling layer to generate a global feature vector with a larger receptive field; finally, the global feature vector is mapped to the multi-layer perceptron and 2 Normalize to get a single mixed feature vector F ui ;

[0021] Based on the hybrid feature F extracted by the RPM-Net feature extraction module ui The calculation formula for calculating the Kendall correlation coefficient τ is:

[0022]

[0023] in, i≠j, && represents the AND logic, || represents the OR logic, P{(·)>0} represents the number of groups of positively correlated points, and the judgment method is e i , p i , d i Every element in is greater than e j , p j , d j ; N{(·)<0} indicates how many groups of negatively correlated points there are, and the judgment method is e i , p i , d i There is any element in the j , p j , d j ; Q{·∪·∪·} means selecting 2 points from k+1 points without considering the combination of sorting; e i , p i , d i They represent the autocorrelation of sampling points, neighborhood points, neighborhood structure information and relative displacement vector respectively.

[0024] Optionally, in step 3, the Copula denoising module uses Kendall rank to measure the positive and negative correlations between each point cloud feature, namely, point coordinate features, neighborhood structure features, and relative displacement vector features, and filters out negatively correlated point clouds, specifically outlier points and outliers, through a cyclic iteration method, to ensure that the point clouds in the local neighborhood are all internal points to the greatest extent.

[0025] Optionally, the local bidirectional attention module in step 4 clarifies the feature encoding process into the following two steps:

[0026] Step 4.1: Bidirectional attention feature encoding of the sampling center, through the self-enhancement of the sampling point, the interaction force of the sampling point on the neighboring points and the influence factor of the neighboring points on the sampling point, the enhanced learning of the neighborhood spatial relationship of the sampling center point is realized;

[0027] Step 4.2: Neighborhood spatial structure feature encoding, local feature encoding is achieved through the spatial Euclidean distance between the sampling point and the neighborhood point, direction vector and normal vector information.

[0028] Optionally, the process of obtaining the annealing parameters of the point cloud dataset first requires sending the features extracted by the local bidirectional attention module into the fully connected network to estimate the parameter values ​​from the feature information; secondly, sending the estimated annealing parameters α and β into the SoftPlus activation function for positive processing.

[0029] Optionally, the two point cloud features and annealing parameters extracted and estimated in steps 4 and 5 are used to solve the optimal distance of the double random restriction matrix using the Sinkhorn algorithm. The initial state of the matrix is ​​as follows:

[0030]

[0031] Where F xj and F yk Represent the characteristics of two point clouds respectively; since each point x in the point cloud X j There is not always a corresponding point y in the point cloud Y k , it is necessary to add a row and a column to the jth row and kth column of the M matrix to satisfy the iteration condition of the double random restriction matrix; for each point x in the point cloud X j , construct the point y corresponding to the tone j , and its solution is shown as follows:

[0032] Get a one-to-one correspondence (x j ,y j ) After that, the rotation matrix R and translation matrix t can be solved by singular value decomposition.

[0033] The present invention provides a noise-resistant bidirectional attention point cloud registration method. Firstly, an RPMNet++ network is constructed based on the RPM-Net basic framework. The feature correlation analysis of point cloud data is performed through the Copula denoising module to remove the negatively correlated features of noise points, outliers and abnormal values, and the positively correlated inliers are retained. Then, the neighborhood information aggregation and enhancement of the sampling point features are performed through the local bidirectional attention mechanism module, while the ability of the network model to learn local neighborhood context information is improved, and the neighborhood correlation features are fully learned from point cloud data containing noise points and uneven density. Finally, in the correspondence estimation module, the Sinkhorn normalized iterative algorithm is used to iterate the matching points, and the corresponding points are sent to the rotation matrix estimation module for singular value decomposition, and a new rotation transformation matrix is ​​calculated. The rotation matrix is ​​introduced in a new round of iteration until the network converges and the final registration result is output. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 It is a schematic diagram of the structure of the RPMNet++ network of the present invention.

[0036] Figure 2 It is a specific flow chart of a noise-resistant bidirectional attention point cloud registration method of the present invention.

[0037] Figure 3 It is a structural schematic diagram of the Copula denoising module of the present invention.

[0038] Figure 4 It is a schematic diagram of the structure of the local bidirectional attention module of the present invention.

[0039] Figure 5 It is a schematic diagram comparing the noise-free point cloud registration effects in a specific embodiment of the present invention.

[0040] Figure 6 It is a schematic diagram comparing the registration effects of Gaussian noise point clouds with different densities according to a specific embodiment of the present invention.

[0041] Figure 7 It is a schematic diagram comparing the effects of structure-missing Gaussian noise point cloud registration according to a specific embodiment of the present invention.

[0042] Figure 8 It is a schematic diagram comparing the residual noise ratios of three groups of data before and after adding the Copula denoising module according to a specific embodiment of the present invention.

[0043] Fig. 9 It is a schematic diagram comparing the visualization effects of denoising models under different Gaussian noise variances according to a specific embodiment of the present invention. DETAILED DESCRIPTION

[0044] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0045] Some English terms appearing in this article are explained as follows:

[0046] robust point matching network, RPM-Net: robust point matching network;

[0047] farthestpoint sampling, FPS: farthest point sampling algorithm;

[0048] multilayerperceptron, MLP: multilayer perceptron;

[0049] singularvalue decomposition, SVD: singular value decomposition.

[0050] The present invention provides a noise-resistant bidirectional attention point cloud registration method, comprising the following steps:

[0051] Step 1: Based on RPM-Net as the basic framework, an RPMNet++ network is constructed, wherein the RPMNet++ network includes a Copula denoising module, a local bidirectional attention module, an annealing parameter prediction module, a correspondence estimation module, and a rotation matrix estimation module;

[0052] Step 2: Use the initial / current rotation matrix to perform a rigid transformation on the origin set X to obtain the rotation point set, and use the RPM-Net feature extraction module to extract the mixed features of the corresponding data;

[0053] Step 3: In the Copula denoising module, the inliers are screened according to the Kendall correlation coefficient and the Clayton Copula model, and stored as a new origin point set and a new target point set;

[0054] Step 4: In the local bidirectional attention module, a bidirectional attention mechanism is used from the sampling point to the neighboring point and from the neighboring point to the sampling point;

[0055] Step 5: Estimate annealing parameters using the annealing parameter prediction module;

[0056] Step 6: In the correspondence estimation module, the Sinkhorn normalized iterative algorithm is used to estimate the corresponding points in the two point clouds;

[0057] Step 7: The corresponding points are sent to the rotation matrix estimation module for singular value decomposition, and a new rotation transformation matrix is ​​calculated. This rotation matrix is ​​introduced in a new round of iteration until the RPMNet++ network converges and outputs the final registration result.

[0058] The structure of the RPMNet++ network is as follows Figure 1 As shown, further, the process of the noise-resistant bidirectional attention point cloud registration method described in the present invention is as follows Figure 2As shown in the figure, firstly, the RPMNet++ network is built based on the RPM-Net basic framework, and the feature correlation analysis of the point cloud data is performed through the Copula denoising module to remove the negatively correlated features of noise points, outliers and abnormal values, and the positively correlated inliers will be retained; then, the neighborhood information aggregation and enhancement of the sampling point features are performed through the local bidirectional attention mechanism module, while improving the ability of the network model to learn local neighborhood context information, and fully learn the neighborhood correlation features from the point cloud data containing noise points and uneven density; finally, in the correspondence estimation module, the Sinkhorn normalized iterative algorithm is used to iterate the matching points, and the corresponding points are sent to the rotation matrix estimation module for singular value decomposition, and the new rotation transformation matrix is ​​calculated. The rotation matrix is ​​introduced in a new round of iteration until the network converges and the final alignment result is output.

[0059] The following is a further explanation of the main component modules in the process of step execution:

[0060] 1. Principle of Copula denoising module:

[0061] From a mathematical perspective, the correlation between noise points and inliers can be quantified using deterministic metrics. The Clayton Copula model is used to quantify the correlation between multiple points, and noise points are removed based on the positive and negative correlation of data points within the K neighborhood. The Clayton Copula model distribution function is defined as

[0062]

[0063] Among them, h i and h i,k Respectively represent the mixed features of the local neighborhood center point and neighborhood points sampled from the point cloud data set U, and λ∈(0,∞) is the Clayton Copula model parameter. When λ→0, C(·)→0, and all points tend to be independent; when λ→∞, C(·)→1, the higher the correlation between all points: Therefore, the value of λ not only determines the range distribution of the Clayton Copula model, but also defines the correlation between the points in the local neighborhood. The λ parameter can be calculated by the Kendall correlation coefficient. The present invention calculates the Kendall correlation coefficient τ according to the spatial position of the sampling center point and the neighborhood point, the neighborhood structure information and the relative displacement vector. The formula is

[0064]

[0065] in,

[0066]

[0067] i≠j, && represents the AND logic, || represents the OR logic, P{(·)>0} represents the number of groups of positively correlated points, and the judgment method is e i , p i , d i Every element in is greater than e j , p j , d j ; N{(·)<0} indicates how many groups of negatively correlated points there are, and the judgment method is e i , p i , d i There is any element in the j , p j , d j ; Q{·∪·∪·} represents the combination of selecting 2 points from k+1 points without considering the order. In formula (2), e i , p i , d i Respectively represent the autocorrelation of sampling points, neighborhood points, neighborhood structure information and relative displacement vector, defined as

[0068]

[0069] in, u i ,u i,k They represent the spatial location information of the sampling point and the neighborhood point respectively. PPF(·) is used to describe the neighborhood structure information of the point pair feature. Δu i,k =u i -u i,k Represents the spatial displacement vector from the sampling point to the neighborhood point. τ describes the change trend between local neighborhood points according to the changes in P(·) and N(·): when P(·)>N(·), τ→1, indicating that the relationship between neighborhood points tends to be completely consistent; when P(·)<N(·), τ→-1, indicating that the relationship between neighborhood points tends to be completely contradictory; when P(·)=N(·), τ=0, indicating that the change relationship between neighborhood points cannot be determined. It can be seen from equations (1) and (2) that the range of λ is (0, ∞), and the domain of τ is (-1,1). Further, the relationship function between τ and λ is defined as

[0070]

[0071] Substituting λ obtained from formula (4) into formula (1), the Copula denoising module proposed in the present invention can be constructed, as follows: Figure 3 This module makes full use of the similarity of the point cloud features of the K-neighborhood system to remove noise points and improve the network model's ability to learn point cloud data features. Figure 3 Medium,U,U I and U iThey represent the original point cloud dataset, the sub-region dataset obtained from U by the farthest point sampling (FPS) algorithm, and the sub-region dataset obtained from U by the random sampling method. I The sampling center point set is obtained by extracting without replacement, where U is much larger than U I and U i , while U i Depending on the degree of noise interference, usually U I 1 / 2 or 1 / 4 of the points. i The K-nearest neighbor (KNN) algorithm is used to obtain the neighboring point U i,k Extract U i and U i,k The local point cloud data features composed of , and use formula (3) to calculate the inner product, and substitute the inner product into formula (2) to calculate the Kendall correlation coefficient: (1) When τ≤0, the corresponding U i Identify as noise points and delete them from the original point cloud data, and randomly sample U I The remaining points in U i ′Additional, the sampling process is also in the form of extraction without replacement, and the point set update is consistent with the original operation. (2) When τ>0, substitute it into formula (4) to calculate the λ parameter of the Clayton Copula model, and substitute λ into formula (1) to quantify the correlation of each neighborhood point. When C(·)→0, calculate the negative correlation coefficient Q{(·)<0} of each neighborhood point according to formula (2) for identification, retain all positively correlated neighborhood points, and delete all negatively correlated neighborhood points as noise points from the original point cloud data; then use the KNN algorithm to sample the remaining points in the local space until the number of neighborhood points meets K; finally, perform local mixed feature extraction and correlation judgment on the recombined neighborhood points until C(·)→1, that is, τ=1, and the output is the sampling point U i ′. When the inliers in the local space cannot meet the sampling requirements of K neighboring points, the inliers are searched by expanding the neighborhood radius by 1 to meet the consistency of the local feature dimension. Obviously, by calculating the correlation between point pair features and features, noise points can be effectively found, the interference of noise features can be reduced to a certain extent, and the network's ability to estimate parameters and learn features can be improved overall.

[0072] The Copula denoising module is based on the assumption that point cloud data in a neighborhood have similar characteristics. I Perform random sampling to obtain U i And select K neighboring points U i,kPerform feature encoding; then construct the ClaytonCopula model distribution function C(·) based on the feature information and calculate the Kendall correlation coefficient τ; finally, use C(·) and τ to filter out noise points and iterate U I points in until C(·)→1 is satisfied. The specific steps of the algorithm are as follows:

[0073] 1. For the target point cloud dataset U, set the sub-region dataset U I , sampling point set U i , neighborhood point set U i,k , neighborhood radius r, number of neighborhood points K, Kendall correlation coefficient τ, Clayton Copula distribution function C(·) and interior point array U i ′.

[0074] 2. Sample U from U using FPS I , using random sampling algorithm and KNN from U I Get {U i ,U i,k}, and substitute the sampled point set into formula (2) to calculate the τ value. If the τ value is less than or equal to 0, then I Re-acquire {U i ′,U i ' ,k} and set the initial {U i ,U i,k} discarded until the τ value is greater than zero and the qualified U i 'Point set saved.

[0075] 3. Use formula (1) to calculate {U i ′,U i ' ,k} point set to calculate the correlation of neighboring points. If C(·)→1, then {U i ′,U i ' ,k}Save and output, otherwise the U in the neighborhood i ' ,k Resample until C(·)→1. It should be noted that if all points in the neighborhood cannot satisfy C(·)→1, the neighborhood radius r is doubled until all U i ' ,k Satisfies C(·)→1.

[0076] The Copula denoising module uses Kendall rank to measure the positive and negative correlations between each point cloud feature (point coordinate feature, neighborhood structure feature and relative displacement vector feature), and filters out negatively correlated point clouds (outlier points and outliers) through cyclic iteration to ensure that the point clouds in the local neighborhood are all internal points to the greatest extent.

[0077] 2. Principle of local bidirectional attention module:

[0078] Although the Copula denoising module can remove noise points with significant feature differences in the neighborhood of a sampling point, when the K neighborhood contains multiple types of targets or the feature differences between the noise data and other data in the neighborhood are not obvious, the Copula denoising result will still contain a small amount of noise with similar features to the internal point data, affecting the local neighborhood feature expression; at the same time, Copula denoising will aggravate the uneven density of the point cloud, especially the local areas with dense noise points will become sparse after denoising, which will destroy the adjacency relationship of the original target point cloud data to a certain extent, affecting the expression of the local geometric structure features of the target and the learning ability of CNN for neighborhood similarity features.

[0079] In order to overcome the influence of a small number of noise points and effectively extract data features from sparse point clouds, and further improve the correlation of data features in the neighborhood, the present invention designs the following Figure 4 The local bidirectional attention module shown is composed of sampling center bidirectional attention feature coding and neighborhood space structure feature coding. Wherein, B represents the batch size; N represents the number of sampling points contained in each batch, and the present invention only considers the 3D spatial position information of K neighborhood points and N sampling points; the sampling center bidirectional attention feature coding realizes enhanced learning of the neighborhood spatial relationship of the sampling center point through self-enhancement of the sampling point, the interaction force of the sampling point on the neighborhood point, and the influence factor of the neighborhood point on the sampling point; the neighborhood space structure feature coding aims to realize local feature coding through the spatial Euclidean distance between the sampling point and the neighborhood point, the direction vector and the normal vector information, so as to improve the network model's ability to learn the local fine-grained features of the point cloud data. First, the input features are feature encoded, and the formula is

[0080]

[0081] in, Enhance the feature F by sampling points i and local geometric structure characteristics F i 'composition, g(·), y(·) and z(·) represent linear transformation operations (1×1 convolution operations), C represents the number of output channels, and After two linear transformations, the feature similarity can be obtained by transposing and calculating the inner product. The feature similarity is obtained by dividing by Adjustment is made to make the training process have a more stable gradient. Formula (5) assigns the self-attention weight calculated by the normalized exponential function softmax to the local spatial coordinate feature after linear transformation, and then adds it to itself to construct the output result of the local bidirectional self-attention mechanism represented by the feature vector, which reflects the fine-grained geometric structure of the sampling point neighborhood. Medium, Fi The purpose is to further aggregate the influence of each point in the neighborhood on the sampling point. The aggregation calculation formula is:

[0082]

[0083] Among them, f i =X δ (u i ), f i,k =X δ (u i,k ),u i (i∈N) and u i,k Represent the spatial coordinates of the sampling point and the neighboring point respectively, according to the PointNet feature encoding method X δ Generate sampling point features f i and neighborhood point features f i,k , and substitute them into F i Perform sampling point feature enhancement. Formula (6) consists of three parts: (1) f i (1+f i T ) represents the self-enhancement of sampling point features; (2)-MLP(f i -f i,k ) First calculate the influence factor of each neighborhood point feature on the sampling point feature, and then project them to the multilayer perceptron (MLP); (3) (f i,k -f i ) represents the relationship between the features of each neighborhood point and the features of the sampling point. By multiplying and accumulating with (2), the learning of the enhanced features of the sampling point can be realized. Formula (6) can be analyzed from the perspective of force field, that is, each f in the local space k All right i There is 1 force, gravity, trying to push f i Zoom in k , and the repulsive force pushes the two apart; the strength of the force is given by -MLP(f i -f i,k ), the direction of the force is determined by (f i,k -f i ) is determined by the difference between the two feature vectors. Therefore, F i The relationship between each neighborhood point and the sampling point is fully integrated, which can better describe the neighborhood correlation characteristics. i ′ and its neighboring point U i ' ,k The Euclidean distance between the direction vector, the normal vector and the spatial position of the neighboring points, F i The formula for spatial encoding is

[0084]

[0085] in,

[0086] PPF(u i ,u i,k )=(∠(n i ,(u i,k -u i )),∠(n i,k ,(u i,k -u i )),∠(n i ,n i,k )).

[0087] It should be pointed out that in formula (7) (u i,k -u i ) and PPF(u i ,u i,k ) represent the Euclidean distance between the sampling point and the neighboring point, the relative displacement (direction) vector and the normal vector information respectively. The three are concatenated to generate a 13-dimensional feature vector

[0088] The present invention also demonstrates the registration experimental effect through specific embodiments to verify the feasibility:

[0089] It is used for the purpose of evaluating the performance of homologous point cloud data registration under different noise conditions. It is divided into three data types: noise-free point cloud, Gaussian noise point cloud under different densities, and Gaussian noise point cloud with structural loss. In addition, ablation experiments are designed to evaluate the role of the proposed network function module. ModelNet40 contains 40 CAD models of different object categories. In order to evaluate the ability of the network to generalize to different object categories, 20 categories of data are randomly selected for training and evaluation, and the data of the remaining categories are all used for testing, totaling 5112 training models, 1202 verification models and 1266 test models. In order to ensure the consistency of the point dimension of each model during the experiment, the PointNet++ farthest point sampling algorithm is used to downsample each model and the number of sampling points is set to 2048. The three Euler angles of the rigid transformation matrix are randomly sampled in the range of [0, 45°], and the random translation value is moved in the range of [-0.5, 0.5] on each axis.

[0090] Noise-free point cloud experiment:

[0091] First, the point cloud (2048 points) randomly extracted from each model of ModelNet40 is copied to obtain the source point cloud X and the reference point cloud Y with precise point correspondence. Then, the randomly generated rigid transformation matrix (R, t) is applied to X and randomly scrambled to obtain X tTable 1 shows the registration results of the method of the present invention and the method in the literature under the above-mentioned noise-free point cloud data conditions, where: the registration evaluation content includes isotropy, anisotropy indicators and chamfer distance, bold indicates the optimal performance parameters, and italics indicate suboptimal performance parameters; the comparison method includes not only the point cloud registration networks PointNetLK, DeepClosest Point (DCP-v2) and RPM-Net based on deep learning, but also the traditional point cloud registration algorithms ICP and FGR. Figure 5 The point cloud registration visualization effects of the present invention and the literature method are given.

[0092] Table 1. Performance statistics of noise-free point cloud registration

[0093]

[0094]

[0095] From the registration accuracy and Figure 5 The visualization effect of the registration shows that the source point cloud X t (blue) deviates from the reference point cloud Y (red) at different angles, such as Figure 5 In (a), the blue trees are obviously left-leaning and excessively right-leaning relative to the red trees. Figure 5 (b) Medium blue chair and Figure 5 The blue stool in (c) is tilted back to some extent. The reason is that the ICP algorithm is sensitive to the initial value of the parameters and tends to converge to the local minimum, resulting in the highest R1_err and T1_err, reaching 6.407 and 0.0506 respectively. The advantage of DCP-v2 is that it can aggregate global-local feature information for soft matching matrix calculation, but its ability to accurately optimize the corresponding relationship is insufficient. Figure 5 The blue point clouds in (a), (b) and (c) still have leftward, forward and backward tilt phenomena, respectively. Therefore, although it has been significantly improved compared with ICP (R1_err and T1_err are reduced by 2.415 and 0.0214 respectively), the registration accuracy ranks second in Table 1. Compared with the above two methods, PointNetLK achieves better registration effects in all three types of targets, and its R1_err and T1_err have achieved good results of 0.847 and 0.0054 respectively. Figure 5 The top branches and leaves of the blue tree in (a) are still offset. This is because PointNetLK does not fully consider the local feature information when extracting the global feature descriptor of each point, which leads to convergence to the local minimum when processing targets with similar local structures. Figure 5It can be found that RPM-Net is slightly inferior to the method of the present invention in the tree registration effect. This is because the method of the present invention adds a local feature learning module under the bidirectional attention mechanism on the basis of RPM-Net. For some models with weak global feature significance and repeated local features, the new module can effectively improve the network's discrimination of local similar structures; at the same time, the new Copula denoising module of the method of the present invention increases the calculation time because a noise check is performed once in each iteration. For point clouds with a large number of precise point correspondences, FGR can obtain highly consistent model outer contour surface information for accurate estimation of transformation parameters, thereby obtaining the optimal registration effect (R1_err and T1_err are 0.022 and 0.0002 respectively). The method of the present invention is close to it as a whole, but is slightly inferior in terms of isotropic rotation error (only R1_err lags behind by 0.008).

[0096] Experiments on Gaussian noise point clouds with different densities:

[0097] First, 2048 points are randomly sampled from the same model to obtain the source point cloud X and the reference point cloud Y without point correspondence. Then, the source point cloud X is spatially transformed using the random rigid transformation matrix (R, t) and 2048 Gaussian noise points are added to the two point clouds, where: 2 ) parameters are μ=0 and σ=0.1 respectively. The point cloud data with Gaussian noise added is sent to the network for training and testing. The registration results of the present invention and the reference method are shown in Table 2. Figure 6 At the same time, the point cloud registration visualization effect of the present invention and the literature method is given.

[0098] Table 2 Statistics of registration performance of Gaussian noise point clouds with different densities

[0099]

[0100] Comparing Table 1 and Table 2, we can see that since the Gaussian noise point cloud data cannot guarantee the correspondence of points, DCP-v2 uses feature difference to estimate the soft assignment matrix, which has a large number of mismatched points, resulting in the highest R1_err and T1_err, which are 8.922 and 0.0707 respectively. Figure 6 In (a), (b) and (c), the angular offset between the source point cloud and the reference point cloud in each coordinate axis direction is the largest; although the R1_err and T1_err of ICP are reduced by 1.923 and 0.0193 respectively compared with DCP-v2, good registration cannot be achieved. It is worth noting that Figure 6 The FGR registration effect on curtains, potted plants and airplanes is far less effective than that on Figure 5The reason is that the noise affects the effective extraction of object surface information by FGR and the accurate estimation of the correspondence between point clouds, which leads to a significant decline in registration accuracy. R1_err and T1_err are 2.991 and 0.0252 respectively. Similarly, the feature encoding method of PointNetLK has weak anti-noise ability. Figure 6 The registration result of PointNetLK in the paper shows that the source point cloud partially deviates from the reference point cloud. By adopting 4D point pair features with rotation invariance and replacing spatial coordinate differences with feature differences to describe local spatial structures, RPM-Net can obtain a robust estimate of the point cloud correspondence, and outperforms the literature methods other than the method of the present invention in terms of R1_err (0.664) and T1_err (0.0062). Compared with the registration effect of RPM-Net, although the visual difference is not obvious, the accuracy of the method of the present invention has been improved by nearly 50% in terms of isotropy and anisotropy indicators. The reason is that the network of the present invention uses the Copula denoising module to remove noise points, outliers and outlier points, while supplementing the unjudged point cloud as internal points, so as to minimize the interference of noise points on the annealing parameter estimation and feature extraction process, thereby achieving a more accurate registration effect than RPM-Net, which fully proves the effectiveness of the Copula denoising model.

[0101] Structure missing Gaussian noise point cloud experiment:

[0102] In order to verify the registration performance of the network of the present invention on data with partial structure missing, 30% of the local area of ​​the model is randomly discarded when sampling the model points. At the same time, Gaussian noise is added to the sampled point cloud using the method in the previous section and then sent to the network for training and testing. Table 3 shows the statistical results of the registration performance of the Gaussian noise point cloud with structure missing by the present invention and the method in the literature. Figure 7 The corresponding point cloud registration visualization effect is given.

[0103] Table 3. Performance statistics of structure-missing Gaussian noise point cloud registration

[0104]

[0105] Comparing Table 3 with Table 2, it can be seen that when there is a partial structural loss in the model, the point cloud registration performance of all algorithms except the algorithm of the present invention and RPM-Net is greatly reduced, especially in terms of isotropic and anisotropic rotation errors. Specifically, FGR has the worst R1_err (30.839) and T1_err (0.192). Figure 7 a) and bookshelf( Figure 7b), which indicates that FGR is not only sensitive to noise, but also has difficulty in effectively processing missing data with complex structures. The PointNetLK feature extraction module cannot effectively cope with complex environments or missing target structures. Obvious offsets are found in all three target registration tasks, with R1_err (12.607) and T1_err (0.169) ranking the last two. Missing target structures also aggravate the mismatching of ICP and DCP-v2. The former also has obvious offsets in all three target registration tasks, while the latter has smaller offsets only in the computer registration task. Compared with the above methods, RPM-Net performs well in the registration of the three types of targets, with R1_err and T1_err of 1.712 and 0.018 respectively. The reason is that its 4D point pair feature extraction method can better describe the characteristics of each local structure, and the adopted Sinkhorn iteration method is conducive to the matching of some identical data under the condition of missing structure; in comparison, the feature information extracted by the local bidirectional attention module of the network of the present invention not only has the 4D point pair feature information of RPM-Net, but also contains the interaction relationship information from the center point to the sampling point and from the sampling point to the center point, and can obtain richer feature information than RPM-Net for corresponding point search, so the optimal registration accuracy is achieved. This shows that the bidirectional attention module of the present invention enhances the local fine-grained feature learning ability through adaptive local feature learning, while facilitating the accurate search of internal point correspondence.

[0106] Copula denoising module experiment:

[0107] To further verify the effectiveness of the Copula denoising model, we first sampled and generated three sets of Gaussian noise point cloud data sets from the Modelnet40 data set. The specific methods are as follows: ① Sample the model with 2048 points and copy it into two copies, of which only one copy adds Gaussian white noise (2048 noise points); ② Sample the model with 2048 points and copy it into two copies, and both copies add Gaussian white noise (2048 noise points) with the same distribution; ③ Randomly sample points twice for the same model with 2048 points and add Gaussian white noise (2048 noise points) with the same distribution. The three sets of data sets were sent to the original RPM-Net and the RPM-Net with only the Copula denoising module added for training, and the noise points of the test data set were searched according to the matching matrix obtained from the training. Figure 8 The statistical results of RPM-Net denoising capabilities of three sets of data before and after adding the Copula denoising module are given. Fig. 9 The denoising model visualization effect of the second set of data under different Gaussian variance parameters σ is given, where: Gaussian white noise follows the distribution N(μ, σ 2 ) and the parameter μ takes the value 0, σ 2The parameters are (a) 0.02, (b) 0.03, (c) 0.04 and (d) 0.05 respectively.

[0108] Depend on Figure 8 From the residual ratio of noise points in the first set of data, it can be found that the internal point features with corresponding relationships are easy to be learned and distinguished by the network, while the randomly generated Gaussian white noise does not have a clear corresponding relationship. Therefore, RPM-Net can remove most of the noise points that do not have a corresponding relationship through the Sinkhorn normalization iterative method. Fig. 9 The residual noise points of the Copula denoising module are far less than those of RPM-Net, so the residual noise points of the Copula denoising module are magnified 3 times for easy observation. Figure 8 The second group and Fig. 9 It can be found that as the noise increases, the proportion of noise points left in the two networks gradually increases. This is because the larger the σ value, the greater the diffusion of Gaussian noise, which causes the feature information extracted by the two algorithms to contain more noise point features, ultimately affecting the accurate estimation of the internal point relationship. Fig. 9 In (b), (c) and (d), the number of noise points left by the Copula denoising module is far less than that of RPM-Net, which indicates that with the increase of Gaussian noise, the ability of RPM-Net to remove noise points is significantly reduced, while the Copula denoising module can still effectively identify and remove noise points. The reason is that the Copula denoising module can pre-eliminate some noise points and outlier points by comparing and analyzing the point feature information, and then send the processed data to the Sinkhorn normalization layer for further noise point removal and corresponding point estimation, and finally obtain more inliers and fewer noise corresponding points than RPM-Net; on the other hand, when the two groups of point clouds do not have a one-to-one correspondence, the proportion of noise points in both RPM-Net and Copula denoising modules is significantly increased. The reason is that the randomly distributed noise points have spatial position information similar to that of the inliers, which makes both networks have the problem of mismatching noise points with inliers and noise points with noise points as corresponding points. In general, both the Copula denoising module and the Sinkhorn normalization iterative method of RPM-Net can process noise points, but the Copula denoising module uses the difference in feature information to distinguish noise points from inliers, making it more effective in removing discrete noise points than RPM-Net.

[0109] In summary, the experimental results of point cloud registration show that on the ModelNet40 dataset, compared with RPM-Net, RPMNet++ has a significant effect on improving the isotropic average rotation error and translation error, which are reduced by (0.026, 0.001), (0.267, 0.0019) and (0.560, 0.007) on the noise-free dataset, the noise dataset and the partially visible noise dataset respectively; on the Stanford 3D dataset, compared with 7 cross-source registration algorithms, RPMNet++ can achieve the best performance and has good generalization performance and application value.

[0110] What is disclosed above is only a preferred embodiment of the present invention, and it certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A noise-resistant bidirectional attention point cloud registration method, characterized in that: The following steps are involved: Step 1: Based on RPM-Net as the basic framework, an RPMNet++ network is constructed, wherein the RPMNet++ network includes a Copula denoising module, a local bidirectional attention module, an annealing parameter prediction module, a correspondence estimation module, and a rotation matrix estimation module; Step 2: Use the initial / network feedback rotation matrix to rigidly transform the origin set X to obtain the rotation point set, and extract the mixed features of the corresponding data through the RPM-Net feature extraction module; Step 3: In the Copula denoising module, the inliers are screened according to the Kendall correlation coefficient and the Clayton Copula model, and stored as a new origin point set and a new target point set; Step 4: In the local bidirectional attention module, a bidirectional attention mechanism is used from the sampling point to the neighboring point and from the neighboring point to the sampling point; Step 5: Estimate annealing parameters using the annealing parameter prediction module; Step 6: In the correspondence estimation module, the Sinkhorn normalized iterative algorithm is used to estimate the corresponding points in the two point clouds; Subsequently, the corresponding points are sent to the rotation matrix estimation module for singular value decomposition, and a new rotation transformation matrix is ​​calculated. This rotation matrix is ​​introduced in a new round of iteration until the RPMNet++ network converges and outputs the final registration result.

2. The noise-resistant bidirectional attention point cloud registration method according to claim 1, characterized in that: The Clayton Copula model distribution function in step 3 is Among them, h i and h i,k They represent the mixed features of the local neighborhood center point and the neighborhood points sampled from the point cloud dataset U, respectively. λ is the Clayton Copula model parameter, 0<λ<∞. When λ→0, C(·)→0, and all points tend to be independent; when λ→∞, C(·)→1, and the correlation between all points is higher.

3. The noise-resistant bidirectional attention point cloud registration method according to claim 2, characterized in that: For the point cloud U={fi|i=1,...,N} obtained by random sampling, where N is the number of point clouds, RMP-Net uses the nearest point sampling algorithm to obtain the K neighboring points of the sampling point, and then performs feature encoding according to the following calculation formula: In the formula, F ui is the sampling center point u i Hybrid features of spatial coordinates and neighborhood topology; f θ It is an encoding calculation formula based on PointNet, which is used to aggregate all point information in the neighborhood into a single feature descriptor. i is the spatial coordinate of the center point, △u i , k is the relative displacement / direction vector between the neighborhood point and the center point; PPF(u i ,u k ) describes the center point u in a rotationally invariant way i and its neighboring point u k The surface between , where ni and nk and are the normal vectors of the center point and the neighborhood point respectively, ∠(·) is to ensure that the value range is The angle calculation formula within the range, ||·|| is the Euclidean distance calculation formula; RPM-Net takes the center point u i And each of its neighboring points u k The features are concatenated into a 10-D feature vector, and the feature is input into the MLP layer and the maximum pooling layer to generate a global feature vector with a larger receptive field; finally, the global feature vector is mapped to the multi-layer perceptron and 2 Normalize to get a single mixed feature vector F ui ; Based on the hybrid feature F extracted by the RPM-Net feature extraction module ui The calculation formula for calculating the Kendall correlation coefficient τ is: in, && indicates AND logic, || indicates OR logic, P{(·)>0} indicates how many groups of positively correlated points there are, and the judgment method is e i , p i , d i Every element in is greater than e j , p j , d j ; N{(·)<0} indicates how many groups of negatively correlated points there are, and the judgment method is e i , p i , d i There is any element in the j , p j , d j ; Q{·∪·∪·} means selecting 2 points from k+1 points without considering the combination of sorting; e i , p i , d i They represent the autocorrelation of sampling points, neighborhood points, neighborhood structure information and relative displacement vector respectively.

4. The noise-resistant bidirectional attention point cloud registration method according to claim 3, characterized in that: In step 3, the Copula denoising module uses Kendall rank to measure the positive and negative correlations between the point cloud features, namely the point coordinate features, the neighborhood structure features, and the relative displacement vector features, and filters out the negatively correlated point clouds, specifically the outlier points and outliers, through a cyclic iteration method, to ensure that the point clouds in the local neighborhood are all internal points to the greatest extent.

5. The noise-resistant bidirectional attention point cloud registration method according to claim 4, characterized in that: The local bidirectional attention module in step 4 clarifies the feature encoding process into the following two steps: Step 4.1: Bidirectional attention feature encoding of the sampling center, through the self-enhancement of the sampling point, the interaction force of the sampling point on the neighboring points and the influence factor of the neighboring points on the sampling point, the enhanced learning of the neighborhood spatial relationship of the sampling center point is realized; Step 4.2: Neighborhood spatial structure feature encoding, local feature encoding is achieved through the spatial Euclidean distance between the sampling point and the neighborhood point, direction vector and normal vector information.

6. The noise-resistant bidirectional attention point cloud registration method according to claim 5, characterized in that: The process of obtaining the annealing parameters of the point cloud dataset first requires sending the features extracted by the local bidirectional attention module into the fully connected network to estimate the parameter values ​​from the feature information; secondly, the estimated annealing parameters α and β are sent to the SoftPlus activation function for positive processing.

7. The noise-resistant bidirectional attention point cloud registration method according to claim 6, characterized in that: The two point cloud features and annealing parameters extracted and estimated in steps 4 and 5 use the Sinkhorn algorithm to solve the optimal distance of the double random restriction matrix. The initial state of the matrix is ​​as follows: Where F xj and F yk Represent the characteristics of two point clouds respectively; since each point x in the point cloud X j There is not always a corresponding point y in the point cloud Y k , it is necessary to add a row and a column to the jth row and kth column of the M matrix to satisfy the iteration condition of the double random restriction matrix; for each point x in the point cloud X j , construct the point y corresponding to the tone j , and its solution is shown as follows: Get a one-to-one correspondence (x j ,y j ) After that, the rotation matrix R and translation matrix t can be solved by singular value decomposition.

Images