Point cloud registration system and method based on cross-offset features and spatial consistency

Abstract

The present invention discloses a point cloud registration system and method based on cross-offset features and spatial consistency, and relates to the technical field of point cloud registration. The system is a network architecture, which consists of a feature extraction module, a key point screening module, a matching module, and a confidence estimation module. In the network architecture, P and Q are respectively represented by the source point cloud and the target point cloud, which are sent into the registration network to obtain the final rotation matrix and translation vector; the feature extraction module extracts the feature descriptors of the source point cloud and the target point cloud through a shared graph feature extraction network. Through a series of improvements, the present invention solves the problem of selecting the most representative point pairs from a large number of noisy points and generating feature descriptors with powerful representation capabilities; through the offset attention mechanism, the two point clouds are cross-learned from each other's key information, completing the screening of key points while providing initial point pair information for subsequent matching tasks.

Description

Technical Field

[0001] The present invention belongs to the technical field of point cloud registration, and in particular relates to a point cloud registration system and method based on cross-offset features and spatial consistency. Background Art

[0002] With the development of sensors such as LiDAR, stereo cameras and structured light, the acquisition of point cloud data has been greatly facilitated. Point cloud registration has also attracted widespread attention in the fields of computer vision and robotics. The goal of point cloud registration is to achieve the rigid transformation relationship between the source point cloud and the target point cloud, that is, to align the source point cloud with the target point cloud by solving the rotation matrix and the balance vector. In traditional methods, the correspondence and transformation between point clouds are estimated in an iterative manner based on the optimization framework, and finally the optimal transformation matrix is ​​calculated and output based on the obtained matching point pairs. Among them, the most widely used is ICP (Iterative Closest Point Algorithm). In deep learning-based methods, it is divided into point pair-related and point pair-independent registration. Point pair-independent registration is based on the difference between the global features of two point clouds. Regression of rigid transformation parameters; point pair related matching criteria is to establish a preliminary correspondence based on the similarity of the two point cloud descriptors after obtaining the local descriptors of the feature points, and use geometric constraints to eliminate mismatches, and calculate the transformation parameters with the final matching subset. The most representative of these is the end-to-end splicing network DeepVCP; the point pair related registration algorithm in deep learning can theoretically be applied to point clouds in all situations, including point clouds with low repetition rates, and its core lies in the selection and description of feature points; however, the existing feature point extraction and description algorithms do not yet have the ability to process local features and ignore the linear matching inside the point cloud. In addition, the large-scale calculation and computational complexity of feature point extraction and description will also affect the practicality of the registration algorithm;

[0003] Among the current point cloud registration methods based on deep learning, PointNet can operate directly on the point cloud and use a simple permutation invariant pooling operation to aggregate the information of a single point. This convenient operation makes it very popular in the point cloud registration task, but this method tends to ignore the local information of the point cloud; DGCNN constructs a connection graph between the point cloud and its domain, uses a graph neural network for feature extraction, and uses similarity convolution to measure the point pair matching relationship, and finally completes the registration task; despite this, there are still problems such as insufficient description ability and difficulty in providing more reliable matching information; although theoretically measuring the similarity of the features of two point clouds can obtain matching point pairs, due to the interference of the scene and the limitations of the equipment, there will still be a large number of mismatched point pairs; the current learning-based elimination method mainly treats the elimination of outliers as a classification task, such as DGR and 3Dreg use sparse convolution and point-by-point MLP to classify the correspondence, but they ignore the rigid properties of the point cloud space; for this reason, we designed a point cloud registration system and method based on cross-offset features and spatial consistency. Summary of the invention

[0004] The object of the present invention is to provide a point cloud registration system and method based on cross-offset features and spatial consistency to solve the problems raised in the above background technology.

[0005] To solve the above technical problems, the present invention is achieved through the following technical solutions:

[0006] The present invention is a point cloud registration system based on cross-offset features and spatial consistency. The system is a network architecture, which consists of a feature extraction module, a key point screening module, a matching module and a confidence estimation module;

[0007] The network architecture allows P and Q to represent the source point cloud and the target point cloud respectively, and is sent into the registration network to obtain the final rotation matrix and translation vector;

[0008] The feature extraction module extracts feature descriptors of the source point cloud and the target point cloud through a shared graph feature extraction network;

[0009] The key point screening module mines the structural information of the point cloud through the offset attention mechanism, and fuses the point cloud features from different perspectives in a complementary manner, thereby improving the quality of the simplified points;

[0010] The matching module matches the key points after screening them;

[0011] The confidence estimation module is used to obtain initial matching point pairs and screen the initial matching set.

[0012] A point cloud registration method based on cross-offset features and spatial consistency is used for the above item, and the steps are as follows:

[0013] Through the network architecture, let the source point cloud be P and the target point cloud be Q;

[0014] Through the feature extraction module, the feature descriptor is obtained

[0015] Let the initial feature F of the source point cloud be P and the initial feature F of the target point cloud Q Learn each other's structural information respectively, and then obtain the source point cloud learning feature F' P And the target point cloud learning feature F' Q ;

[0016] By screening the key points, we can get the key point source cloud p = {x 1 ,x 2 ,…,x m} and target point cloud q = {y 1 ,y 2,…,y m} and the corresponding point source point cloud feature set f' P And the target point cloud feature set f' Q , and through the point source point cloud feature set f' P And the target point cloud feature set f' Q and component feature description T (i,j) ;

[0017] Use two-dimensional convolution to obtain the similarity score corresponding to each point and obtain the final similarity matrix S(i,j);

[0018] Take the maximum value of each row of S to obtain the initial matching pair C and construct the length consistency matrix β ij ,

[0019] Then, the compatibility score is adjusted by introducing the feature similarity matrix of the initial matching point pair to obtain the feature compatibility matrix γ ij ;

[0020] Get the compatibility matrix M ij ;

[0021] Get the validity score v between the point pairs i The weight of the effectiveness score w i ;

[0022] By weighting the remaining scores proportionally to the effectiveness score, the objective function R,t,is obtained;

[0023] Generate the centroid of the source point cloud by adding weight parameters to the center points of the two point clouds and the target point cloud centroid

[0024] The H matrix is ​​obtained by calculating the displacement vector;

[0025] Decompose the H matrix by singular value decomposition SVD, and obtain the rotation matrix R based on the matrix U and the matrix V, and then calculate the translation matrix t from the rotation matrix R;

[0026] The key point detection loss function of the source point cloud P and the target point cloud Q is obtained by calculation and Matching loss function And the confidence estimation loss function And through the key point detection loss function and Matching loss function And the confidence estimation loss function The sum of is the loss function Loss.

[0027] Furthermore, the feature descriptor The calculation formula is:

[0028]

[0029] in, Represents point p i The set of k points in the neighborhood, Indicates p i Feature descriptors under the lth layer of graph convolutional network, h and g represent multi-layer perceptrons used for feature learning, Indicates that the aggregation operation takes the maximum value, the initial feature is the 3D coordinate value of the point cloud.

[0030] Furthermore, the source point cloud learning feature F' P And the target point cloud learning feature F' Q The calculation formulas are:

[0031] F′ P =F p +e(F Q )

[0032] F′ Q =F Q +e(F P )

[0033] Among them, e:R N×K →R N×K represents the offset attention mechanism, K is the feature dimension of the two point clouds, and F' P and F' Q Represent the learned features of the source point cloud and the target point cloud respectively.

[0034] Furthermore, the characteristic description T (i,j) The expression formula

[0035]

[0036] Where “;” indicates the serial interpretation formula, T (i,j) It represents the joint information encoding the geometric features and relative positions of two point clouds.

[0037] Furthermore, the expression of the initial matching pair C is:

[0038]

[0039] The length consistency matrix β ij The expression is:

[0040]

[0041] where β represents the length compatibility matrix based on spatial consistency, t dAs a distance parameter to control the sensitivity to length differences.

[0042] Furthermore, the compatibility matrix γ ij The expression is:

[0043]

[0044] where α ij Indicates the similarity of the initial matching points to the features. The smaller the feature difference, α ij The smaller the value, the f It indicates that the feature parameters are used to control the sensitivity to feature differences;

[0045] The compatibility matrix M ij The expression is:

[0046] M ij =β ij *γ ij

[0047] The validity score v between the point pairs i and the weight of the validity score w i Respectively expressed as:

[0048] v i =σ(g(M ij ))

[0049]

[0050] Where σ represents the sigmoid function, g is a multi-layer perceptron whose purpose is to learn the validity scores between matching point pairs based on the compatibility scores between the point pairs;

[0051] The expression of the objective function R,t is:

[0052]

[0053] Furthermore, the source point cloud centroid and the target point cloud centroid The expressions are:

[0054]

[0055]

[0056] The H matrix formula is as follows:

[0057]

[0058] The H matrix decomposition formula is: UΣV T =H;

[0059] The expression of the rotation matrix R is: R = VU T ;

[0060] The expression formula of the translation matrix t is:

[0061] Furthermore, the key point detection loss function and Respectively expressed as:

[0062]

[0063]

[0064] Among them, s(i) represents the importance score of the i-th point in the source point cloud, and s(j) represents the importance score of the j-th point in the target point cloud;

[0065] The matching loss function The expression is:

[0066]

[0067]

[0068] Among them, j * is the index of the corresponding point in the target point cloud, which is the point closest to the source point cloud after conversion; r is a hyperparameter to control the distance between two points to be small enough;

[0069] The confidence estimation loss function The expression is:

[0070]

[0071] in: Mainly used to monitor the length consistency matrix and feature similarity matrix of the initial matching point pairs; The probability of matching point pairs as corresponding relationships is used as a signal to supervise the effectiveness score, which is expressed as follows:

[0072]

[0073]

[0074] The present invention has the following beneficial effects:

[0075] Through a series of improvements, the present invention solves the problem of selecting the most representative point pairs from a large number of noisy points and generating feature descriptors with powerful characterization capabilities; through the offset attention mechanism, the two point clouds are allowed to cross-learn each other's key information, completing the screening of key points while providing initial point pair information for subsequent matching tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0077] Figure 1 This is a schematic diagram of the point cloud registration network structure of the present invention;

[0078] Figure 2 It is a schematic diagram of the spatial consistency of the point cloud of the present invention;

[0079] Figure 3 This is the point cloud registration effect diagram of the present invention;

[0080] Figure 4 It is a schematic diagram of the key point screening of the present invention;

[0081] Figure 5 It is a schematic diagram of eliminating mismatched point pairs of the present invention. DETAILED DESCRIPTION

[0082] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0083] See also Figure 1-5 As shown, the present invention is a point cloud registration system and method based on cross-offset features and spatial consistency.

[0084] Embodiment 1:

[0085] The present invention is a point cloud registration system based on cross-offset features and spatial consistency.

[0086] This system is a network architecture, consisting of a feature extraction module, a key point screening module, a matching module, and a confidence estimation module. For details, please refer to Figure 1 ;

[0087] Among them, the network architecture allows P and Q to represent the source point cloud and the target point cloud respectively, which are sent into the registration network to obtain the final rotation matrix and translation vector;

[0088] Among them, the feature extraction module extracts feature descriptors of the source point cloud and the target point cloud through a shared graph feature extraction network;

[0089] Among them, the key point screening module mines the structural information of the point cloud through the offset attention mechanism, and fuses the point cloud features from different perspectives in a complementary way, thereby improving the quality of the streamlined points;

[0090] Among them, the matching module matches the key points after screening them;

[0091] The confidence estimation module is used to obtain initial matching point pairs and filter the initial matching set.

[0092] Embodiment 2:

[0093] A point cloud registration method based on cross-offset features and spatial consistency, used in the above-mentioned embodiment 1;

[0094] 1. Network architecture

[0095] It is used to let P and Q represent the source point cloud and the target point cloud respectively and send them into the registration network to obtain the final rotation matrix and translation vector; first, the shared feature extraction network GNN is used to obtain the initial feature descriptors of P and Q, and then the initial Euclidean distance features and geometric feature information features of the two point clouds are combined and sent to the cross feature filtering module; this module uses the offset attention mechanism to make the two point clouds cross-learn each other's features to obtain high-level semantic information, and uses this information to filter out key points and key point feature descriptors; then the key points and their feature descriptors will be sent to the matching module, and a similarity convolution will be used to evaluate the similarity of the key points and generate initial matching point pairs; then the confidence estimation module will use the feature similarity matrix of the matching point pairs and combine the spatial consistency of the point clouds to generate a confidence score to judge the correctness of the point pair matching relationship, and finally obtain the transformation matrix through the weighted SVD algorithm;

[0096] Feature extraction module

[0097] Through the shared graph feature extraction network, the feature descriptors of the source point cloud and the target point cloud are extracted; taking the source point cloud P as an example, Represents point p i The set of k points in the neighborhood, Indicates p i For the feature descriptor under the lth layer of graph convolutional network, the formula for calculating the features of the next layer is as follows:

[0098]

[0099] Where h and g represent multi-layer perceptrons used for feature learning, Indicates that the aggregation operation takes the maximum value, the initial feature The three-dimensional coordinate value of the point cloud is superimposed by five layers, and the high-level features are extracted layer by layer. The output dimension of the final feature is 64

[0100] Key point screening module

[0101] The offset attention mechanism under the attention mechanism is used to mine the structural information of the point cloud, and the point cloud features from different perspectives are fused in a complementary way to improve the quality of the streamlined points. The Euclidean distance feature of the point and the offset attention mechanism are connected in series to complete the feature fusion, where the Euclidean distance is a metric, that is, the real distance between two points, and the Euclidean distance feature represents the geometric features of the point.

[0102] However, concatenation will make the feature dimension redundant, which makes it difficult to refine the useful information in the sub-feature set. Therefore, a cross-feature learning module is designed so that the two point clouds can learn each other's structural information. The specific formula is as follows:

[0103] F′ P =F p +e(F Q )

[0104] F′ Q =F Q +e(F P )

[0105] Let F P and F Q Represents the initial feature descriptors of the source point cloud and the target point cloud, e:R N×K →R N×K represents the offset attention mechanism, K is the feature dimension of the two point clouds, and F' P and F' Q Represent the learned features of the source point cloud and the target point cloud respectively; then use a multi-layer perceptron h:e:R N×K →R N×1 , obtain the importance score of each point; the higher the score means the more obvious the feature of the point (such as a corner point), the more likely it is to be selected as a key point; finally, the source point cloud P and the target point cloud Q will retain M points for subsequent registration tasks, where M = N / 6.

[0106] Matching module

[0107] After screening the key points, the key point source point cloud p = {x 1 ,x 2 ,…,x m}, target point cloud q = {y1 ,y 2 ,…,y m}, and the corresponding two feature sets The feature descriptor is constructed using the matching strategy of IDAM, as shown below:

[0108]

[0109] Where “;” indicates the serial interpretation formula, T (i,j) It represents the joint information of the geometric features and relative positions of the two point clouds. Then we use a series of 1*1 two-dimensional convolutions to obtain the similarity score of each point pair, and then use the Softmax function (activation function) for each row to obtain the final similarity matrix S(i,j), which represents y j With x i How likely is it to be a matching point pair, where this series of 1*1 convolutions learn their weight parameters through matching loss

[0110] 5. Confidence Estimation Module

[0111] The initial matching point pairs are obtained. However, due to the interference of partial overlap, data noise and other problems, there are often a large number of mismatched point pairs in the initial matching point set. The next important task is to filter out high-quality matching point sets from the initial matching set. Therefore, the present invention filters the initial matching set based on the spatial consistency method; the maximum value of each row of S is taken to obtain the initial matching pair Construct the compatibility matrix M of the subset; first construct the length consistency matrix, which is shown as follows:

[0112]

[0113] Where β represents the length compatibility matrix based on spatial consistency, and the present invention uses t d as a distance parameter to control the sensitivity to length differences, [·] + Used to ensure that the compatibility score is a number greater than or equal to 0;

[0114] But if Figure 2 (a) (A, B) is the length between the inline points that conforms to spatial consistency, such as Figure 2 (b) (D) is an outlier point, which does not meet the length consistency requirement with any other point, but Figure 2 Although (C) in (a) is an outlier, (B, C) satisfy the length consistency, while (C, D) does not satisfy the length consistency, so there is often interference information;

[0115] From the above, we can see that when both are inline points, length consistency will be satisfied. If they are outliers, length consistency usually cannot be satisfied. However, when inline points and outliers satisfy length consistency, ambiguous information will appear. Therefore, we introduce the feature similarity matrix of the initial matching point pair obtained by matching the point pairs in the matching module to adjust the compatibility score. The specific formula is as follows:

[0116]

[0117] where γ represents the feature compatibility matrix, α ij Indicates the similarity of the features of the initial matching points. The smaller the feature difference, the smaller the value. f It indicates that the feature parameters are used to control the sensitivity to feature differences;

[0118] The formula for obtaining the compatibility matrix is ​​as follows:

[0119] M ij =β ij *γ ij

[0120] Finally, based on the compatibility matrix, we can obtain the validity score between the point pairs. The formula is as follows:

[0121] v i =σ(g(M ij ))

[0122] Where σ represents the sigmoid function, g is a multi-layer perceptron whose purpose is to learn the validity scores of matching point pairs based on the compatibility scores between the point pairs and obtain the weights w of the validity scores between the matching point pairs. i ;

[0123]

[0124] Where 1[·] represents the indicator function. If the validity score of the current point is less than the threshold, the point is assigned a weight of 0 (eliminating the mismatched point pair), and the remaining scores are weighted proportionally according to the validity score, resulting in the following objective function:

[0125]

[0126] 6. Transformation Matrix Estimation Module

[0127] For two partially overlapping point clouds, the point pairs are not always one-to-one corresponding, and there may be a one-to-many situation. Therefore, when using SVD to calculate the rigid transformation parameters, the present invention introduces a weight score to judge the confidence level of each corresponding relationship. The steps are as follows:

[0128] (1) Finding the centroids of the two point clouds respectively: For the center points of the two point clouds, the present invention uses weight parameters to weight the centroids of the source point cloud and the target point cloud. The specific formula is as follows:

[0129]

[0130]

[0131] (2) Obtaining the H matrix: Calculate the displacement vector of each point relative to the center of mass, that is, the distance with direction between each point and the center of mass, and use this vector to calculate the H matrix. The specific formula is as follows:

[0132]

[0133] (3) Use singular value decomposition (SVD) to decompose the H matrix, and obtain the rotation matrix R based on the matrix U and matrix V in the singular value decomposition matrix. Then calculate the translation matrix t from the rotation matrix. The specific formula is as follows:

[0134] UΣV T =H

[0135] R=VU T

[0136]

[0137] 7. Loss Function

[0138] The loss function is an indispensable part in machine learning tasks, and different loss functions should be selected for different problems. For each module of the present invention, it is mainly composed of the following three types of loss functions, namely, the key point detection loss function of two point clouds and Matching loss function And the confidence estimation loss function The specific formula is as follows:

[0139]

[0140] 7.1 Keypoint Detection Loss Function

[0141] This loss is used to obtain the key points in the two point clouds. The negative entropy of the probability distribution of the i-th row (j-th column) is regarded as a supervisory signal for the importance score. If it is a key point, then there is greater confidence in the matching point pair. At this time, the probability distribution defined by the i-th row (j-th column) should have low entropy. On the contrary, if it is a non-key point, it will have high entropy. The specific formula is as follows:

[0142]

[0143]

[0144] Where s(i) represents the importance score of the i-th point in the source point cloud, and s(j) represents the importance score of the j-th point in the target point cloud. At this point, we hope that the cross-feature filtering module can learn to filter out key point pairs based on the shape information of the point cloud.

[0145] 7.2 Matching Loss Function

[0146] This loss function is used to supervise similarity convolution. It is a standard cross entropy loss function. The specific formula is as follows:

[0147]

[0148]

[0149] where j * is the index of the corresponding point in the target point cloud, which is the point closest to the source point cloud after the conversion; r is a hyperparameter to control the distance between two points to be small enough only when x i and The similarity score between matching point pairs is credible only when the distance is within the given threshold.

[0150] 7.3 Confidence Estimation Loss Function

[0151] The functions in this section are used to remove mismatched point pairs. and The specific formula is as follows:

[0152]

[0153] Where λ = 0.01, Mainly used to monitor the length consistency matrix and feature similarity matrix of the initial matching point pairs; The probability of matching point pairs as corresponding relationships is used as a signal to supervise the effectiveness score. The specific formula is as follows:

[0154]

[0155]

[0156] This loss controls the compatibility score of the initial matching point pairs and creates an annealing schedule during training;

[0157]

[0158] This loss assigns a positive label of 1 to points whose correspondence is found correctly, and a negative label of 0 to points whose correspondence is not found correctly; in the long run, point pairs with a high probability of correct matching will have a higher validity score;

[0159] 8. Experimental Verification

[0160] 1. Experimental results and analysis

[0161] The present invention uses the ModelNet40 public data set for training and testing. For a given shape in the data set, it is randomly rotated [0°, 45°] and translated [-0.5, 0.5]; the original point cloud is used as the source point cloud, and the transformed point cloud is used as the template point cloud; in order to generate the problem of partial overlap, a random point is fixed, which keeps the farthest distance from both point clouds, and the 768 points closest to the farthest point are retained for each point cloud; at the same time, the present invention uses the following indicators for evaluation: for the rotation matrix, the root mean square rotation error (RMSE(R)) and the mean absolute rotation error (MAE(R)) are used in degrees; for the translation vector, the root mean square translation error (RMSE(t)) and the mean absolute translation error (MAE(t)) are used; the present invention is compared with ICP, GO-ICP, FGR, FPFH+RANSAC, PointNetLK, DCP, PRNet and IDAM, and the following experiments prove the performance and efficiency of the method of the present invention:

[0162] First, the present invention trains the model on the training set of ModelNet40 and evaluates it on the test set; both the training set and the test set of ModelNet40 contain point clouds from all 40 categories; this experiment evaluates the ability of registration when the shape of the point cloud is invisible; the results are shown in Table 1;

[0163] Table 1 Testing of point clouds of invisible shapes in ModelNet40

[0164]

[0165]

[0166] According to Table 1, we can see that ICP has poor registration performance, while our method achieves the best results in terms of both rotation error and translation error under root mean square, and is second only to the recent IDAM in terms of mean absolute rotation error and mean translation error;

[0167] In the second experiment, the present invention uses the first 20 categories in the ModelNet40 training set for training and evaluates the other 20 categories in the test set; this experiment tests the ability to generalize point clouds of unknown categories; the results are shown in Table 2; it can be seen that without training on the test categories, all learning-based methods will always perform poorly, while the traditional methods are not affected as expected; the present invention achieves the best results in terms of rotation error and mean error under root mean square, while FPFH+IDAM performs best in terms of mean absolute rotation error and mean translation error;

[0168] Table 2 Point cloud test results of unknown categories in ModelNet40

[0169]

[0170]

[0171] In the third experiment, random Gaussian noise with a standard deviation of 0.01 was added, and then the first experiment (invisible shapes) was repeated; the random noise was clipped to [-0.05, 0.05]; As shown in Table 3, the performance of the traditional method and IDAM based on FPFH features is much worse than that of the noise-free case, while our method and GNN+IDAM have their own advantages in rotation error and translation error, but overall, the two algorithms have the best performance among all the compared algorithms;

[0172] Table 3 Results of testing point clouds of invisible shapes in ModelNet40 with Gaussian noise

[0173]

[0174]

[0175] 2. Visualization effect analysis

[0176] The present invention visualizes the contribution of the registration task, and the results are as follows:

[0177] First, the present invention is trained on the ModelNet40 training set, and the model results on the test set are visualized as follows: Figure 3 , the first and third rows respectively show the initial positions of different objects in the data set, and the second and fourth rows show the registration results; in general, the registration of the present invention achieves better results;

[0178] Secondly, Figure 4 The visualization effect of 128 key points filtered out from 1024 points is shown;

[0179] at last Figure 5 The elimination of mismatched point pairs is visualized; this demonstrates the effectiveness of our work in eliminating low-quality point pairs using confidence scores.

[0180] 9. Summary

[0181] The present invention specifically includes the following:

[0182] (1) The present invention proposes a robust point cloud registration network, which solves the problem of selecting the most representative point pairs from a large number of noisy points and generating feature descriptors with powerful characterization capabilities. The present invention uses an offset attention mechanism to allow two point clouds to cross-learn each other's key information, complete the screening of key points, and provide initial point pair information for subsequent matching tasks. In addition, due to partial overlap of data, there will be a large number of false matches in the initial matching set. The present invention uses the spatial consistency relationship of the points in the subset and the similarity of the features to construct a compatibility matrix of the subset, and then uses a learning method to use a neural network to generate a weight parameter for the correspondence of each point to indicate whether the current point pair correspondence is standardized.

[0183] (2) The present invention proposes a cross-feature based on the offset attention mechanism, which allows two point clouds to learn each other's structural information, thereby filtering out key points and providing more stable information decoding.

[0184] (3) The present invention proposes a classification algorithm for mismatched point pairs based on spatial consistency. The compatibility matrix is ​​constructed in a learning manner to obtain the validity scores between point pairs, thereby eliminating mismatched point pairs. This method is robust to noise and outliers.

[0185] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0186] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the present invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A point cloud registration method based on cross-offset features and spatial consistency, characterized in that: Here are the steps: Through the network architecture, let the source point cloud be P and the target point cloud be Q; Through the feature extraction module, the feature descriptor is obtained Let the initial feature F of the source point cloud be P And the initial feature F of the target point cloud Q Learn each other's structural information respectively, and then obtain the source point cloud learning feature F ′ P And the target point cloud learns feature F ′ Q ; By screening the key points, we can get the key point source cloud p = {x1, x2, ..., x m } and target point cloud q={y1,y2,…,y m } and the corresponding point source point cloud feature set f P ′ And the target point cloud feature set f ′ Q , and through the point source point cloud feature set f P ′ And the target point cloud feature set f ′ Q and component feature description T (i,j) ; Use two-dimensional convolution to obtain the similarity score corresponding to each point and obtain the final similarity matrix S(i,j); Take the maximum value of each row of S to obtain the initial matching pair C and construct the length consistency matrix β ij , Then, the compatibility score is adjusted by introducing the feature similarity matrix of the initial matching point pair to obtain the feature compatibility matrix γ ij ; Get the compatibility matrix M ij ; Get the validity score v between the point pairs i The weight of the effectiveness score w i ; By weighting the remaining scores proportionally to the effectiveness score, the objective function R,t,is obtained; Generate the centroid of the source point cloud by adding weight parameters to the center points of the two point clouds and the target point cloud centroid The H matrix is ​​obtained by calculating the displacement vector; Decompose the H matrix by singular value decomposition SVD, and obtain the rotation matrix R based on the matrix U and the matrix V, and then calculate the translation matrix t from the rotation matrix R; The key point detection loss function of the source point cloud P and the target point cloud Q is obtained by calculation and Matching loss function And the confidence estimation loss function And through the key point detection loss function and Matching loss function And the confidence estimation loss function The sum of gets the loss function Loss; The source point cloud learns the feature F ′ P And the target point cloud learns feature F ′ Q The calculation formulas are: F ′ P =F p +e(F Q ) F ′ Q =F Q +e(F P ) Among them, e:R N×K →R N×K represents the offset attention mechanism, K is the feature dimension of the two point clouds, and F ′ P and F ′ Q Represent the learned features of the source point cloud and the target point cloud respectively.

2. The point cloud registration method based on cross-offset features and spatial consistency according to claim 1, characterized in that: The feature descriptor The calculation formula is: in, Represents point p i The set of k points in the neighborhood, Indicates p i Feature descriptors under the lth layer of graph convolutional network, h and g represent multi-layer perceptrons used for feature learning, Indicates that the aggregation operation takes the maximum value, the initial feature is the 3D coordinate value of the point cloud.

3. The point cloud registration method based on cross-offset features and spatial consistency according to claim 1, characterized in that: The characteristics describe T (i,j) The expression formula Where ";" indicates the serial interpretation formula, T (i,j) It represents the joint information encoding the geometric features and relative positions of two point clouds.

4. The point cloud registration method based on cross-offset features and spatial consistency according to claim 1, characterized in that: The expression of the initial matching pair C is: The length consistency matrix β ij The expression is: where β represents the length compatibility matrix based on spatial consistency, t d As a distance parameter to control the sensitivity to length differences.

5. The point cloud registration method based on cross-offset features and spatial consistency according to claim 4, characterized in that: The compatibility matrix γ ij The expression is: where α ij Indicates the similarity of the initial matching points to features. The smaller the feature difference, α ij The smaller the value, the f It indicates that the feature parameters are used to control the sensitivity to feature differences; The compatibility matrix M ij The expression is: M ij =b ij *c ij The validity score v between the point pairs i and the weight of the validity score w i Respectively expressed as: v i =σ(g(M ij )) Where σ represents the sigmoid function, g is a multi-layer perceptron whose purpose is to learn the validity scores between matching point pairs based on the compatibility scores between the point pairs; The expression of the objective function R,t is:

6. The point cloud registration method based on cross-offset features and spatial consistency according to claim 1, characterized in that: The source point cloud centroid and the target point cloud centroid The expressions are: The H matrix formula is as follows: The H matrix decomposition formula is: UΣV T =H; The expression of the rotation matrix R is: R = VU T ; The expression formula of the translation matrix t is:

7. The point cloud registration method based on cross-offset features and spatial consistency according to claim 1, characterized in that: The key point detection loss function and Respectively expressed as: Among them, s(i) represents the importance score of the i-th point in the source point cloud, and s(j) represents the importance score of the j-th point in the target point cloud; The matching loss function The expression is: Among them, j * is the index of the corresponding point in the target point cloud, which is the point closest to the source point cloud after conversion; r is a hyperparameter to control the distance between two points to be small enough; The confidence estimation loss function The expression is: in: Mainly used to monitor the length consistency matrix and feature similarity matrix of the initial matching point pairs; The probability of matching point pairs as corresponding relationships is used as a signal to supervise the effectiveness score, which is expressed as follows:

8. A point cloud registration system based on cross-offset features and spatial consistency, applied to the point cloud registration method based on cross-offset features and spatial consistency according to any one of claims 1 to 7, characterized in that: The system is a network architecture, consisting of a feature extraction module, a key point screening module, a matching module, and a confidence estimation module; The network architecture allows P and Q to represent the source point cloud and the target point cloud respectively, and is sent into the registration network to obtain the final rotation matrix and translation vector; The feature extraction module extracts feature descriptors of the source point cloud and the target point cloud through a shared graph feature extraction network; The key point screening module mines the structural information of the point cloud through the offset attention mechanism, and fuses the point cloud features from different perspectives in a complementary manner, thereby improving the quality of the simplified points; The matching module matches the key points after screening them; The confidence estimation module is used to obtain initial matching point pairs and screen the initial matching set.

Images