A non-rigid point cloud registration method

Abstract

The application discloses a non-rigid point cloud registration method and relates to the technical field of three-dimensional reconstruction. 2 The application further discloses a non-rigid point cloud registration method, which comprises the following steps: standardizing data points of a collected non-rigid point cloud and reference point cloud; calculating a local linear embedding weight matrix L of the data points; calculating a matrix M of the data points according to the matrix L; calculating a Gram matrix G of the data points; calculating a corresponding matrix P of the data points based on the reference point cloud; iteratively calculating a non-rigid transformation coefficient matrix W by using an L-M algorithm based on the matrix G, the matrix M and the matrix P; updating λ and σ in the non-rigid transformation coefficient matrix W according to the number of iterations; calculating the matrix P and the non-rigid transformation coefficient matrix W again; setting an iteration termination condition, that is, the number of iterations reaches a set value or a relative error of a target function value is less than a threshold value; and outputting a non-rigid transformation T based on the matrix G and the non-rigid transformation coefficient matrix W after the iteration is terminated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional reconstruction, and particularly to a non-rigid point cloud registration method. BACKGROUND

[0002] With the rapid development of high-precision sensors such as laser radar, structured light and binocular stereo camera, point cloud has become the main way to characterize the three-dimensional world. Point cloud registration technology estimates the transformation relationship between two point clouds for point cloud registration, and its value lies in its key role in many computer vision applications, including three-dimensional reconstruction, unmanned driving, virtual reality and pose estimation.

[0003] According to the transformation mode between point clouds, registration technology is divided into rigid and non-rigid. Rigid point cloud registration only involves translation and rotation parameters, and is relatively easy to solve; in contrast, non-rigid point cloud is difficult to solve due to its complex mathematical model and a large number of transformation parameters. Non-rigid point cloud registration is essentially an NPC (Non-deterministic Polynomial Complete) combinatorial optimization problem, and the increase in the number of point clouds will cause the feasible solution space to grow exponentially. When there is a large amount of point cloud data, the search cost will become very high; therefore, how to design an effective point cloud registration algorithm is an important scientific problem in the field of computer vision.

[0004] The most effective and widely used method in point cloud registration is the Iterative Closest Point (ICP) algorithm, which can quickly converge to the optimal solution when the transformation between two point clouds is small. In 2003, Chui et al. made an important contribution in the soft matching direction, and adopted a thin plate spline (TPS) as a parameterization of non-rigid spatial transformation. This method can well estimate two-dimensional non-rigid transformation, but for three-dimensional or high-dimensional point clouds, it needs to construct a new thin plate spline model; in 2009, Myronenko et al. proposed a coherent point drift (CPD) method for point cloud registration, which models the point cloud registration problem as a probability density estimation problem. This method can be well extended to high-dimensional point cloud registration.

[0005] These methods mainly utilize the global structure of point clouds for registration, and the distribution characteristics of the neighboring points of each point are not considered. However, for non-rigid shapes, the local structure of the neighboring points is robust and stable. In order to better utilize the local structure of point clouds, Belinge et al. proposed a two-dimensional feature descriptor, called shape context, which describes the statistical distribution of the distances and angles of the neighboring points relative to a reference point to establish better point correspondences. For the three-dimensional case, Radu et al. proposed a computationally efficient three-dimensional feature, Fast Point Feature Histograms (FPFH), to establish a rough registration relationship between point clouds by feature similarity; using only local structure features is prone to local optimization and is affected by noise, in order to utilize global and local structures, Song Ge et al. combined Locally Linear Embedding (LLE) with CPD; Wang et al. and Ma et al. utilize local features to construct an initial correspondence relationship, and then utilize smoothing constraints or manifold regularization for global non-rigid transformation estimation; Huang et al. introduce high-dimensional representation to incorporate local structure information into point coordinates, thereby improving registration robustness and accuracy; the combination of global and local features can combine the smoothness of global constraints and the specificity of local features, however, this kind of method also inherits the inapplicability of global smoothness constraints to rotation and outlier distortion and the sensitivity to noise distortion; in order to improve the robustness and efficiency of the algorithm, Yao et al. proposed a non-rigid point cloud transformation solver based on the quasi-Newton method; Chaudhury decomposes the non-rigid point cloud registration process into three levels for step-by-step solving, thereby improving the robustness of the algorithm; Hirose introduces variational Bayesian inference to ensure the convergence of the algorithm. These methods are based on smoothness constraint assumptions or utilize local structure information to establish a registration model, and the robustness and efficiency of the algorithm are guaranteed by introducing different optimization algorithms, therefore, this kind of algorithm still cannot well handle rotation, outliers and noise distortion.

[0006] In 2014, Song et al. proposed a non-rigid point cloud registration method based on global and local topology preservation, which includes data registration error, global transformation consistency and local transformation consistency in the objective function, and then utilizes the EM (Expectation Maximization) algorithm for solving. Since the objective function contains the global transformation consistency constraint, its robustness to rotation and outlier distortion is greatly reduced, although the subsequent local transformation consistency constraint can improve the robustness of the algorithm, under the previous global transformation consistency constraint, the subsequent local transformation consistency constraint can only improve the registration effect of the algorithm under rotation and outlier distortion, and cannot well solve the problem. At the same time, based on the local transformation consistency constraint, its robustness to noise and outliers is limited.

[0007] One application of the non-rigid point cloud registration method is pose estimation in robot arm path planning: aligning one point cloud (real-time view) with another point cloud (environmental corresponding map), the pose information of point cloud X 0 relative to point cloud Y 0 can be generated, which can be used for robot decision-making. For example, the pose information of the robot arm can be obtained to determine how to move to accurately grasp the target.

[0008] In summary, for the distorted point cloud containing rotation, outliers and noise, the existing non-rigid point cloud registration method cannot guarantee to obtain the correct solution. SUMMARY

[0009] Therefore, it is of great significance to study a robust non-rigid point cloud registration algorithm that can handle various distortions. The non-rigid point cloud registration method proposed in the present application removes the global transformation consistency constraint, and the objective function only contains the registration error and the local linear embedding consistency constraint. This constraint is not affected by rotation, translation and scaling, and can better depict the deformation of the non-rigid point cloud itself, and is not affected by the rotation and translation between point clouds. For model solving, the L-M (Levenberg-Marquardt) method is added for solving, so as to ensure the stability of the algorithm.

[0010] The non-rigid point cloud registration method proposed in the present application specifically comprises the following steps:

[0011] The data point cloud of the collected robot arm pose information and the reference point cloud are respectively standardized;

[0012] The linear reconstruction coefficients of the neighboring points of the data point cloud are calculated, the local linear embedding weight matrix L of the data point cloud is calculated by combining the minimum cost function method, and the reconstruction coefficient matrix M of the data point cloud is calculated according to the matrix L, M=(I-L)(I-L) T ;

[0013] The Gram matrix G of the data point cloud is calculated;

[0014] Taking the reference point cloud as the reference, the corresponding probability matrix P of the data point cloud is calculated;

[0015] Based on the matrix G, the matrix M and the matrix P, the L-M algorithm is used to iteratively calculate the non-rigid transformation coefficient matrix W;

[0016] According to the number of iterations, the parameters in the non-rigid transformation coefficient matrix W are updated, and the matrix P and the non-rigid transformation coefficient matrix W are calculated again according to the updated parameters;

[0017] Set the iteration termination condition: the number of iterations reaches a set value or the relative error of the objective function value is less than a threshold;

[0018] After the iteration terminates, based on matrix G and the non-rigid transformation coefficient matrix W, the non-rigid transformation T is output, and the accuracy of the robot arm posture information is judged based on the non-rigid transformation T.

[0019] Furthermore, the standardization processing of the collected robot arm pose information data point cloud and reference point cloud specifically includes:

[0020] The original point clouds collected are as follows: Data Point Cloud A real-time view of the robot arm's pose, reference point cloud. This is a map representing the environment of the robotic arm, where D represents the point cloud dimension, M represents the number of data points, and N represents the number of reference points; the two point clouds after standardization are: data point cloud Y... M×D =(y1,...,y M ) T and reference point cloud X N×D =(x1,...,x N ) T ,in, and σ represents the average value of the original point cloud and the reference point cloud, respectively. y and σ x These are the mean square errors of the original point cloud and the reference point cloud, respectively.

[0021] Furthermore, the locally linear embedding weight matrix L of the computational data point cloud specifically includes:

[0022] Through each data point y m The linear reconstruction coefficients of neighboring points for each data point y m To characterize the local linear embedding weights, the cost function is minimized. We obtain, where y i Let i = 1, ..., K be the data points y. m The K-nearest point.

[0023] Furthermore, the calculation of the local linear embedding weight matrix L includes the following steps:

[0024] For each data point y m Calculate its K nearest neighbors;

[0025] For each data point y m Calculate the correlation matrix C after centering the K nearest neighbors. m ,in y i and y jare K-nearest neighbors of data point y m .

[0026] compute the inverse of correlation matrix C m (C m ) -1 and add a multiple of identity matrix to get the inverse of correlation matrix, denoted as R m =(C m +ε·trace(C m )·I K×K ) -1 ·I K×1 , then for each data point y m , the weight coefficient L(m,:) = R m / sum(R m ), where C m is the correlation matrix of K-nearest neighbors of any data point y m in the map point cloud, with dimension KxK, ε·trace(C m )·I K×K is a multiple of identity matrix of the same order as C m , ε is a constant, set to 10 -3 , trace(C m ) is the trace of matrix C m , I K×K is the identity matrix with dimension KxK, and I K×1 is the identity matrix with dimension Kx1.

[0027] Further, the Gram matrix G of the data point cloud is calculated using , where β is initialized to 2, y i and y j are K-nearest neighbors of data point y m .

[0028] Further, the corresponding probability matrix P of the data point cloud is calculated using , where D represents the dimension of the point cloud, M represents the number of data points, N represents the number of reference points, x n is any real-time view point cloud coordinate, y m is any map point cloud coordinate, and T(y m ) is the coordinate of y m after non-rigid transformation, i.e., T(y m ) = y m + G(m, ·)W.

[0029] Further,

[0030] (d(P · 1)G + λ1+ ασ 2MG) W = PX - (d(P * 1) + a * s 2 M) Y calculates the non-rigid transformation coefficient matrix W, wherein λ1 is a unit matrix multiple, λ gradually increases with the number of iterations, λ is initialized to 3e-5, λ is increased by 1.6181 times every 10 iterations, and α is initialized to 0.3.

[0031] Further, the parameters in the non-rigid transformation coefficient matrix W are updated, and specifically comprising:

[0032] By using updating the non-rigid transformation coefficient matrix W in σ 2 , wherein N P = 1 T P1, D represents the dimension of the point cloud, exp(·) is the exponential with the natural constant as the base, π is the circular constant, ω is the outlier ratio, which is set to 0, N is the number of view point clouds, and M is the number of map point clouds.

[0033] Further, by using calculating the relative error of the objective function value in the iteration termination condition, wherein T(y m ) = y m + G(m, ·) W;

[0034] The iteration number in the iteration termination condition reaches a set value of 100, and the relative error threshold of the objective function value is 1x10 -10 .

[0035] Further, the non-rigid transformation T is calculated by using T(Y) = Y+GW.

[0036] Compared with the prior art, the non-rigid point cloud registration method provided by the application has the beneficial effects that:

[0037] The non-rigid point cloud registration method provided by the application removes the global transformation consistency constraint, and the objective function only contains the registration error and the local linear embedding consistency constraint, which is not affected by rotation, translation and scaling, can better depict the deformation of the non-rigid point cloud itself, and is not affected by the rotation and translation between point clouds and other transformations, so the method provided in the application can better handle rotation, translation and outliers. For model solving, the L-M (Levenberg-MarquareLt) method is added for solving, which ensures the non-singularity of the coefficient matrix, thereby improving the reliability of the algorithm. BRIEF DESCRIPTION OF DRAWINGS

[0038] Fig. 1 A flowchart of the non-rigid point cloud registration method provided by the application is provided.

[0039] Fig. 2The fish-shaped and Chinese character point cloud registration statistical results in the embodiments of the present application. DETAILED DESCRIPTION

[0040] The specific embodiments of the present application are further described below. The following embodiments are only used to more clearly illustrate the technical solutions of the present application, and cannot be used to limit the protection scope of the present application. Figs. 1-2 The specific embodiments of the present application are further described below. The following embodiments are only used to more clearly illustrate the technical solutions of the present application, and cannot be used to limit the protection scope of the present application.

[0041] Embodiment 1: As shown in the table, the present application proposes a non-rigid point cloud registration method, including the following steps: Figs. 1-2

[0042] 1) Standardize the two collected point clouds respectively;

[0043] Let the collected original point clouds be: data point cloud and reference point cloud Wherein, D represents the dimension of the point cloud, M represents the number of data points, and N represents the number of reference points. The two standardized point clouds are: data point cloud Y M×D =(y1,...,y M ) T and reference point cloud X N×D =(x1,...,x N ) T , wherein, and are the average values of the original data point cloud and the reference point cloud, and σ y and σ x are the mean square deviations of the original data point cloud and the reference point cloud;

[0044] 2) Calculate the local linear embedding weight matrix L M×K ;

[0045] For each data point y m , this paper describes it through the linear reconstruction coefficient of its neighboring points, and the local linear embedding weight can be obtained by minimizing the following cost function: Wherein, y i ,i=1,...,K are K neighboring points of data point y m ;

[0046] The weight matrix L can be obtained by the following three steps:

[0047] 2.1) For each point y m , calculate its K neighboring points;

[0048] 2.2) For each point y m , calculate the correlation matrix C m centered after the K neighboring points, wherein y​i and y j are K-nearest neighbors of data point y m ;

[0049] 2.3) Compute the inverse of the correlation matrix (C m ) -1 Since the inverse of a singular matrix or a matrix with large condition number is unstable, add a multiple of the identity matrix to the correlation matrix before computing its inverse, denoted as R m = (C m + ε·trace(C m )·I) -1 Then for each data point y m , the weight coefficient L(m,:) = R m / sum(R m );

[0050] 3) Compute the non-rigid transformation T of the data point cloud with respect to the reference point cloud;

[0051] The registration between point clouds Y M×D = (y1,...,y M ) T and X N×D = (x1,...,x N ) T is considered as two sub-problems: one is to find the correspondence between two point sets, and the other is to find the transformation parameters using the correspondence between two point sets. In this paper, the EM algorithm is used to solve it, which includes the following steps:

[0052] 3.1) Compute the Gram matrix G of the data point cloud Y M×D = (y1,...,y M ) T , where Initialize β = 2;

[0053] 3.2) Compute the reconstruction coefficient matrix M = (I-L)(I-L) T ;

[0054] 3.3) Iterate until convergence: set the relative error threshold to 1x10 -10 and the maximum number of iterations to 100;

[0055] 3.3.1) Compute the correspondence matrix P, where

[0056] 3.3.2) Compute the non-rigid transformation coefficient matrix W based on the L-M algorithm;

[0057] (d(P·1)G+λ1+ασ 2 MG)W = PX-(d(P·1)+ασ2 M) Y(2), wherein λ1is the prevention of (d(P·1)G+αs 2 MG) singular or condition number too large to introduce the unit matrix multiple, λ gradually increases with the number of iterations, so that the whole calculation process, the algorithm converges faster. Initialize λ=3e-5, every 10 iterations, λ increases by 1.6181 times; initialize α=0.3;

[0058] 3.3.3) update σ 2 ;

[0059] Where N P =1 T P1;

[0060] 3.3.4) determine whether the iteration is terminated: the iteration number reaches 100 times or the relative error of the objective function value is less than the threshold value, then the iteration is terminated, otherwise continue; the relative error is calculated according to formula (4):

[0061] Where T(y m )=y m +G(m,·)W;

[0062] The iteration is terminated and the non-rigid transformation T(Y)=Y+GW is output.

[0063] In the embodiment of the application, the point cloud registration accuracy test is also carried out by using the consistent point drift and the point cloud registration method based on manifold regularization. In this embodiment, two shape point cloud sets (fish shape and Chinese character) generated by Dr. Hai Li Chui and Prof. Anand Rangarajan are used for testing, each distortion test case contains 500 to 600, and a total of 3400 test cases. The hardware configuration is Intel Core i7-10710U, 16GB RAM, and the experimental platform is MatLab 2021b. Fig. 2 The registration results of the application and the other two methods under rotation, outlier and noise three distortions are shown, Fig. 2 The left is the test result of the fish shape point set, and the right is the test result of the Chinese character. From top to bottom are the test results of rotation, outlier and noise three distortions. From Fig. 2 The mean (the midpoint of each vertical line, marked with a separate shape) and variance (the length of the vertical line) of the residual in can be seen: the error of the point cloud registration of the application is generally lower than that of the other two methods, especially when the distortion degree increases, that is, the robustness of the application to the point cloud registration problem with large distortion degree is better.

[0064] The closest solution to the present application is a non-rigid point cloud registration method based on global and local topology preservation proposed by Song et al. in 2014. The model contains three items of data registration error, global transformation consistency and local transformation consistency, and then uses the EM (Expectation Maximization) algorithm for solving. Since the objective function contains the global transformation consistency constraint, its robustness to rotation and outliers is greatly reduced. At the same time, since the algorithm solving process contains large-scale linear equation solving, the singularity of its coefficient matrix is not guaranteed, which leads to the reliability of the algorithm cannot be guaranteed.

[0065] The robust non-rigid point cloud registration method based on local linear embedding proposed in the present application removes the global transformation consistency constraint, and the objective function only contains the registration error and the local linear embedding consistency constraint. This constraint is not affected by rotation, translation and scaling, and can better describe the deformation of the non-rigid point cloud itself, and is not affected by the rotation and translation between point clouds. Therefore, the method proposed in the present application can better handle rotation, translation and outliers. For model solving, the L-M (Levenberg-MarquareLt) method is added for solving, which guarantees the non-singularity of the coefficient matrix, thereby improving the reliability of the algorithm.

[0066] In summary, the non-rigid point cloud registration method proposed in the present application removes the global transformation consistency constraint, and the objective function only contains the registration error and the local linear embedding consistency constraint. This constraint is not affected by rotation, translation and scaling, and can better describe the deformation of the non-rigid point cloud itself, and is not affected by the rotation and translation between point clouds. Therefore, the method proposed in the present application can better handle rotation, translation and outliers. For model solving, the L-M (Levenberg-MarquareLt) method is added for solving, which guarantees the non-singularity of the coefficient matrix, thereby improving the reliability of the algorithm.

[0067] The above-described embodiments are only the preferred specific embodiments of the present application, and the protection scope of the present application is not limited thereto. Any simple changes or equivalent replacements of the technical solutions within the technical range disclosed in the present application, which can be obviously obtained by those skilled in the art, shall belong to the protection scope of the present application.

Claims

1. A non-rigid point cloud registration method, characterized in that, Includes the following steps: The data point cloud and reference point cloud of the collected robot arm posture information are standardized respectively; Calculate the linear reconstruction coefficients of the neighboring points of the data point cloud, and then use the method of minimizing the cost function to calculate the local linear embedding weight matrix of the data point cloud. This includes: through each data point The linear reconstruction coefficients of neighboring points for each data point To characterize the local linear embedding weights, the cost function is minimized. get, , For data points The K nearest neighbors; wherein the local linear embedding weight matrix The calculation process is as follows: for each data point Calculate its K nearest neighbors; for each data point Calculate the correlation matrix after centering the K nearest neighbors. ,in , and All are data points Find the K nearest neighbors; calculate the correlation matrix. inverse matrix Then add a multiple of the identity matrix and find the inverse of the correlation matrix, denoted as . For each data point Its weighting coefficient ,in For any data point in the map point cloud The correlation matrix obtained by calculating the K-nearest neighbors has dimensions of . , Is and Multiples of an identity matrix of the same order It is a constant, set as , For matrix traces, It is a dimension of The identity matrix, Dimensions The identity matrix; Calculate the Gram matrix of the point cloud data ; Using the reference point cloud as a baseline, calculate the corresponding probability matrix of the data point cloud. ; Based on matrix ,matrix sum matrix The non-rigid transformation coefficient matrix is ​​calculated iteratively using the LM algorithm. Including: utilizing Calculate the non-rigid transformation coefficient matrix ,in Multiples of the identity matrix As the number of iterations gradually increases, initialization Every 10 iterations Increased by a factor of 1.6181; Initialization ; Update the non-rigid transformation coefficient matrix according to the number of iterations. The parameters include: using Update the non-rigid transformation coefficient matrix middle ,in , Represents the dimension of the point cloud. It is an exponent with the natural constant as its base. Pi The out-of-point ratio is set to 0. For the number of point clouds in the view, Number of point clouds on the map; Calculate the matrix again based on the updated parameters. Non-rigid transformation coefficient matrix ; Set the iteration termination condition: the number of iterations reaches a set value or the relative error of the objective function value is less than a threshold; After the iteration terminates, based on the matrix Non-rigid transformation coefficient matrix Output non-rigid transformation And based on non-rigid transformation To determine the accuracy of the robot arm's posture information.

2. The non-rigid point cloud registration method as described in claim 1, characterized in that, The standardization process for the collected robot arm pose information data point cloud and reference point cloud includes: The original point clouds collected are as follows: Data Point Cloud A real-time view of the robot arm's pose, reference point cloud. This is a map corresponding to the environment of the robotic arm, where, Represents the dimension of the point cloud. Represents the number of data points. The number of reference points represents the data point cloud; the two point clouds after standardization are: data point cloud and data point cloud. and reference point cloud ,in, , , and These are the average values ​​of the original point cloud and the reference point cloud, respectively. and These are the mean square errors of the original point cloud and the reference point cloud, respectively.

3. The non-rigid point cloud registration method as described in claim 1, characterized in that: use Calculate the Gram matrix of the data point cloud. , where initialization , and All are data points The K-nearest point.

4. The non-rigid point cloud registration method as described in claim 1, characterized in that: use Calculate the corresponding probability matrix of the data point cloud. ,in Represents the dimension of the point cloud. Represents the number of data points. Represents the number of reference points. For any real-time view point cloud coordinates, For any map point cloud coordinates, for The coordinates after non-rigid transformation, i.e. .

5. The non-rigid point cloud registration method as described in claim 1, characterized in that: use Calculate the relative error of the objective function value in the iteration termination condition, where , ; In the iteration termination condition, the number of iterations reaches a set value of 100, and the relative error threshold of the objective function value is [value missing]. .

6. The non-rigid point cloud registration method as described in claim 1, characterized in that: use Calculate the non-rigid transformation .

Images