A multi-view point cloud registration method with minimum constraints

Abstract

The present invention belongs to the technical field related to optical three-dimensional measurement, and discloses a multi-view point cloud registration method with minimized constraints. The method includes the following steps: S1 collects point cloud data, and transforms all collected point cloud data into the same coordinate system, calculates the three-dimensional bounding box of the point cloud, and judges the coincidence relationship between the point cloud data according to the position relationship between the three-dimensional bounding boxes; S2 determines the corresponding point pairs in the coincident point cloud according to the coincidence relationship of the point cloud, constructs a weighted minimization objective function of the distance and variance of the point pairs, and solves the objective function to obtain the posture transformation matrix; S3 uses the posture transformation matrix to correct the point cloud data to obtain the registered point cloud data. Through the present invention, the problem of low point cloud matching accuracy and only being able to achieve point cloud matching from two perspectives is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field related to optical three-dimensional measurement, and more specifically, relates to a multi-view point cloud registration method with minimized constraints. Background Art

[0002] Large industrial parts such as automobile frames and aircraft wings are large in size, ranging from several meters to more than ten meters in length. Limited by the measurement range of 3D sensors, the size calculation and shape detection of large parts mostly rely on robots equipped with 3D sensors to obtain point cloud data from multiple perspectives (dozens to hundreds of pictures). The amount of point cloud data obtained by the sensors is as high as hundreds of millions to billions. The multi-view point clouds can be unified into the same coordinate system through the robot's acquisition posture, but due to the errors in the repeated positioning of industrial robots, the overlapping areas of the multi-view point cloud data will be layered, which requires point cloud precise registration to correct the point cloud posture.

[0003] The current point cloud registration methods mainly include probability-based registration methods, feature-based registration methods, and distance-based registration methods. The probability-based registration method converts the registration problem between two point clouds into a probability density estimation problem. It mainly uses a Gaussian mixture model to establish a pairing relationship between point clouds and solves it through the maximum likelihood method. The feature-based registration method extracts representative features of point clouds to establish a matching relationship between features of multiple point clouds, and then uses the matched relationship to solve the relative pose relationship of point clouds. The features can be manually designed features or features learned through deep learning. The probability-based registration method and the feature-based registration method are fast, but have low accuracy and are not suitable for the registration of measured point clouds of industrial parts with high accuracy requirements. The distance-based registration method optimizes the point cloud pose relationship by establishing a pairing relationship between corresponding points of multiple point clouds and minimizing the point distance. Distance minimization is easily affected by noise, openness, and uneven density in the actual collected point clouds, and is prone to misregistration. Summary of the invention

[0004] In view of the above defects or improvement needs of the prior art, the present invention provides a multi-view point cloud registration method with minimized constraints, which solves the problems of low point cloud matching accuracy and only being able to achieve point cloud matching from two perspectives.

[0005] To achieve the above object, according to the present invention, a multi-view point cloud registration method with minimized constraints is provided, the method comprising the following steps:

[0006] S1 collects point cloud data from multiple perspectives, transforms all collected point cloud data into the same coordinate system, calculates the three-dimensional bounding box of the point cloud, and determines the overlap relationship between the point cloud data based on the positional relationship between the three-dimensional bounding boxes;

[0007] S2 determines the corresponding point pairs in the overlapping point clouds according to the overlapping relationship of the point clouds, constructs the distance and variance weighted minimization objective function of the point pairs, and solves the objective function to obtain the posture transformation matrix;

[0008] S3 uses the posture transformation matrix to correct the point cloud data to obtain the registered point cloud data.

[0009] Further preferably, in step S1, before calculating the three-dimensional bounding box of the point cloud, all point cloud data need to be filtered.

[0010] Further preferably, the filtering method is performed as follows:

[0011] Set the search radius r of the current point, and search for neighboring points within the radius r. If the number of neighboring points is less than the preset threshold K, the current point is an outlier and is removed.

[0012] Further preferably, in step S1, the three-dimensional bounding box is constructed in the following manner:

[0013] The center of the bounding box of the point cloud is O(o x ,o y ,o z ):

[0014]

[0015] Where p = [p x ,p y ,p z ] is any point in a point cloud, X min is the minimum value of the x coordinates of all points, X max is the maximum value of the x coordinates of all points, Y min is the minimum value of the y coordinates of all points, Y max is the maximum value of the y coordinates of all points, Z min is the minimum value of the z coordinates of all points, Z max is the maximum value of the z coordinates of all points, o x ,o y ,o z are the three coordinates of the center of the bounding box, L, W and H are the length, width and height of the bounding box respectively.

[0016] Further preferably, in step S1, the determining of the overlap relationship between the point cloud data is performed in the following manner: if the bounding boxes of two pieces of point cloud data have an intersection in space, the two pieces of point cloud data overlap, otherwise there is no overlap.

[0017] Further preferably, the intersection in the space is performed as follows:

[0018]

[0019] Among them, O i and O j Represent the i-th and j-th bounding boxes respectively, Yes i The x, y, z axis coordinates of the center point, Yes j The center point x, y, z axis coordinates, L i , W i , H i O i Length, width, height, L j , W j , H j O j The length, width and height of the .

[0020] Further preferably, in step S2, the point pairing is performed as follows: searching for the closest point from a point in one point cloud to all points in another point cloud; if the distance between the two points is less than a certain threshold, the two points are considered to constitute a corresponding point pair.

[0021] Further preferably, the search is performed using KD-Tree.

[0022] Further preferably, in step S2, the objective function is performed as follows:

[0023]

[0024] Among them, n is the total number of point clouds, d ijk is the point distance between the kth corresponding point of the i-th point cloud and the j-th point cloud, is the average distance between all corresponding points between the i-th point cloud and the j-th point cloud, μ is a weight, and m is the number of corresponding point pairs sampled between the i-th point cloud and the j-th point cloud.

[0025] Further preferably, in step S2, the objective function is solved by using the Gauss-Newton method.

[0026] In general, the above technical solutions conceived by the present invention have the following beneficial effects compared with the prior art:

[0027] 1. The present invention constructs a weighted objective function of corresponding point spacing and point spacing variance, thereby reducing the abnormal situation that the local area of ​​the point cloud has too much influence on the objective function due to uneven density, avoiding the registration failure caused by uneven density of point cloud data, and significantly reducing the stratification phenomenon of the point cloud after registration;

[0028] 2. The present invention introduces all multi-view point cloud poses into the objective function in the form of display variables in the form of Lie algebra, so that the objective function optimization process can simultaneously optimize all multi-view point cloud poses, avoiding the problem of excessive cumulative error caused by the previous pairwise point cloud registration, and can more effectively converge to an optimal solution, greatly improving the registration accuracy;

[0029] 3. The present invention utilizes the three-dimensional bounding box position of the point cloud to preliminarily determine the overlap relationship between point clouds, which can avoid searching for the nearest point on a large number of point clouds with no overlap area, avoid a large amount of useless calculations, and improve the efficiency of searching for overlap areas in multi-view point clouds. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a flow chart of a multi-view point cloud registration method with minimized constraints constructed according to a preferred embodiment of the present invention;

[0031] Figure 2 is a schematic diagram of a discrete point filtering effect constructed according to a preferred embodiment of the present invention;

[0032] Figure 3 is a schematic diagram of a three-dimensional bounding box constructed according to a preferred embodiment of the present invention;

[0033] Figure 4 is a schematic diagram of determining the coincidence relationship between point clouds constructed according to a preferred embodiment of the present invention, wherein (a) is when the bounding boxes intersect, and (b) is when the bounding boxes do not intersect;

[0034] Figure 5 It is a schematic diagram of coincident point pairs between calculated point clouds constructed according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0036] A multi-view point cloud registration method with minimized constraints, the method comprising the following steps:

[0037] (1) Rough registration of point cloud pose

[0038] During the point cloud data acquisition process, the acquisition posture of the 3D sensor is recorded, and then the collected point cloud data is unified into the same coordinate system through the acquisition posture to complete the coarse alignment of the multi-view point cloud.

[0039] (2) Outlier Filtering

[0040] Outlier filtering is to find any point in the point cloud as the center and find the neighboring points within the radius r. If the number of neighboring points is less than a certain threshold K, it means that the current point is an outlier and should be removed. This can prevent the point cloud from being affected by the noise of outliers during the registration process. The results are as follows Figure 2 shown.

[0041] (3) Calculate the 3D bounding box of the point cloud

[0042] Define the center of the bounding box of the point cloud as O(o x ,o y ,o z ),have:

[0043]

[0044] Where p = [p x ,p y ,p z ] is any point in a point cloud, X min is the minimum value of the x coordinates of all points, X max is the maximum value of the x coordinates of all points, Y min is the minimum value of the y coordinates of all points, Y max is the maximum value of the y coordinates of all points, Z min is the minimum value of the z coordinates of all points, Z max is the maximum value of the z coordinates of all points, o x ,o y ,o z are the three coordinates of the center of the bounding box, L, W and H are the length, width and height of the bounding box respectively. Figure 3 shown.

[0045] (4) Judgment of overlap between point clouds

[0046] like Figure 4 As shown in the figure, if the bounding boxes of two point clouds intersect in space, then it is considered that the two point clouds may overlap, otherwise there must be no overlap. and The formula for determining overlap is as follows:

[0047]

[0048] Among them, O i and O j Represent the i-th and j-th bounding boxes respectively, Yes i The x, y, z axis coordinates of the center point, Yes j The center point x, y, z axis coordinates, L i , Wi , H i O i Length, width, height, L j , W j , H j O j The length, width and height of the .

[0049] (5) Calculation of corresponding point pairs between overlapping point clouds

[0050] The corresponding point pairs between the point clouds that are judged to be overlapping are calculated by finding the nearest point in KD-Tree, finding the nearest point from a point in one point cloud to all the points in another point cloud. If the distance between the two points is less than a certain threshold, the two points are considered to constitute a corresponding point pair, such as Figure 5 shown.

[0051] (6) Constructing the objective function

[0052] Assumptions is the homogeneous transformation matrix from the point cloud i's own coordinate system to the global coordinate system, i = 1, 2, ... n, where R is the rotation matrix and t is the translation vector. Assume that from the point cloud P i With point cloud P j The matching points corresponding to the sample m groups are then point p ijk Refers to P i With P j For the kth matching point, the distance minimization objective function of the matching point is:

[0053]

[0054] where d ijk =T i p ijk -T j p jik , is the distance between the kth group of corresponding points in the i-th point cloud and the j-th point cloud. Similarly, the objective function of minimizing the variance is:

[0055]

[0056] in The distance and variance objective functions are weighted according to the weight of λ to obtain the objective function:

[0057]

[0058] in

[0059] (7) Objective function solution and pose optimization

[0060] The objective function can be transformed into F w (ξ)=||f(ξ)||2 , where ξ satisfies the relation transformation matrix T = exp(ξ^), then

[0061] Then we have:

[0062]

[0063] make Then we have:

[0064] f(ξ)=AD(ξ)

[0065] in

[0066] And because f(ξ+Δξ)=||f(ξ)+J(ξ) T ·Δξ|| 2 , where J(ξ) is the transpose of the first-order derivative of f(ξ), we can get:

[0067]

[0068] Taking the derivative of the above equation and setting it to 0, we get: Gauss-Newton method solution

[0069]

[0070] in , δξ=[δξ1 δξ2… δξ n ]

[0071] By continuously updating ξ through Δξ until the algorithm converges, the optimized point cloud pose is obtained: T'=exp(ξ^).

[0072] (8) Point cloud pose correction

[0073] The optimized point cloud pose is used to correct the original point cloud to obtain the precisely aligned point cloud.

[0074] The correction method is:

[0075] P i ′=T′P i ,

[0076] Where P i is the origin cloud, P i 'It is the point cloud after pose optimization.

[0077] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-view point cloud registration method with minimum constraints, characterized in that: The method comprises the following steps: S1 collects point cloud data from multiple perspectives, transforms all collected point cloud data into the same coordinate system, calculates the three-dimensional bounding box of the point cloud, and determines the overlap relationship between the point cloud data based on the positional relationship between the three-dimensional bounding boxes; S2 determines the corresponding point pairs in the overlapping point clouds according to the overlapping relationship of the point clouds, constructs the distance and variance weighted minimization objective function of the point pairs, and solves the objective function to obtain the posture transformation matrix; The distance and variance weighted minimization objective function of the constructed point pair is as follows: Assumptions is the homogeneous transformation matrix from the coordinate system of point cloud i to the global coordinate system, i = 1, 2, ... n, where R is the rotation matrix and t is the translation vector; Assume that from the point cloud P i With point cloud P j The matching points corresponding to the sampled m groups, point p ijk Refers to P i With P j The kth matching point, then the distance of the matching point minimizes the objective function F d for: Among them, d ijk =T i p ijk -T j p jik , is the distance between the kth group of corresponding points in the i-th point cloud and the j-th point cloud. Similarly, the objective function F that minimizes the variance is v for: in, The distance and variance objective functions are weighted by weight λ to obtain the objective function F w : in, n is the total number of point clouds, d ijk is the point distance between the kth corresponding point of the i-th point cloud and the j-th point cloud, is the average distance between all corresponding points between the i-th point cloud and the j-th point cloud, μ is a weight, and m is the number of corresponding point pairs sampled between the i-th point cloud and the j-th point cloud; S3 uses the posture transformation matrix to correct the point cloud data to obtain the registered point cloud data.

2. A multi-view point cloud registration method with minimized constraints as claimed in claim 1, characterized in that: In step S1, before calculating the three-dimensional bounding box of the point cloud, all point cloud data need to be filtered.

3. A multi-view point cloud registration method with minimized constraints as claimed in claim 2, characterized in that: The filtering method is performed as follows: Set the search radius r of the current point, and search for neighboring points within the radius r. If the number of neighboring points is less than the preset threshold K, the current point is an outlier and is removed.

4. A multi-view point cloud registration method with minimized constraints as claimed in claim 1 or 2, characterized in that: In step S1, the three-dimensional bounding box is constructed in the following manner: The center of the bounding box of the point cloud is O(o x ,o y ,o z ): Where p = [p x ,p y ,p z ] is any point in a point cloud, X min is the minimum value of the x coordinates of all points, X max is the maximum value of the x coordinates of all points, Y min is the minimum value of the y coordinates of all points, Y max is the maximum value of the y coordinates of all points, Z min is the minimum value of the z coordinates of all points, Z max is the maximum value of the z coordinates of all points, o x ,o y ,o z are the three coordinates of the center of the bounding box, L, W and H are the length, width and height of the bounding box respectively.

5. The multi-view point cloud registration method with minimized constraints as claimed in claim 1, characterized in that: In step S1, the overlapping relationship between the point cloud data is determined in the following manner: if the bounding boxes of two pieces of point cloud data have an intersection in space, the two pieces of point cloud data overlap, otherwise there is no overlap.

6. A multi-view point cloud registration method with minimized constraints as claimed in claim 5, characterized in that: The intersection in space is performed as follows: Among them, O i and O j Represent the i-th and j-th bounding boxes respectively, Yes i The x, y, z axis coordinates of the center point, Yes j The center point x, y, z axis coordinates, L i , W i , H i O i Length, width, height, L j , W j , H j O j The length, width and height of the .

7. The multi-view point cloud registration method with minimized constraints as claimed in claim 1, characterized in that: In step S2, the point pairing is performed as follows: the nearest point from a point in one point cloud to all points in another point cloud is found, and if the distance between the two points is less than a certain threshold, the two points are considered to constitute a corresponding point pair.

8. A multi-view point cloud registration method with minimized constraints as claimed in claim 7, characterized in that: The search is performed using KD-Tree.

9. The multi-view point cloud registration method with minimized constraints as claimed in claim 1, characterized in that: In step S2, the Gauss-Newton method is used to solve the objective function.

Images