A method and device for splicing three-dimensional point cloud data

Abstract

A method and device for splicing three-dimensional point cloud data, which can solve the problems of difficult splicing and low accuracy of two point clouds with low overlap rate, and effectively improve the splicing accuracy and time efficiency of point cloud data with relatively low initial overlap degree. The method includes: (1) The point cloud acquisition system acquires partially overlapping source point cloud and target point cloud, and preprocesses the point cloud; (2) Through coordinate transformation of the two scanning spatial positions, the source point cloud and the target point cloud obtain good initial positions; (3) Calculate the curvature features of the point cloud and use Kdtree to accelerate the search process of the nearest neighbor to find the initial matching point pairs, and then use the direction vector threshold method to set constraints to eliminate the wrong point pairs; (4) Use the singular value decomposition SVD method to obtain the rotation matrix R and translation vector t between the two point clouds to complete the splicing.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly relates to a method for stitching three-dimensional point cloud data and a device for stitching three-dimensional point cloud data. Background Art

[0002] Point cloud stitching is an important part in the process of three-dimensional point cloud processing and is also a very crucial task in fields such as reverse engineering, virtual reality, and cultural relic restoration. In actual measurement, due to the limitations of the camera and projector viewpoints, it is difficult to obtain the complete shape of an object through measurement from a single viewpoint. Therefore, it is necessary to measure from different viewpoints and stitch the point cloud data obtained from each viewpoint into a complete three-dimensional model, and this process is point cloud stitching.

[0003] Currently, common stitching methods include those that rely on external auxiliary information and those that rely on the measurement data itself. Among them, the surface marker point method is a stitching method that relies on external auxiliary information. It obtains the coordinate transformation relationship between the marker points in the initial point cloud and the marker points in the target point cloud by pasting marker points on the surface of the object to be measured, so as to stitch the point cloud data from different viewpoints. However, the applicable range of this method is limited. Stitching methods that rely on the measurement data itself include the Iterative Closest Point (ICP) method, the Principal Component Analysis (PCA) method, etc. The ICP algorithm iteratively searches for the closest points between two point clouds to find the transformation relationship when the Euclidean distance reaches the minimum. However, there are two preconditions for using the ICP algorithm. One is that the two point clouds should be similar enough, and the other is that the two point clouds need to have a good initial position. In addition, the convergence speed of traditional ICP is usually low, and the algorithm requires a large amount of computing time. The PCA first calculates the eigenvectors of the point cloud through the covariance matrix, and then calculates the main direction of the point cloud and the two secondary directions perpendicular to it from the eigenvectors, and uses these three directions to complete the stitching of the point cloud. This method has high computational efficiency, but requires a high enough overlap rate between adjacent point clouds. Summary of the Invention

[0004] To overcome the defects of the prior art, the technical problem to be solved by the present invention is to provide a method for stitching three-dimensional point cloud data, which can solve the problems of difficult stitching and low accuracy of two point clouds with low overlap rate, and effectively improve the stitching accuracy and time efficiency of point cloud data with a relatively low initial overlap degree.

[0005] The technical solution of the present invention is as follows: This method for stitching three-dimensional point cloud data includes the following steps:

[0006] (1) A point cloud acquisition system acquires partially overlapping source point cloud and target point cloud, and preprocesses the point cloud;

[0007] (2) Through coordinate transformation of the spatial positions scanned twice, the source point cloud and the target point cloud are given a good initial position;

[0008] (3) Calculate the curvature features of the point cloud and use Kdtree to accelerate the search process of the nearest neighbor to find the initial matching point pairs, and then use the direction vector threshold method to set constraints to eliminate the wrong point pairs;

[0009] (4) Use the singular value decomposition (SVD) method to obtain the rotation matrix R and translation vector t between the two point clouds to complete the stitching.

[0010] The present invention first obtains the source point cloud and target point cloud with partial overlap through the point cloud acquisition system, and preprocesses the point cloud; then makes the source point cloud and target point cloud obtain good initial positions through the coordinate transformation of the two scanning spatial positions; calculates the curvature features of the point cloud and uses the Kdtree nearest neighbor search method to obtain the initial matching point pairs of the two point clouds, and sets constraints through the direction vector threshold method to eliminate the wrong point pairs; finally, uses the singular value decomposition to obtain the rotation matrix R and translation vector t between the two point clouds. Therefore, it can solve the problems of great difficulty and low accuracy in stitching two point clouds with low overlap rate, and effectively improve the stitching accuracy and time efficiency of point cloud data with relatively low initial overlap degree.

[0011] A stitching device for three-dimensional point cloud data is also provided, which includes:

[0012] A data preprocessing module configured to obtain the source point cloud and target point cloud with partial overlap through the point cloud acquisition system and preprocess the point cloud;

[0013] An initial position acquisition module configured to make the source point cloud and target point cloud obtain good initial positions through the coordinate transformation of the two scanning spatial positions;

[0014] A data elimination module configured to calculate the curvature features of the point cloud and use Kdtree to accelerate the search process of the nearest neighbor to find the initial matching point pairs, and then use the direction vector threshold method to set constraints to eliminate the wrong point pairs;

[0015] A stitching module configured to use the singular value decomposition (SVD) method to obtain the rotation matrix R and translation vector t between the two point clouds to complete the stitching. Description of the Drawings

[0016] Figure 1 is a flowchart of the stitching method for three-dimensional point cloud data according to the present invention.

[0017] Figure 2 shows a schematic flow diagram of an embodiment of the present invention.

[0018] Figure 3 is a schematic structural diagram of eliminating wrong matching point pairs according to the included angle between the corresponding point normals. Detailed Embodiments

[0019] As Figure 1 shown, this method for stitching three-dimensional point cloud data includes the following steps:

[0020] (1) The point cloud acquisition system obtains partially overlapping source point cloud and target point cloud, and preprocesses the point cloud;

[0021] (2) Through coordinate transformation of the scanning spatial positions twice, the source point cloud and the target point cloud obtain good initial positions;

[0022] (3) Calculate the curvature features of the point cloud and use Kdtree to accelerate the search process of the nearest neighbor to find the initial matching point pairs, and then use the direction vector threshold method to set constraint conditions to eliminate the wrong point pairs;

[0023] (4) Use the singular value decomposition SVD method to obtain the rotation matrix R and translation vector t between the two point clouds, and complete the stitching.

[0024] The present invention first obtains the source point cloud and the target point cloud with partial overlap through the point cloud acquisition system, and preprocesses the point cloud; then through coordinate transformation of the scanning spatial positions twice, the source point cloud and the target point cloud obtain good initial positions; by calculating the curvature features of the point cloud and using the Kdtree nearest neighbor search method to obtain the initial matching point pairs of the two point clouds, and by using the direction vector threshold method to set constraint conditions to eliminate the wrong point pairs; finally, use the singular value decomposition to obtain the rotation matrix R and translation vector t between the two point clouds. Therefore, it can solve the problems of difficult stitching and low accuracy of two point clouds with low overlap rate, and effectively improve the stitching accuracy and time efficiency of point cloud data with low initial overlap degree.

[0025] Preferably, in the step (1), the point cloud preprocessing includes filtering and downsampling. The statistical filtering, radius filtering, and Gaussian filtering methods are used to remove the noise and outliers of the original point cloud, so as to improve the calculation efficiency and accuracy of subsequent operations; the improved voxel filtering is used to downsample the point cloud, which can reduce the number of point clouds while not changing the original point cloud data structure, and improve the speed of subsequent stitching.

[0026] Preferably, in the step (1), the VoxelGrid class of PCL creates a three-dimensional voxel grid through the input point cloud data, and uses the centroid of all points in each voxel to approximately represent other points in the voxel, so that all points in the voxel are represented by a centroid point, and this centroid point is not necessarily a point in the original point cloud, losing the fine features of the original point cloud; the improved voxel filtering uses the point closest to the centroid point of the voxel in the original point cloud data to replace the centroid point of the voxel, so as to improve the expression accuracy of the point cloud data.

[0027] Preferably, step (3) includes the following sub-steps:

[0028] (3.1) For the curvature feature, a covariance matrix is constructed through a query point x and its k neighborhood points in the point cloud:

[0029]

[0030]

[0031] where x i is the k neighborhood point of the query point x, is the centroid of x and its neighborhood points;

[0032] (3.2) The three eigenvalues of the covariance matrix are obtained by the singular value decomposition (SVD) method and arranged in descending order, which are λ d1 > λ d2 > λ d3 ;

[0033] (3.3) The surface curvature at the query point x is represented by s d :

[0034]

[0035] (3.4) To enhance robustness, different neighborhood search radii are selected, and the difference in curvature information calculated under different search radii is used as the matching information

[0036] Δs d = s d+1 - s d

[0037] (3.5) The curvature information of another point cloud is calculated using the same method, denoted as Δs d ′. A threshold τ is set. If Δs d - Δs d ′ < τ, it is considered a matching point pair;

[0038] (3.6) For the direction vector threshold method, for a point p i in the point cloud and its k neighborhood points, the least squares method is used to fit a local plane for the neighborhood points. The general expression of the plane equation is Ax + By + Cz - D = 0. Among the four parameters to be solved, the vector (A, B, C) whose elements are not all 0 actually represents the normal vector n of the plane equation, and D is the distance from the origin to the plane; The fitting process of the local plane P l is expressed as:

[0039]

[0040] By minimizing the objective function, the dot product of the vector formed by this point and each of its neighboring points with the normal vector is made to be 0; in the above formula, m is the point x i and the centroid of its neighborhood points

[0041] Meanwhile, make y i = X i - m

[0042] In this way, the objective function is transformed into:

[0043]

[0044] (3.7) Let S=(YY T ), decompose it using the SVD algorithm, and obtain

[0045] Y = U∑V T

[0046] The last column in U is the normal vector n to be solved, that is, the eigenvector corresponding to the smallest eigenvalue;

[0047] (3.8) Normalize the obtained normal vector, calculate the cosine value of the angle between the normal vectors of each pair of matching points, set a threshold ε, if the cosine value of the angle cosθ < ε, then consider this pair of corresponding points as a wrong pair of points and remove it from the set of corresponding points.

[0048] Preferably, in the step (3.5), when searching for the neighborhood points of the query point, Kdtree is used to accelerate the search process of the nearest neighbor.

[0049] Preferably, in the step (3.7), in the process of calculating the normal vector, the method of OpenMP multi-threaded parallel computing is adopted.

[0050] As Figure 2 shown, an embodiment of the present invention and its implementation process are as follows:

[0051] Step 1: Obtain partially overlapping source point cloud and target point cloud required for the experiment.

[0052] The first group is the bunny partial data set, with 4156 points in the source point cloud and 3679 points in the target point cloud, without the need for preprocessing; the second group is the point cloud data actually obtained by the structured light system, with 204833 points in the source point cloud and 218594 points in the target point cloud. The two point clouds are downsampled using improved voxel filtering. After preprocessing, there are 51797 points in the source point cloud and 58216 points in the target point cloud. Before preprocessing the point cloud, the point cloud data format collected by the structured light is.txt, while the required point cloud format for the experiment is.pcd. Here, the software Visual Studio 2017 is used to configure the PCL 1.9.1 environment to convert the.txt point cloud data into the.pcd format. The subsequent stitching process is also carried out in this configured environment.

[0053] Step 2: Use the CorrespondenceEstimationNormalShooting class to determine the initial matching point pairs of the two point clouds.

[0054] The class CorrespondenceEstimationNormalShooting realizes the calculation of the corresponding point pairs between the input point cloud and the target point cloud under the minimum distance constraint through the normal vector. The input is the target and source point clouds, and the output is the point pairs, that is, the set of corresponding points between the two groups of point clouds. It should be noted that the normal vector calculated here is the normal vector corresponding to the source point cloud, not the normal vector corresponding to the target point cloud. There are two ways to determine the correspondence between the source point cloud and the target point cloud. One is determineCorrespondences, which inputs the maximum distance max_distance allowed between the source point cloud and the corresponding target point cloud, and the output found correspondence (query point index, target point index, and the distance between them) is stored in correspondences; the other is determineReciprocalCorrespondences, which is the same as the above function, but does not require the input of the maximum distance allowed between the source point cloud and the target point cloud, and the corresponding points found here are reciprocal. The number of initial matching point pairs determined for the first group of point clouds is 2498 pairs, and the number of initial matching point pairs determined for the second group of point clouds is 22286 pairs.

[0055] Step 3: Use the direction vector threshold method to eliminate the incorrect matching point pairs.

[0056] The class CorrespondenceRejectorSurfaceNormal implements a correspondence removal method based on the normal angle of corresponding points. The input is the normals of the source point cloud and the target point cloud, the initial correspondences, and the set angle threshold between the normals of corresponding points. The output is the exact matching point pairs after removing incorrect correspondences, which are stored in correspondences_after_rejector. During the process of calculating the normals, the method of parallel acceleration with OpenMP multi-threading is adopted, and during the process of searching for neighboring points, the Kdtree accelerated search method is adopted to improve the computational efficiency of the algorithm. The number of OpenMP threads set in the experiment is 8.

[0057] As Figure 3 shown, it is a schematic diagram of removing incorrect matching point pairs by the angle between the normals of corresponding points. Among them, q1, q2, q3,... are some points in the point cloud, n represents the normal vector at the q1 point, and (q1, q′1), (q2, q′2), (q3, q′3) are the correct corresponding point pairs. The normal vector at the q4 point is n4, and the normal vector at the q′4 point is n′4. The angle between n4 and n′4 is θ. Set the threshold ε. If cosθ < ε, then (q4, q′4) is considered an incorrect matching point pair and is removed from the initial matching point pairs. In the experiment, the set angle threshold for the first group of point clouds is 4°, and the number of point pairs after removal is 44 pairs; the set angle threshold for the second group of point clouds is 1°, and the number of point pairs after removal is 127 pairs.

[0058] Step 4: Use the singular value decomposition method (SVD) to obtain the rotation matrix R and translation vector t between the two point clouds.

[0059] The class TransformationEstimationSVD is an implementation of an algorithm for estimating the transformation matrix based on the SVD method for a given correspondence. The input of this algorithm is two groups of point clouds to be registered and their corresponding point pairs, and the output is the transformation matrix solved by the SVD method, which is stored in transformation.

[0060] The result data of the two groups of point cloud data in the experiment are shown in Table 1:

[0061]

[0062] Table 1

[0063] In this embodiment, the two sets of experimental data are the bunny partial data set and the point cloud data actually collected by the structured light system. In the process of removing incorrect matching point pairs by the direction vector threshold method, not only are the matching point pairs reduced, the subsequent calculation speed is accelerated, but also the incorrect corresponding point pairs that affect the stitching are removed, improving the accuracy of the algorithm.

[0064] Those of ordinary skill in the art can understand that all or part of the steps in implementing the method of the above embodiment can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable storage medium. When the program is executed, it includes the steps of the method of the above embodiment, and the storage medium can be: ROM / RAM, magnetic disk, optical disk, memory card, etc. Therefore, corresponding to the method of the present invention, the present invention also includes a device for stitching three-dimensional point cloud data, which is usually represented in the form of functional modules corresponding to the steps of the method. The device includes:

[0065] A data preprocessing module configured to obtain partially overlapping source point cloud and target point cloud through a point cloud acquisition system and preprocess the point cloud;

[0066] An initial position acquisition module configured to obtain a good initial position of the source point cloud and the target point cloud through coordinate transformation of the spatial positions scanned twice;

[0067] A data removal module configured to calculate the curvature feature of the point cloud and use Kdtree to accelerate the search process of the nearest neighbor to find the initial matching point pairs, and then use the direction vector threshold method to set constraint conditions to remove incorrect point pairs;

[0068] A stitching module configured to use the singular value decomposition SVD method to obtain the rotation matrix R and the translation vector t between the two point clouds to complete the stitching.

[0069] The beneficial effects of the present invention are as follows: The present invention provides a nearest neighbor search method based on curvature features and accelerated by Kdtree for the problems of difficult stitching and low accuracy of two point clouds with low overlap rate. In the search method of matching point pairs, for the problem of time-consuming calculation of massive point cloud data, the curvature feature of the point cloud data is used to achieve more accurate matching, and at the same time, the Kdtree nearest neighbor search method is added to improve the search speed. After obtaining the matching point pairs of the two point clouds, in order to further reduce the stitching error, incorrect point pairs are removed by the direction vector threshold method. In the process of calculating the normal vector, the OpenMP multi-threaded parallel calculation method is adopted to improve the algorithm efficiency. Finally, the singular value decomposition method is used to obtain the rotation matrix R and the translation vector t between the two point clouds. The stitching accuracy and time efficiency of point cloud data with a relatively low initial overlap degree are effectively improved.

[0070] The above are only the preferred embodiments of the present invention, and do not impose any form of limitation on the present invention. Any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention still fall within the protection scope of the technical solution of the present invention.

Claims

1. A method for stitching three-dimensional point cloud data, characterized in that: It includes the following steps: (1) The point cloud acquisition system obtains partially overlapping source point cloud and target point cloud, and preprocesses the point cloud; (2) Through the coordinate transformation of the spatial positions of two scans, the source point cloud and the target point cloud are given good initial positions; (3) Calculate the curvature features of the point cloud and use Kdtree to accelerate the search process of the nearest neighbor to find the initial matching point pairs, and then use the direction vector threshold method to set constraint conditions to eliminate the wrong point pairs; (4) Use the singular value decomposition (SVD) method to obtain the rotation matrix R and translation vector t between the two point clouds, and complete the stitching; The step (3) includes the following sub-steps: (3.1) The curvature feature constructs a covariance matrix through a query point x in the point cloud and its k neighborhood points: where x i is the k-nearest neighbor of the query point x, is the centroid of x and its neighboring points; (3.2) Obtain the three eigenvalues of the covariance matrix through the SVD method, and arrange them in descending order, which are λ d1 > λ d2 > λ d3 ; (3.3) The surface curvature at the query point x is represented by s d denotes: (3.4) To enhance the robustness, different neighborhood search radii are selected, and the difference in the curvature information calculated under different search radii is used as the matching information Δs d = s d+1 - s d (3.5) Calculate the curvature information of another point cloud using the same method, denoted as Δs d ′, set a threshold τ. If Δs d - Δs d ′ < τ, it is considered a matching point pair; In the (3.6) direction vector threshold method, for the point p in the point cloud i and its k-neighborhood points, the least squares method is used to fit a local plane for the neighborhood points. The general expression of the plane equation is Ax + By + Cz - D = 0. Among the four parameters to be solved, the vector (A, B, C) where the elements are not all 0 at the same time has the actual meaning of the normal vector n of the plane equation, and D is the distance from the origin to the plane; the local plane P derived according to the least squares method l The fitting process is expressed as: By minimizing the objective function, the dot product of the vector formed by the point and each of its neighboring points with the normal vector is 0; in the above formula, m is the point x i and the centroid of its neighborhood points Simultaneously making y i = x i - m In this way, the objective function is transformed into: (3.7) Let S = (YY T ), decompose it using the SVD algorithm, and obtain Y = U∑V T The last column of U is the normal vector n to be solved, that is, the eigenvector corresponding to the minimum eigenvalue; (3.8) Normalize the obtained normal vector, calculate the cosine value of the angle between the normal vectors of each matching point pair, set a threshold ε, if the cosine value of the angle cosθ < ε, then consider this corresponding point pair as a wrong point pair and remove it from the corresponding point set.

2. The method for stitching three-dimensional point cloud data according to claim 1, characterized in that: In the step (1), the point cloud preprocessing includes filtering and downsampling. The methods of statistical filtering, radius filtering, and Gaussian filtering are used to remove the noise and outliers of the original point cloud, improving the calculation efficiency and accuracy of subsequent operations; An improved voxel filtering is used to downsample the point cloud, reducing the number of point clouds while not changing the original point cloud data structure, and improving the speed of subsequent stitching; In the step (1), the improved voxel filtering replaces the voxel centroid point with the point closest to the voxel centroid point in the original point cloud data, so as to improve the expression accuracy of the point cloud data.

3. The method for stitching three-dimensional point cloud data according to claim 2, wherein: In the step (3.5), Kdtree is used to accelerate the search process of the nearest neighbor when searching for the neighborhood points of the query point.

4. The method for stitching three-dimensional point cloud data according to claim 3, characterized in that: In the step (3.7), in the process of calculating the normal vector, the method of OpenMP multi-threaded parallel calculation is adopted.

5. An apparatus for splicing three-dimensional point cloud data, which is used to execute the method according to any one of claims 1-4, characterized in that: It includes: A data preprocessing module configured to obtain partially overlapping source point cloud and target point cloud through a point cloud acquisition system and preprocess the point cloud; An initial position acquisition module configured to give the source point cloud and the target point cloud good initial positions through the coordinate transformation of the spatial positions of two scans; A data elimination module configured to calculate the curvature features of the point cloud and use Kdtree to accelerate the search process of the nearest neighbor to find the initial matching point pairs, and then use the direction vector threshold method to set constraint conditions to eliminate the wrong point pairs; A stitching module configured to use the singular value decomposition (SVD) method to obtain the rotation matrix R and translation vector t between the two point clouds and complete the stitching.

Images