Point cloud stitching method and device, 3D scanner and electronic device

Abstract

The present application provides a point cloud stitching method and apparatus, a three-dimensional scanner, an electronic device, and a computer-readable storage medium, relating to the technical field of point cloud data processing. The method adaptively downsamples point cloud data according to the information of point cloud normals in the point cloud data to obtain the downsampled point cloud data; initially establishes a set of matching point pairs between at least two pieces of point cloud data according to the FPFH features of the downsampled point cloud data; screens the set of point pairs based on the RANSAC algorithm to calculate an initial transformation matrix; iteratively optimizes the initial transformation matrix based on the ICP algorithm to obtain an optimized rotation and translation matrix, and then stitches at least two pieces of point cloud according to the optimized rotation and translation matrix. It can be seen that the embodiment of the present application can give a correct initial pose, enabling the subsequent iterative optimization to proceed in the correct direction, reducing the number of iterations, and improving the efficiency and accuracy of point cloud stitching.

Description

Technical Field

[0001] The present application relates to the technical field of point cloud data processing, and in particular, to a point cloud stitching method and device, a three-dimensional scanner, an electronic device, and a computer-readable storage medium. Background Art

[0002] Due to its higher reconstruction accuracy compared with laser scanners, structured light scanners are widely used in scientific research, production manufacturing, and online detection in domestic and foreign research institutions, universities, and enterprises, covering industries such as consumer electronics, aerospace, automotive, heavy machinery, and medical. During the scanning process of a structured light scanner, due to problems such as occlusion of the surface of the object to be scanned caused by the scanning angle, light intensity, noise, background, etc., it is necessary to perform multiple scans and register the point cloud data at multiple angles so that the point cloud data is unified into the same coordinate system and stitched into a complete point cloud model. Due to the scanning characteristics of structured light scanners, the overlapping area of the scanned point cloud data is often limited. For such cases where the overlapping area of point cloud data is small, how to accurately and efficiently register it has become a technical problem that urgently needs to be solved. Summary of the Invention

[0003] In view of the above problems, the present application is proposed to provide a point cloud stitching method and device, a three-dimensional scanner, an electronic device, and a computer-readable storage medium that can solve the above problems and improve the efficiency and accuracy of point cloud stitching. The technical solutions are as follows:

[0004] In a first aspect, a point cloud stitching method is provided, which is applied to a three-dimensional scanner and includes:

[0005] Obtain at least two pieces of point cloud data to be stitched, and adaptively downsample the point cloud data according to the information of the point cloud normal in the point cloud data to obtain the downsampled point cloud data;

[0006] Calculate the FPFH (Fast Point Feature Histograms) features of the downsampled point cloud data;

[0007] Based on the FPFH features of the downsampled point cloud data, preliminarily establish a set of matching point pairs between the at least two pieces of point cloud data;

[0008] Based on the RANSAC (RANdom SAmple Consensus) algorithm, screen a preset number of point pairs in the set of point pairs for a preset number of times to obtain the screened point pairs, and calculate an initial transformation matrix based on the screened point pairs;

[0009] Iteratively optimize the initial transformation matrix based on the ICP (Iterative Closest Point) algorithm to obtain an optimized rotation and translation matrix, and then stitch the at least two point clouds according to the optimized rotation and translation matrix.

[0010] In a possible implementation, the adaptive downsampling of the point cloud data according to the information of the point cloud normal in the point cloud data to obtain the downsampled point cloud data includes:

[0011] Create at least one layer of voxel grids for the point cloud data;

[0012] Merge the points with a normal angle less than a preset angle threshold in each voxel of the voxel grid to obtain the downsampled point cloud data.

[0013] In a possible implementation, the calculation of the FPFH feature of the downsampled point cloud data includes:

[0014] For the downsampled point cloud data, assume the source point P t , given a search radius r, find one or more neighboring points P t of point P s , establish a local coordinate system uvw at point P t , calculate the triple descriptor (α, φ, θ) of point P t and its neighboring points, and divide each of the three parameters α, φ, θ into g intervals, where g is a positive integer, and count the distribution of the triple descriptor (α, φ, θ) of point P t and its neighboring points falling into the intervals to obtain a 3×g-dimensional feature vector SPFH (Simplified Point Feature Histogram), denoted as SPFH(P t ), and the uvw coordinate system is as follows:

[0015]

[0016] P t and its neighboring point P s The calculation formula for the triple descriptor (α, φ, θ) is as follows:

[0017]

[0018] where n t is the normal vector of point P t , n s is the normal vector of point P s , d is the Euclidean distance between P t and P s ;

[0019] For point P t and its neighboring point P s calculate the 3×g dimensional feature vector SPFH of the neighboring point P s , denoted as SPFH(P s );

[0020] Use the following formula to weight SPFH(P t ) and SPFH(P s ) to obtain the FPFH feature of Pt:

[0021]

[0022] where w s represents the Euclidean distance between P t and P s .

[0023] In a possible implementation, based on the ICP algorithm, iteratively optimize the initial transformation matrix to obtain an optimized rotation and translation matrix, including:

[0024] Based on the initial transformation matrix, combine the ICP algorithm with point-to-plane metric to calculate the loss function for the selected point pairs, and repeat iterative optimization until the change amount of the rotation and translation matrix is less than the preset change threshold and / or the error value of the loss function is less than the preset error threshold, then terminate the iteration to obtain the optimized rotation and translation matrix. The loss function E symm is as follows:

[0025]

[0026] where R is the rotation matrix, T is the translation matrix, p i and q i are respectively a point in two point clouds, is the normal vector of point p i , is the normal vector of point q i .

[0027] In a possible implementation, based on the RANSAC algorithm, screen a preset number of point pairs in the set of point pairs for a preset number of times to obtain the selected point pairs, and calculate the initial transformation matrix based on the selected point pairs, including:

[0028] Randomly select m point pairs from the set of point pairs, and calculate the first initial transformation matrix based on the RANSAC algorithm; calculate the distance errors of the other point pairs in the set of point pairs except the selected point pairs under the first initial transformation matrix, mark the points with distance errors less than the preset threshold as the first inliers, mark the points with distance errors greater than the preset threshold as the first outliers, and count the number of the first inliers;

[0029] Randomly select m point pairs from the set of point pairs, and calculate the second initial transformation matrix based on the RANSAC algorithm; calculate the distance errors of the other point pairs in the set of point pairs except the selected point pairs under the second initial transformation matrix, mark the points with distance errors less than the preset threshold as the second inliers, mark the points with distance errors greater than the preset threshold as the second outliers, and count the number of the second inliers;

[0030] And so on, randomly select m point pairs from the set of point pairs, and calculate the Nth initial transformation matrix based on the RANSAC algorithm; calculate the distance errors of the other point pairs in the set of point pairs except the selected point pairs under the Nth initial transformation matrix, mark the points with distance errors less than the preset threshold as the Nth inliers, mark the points with distance errors greater than the preset threshold as the Nth outliers, and count the number of the Nth inliers;

[0031] Select the Nth initial transformation matrix with the largest number of inliers among the N samples counted as the initial transformation matrix.

[0032] In a possible implementation, the m point pairs are 3 point pairs; N is a positive integer greater than or equal to 1.

[0033] In a second aspect, a point cloud stitching device is provided, which is applied to a 3D scanner and includes:

[0034] A sampling module, configured to obtain at least two pieces of point cloud data to be stitched, and adaptively downsample the point cloud data according to the information of the point cloud normals in the point cloud data to obtain the downsampled point cloud data;

[0035] An FPFH feature calculation module, configured to calculate the FPFH features of the downsampled point cloud data;

[0036] A point pair set establishment module, configured to preliminarily establish a set of point pairs that match between the at least two pieces of point cloud data according to the FPFH features of the downsampled point cloud data;

[0037] An initial transformation matrix calculation module, configured to perform a preset number of screenings on a preset number of point pairs in the point pair set based on the RANSAC algorithm, obtain the screened point pairs, and calculate an initial transformation matrix based on the screened point pairs;

[0038] An accurate registration module, configured to iteratively optimize the initial transformation matrix based on the ICP algorithm to obtain an optimized rotation and translation matrix, and then splice the at least two point clouds according to the optimized rotation and translation matrix.

[0039] In a third aspect, a 3D scanner is provided, including a 3D scanner body. When the 3D scanner performs a 3D scanning operation, the point cloud splicing method described in any one of the above is used to splice the point cloud data obtained by scanning.

[0040] In a fourth aspect, an electronic device is provided. The electronic device includes a processor and a memory. Among them, a computer program is stored in the memory, and the processor is configured to run the computer program to execute the point cloud splicing method described in any one of the above.

[0041] In a fifth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program, where the computer program is configured to execute the point cloud splicing method described in any one of the above when running.

[0042] By means of the above technical solutions, the point cloud splicing method and device, 3D scanner, electronic device, and computer-readable storage medium provided by the embodiments of the present application can adaptively downsample the point cloud data according to the information of the point cloud normal in the point cloud data to obtain the downsampled point cloud data. The downsampled point cloud data is representative and descriptive, and the amount of the downsampled point cloud data is much less than that of the original point cloud data, which can improve the subsequent processing speed; then calculate the FPFH features of the downsampled point cloud data, and based on the FPFH features of the downsampled point cloud data, initially establish a set of matching point pairs between at least two point cloud data. Furthermore, based on the RANSAC algorithm, perform a preset number of screenings on a preset number of point pairs in the point pair set to obtain the screened point pairs, and calculate an initial transformation matrix based on the screened point pairs. Since the FPFH features estimate the surface change of the point cloud through all the mutual relationships between the normal directions, they can well describe the geometric features of the point cloud, so as to be able to give a correct initial pose, making the subsequent iterative optimization proceed in the correct direction, reducing the number of iterations, and improving the efficiency and accuracy of point cloud splicing; finally, iteratively optimize the initial transformation matrix based on the ICP algorithm to obtain an optimized rotation and translation matrix, and then splice at least two point clouds according to the optimized rotation and translation matrix, so as to perform accurate registration on the basis of initial registration, further improving the accuracy of point cloud splicing. Brief Description of the Drawings

[0043] To more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments of the present application.

[0044] Figure 1 Shows a flowchart of a point cloud stitching method according to an embodiment of the present application;

[0045] Figure 2 Shows a schematic diagram of single voxel downsampling according to an embodiment of the present application;

[0046] Figure 3 Shows a flowchart of a point cloud stitching method according to another embodiment of the present application;

[0047] Figure 4 Shows the point cloud data P and Q to be stitched and their relative poses according to an embodiment of the present application;

[0048] Figure 5 Shows the effect diagram after stitching the point cloud data P and Q using the existing solution;

[0049] Figure 6 Shows the effect diagram after stitching the point cloud data P and Q according to an embodiment of the present application; and

[0050] Figure 7 Shows the structural diagram of a point cloud stitching device according to an embodiment of the present application. Detailed Description of the Embodiments

[0051] The following will describe the exemplary embodiments of the present application in more detail with reference to the drawings. Although the exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present application can be more thoroughly understood and the scope of the present application can be completely conveyed to those skilled in the art.

[0052] It should be noted that the terms "first", "second", etc. in the description and claims of the present application and the above drawings are used to distinguish similar objects and do not necessarily have to be used to describe a specific order or sequence. It should be understood that such use can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the term "including" and its variants should be interpreted as an open term meaning "including but not limited to".

[0053] The embodiments of the present application provide a point cloud stitching method, which can be applied to a 3D scanner. As Figure 1 shown, the point cloud stitching method may include the following blocks S101 to S105:

[0054] Block S101, obtain at least two pieces of point cloud data to be spliced, and adaptively downsample the point cloud data according to the information of the point cloud normals in the point cloud data to obtain the downsampled point cloud data;

[0055] Block S102, calculate the FPFH features of the downsampled point cloud data;

[0056] Block S103, based on the FPFH features of the downsampled point cloud data, initially establish a set of matching point pairs between at least two pieces of point cloud data;

[0057] Block S104, based on the RANSAC algorithm, perform a preset number of screenings on a preset number of point pairs in the set of point pairs to obtain the screened point pairs, and calculate the initial transformation matrix based on the screened point pairs;

[0058] Block S105, iteratively optimize the initial transformation matrix based on the ICP algorithm to obtain the optimized rotation and translation matrix, and then splice at least two pieces of point cloud according to the optimized rotation and translation matrix.

[0059] In the embodiments of the present application, the point cloud data can be adaptively downsampled according to the information of the point cloud normals in the point cloud data to obtain the downsampled point cloud data. The downsampled point cloud data is representative and descriptive, and the amount of the downsampled point cloud data is much less than that of the original point cloud data, which can improve the subsequent processing speed; then calculate the FPFH features of the downsampled point cloud data, and based on the FPFH features of the downsampled point cloud data, initially establish a set of matching point pairs between at least two pieces of point cloud data. Furthermore, based on the RANSAC algorithm, perform a preset number of screenings on a preset number of point pairs in the set of point pairs to obtain the screened point pairs, and calculate the initial transformation matrix based on the screened point pairs. Since the FPFH features estimate the surface change of the point cloud through all the mutual relationships between the normal directions, they can well describe the geometric features of the point cloud, so as to be able to give a correct initial pose, making the subsequent iterative optimization proceed in the correct direction, reducing the number of iterations, and improving the efficiency and accuracy of point cloud splicing; finally, iteratively optimize the initial transformation matrix based on the ICP algorithm to obtain the optimized rotation and translation matrix, and then splice at least two pieces of point cloud according to the optimized rotation and translation matrix, so as to perform precise registration on the basis of the initial registration, further improving the accuracy of point cloud splicing.

[0060] In the embodiments of the present application, in the above-mentioned block S101, the point cloud data is adaptively downsampled according to the information of the point cloud normals in the point cloud data to obtain the downsampled point cloud data, which may specifically include the following A1 to A2:

[0061] A1, for the point cloud data, create at least one layer of voxel grid.

[0062] In A1, you can create a layer of voxel grid, which can be used for high point cloud quality where the point cloud quality meets the preset quality conditions; you can also create multiple layers of voxel grids from fine to coarse.

[0063] A2: In each voxel of the voxel grid, points whose normal angles are less than a preset angle threshold are merged to obtain downsampled point cloud data.

[0064] In A2, the normal angle between points is used to reflect the change of geometric features on the surface of the object. Points with normal angles less than a preset angle threshold can be merged in each voxel of the voxel grid to obtain downsampled point cloud data. The preset angle threshold here can be set according to actual needs, such as a preset angle threshold of 25 degrees or 30 degrees, etc., which is not limited in this embodiment.

[0065] Figure 2 A schematic diagram of single voxel downsampling according to an embodiment of the present application is shown. Figure 2 In the example, a single voxel includes four vertices, and their normal vectors are n1, n2, n3, and n4, respectively. The normal angles of n1 and n3 are smaller than the preset angle threshold, and they are merged into n13; the normal angles of n2 and n4 are smaller than the preset angle threshold, and they are merged into n24, thereby realizing downsampling of a single voxel. It should be noted that the examples here are only for illustration and do not limit this embodiment.

[0066] The embodiments of the present application can retain denser points on the part where the geometric features (such as edges or surfaces with large curvature) on the surface of an object or scene change dramatically, while retaining sparser points on the plane part. These down-sampled points are stable and distinctive point sets on the point cloud. The number of down-sampled points is much reduced compared to the amount of original point cloud data. They are combined with the FPFH local feature descriptor to form a down-sampled point descriptor to form a compact representation of the original point cloud without losing representativeness and descriptiveness, thereby accelerating subsequent processing speed.

[0067] In the embodiments of the present application, PFH (Point Feature Histograms) is first introduced. It is a pose-invariant local feature that describes the geometric properties of points in the k-neighborhood of a source point by calculating the spatial differences between the source point or query point and neighboring points and forming a multi-dimensional histogram. Here, the points in the k-neighborhood of the source point are those whose distance from the source point is less than the radius r. The PFH representation is based on the relationships between all pairs of normal vectors of points within the point k-neighborhood. It estimates the surface variation of the sample through all the mutual relationships between normal directions to describe the geometric features of the sample. FPFH is improved from PFH, reducing the computational complexity of PFH, improving the computational efficiency, and retaining most of the discriminability of PFH. The above-mentioned block S102 calculates the FPFH features of the downsampled point cloud data, which specifically may include the following content:

[0068] For the downsampled point cloud data, assume the source point P t , given the search radius r, find one or more neighboring points P t of point P s . Establish a local coordinate system uvw at point P t , calculate the triple descriptor (α, φ, θ) of point P t and its neighboring points, and divide each of the three parameters α, φ, θ into g intervals, where g is a positive integer. Count the distribution of the triple descriptor (α, φ, θ) of point P t and its neighboring points falling into the intervals to obtain a 3×g-dimensional feature vector SPFH, denoted as SPFH(P t ). The uvw coordinate system is as follows:

[0069]

[0070] The calculation formula for the triple descriptor (α, φ, θ) of point P t and its neighboring point P s is as follows:

[0071]

[0072] Where n t is the normal vector of point P t , n s is the normal vector of point P s , and d is the Euclidean distance between P t and P s ;

[0073] Calculate the 3×g-dimensional feature vector SPFH of the neighboring point P t of point P, denoted as SPFH(P s ); s s s )

[0074] The SPFH(P t ) and SPFH(P s ) are weighted using the following formula to obtain the FPFH feature of Pt:

[0075]

[0076] where w s represents the Euclidean distance between P t and P s .

[0077] In the embodiments of the present application, g can be set according to actual needs. For example, g is 11, and the embodiments of the present application do not limit this. In this way, the distribution of the triple descriptor (α, φ, θ) of the P t point and its neighboring points falling into the interval is statistically obtained, and a 33-dimensional feature vector SPFH is obtained, denoted as SPFH(P t ). The method of using Pt to obtain SPFH(P t ) is used to calculate the 33-dimensional feature vector SPFH of the neighboring point Ps of the Pt point, denoted as SPFH(P s ).

[0078] In the embodiments of the present application, the above-mentioned block S103 initially establishes a set of matching point pairs between at least two point cloud data according to the FPFH features of the downsampled point cloud data. For example, the point cloud data P and Q are respectively downsampled to obtain the downsampled point cloud data P and Q, and the FPFH features of the downsampled point cloud data P and Q are calculated. For the point P t of the downsampled point cloud data P, a point Q t in Q whose FPFH feature matches that of P t is searched for, and a set of matching point pairs (P t , Q t ) between the two point cloud data is initially established.

[0079] In the embodiments of the present application, the above-mentioned block S104, based on the RANSAC algorithm, screens a preset number of point pairs in the set of point pairs for a preset number of times to obtain the screened point pairs, and calculates an initial transformation matrix based on the screened point pairs. Specifically, it may include the following content:

[0080] Randomly screen m point pairs in the set of point pairs, and calculate a first initial transformation matrix based on the RANSAC algorithm; calculate the distance error of other point pairs in the set of point pairs except the screened point pairs under the first initial transformation matrix, mark the point pairs with a distance error less than the preset threshold as the first inlier points, mark the point pairs with a distance error greater than the preset threshold as the first outlier points, and count the number of the first inlier points;

[0081] Randomly select m point pairs from the set of point pairs, and calculate the second initial transformation matrix based on the RANSAC algorithm; calculate the distance error of other point pairs in the set of point pairs except the selected point pairs under the second initial transformation matrix, mark the points with distance error less than the preset threshold as the second inlier points, mark the points with distance error greater than the preset threshold as the second outlier points, and count the number of the second inlier points;

[0082] And so on, randomly select m point pairs from the set of point pairs, and calculate the Nth initial transformation matrix based on the RANSAC algorithm; calculate the distance error of other point pairs in the set of point pairs except the selected point pairs under the Nth initial transformation matrix, mark the points with distance error less than the preset threshold as the Nth inlier points, mark the points with distance error greater than the preset threshold as the Nth outlier points, and count the number of the Nth inlier points;

[0083] Select the Nth initial transformation matrix with the largest number of inlier points among the N samples counted as the initial transformation matrix.

[0084] In the embodiments of the present application, the preset threshold can be set according to actual needs. For example, the preset threshold is 0.01 times the diagonal length of the bounding box, and the embodiments of the present application do not limit this. In addition, randomly selecting m point pairs can reflect that the system of the point cloud stitching scheme in the embodiments of the present application has better robustness, and the robustness here is the ability of the system to survive in abnormal and dangerous situations.

[0085] In the embodiments of the present application, m point pairs can be 3 point pairs, and N is a positive integer greater than or equal to 1. This can improve the screening efficiency, and can give a correct initial pose, so that the subsequent iterative optimization proceeds in the correct direction, reduce the number of iterations, and improve the point cloud stitching efficiency and accuracy.

[0086] In the embodiments of the present application, the above-mentioned block S105 iteratively optimizes the initial transformation matrix based on the ICP algorithm to obtain an optimized rotation and translation matrix, which may specifically include the following content:

[0087] Based on the initial transformation matrix, calculate the loss function for the selected point pairs by combining the ICP algorithm with the point-to-plane metric, and repeat the iterative optimization until the change amount of the rotation and translation matrix is less than the preset change threshold and / or the error value of the loss function is less than the preset error threshold, then terminate the iteration to obtain the optimized rotation and translation matrix, and the loss function E symm is as follows:

[0088]

[0089] where R is the rotation matrix, T is the translation matrix, and pi and q i are respectively a point in two point clouds, is the normal vector of point p i and is the normal vector of point q i .

[0090] In the embodiments of the present application, the initial transformation matrix obtained by screening based on the RANSAC algorithm makes the two point cloud data roughly coincide, but the registration accuracy may not meet the requirements of actual engineering applications. Therefore, precise registration is performed on the basis of the initial registration. The ICP algorithm requires that the point cloud data to be registered has a relatively close initial position. There are many measurement methods for the error between corresponding points. Here, the ICP algorithm combined with the point-to-plane measurement is used to calculate the loss function for the screened matching point pairs, so that the loss function is minimized until the change amount of the rotation and translation matrix is less than the preset change threshold or the error value of the loss function is less than the preset error threshold, and the iteration is terminated.

[0091] Figure 3 shows a flowchart of a point cloud stitching method according to another embodiment of the present application, as Figure 3 shown, the point cloud stitching method may include the following blocks S301 to S306:

[0092] Block S301, downsampling of point cloud data.

[0093] In this block, the method described in A1 to A2 above can be referred to for adaptively downsampling at least two point cloud data to be stitched to obtain the downsampled point cloud data. Here, a structured light scanner can be used to scan to obtain the point cloud data.

[0094] Block S302, calculating the FPFH features of the downsampled point cloud data and establishing a set of feature matching point pairs.

[0095] In this block, the method for calculating the FPFH features of the downsampled point cloud data described above can be referred to, and will not be elaborated here.

[0096] Block S303, screening the set of matching point pairs based on the RANSAC algorithm and solving the initial transformation matrix.

[0097] In this block, the method described above based on the RANSAC algorithm for screening a preset number of point pairs in the set of point pairs for a preset number of times to obtain the screened point pairs and calculating the initial transformation matrix based on the screened point pairs can be referred to, and will not be elaborated here.

[0098] Block S304, ICP iteration, calculating the loss function and the rotation and translation matrix, and updating the point cloud data.

[0099] In this box, the loss function can be referred to the previous introduction and will not be elaborated here. The rotation and translation matrix can be a 4×4 matrix, where the rotation matrix is a 3×3 matrix and the translation matrix is a 3×1 matrix, and the remaining positions are filled with 0 and 1.

[0100] Box S305, determine whether the change amount of the rotation and translation matrix is less than a preset change threshold or whether the error value of the loss function is less than a preset error threshold. If so, that is, the change amount of the rotation and translation matrix is less than the preset change threshold or the error value of the loss function is less than the preset error threshold, then execute box S306; if not, that is, the change amount of the rotation and translation matrix is greater than the preset change threshold or the error value of the loss function is greater than the preset error threshold, then return to continue executing box S304.

[0101] Box S306, terminate the iteration, obtain the optimized rotation and translation matrix, and then splice at least two point clouds according to the optimized rotation and translation matrix.

[0102] For the structured light scanner to perform scanning, the overlapping area of the scanned point cloud data is often limited. Currently, for the case where the overlapping area of such two point cloud data is small, the registration success rate cannot be guaranteed, and the number of iterations is large, the time consumption is long, and it is easy to fall into local optimization. In the embodiment of the present application, the point cloud data is first downsampled, sampling more points in the area with rich features and fewer points in the plane area; calculating the FPFH features of the sampled data points, and establishing a set of matching point pairs according to the FPFH features; screening the point pairs in the set of point pairs based on RANSAC, removing the wrong matching point pairs, and selecting the group with the largest number of remaining point pairs after screening as the sample points, and using the rotation and translation matrix of this group as the initial transformation matrix of the ICP algorithm; iteratively optimizing the initial transformation matrix based on the point-to-plane ICP algorithm, and splicing the point cloud according to the optimized rotation and translation matrix. It can be seen that the embodiment of the present application can give a correct initial pose, make the iteration proceed in the correct direction, reduce the number of iterations, and improve the splicing efficiency and success rate.

[0103] Figure 4 Shows the point cloud data P and Q to be spliced and their relative poses according to the embodiment of the present application. Specifically, (a) is the point cloud data P to be spliced, (b) is the point cloud data Q to be spliced, and (c) is the relative pose of P and Q.

[0104] Figure 5 Shows the effect diagram after splicing the point cloud data P and Q using the existing scheme. Specifically, Figure 5 The left figure in is the effect diagram of rotating and translating the point cloud data Q to P, and the right figure is the effect diagram of another angle of the left figure.

[0105] Figure 6Shows the effect diagram after splicing the point cloud data P and Q according to the embodiments of the present application. Specifically, Figure 6 The left figure in

[0106] Combined with Figure 4 、 Figure 5 And Figure 6 It can be seen that the point cloud splicing method provided by the embodiments of the present application has higher splicing accuracy.

[0107] It should be noted that in practical applications, all the above possible implementation manners can be combined in any combination to form possible embodiments of the present application, which will not be elaborated here one by one.

[0108] Based on the point cloud splicing method provided by the above embodiments, based on the same inventive concept, the embodiments of the present application also provide a point cloud splicing device.

[0109] Figure 7 Shows the structural diagram of the point cloud splicing device according to the embodiments of the present application. As Figure 7 Shown, the point cloud splicing device may include a sampling module 710, an FPFH feature calculation module 720, a point pair set establishment module 730, an initial transformation matrix calculation module 740, and an accurate registration module 750.

[0110] The sampling module 710 is configured to obtain at least two pieces of point cloud data to be spliced, and perform adaptive downsampling on the point cloud data according to the information of the point cloud normal in the point cloud data to obtain the downsampled point cloud data;

[0111] The FPFH feature calculation module 720 is configured to calculate the FPFH features of the downsampled point cloud data;

[0112] The point pair set establishment module 730 is configured to preliminarily establish a set of matching point pairs between at least two pieces of point cloud data according to the FPFH features of the downsampled point cloud data;

[0113] The initial transformation matrix calculation module 740 is configured to, based on the RANSAC algorithm, perform a preset number of screenings on a preset number of point pairs in the point pair set to obtain the screened point pairs, and calculate an initial transformation matrix based on the screened point pairs;

[0114] The accurate registration module 750 is configured to iteratively optimize the initial transformation matrix based on the ICP algorithm to obtain an optimized rotation and translation matrix, and then splice at least two pieces of point cloud according to the optimized rotation and translation matrix.

[0115] In the embodiments of the present application, the sampling module 710 shown above Figure 7 is further configured to:

[0116] For the point cloud data, at least one layer of voxel grids is created;

[0117] In each voxel of the voxel grid, the points with a normal angle less than a preset angle threshold are merged to obtain the downsampled point cloud data.

[0118] In the embodiments of the present application, the above Figure 7 The FPFH feature calculation module 720 shown is further configured to:

[0119] For the downsampled points of the point cloud data, assuming the source point P t , given a search radius r, one or more neighboring points P of the P t point are found. s , a local coordinate system uvw is established at the P t point, and the triple descriptor (α, φ, θ) of the P t point and the neighboring points is calculated. The 3 parameters α, φ, θ are respectively divided into g intervals, where g is a positive integer. The distribution of the triple descriptor (α, φ, θ) of the P t point and the neighboring points falling into the intervals is counted to obtain a 3×g-dimensional feature vector SPFH, denoted as SPFH(P t ). The uvw coordinate system is as follows:

[0120]

[0121] P t point and the neighboring point P s The calculation formula of the triple descriptor (α, φ, θ) is as follows:

[0122]

[0123] where n t is the normal vector of the P t point, n s is the normal vector of the P s point, and d is the Euclidean distance between P t and P s ;

[0124] For the neighboring point P t of the P s point, the 3×g-dimensional feature vector SPFH of the neighboring point P s is calculated, denoted as SPFH(P s );

[0125] The following formula is used to weight SPFH(P t ) and SPFH(P s ) to obtain the FPFH feature of Pt:

[0126]

[0127] where w s represents the Euclidean distance t between P s and P

[0128] In the embodiments of the present application, the precise registration module 750 shown above is further configured to: Figure 7 Based on the initial transformation matrix, combine the ICP algorithm with the point-to-plane metric to calculate the loss function for the selected point pairs, and repeat iterative optimization until the change amount of the rotation and translation matrix is less than the preset change threshold and / or the error value of the loss function is less than the preset error threshold, terminate the iteration, and obtain the optimized rotation and translation matrix. The loss function E

[0129] is as follows: symm is as follows:

[0130]

[0131] where R is the rotation matrix, T is the translation matrix, p i and q i are respectively a point in two point clouds, is the normal vector of point p i and is the normal vector of point q i and

[0132] In the embodiments of the present application, the initial transformation matrix calculation module 740 shown above is further configured to: Figure 7 Randomly select m point pairs from the point pair set, and calculate the first initial transformation matrix based on the RANSAC algorithm; calculate the distance error of the other point pairs in the point pair set except the selected point pairs under the first initial transformation matrix, mark the point pairs with distance error less than the preset threshold as the first inlier points, mark the point pairs with distance error greater than the preset threshold as the first outlier points, and count the number of the first inlier points;

[0133] Randomly select m point pairs from the point pair set, and calculate the second initial transformation matrix based on the RANSAC algorithm; calculate the distance error of the other point pairs in the point pair set except the selected point pairs under the second initial transformation matrix, mark the point pairs with distance error less than the preset threshold as the second inlier points, mark the point pairs with distance error greater than the preset threshold as the second outlier points, and count the number of the second inlier points;

[0134] Randomly select m point pairs from the point pair set, and calculate the second initial transformation matrix based on the RANSAC algorithm; calculate the distance error of the other point pairs in the point pair set except the selected point pairs under the second initial transformation matrix, mark the point pairs with distance error less than the preset threshold as the second inlier points, mark the point pairs with distance error greater than the preset threshold as the second outlier points, and count the number of the second inlier points;

[0135] By analogy, randomly select m point pairs from the point pair set, and calculate the Nth initial transformation matrix based on the RANSAC algorithm; calculate the distance error of other point pairs in the point pair set except the selected point pairs under the Nth initial transformation matrix, mark the points with distance error less than the preset threshold as the Nth inlier points, mark the points with distance error greater than the preset threshold as the Nth outlier points, and count the number of the Nth inlier points;

[0136] Among the N samples counted, select the Nth initial transformation matrix with the largest number of inlier points as the initial transformation matrix.

[0137] In the embodiments of the present application, m point pairs are 3 point pairs; N is a positive integer greater than or equal to 1.

[0138] Based on the same inventive concept, the embodiments of the present application further provide a 3D scanner, including a 3D scanner body. When the 3D scanner performs a 3D scanning operation, the point cloud stitching method as described in any one of the above is used to stitch the point cloud data obtained by scanning. It can be seen that compared with the traditional scanner, the 3D scanner provided by the embodiments of the present application can adaptively downsample the point cloud data according to the information of the point cloud normal in the point cloud data after scanning and obtaining at least two sets of point cloud data, so as to obtain the downsampled point cloud data. The downsampled point cloud data does not lose its representativeness and descriptiveness, and the amount of the downsampled point cloud data is much less than that of the original point cloud data, which can improve the subsequent processing speed; then calculate the FPFH features of the downsampled point cloud data, and based on the FPFH features of the downsampled point cloud data, preliminarily establish a set of matching point pairs between at least two sets of point cloud data, and further based on the RANSAC algorithm, screen a preset number of point pairs in the set of point pairs for a preset number of times to obtain the selected point pairs, and calculate the initial transformation matrix based on the selected point pairs. Since the FPFH features estimate the surface change of the point cloud through all the mutual relationships between the normal directions, it can well describe the geometric features of the point cloud, so as to be able to give a correct initial pose, make the subsequent iterative optimization proceed in the correct direction, reduce the number of iterations, and improve the efficiency and accuracy of point cloud stitching; finally, based on the ICP algorithm, iterate and optimize the initial transformation matrix to obtain the optimized rotation and translation matrix, and then stitch at least two sets of point clouds according to the optimized rotation and translation matrix, so as to perform precise registration on the basis of the initial registration, and further improve the accuracy of point cloud stitching.

[0139] Based on the same inventive concept, the embodiments of the present application further provide an electronic device, including a processor and a memory. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the point cloud stitching method of any one of the above embodiments.

[0140] Based on the same inventive concept, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, and the computer program is configured to execute the point cloud stitching method of any one of the above embodiments when running.

[0141] Those skilled in the art can clearly understand the specific working processes of the above-described systems, devices, and modules, and can refer to the corresponding processes in the foregoing method embodiments. For the sake of brevity, they will not be described in detail here.

[0142] Those of ordinary skill in the art can understand that the technical solution of the present application can essentially or all or part of the technical solution be embodied in the form of a software product. The computer software product is stored in a storage medium, which includes several program instructions for causing an electronic device (such as a personal computer, a server, or a network device, etc.) to execute all or part of the boxes of the method described in each embodiment of the present application when running the program instructions. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc that can store program codes.

[0143] Alternatively, all or part of the boxes of the foregoing method embodiments can be completed by hardware related to program instructions (such as an electronic device such as a personal computer, a server, or a network device), and the program instructions can be stored in a computer-readable storage medium. When the program instructions are executed by the processor of the electronic device, the electronic device executes all or part of the boxes of the method described in each embodiment of the present application.

[0144] The above embodiments are only used to illustrate the technical solution of the present application, rather than to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that within the spirit and principle of the present application, they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the corresponding technical solutions to deviate from the protection scope of the present application.

Claims

1. A point cloud stitching method, applied to a 3D scanner, characterized in that, Including: Obtain at least two pieces of point cloud data to be stitched, and adaptively downsample the point cloud data according to the information of the point cloud normal in the point cloud data to obtain the downsampled point cloud data; Calculate the FPFH features of the downsampled point cloud data; Based on the FPFH features of the downsampled point cloud data, preliminarily establish a set of matching point pairs between the at least two pieces of point cloud data; Based on the RANSAC algorithm, perform a preset number of screenings on a preset number of point pairs in the set of point pairs to obtain the screened point pairs, and calculate an initial transformation matrix based on the screened point pairs; Iteratively optimize the initial transformation matrix based on the ICP algorithm to obtain an optimized rotation and translation matrix, and then stitch the at least two pieces of point cloud according to the optimized rotation and translation matrix; Among them, iteratively optimizing the initial transformation matrix based on the ICP algorithm to obtain an optimized rotation and translation matrix includes: Based on the initial transformation matrix, the ICP algorithm combined with the point-to-plane metric calculates the loss function for the selected point pairs, and repeats iterative optimization until the change amount of the rotation and translation matrix is less than the preset change threshold and / or the error value of the loss function is less than the preset error threshold, terminates the iteration, and obtains the optimized rotation and translation matrix, the loss function E symm as follows: where R is the rotation matrix, T is the translation matrix, p i and q i are respectively a point in two point clouds, is the normal vector of point p i , and is the normal vector of point q i .

2. The point cloud stitching method according to claim 1, characterized in that The adaptively downsampling the point cloud data according to the information of the point cloud normal in the point cloud data to obtain the downsampled point cloud data includes: For the point cloud data, create at least one layer of voxel grid; Merge the points with a normal angle less than a preset angle threshold in each voxel of the voxel grid to obtain the downsampled point cloud data.

3. The point cloud stitching method according to claim 1 or 2, characterized in that, The calculating the FPFH features of the downsampled point cloud data includes: For the downsampled point cloud data, assume the source point P t , given the search radius r, find one or more neighboring points P t of point P s . Establish a local coordinate system uvw at point P t , calculate the triple descriptor (α, φ, θ) of point P t and its neighboring points, and divide each of the three parameters α, φ, θ into g intervals, where g is a positive integer. Count the distribution of the triple descriptor (α, φ, θ) of point P t and its neighboring points falling into the intervals, and obtain a 3×g dimensional feature vector SPFH, denoted as SPFH(P t ). The uvw coordinate system is as follows: P t The triple descriptor (α, φ, θ) of a point and its neighboring point P s is calculated as follows: where n t is the normal vector of point P t , n s is the normal vector of point P s , and d is the Euclidean distance between point P t and point P s ; For P t The neighboring point P of the point s Calculate the neighboring point P s The 3×g-dimensional feature vector SPFH of, denoted as SPFH(P s ); The SPFH(P t ) and SPFH(P s ) are weighted using the following formula to obtain the FPFH feature of Pt: where w s represents the Euclidean distance t between P s and P 4. The point cloud stitching method according to claim 1 or 2, characterized in that The based on the RANSAC algorithm, performing a preset number of screenings on a preset number of point pairs in the set of point pairs to obtain the screened point pairs, and calculating an initial transformation matrix based on the screened point pairs includes: Randomly screen m point pairs in the set of point pairs, and calculate a first initial transformation matrix based on the RANSAC algorithm; calculate the distance error of other point pairs in the set of point pairs except the screened point pairs under the first initial transformation matrix, mark the points with a distance error less than the preset threshold as first inliers, mark the points with a distance error greater than the preset threshold as first outliers, and count the number of first inliers; Randomly screen m point pairs in the set of point pairs, and calculate a second initial transformation matrix based on the RANSAC algorithm; calculate the distance error of other point pairs in the set of point pairs except the screened point pairs under the second initial transformation matrix, mark the points with a distance error less than the preset threshold as second inliers, mark the points with a distance error greater than the preset threshold as second outliers, and count the number of second inliers; And so on, randomly screen m point pairs in the set of point pairs, and calculate an Nth initial transformation matrix based on the RANSAC algorithm; calculate the distance error of other point pairs in the set of point pairs except the screened point pairs under the Nth initial transformation matrix, mark the points with a distance error less than the preset threshold as Nth inliers, mark the points with a distance error greater than the preset threshold as Nth outliers, and count the number of Nth inliers; Select the Nth initial transformation matrix with the largest number of inlier points among the N samples obtained by the above statistics as the initial transformation matrix.

5. The point cloud stitching method according to claim 4, wherein The m point pairs are 3 point pairs; N is a positive integer greater than or equal to 1.

6. A point cloud stitching device, applied to a 3D scanner, characterized in that, Comprising: A sampling module, configured to obtain at least two pieces of point cloud data to be spliced, and adaptively downsample the point cloud data according to the information of the point cloud normal in the point cloud data to obtain the downsampled point cloud data; An FPFH feature calculation module, configured to calculate the FPFH features of the downsampled point cloud data; A point pair set establishment module, configured to preliminarily establish a set of point pairs for matching between the at least two pieces of point cloud data according to the FPFH features of the downsampled point cloud data; An initial transformation matrix calculation module, configured to, based on the RANSAC algorithm, perform screening on a preset number of point pairs in the point pair set for a preset number of times to obtain the screened point pairs, and calculate an initial transformation matrix based on the screened point pairs; An accurate registration module, configured to iteratively optimize the initial transformation matrix based on the ICP algorithm to obtain an optimized rotation and translation matrix, and further splice the at least two pieces of point cloud according to the optimized rotation and translation matrix; Wherein, the accurate registration module is further configured to: Based on the initial transformation matrix, the ICP algorithm combined with the point-to-plane metric calculates the loss function for the selected point pairs, and repeats iterative optimization until the change amount of the rotation and translation matrix is less than the preset change threshold and / or the error value of the loss function is less than the preset error threshold, terminates the iteration, and obtains the optimized rotation and translation matrix, and the loss function E symm is as follows: where R is the rotation matrix, T is the translation matrix, p i and q i are respectively a point in two point clouds, is the normal vector of point p i , and is the normal vector of point q i .

7. A three-dimensional scanner, comprising a three-dimensional scanner body, characterized in that, When the 3D scanner performs a 3D scanning operation, splice the point cloud data obtained by scanning using the point cloud splicing method according to any one of claims 1 to 5.

8. An electronic device, characterized in that, Comprising a processor and a memory, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the point cloud splicing method according to any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, wherein the computer program is configured to execute the point cloud splicing method according to any one of claims 1 to 5 when running.

Images