A 3D point cloud data stitching method, system, device and storage medium

Abstract

The present invention relates to the technical field of depth data stitching in image analysis, and specifically to a 3D point cloud data stitching method, system, device, and storage medium. To solve the technical problem of low stability and accuracy in point cloud stitching in the prior art, the present invention first converts the point cloud data of the object to be measured in the scanner coordinate system to the binocular optical tracker coordinate system based on the conversion matrix from the scanner to the binocular optical tracker, obtaining several groups of point cloud data to be stitched, providing a unified benchmark for subsequent point cloud stitching; then, after the several groups of point cloud data to be stitched are respectively denoised, filtered, and normalized, rough registration is first performed to quickly align the point cloud data at approximately positions, and finally fine registration is performed to finely adjust the remaining registration errors, thereby greatly improving the accuracy and stability of point cloud stitching and enabling the stitched point cloud to more accurately reflect the true shape of the object to be measured.

Description

Technical Field

[0001] The present invention relates to the technical field of depth data stitching in image analysis, and specifically to a 3D point cloud data stitching method, system, device, and storage medium. Background Art

[0002] Three-dimensional measurement has always been a research hotspot in computer vision and is also the research foundation in the field of stereo vision. Compared with other three-dimensional measurement technologies, optical laser scanners are widely used in the field of three-dimensional data reconstruction due to their advantages of flexible and simple operation and strong anti-interference ability.

[0003] However, when an optical laser scanner scans a large-scale workpiece, due to reasons such as the volume, material, and complex surface of the workpiece, it is usually necessary to perform multiple scans at different angles to obtain the complete three-dimensional point cloud data of the workpiece, and then stitch these three-dimensional point cloud data of the workpiece into a whole. However, during the process of stitching multiple three-dimensional point cloud data of the workpiece, manual intervention operations such as feature point selection and parameter adjustment are often relied on, which not only increases the workload but also may introduce errors due to the subjectivity of manual judgment, reducing the stability and accuracy of point cloud stitching. Summary of the Invention

[0004] The purpose of the present invention is to provide a 3D point cloud data stitching method, system, device, and storage medium.

[0005] The technical solution of the present invention is as follows:

[0006] A 3D point cloud data stitching method includes the following operations:

[0007] S1. The scanner scans the marked model body to obtain the point cloud data of the marked model body in the scanner coordinate system, obtaining the marked model body point cloud data; based on several first feature point clouds in the marked model body point cloud data, several spatial transformation equations are constructed to form a transformation linear equation system; based on the transformation linear equation system, the transformation matrix from the scanner to the marked model body is obtained; the binocular optical tracker captures images of the marked model body in different poses, obtaining several model body images; the coordinates of the second feature points in each model body image are obtained, obtaining the coordinate data of several second feature points in the binocular optical tracker coordinate system; based on the coordinate data of several second feature points in the binocular optical tracker coordinate system and the coordinate data of several second feature points in the marked model body coordinate system, the transformation matrix from the marked model body to the binocular optical tracker is obtained; based on the transformation matrix from the scanner to the marked model body and the transformation matrix from the marked model body to the binocular optical tracker, the transformation matrix from the scanner to the binocular optical tracker is obtained;

[0008] S2. The scanner scans the object to be measured multiple times by moving its position, obtains the point cloud data of the object to be measured in the scanner coordinate system, and obtains several groups of point cloud data of the object to be measured; based on the transformation matrix from the scanner to the binocular optical tracker, the several groups of point cloud data of the object to be measured are unified into the binocular optical tracker coordinate system to obtain several groups of point cloud data to be stitched.

[0009] S3. The several groups of point cloud data to be stitched are respectively subjected to denoising, filtering and normalization processing to obtain several groups of optimized point cloud data; after the several groups of optimized point cloud data are roughly registered and precisely registered, the point clouds in the overlapping area are subjected to point cloud fusion, and the point clouds in the non-overlapping area are merged to obtain the three-dimensional point cloud model of the object to be measured.

[0010] The specific method for obtaining the first feature point cloud in S1 is as follows: the marked model body point cloud data is subjected to Gaussian filtering and normalization processing to obtain optimized marked model body point cloud data; the curvature of each point cloud in the optimized marked model body point cloud data is obtained; the point clouds with curvature greater than the curvature threshold are used as candidate point clouds; one point cloud with the largest curvature in two adjacent point clouds with a distance less than the distance threshold is retained to obtain the feature point cloud.

[0011] The operation of constructing several spatial transformation equations based on several first feature point clouds to form a transformation linear equation system in S1 is specifically as follows: from several first feature point clouds, multiple feature point clouds with geometric relationships are selected to form a reference feature point cloud group; the feature point clouds matching the reference feature point cloud group are obtained from several first feature point clouds as the matching reference feature point cloud group; based on each pair of corresponding feature point clouds in the reference feature point cloud group and the matching reference feature point cloud group, several spatial transformation equations are constructed to form a transformation linear equation system.

[0012] The specific operation of rough registration in S3 is as follows:

[0013] Step 1. Randomly select multiple point cloud data from the first group of optimized point cloud data as the first point cloud data to obtain the first point cloud data set; obtain the point cloud data that is a duplicate point of the first point cloud data set from the second group of optimized point cloud data to form the second point cloud data set; the positions of the first group of optimized point cloud data and the second group of optimized point cloud data are adjacent.

[0014] Step 2. Based on the first point cloud data set and the second point cloud data set, obtain the rotation matrix and the translation matrix; based on the rotation matrix and the translation matrix, perform a spatial transformation on the second group of optimized point cloud data to obtain the second group of spatially transformed point cloud data.

[0015] Step 3: Obtain the corresponding point cloud distances of the duplicate points in the second set of spatially transformed point cloud data with respect to the first set of optimized point cloud data, and count the number of point clouds with point cloud distances less than the point cloud distance threshold to obtain the evaluation point quantity. If the evaluation point quantity is less than the evaluation point quantity threshold, record the rotation matrix and the translation matrix.

[0016] Repeat Steps 1 to 3 several times to obtain several rotation matrices and translation matrices. Use the rotation matrix and translation matrix corresponding to the minimum evaluation point quantity as the optimal rotation matrix and the optimal translation matrix. Based on the optimal rotation matrix and the optimal translation matrix, perform a spatial transformation on the second set of optimized point cloud data to obtain the second set of coarsely registered point cloud data.

[0017] And so on, perform the operation of obtaining the optimal rotation matrix and the optimal translation matrix on the remaining adjacent sets of optimized point cloud data to obtain several sets of remaining coarsely registered point cloud data. The first set of optimized point cloud data and the remaining sets of coarsely registered point cloud data form several sets of coarsely registered point cloud data.

[0018] The specific operation of fine registration in S3 is as follows: Use the point clouds in the first set of coarsely registered point cloud data with normal vectors greater than the normal vector threshold as geometric feature point clouds to obtain several first geometric feature point clouds. Based on the KD tree structure search method, obtain the corresponding points of several first geometric feature point clouds from the second coarsely registered point cloud data to obtain several second geometric feature point clouds. Based on several first geometric feature point clouds and second geometric feature point clouds, construct a point cloud change error term. Minimize the point cloud change error term to obtain a rigid body transformation matrix. Based on the rigid body transformation matrix, perform a spatial transformation on the second set of coarsely registered point cloud data to obtain the second set of finely registered point cloud data. And so on, perform the operation of obtaining the rigid body transformation matrix on the remaining adjacent sets of coarsely registered point cloud data to obtain several sets of remaining finely registered point cloud data. The first set of coarsely registered point cloud data and the remaining sets of finely registered point cloud data form several sets of finely registered point cloud data.

[0019] The point cloud change error term is obtained through the following formula:

[0020] ,

[0021] is the change error term, , , are the Euclidean distance weight, the plane distance weight, and the normal vector consistency weight respectively, n is the total number of the first geometric feature point clouds, is the i th first geometric feature point cloud, is the i th second geometric feature point cloud, is the rigid body transformation matrix, , , , are respectively the abscissa coefficient, ordinate coefficient, vertical coordinate coefficient, and constant term of the spatial plane equation after performing a spatial transformation on using the rigid body transformation matrix . , , are respectively 's horizontal, vertical, and vertical coordinate components. is 's normal vector. is the normal vector after performing a spatial transformation on using the rigid body transformation matrix .

[0022] The point cloud fusion at S3 is achieved by the weighted average method.

[0023] A 3D point cloud data stitching system for implementing the above-mentioned 3D point cloud data stitching method, comprising:

[0024] A conversion matrix generation module from the scanner to the binocular optical tracker, which is used for the scanner to scan the marked model body, obtain the point cloud data of the marked model body in the scanner coordinate system, and obtain the point cloud data of the marked model body; based on several first feature point clouds in the point cloud data of the marked model body, construct several spatial transformation equations to form a transformation linear equation system; based on the transformation linear equation system, obtain the conversion matrix from the scanner to the marked model body; the binocular optical tracker takes images of the marked model body in different poses to obtain several model body images; obtain the coordinates of the second feature points in each model body image to obtain the coordinate data of several second feature points in the binocular optical tracker coordinate system; based on the coordinate data of several second feature points in the binocular optical tracker coordinate system and the coordinate data of several second feature points in the marked model body coordinate system, obtain the conversion matrix from the marked model body to the binocular optical tracker; based on the conversion matrix from the scanner to the marked model body and the conversion matrix from the marked model body to the binocular optical tracker, obtain the conversion matrix from the scanner to the binocular optical tracker;

[0025] A to-be-stitched point cloud data generation module, which is used for the scanner to scan the object to be measured multiple times by moving its position, obtain the point cloud data of the object to be measured in the scanner coordinate system, and obtain several groups of point cloud data of the object to be measured; based on the conversion matrix from the scanner to the binocular optical tracker, unify several groups of point cloud data of the object to be measured to the binocular optical tracker coordinate system to obtain several groups of to-be-stitched point cloud data;

[0026] The 3D point cloud model generation module for the object to be measured is used to perform denoising, filtering, and normalization processing on several groups of point cloud data to be stitched respectively, obtaining several groups of optimized point cloud data; after rough registration and fine registration of several groups of optimized point cloud data, the point clouds in the overlapping area are fused, and the point clouds in the non-overlapping area are combined to obtain the 3D point cloud model of the object to be measured.

[0027] A 3D point cloud data stitching device includes a processor and a memory. Among them, when the processor executes the computer program stored in the memory, the above-mentioned 3D point cloud data stitching method is implemented.

[0028] A computer-readable storage medium is used to store a computer program. Among them, the computer program is executed by the processor to implement the above-mentioned 3D point cloud data stitching method.

[0029] The beneficial effects of the present invention are as follows:

[0030] The present invention provides a 3D point cloud data stitching method. First, based on scanning a marker model body by a scanner, the best conversion relationship for aligning the point cloud in the scanner coordinate system with the self-coordinate system of the marker model body is found to obtain the conversion matrix from the scanner to the marker model body; at the same time, based on the coordinates of the feature points in the images of the marker model body at different poses captured by the binocular optical tracker, the best conversion relationship for aligning the coordinate points in the binocular optical tracker coordinate system with the self-coordinate system of the marker model body is obtained to obtain the conversion matrix from the marker model body to the binocular optical tracker; and through the conversion matrix from the scanner to the marker model body and the conversion matrix from the marker model body to the binocular optical tracker, the relationship between the scanner coordinate system and the binocular optical tracker coordinate system is established to obtain the conversion matrix from the scanner to the binocular optical tracker; then, based on the conversion matrix from the scanner to the binocular optical tracker, the point cloud data of the object to be measured in the scanner coordinate system is converted to the binocular optical tracker coordinate system to obtain several groups of point cloud data to be stitched, providing a unified benchmark for subsequent point cloud stitching; finally, after several groups of point cloud data to be stitched are respectively subjected to denoising, filtering, and normalization processing, rough registration is first performed to quickly align the point cloud data in the approximate positions, and then fine registration is performed to finely adjust the remaining registration errors, thereby greatly improving the accuracy and stability of point cloud stitching and making the stitched point cloud more accurately reflect the true shape of the object to be measured. Specific embodiments

[0031] The applicable scenario of this embodiment is to use a binocular scanning system composed of a scanner and a binocular optical tracker to obtain the 3D point cloud data of the object to be measured. Among them, the scanner is located between the binocular cameras in the binocular optical tracker. The scanner is used to obtain the 3D point cloud data of the target object, and the binocular optical tracker is used to obtain the photos of the target object.

[0032] S1. The scanner scans the marked model body to obtain the point cloud data of the marked model body in the scanner coordinate system, resulting in the point cloud data of the marked model body. Based on several first feature point clouds in the point cloud data of the marked model body, several spatial transformation equations are constructed to form a transformation linear equation system. Based on the transformation linear equation system, the transformation matrix from the scanner to the marked model body is obtained. The binocular optical tracker captures images of the marked model body in different poses, resulting in several model body images. The coordinates of the second feature points in each model body image are obtained, resulting in the coordinate data of several second feature points in the binocular optical tracker coordinate system. Based on the coordinate data of several second feature points in the binocular optical tracker coordinate system and the coordinate data of several second feature points in the marked model body coordinate system, the transformation matrix from the marked model body to the binocular optical tracker is obtained.

[0033] Based on the transformation matrix from the scanner to the marked model body and the transformation matrix from the marked model body to the binocular optical tracker, the transformation matrix from the scanner to the binocular optical tracker is obtained.

[0034] Based on the scanner scanning the marked model body to find the best transformation relationship that can align the point cloud in the scanner coordinate system with the coordinate system of the marked model body itself, the transformation matrix from the scanner to the marked model body is obtained. At the same time, based on the coordinates of the feature points in the images of the marked model body in different poses captured by the binocular optical tracker, the best transformation relationship that aligns the coordinate points in the binocular optical tracker coordinate system with the coordinate system of the marked model body itself is obtained, and the transformation matrix from the marked model body to the binocular optical tracker is obtained. And through the transformation matrix from the scanner to the marked model body and the transformation matrix from the marked model body to the binocular optical tracker, the relationship between the scanner coordinate system and the binocular optical tracker coordinate system is established, and the transformation matrix from the scanner to the binocular optical tracker is obtained, which is convenient for subsequent conversion of the point cloud data obtained by the scanner to the binocular optical tracker coordinate system.

[0035] S1.1. Obtain the transformation matrix from the scanner to the marked model body.

[0036] First, use a scanner with calibrated external parameters to scan the marked model body to obtain the point cloud data of the marked model body in the scanner coordinate system, resulting in the point cloud data of the marked model body. It is preferable to scan the marked model body multiple times from different angles and positions to obtain more comprehensive point cloud information of the marked model body and ensure that the features of the marked model body can be accurately captured.

[0037] Then, based on several first feature point clouds in the point cloud data of the marked model body, several spatial transformation equations are constructed to form a transformation linear equation system. The first feature point clouds have uniqueness and identifiability, such as specific patterns, corner points, etc. on the marked model body.

[0038] Among them, the method for obtaining the first feature point cloud is specifically as follows: The point cloud data of the marked model body is subjected to Gaussian filtering and normalization processing. After denoising the point cloud data, the coordinate values of the point cloud data are mapped to a suitable range to eliminate the influence of the data dimension of different dimensions and facilitate subsequent calculations, obtaining optimized marked model body point cloud data; obtaining the curvature of each point cloud in the optimized marked model body point cloud data; taking the point clouds with curvatures greater than the curvature threshold, which usually correspond to the edge, corner and other feature positions of the marked model body, as candidate point clouds; to avoid over-concentration of candidate point clouds, retain the point cloud with the largest curvature among two adjacent point clouds with a distance less than the distance threshold, that is, if the distance between two adjacent point clouds is less than the distance threshold, then retain the point cloud with the largest curvature among the two adjacent point clouds, obtaining a number of feature point clouds.

[0039] The curvature of the above point cloud can be obtained by fitting the quadratic surface formed by the current point cloud and the point cloud within the neighborhood range by the least squares method.

[0040] In addition, based on a number of first feature point clouds, the operation of constructing a number of spatial transformation equations to form a transformation linear equation system is specifically as follows: From a number of first feature point clouds, select a number of feature point clouds with geometric relationships among them. These feature point clouds have unique spatial positions and geometric features and can be accurately identified from different perspectives, forming a reference feature point cloud group; based on the descriptor matching method, obtain the feature point clouds matching the reference feature point cloud group from a number of first feature point clouds as the matching reference feature point cloud group; based on each pair of corresponding feature point clouds in the reference feature point cloud group and the matching reference feature point cloud group, construct a number of spatial transformation equations. The spatial transformation equation formula is , is the i th feature point cloud in the matching reference feature point cloud group, is the i th feature point cloud in the reference feature point cloud group, is 's matching point, is the rotation matrix, is the translation vector, and a number of spatial transformation equations form a transformation linear equation system.

[0041] Finally, based on the transformation linear equation system, the conversion matrix from the scanner to the marked model body is obtained. Specifically, the transformation linear equation system is processed by the least squares method to obtain the rotation matrix and the translation vector , based on the rotation matrix and the translation vector , the conversion matrix from the scanner to the marked model body is obtained.

[0042] S1.2. Obtain the conversion matrix from the marked model body to the binocular optical tracker.

[0043] First, the binocular optical tracker captures images of the marker model body in different poses, ensuring that all angles and features of the marker model body can be clearly captured during shooting, and the lighting conditions are relatively stable to avoid situations such as shadows or reflections that affect the image quality, thereby obtaining several model body images. Specifically, the binocular optical tracker shoots the marker model body from different poses, indirectly obtaining images of the marker model body in different poses and obtaining several model body images.

[0044] Then, obtain the coordinates of the second feature points in each model body image, obtaining the coordinate data of several second feature points in the coordinate system of the binocular optical tracker. The second feature points can be corner points on the marker model body and can be obtained by processing the model body image (a random one of the several model body images or the first model body image captured) using a corner detection method (including but not limited to the Harris corner detection method). To accurately extract the second feature points, before processing the model body image with the corner detection method, it also includes grayscale processing of the model body image, converting the color image to a grayscale image to reduce the amount of data and obtaining the grayscale model body image; the grayscale model body image is processed by median filtering to improve the clarity and quality of the image and obtain the filtered model body image; the filtered model body image is processed by an edge detection algorithm to highlight the contour and features of the marker model body and obtain the edge-enhanced model body image for performing the operation of the corner detection method.

[0045] Finally, based on the coordinate data of several second feature points in the coordinate system of the binocular optical tracker and the coordinate data of several second feature points in the coordinate system of the marker model body, obtain the transformation matrix from the marker model body to the binocular optical tracker. Specifically, based on the coordinate data of each second feature point in the coordinate system of the binocular optical tracker and its coordinate data in the coordinate system of the marker model body, construct several linear equations to form a system of linear equations; the system of linear equations is decomposed by singular value decomposition to obtain the transformation matrix parameters; based on the transformation matrix parameters, obtain the transformation matrix from the marker model body to the binocular optical tracker.

[0046] In addition, to improve the accuracy of the transformation matrix from the marker model body to the binocular optical tracker, it also includes using an iterative algorithm (including but not limited to the Levenberg - Marquardt algorithm) to optimize the transformation matrix from the marker model body to the binocular optical tracker, improving the accuracy of the transformation matrix and obtaining the optimized transformation matrix from the marker model body to the binocular optical tracker for performing the subsequent operation of obtaining the transformation matrix from the scanner to the binocular optical tracker.

[0047] S1.3. Obtain the transformation matrix from the scanner to the binocular optical tracker.

[0048] Based on the transformation matrix from the scanner to the marker model body and the transformation matrix from the marker model body to the binocular optical tracker, the transformation matrix from the scanner to the binocular optical tracker is obtained. Specifically, the transformation matrix from the scanner to the marker model body is multiplied by the transformation matrix from the marker model body to the binocular optical tracker to obtain the transformation matrix from the scanner to the binocular optical tracker.

[0049] S2. The scanner scans the object to be measured multiple times by moving its position, obtains the point cloud data of the object to be measured in the scanner coordinate system, and gets several groups of point cloud data of the object to be measured. Based on the transformation matrix from the scanner to the binocular optical tracker, the several groups of point cloud data of the object to be measured are unified into the binocular optical tracker coordinate system to obtain several groups of point cloud data to be stitched.

[0050] Based on the transformation matrix from the scanner to the binocular optical tracker, the point cloud data of the object to be measured in the scanner coordinate system is transformed into the binocular optical tracker coordinate system, obtaining several groups of point cloud data to be stitched, providing a unified benchmark for subsequent point cloud stitching, facilitating the precise positioning and matching of point cloud data obtained at different positions and angles in the same coordinate system, reducing the error accumulation caused by inconsistent coordinate systems, and improving the accuracy of stitching.

[0051] The scanner scans the object to be measured multiple times by moving its position, obtains the point cloud data of the object to be measured in the scanner coordinate system, and gets several groups of point cloud data of the object to be measured to ensure the integrity of the point cloud data of the object to be measured.

[0052] Based on the transformation matrix from the scanner to the binocular optical tracker, the several groups of point cloud data of the object to be measured are unified into the binocular optical tracker coordinate system to obtain several groups of point cloud data to be stitched. Specifically, the several groups of point cloud data of the object to be measured are respectively multiplied by the transformation matrix from the scanner to the binocular optical tracker to obtain several groups of point cloud data to be stitched.

[0053] S3. The several groups of point cloud data to be stitched are respectively subjected to denoising, filtering, and normalization processing to obtain several groups of optimized point cloud data. After rough registration and fine registration of the several groups of optimized point cloud data, the point clouds in the overlapping area are fused, and the point clouds in the non - overlapping area are merged to obtain the three - dimensional point cloud model of the object to be measured.

[0054] After the several groups of point cloud data to be stitched are respectively subjected to denoising, filtering, and normalization processing, rough registration is first performed to quickly align the point cloud data in the approximate positions, and then fine registration is performed to finely adjust the remaining registration errors, thus greatly improving the accuracy of point cloud stitching and making the stitched point cloud more accurately reflect the true shape of the object to be measured.

[0055] First, several groups of point cloud data to be spliced are respectively denoised, filtered, and normalized. After smoothing the point cloud data and reducing noise, the coordinate values of the point cloud data are mapped to a certain range to obtain several groups of optimized point cloud data.

[0056] Then, several groups of optimized point cloud data are roughly registered to effectively process point cloud data containing noise and outliers. The specific operations of rough registration are as follows.

[0057] Step 1: Randomly select multiple point cloud data from the first group of optimized point cloud data as the first point cloud data to obtain the first point cloud data set; obtain the point cloud data that is a duplicate point of the first point cloud data set from the second group of optimized point cloud data to form the second point cloud data set.

[0058] Step 2: Based on the first point cloud data set and the second point cloud data set, obtain the rotation matrix and the translation matrix; based on the rotation matrix and the translation matrix, perform a spatial transformation on the second group of optimized point cloud data to obtain the second group of spatially transformed point cloud data.

[0059] Step 3: Obtain the corresponding point cloud distances of the points that are duplicate points of the first group of optimized point cloud data in the second group of spatially transformed point cloud data, count the number of point clouds with point cloud distances less than the point cloud distance threshold to obtain the evaluation point number; if the evaluation point number is less than the evaluation point number threshold, record the rotation matrix and the translation matrix.

[0060] Repeat Step 1 to Step 3 several times to obtain several rotation matrices and translation matrices; take the rotation matrix and the translation matrix corresponding to the minimum evaluation point number as the optimal rotation matrix and the optimal translation matrix; based on the optimal rotation matrix and the optimal translation matrix, perform a spatial transformation on the second group of optimized point cloud data to obtain the second group of roughly registered point cloud data, realizing the rough registration of the first group of optimized point cloud data and the second group of optimized point cloud data.

[0061] And so on, perform the operation of obtaining the optimal rotation matrix and the optimal translation matrix on the remaining adjacent groups of optimized point cloud data to obtain several groups of remaining roughly registered point cloud data; the first group of optimized point cloud data and the remaining groups of roughly registered point cloud data form several groups of roughly registered point cloud data.

[0062] Next, several groups of roughly registered point cloud data are finely registered, and the fine registration operation can be realized by the ICP (Iterative Closest Point) method.

[0063] In addition, to improve the registration accuracy, the specific operation of fine registration can also be as follows: taking the point clouds in the first set of roughly registered point cloud data (the first set of optimized point cloud data) whose normal vectors are greater than the normal vector threshold as geometric feature point clouds, these point clouds have obvious geometric features, and several first geometric feature point clouds are obtained; based on the KD-tree structure search method, obtaining the corresponding points of several first geometric feature point clouds from the second roughly registered point cloud data, these points have feature space similarity with the geometric feature point clouds, and several second geometric feature point clouds are obtained; based on several first geometric feature point clouds and second geometric feature point clouds, constructing a point cloud change error term; minimizing the point cloud change error term to obtain a rigid body transformation matrix; based on the rigid body transformation matrix, performing a spatial transformation on the second set of roughly registered point cloud data to achieve the fine registration of the second set of roughly registered point cloud data and obtain the second set of finely registered point cloud data; and so on, performing the operation of obtaining the rigid body transformation matrix on the remaining adjacent sets of roughly registered point cloud data to obtain several remaining sets of finely registered point cloud data; the first set of roughly registered point cloud data (the first set of optimized point cloud data) and the remaining sets of finely registered point cloud data form several sets of finely registered point cloud data.

[0064] The point cloud change error term is obtained through the following formula:

[0065] ,

[0066] is the change error term, , , are the Euclidean distance weight, the plane distance weight, and the normal vector consistency weight respectively, n is the total number of the first geometric feature point clouds, is the i rd first geometric feature point cloud, is the i th second geometric feature point cloud, is the rigid body transformation matrix, , , , are respectively the abscissa coefficient, the ordinate coefficient, the vertical coordinate coefficient, and the constant term of the spatial plane equation after performing a spatial transformation on based on the rigid body transformation matrix , , , are respectively the abscissa, ordinate, and vertical coordinate components, is the distance from the first geometric feature point cloud to the plane where the corresponding spatially transformed second geometric feature point cloud is located; among them, for each spatially transformed second geometric feature point cloud, a plane is fitted by its neighborhood points to obtain the plane equation 0, methods such as the least squares method can be used for plane fitting (if the spatial coordinate positions of the second geometric feature point cloud are different, the , , , will have different values), and then for the first geometric feature point cloud, according to the distance formula from a point to a plane, can be calculated. is 's normal vector, is the normal vector after spatially transforming based on the rigid body transformation matrix .

[0067] Finally, the point clouds in the overlapping regions of several groups of finely registered point cloud data are fused (which can be achieved based on the weighted average method), and the point clouds in the non - overlapping regions are combined to obtain the three - dimensional point cloud model of the object to be measured.

[0068] This embodiment also provides a 3D point cloud data stitching system for implementing the above - mentioned 3D point cloud data stitching method, including:

[0069] A conversion matrix generation module from the scanner to the binocular optical tracker, which is used for the scanner to scan the marked model body, obtain the point cloud data of the marked model body in the scanner coordinate system, and get the point cloud data of the marked model body; based on several first feature point clouds in the point cloud data of the marked model body, construct several spatial transformation equations to form a transformation linear equation system; based on the transformation linear equation system, obtain the conversion matrix from the scanner to the marked model body; the binocular optical tracker takes pictures of the marked model body in different poses to get several model body images; obtain the coordinates of the second feature points in each model body image to get the coordinate data of several second feature points in the binocular optical tracker coordinate system; based on the coordinate data of several second feature points in the binocular optical tracker coordinate system and the coordinate data of several second feature points in the marked model body coordinate system, obtain the conversion matrix from the marked model body to the binocular optical tracker; based on the conversion matrix from the scanner to the marked model body and the conversion matrix from the marked model body to the binocular optical tracker, obtain the conversion matrix from the scanner to the binocular optical tracker;

[0070] A module for generating point cloud data to be stitched, which is used for the scanner to scan the object to be measured multiple times by moving its position, obtain the point cloud data of the object to be measured in the scanner coordinate system, and get several groups of point cloud data of the object to be measured; based on the conversion matrix from the scanner to the binocular optical tracker, unify several groups of point cloud data of the object to be measured into the binocular optical tracker coordinate system to get several groups of point cloud data to be stitched;

[0071] The 3D point cloud model generation module for the object to be measured is used to perform denoising, filtering, and normalization processing on several groups of point cloud data to be spliced, and obtain several groups of optimized point cloud data; after several groups of optimized point cloud data are subjected to rough registration and fine registration, the point clouds in the overlapping areas are fused, and the point clouds in the non-overlapping areas are merged to obtain the 3D point cloud model of the object to be measured.

[0072] This embodiment also provides a 3D point cloud data splicing device, including a processor and a memory. Among them, when the processor executes the computer program stored in the memory, the above-mentioned 3D point cloud data splicing method is implemented.

[0073] This embodiment also provides a computer-readable storage medium for storing a computer program. Among them, the computer program is executed by the processor to implement the above-mentioned 3D point cloud data splicing method.

[0074] This embodiment provides a 3D point cloud data splicing method. First, based on the scanner scanning the marker model body, the best conversion relationship that can align the point cloud in the scanner coordinate system with the coordinate system of the marker model body itself is found, and the conversion matrix from the scanner to the marker model body is obtained; at the same time, based on the coordinates of the feature points in the images of the marker model body at different poses captured by the binocular optical tracker, the best conversion relationship that aligns the coordinate points in the binocular optical tracker coordinate system with the coordinate system of the marker model body itself is obtained, and the conversion matrix from the marker model body to the binocular optical tracker is obtained; and through the conversion matrix from the scanner to the marker model body and the conversion matrix from the marker model body to the binocular optical tracker, the relationship between the scanner coordinate system and the binocular optical tracker coordinate system is established, and the conversion matrix from the scanner to the binocular optical tracker is obtained; then, based on the conversion matrix from the scanner to the binocular optical tracker, the point cloud data of the object to be measured in the scanner coordinate system is converted to the binocular optical tracker coordinate system, and several groups of point cloud data to be spliced are obtained, providing a unified benchmark for subsequent point cloud splicing; finally, after several groups of point cloud data to be spliced are respectively subjected to denoising, filtering, and normalization processing, rough registration is first performed to quickly align the point cloud data in the approximate positions, and then fine registration is performed to finely adjust the remaining registration errors, thereby greatly improving the accuracy and stability of point cloud splicing, and making the spliced point cloud more accurately reflect the true shape of the object to be measured.

Claims

1. A 3D point cloud data splicing method, characterized in that: The following operations are included: S1. The scanner scans the marked model body to obtain point cloud data of the marked model body in the scanner coordinate system, and obtains the marked model body point cloud data; based on a plurality of first feature point clouds in the marked model body point cloud data, a plurality of space transformation equations are constructed to form a transformation linear equation group; based on the transformation linear equation group, a transformation matrix from the scanner to the marked model body is obtained; The binocular optical tracker takes images of the marked model body in different postures to obtain a plurality of model body images; the coordinates of the second feature points in each model body image are obtained to obtain the coordinate data of the plurality of second feature points in the binocular optical tracker coordinate system; based on the coordinate data of the plurality of second feature points in the binocular optical tracker coordinate system and the coordinate data of the plurality of second feature points in the marked model body coordinate system, a transformation matrix from the marked model body to the binocular optical tracker is obtained; Based on the transformation matrix from the scanner to the marked model body and the transformation matrix from the marked model body to the binocular optical tracker, a transformation matrix from the scanner to the binocular optical tracker is obtained; S2, the scanner scans the object to be measured multiple times by moving the position, obtains the point cloud data of the object to be measured in the scanner coordinate system, and obtains several groups of point cloud data of the object to be measured; Based on the conversion matrix from the scanner to the binocular optical tracker, several groups of point cloud data of the objects to be measured are unified into the binocular optical tracker coordinate system to obtain several groups of point cloud data to be spliced; S3. Several groups of point cloud data to be spliced ​​are subjected to denoising, filtering and normalization processing respectively to obtain several groups of optimized point cloud data; after several groups of optimized point cloud data are subjected to coarse alignment and fine alignment, the point clouds in the overlapping areas are fused, and the point clouds in the non-overlapping areas are merged to obtain a three-dimensional point cloud model of the object to be measured.

2. A 3D point cloud data splicing method according to claim 1, characterized in that: In S1, the method for obtaining the first feature point cloud is specifically as follows: The marked model body point cloud data is subjected to Gaussian filtering and normalization processing to obtain optimized marked model body point cloud data; the curvature of each point cloud in the optimized marked model body point cloud data is obtained; the point cloud with a curvature greater than a curvature threshold is used as a candidate point cloud; the point cloud with the largest curvature among two adjacent point clouds with a distance less than a distance threshold is retained to obtain a feature point cloud.

3. The 3D point cloud data splicing method according to claim 1, characterized in that: In S1, based on a plurality of first feature point clouds, a plurality of spatial transformation equations are constructed to form a transformation linear equation group. The specific operation is: From a number of first feature point clouds, multiple feature point clouds with geometric relationships are selected to form a benchmark feature point cloud group; feature point clouds matching the benchmark feature point cloud group are obtained from a number of first feature point clouds as a matching benchmark feature point cloud group; based on each corresponding pair of feature point clouds in the benchmark feature point cloud group and the matching benchmark feature point cloud group, a number of space transformation equations are constructed to form a transformation linear equation group.

4. The 3D point cloud data splicing method according to claim 1, characterized in that: In S3, the specific operation of coarse registration is as follows: Step 1: randomly select multiple point cloud data from the first set of optimized point cloud data as the first point cloud data to obtain a first point cloud data set; obtain point cloud data that are duplicate points of the first point cloud data set from the second set of optimized point cloud data to form a second point cloud data set; The positions of the first set of optimized point cloud data and the second set of optimized point cloud data are adjacent; Step 2: Based on the first point cloud data set and the second point cloud data set, obtain a rotation matrix and a translation matrix; Based on the rotation matrix and the translation matrix, the second set of optimized point cloud data is spatially transformed to obtain a second set of spatially transformed point cloud data; Step 3, obtaining the corresponding point cloud distances of the second set of spatial transformation point cloud data that are duplicate points with the first set of optimized point cloud data, counting the number of point clouds whose point cloud distances are less than the point cloud distance threshold, and obtaining the number of evaluation points; if the number of evaluation points is less than the evaluation point number threshold, recording the rotation matrix and the translation matrix; Repeat steps 1 to 3 several times to obtain several rotation matrices and translation matrices; take the rotation matrix and translation matrix corresponding to the minimum number of evaluation points as the optimal rotation matrix and optimal translation matrix; Based on the optimal rotation matrix and the optimal translation matrix, the second set of optimized point cloud data is spatially transformed to obtain a second set of roughly registered point cloud data; By analogy, the remaining adjacent groups of optimized point cloud data perform operations to obtain the optimal rotation matrix and the optimal translation matrix to obtain several remaining groups of roughly aligned point cloud data; the first group of optimized point cloud data and the remaining groups of roughly aligned point cloud data form several groups of roughly aligned point cloud data.

5. The 3D point cloud data splicing method according to claim 1, characterized in that: In S3, the precise registration operation is as follows: The point clouds whose normal vectors are greater than the normal vector threshold in the first group of coarse registration point cloud data are used as geometric feature point clouds to obtain a number of first geometric feature point clouds; based on the KD tree structure search method, a number of corresponding points of the first geometric feature point clouds are obtained from the second coarse registration point cloud data to obtain a number of second geometric feature point clouds; Based on a plurality of first geometric feature point clouds and a second geometric feature point cloud, a point cloud change error term is constructed; and the point cloud change error term is minimized to obtain a rigid body transformation matrix; Based on the rigid body transformation matrix, the second group of coarsely registered point cloud data is spatially transformed to obtain the second group of finely registered point cloud data; By analogy, the remaining adjacent groups of coarsely registered point cloud data perform the operation of obtaining the rigid body transformation matrix to obtain several remaining groups of finely registered point cloud data; the first group of coarsely registered point cloud data and the remaining groups of finely registered point cloud data form several groups of finely registered point cloud data.

6. A 3D point cloud data splicing method according to claim 5, characterized in that: The point cloud change error term is obtained by the following formula: ， is the variation error term, , , They are Euclidean distance weight, plane distance weight, and normal vector consistency weight. n is the total number of the first geometric feature point cloud, For the i The first geometric feature point cloud, For the i The second geometric feature point cloud, is the rigid body transformation matrix, , , , Based on the rigid body transformation matrix Will The horizontal coordinate coefficient, vertical coordinate coefficient, vertical coordinate coefficient and constant term of the spatial plane equation after spatial transformation. , , They are The horizontal, vertical and vertical coordinate components of for The normal vector of Based on the rigid body transformation matrix Will The normal vector after the space transformation.

7. The 3D point cloud data splicing method according to claim 1, characterized in that: In S3, point cloud fusion is achieved through weighted averaging method.

8. A 3D point cloud data stitching system, used to implement the 3D point cloud data stitching method according to claim 1, characterized in that: include: The conversion matrix generation module from scanner to binocular optical tracker is used for the scanner to scan the marked model body, obtain the point cloud data of the marked model body in the scanner coordinate system, and obtain the marked model body point cloud data; based on several first feature point clouds in the marked model body point cloud data, construct several space transformation equations to form a transformation linear equation group; based on the transformation linear equation group, obtain the conversion matrix from scanner to marked model body; the binocular optical tracker takes images of the marked model body in different postures to obtain several model body images; obtain the coordinates of the second feature point in each model body image to obtain the coordinate data of several second feature points in the binocular optical tracker coordinate system; based on the coordinate data of several second feature points in the binocular optical tracker coordinate system and the coordinate data of several second feature points in the marked model body coordinate system, obtain the conversion matrix from the marked model body to the binocular optical tracker; based on the conversion matrix from scanner to marked model body and the conversion matrix from marked model body to binocular optical tracker, obtain the conversion matrix from scanner to binocular optical tracker; The module for generating point cloud data to be spliced ​​is used for the scanner to scan the object to be measured multiple times by moving the position, obtaining the point cloud data of the object to be measured in the scanner coordinate system, and obtaining several groups of point cloud data of the object to be measured; Based on the conversion matrix from the scanner to the binocular optical tracker, several groups of point cloud data of the objects to be measured are unified into the binocular optical tracker coordinate system to obtain several groups of point cloud data to be spliced; The module for generating the three-dimensional point cloud model of the object to be measured is used to perform denoising, filtering and normalization on several groups of point cloud data to be spliced, and obtain several groups of optimized point cloud data; After coarse and fine registration of several groups of optimized point cloud data, the point clouds in the overlapping areas are fused, and the point clouds in the non-overlapping areas are merged to obtain a three-dimensional point cloud model of the object to be measured.

9. A 3D point cloud data splicing device, characterized in that: The method comprises a processor and a memory, wherein the processor implements the 3D point cloud data stitching method according to any one of claims 1 to 7 when executing the computer program stored in the memory.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the 3D point cloud data stitching method according to any one of claims 1 to 7 is implemented.

Images