A three-dimensional point cloud reconstruction method, device and computer-readable storage medium
Point cloud registration is carried out through binocular camera model and complementary Grey code technology, and combined with denoising and clustering algorithms, the problems of point cloud registration accuracy and noise filtering are solved, and high-precision three-dimensional reconstruction is achieved.
Patent Information
- Application Number
- CN202410786283.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-18
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2044-06-18
AI Technical Summary
The existing three-dimensional reconstruction methods have few overlapping areas between point clouds at different angles during point cloud registration, resulting in poor registration accuracy and difficult to effectively filter out dense noise, affecting the reconstruction accuracy.
The binocular camera model is used to combine phase shift method and complementary Gray code technology to obtain point cloud data, phase matching is performed through polar line correction and global local search, a similarity matrix is constructed, and the rigid body transformation matrix is obtained by using singular value decomposition, and point cloud registration is performed , and extract the main point cloud through distance and radius denoising and clustering.
It improves the accuracy and reconstruction accuracy of point cloud registration, effectively removes noise and redundant information, and ensures seamless connection and efficient reconstruction of point clouds at different angles.
Smart Images

Figure CN118365802B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision technology, and in particular to a three-dimensional point cloud reconstruction method, device and computer-readable storage medium. Background Art
[0002] With the continuous development of industrial technology, the structural complexity and precision requirements of key components commonly used in industry are increasing. Therefore, accurate 3D reconstruction of key components is of great significance for the design, manufacturing, performance optimization and fault diagnosis of key components. However, due to the complexity of the structure of key components and the stringent precision requirements, traditional 3D reconstruction technology often cannot meet actual needs.
[0003] With the continuous advancement of science and technology, point cloud reconstruction technology has made significant progress and has become an important means of obtaining surface information of objects. Point cloud data, as a data set composed of a large number of spatial coordinate points, can accurately depict the three-dimensional shape of objects, especially in the measurement of complex structures such as key industrial components. However, the processing of point cloud data faces many challenges.
[0004] When acquiring the three-dimensional information of key components, it is often necessary to scan from multiple different perspectives. However, how to accurately align and fuse the point cloud data of key components collected from different perspectives has become a technical problem that needs to be solved urgently. Point cloud registration technology was born to solve this problem. It aims to ensure that point cloud data from different perspectives can be seamlessly connected in space, thereby constructing a complete and accurate three-dimensional model. However, the existing point cloud registration methods are difficult to accurately identify and match when there are few overlapping regional features between point clouds at different angles, thus affecting the accuracy of registration.
[0005] In the actual acquisition process, due to the multiple influences of equipment accuracy, environmental factors and the complexity of the surface of key components, the original point cloud data of key components will inevitably be mixed with a large amount of noise and redundant information. These noises and redundant data not only increase the complexity and difficulty of data processing, but also interfere with subsequent tasks such as point cloud registration and dimension measurement, thereby affecting the accuracy and efficiency of the results. Therefore, effective filtering of point cloud data to remove noise and redundant information has become an indispensable part of point cloud processing. However, the existing point cloud data denoising methods mainly focus on tasks such as removing outliers and smoothing point cloud data. They are not effective enough when processing dense noise points and cannot filter out dense noise points, resulting in low reconstruction accuracy, which in turn leads to the inclusion of outliers and noise points in the point cloud data. These points may interfere with the registration results and reduce the accuracy of registration.
[0006] In summary, the existing 3D reconstruction methods are difficult to accurately identify and match when there are few overlapping features between point clouds at different angles, which affects the accuracy of registration and leads to poor accuracy of the acquired 3D point clouds. In addition, the point clouds at different angles used for registration cannot filter out dense noise points after acquisition, resulting in low reconstruction accuracy, which makes the outliers and noise points contained in the reconstructed point clouds, and further leads to poor registration accuracy when the reconstructed point clouds are registered. Summary of the invention
[0007] Therefore, the technical problem to be solved by the present invention is to overcome the problems of low point cloud reconstruction accuracy and poor registration accuracy in the prior art.
[0008] In order to solve the above technical problems, the present invention provides a three-dimensional point cloud reconstruction method, comprising:
[0009] After obtaining the first-angle point cloud data and the second-angle point cloud data of the object to be reconstructed, denoising and clustering are performed respectively to obtain the first-angle main body point cloud and the second-angle main body point cloud;
[0010] The acquisition of the first-angle point cloud data and the second-angle point cloud data includes: establishing a binocular camera model, using checkerboard calibration to obtain the internal and external parameter matrix of the binocular camera; using the phase shift method combined with the complementary Gray code technology to generate a coding pattern; projecting the coding pattern onto the surface of the object to be reconstructed, and using the binocular camera to collect and obtain the left-eye grating image and the right-eye grating image of the object to be reconstructed from a preset angle; performing phase unwrapping on the left-eye grating image and the right-eye grating image respectively to extract the left-eye phase information and the right-eye phase information; based on the left-eye phase information and the right-eye phase information, using epipolar correction, lateral constraints and global local search, phase matching is performed on the left-eye grating image and the right-eye grating image to obtain matching pixel points; based on the matching pixel points and the internal and external parameter matrix of the binocular camera, using the triangulation principle, the three-dimensional spatial coordinates of the matching pixel points are calculated to obtain the point cloud data of the object to be reconstructed at the preset angle;
[0011] Constructing a similarity matrix based on preset matching point pairs between the first angle subject point cloud and the second angle subject point cloud;
[0012] The spectral decomposition technique is used to obtain the matching confidence of each pair of matching points in the similarity matrix, and the matching point pair with the local maximum confidence is used as the matching seed point pair;
[0013] Binarize the similarity matrix, count all matching point pairs in the matching seed point pairs that are compatible with the binarized similarity matrix, and construct a high-order spatial matrix;
[0014] Calculate the similarity of all matching point pairs in the high-order spatial matrix, obtain the preset number of matching point pairs with the highest similarity, and form a target matching point pair set;
[0015] Based on the target matching point pair set, the singular value decomposition method is used to obtain the rigid body transformation matrix, and then the optimal transformation matrix is obtained using the interior point counting criterion;
[0016] Based on the optimal transformation matrix, the first angle point cloud data is transformed to obtain the optimal initial pose point cloud as the three-dimensional point cloud of the object to be reconstructed.
[0017] Preferably, performing phase unwrapping on the grating image comprises:
[0018] The phase shift method is used to obtain the unwrapped phase, and four fringe images with a phase shift difference of π / 2 are collected, which can be expressed as:
[0019] ;
[0020] Let the encoding values of the initial four Gray codes be , the newly added complementary Gray code value is , based on complementary Gray code and phase shift method to solve the absolute phase value , expressed as:
[0021] ;
[0022] in, Represents the unwrapped phase value obtained using the phase shift method, and the expression is
[0023] ; To preset the background light intensity, To preset the modulated light intensity, , , , They respectively represent the grayscale values corresponding to the pixels in the four fringe images.
[0024] Preferably, after obtaining the optimal initial pose point cloud, the method further includes:
[0025] The optimal initial pose point cloud and the second angle point cloud data are uniformly downsampled to obtain the optimal downsampled point cloud and the second downsampled point cloud;
[0026] For the optimal down-sampled point cloud and the second down-sampled point cloud, the KD tree is used to obtain the neighboring points of each point in the point cloud and the FPFH features of the neighboring points, and feature matching is performed to obtain the corresponding point pairs between the optimal down-sampled point cloud and the second down-sampled point cloud, and a corresponding point pair set is constructed;
[0027] Based on the corresponding point pair set, a rigid body transformation matrix is obtained; based on the rigid body transformation matrix, the optimal initial pose point cloud is transformed to obtain an updated optimal down-sampling point cloud;
[0028] Calculate and iteratively optimize based on the root mean square error between the updated optimal down-sampled point cloud and the second angle point cloud data, update the corresponding point pair set, rigid body transformation matrix and optimal down-sampled point cloud until the root mean square error is less than the preset threshold, obtain the optimal rigid body transformation matrix, act on the optimal initial pose point cloud, realize point cloud registration, and obtain the 3D reconstructed point cloud of the object to be reconstructed.
[0029] Preferably, the root mean square error between the updated optimal down-sampled point cloud and the second angle point cloud data is expressed as:
[0030] ;
[0031] in, represents the point in the updated optimal downsampled point cloud, Indicates the second angle point cloud data The corresponding points, Represents the total number of point pairs in the corresponding point pair set.
[0032] Preferably, the method of using spectral decomposition technology to obtain the matching confidence of each pair of matching point pairs in the similarity matrix and using the matching point pair with the local maximum confidence as the matching seed point pair includes:
[0033] The similarity matrix is decomposed by spectral decomposition technology to obtain the eigenvector and eigenvalue of each pair of matching points;
[0034] Select the eigenvector with the largest eigenvalue as the main eigenvector:
[0035] Divide each element of the principal eigenvector by the modulus of the vector to obtain the principal eigenvector;
[0036] Calculate the dot product of the feature vector of each pair of matching points and the main feature unit vector as the matching confidence of the pair of matching points;
[0037] Based on the matching confidence of each pair of matching point pairs in the similarity matrix, all matching point pairs whose matching confidence is higher than the matching confidence of their adjacent matching point pairs in the similarity matrix are obtained as matching seed point pairs.
[0038] Preferably, the first-angle point cloud data and the second-angle point cloud data of the object to be reconstructed are denoised, including distance denoising and radius denoising.
[0039] Preferably, performing distance denoising on the first angle point cloud data and the second angle point cloud data includes:
[0040] Set the number of neighboring points ,according to The neighborhood algorithm searches for each point in the point cloud data. nearest neighbor points;
[0041] Calculate each point and its The mean of the distances to the nearest neighbors , expressed as: ;
[0042] Based on each point to its The distances to the nearest neighbors, as well as the mean of the distances, calculate the standard deviation , expressed as: ;
[0043] Based on mean , Standard Deviation Multiple of the preset standard deviation , the preset average distance range is , the points whose mean is not in the average distance range are identified as noise points and removed to obtain the distance denoised point cloud;
[0044] in, Indicates Point and its The distance between the nearest neighbor points is expressed as , ; Indicates The three-dimensional coordinates of the points, , Indicates the total number of points in the point cloud data; Indicates The 3D coordinates of the nearest neighbor points.
[0045] Preferably, radius denoising is performed on the first angle point cloud data and the second angle point cloud data, both including:
[0046] For each point in the distance denoised point cloud, draw a neighborhood sphere for each point with itself as the center and the preset neighborhood search radius as the radius;
[0047] All points that fall within the neighborhood sphere are regarded as the neighborhood points of the point;
[0048] The number of neighborhood points of each point in the distance denoised point cloud is obtained, and the points whose number of neighborhood points is less than the preset neighborhood point threshold are identified as outliers and removed to obtain the radius denoised point cloud.
[0049] Preferably, the acquisition of the first-angle subject point cloud and the second-angle subject point cloud both includes:
[0050] The cluster neighborhood radius is preset, and for each point in the radius denoised point cloud, the point whose number of points within the cluster neighborhood radius is greater than the preset density threshold is determined as a core point;
[0051] Based on the preset clustering neighborhood radius, obtain the neighborhood of each core point;
[0052] The core points with intersections in the neighborhood are divided into a cluster, and the set of all clusters is the clustering result;
[0053] The largest cluster in the clustering result is extracted as the main cluster, and all the core points in the main cluster are used to form the main point cloud.
[0054] Preferably, the similarity matrix is constructed based on the preset matching point pairs between the first angle subject point cloud and the second angle subject point cloud, which is expressed as:
[0055] ;
[0056] in, represents the similarity matrix, is a monotonically decreasing exponential function; , is the Euclidean distance calculation function; and A pair of preset matching points in the first angle subject point cloud and the second angle subject point cloud; and A matching point pair is preset for another pair in the first angle subject point cloud and the second angle subject point cloud.
[0057] Preferably, binarizing the similarity matrix, counting all matching point pairs in the matching seed point pairs that are compatible with the binarized similarity matrix, and constructing a high-order spatial matrix includes:
[0058] Binarize the similarity matrix to obtain the binary similarity matrix, where the value of the element in the i-th row and j-th column is , expressed as: ; Indicates the preset distance threshold;
[0059] Count all matching point pairs in the matching seed point pairs that are compatible with the binarized similarity matrix to obtain a high-order spatial matrix , expressed as: ;
[0060] in, Represents the first Line The value of the element of the column, Represents the first Line The values of the elements of the column; Represents the total number of matching seed point pairs, .
[0061] The embodiment of the present invention further provides a three-dimensional point cloud reconstruction device, comprising:
[0062] The main point cloud construction module is used to obtain the first-angle point cloud data and the second-angle point cloud data of the object to be reconstructed, and then perform denoising and clustering respectively to obtain the first-angle main point cloud and the second-angle main point cloud; the acquisition of the first-angle point cloud data and the second-angle point cloud data includes: establishing a binocular camera model, using chessboard calibration to obtain the internal and external parameter matrix of the binocular camera; using the phase shift method combined with the complementary Gray code technology to generate a coding pattern; projecting the coding pattern onto the surface of the object to be reconstructed, and using the binocular camera to collect and obtain the left-eye grating image and the right-eye grating image of the object to be reconstructed from a preset angle; performing phase unwrapping on the left-eye grating image and the right-eye grating image respectively to extract the left-eye phase information and the right-eye phase information; based on the left-eye phase information and the right-eye phase information, using epipolar correction, lateral constraints and global local search, phase matching is performed on the left-eye grating image and the right-eye grating image to obtain matching pixels; based on the matching pixels and the internal and external parameter matrix of the binocular camera, using the triangulation principle, the three-dimensional spatial coordinates of the matching pixels are calculated to obtain the point cloud data of the object to be reconstructed at the preset angle;
[0063] A high-order spatial matrix construction module is used to construct a similarity matrix based on preset matching point pairs between the subject point cloud at the first angle and the subject point cloud at the second angle; obtain the matching confidence of each pair of matching point pairs in the similarity matrix using spectral decomposition technology, and use the matching point pair with the local maximum confidence as the matching seed point pair; binarize the similarity matrix, count all matching point pairs in the matching seed point pairs that are compatible with the binarized similarity matrix, and construct a high-order spatial matrix;
[0064] The optimal transformation matrix construction module is used to calculate the similarity of all matching point pairs in the high-order spatial matrix, obtain the preset number of matching point pairs with the highest similarity, and form a target matching point pair set; based on the target matching point pair set, the singular value decomposition method is used to obtain the rigid body transformation matrix, and then the interior point counting criterion is used to obtain the optimal transformation matrix;
[0065] The registration module is used to transform the first angle point cloud data based on the optimal transformation matrix to obtain the optimal initial pose point cloud as the three-dimensional point cloud of the object to be reconstructed.
[0066] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps in the three-dimensional point cloud reconstruction method as described above are implemented.
[0067] The above technical solution of the present invention has the following beneficial effects compared with the prior art:
[0068] The three-dimensional point cloud reconstruction method described in the present invention, after obtaining the first-angle main point cloud and the second-angle main point cloud, constructs a similarity matrix between the two main point clouds; by using the similarity matrix to extract matching seed point pairs of the first-angle main point cloud and the second-angle main point cloud, screening the matching point pairs compatible with the binarized similarity matrix in the matching seed point pairs, constructing a high-order spatial matrix to obtain a matching point pair set, increasing the features between the point clouds, eliminating unmatched points, improving the reconstruction efficiency, and accelerating the reconstruction of the three-dimensional point cloud of the object to be reconstructed; after using the weighted singular value decomposition to obtain the optimal initial pose of the matching point set, it is aligned with the second-angle point cloud data to ensure that the point cloud data at different angles can be accurately fused together to obtain the three-dimensional point cloud of the object to be reconstructed.
[0069] The present invention uses a phase shift method combined with complementary Gray code technology to generate a coding pattern and projects it onto the surface of the object to be reconstructed, thereby constructing the first angle point cloud data and the second angle point cloud data; and through distance denoising and radius denoising, after effectively filtering out different types of discrete points, clustering is performed to accurately extract the main point cloud, thereby solving the problem that the traditional point cloud filtering algorithm has poor denoising effect and is difficult to filter out dense noise points when processing point clouds, improving the quality of point cloud data, and improving the accuracy and reliability of object reconstruction. In addition, when constructing point cloud data, the present invention introduces the idea of complementary Gray code, doubles the codeword on the basis of the last of the original 4 initial Gray codes, introduces a complementary Gray code for decoding, generates a result with a phase order offset of 1 / 2 cycle obtained by decoding the original Gray code, forms the complementarity of the coding, solves the problem of edge jump error, and improves the reconstruction accuracy of the object's three-dimensional point cloud; and uses the polar line correction method combined with global and local search for phase matching, thereby improving the efficiency of binocular matching and the speed of object three-dimensional point cloud reconstruction.
[0070] After obtaining the optimal initial pose point cloud, the present invention can also accurately align the optimal initial pose point cloud with the second angle point cloud data; by downsampling the two point clouds, using the KD tree to accelerate the neighbor point search, and performing accurate alignment based on feature matching, the problems of traditional point cloud alignment algorithms such as many outliers, poor alignment effect, and low efficiency are solved, the alignment speed and accuracy are improved, and the reconstructed three-dimensional point cloud is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS
[0071] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein
[0072] Figure 1 is a flowchart of the steps of the three-dimensional point cloud reconstruction method provided by the present invention;
[0073] Figure 2This is the impeller point cloud data diagram based on binocular structured light reconstruction provided by the present invention;
[0074] Figure 3 It is the impeller point cloud data diagram after preprocessing provided by the present invention;
[0075] Figure 4 (a) is a diagram of the impeller point cloud data after distance denoising with a standard deviation of 1 and a neighborhood point of 50 provided by the present invention; Figure 4 (b) is a diagram of the impeller point cloud data after distance denoising with a standard deviation of 1 and a neighborhood point of 100 provided by the present invention; Figure 4 (c) is a denoised impeller point cloud data diagram with a standard deviation of 1 and a neighborhood point of 300 provided by the present invention; Figure 4 (d) is a denoised impeller point cloud data diagram with a standard deviation of 1 and a neighborhood point of 500 provided by the present invention; Figure 4 (e) is a denoised impeller point cloud data diagram with a standard deviation of 0.5 and a neighborhood point of 300 provided by the present invention; Figure 4 (f) is a denoised impeller point cloud data diagram with a standard deviation of 2 and a neighborhood point of 300 provided by the present invention;
[0076] Figure 5 (a) is a point cloud data diagram of an impeller after radius denoising with a radius of 0.5 and a threshold of 10 provided by the present invention; Figure 5 (b) is a point cloud data diagram of an impeller after radius denoising with a radius of 1 and a threshold of 20 provided by the present invention; Figure 5 (c) is a point cloud data diagram of an impeller after radius denoising with a radius of 1 and a threshold of 50 provided by the present invention; Figure 5 (d) is a point cloud data diagram of an impeller after radius denoising with a radius of 1 and a threshold of 70 provided by the present invention; Figure 5 (e) is a point cloud data diagram of an impeller after radius denoising with a radius of 2 and a threshold of 50 provided by the present invention; Figure 5 (f) is a point cloud data diagram of an impeller after radius denoising with a radius of 2 and a threshold of 70 provided by the present invention;
[0077] Figure 6 It is a relative position diagram of the first angle impeller body point cloud and the second angle impeller body point cloud provided by the present invention;
[0078] Figure 7 (a) is the point cloud image after registration using the RANSAC coarse registration method; Figure 7 (b) is the point cloud image after registration using the SAC-IA coarse registration method; Figure 7 (c) is a point cloud image registered by using the three-dimensional point cloud reconstruction method provided by the present invention;
[0079] Figure 8 (a) is a schematic diagram of the point cloud after ICP registration; Figure 8 (b) is a schematic diagram of a point cloud after being aligned using the precise alignment method provided by the present invention. DETAILED DESCRIPTION
[0080] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention. Embodiment 1:
[0081] Reference Figure 1 As shown, the step flow chart of the three-dimensional point cloud reconstruction method of the present invention includes the following steps:
[0082] S101: After obtaining the first-angle point cloud data and the second-angle point cloud data of the object to be reconstructed, denoising and clustering are performed respectively to obtain the first-angle main body point cloud and the second-angle main body point cloud;
[0083] The acquisition of the first-angle point cloud data and the second-angle point cloud data includes: establishing a binocular camera model, using checkerboard calibration to obtain the internal and external parameter matrix of the binocular camera; using the phase shift method combined with the complementary Gray code technology to generate a coding pattern; projecting the coding pattern onto the surface of the object to be reconstructed, and using the binocular camera to collect and obtain the left-eye grating image and the right-eye grating image of the object to be reconstructed from a preset angle; performing phase unwrapping on the left-eye grating image and the right-eye grating image respectively to extract the left-eye phase information and the right-eye phase information; based on the left-eye phase information and the right-eye phase information, using epipolar correction, lateral constraints and global local search, phase matching is performed on the left-eye grating image and the right-eye grating image to obtain matching pixel points; based on the matching pixel points and the internal and external parameter matrix of the binocular camera, using the triangulation principle, the three-dimensional spatial coordinates of the matching pixel points are calculated to obtain the point cloud data of the object to be reconstructed at the preset angle;
[0084] S102: constructing a similarity matrix based on preset matching point pairs between the first angle subject point cloud and the second angle subject point cloud;
[0085] S103: using spectral decomposition technology to obtain the matching confidence of each pair of matching points in the similarity matrix, and using the matching point pair with the local maximum confidence as the matching seed point pair;
[0086] S104: Binarizing the similarity matrix, counting all matching point pairs in the matching seed point pairs that are compatible with the binarized similarity matrix, and constructing a high-order spatial matrix;
[0087] S105: Calculate the similarity of all matching point pairs in the high-order spatial matrix, obtain a preset number of matching point pairs with the highest similarity, and form a target matching point pair set;
[0088] S106: Based on the target matching point pair set, after obtaining the rigid body transformation matrix by using the singular value decomposition method, the optimal transformation matrix is obtained by using the interior point counting criterion;
[0089] S107: Based on the optimal transformation matrix, transform the first angle point cloud data to obtain the optimal initial pose point cloud as the three-dimensional point cloud of the object to be reconstructed.
[0090] Specifically, in step S101, phase unwrapping is performed on the grating image, including:
[0091] The phase shift method is used to obtain the unwrapped phase, and four fringe images with a phase shift difference of π / 2 are collected, which can be expressed as:
[0092] ;
[0093] Let the encoding values of the initial four Gray codes be , the newly added complementary Gray code value is , based on complementary Gray code and phase shift method to solve the absolute phase value , expressed as:
[0094] ;
[0095] in, Represents the unwrapped phase value obtained using the phase shift method, and the expression is
[0096] ; To preset the background light intensity, To preset the modulated light intensity, , , , They respectively represent the grayscale values corresponding to the pixels in the four fringe images.
[0097] Among them, the left eye phase information and the right eye phase information are polar line corrected, and the left and right phase map rows of the object are aligned after the correction, reducing the 2-dimensional matching search range to 1-dimensional; a global search is performed in the same row of the right phase map for the most similar point with the left phase map with the same phase value or less than the set phase difference value. After the match is successful, the subsequent corresponding points can use the previous matching point as the initial search point to search to the right for the most similar point. After matching all pixel points, sub-pixel interpolation is performed to obtain a complete disparity map.
[0098] Specifically, in an embodiment of the present invention, denoising is performed on the first-angle point cloud data and the second-angle point cloud data of the object to be reconstructed, including distance denoising and radius denoising.
[0099] ① Perform distance denoising on the first angle point cloud data and the second angle point cloud data, including:
[0100] Set the number of neighboring points ,according to The neighborhood algorithm searches for each point in the point cloud data. nearest neighbor points;
[0101] Calculate each point and its The mean of the distances to the nearest neighbors , expressed as: ;
[0102] Based on each point to its The distances to the nearest neighbors, as well as the mean of the distances, calculate the standard deviation , expressed as: ;
[0103] Based on mean , Standard Deviation Multiple of the preset standard deviation , the preset average distance range is , the points whose mean is not in the average distance range are identified as noise points and removed to obtain the distance denoised point cloud;
[0104] in, Indicates Point and its The distance between the nearest neighbor points is expressed as , ; Indicates The three-dimensional coordinates of the points, , Indicates the total number of points in the point cloud data; Indicates The 3D coordinates of the nearest neighbor points.
[0105] ② Perform radius denoising on the first angle point cloud data and the second angle point cloud data, both including:
[0106] For each point in the distance denoised point cloud, draw a neighborhood sphere for each point with itself as the center and the preset neighborhood search radius as the radius;
[0107] All points that fall within the neighborhood sphere are regarded as the neighborhood points of the point;
[0108] The number of neighborhood points of each point in the distance denoised point cloud is obtained, and the points whose number of neighborhood points is less than the preset neighborhood point threshold are identified as outliers and removed to obtain the radius denoised point cloud.
[0109] Specifically, the acquisition of the first angle subject point cloud and the second angle subject point cloud both include:
[0110] The cluster neighborhood radius is preset, and for each point in the radius denoised point cloud, the point whose number of points within the cluster neighborhood radius is greater than the preset density threshold is determined as a core point;
[0111] Based on the preset clustering neighborhood radius, obtain the neighborhood of each core point;
[0112] The core points with intersections in the neighborhood are divided into a cluster, and the set of all clusters is the clustering result;
[0113] The largest cluster in the clustering result is extracted as the main cluster, and all the core points in the main cluster are used to form the main point cloud.
[0114] Specifically, in an embodiment of the present invention, the point cloud data with topological relationships constructed are initially denoised using statistical filtering, the number of neighborhood points is set to 300, the standard deviation multiple threshold is set to 1.0, and some discrete points are filtered out; the point cloud data after the initial denoising is denoised again using a radius filter, the filter radius is set to 1 mm, the neighborhood point threshold is set to 50, and the discrete point clouds that cannot be filtered out by statistical filtering are filtered out again; the point cloud data after the second denoising is screened using the DBSCAN clustering algorithm, the largest piece of point cloud data is the main point cloud, and the remaining dense noise point clusters are all deleted.
[0115] The present invention uses the phase shift method combined with the complementary Gray code technology to generate a coding pattern and projects it onto the surface of the object to be reconstructed, thereby constructing the first angle point cloud data and the second angle point cloud data; and through distance denoising and radius denoising, after effectively filtering out different types of discrete points, clustering is performed to accurately extract the main point cloud, thereby solving the problem that the traditional point cloud filtering algorithm has poor denoising effect and is difficult to filter out dense noise points when processing point clouds, improving the quality of reconstructed point cloud data, and improving the accuracy and reliability of object reconstruction. In addition, when constructing point cloud data, the present invention introduces the idea of complementary Gray code, doubles the codewords on the basis of the last of the original 4 initial Gray codes, introduces a complementary Gray code for decoding, generates a result with a phase order offset of 1 / 2 cycle obtained by decoding the original Gray code, forms the complementarity of the coding, solves the problem of edge jump error, and improves the reconstruction accuracy of the three-dimensional point cloud; and adopts the method of polar line correction combined with global and local search for phase matching, thereby improving the efficiency of binocular matching and the speed of three-dimensional point cloud reconstruction.
[0116] Specifically, in step S102, a similarity matrix is constructed based on the preset matching point pairs between the first angle subject point cloud and the second angle subject point cloud, which is expressed as: ;in, represents the similarity matrix, is a monotonically decreasing exponential function; , is the Euclidean distance calculation function; and A pair of preset matching points in the first angle subject point cloud and the second angle subject point cloud; and A matching point pair is preset for another pair in the first angle subject point cloud and the second angle subject point cloud.
[0117] Specifically, in step S103, the matching confidence of each pair of matching points in the similarity matrix is obtained by using spectral decomposition technology, and the matching point pair with the local maximum confidence is used as the matching seed point pair, including:
[0118] S103-1: Perform eigendecomposition on the similarity matrix using spectral decomposition technology to obtain eigenvectors and eigenvalues of each pair of matching points;
[0119] S103-2: Select the eigenvector with the largest eigenvalue as the main eigenvector:
[0120] S103-3: Divide each element of the main eigenvector by the modulus of the vector to obtain the main eigenvector;
[0121] S103-4: Calculate the dot product of the feature vector of each pair of matching points and the main feature unit vector as the matching confidence of the pair of matching points;
[0122] S103-5: Based on the matching confidence of each pair of matching point pairs in the similarity matrix, all matching point pairs whose matching confidence is higher than the matching confidence of their adjacent matching point pairs in the similarity matrix are obtained as matching seed point pairs.
[0123] Specifically, in step S104, the similarity matrix is binarized, and all matching point pairs in the matching seed point pairs that are compatible with the binarized similarity matrix are counted to construct a high-order spatial matrix, including:
[0124] S104-1: Binarize the similarity matrix to obtain the binary similarity matrix, where the value of the element in the i-th row and j-th column is , expressed as: ; Indicates the preset distance threshold;
[0125] S104-2: Count all matching point pairs in the matching seed point pairs that are compatible with the binarized similarity matrix to obtain a high-order spatial matrix , expressed as: ;
[0126] in, Represents the first Line The value of the element of the column, Represents the first Line The values of the elements of the column; Represents the total number of matching seed point pairs, .
[0127] The three-dimensional point cloud reconstruction method described in the present invention obtains the first-angle main point cloud and the second-angle main point cloud, and then constructs a similarity matrix between the two main point clouds; by using the similarity matrix to extract matching seed point pairs of the first-angle main point cloud and the second-angle main point cloud, the matching point pairs compatible with the binarized similarity matrix are screened in the matching seed point pairs, and a high-order spatial matrix is constructed to obtain a matching point pair set, thereby increasing the features between the point clouds, eliminating mismatched points, improving the reconstruction efficiency, and accelerating the reconstruction of the three-dimensional point cloud of the object; after obtaining the optimal initial pose by using the weighted singular value decomposition matching point set, it is aligned with the second-angle point cloud data to ensure that the point cloud data at different angles can be accurately fused together to obtain the three-dimensional point cloud of the object to be reconstructed.
[0128] The embodiment of the present invention further provides a three-dimensional point cloud reconstruction device, comprising:
[0129] The main point cloud construction module 100 is used to obtain the first angle point cloud data and the second angle point cloud data of the object to be reconstructed, and then perform denoising and clustering respectively to obtain the first angle main point cloud and the second angle main point cloud; the acquisition of the first angle point cloud data and the second angle point cloud data both include: establishing a binocular camera model, using chessboard calibration to obtain the internal and external parameter matrix of the binocular camera; using the phase shift method combined with the complementary Gray code technology to generate a coding pattern; projecting the coding pattern onto the surface of the object to be reconstructed, and using the binocular camera to collect and obtain the left-eye grating image and the right-eye grating image of the object to be reconstructed from a preset angle; performing phase unwrapping on the left-eye grating image and the right-eye grating image respectively to extract the left-eye phase information and the right-eye phase information; based on the left-eye phase information and the right-eye phase information, using epipolar correction, lateral constraints and global local search, phase matching is performed on the left-eye grating image and the right-eye grating image to obtain matching pixels; based on the matching pixels and the internal and external parameter matrix of the binocular camera, using the triangulation principle, the three-dimensional spatial coordinates of the matching pixels are calculated to obtain the point cloud data of the object to be reconstructed at the preset angle;
[0130] The high-order spatial matrix construction module 200 is used to construct a similarity matrix based on the preset matching point pairs between the first angle subject point cloud and the second angle subject point cloud; use the spectral decomposition technology to obtain the matching confidence of each pair of matching point pairs in the similarity matrix, and use the matching point pair with the local maximum confidence as the matching seed point pair; binarize the similarity matrix, count all the matching point pairs in the matching seed point pairs that are compatible with the binarized similarity matrix, and construct a high-order spatial matrix;
[0131] The optimal transformation matrix construction module 300 is used to calculate the similarity of all matching point pairs in the high-order spatial matrix, obtain the preset number of matching point pairs with the highest similarity, and form a target matching point pair set; based on the target matching point pair set, after obtaining the rigid body transformation matrix using the singular value decomposition method, the optimal transformation matrix is obtained using the interior point counting criterion;
[0132] The registration module 400 is used to transform the first angle point cloud data based on the optimal transformation matrix to obtain the optimal initial pose point cloud as the three-dimensional point cloud of the object to be reconstructed.
[0133] The three-dimensional point cloud reconstruction device of this embodiment is used to implement the aforementioned three-dimensional point cloud reconstruction method. Therefore, the specific implementation methods of the three-dimensional point cloud reconstruction device can be seen in the embodiment part of the three-dimensional point cloud reconstruction method in the previous text, for example, the main point cloud construction module 100 is used to implement step S101 in the above-mentioned three-dimensional point cloud reconstruction method; the high-order spatial matrix construction module 200 is used to implement steps S102, S103 and S104 in the above-mentioned three-dimensional point cloud reconstruction method; the optimal transformation matrix construction module 300 is used to implement steps S105 and S106 in the above-mentioned three-dimensional point cloud reconstruction method; the registration module 400 is used to implement step S107 in the above-mentioned three-dimensional point cloud reconstruction method. Therefore, its specific implementation methods can refer to the description of the corresponding embodiments of each part, which will not be repeated here.
[0134] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the three-dimensional point cloud reconstruction method as described above are implemented. Embodiment 2:
[0135] Based on the above embodiment, in another embodiment of the present invention, after obtaining the optimal initial pose point cloud, it also includes precise registration to obtain a more accurate three-dimensional point cloud of the object to be reconstructed, and the specific steps include:
[0136] S201: After obtaining the first-angle point cloud data and the second-angle point cloud data of the object to be reconstructed, denoising and clustering are performed respectively to obtain the first-angle main body point cloud and the second-angle main body point cloud;
[0137] Set the number of neighborhood points K, and search for the nearest neighbor point according to the K-neighbor algorithm; traverse the points in the point cloud, and calculate the average distance between each point and its nearest neighbor point, expressed as: ; According to the average distance, the standard deviation is calculated to describe the degree of dispersion of the data distribution; the standard deviation is expressed as: ; By calculating the mean μ and standard deviation σ, a statistical model of point cloud distribution can be obtained. Based on this model, a standard deviation multiple std is set. If the average distance from a point in the point cloud to its k-neighborhood exceeds If the point is within the range of , then the point is a noise point or an outlier and will be removed.
[0138] For each point in the point cloud after preliminary denoising, draw a sphere with the point as the center and the set neighborhood search radius R as the radius, and find all the points that fall within the sphere as the neighborhood points of the point. If the number of neighborhood points in the sphere is less than the set neighborhood point threshold, the point will be considered an outlier and removed from the point cloud.
[0139] Set the neighborhood radius e of the cluster. For each point in the point cloud data after secondary denoising, check the number of point clouds within its neighborhood radius e. If the number is greater than the set density threshold dth, such points are marked as core points. If the neighborhoods of two core points have intersections, connect them and find all points connected by the neighborhood. These points belong to the same cluster. The set of all clusters is the final clustering result. Calculate the largest cluster in the cluster, which is the main point cloud, extract it and save it.
[0140] S202: Based on the preset matching point pairs between the first angle subject point cloud and the second angle subject point cloud, a similarity matrix is constructed, which is expressed as: ;
[0141] S203: using spectral decomposition technology to obtain the matching confidence of each pair of matching points in the similarity matrix, and using the matching point pair with the local maximum confidence as the matching seed point pair;
[0142] S204: Binarizing the similarity matrix, counting all matching point pairs in the matching seed point pairs that are compatible with the binarized similarity matrix, and constructing a high-order spatial matrix;
[0143] S205: Calculate the similarity of all matching point pairs in the high-order spatial matrix, obtain a preset number of matching point pairs with the highest similarity, and form a target matching point pair set;
[0144] S206: Based on the target matching point pair set, after obtaining the rigid body transformation matrix by using the singular value decomposition method, the optimal transformation matrix is obtained by using the interior point counting criterion;
[0145] S207: Based on the optimal transformation matrix, transform the first angle point cloud data to obtain the optimal initial pose point cloud;
[0146] S208: uniformly down-sampling the optimal initial pose point cloud and the second angle point cloud data respectively to obtain the optimal down-sampled point cloud and the second down-sampled point cloud;
[0147] S209: For the optimal down-sampled point cloud and the second down-sampled point cloud, respectively, the neighboring points of each point in the point cloud and the FPFH features of the neighboring points are obtained by using the KD tree, and feature matching is performed to obtain corresponding point pairs between the optimal down-sampled point cloud and the second down-sampled point cloud, and a corresponding point pair set is constructed;
[0148] S210: Based on the corresponding point pair set, a rigid body transformation matrix is obtained; based on the rigid body transformation matrix, the optimal initial pose point cloud is transformed to obtain an updated optimal down-sampling point cloud;
[0149] S211: Calculate and iteratively optimize based on the root mean square error between the updated optimal down-sampling point cloud and the second angle point cloud data, update the corresponding point pair set, the rigid body transformation matrix and the optimal down-sampling point cloud until the root mean square error is less than a preset threshold, obtain the optimal rigid body transformation matrix, act on the optimal initial pose point cloud, realize point cloud registration, and obtain the three-dimensional reconstructed point cloud of the object to be reconstructed.
[0150] When the value of the root mean square error is smaller, the registration error is smaller and the registration accuracy is higher. The root mean square error is expressed as:
[0151] ;
[0152] in, represents the point in the updated optimal downsampled point cloud, Indicates the second angle point cloud data The corresponding points, Represents the total number of point pairs in the corresponding point pair set.
[0153] After obtaining the optimal initial pose point cloud, the present invention can also accurately align the optimal initial pose point cloud with the second angle point cloud data; the optimal initial pose point cloud and the second angle point cloud data are uniformly downsampled respectively to reduce the number of points in the point cloud, reduce the complexity of the data and the storage space requirements, and at the same time retain the main features and shape information of the point cloud as much as possible to obtain two sampled point clouds. A KD tree is established to accelerate the search for neighboring points of the downsampled point cloud, extract the FPFH features of the points and neighboring points of the sampled point cloud, and match based on the features to determine each unique corresponding point in the sampled target domain and establish a corresponding point set. The rigid body transformation matrix and the root mean square error are calculated, and the iteration is continuously performed until the error value meets the requirements or the maximum number of iterations is reached and the iteration is stopped, and the optimal transformation matrix is applied to the optimal initial pose point cloud to achieve point cloud alignment.
[0154] The present invention solves the problems of many outliers, poor registration effect and low efficiency in traditional point cloud registration algorithms by downsampling the two point clouds, using the KD tree to accelerate the search for neighboring points, and performing precise registration based on feature matching. It improves the registration speed and accuracy, and makes the reconstructed three-dimensional point cloud of the object more accurate.
[0155] In summary, the three-dimensional reconstruction method provided by the embodiment of the present invention first effectively filters out different types of discrete points by combining a statistical model with a radius search-based method, and further uses a clustering algorithm to accurately extract the main point cloud, thereby solving the problem that the traditional point cloud filtering algorithm has poor denoising effect and difficulty in filtering out dense noise points when processing point clouds. Secondly, by using a spatial matrix to extract matching seed points of two point clouds, calculating and screening a set of matching points compatible with a high-order spatial matrix among the matching seed points, using weighted singular value decomposition to obtain the initial pose, downsampling the two point clouds, using a KD tree to accelerate the search for neighboring points, and accurately aligning based on feature matching, the traditional point cloud alignment algorithm solves the problems of many external points, poor alignment effect, and low efficiency.
[0156] Based on the above embodiments, in an embodiment of the present invention, a three-dimensional point cloud reconstruction is performed using an impeller as an object to be reconstructed, and the feasibility and accuracy of the three-dimensional reconstruction method provided by the embodiment of the present invention are demonstrated; the specific steps include:
[0157] S301: After obtaining the first-angle impeller point cloud data and the second-angle impeller point cloud data of the impeller to be reconstructed, denoising and clustering are performed respectively to obtain the first-angle impeller main body point cloud and the second-angle impeller main body point cloud;
[0158] S302: constructing a similarity matrix based on preset matching point pairs between the first angle impeller body point cloud and the second angle impeller body point cloud;
[0159] S303: using spectral decomposition technology to obtain the matching confidence of each pair of matching points in the similarity matrix, and using the matching point pair with the local maximum confidence as the matching seed point pair;
[0160] S304: Binarize the similarity matrix, count all matching point pairs in the matching seed point pairs that are compatible with the binarized similarity matrix, and construct a high-order space matrix;
[0161] S305: Calculate the similarity of all matching point pairs in the high-order spatial matrix, obtain a preset number of matching point pairs with the highest similarity, and form a target matching point pair set;
[0162] S306: Based on the target matching point pair set, after obtaining the rigid body transformation matrix by using the singular value decomposition method, the optimal transformation matrix is obtained by using the interior point counting criterion;
[0163] S307: Based on the optimal transformation matrix, transform the first angle impeller point cloud data to obtain the optimal initial posture point cloud;
[0164] S308: uniformly down-sampling the optimal initial pose point cloud and the second angle impeller point cloud data respectively to obtain the optimal down-sampled point cloud and the second down-sampled point cloud;
[0165] S309: For the optimal down-sampled point cloud and the second down-sampled point cloud, respectively, the neighboring points of each point in the point cloud and the FPFH features of the neighboring points are obtained by using the KD tree, and feature matching is performed to obtain corresponding point pairs between the optimal down-sampled point cloud and the second down-sampled point cloud, and a corresponding point pair set is constructed;
[0166] S310: Based on the corresponding point pair set, a rigid body transformation matrix is obtained; based on the rigid body transformation matrix, the optimal initial pose point cloud is transformed to obtain an updated optimal down-sampling point cloud;
[0167] S311: Calculate and iteratively optimize based on the root mean square error between the updated optimal down-sampling point cloud and the second-angle impeller point cloud data, update the corresponding point pair set, rigid body transformation matrix and the optimal down-sampling point cloud until the root mean square error is less than the preset threshold, obtain the optimal rigid body transformation matrix, act on the optimal initial pose point cloud, realize point cloud registration, and obtain the three-dimensional reconstructed point cloud of the impeller to be reconstructed.
[0168] In the practical application of this application, after the impeller point cloud preprocessing and two-stage fine registration are performed by the 3D reconstruction method provided by the embodiment of the present invention, not only can the noise and redundant information be effectively removed, but also the accurate alignment and fusion of point cloud data from different perspectives can be achieved, which provides strong support for the research and application in related fields. The experimental data and experimental results of the embodiment of the present invention are compared, including:
[0169] This embodiment is used to verify that the impeller 3D reconstruction method provided by the present invention has a good 3D reconstruction effect on the impeller. This experiment uses a Texas Instruments DLP LightCrafter4500 projector for structured light projection with a resolution of 912´1140, and two Daheng MER2-503-36U3M industrial cameras for image acquisition with a resolution of 2448´2048. The experiment is carried out under the Windows 10 operating system, the processor is Intel(R) Xeon(R) Gold 6244, the graphics card is NVIDIAQuadro RTX 5000, and the video memory is 16GB. Reference Figure 2 As shown in Figure 1, it is the impeller point cloud data reconstructed based on binocular structured light; Figure 2 It can be clearly seen that the impeller point cloud has a lot of noise points. Using the preprocessing algorithm provided in this application, the impeller point cloud data after preprocessing is as follows Figure 3 shown; reference Figure 4 As shown, the impeller point cloud data diagram after distance denoising provided by the present invention includes: Figure 4 (a) is a diagram of the impeller point cloud data after distance denoising with a standard deviation of 1 and a neighborhood point of 50 provided by the present invention; Figure 4 (b) is a diagram of the impeller point cloud data after distance denoising with a standard deviation of 1 and a neighborhood point of 100 provided by the present invention; Figure 4 (c) is a denoised impeller point cloud data diagram with a standard deviation of 1 and a neighborhood point of 300 provided by the present invention; Figure 4 (d) is a denoised impeller point cloud data diagram with a standard deviation of 1 and a neighborhood point of 500 provided by the present invention; Figure 4 (e) is a denoised impeller point cloud data diagram with a standard deviation of 0.5 and a neighborhood point of 300 provided by the present invention; Figure 4 (f) is a denoised impeller point cloud data diagram with a standard deviation of 2 and a neighborhood point of 300 provided by the present invention. Figure 5 As shown in FIG. 1 , the impeller point cloud data diagram after radius denoising provided by the present invention includes: Figure 5 (a) is a point cloud data diagram of an impeller after radius denoising with a radius of 0.5 and a threshold of 10 provided by the present invention; Figure 5 (b) is a point cloud data diagram of an impeller after radius denoising with a radius of 1 and a threshold of 20 provided by the present invention; Figure 5 (c) is a point cloud data diagram of an impeller after radius denoising with a radius of 1 and a threshold of 50 provided by the present invention; Figure 5 (d) is a point cloud data diagram of an impeller after radius denoising with a radius of 1 and a threshold of 70 provided by the present invention; Figure 5 (e) is a point cloud data diagram of an impeller after radius denoising with a radius of 2 and a threshold of 50 provided by the present invention; Figure 5(f) is a denoised impeller point cloud data diagram with a radius of 2 and a threshold of 70 provided by the present invention; after comparison, it is found that the pre-processed impeller point cloud effectively removes the interference of noise and improves the quality of the impeller point cloud data.
[0170] Reference Figure 6 As shown in Figure 1, it is the relative position diagram of the first angle impeller main body point cloud and the second angle impeller main body point cloud. Figure 6 It can be clearly seen that the two impeller point clouds are not in the same coordinate system, have few intersections, and have large spatial position differences. The method provided in this application uses the spatial matrix to extract the matching seed points of the two point clouds, calculates and screens the matching point set compatible with the high-order spatial matrix in the matching seed points, and uses the weighted singular value decomposition to obtain the initial pose to align the point cloud. Compared with the impeller point cloud alignment of the traditional coarse alignment algorithm, Figure 7 As shown, Figure 7 (a) is the point cloud image after registration using the RANSAC coarse registration method; Figure 7 (b) is the point cloud image after registration using the SAC-IA coarse registration method; Figure 7 (c) is a point cloud image after registration using the three-dimensional point cloud reconstruction method provided by the present invention. The algorithm of the present application considers the similarity of corresponding points from a global perspective, reduces the influence of external points on the registration, and by comparing with the traditional coarse registration algorithm, it is proved that the proposed coarse registration algorithm can effectively improve the registration accuracy and reduce the registration time.
[0171] The method provided in this application is to downsample the two point clouds, use the KD tree to accelerate the neighbor point search, and perform precise registration based on feature matching. Compared with the impeller point cloud registration of the traditional precise registration algorithm, Figure 8 As shown, Figure 8 (a) is a schematic diagram of the point cloud after ICP registration; Figure 8 (b) is a schematic diagram of the point cloud after alignment using the precise alignment method provided by the present invention; by comparing with the traditional precise alignment algorithm, it is proved that the proposed precise alignment algorithm has smaller error and consumes less time.
[0172] The three-dimensional point cloud reconstruction method described in the present invention uses a phase shift method combined with a complementary Gray code technology to generate a coding pattern and project it onto the surface of the object to be reconstructed, thereby constructing the first angle point cloud data and the second angle point cloud data; and through distance denoising and radius denoising, after effectively filtering out different types of discrete points, clustering is performed to accurately extract the main point cloud, thereby solving the problem that the traditional point cloud filtering algorithm has poor denoising effect and is difficult to filter out dense noise points when processing point clouds, improving the quality of point cloud data, and improving the accuracy and reliability of reconstruction. In addition, when constructing point cloud data, the present invention introduces the idea of complementary Gray code, doubles the codeword on the basis of the last of the original four initial Gray codes, introduces a complementary Gray code for decoding, generates a result with a phase order offset of 1 / 2 cycle obtained by decoding the original Gray code, forms the complementarity of the coding, solves the problem of edge jump error, and improves the reconstruction accuracy of the three-dimensional point cloud; and adopts the method of polar line correction combined with global and local search for phase matching, thereby improving the efficiency of binocular matching and the speed of three-dimensional point cloud reconstruction. After obtaining the main point cloud of the first angle and the main point cloud of the second angle, the present invention constructs a similarity matrix between the two main point clouds; by using the similarity matrix to extract matching seed point pairs of the main point cloud of the first angle and the main point cloud of the second angle, screening the matching point pairs compatible with the binarized similarity matrix in the matching seed point pairs, and constructing a high-order spatial matrix to obtain a matching point pair set, thereby increasing the features between the point clouds, eliminating unmatched points, improving the reconstruction efficiency, and accelerating the reconstruction of the three-dimensional point cloud; after obtaining the optimal initial posture by using the weighted singular value decomposition matching point set, it is aligned with the second angle point cloud data to ensure that the point cloud data at different angles can be accurately fused together to obtain the three-dimensional point cloud of the object to be reconstructed.
[0173] After obtaining the optimal initial pose point cloud, the present invention can also accurately align the optimal initial pose point cloud with the second angle point cloud data; by downsampling the two point clouds, using the KD tree to accelerate the neighbor point search, and performing accurate alignment based on feature matching, the problems of traditional point cloud alignment algorithms such as many outliers, poor alignment effect, and low efficiency are solved, the alignment speed and accuracy are improved, and the reconstructed three-dimensional point cloud of the object is more accurate.
[0174] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0175] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0176] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0177] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0178] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.
Claims
1. A three-dimensional point cloud reconstruction method, characterized in that: include: After obtaining the first-angle point cloud data and the second-angle point cloud data of the object to be reconstructed, denoising and clustering are performed respectively to obtain the first-angle main point cloud and the second-angle main point cloud; wherein, the first-angle point cloud data and the second-angle point cloud data of the object to be reconstructed are denoised, including distance denoising and radius denoising; the distance denoising of the first-angle point cloud data and the second-angle point cloud data includes: setting the number of neighborhood points K, searching for the K nearest neighboring points of each point in the point cloud data according to the K-neighborhood algorithm; calculating the mean value μ of the distance between each point and its K nearest neighboring points, expressed as: Based on the distance from each point to its K nearest neighbors and the mean of the distances, the standard deviation σ is calculated, expressed as: Based on the mean μ, standard deviation σ and the preset standard deviation multiple std, the preset average distance range is (μ-std·σ,μ+std·σ), and the points whose mean is not in the average distance range are identified as noise points and removed to obtain the distance denoised point cloud; where, Represents the distance between the nth point and its mth nearest neighbor, expressed as (X n ,Y n ,Z n ) represents the three-dimensional coordinates of the nth point, 1≤n≤N, N represents the total number of points in the point cloud data; (X m ,Y m ,Z m ) represents the three-dimensional coordinates of the mth nearest neighbor point; The acquisition of the first-angle point cloud data and the second-angle point cloud data includes: establishing a binocular camera model, using checkerboard calibration to obtain the internal and external parameter matrix of the binocular camera; using the phase shift method combined with the complementary Gray code technology to generate a coding pattern; projecting the coding pattern onto the surface of the object to be reconstructed, and using the binocular camera to collect and obtain the left-eye grating image and the right-eye grating image of the object to be reconstructed from a preset angle; performing phase unwrapping on the left-eye grating image and the right-eye grating image respectively to extract the left-eye phase information and the right-eye phase information; based on the left-eye phase information and the right-eye phase information, using epipolar correction, lateral constraints and global local search, phase matching is performed on the left-eye grating image and the right-eye grating image to obtain matching pixel points; based on the matching pixel points and the internal and external parameter matrix of the binocular camera, using the triangulation principle, the three-dimensional spatial coordinates of the matching pixel points are calculated to obtain the point cloud data of the object to be reconstructed at the preset angle; Based on the preset matching point pairs between the first angle subject point cloud and the second angle subject point cloud, a similarity matrix is constructed, which is expressed as: SM = φ(d ij ), where SM represents the similarity matrix and φ() is a monotonically decreasing exponential function; d ij =|D(α i ,α j )-D(β i -β j )|, D() is the Euclidean distance calculation function; α i and β i A pair of preset matching points in the first angle subject point cloud and the second angle subject point cloud; α j and β j Preset a matching point pair for another pair of the first angle subject point cloud and the second angle subject point cloud; The matching confidence of each pair of matching point pairs in the similarity matrix is obtained by using the spectral decomposition technology, and the matching point pairs with the local maximum confidence are used as the matching seed point pairs, including: using the spectral decomposition technology to perform eigendecomposition on the similarity matrix to obtain the eigenvector and eigenvalue of each pair of matching point pairs; selecting the eigenvector with the largest eigenvalue as the main eigenvector: dividing each element of the main eigenvector by the modulus of the vector to obtain the main eigenunit vector; calculating the dot product of the eigenvector of each pair of matching point pairs and the main eigenunit vector as the matching confidence of the pair of matching point pairs; based on the matching confidence of each pair of matching point pairs in the similarity matrix, obtaining all matching point pairs whose matching confidence is higher than the matching confidence of their adjacent matching point pairs in the similarity matrix as matching seed point pairs; Binarize the similarity matrix, count all matching point pairs in the matching seed point pairs that are compatible with the binarized similarity matrix, and construct a high-order spatial matrix, including: binarize the similarity matrix, obtain the binarized similarity matrix, where the value M of the element in the i-th row and j-th column ij , expressed as: d thr Represents the preset distance threshold; count all matching point pairs in the matching seed point pairs that are compatible with the binarized similarity matrix to obtain a high-order spatial matrix It is expressed as: Among them, M ik Represents the value of the element in the i-th row and k-th column of the binary similarity matrix, M kj Represents the value of the element in the kth row and jth column of the binary similarity matrix; N" represents the total number of matching seed point pairs, 1≤k≤N"; Calculate the similarity of all matching point pairs in the high-order spatial matrix, obtain the preset number of matching point pairs with the highest similarity, and form a target matching point pair set; Based on the target matching point pair set, the singular value decomposition method is used to obtain the rigid body transformation matrix, and then the optimal transformation matrix is obtained using the interior point counting criterion; Based on the optimal transformation matrix, the first angle point cloud data is transformed to obtain the optimal initial pose point cloud, and the optimal initial pose point cloud and the second angle point cloud data are uniformly downsampled to obtain the optimal downsampled point cloud and the second downsampled point cloud; For the optimal down-sampled point cloud and the second down-sampled point cloud, the KD tree is used to obtain the neighboring points of each point in the point cloud and the FPFH features of the neighboring points, and feature matching is performed to obtain the corresponding point pairs between the optimal down-sampled point cloud and the second down-sampled point cloud, and a corresponding point pair set is constructed; Based on the corresponding point pair set, a rigid body transformation matrix is obtained; based on the rigid body transformation matrix, the optimal initial pose point cloud is transformed to obtain an updated optimal down-sampling point cloud; Calculate and iteratively optimize based on the root mean square error between the updated optimal down-sampled point cloud and the second angle point cloud data, update the corresponding point pair set, rigid body transformation matrix and optimal down-sampled point cloud until the root mean square error is less than the preset threshold, obtain the optimal rigid body transformation matrix, act on the optimal initial pose point cloud, realize point cloud registration, and obtain the 3D reconstructed point cloud of the object to be reconstructed.
2. The three-dimensional point cloud reconstruction method according to claim 1, characterized in that: Phase unwrapping of grating images, including: The phase shift method is used to obtain the unwrapped phase, and four fringe images with a phase shift difference of π / 2 are collected, which can be expressed as: Let the encoding values of the initial four Gray codes be k1(x, y), and the encoding value of the newly added complementary Gray code be k2(x, y). The absolute phase value is solved based on the complementary Gray code and phase shift method, expressed as: in, Represents the unwrapped phase value obtained using the phase shift method, and the expression is A(x,y) is the preset background light intensity, B(x,y) is the preset modulated light intensity, I1(x,y), I2(x,y), I3(x,y), and I4(x,y) respectively represent the grayscale values corresponding to the pixels in the four fringe images.
3. The three-dimensional point cloud reconstruction method according to claim 1, characterized in that: The root mean square error between the updated optimal down-sampled point cloud and the second angle point cloud data is expressed as: Among them, s i represents the point in the updated optimal downsampled point cloud, t i Indicates the second angle point cloud data with s i Corresponding points, N' represents the total number of point pairs in the corresponding point pair set.
4. The three-dimensional point cloud reconstruction method according to claim 1, characterized in that: The radius denoising is performed on the first angle point cloud data and the second angle point cloud data, including: For each point in the distance denoised point cloud, draw a neighborhood sphere for each point with itself as the center and the preset neighborhood search radius as the radius; All points that fall within the neighborhood sphere are regarded as the neighborhood points of the point; The number of neighborhood points of each point in the distance denoised point cloud is obtained, and the points whose number of neighborhood points is less than the preset neighborhood point threshold are identified as outliers and removed to obtain the radius denoised point cloud.
5. The three-dimensional point cloud reconstruction method according to claim 4, characterized in that: The acquisition of the first angle subject point cloud and the second angle subject point cloud both include: The cluster neighborhood radius is preset, and for each point in the radius denoised point cloud, the point whose number of points within the cluster neighborhood radius is greater than the preset density threshold is determined as a core point; Based on the preset clustering neighborhood radius, obtain the neighborhood of each core point; The core points with intersections in the neighborhood are divided into a cluster, and the set of all clusters is the clustering result; The largest cluster in the clustering result is extracted as the main cluster, and all the core points in the main cluster are used to form the main point cloud.
6. A device using the three-dimensional point cloud reconstruction method according to any one of claims 1 to 5, characterized in that: include: The main point cloud construction module is used to obtain the first-angle point cloud data and the second-angle point cloud data of the object to be reconstructed, and then perform denoising and clustering respectively to obtain the first-angle main point cloud and the second-angle main point cloud; The acquisition of the first-angle point cloud data and the second-angle point cloud data includes: establishing a binocular camera model, using checkerboard calibration to obtain the internal and external parameter matrix of the binocular camera; using the phase shift method combined with the complementary Gray code technology to generate a coding pattern; projecting the coding pattern onto the surface of the object to be reconstructed, and using the binocular camera to collect and obtain the left-eye grating image and the right-eye grating image of the object to be reconstructed from a preset angle; performing phase unwrapping on the left-eye grating image and the right-eye grating image respectively to extract the left-eye phase information and the right-eye phase information; based on the left-eye phase information and the right-eye phase information, using epipolar correction, lateral constraints and global local search, phase matching is performed on the left-eye grating image and the right-eye grating image to obtain matching pixel points; based on the matching pixel points and the internal and external parameter matrix of the binocular camera, using the triangulation principle, the three-dimensional spatial coordinates of the matching pixel points are calculated to obtain the point cloud data of the object to be reconstructed at the preset angle; A high-order spatial matrix construction module is used to construct a similarity matrix based on preset matching point pairs between the subject point cloud at the first angle and the subject point cloud at the second angle; obtain the matching confidence of each pair of matching point pairs in the similarity matrix using spectral decomposition technology, and use the matching point pair with the local maximum confidence as the matching seed point pair; binarize the similarity matrix, count all matching point pairs in the matching seed point pairs that are compatible with the binarized similarity matrix, and construct a high-order spatial matrix; The optimal transformation matrix construction module is used to calculate the similarity of all matching point pairs in the high-order spatial matrix, obtain the preset number of matching point pairs with the highest similarity, and form a target matching point pair set; based on the target matching point pair set, the singular value decomposition method is used to obtain the rigid body transformation matrix, and then the interior point counting criterion is used to obtain the optimal transformation matrix; The registration module is used to transform the first angle point cloud data based on the optimal transformation matrix to obtain the optimal initial pose point cloud as the three-dimensional point cloud of the object to be reconstructed.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the three-dimensional point cloud reconstruction method according to any one of claims 1 to 5 are implemented.
