Three-dimensional point cloud registration method for image pre-registration and multi-criterion mismatching suppression

Abstract

The invention discloses a three-dimensional point cloud registration method for image pre-registration and multi-criterion mismatching suppression, which relates to the technical field of point cloud registration, and comprises the following steps of: firstly, enhancing surface texture information of a to-be-measured object by using a projection speckle image to perform data acquisition, and then positioning an optimal registration area of the to-be-registered image by using an image partitioning and template matching algorithm, image pre-registration is realized by performing feature point detection and registration on the region, and adverse effects caused by the non-overlapping region of the to-be-registered image are inhibited; thirdly, according to the coordinate transformation relation, obtaining corresponding three-dimensional matching point pairs through the two-dimensional matching point pairs obtained through image pre-registration, and completing point cloud coarse registration; and finally, adopting a classic ICP algorithm to realize point cloud precise registration. In order to improve the precision and efficiency of registration, multiple error matching suppression strategies are integrated in the stages of image pre-registration and point cloud coarse registration. According to the method, high-efficiency and high-precision registration of the point clouds which are limited in overlapping rate and lack of features can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud registration, and in particular to a three-dimensional point cloud registration method for image pre-registration and multi-criteria mismatch suppression. Background Art

[0002] Point cloud registration is to transform point cloud data collected from different perspectives into a unified reference coordinate system by estimating the rigid body transformation relationship between point clouds. It is the basis for high-level information acquisition tasks such as industrial inspection, robot environment perception, and 3D scene reconstruction. In recent years, with the rapid development of technologies such as intelligent production lines and unmanned driving, people have higher and higher requirements for point cloud registration.

[0003] Domestic and foreign scholars have developed a large number of point cloud registration methods to meet the needs of different application scenarios. According to whether the registration process relies on feature extraction and matching, they can be divided into two categories: feature-based registration methods and featureless registration methods. Feature-based registration methods first extract key feature points with recognition and stability, and then use these feature points to establish the correspondence between point clouds to complete the registration. They can be divided into methods based on physical assistance and methods based on point cloud geometric features. The method based on physical assistance relies on landmark points or high-precision motion mechanisms to calculate the transformation matrix. The method based on point cloud geometric features describes the features based on the point neighborhood information, establishes the feature descriptor of the point, and thus determines the correspondence between the two point clouds to complete the registration. He et al. defined the cross entropy of the point according to the geometric features of the point, and used statistical methods to select points with cross entropy tending to the middle value as registration points to complete the registration. Zhao et al. proposed a local normal deviation statistical histogram descriptor based on the local reference axis and the average normal deviation value, and achieved good registration results in multiple public data sets.

[0004] Unlike feature-based methods, featureless registration methods do not rely on feature extraction and matching. Representative methods include iterative closest point (ICP) based methods, probability-based methods, and exhaustive methods. The ICP proposed by Besl et al. is a classic point cloud registration method. This method regards the point pairs with the smallest distance in two point clouds as corresponding points, and iteratively optimizes so that the distance error function of the point pair set converges to the minimum value to estimate the transformation matrix. It has high accuracy but strict requirements on the initial pose. In order to improve the performance of the ICP algorithm, scholars at home and abroad have proposed a variety of improved algorithms. The Trimmed ICP algorithm proposed by Chetverikov et al. uses the overlap rate coefficient to remove outliers and effectively register low overlap rate point clouds. LV et al. reduce the influence of outliers by weighting the distance function of positive and negative measurement points. The most representative method based on statistical probability is the Normal Distributions Transform (NDT) algorithm proposed by Magnusson et al., which estimates the point cloud distribution through the probability density function. Although it is efficient, it is sensitive to the initial value. The most representative method based on exhaustive thinking is 4PCS (Four Point Congruent Set) proposed by Aiger et al., which determines the optimal rigid body transformation by searching for congruent four-point sets.

[0005] The registration accuracy of methods based on physical assistance, point cloud geometric features, and exhaustive thinking is not high, but the robustness is good; the ICP-based and NDT-based methods are high in accuracy, but sensitive to initial values. In engineering practice, in order to obtain better matching results, the "coarse-fine" two-step combination method is often used in engineering applications. Wang Xiaoming et al. proposed a fast registration strategy based on Harris feature coarse registration and NDT-ICP fine registration, which has been successfully applied to the size detection of large-scale span arch beam composite structure steel box arch prefabricated parts. Li et al. first established the initial registration point set through mutual projection between similar local areas, and then determined the final registration point set through rigid transformation consistency constraints to achieve high-precision registration. Tang Yuyang et al. proposed a point cloud registration method that combines PCA-based coarse registration and KD-ICP-based fine registration. Based on this method, the accuracy of the rotor blade torsion angle was achieved.

[0006] The above studies have improved the performance of the point cloud registration algorithm to a certain extent, and have a good registration accuracy when the overlap is large and the quality of the visual features of the surface of the measured image is high. However, in applications such as automated flexible inspection lines for parts, the overlapping area of ​​two point clouds is difficult to strictly guarantee or guaranteeing the overlapping area will seriously affect the efficiency; the surface visual features of the measured object are usually not rich and obvious enough. When the overlap rate of the two point clouds is limited (<50%) and the target surface features are lacking, the above methods are difficult to achieve high-precision and high-efficiency point cloud registration. Therefore, how to quickly find the overlapping area and ensure the correct transformation relationship to complete the registration under low overlap rates is still a hot issue at present. Summary of the invention

[0007] In view of the above technical problems to be solved, the present invention provides a three-dimensional point cloud registration method with image pre-registration and multi-criteria mismatch suppression, which can achieve high-efficiency and high-precision registration of point clouds with limited overlap and lack of surface features.

[0008] In order to solve the above technical problems, the technical solution proposed by the present invention is: A 3D point cloud registration method with image pre-registration and multi-criteria mismatch suppression includes three stages: data acquisition, texture image pre-registration, and 3D point cloud registration. Specifically, the method includes the following steps: Step S1, data acquisition stage, projecting speckle image to cover the surface of the part to be tested, collecting surface point cloud data of the object to be tested and its corresponding two-dimensional image; Step S2, texture image pre-registration stage, first, the best registration area is located for two frames of images with different viewing angles, and feature point detection and registration are performed only in this area to suppress the adverse effects brought by the non-overlapping area information of the two frames of images, and then the distance constraint and the random sampling consistency of the homography matrix are used to suppress the wrong matching points to obtain the final two-dimensional matching point pair; Step S3, the three-dimensional point cloud registration stage, the corresponding three-dimensional matching point pairs are calculated through the two-dimensional matching point pairs obtained by texture image pre-registration and the coordinate transformation relationship, the wrong matching is suppressed based on the random sampling consistency of the rigid body transformation matrix, and the transformation matrix is ​​calculated to complete the rough registration; finally, the classic ICP algorithm is used to complete the fine registration of the point cloud.

[0009] As a further improvement of the above technical solution: Preferably, the step S2 specifically includes the following contents: S2-1, optimal registration area positioning: First, the most similar area of ​​the two frames is found as the best registration area, and only the information of the best registration area is used for image registration to suppress the adverse effects of the information of the non-overlapping area; then the template with the largest normalized mutual correlation coefficient with the matching position and its corresponding matching area are taken as the most similar area of ​​the two perspective images, that is, the best registration area; S2-2, feature point detection and registration: The AKAZE algorithm is used to extract features from the best matching area of ​​the texture image; S2-3, image mismatch suppression: Distance constraint and sampling consistency of homography matrix are used to suppress false matches respectively. Distance constraint false match suppression calculates the distance between feature description vectors of matching point pairs and regards matching point pairs with too large distance as false matches.

[0010] Preferably, the normalized correlation coefficient is , the calculation formula is: ; in, is the template image; For the template image in the image The center position of the match, is the average gray value of the template pixels, For images The average gray value of the pixels overlapping the template image. is the horizontal coordinate of the pixel in the template image, and its value range is [0, ], is the pixel ordinate in the template image, ranging from [0, ], The upper left corner of the template image is The horizontal coordinate in the image, The upper left corner of the template image is The vertical coordinate in the image, is the width of the template image, is the height of the template image; is the pixel value in the template image in the image The actual location in The value range of is [-1,1], When it is 1, it indicates a perfect match, and when it is -1, it indicates the maximum difference between the two image blocks.

[0011] Preferably, the feature point detection and registration includes the following contents: S2-2-1, constructing a nonlinear scale space through a nonlinear diffusion filter: ; in, Represents the original image from which features are to be extracted; is the evolution time, the larger its value is, the simpler the image representation is; and They are the divergence operator and the gradient operator respectively; is the conduction function, which is used to convert The local structure is diffused and the detail information is preserved; The explicit expression of the nonlinear diffusion filter is: ; in, for The image after one display diffusion iteration, Time for evolution Evolutionary images; is the identity matrix; is the time step; For images The conduction matrix of A is the discretized diffusion operator; the original image is obtained by using fast display diffusion. Nonlinear scale space of S2-2-2, calculate images at different nonlinear scales After obtaining the Hessian matrix, the AKAZE algorithm completes the feature point detection by finding the local extreme value in the nonlinear scale space; S2-2-3, use M-LDB descriptor to generate feature point description vector, and complete the feature point matching of the best registration area of ​​the image through brute force matching.

[0012] Preferably, the step S3 specifically includes the following steps: S3-1, 3D projection of 2D matching point pairs: After obtaining the two-dimensional coordinates of the matching point pair, the coordinates of the matching point pair in three-dimensional space are calculated by combining the corresponding depth image and the coordinate change relationship; S3-2, mismatch suppression of matching point pairs and coarse registration of point clouds: Based on the RANSAC algorithm, the sampling consistency error matching suppression process of the point cloud transformation matrix is ​​carried out to find the optimal transformation matrix with the largest number of inliers, and the corresponding inliers set is output; the obtained inliers set is the three-dimensional matching point pair after error matching suppression, and the matching situation of the three-dimensional matching point pair after error matching suppression is solved by the singular value decomposition method to solve the transformation relationship between the source point cloud and the target point cloud in this matching situation, that is, the rotation and translation transformation matrix between the point clouds; finally, the source point cloud is transformed according to the rotation and translation transformation matrix, and the source point cloud and the target point cloud are converted to the same coordinate system to complete the point cloud coarse registration; S3-3, pre-extract ICP fine registration of overlapping areas: First, the nearest neighbor search algorithm is used to extract the overlapping area of ​​the point cloud to be registered, and the point cloud is downsampled. Then the ICP algorithm is used to calculate the optimal transformation matrix of the two point clouds to complete the precise registration.

[0013] Preferably, the sampling consistency error matching suppression process of the point cloud transformation matrix is ​​as follows: randomly selecting 6 point pairs from the three-dimensional matching point pairs as sample points, and calculating the rigid body transformation matrix using the three-dimensional coordinates of the sample points and their corresponding relationships ,pass Calculate the corresponding positions of the remaining matching point pairs after transformation, and calculate the distance between them. If the distance of a matching point pair is less than the preset threshold TS, the point pair is considered as an inner point, otherwise it is considered as an outer point. Repeat the above steps until the maximum number of iterations K is reached. b , find the optimal transformation matrix with the largest number of interior points and output the corresponding interior point set.

[0014] Preferably, the process of extracting the overlapping area of ​​the point cloud to be registered by the nearest neighbor search algorithm is as follows: First, two KD-trees are constructed for the source point cloud and the target point cloud respectively. For each point in the source point cloud , by performing a nearest neighbor search in the KD-tree of the target point cloud, find its nearest neighbor point , and calculate and Then judge the distance between and Is the distance between them less than the distance threshold? If so, then and If a point pair is identified as a matching point pair, the area where it is located is considered the overlapping area of ​​the two point clouds. If not, it is considered an incorrect matching point pair and is discarded. Repeat the above steps for all points in the two point clouds to obtain the entire overlapping area of ​​the two point clouds.

[0015] Preferably, the ICP algorithm calculates the optimal transformation matrix of two point clouds to complete precise registration, the steps are: (1) In the overlapping area, find the nearest point in the target point cloud for each point in the source point cloud; (2) Calculate the rotation matrix R and translation vector T between the point clouds based on the nearest point pair, and apply the calculated transformation matrix to the source point cloud to update its position; after the update, the mean square error of the Euclidean distance between the corresponding point pairs of the source point cloud and the target point cloud is : ;

[0016] in, is the number of nearest neighbor pairs in the overlapping area, and are points in the source point cloud and the target point cloud for a nearest neighbor point pair respectively; (3) Setting the error threshold and the maximum number of iterations K c ,like Or reach the maximum number of iterations K c , then stop the iteration and obtain the final transformation matrix, otherwise return to step (1).

[0017] The three-dimensional point cloud registration method with image pre-registration and multi-criteria mismatch suppression provided by the present invention has the following advantages compared with the prior art: The three-dimensional point cloud registration method of the present invention with image pre-registration and multi-criteria mismatch suppression first realizes the rough matching of the three-dimensional point cloud based on the two-dimensional image pre-registration result suppressed by non-overlapping area information, and completes the overlapping area identification and position transformation estimation through the optimal area positioning and AKAZE feature extraction algorithm. This method effectively suppresses the mismatch caused by the change of scene illumination and the lack of texture features, so that it can achieve good alignment under the condition of limited overlap rate. The present invention adopts the classic ICP algorithm to realize the precise registration of point clouds, and integrates a variety of mismatch suppression strategies in the image pre-registration and point cloud rough registration stages to improve the accuracy and efficiency of registration. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flow chart of the optimal registration area positioning of the present invention.

[0019] Figure 2 These are six different types of parts to be tested used in the experimental verification of the present invention.

[0020] Figure 3 This is a comparison chart of the registration results of the method of the present invention and comparison algorithm 1 and comparison algorithm 2 in the experimental verification of the present invention. DETAILED DESCRIPTION

[0021] The specific embodiments of the present invention are described in detail below. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0022] The three-dimensional point cloud registration method of the present application for image pre-registration and multi-criteria mismatch suppression includes three stages: data acquisition, texture image pre-registration, and three-dimensional point cloud registration, including the following steps: Step S1, data acquisition stage, the projected speckle image covers the surface of the part to be tested to enhance its texture information, and at the same time, the surface point cloud data of the object to be tested and its corresponding two-dimensional image and depth image are collected.

[0023] Step S2, the texture image pre-registration stage, first, locate the best registration area for the two frames of images with different viewing angles, and only perform feature point detection and registration on this area, so as to suppress the adverse effects brought by the non-overlapping area information of the two frames of images; use the distance constraint and the random sampling consistency of the homography transformation matrix to suppress the erroneous matching points to obtain the final two-dimensional matching point pair.

[0024] In the two-dimensional texture image pre-registration stage, similar areas of the images to be detected under two different perspectives of the part are located as the best registration area through image segmentation and template matching; feature point detection and registration are performed only on the best registration area, thereby suppressing the adverse effects of non-overlapping areas of the images to be registered on image registration; finally, the mismatched point pairs are eliminated to obtain a more accurate registration result.

[0025] Specifically include the following: S2-1, optimal registration area positioning: During the image registration process, images of the same detection object collected from different viewing angles need to overlap partially. However, if the proportion of overlapping areas is too low, the large number of non-overlapping areas in the two frames of images will have a serious adverse effect on image registration, and even cause registration failure. In response to the above problems, the method proposed in the present invention first finds the most similar area of ​​the two frames of images as the best registration area, and only uses the information of the best registration area for image registration, thereby effectively suppressing the adverse effects of the non-overlapping area information.

[0026] The optimal registration region positioning method process is as follows Figure 1 As shown in Figure 2. Assume that the images to be detected under two different viewing angles are and , the size is × , the size of the template image is b×b. First, the contrast-limited adaptive histogram equalization is used to improve and The contrast of the image is then segmented into sub-blocks, each sub-block is used as a template, in the image The template matching method that calculates the normalized cross-correlation coefficient is used to slide and find the area that best matches each template, and record The normalized correlation coefficient between the sub-block and its most matching area and its matching area. Template image and image The normalized cross-correlation coefficient The calculation formula is: (1); in, is the template image; For the template image in the image The center position of the match, is the average gray value of the template pixels, For images The average gray value of the pixels overlapping the template image. is the horizontal coordinate of the pixel in the template image, and its value range is [0, ], is the pixel ordinate in the template image, ranging from [0, ], The upper left corner of the template image is The horizontal coordinate in the image, The upper left corner of the template image is The vertical coordinate in the image, is the width of the template image, is the height of the template image; is the pixel value in the template image in the image The actual location in The value range of is [-1,1], When it is 1, it means a perfect match, and when it is -1, it means the difference between the two image blocks is the largest. It is stored in the computer as a two-dimensional matrix, each Represents the image at coordinates The gray value at .

[0027] Finally, the template with the largest normalized correlation coefficient with the matching position and its corresponding matching area are taken as the most similar area of ​​the two view images, that is, the best registration area.

[0028] S2-2, Feature point detection and registration: After successfully locating the best registration area, the AKAZE algorithm is used to extract features from the best matching area of ​​the texture image. This algorithm is a feature detection algorithm based on nonlinear scale space.

[0029] S2-2-1, constructing a nonlinear scale space through a nonlinear diffusion filter: (2) in, Represents the original image from which features are to be extracted; The evolution time controls the degree to which the filter smoothes the image and retains the image details. The larger the value, the simpler the image representation. and They are the divergence operator and the gradient operator respectively; is the conduction function, which is used to convert The local structure is diffused and the details are preserved: (3) in, For the original image After Gaussian smoothing, the evolution time ,space The gradient vector at . Function The expression is: (4) In order to perform effective feature detection at different scales, continuous scale parameters must be sampled. Such sampling is often performed in a logarithmic manner. Such a discrete scale set can be visualized as a "pyramid". The function The scale of the constructed nonlinear scale space increases logarithmically with the number of pyramid layers. If the image pyramid has Layer images, each layer has sub-layers, then the pyramid Tier The scale of the sublayer for: (5) in, is the serial number of the layer where the scale is located, , is the total number of images in the nonlinear scale space, ; is the initial scale value. Transformed into an evolutionary moment , the explicit expression of the nonlinear diffusion filter can be obtained as: (6) in, for The image after one display diffusion iteration, for Evolutionary images; is the identity matrix; is the time step; For images The conduction matrix of , A is the discretized diffusion operator. Using fast display diffusion to solve equation (6), we can get the original image Nonlinear scale space.

[0030] S2-2-2, calculate images at different nonlinear scales The Hessian matrix of : (7) in, is the normalized scale factor; and are the second-order horizontal and vertical partial derivatives of the Laplace function of Gaussian; is the second-order mixed cross partial derivative of the Gaussian Laplace function. After obtaining the Hessian matrix, the AKAZE algorithm completes feature point detection by finding local extreme values ​​in the nonlinear scale space.

[0031] S2-2-3, use M-LDB descriptor to generate feature point description vector, and complete the image through brute force matching and The feature points in the best registration area are matched.

[0032] S2-3, image mismatch suppression: In the field environment, M-LDB feature point description and brute force matching will result in a large number of wrong matching point pairs. In order to obtain high-quality matching results, distance constraints and sampling consistency of homography matrices are used to suppress wrong matching.

[0033] Distance-constrained mismatch suppression calculates the distance between the feature description vectors of matching point pairs, and regards matching point pairs with too large distance (i.e. too low similarity) as mismatches. The feature points in and The feature points in is a pair of matching points, and The Euclidean distance between feature vectors is described as , When it is greater than the set threshold TH, and For wrong matching.

[0034] Because under the assumption that the scene is rigid and unchanged and satisfies the homography transformation, the mapping between the two images can be described by a set of fixed parameters, which reflect the camera motion, perspective change or geometric relationship of the planar scene. and The transformation relationship between them is certain. Therefore, if the transformation relationship of some matching point pairs is too different from that of most other matching point pairs, these matching point pairs can be regarded as wrong matches. Based on the RANSAC algorithm, the sampling consistency error matching suppression process of the homography matrix is ​​as follows: 4 pairs are randomly selected from m groups of matching point pairs to calculate the homography matrix H; the image is calculated by H All matching points in the image The coordinates in the image; if a matching point The matching points in the image Coordinates and images in If the distance between the matching points in is greater than the threshold TR, then this matching point is regarded as an external point of the homography matrix H, otherwise it is regarded as an internal point of the homography matrix H; repeat the above steps until the number of runs reaches the maximum number of iterations Ka When , the homography matrix with the most inliers is found, and the outliers of this homography matrix are regarded as false matches.

[0035] Step S3, the 3D point cloud registration stage, uses the 2D matching point pairs obtained by texture image pre-registration and the coordinate transformation relationship to calculate the corresponding 3D matching point pairs, uses the random sampling consistency of the rigid body transformation matrix to suppress erroneous matching, calculates the transformation matrix to complete the coarse registration; finally, uses the classic ICP algorithm to complete the fine registration of the point cloud.

[0036] The 3D point cloud registration stage is divided into two steps: coarse registration and fine registration. In the coarse registration step, the corresponding 3D matching point pairs are obtained from the 2D matching point pairs obtained in the texture image pre-registration stage according to the coordinate transformation relationship; then, the depth information and RANSAC are applied to further suppress the mismatch of the 3D matching point pairs; finally, the classic ICP algorithm is used to achieve point cloud fine registration.

[0037] S3-1, 3D projection of 2D matching point pairs: After obtaining the two-dimensional coordinates of the matching point pair, the coordinates of the matching point pair in three-dimensional space are calculated by combining the corresponding depth image and the coordinate change relationship. If the world coordinate system coincides with the camera coordinate system used to obtain the two-dimensional image, the three-dimensional coordinates corresponding to the image matching point The calculation formula is: (8) in, is the pixel coordinate of the two-dimensional matching point; is the camera principal point, which is the image coordinate at the intersection of the camera optical axis and the imaging plane; and The camera is The equivalent focal length on the axis; d is the coordinate in the depth image The corresponding gray value; is the depth scale factor of the camera.

[0038] S3-2, mismatch suppression of matching point pairs and coarse registration of point clouds: Similar to the situation in 2D image matching, the transformation relationship between the source point cloud and the target point cloud is determined. Therefore, if the transformation relationship of the matching point pairs is too different from that of most other matching point pairs, these matching point pairs can be regarded as mismatches. Based on the RANSAC algorithm, the sampling consistency error matching suppression process of the point cloud transformation matrix is ​​as follows: 6 point pairs are randomly selected from the 3D matching point pairs, and the 3D coordinates of these sample points and their corresponding relationships are used to calculate the rigid body transformation matrix. ,pass Calculate the corresponding positions of the remaining matching point pairs after transformation, and calculate the distance between them. If the distance of a matching point pair is less than the preset threshold TS, the point pair is considered as an inner point, otherwise it is considered as an outer point. Repeat the above steps until the maximum number of iterations K is reached. b , find the optimal transformation matrix with the largest number of interior points and output the corresponding interior point set.

[0039] The internal point set obtained by the above method is the three-dimensional matching point pair after false matching is suppressed. The singular value decomposition (SVD) method is used to solve the transformation relationship between the source point cloud and the target point cloud in this matching situation, that is, the rotation and translation transformation matrix between the point clouds. Finally, the source point cloud is transformed according to the rotation and translation transformation matrix, and the source point cloud and the target point cloud are converted to the same coordinate system to complete the rough registration of the point cloud.

[0040] S3-3, pre-extract ICP fine registration of overlapping areas: Only in the overlapping area of ​​two point clouds are there real matching point pairs. When the overlap of the two point clouds is insufficient, if all point clouds are involved in precise matching, it will not only seriously affect the efficiency, but also may lead to a series of problems such as being more likely to fall into local optimality. Therefore, before using the classic ICP algorithm for registration, the method of the present invention first uses the nearest neighbor search algorithm to extract the overlapping area of ​​the point cloud to be registered, and downsamples the point cloud, thereby improving efficiency and stability.

[0041] The process of finding the overlapping area of ​​two point clouds using the nearest neighbor search algorithm is as follows: First, two KD-trees are constructed for the source point cloud and the target point cloud respectively. , by performing a nearest neighbor search in the KD-tree of the target point cloud, find its nearest neighbor point , and calculate and The distance between and Is the distance between them less than the distance threshold? If so, then and If a point pair is identified as a match, the area where it is located is considered the overlapping area of ​​the two point clouds. If not, it is considered an incorrect match and is discarded. Repeat the above steps for all points in the two point clouds to obtain the entire overlapping area of ​​the two point clouds.

[0042] After obtaining the overlapping area, the ICP algorithm is used to calculate the optimal transformation matrix of the two point clouds to complete the precise registration. The steps are: (1) In the overlapping area, find the nearest point in the target point cloud for each point in the source point cloud.

[0043] (2) Calculate the rotation matrix R and translation vector T between the point clouds based on the nearest point pair, and apply the calculated transformation matrix to the source point cloud to update its position. The mean square error of the Euclidean distance between the corresponding point pairs between the source point cloud and the target point cloud after the update is : (9) in, is the number of nearest neighbor pairs in the overlapping area, and are points in the source point cloud and the target point cloud, respectively.

[0044] (3) Setting the error threshold and the maximum number of iterations K c ,like Or reach the maximum number of iterations K c , then stop the iteration and obtain the final transformation matrix, otherwise return to step (1).

[0045] Experimental verification: Three experiments were designed to verify the effectiveness of the method of the present invention, including two-dimensional texture image registration, point cloud rough registration and overall performance evaluation. The experimental program was implemented on the Visual Studio 2017 platform in combination with PCL1.8.1 and OpenCV3.4.1 open source libraries. The hardware platform on which the program runs is a 2.90GHz AMD Ryzen7 4800H CPU with 16GB of memory.

[0046] Six different types of parts are used for point cloud registration experiments, namely (a) rectangular parts, (b) gears, (c) bolts, (d) square parts, (e) U-shaped fixtures, and (f) small fan bases. Figure 2 shown.

[0047] The method of this application is compared with two commonly used point cloud registration methods in engineering. Comparison method 1 first uses FPFH to calculate the point feature histogram, then uses SAC-IA for coarse registration of similar points, and finally uses ICP to complete fine registration. Comparison method 2 uses the 4PCS algorithm for coarse point cloud registration, and the fine registration scheme is the same as comparison method 1. The maximum mean square error and transformation matrix error thresholds of the three algorithms are taken as 1e-6m and 1e-10m respectively, the maximum number of iterations is 1000, and the number of nearest neighbor searches in the comparison algorithm is set to 7.

[0048] The overlap ratio used in the experiment Quantitatively characterize the degree of overlap of point clouds: (10) in, is the number of point pairs in the overlapping area, and are the number of points in the source point cloud and the target point cloud, respectively. Figure 3 Visualization of the registration results of the three methods. Figure 3 The point cloud overlap rates of (a) and (b) are low, approximately 40% and 50% respectively. Figure 3 The overlap rates of (c), (d), (e), and (f) in Fig. 1 are all greater than 80%. The registration effect of the present invention is significantly better than that of the comparison algorithm. Figure 3 (c), (d) and (e) are successfully registered, compared with Algorithm 2 Figure 3 (c) and (e) of are successfully registered, but the uniformity of the cross-distribution of the two point clouds shows that the registration effect is worse than that of the present application method. Figure 3 There is obvious misalignment in (a), (b) and (f).

[0049] Table 1 shows the point cloud registration accuracy and time consumption data of the three methods. The average time consumption of the method in this application is 3.540S, and the average mean square error of the registration is 0.830mm. 2 , and outperforms the two compared algorithms in terms of mean square error and average algorithm time consumption. The proposed method effectively solves the problem that parts are difficult to align effectively due to low overlap, lack of features and repetition, and has good robustness for different types of point cloud data with different initial pose relationships.

[0050] Table 1: Comparison of point cloud registration results of three methods:

[0051] In the process of point cloud registration, it is difficult to achieve high-precision and high-efficiency registration due to the inability to guarantee the overlap rate of two point clouds, the lack of target surface features and the influence of factors such as on-site light sources, and it cannot meet the increasingly high requirements for point cloud registration in applications such as intelligent production lines. Experimental results show that the method proposed in this application can achieve high-efficiency and high-precision registration when processing part point clouds with limited overlap rate, lack of features and duplication, and its effectiveness is significantly better than the existing mainstream point cloud registration scheme. The method of this application is expected to further enhance the three-dimensional information acquisition capabilities of intelligent systems in complex application scenarios.

[0052] The above implementation cases are only preferred embodiments of the present invention and are not intended to limit the present invention in any form. Although the present invention has been disclosed as above with preferred embodiments, they are not intended to limit the present invention. Therefore, any simple modification, equivalent changes and modifications made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the scope of protection of the technical solution of the present invention.

Claims

1. A three-dimensional point cloud registration method with image pre-registration and multi-criteria mismatch suppression, characterized in that: It includes three stages: data acquisition, texture image pre-registration, and 3D point cloud registration, which specifically includes the following steps: Step S1, data acquisition stage, projecting speckle image to cover the surface of the part to be tested, collecting surface point cloud data of the object to be tested and its corresponding two-dimensional image; Step S2, texture image pre-registration stage, first, the best registration area is located for two frames of images with different viewing angles, and feature point detection and registration are performed only in this area to suppress the adverse effects brought by the non-overlapping area information of the two frames of images, and then the distance constraint and the random sampling consistency of the homography matrix are used to suppress the wrong matching points to obtain the final two-dimensional matching point pair; Step S3, the three-dimensional point cloud registration stage, the corresponding three-dimensional matching point pairs are calculated through the two-dimensional matching point pairs obtained by texture image pre-registration and the coordinate transformation relationship, the wrong matching is suppressed based on the random sampling consistency of the rigid body transformation matrix, and the transformation matrix is ​​calculated to complete the rough registration; finally, the classic ICP algorithm is used to complete the fine registration of the point cloud.

2. The three-dimensional point cloud registration method with image pre-registration and multi-criteria mismatch suppression according to claim 1, characterized in that: The step S2 specifically includes the following contents: S2-1, optimal registration area positioning: First, the most similar area of ​​the two frames is found as the best registration area, and only the information of the best registration area is used for image registration to suppress the adverse effects of the information of the non-overlapping area; then the template with the largest normalized mutual correlation coefficient with the matching position and its corresponding matching area are taken as the most similar area of ​​the two perspective images, that is, the best registration area; S2-2, feature point detection and registration: The AKAZE algorithm is used to extract features from the best matching area of ​​the texture image; S2-3, image mismatch suppression: Distance constraint and sampling consistency of homography matrix are used to suppress false matches respectively. Distance constraint false match suppression calculates the distance between feature description vectors of matching point pairs and regards matching point pairs with too large distance as false matches.

3. The three-dimensional point cloud registration method with image pre-registration and multi-criteria mismatch suppression according to claim 2, characterized in that: The normalized cross-correlation coefficient is , the calculation formula is: ； in, is the template image; For the template image in the image The center position of the match, is the average gray value of the template pixels, For images The average gray value of the pixels overlapping the template image. is the horizontal coordinate of the pixel in the template image, and its value range is [0, ], is the pixel ordinate in the template image, ranging from [0, ], The upper left corner of the template image is The horizontal coordinate in the image, The upper left corner of the template image is The vertical coordinate in the image, is the width of the template image, is the height of the template image; is the pixel value in the template image in the image The actual location in The value range of is [-1,1], When it is 1, it indicates a perfect match, and when it is -1, it indicates the maximum difference between the two image blocks.

4. The three-dimensional point cloud registration method with image pre-registration and multi-criteria mismatch suppression according to claim 2, characterized in that: The feature point detection and registration includes the following contents: S2-2-1, constructing a nonlinear scale space through a nonlinear diffusion filter: ； in, Represents the original image from which features are to be extracted; is the evolution time, the larger its value is, the simpler the image representation is; and They are the divergence operator and the gradient operator respectively; is the conduction function, which is used to convert The local structure is diffused and the detail information is preserved; The explicit expression of the nonlinear diffusion filter is: ； in, for The image after one display diffusion iteration, Time for evolution Evolutionary images; is the identity matrix; is the time step; For images The conduction matrix of A is the discretized diffusion operator; the original image is obtained by using fast display diffusion. Nonlinear scale space of S2-2-2, calculate images at different nonlinear scales After obtaining the Hessian matrix, the AKAZE algorithm completes the feature point detection by finding the local extreme value in the nonlinear scale space; S2-2-3, use M-LDB descriptor to generate feature point description vector, and complete the feature point matching of the best registration area of ​​the image through brute force matching.

5. The three-dimensional point cloud registration method with image pre-registration and multi-criteria mismatch suppression according to claim 2, characterized in that: The step S3 specifically includes the following steps: S3-1, 3D projection of 2D matching point pairs: After obtaining the two-dimensional coordinates of the matching point pair, the coordinates of the matching point pair in three-dimensional space are calculated by combining the corresponding depth image and the coordinate change relationship; S3-2, mismatch suppression of matching point pairs and coarse registration of point clouds: Based on the RANSAC algorithm, the sampling consistency error matching suppression process of the point cloud transformation matrix is ​​carried out to find the optimal transformation matrix with the largest number of inliers, and the corresponding inliers set is output; the obtained inliers set is the three-dimensional matching point pair after error matching suppression, and the matching situation of the three-dimensional matching point pair after error matching suppression is solved by the singular value decomposition method to solve the transformation relationship between the source point cloud and the target point cloud in this matching situation, that is, the rotation and translation transformation matrix between the point clouds; finally, the source point cloud is transformed according to the rotation and translation transformation matrix, and the source point cloud and the target point cloud are converted to the same coordinate system to complete the point cloud coarse registration; S3-3, pre-extract ICP fine registration of overlapping areas: First, the nearest neighbor search algorithm is used to extract the overlapping area of ​​the point cloud to be registered, and the point cloud is downsampled. Then the ICP algorithm is used to calculate the optimal transformation matrix of the two point clouds to complete the precise registration.

6. The three-dimensional point cloud registration method with image pre-registration and multi-criteria mismatch suppression according to claim 5, characterized in that: The sampling consistency error matching suppression process of the point cloud transformation matrix is ​​as follows: randomly select 6 point pairs from the three-dimensional matching point pairs as sample points, and use the three-dimensional coordinates of the sample points and their corresponding relationships to calculate the rigid body transformation matrix. ,pass Calculate the corresponding positions of the remaining matching point pairs after transformation, and calculate the distance between them. If the distance of a matching point pair is less than the preset threshold TS, the point pair is considered as an inner point, otherwise it is considered as an outer point. Repeat the above steps until the maximum number of iterations K is reached. b , find the optimal transformation matrix with the largest number of interior points and output the corresponding interior point set.

7. The three-dimensional point cloud registration method with image pre-registration and multi-criteria mismatch suppression according to claim 5, characterized in that: The process of extracting the overlapping area of ​​the point cloud to be registered by the nearest neighbor search algorithm is as follows: First, construct two KD-trees for the source point cloud and the target point cloud respectively. ； For each point in the source point cloud , by performing a nearest neighbor search in the KD-tree of the target point cloud, find its nearest neighbor point , and calculate and Then judge the distance between and Is the distance between them less than the distance threshold? If so, then and If a point pair is identified as a matching point pair, the area where it is located is considered the overlapping area of ​​the two point clouds. If not, it is considered an incorrect matching point pair and is discarded. Repeat the above steps for all points in the two point clouds to obtain the entire overlapping area of ​​the two point clouds.

8. The three-dimensional point cloud registration method with image pre-registration and multi-criteria mismatch suppression according to claim 5, characterized in that: The ICP algorithm calculates the optimal transformation matrix of two point clouds to complete precise registration. The steps are: (1) In the overlapping area, find the nearest point in the target point cloud for each point in the source point cloud; (2) Calculate the rotation matrix R and translation vector T between the point clouds based on the nearest point pair, and apply the calculated transformation matrix to the source point cloud to update its position; The mean square error of the Euclidean distance between the corresponding point pairs between the source point cloud and the target point cloud after updating is : ； in, is the number of nearest neighbor pairs in the overlapping area, and are points in the source point cloud and the target point cloud for a nearest neighbor point pair respectively; (3) Setting the error threshold and the maximum number of iterations K c ,like Or reach the maximum number of iterations K c , then stop the iteration and obtain the final transformation matrix, otherwise return to step (1).

Images