A three-dimensional point cloud splicing method fusing two-dimensional contours and three-dimensional features

Abstract

This invention discloses a method for stitching 3D point clouds by fusing 2D contours and 3D features. The method includes: acquiring a first 3D point cloud and a second 3D point cloud to be stitched; generating multiple 3D rigid body transformation candidate matrices; transforming the first 3D point cloud based on each candidate matrix to obtain a corresponding third 3D point cloud; projecting the third and second point clouds onto the same 2D plane and extracting their respective 2D contours; calculating matching scores in both the 2D contour space and the 3D feature space; obtaining a joint score through a weighted fusion joint scoring function; selecting the candidate matrix with the highest joint score; performing coarse registration of the first and second 3D point clouds; and introducing curvature weights and normal consistency weights into the ICP algorithm to perform fine registration of the coarsely registered point clouds. This invention effectively improves the robustness of coarse registration by fusing 2D contour information and 3D feature information.

Description

Technical Field

[0001] This invention relates to the field of computer vision and point cloud processing technology, and in particular to a method for stitching three-dimensional point clouds that integrates two-dimensional contours and three-dimensional features. Background Technology

[0002] With the development of 3D sensing and computer vision technologies, structured light-based 3D measurement methods have been widely applied in fields such as industrial inspection, digital modeling, and reverse engineering. By using structured light projection combined with imaging equipment, point cloud data of the workpiece surface can be acquired from different perspectives, providing a data foundation for subsequent morphology analysis and 3D reconstruction. However, limited by the perspective and occlusion of a single scan, it is usually necessary to acquire point clouds from multiple perspectives and stitch them together to obtain a complete 3D model of the workpiece.

[0003] Existing point cloud stitching methods mainly rely on feature matching or iterative optimization. Feature-based methods often extract and match geometric features of the point cloud to obtain the initial pose. However, when the workpiece surface lacks significant features or has high noise levels, these methods are prone to mismatches, leading to stitching failure. In iterative optimization-based methods, the typical ICP algorithm can achieve high registration accuracy when the initial pose is relatively accurate. However, if the initial estimation deviation is large, it is prone to getting trapped in local optima, failing to guarantee the reliability of the stitching result. Furthermore, some methods attempt to project the point cloud onto a two-dimensional plane and achieve coarse alignment by comparing contour shapes. This approach has certain advantages in computational efficiency, but due to the lack of three-dimensional geometric constraints, the stitching accuracy is limited, especially on complex surfaces and high-curvature regions where errors are more likely to occur.

[0004] In summary, existing technologies suffer from insufficient robustness and limited accuracy in point cloud stitching, particularly in coarse registration which is highly dependent on the initial pose, and fine registration which struggles to guarantee accuracy at geometric boundaries and in high curvature regions. Summary of the Invention

[0005] To address the problems of existing point cloud stitching methods, such as strong dependence on initial pose and insufficient robustness in the coarse registration stage, and easy generation of errors in high curvature and geometric boundary regions in the fine registration stage, this invention provides a three-dimensional point cloud stitching method that integrates two-dimensional contours and three-dimensional features. This method achieves robust coarse registration of point clouds by introducing joint constraints of two-dimensional projected contours and three-dimensional feature information, and improves the overall accuracy and stability of stitching by using curvature and normal consistency weighting in the fine registration process.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides a method for stitching together three-dimensional point clouds that integrates two-dimensional contours and three-dimensional features, the method comprising: Obtain the first and second 3D point clouds to be stitched together; The ISS algorithm is used to extract three-dimensional feature points from the first and second three-dimensional point clouds. The FPFH descriptor of each three-dimensional feature point is calculated. The RANSAC algorithm is used to match the three-dimensional feature points, and multiple three-dimensional rigid body transformation candidate matrices are generated based on the matched three-dimensional feature point pairs. Based on each 3D rigid body transformation candidate matrix, the first 3D point cloud is transformed to obtain the corresponding third 3D point cloud. The third 3D point cloud and the second 3D point cloud are projected onto the same 2D plane and their respective 2D contours are extracted. The matching scores are calculated in the 2D contour space and the 3D feature space respectively. The joint score of the 3D rigid body transformation candidate matrix is ​​obtained through a weighted fusion joint scoring function. The candidate matrix of the three-dimensional rigid body transformation with the highest joint score is selected as the coarse registration transformation matrix, and the first three-dimensional point cloud and the second three-dimensional point cloud are coarsely registered based on the coarse registration transformation matrix. Curvature weight and normal consistency weight are introduced into the ICP algorithm to perform fine registration on the point cloud after coarse registration.

[0007] Following the above technical solution, multiple candidate matrices for three-dimensional rigid body transformation are generated based on matched three-dimensional feature point pairs, including: Each time, multiple pairs of matching 3D feature points are randomly selected, and a rigid body transformation candidate matrix is ​​obtained by solving the SVD decomposition method. Repeating the above steps multiple times yields several candidate matrices for three-dimensional rigid body transformation.

[0008] Following the above technical solution, extracting the two-dimensional contour includes: Projecting the 3D point cloud onto the target 2D plane yields a 2D point set; The convex hull algorithm is used to extract the two-dimensional contour line point set of the two-dimensional point set.

[0009] Following the above technical solution, the joint scoring function is defined as follows: ; In the formula, This is a candidate matrix for three-dimensional rigid body transformation. For 2D contour matching scores, For 3D feature matching score, For joint scoring, and These are the weighting coefficients.

[0010] Following the above technical solution, the two-dimensional contour matching score is defined as: ; In the formula, For the three-dimensional point cloud, it is the set of two-dimensional contour lines. The set of two-dimensional contour points for the second three-dimensional point cloud; This represents the minimum distance from a point on the 2D contour of the third 3D point cloud to the 2D contour of the second 3D point cloud. For the two-dimensional contour points of the third three-dimensional point cloud, These are the two-dimensional contour points of the second three-dimensional point cloud; This is the distance decay index.

[0011] Following the above technical solution, the three-dimensional feature matching score is defined as: ; In the formula, and This represents the i-th pair of matched 3D feature points in the third and second 3D point clouds. and These are three-dimensional feature point pairs. and The FPFH descriptor vector, where N is the number of 3D feature point pairs.

[0012] Following the above technical solution, curvature weight and normal consistency weight are introduced into the ICP algorithm to perform fine registration on the point cloud after coarse registration, including: For each point in any 3D point cloud after coarse registration Find the nearest point from another 3D point cloud. , forming matching point pairs ; For each pair of matching points Determine the curvature of the two matching points respectively. and the corresponding curvature weights The average curvature weights of the two matching points are taken as the curvature weight of the matching point pair. ; and determine the normals of the two matching points respectively. and Calculate the normal consistency weight of the matching point pair. ; The curvature weight and normal consistency weight of each pair of matched points are combined to define a comprehensive weight. ; Based on each pair of matching points Based on the corresponding comprehensive weights, a weighted objective function is constructed. ;in, For rotation matrix, It is a translation matrix. To match the number of point pairs; The optimal rotation matrix is ​​obtained by iteratively optimizing the objective function. Translation matrix This is to perform fine registration on the point cloud after coarse registration.

[0013] Secondly, the present invention provides a three-dimensional point cloud stitching system that integrates two-dimensional contours and three-dimensional features, for implementing the three-dimensional point cloud stitching method that integrates two-dimensional contours and three-dimensional features as described in the first aspect. The system includes: The 3D point cloud acquisition module acquires the first and second 3D point clouds to be stitched together. The candidate matrix generation module is used to extract three-dimensional feature points from the first and second three-dimensional point clouds using the ISS algorithm, calculate the FPFH descriptor of each three-dimensional feature point, match the three-dimensional feature points using the RANSAC algorithm, and generate multiple three-dimensional rigid body transformation candidate matrices based on the matched three-dimensional feature point pairs. The transformation projection and scoring module is used to transform the first three-dimensional point cloud based on each three-dimensional rigid body transformation candidate matrix to obtain the corresponding third three-dimensional point cloud. The third three-dimensional point cloud and the second three-dimensional point cloud are projected onto the same two-dimensional plane and their respective two-dimensional contours are extracted. The matching scores are calculated in the two-dimensional contour space and the three-dimensional feature space respectively. The joint score of the three-dimensional rigid body transformation candidate matrix is ​​obtained through a weighted fusion joint scoring function. The coarse registration module is used to select the candidate 3D rigid body transformation matrix with the highest joint score as the coarse registration transformation matrix, and to perform coarse registration on the first 3D point cloud and the second 3D point cloud based on the coarse registration transformation matrix. The fine registration module is used to introduce curvature weights and normal consistency weights in the ICP algorithm to perform fine registration on the point cloud after coarse registration.

[0014] Thirdly, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the first aspect.

[0015] Fourthly, the present invention provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0016] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects: This invention provides a method for stitching three-dimensional point clouds by integrating two-dimensional contours and three-dimensional features. In the coarse registration stage, the method integrates two-dimensional contours and three-dimensional features for joint scoring, which improves the tolerance to noise and initial pose deviation and enhances the registration robustness. In the fine registration stage, curvature and normal consistency weights are introduced to enhance the constraints of high curvature regions and geometric boundaries, improve the matching accuracy of high curvature regions and geometric boundary regions, and thus improve the overall accuracy and stability of stitching. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a method for stitching three-dimensional point clouds that integrates two-dimensional contours and three-dimensional features according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the three-dimensional point cloud of two parts of a workpiece to be spliced ​​according to an embodiment of the present invention. Figure 3 This is a two-dimensional contour diagram of the three-dimensional point cloud of two parts of the workpiece to be spliced ​​according to an embodiment of the present invention. Figure 4 This is a schematic diagram of the coarse registration effect of the three-dimensional point clouds of two parts of the workpiece to be spliced ​​in an embodiment of the present invention. Figure 5 This is a schematic diagram of the fine registration effect of the three-dimensional point clouds of two parts of the workpiece to be spliced ​​in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments provided by this invention without inventive effort are within the scope of protection of this invention.

[0019] Obviously, the accompanying drawings described below are merely some examples or embodiments of the present invention. Those skilled in the art can apply the present invention to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this invention, modifications to design, manufacturing, or production based on the technical content disclosed in this invention are merely conventional technical means and should not be construed as insufficient disclosure of the present invention.

[0020] In this invention, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this invention may be combined with other embodiments without conflict.

[0021] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "a," "an," "an," "the," and similar words used in this invention do not indicate quantity limitation and may indicate singular or plural. The terms "comprising," "including," "having," and any variations thereof used in this invention are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms "connected," "linked," "coupled," and similar words used in this invention are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "A plurality" used in this invention refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships may exist; for example, "A and / or B" can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects have an "or" relationship. The terms "first," "second," and "third" used in this invention are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0022] This invention provides a method for stitching 3D point clouds by fusing 2D contours and 3D features, comprising: acquiring point cloud data from different viewpoints and preprocessing it; extracting 3D feature points and calculating FPFH descriptors based on the ISS algorithm, and generating a set of candidate 3D rigid body transformation matrices using RANSAC; for each candidate transformation matrix, transforming one point cloud, projecting the transformed point cloud and the other point cloud onto the same 2D plane, and extracting the contour using the convex hull algorithm; calculating matching scores in the 2D contour space and 3D feature space respectively, and constructing a joint scoring function to filter the optimal coarse registration result, wherein the 2D score is based on an exponential function of contour distance, and the 3D score is based on feature cosine similarity; finally, curvature and normal consistency weights are introduced into the ICP algorithm to achieve fine registration, thereby improving the accuracy and robustness of point cloud stitching. This invention effectively improves the robustness of coarse registration by fusing 2D contour information and 3D feature information; at the same time, the introduction of a curvature and normal consistency weighted fine registration mechanism improves the matching accuracy in high curvature regions and geometric boundary regions.

[0023] like Figure 1 As shown, the 3D point cloud stitching method integrating two-dimensional contours and three-dimensional features of the present invention includes the following steps: S1. Obtain the first and second 3D point clouds to be stitched together; S2. The ISS algorithm is used to extract the three-dimensional feature points in the first and second three-dimensional point clouds, calculate the FPFH descriptor of each three-dimensional feature point, match the three-dimensional feature points using the RANSAC algorithm, and generate multiple three-dimensional rigid body transformation candidate matrices based on the matched three-dimensional feature point pairs. S3. Based on each 3D rigid body transformation candidate matrix, transform the first 3D point cloud to obtain the corresponding third 3D point cloud. Project the third 3D point cloud and the second 3D point cloud onto the same 2D plane and extract their respective 2D contours. Calculate the matching score in the 2D contour space and the 3D feature space respectively. Obtain the joint score of the 3D rigid body transformation candidate matrix through a weighted fusion joint scoring function. S4. Select the candidate matrix of the three-dimensional rigid body transformation with the highest joint score as the coarse registration transformation matrix, and perform coarse registration of the first three-dimensional point cloud and the second three-dimensional point cloud based on the coarse registration transformation matrix. S5. Curvature weight and normal consistency weight are introduced into the ICP algorithm to perform fine registration on the point cloud after coarse registration.

[0024] This method integrates 2D contours and 3D features for joint scoring in the coarse registration stage, improving tolerance to noise and initial pose deviations and enhancing registration robustness. In the fine registration stage, it introduces curvature and normal consistency weights, strengthening constraints on high-curvature regions and geometric boundaries, thus improving the matching accuracy between these regions and enhancing the overall accuracy and stability of the stitching. This invention effectively solves the problems of existing point cloud stitching methods, such as strong dependence on initial pose and insufficient robustness in the coarse registration stage, and the tendency to generate errors in high-curvature and geometric boundary regions in the fine registration stage, meeting the high-precision industrial inspection requirements.

[0025] Specifically, the three-dimensional point cloud stitching method that integrates two-dimensional contours and three-dimensional features according to embodiments of the present invention includes the following steps: S1. Acquire two three-dimensional point cloud data of the workpiece under test in different directions through a dual-line structured light device: the first three-dimensional point cloud P and the second three-dimensional point cloud Q, and perform filtering and downsampling processing on the three-dimensional point clouds P and Q respectively. S2. For the preprocessed 3D point cloud, the ISS algorithm is used to extract stable 3D feature points, calculate the FPFH descriptor and generate the matching relationship of 3D feature points through the RANSAC algorithm, thereby calculating the set of candidate matrices for 3D rigid body transformation. S3. Based on the 3D transformation candidate matrix, one of the point clouds is transformed in one go. The transformed point cloud and the other point cloud are projected onto the XOY plane. The convex hull algorithm is used to extract the 2D contour. The matching score is calculated in the 2D contour space and the 3D feature space respectively. The candidate transformation matrix is ​​scored by a weighted fusion joint scoring function. S4. Select the transformation with the highest score as the coarse registration transformation matrix, and perform coarse registration on the preprocessed 3D point cloud based on the coarse registration transformation matrix. S5. Introduce curvature and normal consistency weights into the ICP algorithm to perform fine registration of point clouds.

[0026] In step S1 above, the process of acquiring the point cloud is as follows: two three-dimensional point clouds of the workpiece under test in different directions are acquired through dual-line structured light, namely the first three-dimensional point cloud P and the second three-dimensional point cloud Q, as shown below. Figure 2 As shown. Specifically: the left laser of the dual-line structured light measurement device is turned on, and the workpiece is scanned along the positive direction parallel to the X-axis to obtain the workpiece point cloud P; keeping the position of the laser measurement device unchanged, the right laser is turned on, and the workpiece is scanned along the negative direction parallel to the X-axis to obtain the workpiece point cloud Q. The dual-line structured light device includes an X, Y, and Z-axis three-axis moving platform, a high-precision dual telecentric camera perpendicular to the coordinate axis moving platform, and left and right laser emitters arranged at the same angle to the camera optical axis and parallel to the X-axis. The dual telecentric camera ensures minimal distortion during the imaging process, and the symmetrical arrangement of the dual lasers improves the integrity and symmetry of the scan data.

[0027] Then, point clouds P and Q are filtered and downsampled respectively to obtain point clouds P′ and Q′. The filtering method is statistical filtering, which is used to remove isolated points and noise points; the downsampling method uses voxel grid filtering to reduce the amount of point cloud data, improve the efficiency of subsequent calculations, and at the same time preserve the overall geometric features of the workpiece.

[0028] In step S2 above, the ISS algorithm is used to extract stable 3D feature points, the FPFH descriptor is calculated, and the RANSAC algorithm is used to generate the matching relationship of the 3D feature points, thereby calculating the set of candidate matrices for 3D rigid body transformation. Specifically: Use ISS to extract stable keypoint sets from point clouds P′ and Q′. The ISS algorithm selects feature points with high geometric saliency by analyzing the eigenvalues ​​of the covariance matrix of the local neighborhood of the point cloud.

[0029] Calculate the FPFH descriptor for each feature point, as follows: and Calculate the FPFH descriptor for each key feature point to characterize the local geometry of the point, in order to form a stable three-dimensional feature table.

[0030] Then match using the RANSAC matching algorithm Generate a one-to-one matching relationship. , ), and based on the matched 3D feature point pairs ( , ), calculate the rigid body transformation matrix : ; in, and These are the rotation matrix and the translation vector, respectively, solved using SVD.

[0031] Finally, based on all 3D matching point pairs, the set of candidate 3D transformation matrices is obtained using SVD: ; Where N is the number of candidate matrices for the three-dimensional transformation.

[0032] In steps S3 and S4 above, for each candidate matrix of three-dimensional rigid body transformation A transformation is performed on the preprocessed first 3D point cloud P′ to obtain the corresponding third 3D point cloud P′′. The third 3D point cloud P′′ and the preprocessed second 3D point cloud Q′ are then projected onto the XOY plane, and the convex hull algorithm is used to extract the 2D contour to obtain the 2D point set. and ,as follows: First, define the projection operator: ; A two-dimensional point set is obtained using the projection operator:

[0033] Then, the convex hull algorithm is used to extract the two-dimensional contour point set. and : ; in, For the three-dimensional point cloud, it is the set of two-dimensional contour lines. It is the set of two-dimensional contour points of the second three-dimensional point cloud.

[0034] like Figure 3 As shown, the convex hull profile can effectively describe the overall boundary shape of an object, providing global geometric constraints for subsequent matching.

[0035] Then, matching scores are calculated in both two-dimensional and three-dimensional spaces, and each candidate transformation matrix is ​​scored using a weighted fusion joint scoring function. The transformation with the highest score is selected as the coarse registration transformation matrix. Specifically: To measure the degree of matching between the transformed contour and the target contour, a point-by-point distance evaluation is performed on the source contour points, and a two-dimensional contour matching score is defined: ; in: This represents the minimum distance from a point to the contour. This is a distance scale parameter used to adjust the degree of influence of contour matching error on the scoring function. Its value is related to the spatial resolution of the point cloud, and is preferably 2-5 times the average point spacing of the point cloud or a certain proportion of the bounding box size of the point cloud.

[0036] To measure the consistency of local geometric structure, feature cosine similarity is used, and a three-dimensional feature matching score is defined: ; in, and This represents the i-th pair of matched 3D feature points in the third and second 3D point clouds. and These are three-dimensional feature point pairs. and The FPFH descriptor vector, where N is the number of matched 3D feature point pairs.

[0037] The joint scoring function is calculated based on a weighted sum of the 2D contour matching score and the 3D feature matching score: ; in, , These are the weighting coefficients. This represents the score for two-dimensional contour matching. This represents the three-dimensional feature matching score. and , The value is adaptively adjusted based on the characteristics of the point cloud data: it increases when the noise is high. Increase when there are more feature points The initial values ​​of the weighting coefficients are set to... .

[0038] By calculating a joint scoring function, all candidate transformation matrices are scored, and the transformation with the highest score is selected as the optimal transformation. ; Finally, based on the optimal transformation, the first 3D point cloud P is transformed to complete the coarse registration of the first and second 3D point clouds, as follows: Figure 4 As shown.

[0039] In step S5 above, curvature and normal consistency weights are introduced into the ICP algorithm for accurate registration. Specifically, curvature and normal consistency weights are introduced during the ICP iteration process to improve the matching accuracy in high curvature regions and geometric boundary regions.

[0040] First, for each point in the point cloud P Find the point in the point cloud Q that is closest to it. : ; This results in a set of matching point pairs: .

[0041] Then, the curvature weights are calculated. For each point in the coarsely registered point cloud, the eigenvalues ​​of its neighborhood covariance matrix are calculated. , , Curvature is defined as: ; For each point in the first and second 3D point clouds after coarse registration, calculate the curvature and the corresponding point curvature weight. ; For each pair of matching points, the average curvature weights of the two matching points are taken as the curvature weight of that pair. ; Calculate the normal consistency weight. For matched point pairs... Their normal directions are respectively , Define the normal consistency weight as: .

[0042] The ICP algorithm is based on curvature and normal consistency weighting. The curvature weight and normal consistency weight are combined to define a comprehensive weight: ; Based on matching point pairs and their corresponding comprehensive weights Construct a weighted objective function: ; By iteratively optimizing the objective function, more accurate registration results can be obtained, such as... Figure 5 As shown, the matching accuracy can be significantly improved, especially in high curvature regions and geometric boundary regions.

[0043] Therefore, this invention provides a three-dimensional point cloud stitching method that integrates two-dimensional contours and three-dimensional features. It combines two-dimensional projection contours and three-dimensional geometric features to improve the stability of coarse registration of point clouds. At the same time, it uses curvature and normal consistency weighting to improve the overall accuracy of fine registration during fine registration, thereby meeting the needs of three-dimensional modeling and high-precision detection of complex workpieces.

[0044] This invention also provides a 3D point cloud stitching system that integrates two-dimensional contours and three-dimensional features, used to implement the 3D point cloud stitching method that integrates two-dimensional contours and three-dimensional features described in the above method embodiments. The system includes: The 3D point cloud acquisition module acquires the first and second 3D point clouds to be stitched together. The candidate matrix generation module is used to extract three-dimensional feature points from the first and second three-dimensional point clouds using the ISS algorithm, calculate the FPFH descriptor of each three-dimensional feature point, match the three-dimensional feature points using the RANSAC algorithm, and generate multiple three-dimensional rigid body transformation candidate matrices based on the matched three-dimensional feature point pairs. The transformation projection and scoring module is used to transform the first three-dimensional point cloud based on each three-dimensional rigid body transformation candidate matrix to obtain the corresponding third three-dimensional point cloud. The third three-dimensional point cloud and the second three-dimensional point cloud are projected onto the same two-dimensional plane and their respective two-dimensional contours are extracted. The matching scores are calculated in the two-dimensional contour space and the three-dimensional feature space respectively. The joint score of the three-dimensional rigid body transformation candidate matrix is ​​obtained through a weighted fusion joint scoring function. The coarse registration module is used to select the candidate 3D rigid body transformation matrix with the highest joint score as the coarse registration transformation matrix, and to perform coarse registration on the first 3D point cloud and the second 3D point cloud based on the coarse registration transformation matrix. The fine registration module is used to introduce curvature weights and normal consistency weights in the ICP algorithm to perform fine registration on the point cloud after coarse registration.

[0045] Furthermore, the present invention also provides a computer device / apparatus / system, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above-described method.

[0046] The present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0047] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, systems, and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or block diagrams.

[0048] These computer program instructions may also be stored in a computer-readable storage medium capable of directing a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more blocks in a block diagram.

[0049] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable data processing apparatus to produce a computer-implemented process, thereby providing steps for implementing the functions specified in one or more flowcharts and / or one or more blocks in a block diagram.

[0050] It should be noted that, depending on the implementation needs, the various steps / components described in this invention can be broken down into more steps / components, or two or more steps / components or parts of the operation of steps / components can be combined into new steps / components to achieve the purpose of this invention.

[0051] Those skilled in the art will readily understand that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for fusing two-dimensional contours and three-dimensional features of a three-dimensional point cloud, the method comprising: The method includes: Obtain the first and second 3D point clouds to be stitched together; The ISS algorithm is used to extract three-dimensional feature points from the first and second three-dimensional point clouds. The FPFH descriptor of each three-dimensional feature point is calculated. The RANSAC algorithm is used to match the three-dimensional feature points, and multiple three-dimensional rigid body transformation candidate matrices are generated based on the matched three-dimensional feature point pairs. Based on each 3D rigid body transformation candidate matrix, the first 3D point cloud is transformed to obtain the corresponding third 3D point cloud. The third 3D point cloud and the second 3D point cloud are projected onto the same 2D plane and their respective 2D contours are extracted. The matching scores are calculated in the 2D contour space and the 3D feature space respectively. The joint score of the 3D rigid body transformation candidate matrix is ​​obtained through a weighted fusion joint scoring function. The candidate matrix of the three-dimensional rigid body transformation with the highest joint score is selected as the coarse registration transformation matrix, and the first three-dimensional point cloud and the second three-dimensional point cloud are coarsely registered based on the coarse registration transformation matrix. Curvature weight and normal consistency weight are introduced into the ICP algorithm to perform fine registration on the point cloud after coarse registration. The joint scoring function is defined as follows: ； wherein is a three-dimensional rigid body transformation candidate matrix, is a two-dimensional contour matching score, is a three-dimensional feature matching score, is a joint score, and is a weight coefficient; The two-dimensional contour matching score is defined as: ； In the formula, For the three-dimensional point cloud, it is the set of two-dimensional contour lines. The set of two-dimensional contour points for the second three-dimensional point cloud; This represents the minimum distance from a point on the 2D contour of the third 3D point cloud to the 2D contour of the second 3D point cloud. For the two-dimensional contour points of the third three-dimensional point cloud, These are the two-dimensional contour points of the second three-dimensional point cloud; The distance decay exponent; The three-dimensional feature matching score is defined as: ； In the formula, and This represents the i-th pair of matched 3D feature points in the third and second 3D point clouds. and These are three-dimensional feature point pairs. and The FPFH descriptor vector, where N is the number of 3D feature point pairs.

2. The three-dimensional point cloud stitching method integrating two-dimensional contours and three-dimensional features according to claim 1, characterized in that, Multiple candidate matrices for 3D rigid body transformation are generated based on matched 3D feature point pairs, including: Each time, multiple pairs of matching 3D feature points are randomly selected, and a rigid body transformation candidate matrix is ​​obtained by solving the SVD decomposition method. Repeating the above steps multiple times yields several candidate matrices for three-dimensional rigid body transformation.

3. The method for stitching three-dimensional point clouds by integrating two-dimensional contours and three-dimensional features according to claim 1, characterized in that, Extracting the 2D contour includes: Projecting the 3D point cloud onto the target 2D plane yields a 2D point set; The convex hull algorithm is used to extract the two-dimensional contour line point set of the two-dimensional point set.

4. The three-dimensional point cloud stitching method integrating two-dimensional contours and three-dimensional features according to claim 1, characterized in that, Curvature weight and normal consistency weight are introduced into the ICP algorithm to perform fine registration on the point cloud after coarse registration, including: For each point in any 3D point cloud after coarse registration Find the nearest point from another 3D point cloud. , forming matching point pairs ; For each pair of matching points Determine the curvature of the two matching points respectively. and the corresponding curvature weights The average curvature weights of the two matching points are taken as the curvature weight of the matching point pair. ; and determine the normals of the two matching points respectively. and Calculate the normal consistency weight of the matching point pair. ; The curvature weight and normal consistency weight of each pair of matched points are combined to define a comprehensive weight. ; Based on each pair of matching points Based on the corresponding comprehensive weights, a weighted objective function is constructed. ;in, Let be a rotation matrix. It is a translation matrix. To match the number of point pairs; The optimal rotation matrix is ​​obtained by iteratively optimizing the objective function. Translation matrix This is used to perform fine registration on the point cloud after coarse registration.

5. A three-dimensional point cloud stitching system integrating two-dimensional contours and three-dimensional features, characterized in that, The system is used to implement the three-dimensional point cloud stitching method for fusing two-dimensional contours and three-dimensional features as described in any one of claims 1 to 4, the system comprising: The 3D point cloud acquisition module acquires the first and second 3D point clouds to be stitched together. The candidate matrix generation module is used to extract three-dimensional feature points from the first and second three-dimensional point clouds using the ISS algorithm, calculate the FPFH descriptor of each three-dimensional feature point, match the three-dimensional feature points using the RANSAC algorithm, and generate multiple three-dimensional rigid body transformation candidate matrices based on the matched three-dimensional feature point pairs. The transformation projection and scoring module is used to transform the first three-dimensional point cloud based on each three-dimensional rigid body transformation candidate matrix to obtain the corresponding third three-dimensional point cloud. The third three-dimensional point cloud and the second three-dimensional point cloud are projected onto the same two-dimensional plane and their respective two-dimensional contours are extracted. The matching scores are calculated in the two-dimensional contour space and the three-dimensional feature space respectively. The joint score of the three-dimensional rigid body transformation candidate matrix is ​​obtained through a weighted fusion joint scoring function. The coarse registration module is used to select the candidate 3D rigid body transformation matrix with the highest joint score as the coarse registration transformation matrix, and to perform coarse registration on the first 3D point cloud and the second 3D point cloud based on the coarse registration transformation matrix. The fine registration module is used to introduce curvature weights and normal consistency weights in the ICP algorithm to perform fine registration on the point cloud after coarse registration.

6. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1 to 4.

Images