Progressive point cloud fine matching method based on mixed features
By adopting a mixed feature progressive method in point cloud precision registration, using mixed features of depth and color features, the problem of poor performance of the prior art in challenging scenarios is solved, and higher precision registration accuracy and less drift problems are achieved.
Patent Information
- Application Number
- CN202411681608.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-22
- Publication Date
- 2025-03-11
AI Technical Summary
The existing point cloud precision registration method is not effective when dealing with challenging scenes, especially in environments with repeated textures, weak textures, poor light and weak three-dimensional structural characteristics, which are prone to interlacing and stratification, and there are drift problems.
The point cloud fine matching method based on the gradual mixed feature is adopted. By extracting depth and color features, mixing features are constructed, and geometric consistency checks and sparse feature matching are used to gradually narrow the search range to establish a matching relationship, tolerate greater rough matching errors, and improve the accuracy of fine matching results.
This method is suitable for more challenging scenarios, improves the accuracy of point cloud registration, reduces degradation problems such as drift, and can tolerate large errors of rough matching results, and achieves rapid convergence.
Smart Images
Figure CN119600075A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of point cloud precise registration, and in particular to a point cloud precise registration method based on hybrid feature progression. Background Art
[0002] Point cloud registration is generally divided into two processes: coarse registration and fine registration. Coarse registration refers to finding a set of approximate coordinate transformations to unify the point clouds from two viewpoints into the same coordinate system without any prior information, that is, roughly aligning the two point clouds with visual overlap, including position, rotation angle, etc. The result of coarse registration is usually an approximation of the precise coordinate transformation, so that the two point clouds have a certain degree of fit in the overlapping area. However, this approximate position transformation makes it difficult to accurately fit the overlapping areas of the point clouds, and there are often interlacing and stratification phenomena, which is not conducive to subsequent point cloud fusion operations. Therefore, it is necessary to further adjust the position of the point cloud to improve the registration accuracy. This process is called fine registration.
[0003] The point cloud rough registration methods include: RANSAC solution based on feature matching, geometric consistency verification solution based on vision, four-point method and deep learning-based method.
[0004] Point cloud precision registration methods include: ICP and its variants NICP, GICP; NDT method, etc. These methods are based on dense point clouds for precision registration processing.
[0005] However, there are the following defects in the prior art:
[0006] 1. The effectiveness of methods represented by ICP is limited by the results of coarse registration. Using coarse registration results with slightly larger errors as input may cause ICP to iteratively optimize to a local optimal solution; secondly, ICP has a drift problem. When the point cloud only has a single wall or two non-parallel walls, the convergence speed of ICP will be slow and the results are prone to drift, deviating from the true value. Typical scenes include long corridors, relatively empty museums or outdoors.
[0007] 2. Environmental Challenges
[0008] Rough registration is generally based on sparse feature matching and RANSAC consistency check to calculate the results. The use of a single feature is limited by the scene. For example, a solution based solely on visual features is not suitable for repeated textures (wallpaper with repeated patterns or brick walls, brick floors, etc.), weak textures (white walls), poor lighting environments, etc.; a solution based solely on three-dimensional features is not suitable for environments with few point cloud structural features.
[0009] To this end, we provide a point cloud matching method based on hybrid features and progressiveness to solve the above problems. Summary of the invention
[0010] In response to the problems existing in the above-mentioned prior art, the present invention provides a point cloud fine matching method based on hybrid features in a progressive manner, which is suitable for more challenging scenarios. It can tolerate larger rough matching errors by gradually narrowing the search range to establish a matching relationship. The fine matching results are highly accurate and can suppress degradation problems such as drift.
[0011] In order to achieve the above object, the present invention adopts a point cloud precision matching method based on hybrid feature progressive, including naming the two data to be precisely matched as source data and target data, the source data and the target data both contain depth and color data, and also includes the following steps:
[0012] S1: Extract feature data from source data and target data respectively to obtain point cloud and color features;
[0013] S2: The point cloud and color features obtained by constructing process S1 are mixed features in, is the 3D point coordinate corresponding to the feature, is the normal direction of the 3D point corresponding to the feature, M is the feature descriptor, and the construction method is as follows:
[0014] S201, project the color feature points from the two-dimensional plane to the corresponding three-dimensional point cloud, and mark the corresponding three-dimensional point coordinates as The normal corresponding to this point is recorded as The descriptor corresponding to the color feature point is denoted as M;
[0015] S202, the 3D coordinates and normals corresponding to the 3D feature points of the point cloud are also recorded as The descriptor is also denoted as M;
[0016] The n mixed feature data extracted from the source data are recorded as in, They refer to the mixed feature data from the 1st to the nth source data, and the m mixed feature data extracted from the target data are recorded as in, Refers to the mixed feature data from the 1st to the nth target data;
[0017] S3: Create feature matching pairs in, They refer to the mixed feature data of the i-th source data and the mixed feature data of the i-th target data respectively;
[0018] S301, the three-dimensional transformation result of the known rough registration is in, is a 3x3 rotation matrix, is a 3x1 translation matrix that transforms FList according to the current three-dimensional transformationsrc Each mixed feature in The 3D point coordinates are transformed into the target point cloud coordinate system, and then from FList tar Find the feature sequence whose three-dimensional Euclidean distance is less than the distance parameter param_dist_P in, They represent the feature sequences from the 1st to the hth that satisfy the three-dimensional Euclidean distance less than the distance parameter. and The descriptor of the same type, is the jth feature sequence that satisfies the three-dimensional Euclidean distance less than the distance parameter, and They are all converted from color features, and the same goes for point cloud features;
[0019] S302, calculation Each element of and The Euclidean distance of the descriptor is obtained, and the distance sequence dist_M1, dist_M2, ..., dist_M h}, where dist_M1, dist_M2, ..., dist_M h Refers to the corresponding elements from 1 to h respectively and The descriptor Euclidean distance is filtered according to the parameter param_dist_M. Elements with values exceeding this parameter are deleted. Finally, the removed distance sequence is sorted according to the value size to obtain the two objects with the smallest value: dist_M min1 and dist_M min2 ;
[0020] S303, judging dist_M according to the parameter param_ratio min1 and dist_M min2 If the difference is large, a matching pair is created. in dist_M min1 The corresponding target mixed features; if the difference is not big, no matching pair is established;
[0021] S304, normal check, for matching pairs check and Normal If the angle between them exceeds the parameter param_degree_normal, the matching pair will be deleted;
[0022] S4: Geometric consistency check for matching pair sequences Establish a 3D transformation from the source mixed feature 3D point set to the target mixed feature 3D point set satisfy: in is a 3x3 rotation matrix, is a 3x1 translation matrix, Target mixed features The three-dimensional point coordinates, Source Mix Features The three-dimensional point coordinates are estimated using the ransac method And increase the inner point distance, edge length consistency ratio and normal angle threshold to construct the inner point screening principle and filter the sequence The final mixed feature matching pair sequence is recorded as
[0023] S5: Mixed icp:
[0024] S501, assuming that the source data point cloud sequence is in, The point clouds of the original data correspond to the three-dimensional point coordinates corresponding to the point cloud features of the 1st to nth source data, and the point cloud of the target data is in, The three-dimensional point coordinates corresponding to the point cloud features of the 1st to mth target data are respectively converted according to the current three-dimensional transformation P src The point cloud sequence is transformed into the target point cloud coordinate system. Refers to the conversion to the three-dimensional coordinate point in the target point cloud coordinate system, and finding its tar The closest point in The point cloud matching pairs found by the nearest neighbor search are recorded as
[0025] S502, using the sparse mixed feature matching sequence MatchList obtained in S4 F MatchList with dense point cloud Dense , establish the following objective function:
[0026]
[0027] in is the parameter to be optimized, param_alpha is the weight parameter, Not limited to point-to-point models, iterative solution using Gauss-Newton method Set the iteration upper limit parameter param_iter_maxcount, and determine whether to end the iteration optimization early based on the residual change. is the optimized three-dimensional transformation;
[0028] S503, use renew And adjust the distance parameter param_dist_P and re-execute S3, adjust the value of param_dist_P to half of the previous value, and stop the precise matching when param_dist_P reaches a reasonable range.
[0029] As a further optimization of the above solution, the three-dimensional point features and descriptors are extracted from the point cloud data in S1, including but not limited to Sift3D, Harris3D, and FPFH feature extraction methods;
[0030] The S1 acquires point features and descriptors for color photos, including but not limited to sift, orb, surf feature extraction methods and SuperPoint deep learning-based methods.
[0031] As a further optimization of the above solution, if the distance sequence in S303 contains only one element, the target mixed feature corresponding to the element is for Feature matching object, establish matching relationship
[0032] As a further optimization of the above solution, in S502 Select the icp model: point to line, point to surface for replacement.
[0033] As a further optimization of the above solution, the iteration upper limit parameter in S502 does not exceed 20 times.
[0034] As a further optimization of the above solution, when param_dist_P in S503 reaches a reasonable range, it is related to the density of the point cloud.
[0035] The point cloud precision matching method based on hybrid feature progressive method of the present invention has the following beneficial effects:
[0036] The present invention is a point cloud fine matching method based on mixed feature progressive, which is suitable for more scenes (including repeated color texture, weak color texture, weak three-dimensional structure features, etc.), gradually narrows the search range to establish matching relationships and improves the tolerance for large errors in the rough registration results. Although the present invention does not belong to the rough registration technology, in order to correct the rough registration results, the present invention needs to adapt to more scenes.
[0037] The invention discloses a point cloud precise matching method based on hybrid feature progressive method, which improves the matching accuracy and effectively suppresses the degradation problems such as drift.
[0038] With reference to the following description and drawings, a specific embodiment of the present invention is disclosed in detail, indicating the manner in which the principles of the present invention can be adopted. It should be understood that the scope of the embodiments of the present invention is not limited thereby, and within the spirit and scope of the appended claims, the embodiments of the present invention include many changes, modifications and equivalents. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 It is a schematic diagram of the flow of the hybrid feature-based progressive point cloud precision matching method of the present invention. DETAILED DESCRIPTION
[0040] In order to make the purpose, technical solution and advantages of the present invention more clear, the present invention is further described in detail below through the accompanying drawings and embodiments. However, it should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the scope of the present invention.
[0041] Please refer to the instruction manual Figure 1 The present invention provides a technical solution: a point cloud precision matching method based on hybrid feature progressive method.
[0042] The present invention is applicable to devices that can simultaneously collect depth and color data. Depth data includes depth maps (data collected by structured light devices) or point cloud data (data collected by radar and other devices); color data mainly refers to photos collected by RGB color cameras, which can be single-frame, single-angle picture data or panoramic picture data after single-station multi-angle stitching.
[0043] name:
[0044] The two sets of data to be precisely aligned are named source data and target data. The present invention realizes precise alignment of the point cloud of the source data with the point cloud of the target data. Both the source data and the target data contain two types of data: depth and color. The depth data is uniformly converted into point cloud data; the color data can be a single-frame, single-angle picture captured by a camera, or a panoramic picture stitched from multi-angle photos of a single station.
[0045] process:
[0046] S1, extract feature data from source data and target data respectively to obtain point cloud and color features. Including:
[0047] S101, extracting 3D point features and descriptors from point cloud data, including but not limited to Sift3D, Harris3D, FPFH and other feature extraction methods.
[0048] S102, obtaining point features and descriptors for color photos, including but not limited to feature extraction methods such as sift, orb, surf, and deep learning-based methods such as SuperPoint.
[0049] S2, construct the point cloud and color features obtained in process S1 as mixed features in, is the three-dimensional point coordinate corresponding to the feature; is the normal direction of the 3D point corresponding to the feature; M is the feature descriptor. The construction method is as follows:
[0050] S201, project the color feature points from the two-dimensional plane to the corresponding three-dimensional point cloud, and mark the corresponding three-dimensional point coordinates as The normal corresponding to this point is recorded as The descriptor corresponding to the color feature point is denoted by M.
[0051] S202, the 3D coordinates and normals corresponding to the 3D feature points of the point cloud are also recorded as The descriptor is also denoted as M.
[0052] The n mixed feature data extracted from the source data are recorded as in, They refer to the mixed feature data from the 1st to the nth source data, and the m mixed feature data extracted from the target data are recorded as in, Refers to the mixed feature data from the 1st to the nth target data.
[0053] S3, establish feature matching pairs
[0054] S301, the three-dimensional transformation result of the known rough registration is in is a 3x3 rotation matrix, is a 3x1 translation matrix. src Each mixed feature in The 3D point coordinates are transformed into the target point cloud coordinate system, and then from FList tar Find the feature sequence whose three-dimensional Euclidean distance is less than the distance parameter param_dist_P in, They represent the feature sequences from the 1st to the hth that satisfy the three-dimensional Euclidean distance less than the distance parameter. Note: and The descriptor of the same type, is the jth feature sequence that satisfies the three-dimensional Euclidean distance less than the distance parameter, that is, if is converted from color features, then It is also converted from color features; the same goes for point cloud features.
[0055] S302, calculation Each element of and The Euclidean distance of the descriptor is obtained, and the distance sequence dist_M1, dist_M2, ..., dist_M h}, where dist_M1, dist_M2, ..., dist_M h Refers to the corresponding elements from 1 to h respectively and The distance sequence is filtered according to the parameter param_dist_M, and the elements with values exceeding this parameter are deleted. Finally, the removed distance sequence is sorted according to the value size, and the two objects with the smallest value are obtained: dist_M min1 and dist_M min2 .
[0056] S303, judging dist_M according to the parameter param_ratio min1 and dist_M min2 If the difference is large, a matching pair is created. in dist_M min1 The corresponding target mixed feature; if the difference is not big, no matching pair is established. Note: If the distance sequence contains only one element, the target mixed feature corresponding to the element for Feature matching object, establish matching relationship
[0057] S304, normal check, for matching pairs check and Normal If the angle between them exceeds the parameter param_degree_normal, the matching pair will be deleted.
[0058] S4, geometric consistency check, for matching pair sequences Establish a 3D transformation from the source mixed feature 3D point set to the target mixed feature 3D point set satisfy: in is a 3x3 rotation matrix, is a 3x1 translation matrix, Target mixed features The three-dimensional point coordinates, Source Mix Features The three-dimensional point coordinates of . Use the ransac method to estimate And increase the inner point distance, edge length consistency ratio and normal angle threshold to construct the inner point screening principle and filter the sequence The final mixed feature matching pair sequence is recorded as
[0059] S5, mixed icp:
[0060] S501, assuming that the source data point cloud sequence is The target data point cloud is in, The point clouds of the original data are the three-dimensional point coordinates corresponding to the point cloud features of the 1st to nth source data, The three-dimensional point coordinates corresponding to the point cloud features of the 1st to mth target data are respectively converted according to the current three-dimensional transformation P src The point cloud sequence is transformed into the target point cloud coordinate system. Refers to the conversion to the three-dimensional coordinate point in the target point cloud coordinate system, and finding its tar The closest point in The point cloud matching pairs found by the nearest neighbor search are recorded as
[0061] S502, using the sparse mixed feature matching sequence MatchList obtained in S4 F MatchList with dense point cloud Dense , establish the following objective function:
[0062]
[0063] in is the parameter to be optimized, and param_alpha is the weight parameter. It is not limited to the point-to-point model, and other ICP models can also be selected: point-to-line, point-to-surface, etc. The present invention uses the Gauss-Newton method to iteratively solve Set the iteration upper limit parameter param_iter_maxcount (usually no more than 20-30 times), and determine whether to end the iterative optimization early based on the residual change. is the optimized three-dimensional transformation.
[0064] S503, use renew Adjust the distance parameter param_dist_P and re-execute step 3. Generally, adjust the value of param_dist_P to half of the previous value. When param_dist_P reaches a reasonable range (related to the density of the point cloud), stop the fine matching.
[0065] The key points of the present invention are:
[0066] 1. Establish mixed features and use both color and point cloud features;
[0067] 2. Gradually narrow the search range to establish a matching relationship. For rough matching results with large errors, it can improve the error correction capability and achieve the purpose of rapid convergence;
[0068] 3. The establishment of matching relationships between features takes into account more solutions for removing false matches;
[0069] 4. Hybrid ICP considers the matching constraints between dense point clouds and sparse feature points at the same time, effectively suppressing degradation problems such as drift while ensuring accuracy.
[0070] It is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modification, equivalent substitution or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A point cloud precision matching method based on hybrid feature progressive method, characterized in that: The two sets of data to be precisely aligned are named source data and target data. Both the source data and the target data contain depth and color data. The following steps are involved: S1: Extract feature data from source data and target data respectively to obtain point cloud and color features; S2: The point cloud and color features obtained by constructing process S1 are mixed features in, is the 3D point coordinate corresponding to the feature, is the normal direction of the 3D point corresponding to the feature, M is the feature descriptor, and the construction method is as follows: S201, project the color feature points from the two-dimensional plane to the corresponding three-dimensional point cloud, and mark the corresponding three-dimensional point coordinates as The normal corresponding to this point is recorded as The descriptor corresponding to the color feature point is denoted as M; S202, the 3D coordinates and normals corresponding to the 3D feature points of the point cloud are also recorded as The descriptor is also denoted as M; The n mixed feature data extracted from the source data are recorded as in, They refer to the mixed feature data from the 1st to the nth source data, and the m mixed feature data extracted from the target data are recorded as in, Refers to the mixed feature data from the 1st to the nth target data; S3: Create feature matching pairs in, They refer to the mixed feature data of the i-th source data and the mixed feature data of the i-th target data respectively; S301, the three-dimensional transformation result of the known rough registration is in, is a 3x3 rotation matrix, is a 3x1 translation matrix that transforms FList according to the current three-dimensional transformation src Each mixed feature in The 3D point coordinates are transformed into the target point cloud coordinate system, and then from FList tar Find the feature sequence whose three-dimensional Euclidean distance is less than the distance parameter param_dist_P in, They represent the feature sequences from the 1st to the hth ones whose three-dimensional Euclidean distance is less than the distance parameter. and The descriptor of the same type, is the jth feature sequence that satisfies the three-dimensional Euclidean distance less than the distance parameter, and They are all converted from color features, and the same goes for point cloud features; S302, calculation Each element of and The Euclidean distance of the descriptor is obtained, and the distance sequence {dist_M1, dist_M2, …, dist_M h }, where dist_M1, dist_M2, ..., dist_M h Refers to the corresponding elements from 1 to h respectively and The descriptor Euclidean distance is filtered according to the parameter param_dist_M. Elements with values exceeding this parameter are deleted. Finally, the removed distance sequence is sorted according to the value size to obtain the two objects with the smallest value: dist_M min1 and dist_M min2 ; S303, judging dist_M according to the parameter param_ratio min1 and dist_M min2 If the difference is large, a matching pair is created. in dist_M min1 The corresponding target mixed features; if the difference is not big, no matching pair is established; S304, normal check, for matching pairs check and Normal If the angle between them exceeds the parameter param_degree_normal, the matching pair will be deleted; S4: Geometric consistency check for matching pair sequences Establish a 3D transformation from the source mixed feature 3D point set to the target mixed feature 3D point set satisfy: in is a 3x3 rotation matrix, is a 3x1 translation matrix, Target mixed features The three-dimensional point coordinates, Source Mix Features The three-dimensional point coordinates are estimated using the ransac method And increase the inner point distance, edge length consistency ratio and normal angle threshold to construct the inner point screening principle and filter the sequence The final mixed feature matching pair sequence is recorded as S5: Mixed icp: S501, assuming that the source data point cloud sequence is in, The point clouds of the original data correspond to the three-dimensional point coordinates corresponding to the point cloud features of the 1st to nth source data, and the point cloud of the target data is in, The three-dimensional point coordinates corresponding to the point cloud features of the 1st to mth target data are respectively converted according to the current three-dimensional transformation P src The point cloud sequence is transformed into the target point cloud coordinate system. Refers to the conversion to the three-dimensional coordinate point in the target point cloud coordinate system, and finding its tar The closest point in The point cloud matching pairs found by the nearest neighbor search are recorded as S502, using the sparse mixed feature matching sequence MatchList obtained in S4 F MatchList with dense point cloud Dense , establish the following objective function: in is the parameter to be optimized, param_alpha is the weight parameter, Not limited to point-to-point models, iterative solution using Gauss-Newton method Set the iteration upper limit parameter param_iter_maxcount, and determine whether to end the iteration optimization early based on the residual change. is the optimized three-dimensional transformation; S503, use renew And adjust the distance parameter param_dist_P and re-execute S3, adjust the value of param_dist_P to half of the previous value, and stop the precise matching when param_dist_P reaches a reasonable range.
2. The point cloud precision matching method based on hybrid feature progressive method according to claim 1, characterized in that: In S1, three-dimensional point features and descriptors are extracted from point cloud data, including but not limited to Sift3D, Harris3D, and FPFH feature extraction methods; In the S1, point features and descriptors are obtained for color photos, including but not limited to sift, orb, surf feature extraction methods and SuperPoint deep learning-based methods.
3. The point cloud precision matching method based on hybrid feature progressive method according to claim 1, characterized in that: If the distance sequence in S303 contains only one element, the target mixed feature corresponding to the element is for Feature matching object, establish matching relationship 4. The point cloud precision matching method based on hybrid feature progressive method according to claim 1, characterized in that: In S502 Select the icp model: point to line, point to surface for replacement.
5. The point cloud precision matching method based on hybrid feature progressive method according to claim 1, characterized in that: The iteration upper limit parameter in S502 does not exceed 20 times.
6. The point cloud precision matching method based on hybrid feature progressive method according to claim 1, characterized in that: When param_dist_P in S503 reaches a reasonable range, it is related to the density of the point cloud.