Point cloud matching method, device and equipment combining feature entropy screening with geometric constraint

Abstract

The application relates to a point cloud matching method, device and equipment combining feature entropy screening and geometric constraints. The method comprises the following steps: voxel sampling is performed on a source point cloud and a target point cloud to obtain key points corresponding to the source point cloud and the target point cloud; adaptive radius searching is performed based on the local average neighbor distance of each key point and the global average neighbor distance of the corresponding point cloud to determine the target radius corresponding to each key point; in the case that the local average neighbor distance is smaller than the global average neighbor distance of the corresponding point cloud, the target radius is reduced; in the case that the local average neighbor distance is greater than the global average neighbor distance of the corresponding point cloud, the target radius is increased; the neighbor point set corresponding to each key point is determined based on the target radius corresponding to each key point; the feature descriptor corresponding to each key point is generated based on the neighbor point set of each key point; and the source point cloud and the target point cloud are matched based on the feature descriptor corresponding to each key point. The method can improve the matching accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional point cloud processing, in particular to a point cloud matching method, device and equipment combining feature entropy screening and geometric constraint. BACKGROUND

[0002] With the rapid development of three-dimensional sensing technology (such as structured light, laser scanning, binocular stereo vision, etc.), point cloud data has become an important form to describe the spatial geometric information of an object. In order to unify point cloud data collected from different angles, different devices or different times into the same coordinate system, point cloud registration becomes a key step in three-dimensional reconstruction and spatial alignment. The core goal of point cloud registration is to solve an optimal rigid body transformation in three-dimensional space, so that the source point cloud and the target point cloud are best aligned in space. According to whether it depends on the initial pose and the optimization method.

[0003] In the traditional technology, point cloud registration methods are mainly divided into two categories: local registration and global registration.

[0004] (1) Local registration: this kind of algorithm assumes that the initial position is relatively close, and gradually optimizes the rigid body transformation matrix by iteratively minimizing the distance error between point pairs. Although it has high computational efficiency, it strongly depends on the initial pose and is easy to fall into a local optimal solution, and it is not stable in the case of noise, occlusion or low overlap rate.

[0005] (2) Global registration: global method does not depend on the initial alignment state, and usually obtains the global optimal pose estimation through feature matching, geometric constraint or consistency optimization.

[0006] However, the neighborhood of the traditional geometric feature is fixed, and it is difficult to adapt to the local density of the point cloud. In the dense area, it is redundant, and in the sparse area, the information is insufficient. In the repeated structure area, it is easy to confuse, resulting in a high error rate of initial matching. SUMMARY

[0007] Therefore, it is necessary to provide a point cloud matching method, device and equipment combining feature entropy screening and geometric constraint, which can improve the matching accuracy.

[0008] In a first aspect, the present application provides a point cloud matching method combining feature entropy screening and geometric constraint, which comprises:

[0009] performing voxel sampling on the source point cloud and the target point cloud to obtain key points corresponding to the source point cloud and the target point cloud;

[0010] perform adaptive radius search based on a local average neighbor distance of each of the key points and a global average neighbor distance of a corresponding point cloud to determine a target radius corresponding to each of the key points, wherein the target radius is reduced when the local average neighbor distance is less than the global average neighbor distance of the corresponding point cloud, and the target radius is increased when the local average neighbor distance is greater than the global average neighbor distance of the corresponding point cloud;

[0011] determine a neighbor point set corresponding to each of the key points based on the target radius corresponding to each of the key points;

[0012] generate a feature descriptor corresponding to each of the key points based on the neighbor point set corresponding to each of the key points through feature entropy screening and geometric constraint;

[0013] match the source point cloud and the target point cloud based on the feature descriptor corresponding to each of the key points.

[0014] In one embodiment, the generating a feature descriptor corresponding to each of the key points based on the neighbor point set corresponding to each of the key points comprises:

[0015] generating a surface normal vector corresponding to each of the key points based on the neighbor point set corresponding to each of the key points;

[0016] dividing neighbor points in the neighbor point set into at least one subset according to a distance between each of the key points and the neighbor points in the neighbor point set, each of the subsets comprising a plurality of neighbor point pairs, each of the neighbor point pairs comprising the key point and one of the neighbor points;

[0017] constructing a Dab coordinate system based on the surface normal vector corresponding to each of the key points for each of the neighbor points in the neighbor point set corresponding to each of the key points and the key point, and calculating a rotation matrix corresponding to each of the neighbor point pairs based on each of the Dab coordinate systems;

[0018] screening each of the subsets based on an entropy of each of the neighbor point pairs in the subset to obtain a target subset;

[0019] converting the rotation matrix into Euler angles, calculating a geometric property of each of the target subsets based on the Euler angles and the distance between each of the key points and the neighbor points in the neighbor point set corresponding to each of the key points, and obtaining the feature descriptor corresponding to each of the key points based on the geometric property.

[0020] In one embodiment, the converting the rotation matrix into Euler angles, calculating a geometric property of each of the target subsets based on the Euler angles and the distance between each of the key points and the neighbor points in the neighbor point set corresponding to each of the key points, and obtaining the feature descriptor corresponding to each of the key points based on the geometric property comprises:

[0021] For each of the key points, a geometric property of each of the target subsets is calculated based on a rotation matrix of each of the point pairs in the target subset and a surface normal vector corresponding to the key point;

[0022] The geometric properties are discretely divided into corresponding intervals, and a frequency of occurrence of the geometric properties in each of the intervals is counted to obtain a multi-dimensional joint histogram;

[0023] A feature descriptor corresponding to each of the key points is constructed based on the multi-dimensional joint histogram.

[0024] In one of the embodiments, the matching of the source point cloud and the target point cloud based on the feature descriptors corresponding to the key points comprises:

[0025] The feature descriptor of each key point in the source point cloud and the feature descriptor of each key point in the target point cloud are matched to obtain an initial matching point pair;

[0026] The initial matching point pair is screened to obtain a target matching point pair;

[0027] The source point cloud and the target point cloud are matched based on the target matching point pair.

[0028] In one of the embodiments, the screening of the initial matching point pair to obtain a target matching point pair comprises:

[0029] A second-order consistency matrix of each of the initial matching point pairs is calculated;

[0030] An adaptive threshold value is obtained based on the second-order consistency matrix of each of the initial matching point pairs, and the initial matching point pair is screened based on the adaptive threshold value and the second-order consistency matrix.

[0031] In one of the embodiments, the screening of the initial matching point pair to obtain a target matching point pair further comprises:

[0032] Based on the second-order consistency matrix of each of the initial matching point pairs, a second point in the target point cloud is found to match a first point in the source point cloud in the screened initial matching point pair;

[0033] Based on the second-order consistency matrix of each of the initial matching point pairs, a fourth point in the source point cloud is found to match a third point in the target point cloud in the screened initial matching point pair;

[0034] In a case where the first point and the fourth point are the same, and the second point and the third point are the same, the initial matching point pair is retained;

[0035] In a case that the first point is the same as the fourth point and the second point is different from the third point, deleting the initial matching point pair.

[0036] In one of the embodiments, the screening the initial matching point pair to obtain a target matching point pair further comprises:

[0037] Selecting point pair triplets from the retained initial matching point pairs;

[0038] Calculating geometric features of each of the point pair triplets;

[0039] Calculating similarity scores of each type of geometric feature, and obtaining a comprehensive geometric consistency score based on the similarity scores of each type of geometric feature;

[0040] Screening the triplets based on the comprehensive geometric consistency score, and the number of the triplets is greater than or equal to a number threshold.

[0041] In one of the embodiments, the matching the source point cloud and the target point cloud based on the target matching point pair comprises:

[0042] Generating candidate rigid body transformation matrices based on the screened triplets;

[0043] Applying each of the candidate rigid body transformation matrices to the source point cloud, and counting the number of inliers in the target point cloud that satisfy a distance threshold;

[0044] Determining a target rigid body transformation matrix based on the number of inliers corresponding to each of the candidate rigid body transformation matrices;

[0045] Matching the source point cloud and the target point cloud based on the target rigid body transformation matrix.

[0046] In a second aspect, the present application further provides a point cloud matching device combining feature entropy screening and geometric constraints, the device comprising:

[0047] A key point determination module for voxel sampling of a source point cloud and a target point cloud to obtain key points corresponding to the source point cloud and the target point cloud;

[0048] An adaptive radius determination module for adaptive radius search to determine target radii corresponding to the key points based on local average neighbor distances of the key points and global average neighbor distances of the corresponding point clouds, wherein the target radius is reduced in a case that the local average neighbor distance is less than the global average neighbor distance of the corresponding point cloud, and the target radius is increased in a case that the local average neighbor distance is greater than the global average neighbor distance of the corresponding point cloud;

[0049] The neighborhood point set determination module is configured to determine, based on the target radius corresponding to each key point, a neighborhood point set corresponding to each key point;

[0050] The feature descriptor generation module is configured to generate, based on the neighborhood point set of each key point, a feature descriptor corresponding to each key point through feature entropy screening and geometric constraint.

[0051] The matching module is configured to match the source point cloud and the target point cloud based on the feature descriptor corresponding to each key point.

[0052] In a third aspect, the present application also provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method in any one of the above embodiments when executing the computer program.

[0053] The point cloud matching method, device and computer device described above combine feature entropy screening and geometric constraint, perform voxel sampling on a source point cloud and a target point cloud to obtain key points corresponding to the source point cloud and the target point cloud, perform adaptive radius search based on a local average neighbor distance of each key point and a global average neighbor distance of the point cloud corresponding to each key point to determine a target radius corresponding to each key point, wherein the target radius is reduced when the local average neighbor distance is less than the global average neighbor distance of the point cloud corresponding to each key point, and the target radius is increased when the local average neighbor distance is greater than the global average neighbor distance of the point cloud corresponding to each key point, determine a neighborhood point set corresponding to each key point based on the target radius corresponding to each key point, generate a feature descriptor corresponding to each key point based on the neighborhood point set of each key point through feature entropy screening and geometric constraint, and match the source point cloud and the target point cloud based on the feature descriptor corresponding to each key point. In this way, the local average neighbor distance of each key point and the global average neighbor distance of the point cloud corresponding to each key point are used to perform adaptive radius search to determine the target radius corresponding to each key point, thereby ensuring the accuracy of the neighborhood point set and the accuracy of the feature descriptor, and improving the accuracy of subsequent matching. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the description of the embodiments of the present application or the related art will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other related drawings can also be obtained without creative labor.

[0055] Figure 1 It is a flowchart of the point cloud matching method combining feature entropy screening and geometric constraint in one embodiment.

[0056] Figure 2 This is a flowchart illustrating a point cloud matching method that combines feature entropy filtering with geometric constraints in another embodiment.

[0057] Figure 3 This is a flowchart of the feature descriptor generation steps in one embodiment;

[0058] Figure 4 This is a structural block diagram of a point cloud matching device that combines feature entropy filtering with geometric constraints in one embodiment;

[0059] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0061] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0062] In one embodiment, such as Figure 1 As shown, a point cloud matching method combining feature entropy filtering and geometric constraints is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, and to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0063] S102: Perform voxel sampling on the source point cloud and the target point cloud to obtain the key points corresponding to the source point cloud and the target point cloud.

[0064] This involves voxel sampling of the source point cloud and the target point cloud, with the sampled point cloud serving as key points. For example, voxel sampling of the source point cloud yields the corresponding key points, and voxel sampling of the target point cloud yields the corresponding key points.

[0065] S104: Based on the local average neighbor distance of each key point and the global average neighbor distance of the corresponding point cloud, perform adaptive radius search to determine the target radius corresponding to each key point. If the local average neighbor distance is less than the global average neighbor distance of the corresponding point cloud, the target radius is reduced; if the local average neighbor distance is greater than the global average neighbor distance of the corresponding point cloud, the target radius is increased.

[0066] S106: Based on the target radius corresponding to each key point, determine the neighborhood point set corresponding to each key point.

[0067] The local average neighbor distance of each keypoint is determined based on the radius to define the neighborhood point set. Then, the local average neighbor distance of the keypoint is obtained based on the distances between each point in the neighborhood point set and the keypoint. The global average neighbor distance of the point cloud is determined based on the local average neighbor distances of each keypoint. For example, the local average neighbor distances of each keypoint are summed and then divided by the number of keypoints to obtain the global average neighbor distance of the point cloud.

[0068] To facilitate understanding, we will use the source point cloud as an example. First, we assign an initial radius to each keypoint. Then, based on the initial radius, we determine the neighborhood point set corresponding to each keypoint. Based on the distance between each point in the neighborhood point set and the keypoint, we obtain the local average neighbor distance of the keypoint. Based on the local average neighbor distance of each keypoint, we determine the global average neighbor distance of the point cloud. Then, we calculate a target radius. If the local average neighbor distance is less than the corresponding global average neighbor distance of the point cloud, we reduce the target radius. If the local average neighbor distance is greater than the corresponding global average neighbor distance of the point cloud, we increase the target radius. This process continues until we obtain the target radius corresponding to each keypoint. For example, we determine the radius at which the local average neighbor distance equals the corresponding global average neighbor distance of the point cloud as the target radius.

[0069] The formula for calculating the adaptive radius can be:

[0070]

[0071] in It is the base radius, that is, the reference radius under average density conditions. This is the key point. The local average neighbor distance, Global average neighbor distance (cloud computing once for the entire point). The adjustment parameter determines the sensitivity of exponential growth decay (0.2~0.5), at which point the neighborhood point set of all key points is obtained.

[0072] S108: Through feature entropy filtering and geometric constraints, feature descriptors corresponding to each key point are generated based on the neighborhood point set of each key point.

[0073] The generation of feature descriptors can include first estimating the normal vector based on the neighborhood point set of each key point, then constructing a neighborhood point pair set, defining the coordinate system, and performing subset selection based on feature entropy. Finally, feature histogram encoding is performed on the selected subset to obtain the feature descriptor.

[0074] S110: Match the source point cloud and the target point cloud based on the feature descriptors corresponding to each key point.

[0075] The matching of source point cloud and target point cloud based on the feature descriptors corresponding to each key point can be achieved by first determining the target matching point pair. The target matching point pair can be obtained by first determining the initial matching pair based on the feature descriptors corresponding to each key point, and then filtering the initial matching pair. The filtering method includes, but is not limited to, at least one of the following: second-order consistency filtering method, bidirectional matching filtering method, triplet filtering method, and inlier number filtering method.

[0076] Among them, the combination Figure 2 As shown, key points are selected through voxel sampling, and the neighborhood radius is adaptively determined based on local density. Normal vectors are estimated for key points and their neighborhoods to construct rotation-invariant local features. Subsequently, feature discriminativeness is improved through subset entropy screening, and a robust feature descriptor operator is constructed. In the coarse registration stage, the robustness of point pairs is evaluated using a second-order spatial consistency matrix. Matching points are screened by combining adaptive thresholding and bidirectional verification, and the most reliable point pair triples are selected through comprehensive geometric consistency scoring. Finally, SVD is used to obtain a coarse registration result with high robustness and high accuracy.

[0077] The point cloud matching method combining feature entropy filtering and geometric constraints described above performs voxel sampling on the source and target point clouds to obtain key points corresponding to the source and target point clouds. Based on the local average neighbor distance of each key point and the global average neighbor distance of the corresponding point cloud, an adaptive radius search is performed to determine the target radius corresponding to each key point. Specifically, if the local average neighbor distance is less than the global average neighbor distance of the corresponding point cloud, the target radius is reduced; if the local average neighbor distance is greater than the global average neighbor distance of the corresponding point cloud, the target radius is increased. Based on the target radius corresponding to each key point, the neighborhood point set corresponding to each key point is determined. Based on the neighborhood point set of each key point, a feature descriptor corresponding to each key point is generated. Based on the feature descriptor corresponding to each key point, the source and target point clouds are matched. In this way, the adaptive radius search based on the local average neighbor distance of each key point and the global average neighbor distance of the corresponding point cloud to determine the target radius corresponding to each key point ensures the accuracy of the neighborhood point set, thereby ensuring the accuracy of the feature descriptor and improving the accuracy of subsequent matching.

[0078] In some optional embodiments, the method further comprises Figure 3 as shown, Figure 3 a flowchart for generating feature descriptors for each key point in an embodiment, the feature descriptor generation step, i.e., generating a feature descriptor corresponding to each key point based on the neighborhood point set of each key point, includes:

[0079] S302: generating a surface normal vector corresponding to each key point based on the neighborhood point set of each key point.

[0080] wherein surface normal vector estimation is performed for each key point and the point set within its adaptive neighborhood. Using the principal component analysis (PCA) method, by calculating the covariance matrix of the neighborhood points, the eigenvector corresponding to the smallest eigenvalue is determined as the surface normal vector of the key point. Distance weight can be introduced in the process of normal vector calculation to improve the estimation accuracy.

[0081]

[0082]

[0083] wherein, is the covariance matrix, is the number of neighborhood points in the neighborhood point set, is the key distance between the key point P and the neighborhood point, is the kth key point, is the ith neighborhood point corresponding to the kth key point, is a scale parameter, generally 1-3 times the resolution, is the weight, here T means the transpose matrix, and is decomposed into a matrix, and the eigenvector corresponding to the smallest eigenvalue is taken as the normal vector. In this application, is also the meaning of i, i.e. i.e. , the of the following is also similar processing.

[0084] S304: dividing the neighborhood points in the neighborhood point set into at least one subset according to the distance between each key point and the neighborhood points in the neighborhood point set corresponding to the key point, each subset including a plurality of neighborhood point pairs, and each neighborhood point pair including a key point and a neighborhood point.

[0085] wherein according to the distance between the key point and the neighborhood point, M subsets are divided, and subsequent subset entropy screening is performed.

[0086]

[0087] is a neighborhood point pair, including point and point .

[0088] S306: For each neighbor point in the neighbor point set corresponding to each key point and the key point, a Darboux coordinate system is constructed based on the surface normal vector corresponding to the key point, and a rotation matrix is calculated for each neighbor point pair based on the Darboux coordinate system.

[0089] The core advantage of using this coordinate system is its inherent rotation invariance. The axis of the coordinate system is constructed by the point-to-point vector and the local normal vector. When the entire point cloud is rotated, the coordinate system will rotate with it, so that the relative position of the point in its own Darboux coordinate system remains unchanged. Therefore, the features extracted based on the Darboux coordinate system are not affected by global pose changes, and have good stability and consistency. At the same time, the Darboux coordinate system provides a standardized local reference frame, so that even in areas with repeated structures, the geometric information around different key points can be uniquely and robustly described, significantly improving feature discrimination and effectively reducing the false match rate. The specific process is as follows:

[0090] For each key point and its neighbor points , a Darboux coordinate system is constructed, which has the characteristic of rotation invariance, and is not easy to confuse in parts with repeated features. The Darboux coordinate system is established according to the following formula, which includes three coordinate axes: , , , where is the surface normal vector:

[0091]

[0092] The Darboux coordinate system of the key point and the neighbor point is established:

[0093] The Darboux coordinate system of the key point : ;

[0094] The Darboux coordinate system of the neighbor point : .

[0095] The following is the representation method of the rotation matrix between each neighbor point pair (i.e. the rotation matrix from the key point to the neighbor point ):

[0096]

[0097] S308: Screen the subsets based on the entropy of each neighbor point pair in each subset to obtain the target subset.

[0098] In the embodiment, a statistical analysis mechanism based on feature variance is introduced to evaluate and optimize the information of the subset of neighborhood point pairs. By retaining the high-information subset with larger variance, the method effectively eliminates noise-sensitive, single-structure, or redundant features, thereby significantly improving the stability and discrimination of feature description. At the same time, the number of filtered features is reduced, making subsequent matching and optimization calculation more efficient, achieving dual improvement of feature quality and processing efficiency. This mechanism makes the feature descriptor focus on areas with significant geometric differences and unique structures, enhancing the robustness and accuracy of point cloud matching. The specific process is as follows:

[0099] First, normalize to eliminate the influence of different scales of distance and angle.

[0100] Then calculate the variance of the feature dimension of each point pair in the subset:

[0101]

[0102] wherein is the mean, is the mth subset in the M subsets, and N1 is the number of neighborhood point pairs in the subset, is the variance, wherein and are the point pairs in the subset, and the total variance of the subset is calculated as:

[0103]

[0104] wherein, is the variance corresponding weight, and only the top 50% is retained as the output This can retain the obvious part of the feature, which has a positive effect on efficiency and stability. In other embodiments, it can also retain 40% or other values, without specific limitation.

[0105] S310: Convert the rotation matrix to Euler angles, calculate the geometric properties of each target subset based on the Euler angles and the distances of the neighborhood points in the neighborhood point sets corresponding to each key point and key point pair, and obtain the feature descriptor corresponding to each key point based on the geometric properties.

[0106] After completing the subset filtering, the retained feature vector corresponding to each key point is histogram encoded and normalized.

[0107] In some optional embodiments, the rotation matrix is converted into Euler angles, the geometric property of each target subset is calculated based on the Euler angles and the distance between each key point and the neighborhood points corresponding to the key point in the neighborhood point set, and the feature descriptor corresponding to the key point is obtained based on the geometric property, including: for each key point, the geometric property of each target subset is calculated based on the rotation matrix of each point pair in the target subset and the surface normal vector corresponding to the key point; the geometric property is discretized and divided into corresponding intervals, and the frequency of occurrence of the geometric property in each interval is counted to obtain a multi-dimensional joint histogram; and the feature descriptor corresponding to each key point is constructed based on the multi-dimensional joint histogram.

[0108] Specifically, the geometric property (such as distance, included angle, normal deviation, Euler angle difference, etc.) of each subset is discretized into a fixed number of intervals, and a multi-dimensional joint histogram is constructed by counting the frequency.

[0109] In this application, the rotation matrix is converted into Euler angles, and the rotation property is represented by Euler angles:

[0110]

[0111] The histogram can be regarded as a statistical representation of the feature distribution, which can effectively capture the change rule of the local geometric structure. Then, the histogram is L2 normalized to eliminate the scale difference and enhance the consistency of the descriptor. Finally, the feature of each key point is encoded into a high-dimensional but compact feature vector, at this time the dimension of each key point is M_select*N_angle*N_dist, M_select is the number of subsets selected; the number of angle intervals N_angle, the number of histogram intervals (bins) for discretizing the angle feature; the number of distance intervals N_dist, the number of histogram intervals (bins) for discretizing the distance feature. Both retain the local geometric detail difference and have global rotation invariance and strong discriminative ability.

[0112] In some optional embodiments, based on the feature descriptor corresponding to each key point, the source point cloud and the target point cloud are matched, including: matching the feature descriptor of each key point in the source point cloud and the feature descriptor of each key point in the target point cloud to obtain an initial matching point pair; screening the initial matching point pair to obtain a target matching point pair; and matching the source point cloud and the target point cloud based on the target matching point pair.

[0113] First, for each key point in the source point cloud, find its nearest neighbor of the feature descriptor in the target point cloud (e.g., using the Euclidean distance), to form an initial matching point pair. Then, the initial matching point pair is screened to obtain a target matching point pair, wherein the screening method includes but is not limited to at least one of the following: a second-order consistency screening method, a bidirectional matching screening method, a triplet screening method, and an inner point number screening method. Finally, based on the target matching point pair, the source point cloud and the target point cloud are matched.

[0114] The present application aims at the limitations of traditional point cloud matching methods based on nearest neighbor or feature space distance in repeated structure, noise interference, and rigid mismatching, etc. A robust coarse registration method based on second-order consistency matrix (SC2) is proposed. The method analyzes the high-order structure relationship between point pairs, measures their consistency degree in global and local geometric topology, and thus realizes effective elimination of false matching and preliminary estimation of stable transformation. The overall process includes four main stages: second-order consistency construction, adaptive threshold filtering, bidirectional verification, and triplet geometric consistency test, finally outputting the optimal rigid transformation initial value to provide high-quality input for subsequent fine registration.

[0115] In some optional embodiments, screening the initial matching point pairs to obtain the target matching point pairs includes: calculating the second-order consistency matrix of each initial matching point pair; obtaining an adaptive threshold based on the second-order consistency matrix of each initial matching point pair, and screening the initial matching point pairs based on the adaptive threshold and the second-order consistency matrix.

[0116] wherein, the source point cloud and the target point cloud wherein and are initial matching point pairs.

[0117]

[0118] Construct the C2 distance consistency matrix:

[0119]

[0120] Set the threshold to determine the consistency:

[0121]

[0122]

[0123] The final first-order consistency matrix is: . is the threshold.

[0124] For each source point pair , compute its accumulated consistency score with all other points , 1, 2....N-1:

[0125]

[0126] where SC2 reflects the reliability of the point pair in the local point cloud map, the higher the score, the more robust the match.

[0127] Subsequent filtering based on adaptive threshold to preliminarily screen out high similarity matching point pairs.

[0128] Compute the mean and standard deviation of SC2 matrix, distance threshold The following formula is used to calculate

[0129]

[0130]

[0131]

[0132] q is an adjustable parameter, generally 2, if you want to retain more data points, reduce q, otherwise increase q. N is the total number of key point matching pairs for constructing the SC2 matrix.

[0133] In some optional embodiments, the screening of the initial matching point pairs to obtain the target matching point pairs further includes: based on the second-order consistency matrix of each initial matching point pair, finding a matching second point in the target point cloud for a first point in the source point cloud in the screened initial matching point pair; based on the second-order consistency matrix of each initial matching point pair, finding a matching fourth point in the source point cloud for a third point in the target point cloud in the screened initial matching point pair; in the case that the first point and the fourth point are the same, and the second point and the third point are the same, retaining the initial matching point pair; in the case that the first point and the fourth point are the same, and the second point and the third point are not the same, deleting the initial matching point pair.

[0134] In this embodiment, the reliability of the match is further improved by bidirectional verification, and ambiguous matches are eliminated, specifically:

[0135] Forward matching: for each key point in the source point cloud, find its k most similar neighbors in the target point cloud (based on the SC2 score matrix).

[0136] Reverse matching: for each key point in the target point cloud, find its k most similar neighbors in the source point cloud.

[0137] Bidirectional confirmation: only when the best match of is and the best match of is also The matching pair is retained only when the verification is passed.

[0138] Finally, the matching pairs that pass the bidirectional verification are obtained to filter the point pairs and obtain a more stable point set.

[0139] In some optional embodiments, the filtering of the initial matching point pairs to obtain the target matching point pairs further includes: selecting point pair triplets from the retained initial matching point pairs; calculating geometric features of each point pair triplet; calculating similarity scores of each type of geometric feature, and obtaining a comprehensive geometric consistency score based on the similarity scores of each type of geometric feature; and filtering the triplets based on the comprehensive geometric consistency score, and the number of triplets is greater than or equal to the number threshold.

[0140] The point pair triplets that are most consistent in the overall geometric structure are selected through the comprehensive scoring method (combining the edge length ratio similarity, the angle consistency, and the volume similarity), to provide robust input for subsequent coarse matching transformation estimation.

[0141] From the matching point set that passes the bidirectional verification, point pair triplets are randomly or systematically selected:

[0142]

[0143] wherein is the source point cloud triplet, is the target point cloud triplet.

[0144] The geometric features of each triplet are calculated, L is the edge length feature, is the angle feature, is the volume feature:

[0145]

[0146]

[0147]

[0148] The following three types of similarity scores are calculated respectively:

[0149]

[0150]

[0151]

[0152] The comprehensive geometric consistency score is:

[0153]

[0154]

[0155] According to the threshold, the triplets meeting the geometric consistency are screened, and if the result of the screening is less than 30%, 30% is automatically completed to prevent too little data, and in other embodiments, 30% can be replaced by other values, which are not specifically limited here.

[0156] In this embodiment, it is further verified whether the point pairs are consistent under the overall geometric structure. All triplets meeting the conditions are retained.

[0157] In some optional embodiments, based on the target matching point pairs, the source point cloud and the target point cloud are matched, including: generating candidate rigid transformation matrices based on the screened triplets; applying each candidate rigid transformation matrix to the source point cloud, and counting the number of inliers in the target point cloud that meet the distance threshold; determining the target rigid transformation matrix based on the number of inliers corresponding to each candidate rigid transformation matrix; and matching the source point cloud and the target point cloud based on the target rigid transformation matrix.

[0158] In this embodiment, SVD is used to calculate the matrix for all consistent triplets, and the one with the most inliers is selected as the output matrix of the final coarse matching.

[0159] Specifically, for each triplet that passes the test, a candidate rigid transformation matrix is calculated using the three matching point pairs it contains through the SVD method. Each candidate transformation is applied to the entire source point cloud key point set, and the number of inliers in the target point cloud that meet a certain distance threshold is counted. The candidate transformation matrix with the most inliers is selected as the final output of the coarse registration method.

[0160] In the above embodiments, by combining adaptive local feature description and second-order geometric consistency constraint, a high-robustness and high-precision global point cloud coarse registration effect is achieved in complex scenes.

[0161] In the local feature construction stage, the neighborhood radius is automatically adjusted through an adaptive neighborhood search mechanism, so as to balance the stability of dense and sparse areas; then a Darboux coordinate system is constructed to realize rotation-invariant geometric expression. Through the subset entropy screening mechanism, the feature area with high information quantity and significant geometric difference is retained, further enhancing the feature discrimination degree and reducing redundancy. After histogram encoding and normalization, high-quality local features are obtained which are not sensitive to noise and attitude changes.

[0162] In the global coarse registration stage, the robustness of the matching point pairs is measured by constructing a second-order geometric consistency matrix (SC2), and inconsistent or ambiguous matches are screened out by using adaptive threshold and bidirectional verification; then the most consistent point pair triplets are selected based on the comprehensive geometric score of the fusion edge length ratio, angle consistency and volume similarity, and the optimal rigid transformation matrix is calculated by using SVD decomposition.

[0163] The method can still obtain correct global pose relationship stably in a complex point cloud scene with noise, occlusion and repeated structure, and significantly improves the robustness, matching accuracy and convergence efficiency of the registration algorithm, and provides reliable initial value for subsequent fine registration and three-dimensional reconstruction. The present application takes a cad model as a target object, and performs experiments on point clouds captured by a camera as matching data. At this time, data is captured at 24 angles, and finally 19 can be successfully matched, with a matching success rate of 79%.

[0164] It should be understood that, although each step in the flowchart involved in each of the above embodiments is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each of the above embodiments can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be alternately executed with at least part of other steps or steps or stages in other steps. It can be understood that the steps in different embodiments can be freely combined as needed, and various non-contradictory schemes formed by the combination are within the scope of protection of the present application.

[0165] Based on the same inventive concept, the embodiments of the present application also provide a point cloud matching device for implementing the point cloud matching method involving the above-mentioned feature entropy screening and geometric constraint combination. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more point cloud matching device embodiments provided below can refer to the limitations of the feature entropy screening and geometric constraint combination point cloud matching method in the above text, which will not be repeated here.

[0166] In an exemplary embodiment, as shown in Figure 4 A feature entropy screening and geometric constraint combination point cloud matching device is provided, comprising: a key point determination module 401, an adaptive radius determination module 402, a neighborhood point set determination module 403, a feature descriptor generation module 404, and a matching module 405, wherein:

[0167] The key point determination module 401 is configured to perform voxel sampling on the source point cloud and the target point cloud to obtain corresponding key points of the source point cloud and the target point cloud.

[0168] The adaptive radius determination module 402 is configured to perform adaptive radius searching to determine a target radius corresponding to each key point based on a local average neighbor distance of each key point and a global average neighbor distance of the corresponding point cloud, wherein the target radius is reduced when the local average neighbor distance is less than the global average neighbor distance of the corresponding point cloud, and the target radius is increased when the local average neighbor distance is greater than the global average neighbor distance of the corresponding point cloud.

[0169] The neighborhood point set determination module 403 is configured to determine a neighborhood point set corresponding to each key point based on the target radius corresponding to each key point.

[0170] The feature descriptor generation module 404 is configured to generate a feature descriptor corresponding to each key point based on the neighborhood point set of each key point by feature entropy screening and geometric constraint.

[0171] The matching module 405 is configured to match the source point cloud and the target point cloud based on the feature descriptor corresponding to each key point.

[0172] In some optional embodiments, the feature descriptor generation module 404 is specifically configured to generate a surface normal vector corresponding to each key point based on the neighborhood point set of each key point; divide the neighborhood points in the neighborhood point set into at least one subset according to the distance between each key point and the neighborhood points in the neighborhood point set corresponding to the key point, each subset including a plurality of neighborhood point pairs, and each neighborhood point pair including the key point and one neighborhood point; for each neighborhood point in the neighborhood point set corresponding to each key point and the key point, construct a Dab coordinate system based on the surface normal vector corresponding to the key point, and calculate a rotation matrix corresponding to each neighborhood point pair based on the Dab coordinate systems; screen the subsets based on the entropy of each neighborhood point pair in each subset to obtain a target subset; convert the rotation matrix into Euler angles, calculate the geometric properties of each target subset based on the Euler angles and the distance between each key point and the neighborhood points in the neighborhood point set corresponding to the key point, and obtain the feature descriptor corresponding to each key point based on the geometric properties.

[0173] In some optional embodiments, the feature descriptor generation module 404 is specifically configured to, for each key point, calculate the geometric properties of each target subset based on the rotation matrix of each point pair in the target subset and the surface normal vector corresponding to the key point; discretize the geometric properties into corresponding intervals, and count the frequency of occurrence of the geometric properties in each interval to obtain a multi-dimensional joint histogram.

[0174] The feature descriptor corresponding to each key point is constructed based on the multi-dimensional joint histogram.

[0175] In some optional embodiments, the matching module 405 is specifically configured to match the feature descriptor of each key point in the source point cloud and the feature descriptor of each key point in the target point cloud to obtain initial matching point pairs; screen the initial matching point pairs to obtain target matching point pairs; and match the source point cloud and the target point cloud based on the target matching point pairs.

[0176] In some optional embodiments, the matching module 405 is specifically configured to calculate a second-order consistency matrix of each initial matching point pair; obtain an adaptive threshold based on the second-order consistency matrix of each initial matching point pair, and screen the initial matching point pairs based on the adaptive threshold and the second-order consistency matrix.

[0177] In some optional embodiments, the matching module 405 is specifically configured to find, based on the second-order consistency matrix of each initial matching point pair, a second point in the target point cloud that matches a first point in the source point cloud in the screened initial matching point pairs; find, based on the second-order consistency matrix of each initial matching point pair, a fourth point in the source point cloud that matches a third point in the target point cloud in the screened initial matching point pairs; retain the initial matching point pair in a case where the first point and the fourth point are the same and the second point and the third point are the same; and delete the initial matching point pair in a case where the first point and the fourth point are the same and the second point and the third point are not the same.

[0178] In some optional embodiments, the matching module 405 is specifically configured to select, from the retained initial matching point pairs, point pair triplets; calculate each type of geometric feature of each point pair triplet; calculate a similarity score of each type of geometric feature, and obtain a comprehensive geometric consistency score based on the similarity scores of each type of geometric feature; and screen the triplets based on the comprehensive geometric consistency score, and the number of the triplets is greater than or equal to a number threshold.

[0179] In some optional embodiments, the matching module 405 is specifically configured to generate candidate rigid transformation matrices based on the screened triplets; apply each candidate rigid transformation matrix to the source point cloud, and count the number of inliers in the target point cloud that satisfy a distance threshold; determine a target rigid transformation matrix based on the number of inliers corresponding to each candidate rigid transformation matrix; and match the source point cloud and the target point cloud based on the target rigid transformation matrix.

[0180] Each module in the point cloud matching device with feature entropy screening and geometric constraint combination can be realized by software, hardware, and combinations thereof, in whole or in part. Each module can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in a computer device in software form, so as to be called and executed by a processor to perform the operations corresponding to each module.

[0181] In an exemplary embodiment, a computer device, which can be a terminal, is provided, and an internal structure diagram of the computer device can be as shown in FIG. 1. Figure 5 The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to perform wired or wireless communication with external terminals. The wireless communication can be achieved through WIFI, mobile cellular network, Near Field Communication (NFC) or other technologies. The computer program is executed by the processor to implement a point cloud matching method combining feature entropy screening and geometric constraints. The display unit of the computer device is configured to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0182] Those skilled in the art can understand that Figure 5 the structure shown in the above embodiment is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0183] In an embodiment, a computer device is also provided, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0184] In an embodiment, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the steps in the above method embodiments.

[0185] In an embodiment, a computer program product is provided, which includes a computer program. The computer program is executed by a processor to implement the steps in the above method embodiments.

[0186] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0187] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments of each method. In the embodiments provided in the present application, any reference to memory, database or other medium can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (Resistive Random Access Memory, ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (Artificial Intelligence, AI) processor, etc., without being limited thereto.

[0188] Any technical features in the above embodiments can be combined, and for the sake of brevity, not all possible combinations are described above, however, any combination of these technical features is deemed to be within the scope of the present application.

[0189] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the patent scope of the present application. It should be pointed out that, for ordinary skilled persons in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A point cloud matching method combining feature entropy screening and geometric constraints, characterized in that, The method comprises: performing voxel sampling on a source point cloud and a target point cloud to obtain key points corresponding to the source point cloud and the target point cloud; performing adaptive radius search based on a local average neighbor distance of each key point and a global average neighbor distance of a corresponding point cloud to determine a target radius corresponding to each key point, wherein the target radius is reduced in a case where the local average neighbor distance is less than the global average neighbor distance of the corresponding point cloud, and the target radius is increased in a case where the local average neighbor distance is greater than the global average neighbor distance of the corresponding point cloud; determining a neighbor point set corresponding to each key point based on a target radius corresponding to each key point; generating a feature descriptor corresponding to each key point based on the neighbor point set of each key point through feature entropy screening and geometric constraint; matching the source point cloud and the target point cloud based on the feature descriptor corresponding to each key point. The generating of the feature descriptor corresponding to each key point based on the neighbor point set of each key point comprises: generating a surface normal vector corresponding to each key point based on the neighbor point set of each key point; dividing neighbor points in the neighbor point set into at least one subset according to distances between each key point and the neighbor points in the neighbor point set corresponding to the key point, each subset comprising a plurality of neighbor point pairs, each neighbor point pair comprising the key point and one neighbor point; constructing a Dab coordinate system for each neighbor point in the neighbor point set corresponding to each key point and the key point based on the surface normal vector corresponding to the key point, and calculating a rotation matrix corresponding to each neighbor point pair based on each Dab coordinate system; screening each subset based on an entropy of each neighbor point pair in the subset to obtain a target subset; converting the rotation matrix into Euler angles, calculating a geometric property of each target subset based on the Euler angles and distances between each key point and the neighbor points in the neighbor point set corresponding to the key point, and obtaining the feature descriptor corresponding to the key point based on the geometric property.

2. The method of claim 1, wherein, The converting of the rotation matrix into Euler angles, the calculating of the geometric property of each target subset based on the Euler angles and distances between each key point and the neighbor points in the neighbor point set corresponding to the key point, and the obtaining of the feature descriptor corresponding to the key point based on the geometric property comprise: calculating the geometric property of each target subset based on the rotation matrix of each point pair in the target subset and the surface normal vector corresponding to the key point for each key point; discretely dividing the geometric property into a corresponding interval, and counting frequencies of the geometric property in each interval to obtain a multi-dimensional joint histogram; constructing the feature descriptor corresponding to each key point based on the multi-dimensional joint histogram.

3. The method according to claim 1 or 2, characterized in that, The matching of the source point cloud and the target point cloud based on the feature descriptor corresponding to each key point comprises: matching the feature descriptor of each key point in the source point cloud and the feature descriptor of each key point in the target point cloud to obtain an initial matching point pair. screening the initial matching point pairs to obtain target matching point pairs; matching the source point cloud and the target point cloud based on the target matching point pairs.

4. The method of claim 3, wherein, The screening the initial matching point pairs to obtain target matching point pairs comprises: calculating a second-order consistency matrix of each of the initial matching point pairs; obtaining an adaptive threshold based on the second-order consistency matrix of each of the initial matching point pairs, and screening the initial matching point pairs based on the adaptive threshold and the second-order consistency matrix.

5. The method of claim 4, wherein, The screening the initial matching point pairs to obtain target matching point pairs further comprises: finding a matching second point in the target point cloud for a first point in the source point cloud in the screened initial matching point pairs based on the second-order consistency matrix of each of the initial matching point pairs; finding a matching fourth point in the source point cloud for a third point in the target point cloud in the screened initial matching point pairs based on the second-order consistency matrix of each of the initial matching point pairs; in a case where the first point and the fourth point are the same and the second point and the third point are the same, retaining the initial matching point pair; in a case where the first point and the fourth point are the same and the second point and the third point are not the same, deleting the initial matching point pair.

6. The method of claim 5, wherein, The screening the initial matching point pairs to obtain target matching point pairs further comprises: selecting point pair triplets from the retained initial matching point pairs; calculating each type of geometric feature of each of the point pair triplets; calculating a similarity score of each type of geometric feature, and obtaining a comprehensive geometric consistency score based on the similarity scores of each type of geometric feature; screening the triplets based on the comprehensive geometric consistency score, and the number of the triplets being greater than or equal to a number threshold.

7. The method of claim 6, wherein, The matching the source point cloud and the target point cloud based on the target matching point pairs comprises: generating candidate rigid body transformation matrices based on the screened triplets; applying each of the candidate rigid body transformation matrices to the source point cloud, and counting the number of inliers in the target point cloud that satisfy a distance threshold; determining a target rigid body transformation matrix based on the number of inliers corresponding to each of the candidate rigid body transformation matrices; matching the source point cloud and the target point cloud based on the target rigid body transformation matrix.

8. A point cloud matching device combining feature entropy screening and geometric constraints, characterized in that, The device comprises: a key point determination module configured to perform voxel sampling on a source point cloud and a target point cloud to obtain key points corresponding to the source point cloud and the target point cloud; an adaptive radius determination module configured to perform adaptive radius search to determine a target radius corresponding to each of the key points based on a local average neighbor distance of each of the key points and a global average neighbor distance of the point cloud corresponding to each of the key points, wherein the target radius is reduced in a case where the local average neighbor distance is less than the global average neighbor distance of the point cloud corresponding to each of the key points, and the target radius is increased in a case where the local average neighbor distance is greater than the global average neighbor distance of the point cloud corresponding to each of the key points; a neighborhood point set determination module configured to determine a neighborhood point set corresponding to each of the key points based on the target radius corresponding to each of the key points. The feature descriptor generation module is configured to generate a feature descriptor corresponding to each of the key points based on a neighborhood point set of each of the key points through feature entropy screening and geometric constraint; The matching module is configured to match the source point cloud and the target point cloud based on the feature descriptors corresponding to the key points. The feature descriptor generation module is specifically configured to generate a surface normal vector corresponding to each of the key points based on the neighborhood point set of each of the key points; divide neighborhood points in the neighborhood point set into at least one subset according to distances between each of the key points and the neighborhood points in the neighborhood point set corresponding to the key point, each of the subsets including a plurality of neighborhood point pairs, and each of the neighborhood point pairs including the key point and one of the neighborhood points; for each of the neighborhood points in the neighborhood point set corresponding to each of the key points and the key point, construct a Dab coordinate system based on the surface normal vector corresponding to the key point, and calculate a rotation matrix corresponding to each of the neighborhood point pairs based on the Dab coordinate systems; screen each of the subsets based on entropies of each of the neighborhood point pairs in the subset to obtain a target subset; convert the rotation matrix into Euler angles, calculate geometric properties of each of the target subsets based on the Euler angles and distances between each of the key points and the neighborhood points in the neighborhood point set corresponding to the key point, and obtain the feature descriptor corresponding to each of the key points based on the geometric properties.

9. The apparatus of claim 8, wherein, The feature descriptor generation module is specifically configured to, for each of the key points, calculate geometric properties of each of the target subsets based on the rotation matrix of each of the point pairs in the target subset and the surface normal vector corresponding to the key point; discretize the geometric properties into corresponding intervals, and count frequencies of the geometric properties in each of the intervals to obtain a multi-dimensional joint histogram. The feature descriptor generation module is specifically configured to generate a feature descriptor corresponding to each of the key points based on the multi-dimensional joint histogram.

10. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor, when executing the computer program, implements the steps of the method in any one of claims 1 to 7.

Images