Cross-source point cloud adaptive registration method based on instance segmentation and hierarchy

Abstract

The invention provides a cross-source point cloud adaptive registration method based on instance segmentation and hierarchy. The method comprises the following steps: acquiring a source point cloud P and a target point cloud Q; scene-level matching based on geometric adaptive feature coding is carried out; carrying out object level correspondence based on adaptive instance segmentation and matching ambiguity elimination; and point cloud level registration based on object level point surface residual optimization and global modal consistency fusion optimization is carried out. According to the cross-source point cloud self-adaptive registration method based on instance segmentation and hierarchy provided by the invention, the balance between local geometric detail retention and global space consistency is realized through local point-surface residual optimization and global modal fusion, and finally the precision and recall rate of cross-source point cloud registration are improved.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a cross-source point cloud adaptive registration method based on instance segmentation and hierarchical modeling. Background Technology

[0002] With the widespread application of 3D perception technology in fields such as robot navigation, autonomous driving, and indoor scene reconstruction, cross-source point cloud registration technology, as a core component of multi-sensor data fusion, is becoming increasingly important. The core objective of cross-source point cloud registration is to spatially align 3D point cloud data acquired by different sensors to integrate the advantages of each sensor—for example, LiDAR has high measurement accuracy but uneven point cloud density, depth cameras have high spatial resolution but limited measurement distance, and SfM visual point clouds have low cost but insufficient geometric accuracy. After fusion, it can provide more reliable environmental perception information for mobile intelligent agents.

[0003] However, current cross-source point cloud registration faces three major challenges: First, significant density differences: point clouds from different sensors differ greatly in sampling density and data structure (e.g., sparse point clouds from LiDAR and dense point clouds from Kinect), resulting in poor adaptability of traditional feature extraction methods; Second, low field-of-view overlap: in real-world scenarios, the sensor acquisition ranges often partially overlap, leading to a low ratio of effective matching point pairs and affecting registration stability; Third, ambiguity in multi-object matching: in complex scenes with multiple objects of similar shapes, traditional methods are prone to confusing object correspondences, resulting in decreased registration accuracy.

[0004] Existing technologies are mainly divided into two categories: methods based on traditional optimization and methods based on deep learning. Methods based on traditional optimization often employ a "coarse-to-fine" strategy (such as CSC2F and CSGMM algorithms), graph matching and tensor optimization (such as CSGM and GCTR algorithms), or cluster fusion. However, these methods struggle to simultaneously achieve global consistency and local accuracy, and are insufficiently adaptable to differences in cross-source data density. Methods based on deep learning, such as FMR and FF-LOGO, improve robustness through feature metric optimization or local-global fusion, but lack geometric constraints at the object instance level. They are prone to matching ambiguities in scenes with many similar objects and have limited processing capabilities for extremely sparse point clouds (such as distant LiDAR point clouds).

[0005] Existing technologies still suffer from low registration accuracy and insufficient robustness in complex scenarios with multiple objects and large differences in cross-source density. There is an urgent need for a hierarchical registration scheme that can integrate global structure and local instance features.

[0006] Chinese invention patent application CN119741349A discloses a method and apparatus for multi-view heterogeneous point cloud registration based on semantic segmentation. The method includes: collecting heterogeneous data from top-down and ground-level perspectives of a real scene using different sensors; constructing heterogeneous 3D scene point clouds using a 3D reconstruction algorithm and performing downsampling preprocessing to obtain a training dataset; importing the dataset into the RandLA-Net algorithm for training; using a semantic segmentation algorithm to obtain semantic labels for point clouds from different perspectives, extracting specific label point clouds and performing point cloud filtering; constructing 3D bounding boxes for the filtered label point clouds; indirectly obtaining two heterogeneous point cloud transformation matrices by registering the bounding boxes in two world coordinate systems, thereby achieving multi-view heterogeneous point cloud registration and fusion. The drawback of this method is that it has significant limitations in accuracy and robustness in practical applications. It can only achieve low-precision coarse alignment, which is difficult to meet the requirements of high-precision spatial registration. Moreover, for specific labeled objects with geometric symmetry, relying solely on bounding box registration is prone to directional ambiguity or errors. Without subsequent point-to-point fine calibration steps, the final reliability of multi-view heterogeneous point cloud fusion cannot be guaranteed. Summary of the Invention

[0007] To address the aforementioned technical issues, this invention proposes a cross-source point cloud adaptive registration method based on instance segmentation and hierarchical modeling. By optimizing local point-surface residuals and fusing global modalities, a balance is achieved between preserving local geometric details and maintaining global spatial consistency, ultimately improving the accuracy and recall of cross-source point cloud registration.

[0008] This invention provides a cross-source point cloud adaptive registration method based on instance segmentation and hierarchical classification, including obtaining source point cloud P and target point cloud Q, and further including the following steps:

[0009] Step 1: Perform scene-level matching based on geometric adaptive feature encoding;

[0010] Step 2: Object-level correspondence based on adaptive instance segmentation and matching ambiguity resolution;

[0011] Step 3: Point cloud-level registration based on object-level point-surface residual optimization and global modal consistency fusion optimization.

[0012] Preferably, step 1 includes achieving initial matching of cross-source point clouds through dynamic feature fusion to obtain a global initial transformation matrix. This includes the following sub-steps:

[0013] Step 11: Calculate the complexity of local sets;

[0014] Step 12: Perform geometric encoding;

[0015] Step 13: Match network features.

[0016] In any of the above schemes, step 11 preferably includes initial selection for each point in the source point cloud P and the target point cloud Q. neighboring points as point set Local geometric complexity The calculation formula is:

[0017]

[0018] in, for Neighboring points With the center point Euclidean distance, and and represent the mean and standard deviation of the average neighbor distances of all points in the point cloud, respectively. This is the Sigmoid function.

[0019] In any of the above schemes, step 12 preferably includes fusing pairwise distance embedding and triplet angle embedding to generate point pairs. Geometric embedding The formula is

[0020]

[0021] in, and The projections are embedded for distance and angle, respectively. For pairwise distance embedding, For triplet angle embedding.

[0022] Preferably, in any of the above schemes, the pairwise distance embedding The calculation formula is:

[0023]

[0024]

[0025] in, For point and The Euclidean distance between them The dimension of the data.

[0026] In any of the above schemes, the preferred option is the triplet angle embedding. The calculation formula is:

[0027]

[0028]

[0029] in, For the triple angle embedding vector in the th Encoded values ​​in each dimension For the triple angle embedding vector in the th Encoded values ​​in each dimension Let be the angle value corresponding to the triple formed by node x, node i, and node j. Angle adjustment factor, For dimensional indexing, The total dimension of the angle embedding vector.

[0030] Of any of the above solutions, the preferred one is... The calculation formula is:

[0031]

[0032] in, , .

[0033] In any of the above solutions, step 13 preferably includes the following sub-steps:

[0034] Step 131: Construct a scene-level geometry perception network based on the Transformer architecture, embedding the geometric structure directly into the attention calculation process:

[0035]

[0036] in, and These are the projections of query embedding and key embedding, respectively. For geometric embedding, , Projection embedded in the geometry;

[0037] Step 132: Calculate self-attention and cross-attention based on geometric structure embedding for the point set. and Perform feature extraction and generate features respectively. and and normalized to a unit hypersphere;

[0038] Step 133: Calculate the Gaussian correlation matrix Its elements The formula is:

[0039]

[0040] in, For feature set The Middle Feature vectors of points, For feature set The Middle Feature vectors of points;

[0041] Step 134: Select the Top-N highly relevant feature pairs as the initial matching set:

[0042]

[0043] in, For use in extracting the top [top] of the correlation matrix Indexing operations for the maximum value, Gaussian correlation matrix Corresponding node and nodes The correlation score;

[0044] Step 135: Solve for the initial transformation matrix using C. Complete scene-level coarse registration.

[0045] In any of the above solutions, step 2 preferably includes the following sub-steps:

[0046] Step 21: Adaptive Euclidean clustering segmentation;

[0047] Step 22: Construct instance mappings;

[0048] Step 23: Verify the consistency of spatial layout.

[0049] In any of the above schemes, step 21 preferably includes calculating each point. Local density Based on the density distribution characteristics, an adaptive neighborhood threshold is designed. :

[0050]

[0051] After segmentation, the source point cloud yields a set of instances. The target point cloud yields a set of instances. ;

[0052] in, For point of Nearest neighbor set For Centered on, with radius The volume of the sphere, This is the density adjustment coefficient. For density adjustment parameters, For The number of points in the center's neighborhood. For point radius The neighborhood point set, where m is the total number of source point cloud instance sets and n is the total number of target point cloud instance sets.

[0053] In any of the above solutions, step 22 preferably includes the following sub-steps:

[0054] Step 221: Calculate the superpoint matching propagation correlation degree between instances, for the superpoint correspondence set obtained by scene-level feature matching. Statistical examples The number of matching superpoints included.

[0055] ;

[0056] Step 222: Normalize the number of matches to the match frequency.

[0057] ;

[0058] Step 223: Calculate instance centroid similarity ,

[0059] ;

[0060] Step 224: Integrate the super-point matching propagation correlation and geometric feature similarity to obtain the comprehensive similarity matrix:

[0061] ;

[0062] Given a comprehensive similarity matrix The goal is to find the optimal match. This maximizes the total similarity:

[0063]

[0064] in, The instance index in set Q. For the target point cloud and the source point cloud The instance index that matches each instance, where n is the total number of instances in the target point cloud instance set. For source instance With target instance Superpoint matching propagation correlation between them For source instance With target instance Let M be the set of optimal binary matching pairs that maximizes the total similarity, representing the geometric feature similarity between the pairs.

[0065] In any of the above schemes, step 23 preferably includes calculating the spatial consistency score of instance triples and removing low-consistency matching pairs, as shown in the formula:

[0066]

[0067] in, A triplet consisting of three instances selected from the source point cloud instance set represents the set. , A triplet consisting of three instances selected from the target point cloud instance set represents the set. , For instances in the source point cloud and Euclidean distance between the centroids Parameters for controlling spatial consistency sensitivity, The distance between the centroids of corresponding instances in the target point cloud.

[0068] In any of the above schemes, step 23 preferably includes, for each matching pair Calculate the average spatial consistency score of all triples it participates in, and filter out those with consistency scores below a threshold. Matching pairs:

[0069]

[0070] in, For containing matching pairs The set of all triples, Let be the cardinality of the set.

[0071] In any of the above solutions, step 3 preferably includes the following sub-steps:

[0072] Step 31: Perform point-to-surface residual optimization on each pair of matched object instances to obtain local fine transformation;

[0073] Step 32: Integrate the various local transformations by fusing key point transformations with global consistency.

[0074] In any of the above solutions, step 31 preferably includes the following sub-steps:

[0075] Step 311: For the target point cloud object instance Calculate the normal vector for each point in the vector.

[0076] Step 312: For the source point cloud object instance Each point in Find the plane that is closest to the target point cloud;

[0077] Step 313: Obtain the object-level local fine transformation by solving the least squares problem. ,

[0078]

[0079] in, The distance residual from each point to its corresponding plane.

[0080] In any of the above solutions, step 312 preferably includes the following sub-steps:

[0081] Step 3121: Through initial transformation Will Transform to the target point cloud coordinate system;

[0082] Step 3122: Find points in the target point cloud that satisfy the following formula ,

[0083]

[0084] in, The initial transformation matrix is ​​obtained from object-level matching. For point Time Distance to the plane in which it is located;

[0085] Step 3123: Calculate the distance residual from each point to its corresponding plane:

[0086] .

[0087] In any of the above solutions, step 32 preferably includes the following sub-steps:

[0088] Step 321: From each source object instance Selecting a set of key points ,

[0089]

[0090] in, To from the object The number of key points selected in the text;

[0091] Step 322: Estimate based on local transformation Transform the set of key points of an object instance to obtain the transformed point set. :

[0092] .

[0093] In any of the above schemes, step 322 preferably includes processing the original keypoint set. and the transformed key point set For each pair of corresponding points in the array, solve the global least squares problem to minimize the squared distance error among all points:

[0094] .

[0095] In any of the above schemes, it is preferred to solve the least squares problem using singular value decomposition (SVD), which includes the following sub-steps:

[0096] Step 3221: Calculate the centroids of the original keypoint set and the transformed keypoint set:

[0097]

[0098]

[0099] in, This represents the total number of key points;

[0100] Step 3222: Construct the covariance matrix and perform SVD decomposition:

[0101]

[0102] ;

[0103] Step 3223: Based on the SVD decomposition results, calculate the rotation matrix and translation vector:

[0104] ;

[0105] Step 3224: Combine to obtain the globally optimal transformation:

[0106] .

[0107] This invention proposes a cross-source point cloud adaptive registration method based on instance segmentation and hierarchical modeling, which overcomes the shortcomings of existing cross-source point cloud registration methods in terms of density difference adaptation, multi-object matching ambiguity, and global-local accuracy balance. Attached Figure Description

[0108] Figure 1 This is a flowchart of a preferred embodiment of the cross-source point cloud adaptive registration method based on instance segmentation and hierarchical structure according to the present invention.

[0109] Figure 2 This is a flowchart of a preferred embodiment of the cross-source point cloud adaptive registration method based on instance segmentation and hierarchical structure according to the present invention.

[0110] Figure 3This is a visual schematic diagram of an embodiment of the registration result of the cross-source point cloud adaptive registration method based on instance segmentation and hierarchical classification according to the present invention. Detailed Implementation

[0111] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0112] Example 1

[0113] like Figure 1 As shown, a cross-source point cloud adaptive registration method based on instance segmentation and hierarchical processing is implemented in step 10000 to obtain the source point cloud P and the target point cloud Q.

[0114] Execute step 11000 to perform scene-level matching based on geometric adaptive feature encoding, including initial matching across source point clouds through dynamic feature fusion and obtaining the global initial transformation matrix. This includes the following sub-steps:

[0115] Execute step 11100 to calculate the local set complexity, including initial selection for each point in the source point cloud P and the target point cloud Q. neighboring points as point set Local geometric complexity The calculation formula is:

[0116]

[0117] in, for Neighboring points With the center point Euclidean distance, for Points in Define distance , and and represent the mean and standard deviation of the average neighbor distances of all points in the point cloud, respectively. This is the Sigmoid function.

[0118] Step 11200 involves geometric encoding, including fusing pairwise distance embeddings and triplet angle embeddings to generate point pairs. Geometric embedding The formula is

[0119]

[0120] in, and The projections are embedded for distance and angle, respectively. For pairwise distance embedding, For triplet angle embedding.

[0121] The pairwise distance embedding The calculation formula is:

[0122]

[0123]

[0124] in, For point and The Euclidean distance between them The dimension of the data.

[0125] The triplet angle embedding The calculation formula is:

[0126]

[0127]

[0128] in, For the triple angle embedding vector in the th Encoded values ​​in each dimension For the triple angle embedding vector in the th Encoded values ​​in each dimension Let be the angle value corresponding to the triple formed by node x, node i, and node j. Angle adjustment factor, For dimensional indexing, The total dimension of the angle embedding vector.

[0129] The calculation formula is:

[0130]

[0131] in, , .

[0132] Execute step 11300, matching network features, including the following sub-steps:

[0133] Execute step 11310 to construct a scene-level geometry perception network based on the Transformer architecture, embedding the geometric structure directly into the attention calculation process:

[0134]

[0135] in, and These are the projections of query embedding and key embedding, respectively. For geometric embedding, , Projection embedded in the geometry;

[0136] Execute step 11320, calculate self-attention and cross-attention based on geometric structure embedding, for the point set. and Perform feature extraction and generate features respectively. and and normalized to a unit hypersphere;

[0137] Perform step 11330 to calculate the Gaussian correlation matrix. Its elements The formula is:

[0138]

[0139] in, For feature set The Middle Feature vectors of points, For feature set The Middle Feature vectors of points;

[0140] Execute step 11340, selecting the Top-N highly relevant feature pairs as the initial matching set:

[0141]

[0142] in, For use in extracting the top [top] of the correlation matrix Indexing operations for the maximum value, Gaussian correlation matrix Corresponding node and nodes The correlation score;

[0143] Execute step 11350 to solve for the initial transformation matrix using C. Complete scene-level coarse registration.

[0144] Step 12000, based on adaptive instance segmentation and matching ambiguity resolution, involves object-level correspondence and includes the following sub-steps:

[0145] Execute step 12100, adaptive Euclidean clustering segmentation, including calculating each point Local density Based on the density distribution characteristics, an adaptive neighborhood threshold is designed. :

[0146]

[0147] After segmentation, the source point cloud yields a set of instances. The target point cloud yields a set of instances. ;

[0148] in, For point of Nearest neighbor set For Centered on, with radius The volume of the sphere, This is the density adjustment coefficient. For density adjustment parameters, For The number of points in the center's neighborhood. For point radius Neighborhood point set.

[0149] Execute step 12200 to build the instance mapping relationship, including the following sub-steps:

[0150] Execute step 12210 to calculate the superpoint matching propagation correlation degree between instances, for the superpoint correspondence set obtained by scene-level feature matching. Statistical examples The number of matching superpoints included.

[0151] ;

[0152] Execute step 12220 to normalize the number of matches to the matching frequency.

[0153] ;

[0154] Perform step 12230 to calculate instance centroid similarity. ,

[0155] ;

[0156] Execute step 12240 to fuse the superpoint matching propagation correlation degree and geometric feature similarity to obtain the comprehensive similarity matrix:

[0157] ;

[0158] Given a comprehensive similarity matrix The goal is to find the optimal match. This maximizes the total similarity:

[0159]

[0160] in, The instance index in set Q. For the target point cloud and the source point cloud The instance index that matches each instance, where n is the total number of instances in the target point cloud instance set. For source instance With target instance Superpoint matching propagation correlation between them For source instance With target instance Let M be the set of optimal binary matching pairs that maximizes the total similarity, representing the geometric feature similarity between the pairs.

[0161] Execute step 12300 to verify spatial layout consistency, including calculating the spatial consistency score of instance triples and removing low-consistency matching pairs. The formula is as follows:

[0162]

[0163] in, A triplet consisting of three instances selected from the source point cloud instance set represents the set. That is, three specific instances selected from the source point cloud. A triplet consisting of three instances selected from the target point cloud instance set represents the set. That is, three instances selected from the target point cloud that are considered to be likely to match the aforementioned source instances. For instances in the source point cloud and Euclidean distance between the centroids Parameters for controlling spatial consistency sensitivity, The distance between the centroids of corresponding instances in the target point cloud.

[0164] For each matching pair Calculate the average spatial consistency score of all triples it participates in, and filter out those with consistency scores below a threshold. Matching pairs:

[0165]

[0166] in, For containing matching pairs The set of all triples, Let be the cardinality of the set.

[0167] Step 13000, point cloud-level registration based on object-level point-surface residual optimization and global modal consistency fusion optimization, includes the following sub-steps:

[0168] Execute step 13100, perform point-surface residual optimization on each pair of matched object instances to obtain local fine transformation, including the following sub-steps:

[0169] Execute step 13110 for the target point cloud object instance. Calculate the normal vector for each point in the vector. Includes the following sub-steps:

[0170] Execute step 13121, through initial transformation Will Transform to the target point cloud coordinate system;

[0171] Perform step 13122 to find points in the target point cloud that satisfy the following formula. ,

[0172]

[0173] in, The initial transformation matrix is ​​obtained from object-level matching. For point Time Distance to the plane in which it is located;

[0174] Perform step 13123 to calculate the distance residual from each point to its corresponding plane:

[0175] .

[0176] Execute step 13130 ​​to obtain the object-level local fine transformation by solving the least squares problem. ,

[0177]

[0178] in, The distance residual from each point to its corresponding plane.

[0179] Execute step 13120 for the source point cloud object instance. Each point in Find the nearest plane in the target point cloud, including the following sub-steps:

[0180] Execute step 13121, through initial transformation Will Transform to the target point cloud coordinate system;

[0181] Perform step 13122 to find points in the target point cloud that satisfy the following formula. ,

[0182]

[0183] in, The initial transformation matrix is ​​obtained from object-level matching. For point Time Distance to the plane in which it is located;

[0184] Perform step 13123 to calculate the distance residual from each point to its corresponding plane:

[0185] .

[0186] Execute step 13200, which integrates various local transformations through keypoint transformation and global consistency fusion, including the following sub-steps:

[0187] Execute step 13210, from each source object instance Selecting a set of key points ,

[0188]

[0189] in, To from the object The number of key points selected in the text;

[0190] Perform step 13220 to estimate based on local transformation. Transform the set of key points of an object instance to obtain the transformed point set. :

[0191] .

[0192] For the original key point set and the transformed key point set For each pair of corresponding points in the array, solve the global least squares problem to minimize the squared distance error among all points:

[0193] .

[0194] Solving the least squares problem using Singular Value Decomposition (SVD) includes the following sub-steps:

[0195] Perform step 13221 to calculate the centroids of the original keypoint set and the transformed keypoint set:

[0196]

[0197]

[0198] in, This represents the total number of key points;

[0199] Perform step 13222 to construct the covariance matrix and perform SVD decomposition:

[0200]

[0201] ;

[0202] Perform step 13223, based on the SVD decomposition results, to calculate the rotation matrix and translation vector:

[0203] ;

[0204] Execute step 13224 to obtain the globally optimal transformation:

[0205] .

[0206] Example 2

[0207] The purpose of this invention is to overcome the shortcomings of existing cross-source point cloud registration methods in terms of density difference adaptation, multi-object matching ambiguity, and global-local accuracy balance, and to provide a hierarchical cross-source point cloud registration method (AIS-HCSR) based on adaptive instance segmentation. Specific objectives include:

[0208] ① Construct a three-level progressive registration framework of scene-object-point cloud to overcome the limitations of single-strategy registration and achieve a hierarchical improvement in registration accuracy of "global coarse registration - object-level instance matching - point cloud-level fine optimization";

[0209] ② Design an adaptive geometric feature encoding mechanism to dynamically adjust the distance and angle feature weights based on the local geometric complexity of the point cloud, thereby enhancing the adaptability to cross-source data (such as sparse LiDAR and dense Kinect point clouds);

[0210] ③ We propose an object instance-level matching and spatial layout verification strategy to eliminate matching ambiguity in scenes with many similar objects and improve the robustness of registration in complex scenes.

[0211] By optimizing local point and surface residuals and fusing global modalities, a balance is achieved between preserving local geometric details and maintaining global spatial consistency, ultimately improving the accuracy and recall of cross-source point cloud registration.

[0212] The core of this invention is to construct a three-level progressive registration framework of "scene-level - object-level - point cloud level". Through three key technologies—adaptive feature encoding, instance segmentation and matching, and local-global optimization—it addresses the core challenges of cross-source point cloud registration. The specific implementation steps are as follows:

[0213] Scene-level matching based on geometric adaptive feature encoding

[0214] The scenario-level goal is to achieve initial matching across point clouds through dynamic feature fusion and obtain the global initial transformation matrix. The specific steps include:

[0215] (1) Calculation of local geometric complexity: The local geometric complexity introduced It is a core element for achieving adaptive coding; this parameter can be adjusted based on the point... The local geometric complexity is dynamically adjusted by rebalancing the weights of distance and angle features. For each point in the source point cloud P and the target point cloud Q, an initial selection is made. neighboring points as point set .for Points in Define distance , where represents the Euclidean distance between neighboring points and the center point. Local geometric complexity. The calculation method is as follows:

[0216] (1)

[0217] The local geometric complexity parameter is calculated using the above formula. This parameter is mapped to the [0,1] interval via the Sigmoid function and is used to dynamically adjust the feature weights. and and represent the mean and standard deviation of the average neighbor distances of all points in the point cloud, respectively. This is the Sigmoid function. The first term in the fraction calculates the point. The local geometric complexity is characterized by measuring the average distance to neighboring points to represent the distribution characteristics of the local point cloud. In flat regions... →0 (more dependent on distance features), in complex edge regions →1 (More dependent on angle features).

[0218] (2) Geometric feature encoding: The pairwise distance embedding and triple angle embedding are fused together, and point pairs are generated by formula (2). Geometric embedding To achieve cross-source invariant feature extraction:

[0219] (2)

[0220] in, and These are projections embedded in distance and angle, respectively. It is a geometric structure embedding. It is a pairwise distance embedding. It is a triplet angle embedding.

[0221] Paired distance embedding The calculation formula is:

[0222] (3)

[0223] (4)

[0224] in, Point and The Euclidean distance between them It is a hyperparameter that adjusts distance changes. It is the dimension of the data.

[0225] Triple angle embedding The calculation formula is:

[0226] (5)

[0227] (6)

[0228] Initial selection neighboring points as point set .for Points in , The calculation formula is:

[0229] (7)

[0230] in Defined as .

[0231] (3) Network feature matching: The geometric structure is embedded into the GeoTransformer network. First, a scene-level geometry perception network is constructed. This network is based on the Transformer architecture and, taking into account the geometric characteristics of point cloud data, the geometric structure is embedded directly into the attention calculation process.

[0232] (8)

[0233] in, and These represent the projections of query embedding and key embedding, respectively. Geometric structure embedding, generated by encoding geometric features:

[0234] in This represents the projection of the geometric embedding.

[0235] Self-attention and cross-attention are computed based on geometric structure embedding for point sets. and Perform feature extraction and generate features respectively. and Normalize to a unit hypersphere; calculate the Gaussian correlation matrix. Its elements Defined as:

[0236] (9)

[0237] Select the top-N highly relevant feature pairs as the initial matching set:

[0238] (10)

[0239] Solving the initial transformation matrix using C Complete scene-level coarse registration.

[0240] 1. Object-level correspondence based on adaptive instance segmentation and matching ambiguity cancellation

[0241] The object-level goal is to segment the point cloud into physically meaningful object instances and establish reliable instance-level correspondences. Specific steps include:

[0242] (1) Adaptive Euclidean Clustering Segmentation: To address the problem of uneven density in cross-source point clouds, the following formula is used: Calculate each point Local density Based on the density distribution characteristics, an adaptive neighborhood threshold is designed:

[0243] (11)

[0244] By dynamically adjusting the clustering neighborhood threshold To achieve adaptive instance segmentation: where, Point of Nearest neighbor set Representative with Centered on, with radius The volume of the sphere, This is the density adjustment coefficient. The density adjustment parameter (the threshold is dynamically adjusted based on the ratio of local density to global density) is used, and the minimum instance point threshold is set to 100 (to filter noise). After segmentation, the source point cloud yields an instance set. The target point cloud yields a set of instances.

[0245] (2) Instance correspondence construction: By integrating superpoint matching propagation and geometric centroid similarity, an instance-level matching matrix is ​​established, and an optimization algorithm is applied to determine the optimal instance correspondence. Specific steps include:

[0246] Calculate the superpoint matching propagation correlation degree between instances. For the superpoint correspondence set obtained from scene-level feature matching... Statistical examples Number of matching superpoints included:

[0247] (12)

[0248] Normalize the number of matches to the match frequency:

[0249] (13)

[0250] Calculate instance centroid similarity :

[0251] (14)

[0252] This formula uses a Gaussian kernel function to evaluate the similarity of geometric features. The initial transformation matrix obtained from scene-level matching. and Representing object instances respectively and The coordinates of the centroid, This represents the distance difference between the centroid coordinates. This is the standard deviation parameter for spatial distance.

[0253] Finally, by fusing the superpoint matching propagation correlation and geometric feature similarity, a comprehensive similarity matrix is ​​obtained:

[0254] (15)

[0255] The instance matching problem can be formalized as a bipartite graph matching problem. Given a comprehensive similarity matrix... The goal is to find the optimal match. This maximizes the total similarity:

[0256] (16)

[0257] This optimization problem represents finding the set of matches that maximizes the total similarity among all possible combinations.

[0258] (3) Spatial layout consistency verification: To address the ambiguity problem of multiple similar objects, the spatial consistency score of instance triples is calculated using a formula, and low-consistency matching pairs are eliminated: (17)

[0259] Here, Indicates instances in the source point cloud and Euclidean distance between the centroids Parameters used to control spatial consistency sensitivity. This represents the distance between the centroids of corresponding instances in the target point cloud. This represents the sum of the Euclidean distances between the three instances.

[0260] For each matching pair Calculate the average spatial consistency score of all triples it participates in, and filter out those with consistency scores below a threshold. Matching pairs:

[0261] (18)

[0262] In the formula, Indicates that it contains matching pairs The set of all triples, The cardinality of the set. Remove. This matching mechanism effectively suppresses erroneous matches caused by similar object shapes, improving the reliability of the matching results. The final instance matching set is then obtained. .

[0263] 2. Point cloud-level registration based on object-level point-surface residual optimization and global modal consistency fusion optimization

[0264] The goal at the point cloud level is to achieve registration results that retain local accuracy while ensuring global consistency through this local-to-global optimization strategy.

[0265] The process is mainly divided into two key stages: performing point-surface residual optimization on each pair of matched object instances to obtain local fine transformations; and integrating the various local transformations through key point transformation and global consistency fusion.

[0266] (1) Object-level point and surface residual optimization

[0267] For each pair of object instances obtained through comprehensive similarity matching First, point-to-surface residual optimization is needed to obtain a more accurate local transformation. The specific implementation steps are as follows:

[0268] First, for the target point cloud object instance Calculate the normal vector for each point in the vector. .

[0269] Secondly, for source point cloud object instances Each point in This requires finding the nearest plane in the target point cloud. This step begins with an initial transformation. Will Transform to the target point cloud coordinate system, and then find points in the target point cloud that satisfy the following formula. :

[0270] (19)

[0271] In the above formula, It is the initial transformation matrix obtained from object-level matching. Point Time The distance between the points and the plane. By minimizing this distance, the most suitable point-to-plane correspondence can be found.

[0272] Then, the distance residual from each point to its corresponding plane is calculated:

[0273] (20)

[0274] The residual value represents the perpendicular distance from the transformed point to the corresponding plane, and is a direct measure of registration accuracy. The smaller the residual value, the more accurate the registration result.

[0275] Finally, by solving the least squares problem, the object-level local fine transformation is obtained. :

[0276] (twenty one)

[0277] Through the above optimization process, a local fine-tuning transformation was obtained for each pair of matched object instances. ,in This represents the number of successfully matched object pairs. These local transformations can accurately describe the registration relationship of each object, but since they are optimized independently, there may be some inconsistencies, requiring further global optimization.

[0278] (2) Global modal consistency fusion optimization

[0279] After completing the object-level point-surface residual optimization, the local fine transformation of each pair of matched object instances is obtained, and a global optimization strategy integrating the local transformation is proposed based on this.

[0280] After obtaining the local optimal transformation, global key point least squares optimization is required to integrate local modal consistency adjustment.

[0281] In the specific implementation, firstly, from each source object instance... Selecting a set of key points :

[0282] (twenty two)

[0283] in From the object The number of key points selected. The key point selection adopts the farthest point sampling (FPS) algorithm to ensure uniform coverage within the spatial range of the point cloud, so as to preserve complete geometric feature information.

[0284] Subsequently, estimation based on local transformation Transform the set of key points of an object instance to obtain the transformed point set. :

[0285] (twenty three)

[0286] For the original key point set and the transformed key point set For each pair of corresponding points in the array, solve the following global least squares problem to minimize the squared distance error among all points:

[0287] (twenty four)

[0288] This least squares problem can be solved efficiently using singular value decomposition (SVD). First, the centroids of the original keypoint set and the transformed keypoint set are calculated:

[0289] (25)

[0290] in It is the total number of key points.

[0291] Then, the covariance matrix is ​​constructed and SVD decomposition is performed:

[0292] (26)

[0293] (27)

[0294] Based on the SVD decomposition results, calculate the rotation matrix and translation vector:

[0295] (28)

[0296] The final combination yields the globally optimal transformation:

[0297] (29)

[0298] This yields the final optimized transformation through local-to-global modal consistency transformation estimation. .

[0299] From the perspectives of registration accuracy and robustness, this invention effectively balances global consistency and local accuracy.

[0300] From the perspective of cross-source data adaptability, this invention employs an innovatively designed adaptive mechanism to effectively address the differences in characteristics between point clouds from different sensors. Scene-level adaptive geometric feature encoding dynamically adjusts the weights of distance and angle features based on local geometric complexity, relying on distance features in flat areas and emphasizing angle features in complex edge areas, thus adapting to the modal heterogeneity of LiDAR (sparse, high precision), Kinect depth camera (dense, medium precision), and SfM visual point clouds (low precision, rich visual features). The object-level adaptive Euclidean clustering algorithm breaks through the fixed threshold limitation, dynamically adjusting the neighborhood threshold based on local density, preserving details in high-density areas and ensuring connectivity in low-density areas, solving the segmentation failure problem caused by uneven density in cross-source point clouds. Point cloud-level point-surface residual optimization and global modal consistency fusion ensure overall spatial consistency while preserving local geometric details of objects, meeting the high-precision pose estimation requirements of mobile intelligent agents (such as robot navigation and autonomous driving).

[0301] From the perspective of the synergy and reliability of the method design, the hierarchical framework modules of this invention form a highly efficient complementarity. The two modules, scene-level matching providing a globally consistent initial transformation and object-level matching and local-global optimization eliminating local ambiguity, have a significant synergistic effect, proving the rationality of the framework design and avoiding the shortcomings of traditional single-level methods that are prone to getting trapped in local optima and pure deep learning methods that lack geometric constraints.

[0302] Example 3

[0303] Table 1 shows a comparative analysis of the cross-source point cloud registration performance between the method of this invention and existing methods. Experimental results show that the proposed method achieves excellent results in the three key indicators of recall, translation error, and rotation error, and exhibits significant advantages, especially in handling complex multi-object scenes.

[0304] Table 1. Comparison Experiment Results of 3DCSR Dataset Example 4

[0305] Table 2 shows the ablation experiment results of the method of this application, which can be used to explore in depth the impact of scene registration, object registration, and local-global optimization on the registration performance of the method of this invention.

[0306] Table 2 Ablation experiments using this method Scene Registration Object Matching and Logo Recall (%) √ 63.86 √ 11.38 √ √ 81.19 Ablation experiments show that scene-level matching provides a globally consistent initial transformation (63.86% recall when used alone), while object-level matching and local-global optimization eliminate local ambiguity (11.38% recall when used alone). When combined, the recall jumps to 81.19%. The significant synergistic effect proves the rationality of the framework design and avoids the shortcomings of traditional single-level methods that are prone to getting trapped in local optima and pure deep learning methods that lack geometric constraints.

[0307] Example 5 The overall framework design of this method is as follows: Figure 2 As shown, the core logic is "from coarse to fine, from global to local". First, the global initial pose is obtained through scene-level matching. Then, a reliable correspondence is established through object-level instance segmentation and matching. Finally, fine alignment is achieved through point cloud-level optimization.

[0308] Example 6 On the 3DCSR dataset, the core evaluation metric of the method of this invention, recall, reaches 81.19%, which is 5.45% higher than the current best method FF-LOGO (75.74%), and significantly surpasses traditional optimization methods (such as ICP recall of 24.3% and FilterReg recall of 30.96%) and other deep learning methods.

[0309] Example 7 like Figure 3 As shown in the visualization results, it is evident that the method of this invention achieves excellent accuracy in scenes with complex structures and a large number of objects, such as lidar_scene19_pair11 and lidar_scene16_pair6. Taking lidar_scene19_pair11 as an example, this scene contains objects of multiple shapes and a complex spatial layout; the rotation error of this method is only 1.33°, and the translation error is 0.03m.

[0310] Example 8 In relatively simple scenarios, the registration accuracy difference between the method of this invention and FF-LOGO is not significant, and both methods can achieve relatively accurate point cloud alignment. For example, in the sfm_hard_sofawhole scene, the rotation error of the method of this invention is 1.52° and the translation error is 0.03m, while the rotation error of FF-LOGO is 1.48° and the translation error is 0.02m. This indicates that in scenarios with a small number of objects and relatively simple structures, the FF-LOGO method can provide good registration performance, while the method of this invention maintains stable high accuracy.

[0311] To better understand this invention, specific embodiments have been described in detail above, but these are not intended to limit the invention. Any simple modifications made to the above embodiments based on the technical essence of this invention still fall within the scope of this invention. Each embodiment in this specification focuses on its differences from other embodiments; similar or identical parts between embodiments can be referred to mutually. For system embodiments, since they basically correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

Claims

1. A cross-source point cloud adaptive registration method based on instance segmentation and hierarchical classification, comprising acquiring source point cloud P and target point cloud Q, characterized in that, It also includes the following steps: Step 1: Perform scene-level matching based on geometric adaptive feature encoding; Step 2: Object-level correspondence based on adaptive instance segmentation and matching ambiguity resolution; Step 3: Point cloud-level registration based on object-level point-surface residual optimization and global modal consistency fusion optimization.

2. The cross-source point cloud adaptive registration method based on instance segmentation and hierarchical classification as described in claim 1, characterized in that, Step 1 includes achieving initial matching across source point clouds through dynamic feature fusion and obtaining a global initial transformation matrix. This includes the following sub-steps: Step 11: Calculate the local set complexity, including the initial selection of each point in the source point cloud P and the target point cloud Q. neighboring points as point set Local geometric complexity The calculation formula is: ； in, for Neighboring points With the center point Euclidean distance, and and represent the mean and standard deviation of the average neighbor distances of all points in the point cloud, respectively. For the Sigmoid function; Step 12: Perform geometric encoding, including fusing pairwise distance embeddings and triplet angle embeddings to generate point pairs. Geometric embedding The formula is ； in, and The projections are embedded for distance and angle, respectively. For pairwise distance embedding, For triplet angle embedding; Step 13: Match network features.

3. The cross-source point cloud adaptive registration method based on instance segmentation and hierarchical classification as described in claim 2, characterized in that, The pairwise distance embedding The calculation formula is: ； ； in, For point and The Euclidean distance between them The dimension of the data.

4. The cross-source point cloud adaptive registration method based on instance segmentation and hierarchical classification as described in claim 3, characterized in that, The triplet angle embedding The calculation formula is: ； ； The calculation formula is: ； in, For the triple angle embedding vector in the th Encoded values ​​in each dimension For the triple angle embedding vector in the th Encoded values ​​in each dimension Let be the angle value corresponding to the triple formed by node x, node i, and node j. Angle adjustment factor, For dimensional indexing, The total dimension of the angle embedding vector. , .

5. The cross-source point cloud adaptive registration method based on instance segmentation and hierarchical classification as described in claim 4, characterized in that, Step 13 includes the following sub-steps: Step 131: Construct a scene-level geometry perception network based on the Transformer architecture, embedding the geometric structure directly into the attention calculation process: ； in, and These are the projections of query embedding and key embedding, respectively. For geometric embedding, , Projection embedded in the geometry; Step 132: Calculate self-attention and cross-attention based on geometric structure embedding for the point set. and Perform feature extraction and generate features respectively. and and normalized to a unit hypersphere; Step 133: Calculate the Gaussian correlation matrix Its elements The formula is: ； in, For feature set The Middle Feature vectors of points, For feature set The Middle Feature vectors of points; Step 134: Select the Top-N highly relevant feature pairs as the initial matching set: ； in, For use in extracting the top [top] of the correlation matrix Indexing operations for the maximum value, Gaussian correlation matrix Corresponding node and nodes The correlation score; Step 135: Solve for the initial transformation matrix using C. Complete scene-level coarse registration.

6. The cross-source point cloud adaptive registration method based on instance segmentation and hierarchical classification as described in claim 5, characterized in that, Step 2 includes the following sub-steps: Step 21: Adaptive Euclidean clustering segmentation, including calculating each point Local density Based on the density distribution characteristics, an adaptive neighborhood threshold is designed. : ； After segmentation, the source point cloud yields a set of instances. The target point cloud yields a set of instances. ; in, For point of nearest neighbor set For Centered on, with radius The volume of a sphere, This is the density adjustment coefficient. For density adjustment parameters, For The number of points in the center's neighborhood. For point radius The neighborhood point set, where m is the total number of source point cloud instance sets and n is the total number of target point cloud instance sets; Step 22: Construct instance mappings; Step 23: Verify the consistency of spatial layout.

7. The cross-source point cloud adaptive registration method based on instance segmentation and hierarchical classification as described in claim 6, characterized in that, Step 22 includes the following sub-steps: Step 221: Calculate the superpoint matching propagation correlation degree between instances, for the superpoint correspondence set obtained by scene-level feature matching. Statistical examples The number of matching superpoints included. ； Step 222: Normalize the number of matches to the match frequency. ； Step 223: Calculate instance centroid similarity , ； Step 224: Integrate the super-point matching propagation correlation and geometric feature similarity to obtain the comprehensive similarity matrix: ； Given a comprehensive similarity matrix The goal is to find the optimal match. This maximizes the total similarity: ； in, The instance index in set Q. For the target point cloud and the source point cloud The instance index that matches each instance, where n is the total number of instances in the target point cloud instance set. For source instance With target instance Superpoint matching propagation correlation between them For source instance With target instance Let M be the set of optimal binary matching pairs that maximizes the total similarity, representing the geometric feature similarity between the pairs.

8. The cross-source point cloud adaptive registration method based on instance segmentation and hierarchical classification as described in claim 7, characterized in that, Step 23 includes calculating the spatial consistency score of instance triples and removing low-consistency matching pairs, using the following formula: ； in, This is a triplet consisting of three instances selected from the source point cloud instance set. A triplet consisting of three instances selected from the target point cloud instance set. For instances in the source point cloud and Euclidean distance between the centroids Parameters for controlling spatial consistency sensitivity, The distance between the centroids of corresponding instances in the target point cloud.

9. The cross-source point cloud adaptive registration method based on instance segmentation and hierarchical classification as described in claim 8, characterized in that, Step 23 also includes, for each matching pair Calculate the average spatial consistency score of all triples it participates in, and filter out those with consistency scores below a threshold. Matching pairs: ； in, For containing matching pairs The set of all triples, Let be the cardinality of the set.

Images