A point cloud registration method and system based on geometric perception and global consistency

Abstract

The present application relates to the field of computer vision, and more particularly to a point cloud registration method and system based on geometric perception and global consistency; the method comprises the following steps: S1: obtaining source point cloud and target point cloud to be registered; S2: using a feature extraction network with shared weights to extract initial high-dimensional features of the source point cloud and the target point cloud respectively; S3: inputting the initial high-dimensional features into a geometric vector Transformer to obtain geometric vector features and scalar semantic features; S4: evaluating the importance of the geometric vector features and the scalar semantic features to obtain enhanced features of the source point cloud and the target point cloud; S5: calculating an inner product similarity matrix between the two enhanced features; S6: inputting the inner product similarity matrix into a Sinkhorn global consistency matching network to obtain a correspondence matrix; S7: establishing a weighted correspondence relationship between the source point cloud and the target point cloud according to the correspondence matrix to obtain a rigid transformation matrix; the accuracy of the registration process is improved.

Description

Technical Field

[0001] This invention relates to the field of computer vision, specifically to a point cloud registration method and system based on geometric perception and global consistency. Background Technology

[0002] In Transformer-based point cloud registration methods, such as DCP, feature extraction primarily relies on scalar embeddings and scalar self-attention mechanisms for global context modeling. However, this scalar feature representation inherently ignores the three-dimensional geometric properties of the point cloud, such as directional vectors, local curvature, and edge structure information. Due to the lack of explicit geometric inductive bias, the network cannot establish robust shape representations when dealing with objects with large-angle rotations, geometric symmetry, or inconsistent local structures. Scalar attention mechanisms struggle to capture the geometric consistency across point clouds, leading to a significant decrease in feature discriminative power under complex transformations and limiting registration accuracy.

[0003] Existing techniques typically employ row-wise Softmax normalization to calculate the similarity matrix, generating point-to-point matching probabilities. The drawback of this matching method is that it is essentially locally normalized, guaranteeing only that each source point has a probability distribution sum in the target point cloud (i.e., the row sum is 1), but failing to constrain the number of times the target point is selected (i.e., the column sum is not 1). This leads to a severe "many-to-one" matching problem, where multiple source points may competitively match the same target point. This asymmetric matching logic not only introduces high-entropy ambiguity in the correspondence but also lacks global consistency constraints. When point clouds exhibit partial overlap, symmetrical structures, or noise interference, this ambiguity can cause significant rotational drift and registration failures.

[0004] In practical registration tasks, point clouds are often accompanied by Gaussian noise, outliers, and varying degrees of random sampling loss. Existing non-equivariant networks are extremely sensitive to spatial transformations (especially large-scale rotations). Even small changes in coordinate noise or sampling density can cause drastic fluctuations in the feature space, lacking the equivariance required to handle rigid body transformations. Although some strictly equivariant networks attempt to solve this problem, such as TFN and SE(3)-Transformer, they often have excessive computational overhead, making it difficult to meet real-time requirements.

[0005] Existing lightweight methods lack an effective geometric "hardening" mechanism in the feature enhancement stage, which leads to feature degradation and poor generalization ability when faced with unseen categories, large rotations, or low overlap scenes. Summary of the Invention

[0006] The purpose of this invention is to provide a point cloud registration method and system based on geometric perception and global consistency, which can improve the discriminative power and geometric perception capability of features in complex rotating scenes.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: A point cloud registration method based on geometric perception and global consistency includes the following steps performed sequentially: S1: Obtain the source point cloud and target point cloud to be registered; S2: Use a feature extraction network with shared weights to extract the initial high-dimensional features of the source point cloud and the target point cloud respectively; S3: Input the initial high-dimensional features into a geometric vector Transformer, which includes a vector branch and a scalar branch. The vector branch uses linear projection to map the undirected scalar features into directed vectors in a three-dimensional vector space. It explicitly captures the geometric properties of the point cloud's normal vector, curvature, and edge direction through the rotation properties of the vectors, thus obtaining geometric vector features. The scalar branch uses a standard Transformer network to extract features from the initial high-dimensional features. It performs global context modeling on the input point cloud features through the self-attention mechanism of the standard Transformer to obtain high-dimensional semantic feature embeddings with long-distance dependencies, thus obtaining scalar semantic features. S4: Utilize the gated adaptive feature fusion module to dynamically evaluate the importance of the geometric vector feature and the scalar semantic feature in the current context, and obtain the enhanced features of the source point cloud and the target point cloud respectively; S5: Calculate the inner product similarity matrix between the two enhanced features; S6: Input the inner product similarity matrix into the Sinkhorn global consistency matching network. By alternately performing row normalization and column normalization operations, the inner product similarity matrix is ​​transformed into a correspondence matrix that satisfies the double random constraint. ; S7: Based on the corresponding matrix A weighted correspondence between the source point cloud and the target point cloud is established, and the optimal rotation matrix and translation vector are solved using the weighted singular value decomposition algorithm to obtain the rigid body transformation matrix.

[0008] Preferably, the specific processing flow of the geometric vector Transformer in step S3 is as follows: S3-1: Linear projection from scalar to vector: for input features Through a learnable weight matrix and Project it into a set of three-dimensional vectors : ; in, Indicates the number of vectors generated; S3-2: Local geometric structure encoding: Reshape the three-dimensional vector U to obtain the three-dimensional geometric vector features V; S3-3: Vector-to-Scalar Reprojection: This is achieved by flattening the 3D geometric vector feature V and passing it through a linear layer and activation function. Processing yields geometric vector features. : ; in, The weight matrix is ​​a learnable linear layer. For the corresponding bias vector, This indicates a flattening operation on the three-dimensional geometric vector feature V.

[0009] Preferably, the processing flow of the gated adaptive feature fusion module in step S4 is as follows: S4-1: Feature concatenation and joint perception: The scalar semantic features extracted from the scalar branch and the geometric vector features extracted from the vector branch are concatenated along the channel dimension to form a joint feature vector; S4-2: Gated Weight Generation: A lightweight perceptron is used to process the joint feature vector to generate a gated weight vector in the range [0,1]. ; S4-3: Adaptive Weighted Fusion: Utilizing the generated gating weight vector The fusion process is dynamically controlled. After the geometric vector feature is gated and modulated, it is injected into the scalar semantic feature in the form of residual connection to generate enhanced features.

[0010] Preferably, in step S6, the specific implementation steps of the Sinkhorn global consistency matching network are as follows: S6-1: Similarity Matrix Initialization: Initialize the similarity matrix... Apply temperature coefficient Perform exponential operations to obtain the initial matrix. : ; in, Temperature coefficient; S6-2: Sinkhorn Iterative Optimization: Initial Matrix Input the Sinkhorn layer and perform multiple rounds of iterative updates. In each round of iteration, row normalization and column normalization operations are performed alternately. ; in, Ensure that the sum of the elements in each row is 1. Ensure that the sum of the elements in each column is 1. This is the intermediate matrix obtained in the k-th iteration. The final matrix obtained; S6-3: Generation and Ambiguity Removal of a Double-Random Matrix: After K iterations, the matrix converges to an approximate double-random matrix. .

[0011] A point cloud registration system includes a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the point cloud registration method based on geometry perception and global consistency as described in any of the preceding claims.

[0012] By adopting the aforementioned design scheme, the beneficial effects of the present invention are as follows: This application uses a geometric vector Transformer to introduce parallel learnable vector branching and gating fusion mechanism, projecting scalar features into direction-sensitive three-dimensional vectors, thereby effectively capturing the geometric properties of point clouds such as local curvature and edge direction, overcoming the limitation of pure scalar features ignoring spatial structure, and fundamentally improving the discriminative power and geometric perception ability of features in complex rotating scenes. This application uses the Sinkhorn global consistency matching network to replace the traditional Softmax normalization. By alternately performing row and column normalization, a correspondence matrix that satisfies the double random constraint is generated, ensuring a strict "one-to-one" matching relationship between the source point cloud and the target point cloud. This effectively eliminates many-to-one erroneous assignments, suppresses high-entropy fuzzy distributions, and improves the global consistency and accuracy of the registration process. Attached Figure Description

[0013] Figure 1 This is a flowchart of the point cloud registration method of the present invention; Figure 2 This is a flowchart of the geometric vector Transformer processing of the present invention; Figure 3 This is a flowchart of the Sinkhorn global consistency matching network of the present invention. Figure 4 This is a schematic diagram comparing the point cloud registration method of the present invention with other methods on the Bunny and Dragon datasets. Figure 5 This is a schematic diagram illustrating the point cloud registration method of the present invention performing point cloud registration on the ModelNet40 dataset. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0015] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0016] A point cloud registration method based on geometric perception and global consistency, such as Figure 1 As shown, the steps are executed sequentially as follows: S1: Obtain the source point cloud to be registered. and target point cloud Typically, the input point cloud is subjected to mean removal and normalization to ensure that it is distributed within a unit sphere, thereby eliminating the impact of scale differences on feature extraction.

[0017] S2: Use a shared-weight feature extraction network to extract initial high-dimensional features from the source and target point clouds respectively. and In this embodiment, the feature extraction network with shared weights can be DGCNN or PointNet++ based on dynamic graph convolution, or other conventional networks. This step aims to obtain the local topological structure and basic semantic information of the point cloud.

[0018] S3: Input the initial high-dimensional features into the geometric vector Transformer (GVT). The geometric vector Transformer includes a vector branch and a scalar branch. The vector branch extracts features from the initial high-dimensional features to obtain geometric vector features. The vector branch uses linear projection to map the undirected scalar features into directed vectors in the three-dimensional vector space. Through the rotation characteristics of the vector, it explicitly captures the geometric properties of the point cloud, such as the normal vector, curvature, and edge direction, thereby introducing weak SE(3) equivariance and enhancing the sensitivity of the features to rigid body transformation.

[0019] The scalar branch extracts features from the initial high-dimensional features to obtain scalar semantic features. In this embodiment, the scalar semantic features are extracted by the standard Transformer network. This process is a routine operation in the field of point cloud feature processing. Specifically, the input point cloud features are modeled globally using the self-attention mechanism of the standard Transformer to obtain high-dimensional semantic feature embeddings with long-distance dependencies.

[0020] In this embodiment, to address the problem of missing geometric inductive bias caused by existing Transformer methods using only scalar features, this application designs a method as follows: Figure 2 The diagram shows a geometric vector Transformer. This geometric vector Transformer aims to overcome the limitation of scalar features in representing directionality by explicitly introducing 3D geometric modeling, thereby improving the network's discriminative power under large-angle rotations. The specific processing flow of this geometric vector Transformer is as follows: S3-1: Linear Projection from Scalar to Vector: The core idea of ​​this geometric vector Transformer is to use linear transformations to map undirected scalar features to a three-dimensional vector space, thereby making them rotation-sensitive. Specifically, for input features... Through a learnable weight matrix and Project it into a set of three-dimensional vectors This process can be represented by the following formula: ; in, This indicates the number of vectors generated. These vectors can undergo corresponding rotational transformations as the input point cloud rotates, thus introducing weak SE(3) isovariability. This allows the network to "sense" changes in the pose of the point cloud, rather than just its shape.

[0021] S3-2: Local Geometric Structure Encoding: The 3D vector U is reshaped to obtain the 3D geometric vector features V. In this embodiment, this process is a common operation in deep learning to adjust the tensor dimension. The specific transformation process is as follows: vector Channel dimension It is split into two independent dimensions: the number of vectors. With three-dimensional space dimension 3, thus Reconstructed into three-dimensional geometric vector features V ,in, For the number of points in the point cloud, The number of geometric vectors generated for each point, where 3 represents the coordinate dimension in three-dimensional space; this three-dimensional geometric vector feature V explicitly encodes the local geometric properties of the point cloud in three-dimensional space, such as the direction of the surface normal vector, edge curvature, and the orientation of the local reference frame. This explicit geometric representation enables the network to maintain high discriminative power and stability when dealing with objects with large-angle rotations, symmetrical structures, or missing local geometry.

[0022] S3-3: Vector-to-Scalar Reprojection: To ensure compatibility with subsequent standard Transformer modules, the extracted 3D geometric vector features V are remapped back to scalar space. This is achieved by flattening the 3D geometric vector features V. And through a linear layer and activation function Processing yields geometric vector features. The activation function here is either ReLU or GELU; the formula is as follows: ; in, The weight matrix is ​​a learnable linear layer. This corresponds to the bias vector. This step compresses explicit geometric orientation information into high-dimensional semantic features, providing rich geometric context for subsequent fusion.

[0023] S4: Utilize a gated adaptive feature fusion module to dynamically evaluate the importance of the geometric vector feature and the scalar semantic feature in the current context, and obtain the enhanced features of the source point cloud and the target point cloud, respectively. and In this embodiment, a learnable gating unit generates the fusion weights of the geometric vector features and the scalar semantic features, adaptively injecting key geometric details into the global semantic features to generate enhanced features that contain both long-range dependencies and local geometric sensitivity. and .

[0024] In this embodiment, to address the issue that directly superimposing geometric and semantic features may lead to feature distribution disruption or training instability, this application designs a gated adaptive feature fusion module (GatedFusion in GVT). This gated adaptive feature fusion module can dynamically evaluate the relative importance of geometric vector features and semantic features based on the feature context of the current point, and adaptively adjust the fusion weights. The specific processing flow of this gated adaptive feature fusion module is as follows: S4-1: Feature Concatenation and Joint Perception: Scalar semantic features extracted from scalar branches Geometric vector features extracted with vector branch Concatenate the channels to form a joint feature vector. : ; This joint feature contains both global contextual information and local geometric details of the point cloud.

[0025] S4-2: Gated Weight Generation: A lightweight perceptron is used to process the joint feature vector to generate a gated weight vector in the range [0,1]. : ; in, A learnable linear layer weight matrix used to combine joint features Mapped to the feature space of the gating weights, For the corresponding learnable bias vector, The Sigmoid activation function is used to map the output value to the [0,1] interval, thereby achieving dynamic control over the proportion of geometric information retained.

[0026] In this embodiment, the lightweight perceptual network can be composed of a single linear layer and a sigmoid activation function, or it can be an MLP network composed of two linear layers, a ReLU activation function, and a sigmoid activation function.

[0027] The gating weight vector This determines how much geometric information should be retained at the current location. When the input point is located in a region with significant geometric vector features (such as sharp edges or corners), the gating weights tend to be 1; while in flat regions, the weights may be smaller.

[0028] S4-3: Adaptive Weighted Fusion: Utilizing the Generated Gated Weight Vector The fusion process is dynamically controlled. Geometric vector features are... After gating modulation, it is injected into the scalar semantic features via residual connections. In the middle. The final enhanced features. It preserves the original semantic context while organically incorporating key geometric details to enhance features. It is expressed by the following formula: ; Through the above mechanism, this module effectively avoids interference from erroneous or redundant geometric information, ensures the robustness of feature enhancement, and improves the network's adaptability under different noise levels.

[0029] S5: Calculate the enhanced features of the source point cloud Enhanced features of target point cloud Inner product similarity matrix between ,in Represents source point cloud With target point cloud The degree of similarity in the feature space is calculated using the following formula: ;in, This is a vector inner product, which is a standard calculation method in this field.

[0030] S6: Calculate the inner product similarity matrix Input as follows Figure 3 The Sinkhorn global consistency matching network shown alternatingly performs row normalization and column normalization operations to adjust the inner product similarity matrix. Transformed into a corresponding matrix satisfying the double-stochastic constraint. This process enforces a globally consistent "one-to-one" matching between the source point cloud and the target point cloud, effectively eliminating "many-to-one" ambiguity.

[0031] In this embodiment, to address the problem that traditional Softmax matching lacks global consistency and is prone to many-to-one ambiguity, this application proposes a Sinkhorn global consistency matching network. By utilizing optimal transmission theory, the similarity matrix is ​​transformed into a double random matrix, fundamentally solving the matching conflict problem.

[0032] In this embodiment, the specific implementation steps of the Sinkhorn global consistency matching network are as follows: S6-1: Inner Product Similarity Matrix Initialization: Based on the enhanced features obtained in the previous steps, calculate the inner product similarity matrix between the source point cloud and the target point cloud. To enhance numerical stability, an entropy regularization term is typically introduced before computation, specifically by adjusting the inner product similarity matrix. Apply temperature coefficient Perform exponential operations to obtain the initial matrix. : ; in, Temperature coefficient; S6-2: Sinkhorn Iterative Optimization: Initial Matrix Input the Sinkhorn layer and perform multiple rounds of iterative updates. In each round of iteration, row normalization and column normalization operations are performed alternately. ; in, Ensure that the sum of the elements in each row is 1. Ensure that the sum of the elements in each column is 1. This is the intermediate matrix obtained in the k-th iteration. The final matrix is ​​obtained; in this embodiment, the number of iterations can be set to 10-20.

[0033] S6-3: Generation and Ambiguity Removal of a Double-Random Matrix: After K iterations, the matrix converges to an approximate double-random matrix. This matrix possesses two important properties: first, all rows and columns are 1, naturally satisfying a "one-to-one" matching constraint, meaning each source point can only match one target point, and each target point can only be matched by one source point; second, it exhibits sparsity and high contrast, resulting in sharper matching relationships. This allows the network to automatically filter out ambiguous matching relationships, forcing each source point to find a unique and definite target point, thereby significantly improving registration accuracy in scenarios with partial overlap, symmetrical structures, and outliers, and effectively suppressing rotation drift.

[0034] S7: Based on the corresponding matrix A weighted correspondence between the source point cloud and the target point cloud is established, and the optimal rotation matrix is ​​solved using the weighted singular value decomposition (SVD) algorithm. Translation vector The rigid body transformation matrix is ​​obtained. .

[0035] Specific applications of the rigid body transformation matrix T include: transforming source point clouds... Rigid body transformation is performed using the rigid body transformation matrix T to obtain the target point cloud. Aligned registration point cloud Based on the aligned point cloud, it can be further used for subsequent tasks such as 3D reconstruction, point cloud stitching, model alignment, target recognition, or pose estimation.

[0036] This embodiment also provides a system for implementing the above-described point cloud registration method.

[0037] A point cloud registration system includes a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the point cloud registration method based on geometry perception and global consistency as described in any of the preceding claims.

[0038] like Figure 4 As shown in the figure, the first column represents the initial state (Source / Target), displaying the source point cloud (red) and target point cloud (blue) to be registered; the second column (a) and the third column (b) show the registration results of existing methods DCP-v2 and MDCP-v2, respectively; the fourth column (c) shows the registration result of this application; and the fifth column (d) shows the true registration result (GT). Figure 4 As can be seen from the comparison, when processing point clouds with complex shapes, symmetrical structures, or partial overlap, the registration results of the present application deviate less from the true values, and the alignment accuracy of geometric details is significantly better than that of existing methods, verifying the robustness and accuracy of the present invention in complex scenes.

[0039] Figure 5 Each row in the figure corresponds to a test sample, including three types of objects: airplanes, flower pots, and toilets. The first column is the initial state (Overlay), showing the superposition effect of the source point cloud (blue) and the target point cloud (red). The second column (Aligned(Pred)) is the registration result predicted by this application, and the rotation error and translation error are also marked. The third column (Aligned(GT)) is the actual registration result. As can be seen from the figure, the point cloud predicted by the method of this application highly overlaps with the target point cloud, and the rotation error is controlled within 1°, verifying that this application can achieve high-precision registration across multiple object categories.

[0040] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A point cloud registration method based on geometric perception and global consistency, characterized in that: The steps are as follows, performed sequentially: S1: Obtain the source point cloud and target point cloud to be registered; S2: Use a feature extraction network with shared weights to extract the initial high-dimensional features of the source point cloud and the target point cloud respectively; S3: Input the initial high-dimensional features into a geometric vector Transformer, which includes a vector branch and a scalar branch. The vector branch uses two learnable weight matrices to project the initial high-dimensional features into a 3D vector. The 3D vector is then reshaped to obtain 3D geometric vector features. These 3D geometric vector features are flattened and processed by a linear layer and activation function to be remapped back to the scalar space to obtain geometric vector features. The scalar branch uses a standard Transformer network to extract features from the initial high-dimensional features. Through the self-attention mechanism of the standard Transformer, global context modeling is performed on the input point cloud features to obtain high-dimensional semantic feature embeddings with long-distance dependencies, thus obtaining scalar semantic features. S4: Utilize the gated adaptive feature fusion module to dynamically evaluate the importance of the geometric vector feature and the scalar semantic feature in the current context, and obtain the enhanced features of the source point cloud and the target point cloud respectively; S5: Calculate the inner product similarity matrix between the two enhanced features; S6: Input the inner product similarity matrix into the Sinkhorn global consistency matching network to obtain the corresponding matrix. ; S7: Based on the corresponding matrix A weighted correspondence between the source point cloud and the target point cloud is established, and the optimal rotation matrix and translation vector are solved using the weighted singular value decomposition algorithm to obtain the rigid body transformation matrix.

2. The point cloud registration method based on geometric perception and global consistency as described in claim 1, characterized in that: The specific processing flow of the geometric vector Transformer in step S3 is as follows: S3-1: Linear projection from scalar to vector: for input features Through a learnable weight matrix and Project it into a set of three-dimensional vectors : ； in, Indicates the number of vectors generated; S3-2: Local geometric structure encoding: Reshape the three-dimensional vector U to obtain the three-dimensional geometric vector features V; S3-3: Vector-to-Scalar Reprojection: This is achieved by flattening the 3D geometric vector feature V and passing it through a linear layer and activation function. Processing yields geometric vector features. : ; in, The weight matrix is ​​a learnable linear layer. For the corresponding bias vector, This indicates a flattening operation on the three-dimensional geometric vector feature V.

3. The point cloud registration method based on geometric perception and global consistency as described in claim 2, characterized in that: The specific processing flow of the gated adaptive feature fusion module in step S4 is as follows: S4-1: Feature concatenation and joint perception: The scalar semantic features extracted from the scalar branch and the geometric vector features extracted from the vector branch are concatenated along the channel dimension to form a joint feature vector; S4-2: Gated Weight Generation: A lightweight perceptron is used to process the joint feature vector to generate a gated weight vector in the range [0,1]. ; S4-3: Adaptive Weighted Fusion: Utilizing the generated gating weight vector The fusion process is dynamically controlled. After the geometric vector feature is gated and modulated, it is injected into the scalar semantic feature in the form of residual connection to generate enhanced features.

4. The point cloud registration method based on geometric perception and global consistency as described in claim 3, characterized in that: In step S6, the specific implementation steps of the Sinkhorn global consistency matching network are as follows: S6-1: Similarity Matrix Initialization: Initialize the similarity matrix... Apply temperature coefficient Perform exponential operations to obtain the initial matrix. : ； in, Temperature coefficient; S6-2: Sinkhorn Iterative Optimization: Initial Matrix Input the Sinkhorn layer and perform multiple rounds of iterative updates. In each round of iteration, row normalization and column normalization operations are performed alternately. ； in, Ensure that the sum of the elements in each row is 1. Ensure that the sum of the elements in each column is 1. This is the intermediate matrix obtained in the k-th iteration. The final matrix obtained; S6-3: Generation and Ambiguity Removal of a Double-Random Matrix: After K iterations, the matrix converges to a double-random matrix. .

5. A point cloud registration system, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the point cloud registration method based on geometric perception and global consistency as described in any one of claims 1-4 above.

Images