Cross-domain point cloud registration method based on diffusion transformation generation and graph equivariant coding

Abstract

The invention discloses a cross-domain point cloud registration method based on a diffusion model and graph equivariant, and aims to improve the generalization performance of a deep learning pre-training model among different data sets. The method comprises the following steps: firstly, constructing a proxy transformation matrix by adopting the principle of a diffusion model to adapt to different spatial distributions so as to reduce the dependence on specific data distribution; secondly, a graph isovariant coding module is designed, feature extraction is conducted on point cloud data through a graph neural network in an Euclidean space, the module does not depend on the global spatial position, noise interference introduced by a diffusion model can be relieved, and the stability and accuracy of an algorithm are improved; and finally, implementing overlapped region contrast learning, and enhancing the adaptability of the model to complex transformation. The registration algorithm provided by the invention not only has high precision, but also shows excellent robustness and generalization ability.

Description

Technical Field

[0001] The present invention relates to the technical field of processing three-dimensional point cloud data, and more specifically, to a method for cross-domain point cloud registration. The method combines a diffusion model and graph equivariant coding technology to improve the generalization of deep learning pre-training algorithms between different data sets, thereby optimizing the spatial registration accuracy and robustness of point cloud data. Background Art

[0002] Point cloud registration is one of the important tasks in 3D point cloud processing. It is an upstream task for 3D data analysis, 3D reconstruction and other tasks. It is widely used in many fields such as urban reconstruction, autonomous driving, intelligent robots, and environmental perception. At present, 3D point cloud registration has become a hot research topic in the field of computer vision. A lot of work has been devoted to improving the performance of 3D matching, and great progress has been made.

[0003] Methods such as SpinNet and Predator extract complex geometric information through deep learning frameworks, use attention mechanisms and local area matching methods to improve the algorithm's context perception, and introduce rotational invariance into the algorithm or use specific data preprocessing to improve the algorithm's registration accuracy. Methods such as PointDSC and SC2-PCR focus on improving the robustness of the algorithm, pay more attention to the spatial consistency after registration, and eliminate or reduce the influence of external points, which greatly improves the matching accuracy and efficiency.

[0004] The above methods have good performance on data sets with the same or similar data distribution, but the test results of directly generalizing the trained algorithm to other data distributions are quite different. The present invention believes that the poor generalization results are caused by the distribution offset between different data. In the real world, changes in acquisition scenes or acquisition equipment often occur, resulting in changes in data distribution, which greatly affects point cloud data acquisition operations.

[0005] During the pre-training process, a wider data distribution that conforms to the real world is generated, and then the pre-training is generalized to the new data distribution, which improves the registration results of the new data. Recent studies have noticed the problem of domain adaptation in the point cloud registration process and have made improvements in previous work. The BUFFER algorithm uses a block embedder to extract efficient and generalizable local information, so that it has strong inter-data generalization results; the SIRA-PCR algorithm adaptively adjusts the synthetic data by considering different perception mechanisms and changes in object layout, so that the algorithm can better adapt to the data distribution of real-world point clouds. However, the above methods all use deep learning algorithms to design a wider range of point cloud data spatial alignment methods, or use the method of minimizing domain differences to make the algorithm better adapt to the new data domain, and cannot dynamically adjust the point cloud data distribution.

[0006] Therefore, how to provide a cross-domain generalized point cloud registration method is an urgent problem that technicians in this field need to solve. Summary of the invention

[0007] In view of this, the present invention provides a cross-domain point cloud registration method based on diffusion transformation generation and graph equivariant coding. This method optimizes the registration accuracy of point cloud data by combining advanced diffusion models with graph equivariant technology, has the ability to efficiently process different data distributions, and effectively resists rotation changes and noise interference while maintaining the point cloud structure, thereby improving the generalization ability and robustness of the algorithm. To achieve the above purpose, the present invention adopts the following technical solutions:

[0008] A cross-domain point cloud registration method based on diffusion transform generation and graph equivariant coding, comprising:

[0009] S1: Initialize point cloud data, standardize the source point cloud and target point cloud through preprocessing steps to ensure that the data quality meets the requirements of the registration process;

[0010] S2: Using the diffusion model based on the special Lie group of Euclidean space, the source point cloud data is injected with sequence noise, and the point cloud data is gradually transformed from the original state to the state with Gaussian noise;

[0011] S3: A graph equivariant encoding module is used to process noisy point cloud data. The graph neural network is used to extract point cloud data without relying on the global spatial position, thereby enhancing the local structure representation capability of point cloud data.

[0012] S4: Perform overlapping area sample set statistics and optimize point cloud matching by constructing positive sample sets and negative sample sets; the positive sample set consists of spatially close point pairs, while the negative sample set consists of distant point pairs; this step improves the registration accuracy by minimizing the distance between positive samples and maximizing the distance between negative samples;

[0013] S5: Use the decoder to restore the point cloud data processed by the graph equivariant encoding module. This step involves converting the encoded high-dimensional point cloud data back to the geometric space of the point cloud data, ready for subsequent transformation matrix calculation;

[0014] S6: Predict and calculate the final transformation matrix through multi-layer perceptron and singular value decomposition method;

[0015] S7: Calculate the loss function and perform back propagation; Calculate the loss function value based on the difference between the predicted transformation matrix and the actual transformation matrix to evaluate the accuracy of the point cloud registration; Then, use the calculation result of the loss function to update the weights and parameters of the algorithm through back propagation to optimize the registration performance of the algorithm;

[0016] S8: End.

[0017] It can be seen from the above technical solutions that, compared with the prior art, the technical solution of the present invention, through the diffusion transformation generator and the graph equivariant coding module, not only improves the accuracy of point cloud registration, but also enhances the algorithm's adaptability to complex environmental changes. This method effectively expands the data domain and reduces the model's dependence on specific data distribution by generating a new transformation matrix and using graph neural networks to extract and process point cloud data. This method does not rely on empirical thresholds, and effectively avoids the problem of mismatching caused by improper threshold setting in traditional methods. In addition, the present invention also ensures that while retaining important structural point cloud information through an iterative optimization process, the algorithm's adaptability and generalization ability between different data sets are greatly enhanced, and can be widely used in many application scenarios such as autonomous driving, robot navigation, and urban modeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0019] Figure 1 A flow chart of generating a point cloud registration transformation matrix for the diffusion model provided by the present invention;

[0020] Figure 2 The cross-domain point cloud registration architecture diagram based on diffusion transformation generation and graph equivariant coding provided by the present invention; Figure 3 A flow chart of cross-domain point cloud registration based on diffusion transform generation and graph equivariant coding provided by the present invention;

[0021] Figure 4 A schematic diagram of graph equivariance local space vectors and hyperplanes controlling local equivariance provided by the present invention;

[0022] Figure 5 The matching result of the cross-domain point cloud registration method provided by the present invention;

[0023] Figure 6 This is the matching result of the iterative process of the cross-domain point cloud registration method provided by the present invention. DETAILED DESCRIPTION

[0024] The technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] See attached Figure 1 The embodiment of the present invention discloses a cross-domain point cloud registration method based on a diffusion model of a special Lie group in Euclidean space and graph equivariant coding, comprising the following steps:

[0026] S1: Initialize point cloud data, standardize the source point cloud and target point cloud through preprocessing steps to ensure that the data quality meets the requirements of the registration process;

[0027] S2: Using the diffusion model based on the special Lie group of Euclidean space, the source point cloud data is injected with sequence noise, and the point cloud data is gradually transformed from the original state to the state with Gaussian noise;

[0028] S3: A graph equivariant encoding module is used to process noisy point cloud data. The graph neural network is used to extract point cloud data without relying on the global spatial position, thereby enhancing the local structure representation capability of point cloud data.

[0029] S4: Perform overlapping area sample set statistics and optimize point cloud matching by constructing positive sample sets and negative sample sets; the positive sample set consists of spatially close point pairs, while the negative sample set consists of distant point pairs; this step improves the registration accuracy by minimizing the distance between positive samples and maximizing the distance between negative samples;

[0030] S5: Use the decoder to restore the point cloud data processed by the graph equivariant encoding module. This step involves converting the encoded high-dimensional point cloud data back to the geometric space of the point cloud data, ready for subsequent transformation matrix calculation;

[0031] S6: Predict and calculate the final transformation matrix through multi-layer perceptron and singular value decomposition method;

[0032] S7: Calculate the loss function and perform back propagation; Calculate the loss function value based on the difference between the predicted transformation matrix and the actual transformation matrix to evaluate the accuracy of the point cloud registration; Then, use the calculation result of the loss function to update the weights and parameters of the algorithm through back propagation to optimize the registration performance of the algorithm;

[0033] S8: End.

[0034] The method involved in the present invention is further described below.

[0035] The present invention proposes a cross-domain point cloud registration method based on diffusion and graph equivariance, and its architecture is as follows: Figure 2 As shown. The present invention is composed of two independent branches, one above and one below, to process the data streams of the source point cloud and the target point cloud respectively. First, the source point cloud is processed by a diffusion proxy generator to generate a source point cloud with Gaussian noise, thereby achieving the purpose of changing the data distribution. Compared with the original source point cloud, the spatial position of the generated source point cloud will be offset and the shape of the point cloud will be distorted, but the relative positions between the points inside the point cloud remain unchanged. Therefore, the source point cloud generated based on diffusion still meets the requirements of point cloud rigid registration. Secondly, due to the addition of noise, there is a large rotation error between the two point clouds during the registration process. Therefore, an equivariant coding structure based on a dynamic graph is added to resist rotation changes. Thirdly, by using regional contrast learning to distinguish overlapping and non-overlapping areas in the new feature representation, the two point clouds are guided to be registered and mismatches are corrected. Finally, the present invention designs a progressive diffusion sequence loss function for the diffusion transform generator to constrain the influence of diffusion on the registration results.

[0036] Since the point cloud registration task is to optimize the spatial transformation parameters, the method of directly generating the point cloud spatial coordinates is not the best choice. The present invention generates a new source point cloud indirectly by generating a new transformation matrix using a rigid body transformation method. The source point cloud generated in this way can have the position disturbance, shape distortion, and noise accumulation caused by noise, while having the characteristics of maintaining the point cloud structure. The diffusion transformation generator structure is as follows: Figure 3 shown.

[0037] In step S2, a diffusion model based on a special Lie group in Euclidean space is used to define the initial data sample as z0, and the probability distribution of the training data set sample is represented by p(z). Then, by injecting noise in a serialized manner, z0 is transformed into a completely random state z t Among them, z t It conforms to the standard Gaussian distribution. The diffusion model of this structure is divided into two parts: the forward process and the reverse process. Gaussian noise is injected through the forward process, and the initial data sample is restored by the reverse process.

[0038] In the forward process defined in step S2, n is used to represent any state, z n-1 represents the noise sample in the n-1 state, and similarly z n represents the noise sample under the n state, and the noise sample under the n state is shown in formula (1):

[0039]

[0040] Among them, ε represents the noise vector drawn from the standard Gaussian distribution, α n represents the noise attenuation coefficient, β nRepresents the noise ratio coefficient injected into state n. N represents a multivariate normal distribution with a mean of 0 and a covariance matrix of the unit matrix, which makes the generated random number vectors linearly uncorrelated and have the same variability. n and z0 represent data samples in state n and state 0, respectively.

[0041] In order to achieve a smoother transition between different states of the diffusion process, the transformation matrix generator uses the cosine strategy to calculate the diffusion rate, and the diffusion parameters are generated by the cosine strategy as shown in formula (2). In the diffusion transformation generator, the cosine strategy generation is the core of the diffusion transformation generator.

[0042]

[0043] Where N represents the total number of steps in the diffusion process, and n represents the current step number. S represents an adjustable parameter used to adjust the shape of the cosine function to finely control the change in diffusion rate. n represents the diffusion coefficient in the n state.

[0044] According to the diffusion coefficient α n Calculating the Noise Figure β n , as shown in formula (3). n It determines the proportion of noise injected in each diffusion step, simulating the uncertainty and measurement error in the real world, which is crucial for the algorithm to effectively learn and generate transformation matrices.

[0045]

[0046] Among them, α n-1 represents the diffusion coefficient of the n-1 term, and 0.999 represents β n The maximum value of .

[0047] Since the diffusion model uses a progressive degradation method, in order to ensure that α n To achieve a smooth transition, the progress of diffusion needs to be uniformly controlled according to the α in each state. n Cumulative product In the cumulative diffusion series, the diffusion coefficient is the 1-β of all the first n states s The continuous product of s is the noise coefficient determined by cosine scheduling. The scheduling strategy determines the rate and mode of the noise addition process. The first n noise coefficients are shown in formula (4).

[0048]

[0049] In the forward diffusion process, define T nrepresents the transformation matrix generated by the diffusion process, the target transformation matrix is ​​T0, and T0=T represents the real transformation of the point cloud X to Y after removing noise. The forward process uses the exponential and logarithmic calculation accumulation, which continuously simulates diffusion between algebraic space and group space. When the diffusion step is n, the transformation matrix T n The calculation is shown in formula (5).

[0050]

[0051] in, represents the cumulative noise coefficient in state n, exp(·) represents the mapping from the algebraic group to the SE(3) group, and log(·) exists as the inverse mapping of exp(·). I is the identity matrix of the same dimension as T0, It is actually the torsion vector of T0.

[0052] In addition, in order to simulate the uncertainty in the real scene, Gaussian noise is introduced based on the original diffusion transformation matrix, and noise is used to represent the generated noise, as shown in formula (6).

[0053]

[0054] Finally, in T n Gaussian noise is added to generate a transformation matrix, and a source point cloud based on diffusion noise is generated with the help of rigid transformation to participate in subsequent registration tasks.

[0055]

[0056] In order to further optimize the point cloud data with diffuse noise, a graph neural network based on Euclidean space is designed in step S3 to sample point cloud features and mine the spatial relationship and geometric properties of point cloud data. In order to ensure the rotation invariance of the point cloud registration result, an equivariant graph convolution sampling point cloud feature is constructed, which is defined as shown in formula (8). Where G represents a directed graph, V represents a vertex, and E represents an edge.

[0057] G=(V,E),v i ∈V,e ij ∈E (8)

[0058] In the sampling process of step S3, local features are first sampled using the K-nearest neighbor algorithm, p i and q i are points in the neighborhood, using m ij =(p i -p j ) / ||p i -p j || represents the vector direction between them, and n ij=(p i ×p j ) / ||p i ×p j || represents the unit normal vector between them. Then use c ij =m ij ×n ij Build p i ,q i The corresponding unit plane is Figure 4 As shown. According to p i ,q i The projection coefficient of the connection line is used to obtain the equivariant characteristics of the l-th layer based on SO(3) As shown in formula (9).

[0059]

[0060] in, is the p in the neighborhood i ,q i 's associated features, and It is globally equivariant, and Equiv is used in the following text to represent equivariant calculations.

[0061] In addition, the present invention builds a message propagation module based on the original graph model, aggregates rich point cloud information using the neighbor nodes of each node, and processes the slightly non-fixed topology between points introduced by the diffusion process to maintain the equivariant result in the Euclidean space. Then, the nodes and edges are processed by MLP to update the node and edge features in the graph, and the update process follows formula (10).

[0062]

[0063] in, Indicates the fusion of layer l nodes and e ij features and update edges, Indicates that the node features are updated using the l-layer edge features, MLP (1) 、MLP (2) Two different multilayer perceptron operations.

[0064] In order to reduce the impact of non-overlapping areas on registration, in step S4, in order to make the overlapping area features play a greater role, a method based on overlapping area contrast learning is proposed. Specifically, the source point cloud X is used to calculate the geodesic transformation, and the relationship between the source point cloud X and the target point cloud Y after the transformation is statistically analyzed. The overlap threshold τ is set according to the distance between the source point cloud and the target point cloud, and the indicator function I is used to calculate the overlap threshold τ. ovlp (x, Y) calculates the minimum distance between the real transformation and the target point cloud and the size of the overlap threshold to determine whether it is an overlapping area. ovlpWhen (x,Y)>τ, x and point cloud Y are classified as overlapping, otherwise they are considered non-overlapping. ovlp Obtain overlapping point cloud data and express.

[0065] In actual operation, step S5 uses Transform to further extract context information of the two point clouds to promote information interaction between point clouds. Step S6 predicts the transformation matrix T by singular value decomposition and multi-layer perceptron.

[0066] In the multi-layer perceptron prediction registration transformation matrix, the decoder is used to aggregate and restore the point cloud data to predict the rotation and translation matrices. The multi-layer perceptron prediction process can be expressed by formula (11), and D pcd represents the decoder mapping, X src , Y tgt Represents embedded point cloud data.

[0067] T(R,t)=D pcd (X src ,Y tgt ) (11)

[0068] By calculating the optimal rotation matrix and translation matrix, a set of source point clouds are aligned to another set of target point clouds, and they are decentralized to obtain X zero , Y zero , and construct the correlation matrix H, w i represents the weight of the i-th point, and finally the rotation matrix R and translation matrix t are obtained by singular value decomposition, as shown in formula (12).

[0069]

[0070] In step S7, two loss functions, overlap contrast loss and diffusion loss, are designed to evaluate the algorithm performance. ovlp As shown in formula (13).

[0071]

[0072] Where D(·,·) represents the distance function between point cloud data, σ p and σ n Represent the distance thresholds of positive and negative pairs, P + Represents the set of overlapping point information pairs between two point clouds, Represents the negative pair set of overlapping point information pairs of point cloud X and non-overlapping point information pairs of point cloud Y, Represents the negative pair set of overlapping point information pairs of point cloud Y and non-overlapping point information pairs of point cloud X. [x] +Represents max(0,x), ensuring that the loss is only counted when the distance exceeds the threshold.

[0073] In addition, it is hoped that each state transformation matrix T n Constrained diffusion process. The loss function in the n state is shown in formula (14), L n represents the loss in state n, T n , represents the transformation matrix under state n, T gt represents the real transformation. Then, the total loss of N in the diffusion process is calculated using equation (15). N Represents a decreasing factor used to adjust the loss contribution of each state.

[0074]

[0075]

[0076] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0077] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A cross-domain point cloud registration method based on diffusion and graph equivariance, comprising the following steps: S1: Initialize point cloud data, standardize the source point cloud and target point cloud through preprocessing steps to ensure that the data quality meets the requirements of the registration process; S2: Using the diffusion model based on the special Lie group of Euclidean space, the source point cloud data is injected with sequence noise, and the point cloud data is gradually transformed from the original state to the state with Gaussian noise; S3: A graph equivariant encoding module is used to process noisy point cloud data. The graph neural network is used to extract point cloud data without relying on the global spatial position, thereby enhancing the local structure representation capability of point cloud data. S4: Perform overlapping area sample set statistics and optimize point cloud matching by constructing positive sample sets and negative sample sets; the positive sample set consists of spatially close point pairs, while the negative sample set consists of distant point pairs; this step improves the registration accuracy by minimizing the distance between positive samples and maximizing the distance between negative samples; S5: Use the decoder to restore the point cloud data processed by the graph equivariant encoding module. This step involves converting the encoded high-dimensional point cloud data back to the geometric space of the point cloud data, ready for subsequent transformation matrix calculation; S6: Predict and calculate the final transformation matrix through multi-layer perceptron and singular value decomposition method; S7: Calculate the loss function and perform back propagation; First, calculate the loss function value based on the difference between the predicted transformation matrix and the actual transformation matrix to evaluate the accuracy of the point cloud registration; then, use the calculation result of the loss function to update the weights and parameters of the algorithm through back propagation to optimize the registration performance of the algorithm; S8: End.

2. According to claim 1, a cross-domain point cloud registration method based on diffusion and graph equivariance is characterized in that: In step S2, the graph message propagation mechanism is used to integrate local information to ensure the consistency of local and global information representation, and the rotation invariance before and after registration is guaranteed through the graph equivariant framework.

3. A cross-domain point cloud registration method based on diffusion and graph equivariance according to claim 1 or 2, characterized in that: In step S7, the overlapping region contrastive learning loss improves the accuracy of the registration and effectively generalizes to unknown data by shortening the distance between positive sample pairs in the data space and pushing the distance between negative sample pairs away.

Images