An unsupervised end-to-end point cloud non-rigid registration method and system

Abstract

The application discloses a kind of unsupervised end-to-end point cloud non-rigid registration method, comprising the following steps: based on hierarchical extension graph convolution network, point-by-point feature extraction containing point cloud high-dimensional feature is carried out, so as to extract global feature;Calculate the point-by-point distance similarity matrix, and each row is regarded as the high-dimensional feature of point, to build point-to-point matching matrix;The global feature, high-dimensional hidden code of rearranged point cloud and the point cloud to be registered are connected, then its high-dimensional point-by-point data is sent into multilayer perception, to predict point-by-point displacement;Wherein, the training of hidden code is carried out with other parameters of network, after training is completed, when testing the non-rigid registration result of new pair of point clouds, fix other parameters except hidden code, use loss function to train hidden code.The application provides more perfect point cloud high-dimensional feature and point cloud point-by-point displacement, so that the learning of network to orthogonal matching matrix becomes more accurate, and then promotes the learning of all other parameters in network.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and 3D point cloud technology, and in particular to an unsupervised end-to-end non-rigid registration method and system for point clouds. Background Technology

[0002] With the urgent need for precise perception of indoor and outdoor environments in robotics, autonomous driving, and drones, 3D point clouds, as the most basic data structure in current 3D data, can be used to accurately describe 3D objects or environments. Currently, in the three-dimensional world, objects are divided into rigid and flexible types. Flexible objects are professionally referred to as non-rigid objects, such as linear ropes, planar clothing, and three-dimensional sponges. When robots perceive non-rigid objects in real time using 3D point clouds, if the relative position between the sensing sensor and the object changes at different time steps, the object's 3D point cloud coordinate data will change, and in most cases, this change will manifest as a non-rigid change. For rigid objects, this change in point cloud data is rigid; that is, the point cloud data of the two n×3 dimensional matrices before and after the change (n represents the number of points in the point cloud) can be rigidly registered using a set of rotation matrices R and translation matrices t. However, for a pair of point clouds of non-rigid objects, it is not possible to find a set of rotation and translation matrices. Instead, it is necessary to find the corresponding point of each point in the other point cloud. If the corresponding point is found, the coordinates of the two points can be obtained and the three-dimensional displacement of each point to the corresponding point can be realized. This process is called non-rigid registration of point clouds.

[0003] Currently, non-rigid registration of point clouds is a cutting-edge research direction, and common problems in its analysis and processing include:

[0004] The disordered nature of point clouds. Each point in a sampled point cloud can still represent the same object after its position is swapped with other points, and the same point cloud can have countless different arrangements.

[0005] The data is unstructured. Each point in a point cloud has no direct relationship with any other point. Different 3D meshes and voxels make it impossible for point cloud data to form an ordered grid, thus making it impossible to learn using conventional convolutional neural networks.

[0006] The non-rigid registration of point clouds is still at the forefront of development, with only a small amount of literature available for reference. Currently, there is no significant lack in aspects such as data annotation, evaluation metrics, and baselines. Especially in data annotation, it means that each point in the point cloud and its corresponding point in another point cloud are annotated, and an orthogonal permutation matrix P is formed. That is, for an unordered n×3-dimensional point cloud data matrix A, multiplying it by a P matrix can rearrange all rows of A to obtain a matrix and each row of which is aligned with each row of another unordered point cloud matrix B.

[0007] It should be noted that the non-rigid registration of point clouds in this article, as Figure 1 shown, refers to the non-rigid registration of a pair of point clouds with the same number of points. In this research field, there is non-rigid registration of point clouds based on traditional iterative algorithms, and its problem is the registration of "a large number of points to a small number of points", that is, how to represent the shape of an object with N points using M points (where M << N). The non-rigid registration in this article does not consider this situation.

[0008] The non-rigid registration of point clouds in the prior art has the following problems:

[0009] First, DGCNN itself as a feature extraction module cannot extract sufficiently good features.

[0010] When using EdgeConv, DGCNN only performs edge convolution on the nearest 20 points each time. After increasing the number of its layers, there will be a large amount of information reuse, which is not conducive to expanding the receptive field and also makes it difficult for the network to find the optimal solution.

[0011] Second, the learnability of the point-to-point matching matrix is not yet robust and flexible enough.

[0012] Currently, it uses a strategy of probability theory plus engineering to make the matching matrix learnable. The matrix goes through the processes of dimensionality increase, column normalization, dimensionality reduction, and row normalization, and continuously regresses to an approximate orthogonal matrix during backpropagation. The learnability of this local module can be further enhanced through strategies such as deepening the network.

[0013] Third, there is still room for further optimization of the shared displacement module.

[0014] Its current input and three-layer perceptron cannot guarantee that the rearranged point cloud generates a perfect point-by-point displacement to form the point cloud to be registered. Summary of the Invention

[0015] To address the problems of existing point cloud feature extraction modules based on DGCNN, such as limited receptive field, a large amount of repetitive information during the learning process, inaccurate learning and slow learning efficiency of point-to-point matching modules, and insufficient robustness of shared deformation modules, this invention proposes an unsupervised end-to-end non-rigid point cloud registration method and system.

[0016] The technical problem of this invention is solved by the following technical solution:

[0017] An unsupervised end-to-end non-rigid registration method for point clouds includes the following steps:

[0018] S1. Based on the Hierarchical Dilated Graph Convolutional Network (HDGCN), point-by-point features containing high-dimensional features of point clouds are extracted, thereby extracting global features.

[0019] S2. Calculate the point-to-point distance similarity matrix based on the point-to-point features, and treat each row as a high-dimensional feature of a point, thereby constructing a point-to-point matching matrix;

[0020] S3. Connect the global features of the rearranged point cloud and the point cloud to be registered, and then feed its high-dimensional point-by-point data into a multilayer perceptron to predict point-by-point displacement. Here, an implicit code mechanism is introduced to concatenate the high-dimensional implicit code to each dimension. The training of the implicit code is carried out together with other parameters of the network. After training, when testing non-rigid registration on a new point cloud, fix all parameters except the implicit code, and use an unsupervised loss function to train the model only on the implicit code, so that the model can converge quickly and predict the point cloud after non-rigid registration.

[0021] In some embodiments, in step S1, point clouds A and B are fed into a shared hierarchical dilated graph convolutional network (HDGCN) to extract n×d pointwise features F respectively. a and F b Thus, the global feature V of 1×d is obtained respectively. a and V b .

[0022] In some embodiments, in step S2, based on the point-by-point feature F a and F b Calculate the point-to-point distance similarity matrix Then, each row is increased in dimension using one-dimensional convolution, and each column is normalized. Then, each row is reduced to d dimensions using one-dimensional convolution, and then each row is normalized again to output an n×n point-to-point matching matrix P.

[0023] In some embodiments, in step S2, after outputting the row normalization result, the result is concatenated to the similarity matrix. Above, continuously repeating and connecting.

[0024] In some embodiments, in step S3, when A registers with B, after outputting the point-to-point matching matrix P, let Each row is aligned with the point cloud B; the global feature V is... b Hidden code Z ab Broadcast separately and connect together Form an n×(3+d+dim_Z) ab High-dimensional pointwise data is then fed into a shared three-layer perceptron with dimensions (3 + d + dim_Z). ab (3+d+dim_Z) ab / / 2、(3+d+dim_Z) ab / / 4; Finally, output the registered point cloud. The goal is to make it identical to point cloud B.

[0025] In some embodiments, in step S3, when B registers with A, after outputting the point-to-point matching matrix P, let Each row is aligned with the point cloud A; the global feature V is... a Hidden code Z ba Broadcast separately and connect together Form an n×(3+d+dim_Z) ba High-dimensional pointwise data is then fed into a shared three-layer perceptron with dimensions (3 + d + dim_Z). ba (3+d+dim_Z) ba / / 2、(3+d+dim_Z) ba / / 4; Finally, output the registered point cloud. The goal is to make it identical to point cloud A.

[0026] In some embodiments, in step S3, implicit codes are used to regress the nonlinear function from high-dimensional data to low-dimensional data in solving for highly nonlinear pointwise displacements.

[0027] This invention also proposes an unsupervised end-to-end point cloud non-rigid registration system, comprising:

[0028] The point cloud feature extraction module is based on the Hierarchical Dilated Graph Convolutional Network (HDGCN) to extract point-by-point features containing high-dimensional features of the point cloud, thereby extracting global features.

[0029] The point-to-point matching module calculates the point-to-point distance similarity matrix based on the point-to-point features, treating each row as a high-dimensional feature of a point, and then constructs the point-to-point matching matrix.

[0030] A shared deformation module connects the global features of the rearranged point cloud and the point cloud to be registered, and then feeds its high-dimensional point-by-point data into a multilayer perceptron to predict point-by-point displacement. An implicit code mechanism is introduced, where a high-dimensional implicit code is concatenated to each dimension. The training of the implicit code is performed together with other network parameters. After training, when testing non-rigid registration on a new point cloud, all parameters except the implicit code are fixed, and an unsupervised loss function is used to train the model only on the implicit code, enabling the model to converge quickly and predict the non-rigidly registered point cloud.

[0031] The present invention also proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described unsupervised end-to-end point cloud non-rigid registration method.

[0032] The beneficial effects of this invention compared to the prior art include:

[0033] This invention provides an unsupervised end-to-end non-rigid registration method for point clouds and designs a new unsupervised end-to-end non-rigid registration framework for point clouds. When point clouds are generally unordered, the unsupervised end-to-end model of this invention can be directly applied to more scenarios and improve the accuracy of non-rigid registration of point clouds.

[0034] Hierarchical dilated edge convolution can minimize the reuse of neighboring point information and expand the receptive field, providing more complete high-dimensional features of point clouds. In this invention, point cloud feature extraction based on hierarchical dilated graph convolutional networks extracts more accurate point-by-point features, thereby extracting more accurate global features. Experiments on non-rigid registration of point clouds have confirmed its superiority.

[0035] The engineering processing of the point-to-point matching module has been optimized. Each row of the matrix is ​​regarded as a high-dimensional feature of a point. Two learnable matrices are constructed by increasing and decreasing the dimensionality respectively. Some network parameters are added to make the matching matrix more accurate. Furthermore, the network can be deepened continuously to prevent gradient vanishing caused by increasing the number of network layers, making the network's learning of orthogonal matching matrices more accurate.

[0036] The pre-processing of the rearranged point cloud to the target point cloud was optimized. By embedding the latent code, the high-dimensional latent code was spliced ​​onto each dimension, making the point-by-point displacement more accurate. When testing non-rigid registration on a new point cloud, all parameters except the latent code were fixed, and an unsupervised loss function was used to continuously optimize only the latent code (in the next training cycle), which enabled it to converge quickly and successfully predict more accurate non-rigidly registered point clouds. This promoted the learning of all other parameters in the network, especially the parameters in the matching module.

[0037] Other beneficial effects of the embodiments of the present invention will be further described below. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of non-rigid registration of point clouds in existing technology;

[0039] Figure 2 This is a flowchart of the unsupervised end-to-end non-rigid registration method for point clouds in an embodiment of the present invention;

[0040] Figure 3 This is a flowchart of the unsupervised end-to-end learning point cloud non-rigid registration model in this embodiment of the invention;

[0041] Figure 4 This is a schematic diagram of the point-to-point matching module of the unsupervised end-to-end learning point cloud non-rigid registration model in an embodiment of the present invention;

[0042] Figure 5 This is a schematic diagram of the deformation module of the point cloud non-rigid registration model for unsupervised end-to-end learning in an embodiment of the present invention. Detailed Implementation

[0043] The present invention will be further described below with reference to the accompanying drawings and preferred embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0044] It should be noted that the directional terms such as left, right, up, down, top, and bottom used in this embodiment are only relative concepts or are based on the normal use of the product, and should not be considered as restrictive.

[0045] The following is an explanation of the technical terms used in the embodiments of this invention:

[0046] (1) Matching matrix:

[0047] The matching matrix, short for point-to-point matching matrix, is also an orthogonal permutation matrix. Define two point clouds with the same number of points; both matrices will then have a dimension of n×3. Multiplying the n×n matching matrix and its transpose on the left of the two point clouds yields two rearranged matrices, each aligned one-to-one with its corresponding source point cloud. Since point clouds are unordered, obtaining the matching matrix is ​​crucial for non-rigid point cloud registration tasks because determining the one-to-one correspondence between points allows us to calculate the displacement of all points.

[0048] (2) CorrNet3D:

[0049] CorrNet3D is the first unsupervised end-to-end non-rigid point cloud registration model. It utilizes unsupervised end-to-end methods to achieve the ability to register two point clouds facing each other. Given the unordered nature of point clouds, its unsupervised characteristics are very suitable for avoiding tedious manual annotation and outputting a considerable point-to-point correspondence matrix (hereinafter referred to as the correspondence matrix), which is a permutation matrix.

[0050] The main framework of CorrNet3D involves first extracting high-dimensional features from two point clouds, with the feature extraction module utilizing the classic DGCNN (Dynamic Point Graph Convolution). Then, the high-dimensional features of all points are fed into a correlation indicator, which outputs a correlation matrix (permutation matrix), thus aligning the point clouds. Finally, the rearranged point cloud is concatenated with the global features of the other point cloud and fed into a multilayer perceptron to transform it into the point cloud to be registered. At this point, the complete end-to-end network is established, and it can also serve as a supervised non-rigid point cloud registration model.

[0051] The core of CorrNet3D lies in its innovative design of the pairing indicator. Previous research has used Sinkhorn layers to iteratively calculate a normalized exponential function for each row and column of the pairing matrix, resulting in a biased orthogonal permutation matrix. It's important to note that a strictly orthogonal matrix has only one 1 in each row and column, with the rest being 0s; therefore, such data is non-differentiable and unlearnable in neural networks. CorrNet3D introduces a smoothing module, assuming that each row of the pairing matrix represents an independent, identically distributed event. Theoretically, to make the pairing matrix tend towards orthogonality, it's necessary to ensure that each row and column has only one prominent element. Through normalization and linear transformations, the number of maximum values ​​in each row can theoretically be controlled. Therefore, in its neural network design, CorrNet3D achieves learnability of the pairing matrix by first increasing and then decreasing dimensionality, and then normalizing the rows and columns separately.

[0052] This invention proposes a novel unsupervised end-to-end point cloud non-rigid registration model. Specifically, improvements have been made to the feature extraction module, matching module, and shared deformation module, resulting in performance metrics that outperform other models currently used in this research field.

[0053] The problems to be solved by the embodiments of the present invention are the three problems existing in the current unsupervised end-to-end point cloud non-rigid registration:

[0054] Point cloud feature extraction modules based on DGCNN suffer from limited receptive fields and a large amount of information being repeatedly used during the learning process. This invention employs a point cloud feature extraction module based on a Hierarchical Dilated Graph Convolution Network (HDGCN) to extract more accurate point-wise features, thereby extracting more accurate global features. Subsequent non-rigid point cloud registration experiments further demonstrate the superiority of this module in this field.

[0055] Point-to-point matching modules suffer from inaccurate learning and slow learning efficiency. This invention optimizes the matching module by treating each row of the matrix as a high-dimensional feature of a point, constructing two learnable matrices through dimensionality increase and dimensionality reduction. Unlike CorrNet3D, the matrix is ​​concatenated to the dimensionality-reduced matrix. This allows for the addition of network parameters to improve the accuracy of the matching matrix and enables the network to be progressively deepened, preventing gradient vanishing due to increased network layers.

[0056] The shared deformation module lacks robustness, resulting in a discrepancy between the predicted non-rigidly registered point cloud and the source point cloud. In existing network models, the rearranged point cloud and the global features of the point cloud to be registered are concatenated, and then their high-dimensional point-by-point data is fed into a multilayer perceptron to predict point-by-point displacement. This invention introduces an implicit code mechanism, concatenating a high-dimensional implicit code to each dimension. Theoretically, the data in all dimensions of the implicit code follows the same Gaussian distribution. In solving highly nonlinear point-by-point displacements, the implicit code can be used to better regress the nonlinear function from high-dimensional data to low-dimensional data. The training of the implicit code is performed concurrently with other network parameters. After training, when testing non-rigid registration on a new point cloud, all parameters except the implicit code are fixed, and an unsupervised loss function is used to continuously optimize only the implicit code (a new training iteration), enabling it to converge quickly and successfully predict a more accurate non-rigidly registered point cloud.

[0057] This invention proposes a novel unsupervised end-to-end learning model for non-rigid point cloud registration, named HCDNet3D. The functionalities of this invention are as follows:

[0058] First, hierarchical dilated edge convolution can minimize the reuse of neighboring point information and expand the receptive field, providing more complete high-dimensional features of point clouds.

[0059] Second, the engineering processing of the point-to-point matching module has been optimized, making the network's learning of orthogonal matching matrices more accurate.

[0060] Third, the pre-processing of point cloud transformation to target point cloud after rearrangement is optimized. By embedding hidden codes, point-by-point displacement is made more accurate, thereby promoting the learning of all other parameters in the network, especially the parameters in the matching module.

[0061] like Figure 2 and Figure 3 As shown in the figure, an unsupervised end-to-end point cloud non-rigid registration method proposed in this embodiment of the invention includes the following steps:

[0062] S1. Based on the Hierarchical Dilated Graph Convolutional Network (HDGCN), point-by-point features containing high-dimensional point cloud features are extracted, thereby extracting global features; the specific operation is as follows:

[0063] like Figure 3 As shown, point clouds A and B are fed into the shared point cloud feature extraction module HDGCN (Hierarchical Dilated Graph Convolutional Network) to extract n×d point-by-point features F respectively. a and F b Thus, the global feature V of 1×d is obtained respectively. a and V b .

[0064] S2. Calculate the point-to-point distance similarity matrix (here, distance refers to Euclidean distance) based on the point-to-point features, treating each row as a high-dimensional feature of a point, and thus constructing the point-to-point matching matrix; the specific operation is as follows:

[0065] like Figure 4 As shown, the point-by-point feature F a and F b The data is fed into the point-to-point matching module, which first calculates the point-to-point distance similarity matrix. Then, Conv1d (convolution on one-dimensional data) is used to increase the dimension of each row, and then each column is normalized. Conv1d is then used to reduce the dimension of each row to d dimensions, and then each row is normalized to output an n×n matching matrix P.

[0066] Note that after outputting the row normalization result, a second optimization strategy can be adopted here, which is to concatenate the result to the similarity matrix. If this strategy is used, this module can be repeatedly connected, and as the network deepens, there will be no gradient vanishing problem.

[0067] S3. Connect the global features of the rearranged point cloud and the point cloud to be registered, and then feed their high-dimensional point-by-point data into a multilayer perceptron to predict point-by-point displacement. An implicit code mechanism is introduced, where a high-dimensional implicit code is concatenated to each dimension. The training of the implicit code is performed together with the other parameters of the network. After training, when testing for non-rigid registration of a new point cloud, all parameters except the implicit code are fixed, and an unsupervised loss function is used to train the model only on the implicit code, enabling the model to converge quickly and predict the non-rigidly registered point cloud. The specific operation is as follows:

[0068] like Figure 5 As shown, after outputting the matching matrix P, when A registers with B, after outputting the point-to-point matching matrix P, let... but Each row is aligned with B. At this point, the global feature V is... b Hidden code Z ab (dimm_Z) ab (Optional) Broadcast separately and connect together Form an n×(3+d+dim_Z) ab High-dimensional pointwise data is then fed into a shared three-layer perceptron (shared deformable module), with dimensions (3 + d + dim_Z). ab (3+d+dim_Z) ab / / 2、(3+d+dim_Z) ab 4. Finally, output the registered point cloud. The goal is to make it identical to point cloud B.

[0069] When B is registered with A, after outputting the point-to-point matching matrix P, let Each row is aligned with the point cloud A; the global feature V is... a Hidden code Z ba Broadcast separately and connect together Form an n×(3+d+dim_Z) ba High-dimensional pointwise data is then fed into a shared three-layer perceptron of the shared deformation module, with dimensions (3 + d + dim_Z). ba (3+d+dim_Z) ba / / 2、(3+d+dim_Z) ba / / 4; Finally, output the registered point cloud. The goal is to make it identical to point cloud A.

[0070] It's important to note that, similar to the matching matrix approach, there's a second optimization strategy here. In this case, the model isn't strictly end-to-end, but it can output more accurate results. Specifically, during training, all parameters in the network and the implicit code are trained together. After training, if testing on a new point cloud pair is needed, all parameters except the implicit code can be fixed, and the implicit code can be retrained using an unsupervised loss function. The model will then converge quickly. After convergence, the model can be used to output the corresponding non-rigid registration result on this point cloud pair. In solving highly nonlinear point-by-point displacements, the implicit code can be used to better regress the nonlinear function from high-dimensional data to low-dimensional data.

[0071] This invention also proposes an unsupervised end-to-end point cloud non-rigid registration system, comprising:

[0072] The point cloud feature extraction module is based on the Hierarchical Dilated Graph Convolutional Network (HDGCN) to extract point-by-point features containing high-dimensional features of the point cloud, thereby extracting global features.

[0073] The point-to-point matching module calculates the point-to-point distance similarity matrix based on point-to-point features, treating each row as a high-dimensional feature of a point, and thus constructs the point-to-point matching matrix.

[0074] The shared deformation module connects the global features of the rearranged point cloud and the point cloud to be registered, and then feeds its high-dimensional point-by-point data into a multilayer perceptron to predict point-by-point displacement. An implicit code mechanism is introduced, where a high-dimensional implicit code is concatenated to each dimension. The training of the implicit code is performed together with other network parameters. After training, when testing non-rigid registration on a new point cloud, all parameters except the implicit code are fixed, and an unsupervised loss function is used to train the model only on the implicit code, enabling the model to converge quickly and predict the non-rigidly registered point cloud.

[0075] This invention also proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the aforementioned unsupervised end-to-end point cloud non-rigid registration method.

[0076] Test example:

[0077] Currently, one evaluation metric in this research field has a certain degree of reliability: the accuracy of the matching matrix, defined as:

[0078] Where P is the predicted matching matrix, P gt For the true matching matrix, The operator represents the Hadamard product of two matrices. As shown in the formula, P and P gtThe greater the similarity, the higher the Corr value. Furthermore, the tolerance distance r is defined as the furthest distance dist between two points in the point cloud. max The ratio is the fault tolerance rate, i.e., r / dist. max If the registered point cloud is within a distance r under a certain fault tolerance rate, it is considered that the point has been matched with its corresponding point. Then the corresponding element in the corresponding P matrix is ​​1. Therefore, the higher the fault tolerance rate, the higher the Corr value.

[0079] In the Surreal human point cloud dataset, 115,000 pairs were selected as non-rigid registration pairs for training. Then, the methods were tested on the Surreal dataset (test set 1) and the SHREC human dataset (test set 2) at different fault tolerance rates. The performance of each method on test set 1 is shown in Table 1, and the performance of each method on test set 2 is shown in Table 2.

[0080] Table 1

[0081]

[0082] Table 2

[0083]

[0084]

[0085] As shown in Tables 1 and 2, the unsupervised end-to-end point cloud non-rigid registration method proposed in this embodiment effectively improves the performance of point cloud non-rigid registration. Since training was performed on Surreal, the model's overall performance on the Surreal dataset is higher than that on SHREC. Tables 1 and 2 also show that the unsupervised end-to-end point cloud non-rigid registration model outperforms the current best model in terms of the accuracy (Corr) of the matching matrix, except that it slightly lags behind NrtNet at 0% fault tolerance. FlowNet3D and CorrNet3D are influential models in recent years, and this method far surpasses FlowNet3D and significantly exceeds CorrNet3D. The above experiments verify the effectiveness of the unsupervised end-to-end point cloud non-rigid registration method. It is worth noting that the above experiments did not utilize the second optimization strategy for the matching module and the second optimization strategy for the shared deformation module; if these were used, the model could be further improved.

[0086] The embodiments of the present invention have the following beneficial effects:

[0087] To promote the development of unsupervised end-to-end cloud non-rigid registration models given the scarcity of datasets in current research.

[0088] This invention does not rely on manually labeled datasets; here, manual labeling refers to point-by-point pairing relationships between two point clouds. This invention designs a novel unsupervised end-to-end non-rigid point cloud registration framework. Given that point clouds are generally unordered, the unsupervised end-to-end model can be directly applied to more scenarios. For complex 3D non-rigid objects, collecting large datasets of point cloud data with point-by-point pairing annotations is very costly; therefore, models based on unsupervised end-to-end learning are necessary.

[0089] Improve the accuracy of non-rigid registration of point clouds.

[0090] This invention optimizes various sub-modules, including the point cloud feature extraction module, the point-to-point matching module, and the shared deformation module. In each sub-module, the improvements made in this invention promote the learning of the overall model. Experimental results confirm that these improvements can further improve accuracy on non-rigid registration tasks of point clouds.

[0091] The optimization strategies for each submodule are not singular and can be adjusted based on experience.

[0092] In the point-to-point matching module and the shared deformation module, this embodiment of the invention provides a second optimization strategy, which can improve the flexibility of network deployment. Meanwhile, in the feature extraction module, HDGCN can also be configured with hyperparameters such as the sparse convolution rate, and it has the feasibility of repeated nesting. Therefore, each of the three sub-modules in this embodiment of the invention has two or more optimization strategies, which are optional and can be flexibly changed according to actual needs. However, the core network framework of each module is necessary. After the core framework is built, other optimization strategies can be selected to further improve the model's performance.

[0093] In other embodiments, for the shared deformation module, global features may not be added. Taking A-to-B registration as an example, it is only necessary to add a higher-dimensional latent code to stitch it into the rearranged point cloud. During training, all weights and latent codes are fixed. During testing, the embodiments of the present invention first perform an "inference" process on unfamiliar point cloud pairs, and use the same unsupervised loss function to quickly train the latent code. After the model converges, output testing is performed only on the test point cloud pair.

[0094] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, several equivalent substitutions or obvious modifications can be made without departing from the concept of the present invention, and all such modifications, achieving the same performance or purpose, should be considered within the scope of protection of the present invention.

Claims

1. An unsupervised end-to-end non-rigid registration method for point clouds, characterized in that, Includes the following steps: S1. Based on the Hierarchical Dilated Graph Convolutional Network (HDGCN), point-by-point features containing high-dimensional features of point clouds are extracted, thereby extracting global features. S2. Calculate the point-to-point distance similarity matrix based on the point-to-point features, and treat each row as a high-dimensional feature of a point, thereby constructing a point-to-point matching matrix; S3. The point cloud is rearranged based on the point-to-point matching matrix. The global features of the rearranged point cloud and the point cloud to be registered are concatenated. Then, its high-dimensional point-by-point data is fed into a multilayer perceptron to predict point-by-point displacement. A hidden code mechanism is introduced to concatenate the high-dimensional hidden code to each dimension. The training of the hidden code is carried out together with other parameters of the network. After training, when testing non-rigid registration of a new point cloud, the other parameters are fixed except for the hidden code. An unsupervised loss function is used to train the model only for the hidden code, so that the model can converge quickly and predict the point cloud after non-rigid registration.

2. The unsupervised end-to-end point cloud non-rigid registration method as described in claim 1, characterized in that, In step S1, n×3 point clouds A and B with the same number of points are fed into a shared hierarchical dilated graph convolutional network (HDGCN) to extract n×d point-by-point features F from point clouds A and B respectively. a and F b Thus, the global feature V of 1×d is obtained respectively. a and V b .

3. The unsupervised end-to-end point cloud non-rigid registration method as described in claim 2, characterized in that, In step S2, based on the point-by-point feature F a and F b Calculate the point-to-point distance similarity matrix Then, each row is increased in dimension using one-dimensional convolution, each column is normalized, each row is reduced to d dimensions using one-dimensional convolution, and then each row is normalized again to output an n×n point-to-point matching matrix P.

4. The unsupervised end-to-end non-rigid registration method for point clouds as described in claim 3, characterized in that, In step S2, after outputting the row normalization result, the result is concatenated to the similarity matrix. Above, continuously repeating and connecting.

5. The unsupervised end-to-end non-rigid registration method for point clouds as described in claim 3 or 4, characterized in that, In step S3, when A registers with B, after outputting the point-to-point matching matrix P, let... , Each row is aligned with the point cloud B; the global feature V is... b Hidden code Z ab Broadcast separately and connect together Form an n×(3+d+dim_Z) ab High-dimensional pointwise data, dim_Z ab Hidden code Z ab The dimensions are then fed into a shared three-layer perceptron with dimensions (3 + d + dim_Z). ab (3+d+dim_Z) ab / / 2、(3+d+dim_Z) ab / / 4; Finally, output the registered point cloud. The goal is to make it identical to point cloud B.

6. The unsupervised end-to-end non-rigid registration method for point clouds as described in claim 3 or 4, characterized in that, In step S3, when B registers with A, after outputting the point-to-point matching matrix P, let... , Each row is aligned with the point cloud A; the global feature V is... a Hidden code Z ba Broadcast separately and connect together Form an n×(3+d+dim_Z) ba High-dimensional pointwise data, dim_Z ba Hidden code Z ba The dimensions are then fed into a shared three-layer perceptron with dimensions (3 + d + dim_Z). ba (3+d+dim_Z) ba / / 2、(3+d+dim_Z) ba / / 4; Finally, output the registered point cloud. The goal is to make it identical to point cloud A.

7. The unsupervised end-to-end non-rigid registration method for point clouds as described in any one of claims 1 to 4, characterized in that, In step S3, in solving the highly nonlinear point-by-point displacement, implicit codes are used to regress the nonlinear function from high-dimensional data to low-dimensional data.

8. An unsupervised end-to-end point cloud non-rigid registration system, characterized in that, include: The point cloud feature extraction module is based on the Hierarchical Dilated Graph Convolutional Network (HDGCN) to extract point-by-point features containing high-dimensional features of the point cloud, thereby extracting global features. The point-to-point matching module calculates the point-to-point distance similarity matrix based on the point-to-point features, treating each row as a high-dimensional feature of a point, and then constructs the point-to-point matching matrix. The shared deformation module rearranges the point cloud based on a point-to-point matching matrix, connects the global features of the rearranged point cloud with the point cloud to be registered, and then feeds its high-dimensional point-by-point data into a multilayer perceptron to predict point-by-point displacement. A hidden code mechanism is introduced, concatenating a high-dimensional hidden code to each dimension. The training of the hidden code is performed together with other network parameters. After training, when testing non-rigid registration on a new point cloud, all parameters except the hidden code are fixed, and an unsupervised loss function is used to retrain only on the hidden code, enabling the model to converge quickly and predict the non-rigidly registered point cloud.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the unsupervised end-to-end point cloud non-rigid registration method as described in any one of claims 1 to 7.

Images