A non-rigid point cloud registration method and storage medium

Abstract

The application discloses a non-rigid point cloud registration method and a storage medium, and the non-rigid point cloud registration method comprises the following steps: establishing a first neural network for generating a high-dimensional feature embedding matrix, and training the first neural network according to a first target, wherein the first target is to make the Euclidean distance between each two high-dimensional feature embeddings in the generated high-dimensional feature embedding matrix equal to the geodesic distance of the corresponding two points in the input point cloud; establishing a second neural network for generating a high-dimensional feature descriptor matrix, and training the second neural network based on the high-dimensional feature embedding matrix; inputting two point clouds to be registered into the trained first neural network and second neural network to obtain two high-dimensional feature embedding matrices and two high-dimensional feature descriptor matrices; further calculating a spectral function mapping, and then calculating a corresponding relationship mapping of the two point clouds to be registered. The application not only improves the calculation efficiency, but also has robustness in processing noise and various incompleteness.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a non-rigid point cloud registration method and storage medium. Background Technology

[0002] Estimating correspondences between non-rigidly aligned point clouds is a crucial component of many computer vision and graphics applications, including animation, robotics, and autonomous driving. Point clouds, as a primitive representation of 3D data, are popular in 3D perception but lack the intrinsic structural information of the underlying objects. This discrepancy presents a significant challenge when directly establishing correspondences between point clouds derived from deformable shapes. Furthermore, more complex deformation models are needed to describe non-rigid motion, such as the joint motion of human body shapes, compared to the well-known rigid case.

[0003] Modeling non-rigid deformations is challenging due to the complexity of the process. There are two main approaches to non-rigid point cloud matching: extrinsic and intrinsic methods. Extrinsic methods approximate complex global non-rigid deformations using a set of local rigidities and / or affine transformations, such as pointwise affine transformations, deformation maps, and deformations based on local patches. While the deformation model using extrinsic methods is intuitive and simple, it is often redundant and lacks global structure. On the other hand, intrinsic methods leverage the geometric information encoded in the eigenbase of the Laplace operator to elevate the matching problem to a high-dimensional space, where a family of isometric non-rigid deformations are well characterized. Specifically, intrinsic methods first transform the extrinsic coordinates into an alternative representation, under which shape alignment is performed. For example, the pioneering functional maps framework utilizes the eigenbase of the Laplace-Beltrami operator as a spectral embedding and, under the assumption of isometric deformation, transforms non-rigid 3D shape matching into rigid alignment of high-dimensional spectral embeddings. However, spectral embeddings are typically obtained through inefficient, non-differentiable offline eigenbase decomposition of the Laplace operator, which is represented as a polygonal mesh or point cloud, and this computation is very time-consuming. Furthermore, spectral embeddings are highly sensitive to various practical influencing factors, such as noise, incompleteness, and disconnectivity.

[0004] The above background information is provided only to aid in understanding the concept and technical solution of this invention. It does not necessarily belong to the prior art of this patent application. In the absence of clear evidence that the above information was disclosed on the filing date of this patent application, the above background information should not be used to evaluate the novelty and inventiveness of this application. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes a non-rigid point cloud registration method and storage medium, which not only improves computational efficiency but also exhibits robustness in handling noise and various incompletenesses.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention discloses a non-rigid point cloud registration method, comprising the following steps:

[0008] S1: Establish a first neural network for generating a high-dimensional feature embedding matrix, and train the first neural network according to a first objective, the first objective being to make the Euclidean distance between every two high-dimensional feature embeddings in the generated high-dimensional feature embedding matrix equal to the geodesic distance between the corresponding two points in the input point cloud.

[0009] S2: Establish a second neural network for generating a high-dimensional feature descriptor matrix, and train the second neural network based on the high-dimensional feature embedding matrix generated by the first neural network;

[0010] S3: Input the two point clouds to be registered into the trained first neural network to obtain two high-dimensional feature embedding matrices, and then input the two point clouds to be registered into the trained second neural network to obtain two high-dimensional feature descriptor matrices.

[0011] S4: Based on the two high-dimensional feature embedding matrices and two high-dimensional feature descriptor matrices obtained in step S3, obtain the spectral function mapping, and then obtain the correspondence mapping of the two point clouds to be registered based on the spectral function mapping.

[0012] Preferably, generating a high-dimensional feature embedding matrix through the first neural network includes a local feature aggregation step, wherein the local feature aggregation step includes: sampling the farthest point of the input point cloud to obtain a uniformly distributed subset, finding the neighboring points corresponding to the preset point in the subset, and assigning the local feature set of the corresponding neighboring points in the subset to the preset point.

[0013] Preferably, the first total loss function for training the first neural network in step S1 includes a first sub-loss function L. G The first sub-loss function L G The formula is:

[0014]

[0015] In the formula, i represents the i-th point cloud, and v p and v q S represents two points in a point cloud. i It is the shape of the i-th point cloud. In the i-th point cloud, v p Point and v q The Euclidean distance between the high-dimensional feature embeddings corresponding to the points, d S (v p ,v q ) represents v in the point cloud p Point and v q The geodesic distance between points.

[0016] Preferably, the first total loss function for training the first neural network in step S1 includes a second sub-loss function L. KL The second sub-loss function L KL The formula is:

[0017]

[0018] In the formula, i represents the i-th point cloud, and v p Let KL represent the points in the point cloud, and let KL represent the loss based on information divergence. The high-dimensional features of all points in the i-th point cloud are embedded into v. p The distance matrix of high-dimensional feature embedding of points. It is all points in the i-th point cloud that reach v p The geodesic distance matrix of a point.

[0019] Preferably, the first total loss function for training the first neural network in step S1 includes a third sub-loss function L. B The third sub-loss function L B The formula is:

[0020]

[0021] in, I is the identity matrix, i represents the i-th point cloud, a represents the set of the first M points of the total set obtained by sampling the farthest point of the point cloud according to odd indices, and b represents the set of the first M points of the total set obtained by sampling the farthest point of the point cloud according to even indices. M takes the value of 0.2 to 0.5 times the total number of points in the point cloud. This represents the high-dimensional feature embedding matrix corresponding to set a. This represents the high-dimensional feature embedding matrix corresponding to set b; express The result of the inverse operation, express The result of the inverse operation, Π ab Π represents the permutation matrix of points in set a to points in set b. ba This represents the permutation matrix of points in set b to points in set a.

[0022] Preferably, step S2 specifically includes the following steps:

[0023] S21: Input point cloud X and point cloud Y into the trained first neural network respectively to obtain the high-dimensional feature embedding matrix Φ. X and Φ Y ;

[0024] S22: Establish a second neural network for generating a high-dimensional feature descriptor matrix. Input point clouds X and Y into the second neural network to obtain the high-dimensional feature descriptor matrix G. X and G Y ;

[0025] S23: Calculate the spectral function C: in and Φ X and Φ Y The result of the inverse operation, A represents Y The result of the inverse operation;

[0026] S24: Calculate the mapping P between point cloud X and point cloud Y: P = softmax(-α||Φ) X C-Φ Y ||2), where α is a hyperparameter;

[0027] S25: Input the correspondence mapping into the second total loss function to train the second neural network.

[0028] Preferably, the second total loss function includes a fourth sub-loss function L. cyclic (X,Y), fourth sub-loss function L cyclic The formula for (X,Y) is:

[0029]

[0030] In the formula, |X| represents the total number of points in point cloud X, |Y| represents the total number of points in point cloud Y, and D X and D Y Let X and Y represent the geodesic distance matrices of point cloud X and point cloud Y, respectively.

[0031] Preferably, the second total loss function includes a fifth sub-loss function L. isometric (X,Y), the fifth sub-loss function L isometric The formula for (X,Y) is:

[0032]

[0033] In the formula, |X| represents the total number of points in point cloud X, |Y| represents the total number of points in point cloud Y, and D X and D Y Let X and Y represent the geodesic distance matrices of point cloud X and point cloud Y, respectively.

[0034] Preferably, the first neural network and the second neural network each use a DGCNN network as their backbone network.

[0035] In a second aspect, the present invention discloses a storage medium storing a computer program, wherein the computer program is configured to be run by a processor to perform the steps of the non-rigid point cloud registration method described in the first aspect.

[0036] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention proposes a first neural network for generating a high-dimensional feature embedding matrix, and uses the high-dimensional feature embedding matrix to replace the feature basis to train a second neural network that obtains the mapping relationship. Finally, the spectral function mapping is obtained based on the high-dimensional feature embedding matrix generated by the first neural network and the high-dimensional feature descriptor matrix generated by the second neural network, and then the correspondence mapping between the two point clouds to be registered is obtained. In this way, it is no longer necessary to calculate the feature basis, nor is it necessary to have real labels and a large amount of training data, which greatly improves the computational efficiency; moreover, it is robust in dealing with noise and various incompleteness. Attached Figure Description

[0037] Figure 1 This is a flowchart of the non-rigid point cloud registration method according to Embodiment 1 of the present invention;

[0038] Figure 2 This is a schematic diagram of the aggregation method used in the first neural network of this invention;

[0039] Figure 3 This is a schematic diagram of the training of the first neural network in this invention;

[0040] Figure 4 This is a schematic diagram of the training of the second neural network in this invention;

[0041] Figure 5 This is a comparison diagram of the geometric information presented by the method of the present invention and the method of the prior art;

[0042] Figure 6 This is a comparison chart of the qualitative results of the method of the present invention and the method of the prior art based on the noise-free instance of the FAUST_r dataset;

[0043] Figure 7 This is a comparison chart of the qualitative results of the method of this invention and the method of the prior art based on noisy instances of the SCAPE_r dataset;

[0044] Figure 8 This is a qualitative example comparison diagram of point cloud matching between the method of this invention and the method of the prior art. Detailed Implementation

[0045] The embodiments of the present invention will be described in detail below. It should be emphasized that the following description is merely exemplary and not intended to limit the scope and application of the present invention.

[0046] It should be noted that when a component is referred to as "fixed to" or "set on" another component, it can be directly on or indirectly on that other component. When a component is referred to as "connected to" another component, it can be directly connected to or indirectly connected to that other component. Furthermore, a connection can be used for both fixing and circuit / signal connectivity.

[0047] It should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", and "outer" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the present invention.

[0048] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0049] like Figure 1 As shown, Embodiment 1 of the present invention discloses a non-rigid point cloud registration method. This non-rigid point cloud registration method is based on a geodesic prior neural network and specifically includes the following steps:

[0050] S1: Establish a first neural network for generating a high-dimensional feature embedding matrix, and train the first neural network according to a first objective, which is to make the Euclidean distance between every two high-dimensional feature embeddings in the generated high-dimensional feature embedding matrix equal to the geodesic distance between the corresponding two points in the input point cloud.

[0051] The first neural network in this embodiment is an improvement upon the DGCNN (Dynamic Graph CNN for Learning on Point Clouds) network as its backbone. Since point clouds may exhibit different sampling densities under different sampling methods, and DGCNN achieves local feature aggregation through k-nearest neighbor search without knowing the density distribution, its generalization ability is significantly reduced. Therefore, this embodiment modifies DGCNN: given a point cloud X, it first samples the farthest point of X to obtain a uniformly distributed subset X. s Given another point v q In this embodiment, instead of directly searching for its k-NN (k-nearest neighbor sampling method) in X, it first finds its nearest neighbor vp in Xs, and then assigns the k-NN of vp in Xs to vq, such as... Figure 2 As shown.

[0052] use Let X represent a point cloud, and ds represent the geodesic distance function with respect to the underlying surface. It can be discretized into a dense matrix recording all pairs of geodesic distances. i In the expression, R represents the set of real numbers, n i 3 represents the number of points in the i-th point cloud, and 3 represents the three-dimensional spatial coordinates of each point in the point cloud.

[0053] The first neural network uses Specifically, this refers to a neural intrinsic embedding network (NIE) that generates a high-dimensional feature embedding matrix, where Θ B These are learnable parameters (neural network parameters). Let represent the high-dimensional feature embedding matrix of the i-th point cloud, where k is the dimension of the embedding. For shape S i Assume v p and v q If v is one of its two vertices (two points in the point cloud), then the ultimate goal of training this neural intrinsic embedding network is to make v p and v q The relative geodesic distance between them is preserved in the embedding space ||Φ i (p,:)-Φ i (q,:)||2=d S (v p v q That is, to ensure that the Euclidean distance between any two high-dimensional feature embeddings in the generated high-dimensional feature embedding matrix is ​​equal to the geodesic distance between the corresponding two points in the input point cloud. See the detailed process below. Figure 3 .

[0054] In this embodiment, the first total loss function L is used to train the first neural network. total1 The first total loss function is the first sub-loss function L. G (Θ B ), second sub-loss function L KL (Θ B ) and the third sub-loss function L B (Θ B The sum of the two is L. total1 (Θ B )=λ1L G (Θ B )+λ2L KL (Θ B )+λ3L B (Θ B ), where λ1, λ2, and λ3 are hyperparameters.

[0055] The first sub-loss function L in this embodiment G (Θ B ) is the loss function that compensates for relative geodesic errors. By adding this first sub-loss function L G (Θ B This method can preserve relatively long geodesic distances within the first neural network and maintain local distances even with limited network capacity and embedding dimensions. Specifically, the first sub-loss function L... G (Θ B The formula for ) is:

[0056]

[0057] In the formula, i represents the i-th point cloud, and v p and v q S represents two points in a point cloud. i It is the shape of the i-th point cloud. In the i-th point cloud, v p Point and v q The Euclidean distance between the high-dimensional feature embeddings corresponding to the points, d S (v p ,v q ) represents v in the point cloud p Point and v q The geodesic distance between points.

[0058] The second sub-loss function L in this embodiment KL (Θ B The loss function is based on Kullback-Leibler divergence (information divergence), and a second sub-loss function L is added. KL (Θ BThis can enhance short-range recovery from a statistical perspective, preserving local geometry and thus enabling the acquisition of more refined correspondences. Specifically, the second sub-loss function L... KL (Θ B The formula for ) is:

[0059]

[0060] In the formula, i represents the i-th point cloud, and v p Let KL represent the points in the point cloud, and let KL represent the loss based on information divergence. The high-dimensional features of all points in the i-th point cloud are embedded into v. p The distance matrix of high-dimensional feature embedding of points. It is all points in the i-th point cloud that reach v p The geodesic distance matrix of a point.

[0061] Using the first sub-loss function L mentioned above G (Θ B ) and the second sub-loss function L KL (Θ B During training, the relative geodesic error of the first neural network saturates at a high-dimensional feature dimension k=8. Therefore, further increasing the embedding dimension leads to insufficient rank, resulting in an irreversible transformation of the insufficient-rank embedding. Based on this, this embodiment adopts a self-supervised approach and further introduces a third sub-loss function L. B (Θ B ), for X i Perform farthest point sampling, selecting M vertices (M is 0.2 to 0.5 times the total number of points in the point cloud, for example, M is 2000), and then using these M vertices with odd indices to obtain the point cloud of set a. Obtain the point cloud of set b according to even index. Each set contains M / 2 vertices. The nearest neighbor search is used to calculate... and Point-by-point correspondence T between ab and T ba Write the point correspondence as a permutation matrix Π ab Π ba Then, the permutation matrix is ​​in the form of... and The corresponding high-dimensional feature embedding matrix and Used for calculating the loss function, we obtain L. B (Θ B Specifically, the third sub-loss function L B (Θ B The formula for ) is:

[0062]

[0063] in, I is the identity matrix, i represents the i-th point cloud, a represents the set of the first M points of the total set obtained by sampling the farthest point of the point cloud according to odd indices, and b represents the set of the first M points of the total set obtained by sampling the farthest point of the point cloud according to even indices. M takes the value of 0.2 to 0.5 times the total number of points in the point cloud. This represents the high-dimensional feature embedding matrix corresponding to set a. This represents the high-dimensional feature embedding matrix corresponding to set b; express The result of the inverse operation, express The result of the inverse operation, ∏ ab Π represents the permutation matrix of points in set a to points in set b. ba This represents the permutation matrix of points in set b to points in set a, ||·|| F This represents the F-norm.

[0064] S2: Establish a second neural network for generating a high-dimensional feature descriptor matrix, and train the second neural network based on the high-dimensional feature embedding matrix generated by the first neural network according to a second objective, the second objective being to obtain the mapping relationship between the two point clouds;

[0065] The second neural network in this embodiment is a Neural Intrinsic Mapping Network (NIM), which belongs to the family of deep functional mappings. However, there are two main modifications: (1) The network uses the high-dimensional feature embedding matrix generated by the first neural network in step S1 to replace the feature basis in the deep functional mapping; (2) The original structural loss in the functional mapping is removed, and a self-supervised loss defined according to the geodesic distance information is used to guide feature learning.

[0066] In short, the Neural Intrinsic Mapping Network (NIM) learns to predict a set of optimal high-dimensional feature descriptor matrices from input point clouds X and Y. and HereΘ D It is a set of learnable parameters. After learning, the mapping from Y to X encoded in the Neural Intrinsic Embedding Network (NIE) is C, and the mapping from X to Y is... Since no corresponding labels are required, in order to make full use of the geodesic distance information, this embodiment converts the function mapping C into a soft correspondence mapping and designs an unsupervised loss according to the depth cyclic mapping.

[0067] Combination Figure 4 Step S2 specifically includes the following steps:

[0068] S21: Input point clouds X and Y into the trained first neural network (neural intrinsic embedding network) respectively, and obtain the corresponding high-dimensional feature embedding matrix Φ. X and Φ Y ;

[0069] S22: Establish a second neural network (neural intrinsic mapping network) for generating the high-dimensional feature descriptor matrix. Input point clouds X and Y into the second neural network to obtain the high-dimensional feature descriptor matrix G. X and G Y ;

[0070] S23: Calculate the spectral function C: in and Φ X and Φ Y The result of the inverse operation, A represents Y The result of the inverse operation;

[0071] S24: Calculate the mapping P between point cloud X and point cloud Y: P = softmax(-α||Φ) X C-Φ Y ||2), where α is a hyperparameter;

[0072] S25: Input the correspondence mapping into the second total loss function to train the second neural network.

[0073] Specifically, given the spectral function C and the high-dimensional feature embedding matrix Φ X and Φ Y Input point clouds X and Y, soft correspondence matrix mapping P: P = softmax(-α||Φ) X C-Φ Y ||2), where α is the hyperparameter controlling the entropy of the probability distribution, and each element P jk This is the probability that the j-th point in point cloud X corresponds to the k-th point in point cloud Y; further, the inverse mapping can be calculated based on the soft correspondence matrix mapping P. Given the geodesic distance matrix D of point cloud X and point cloud Y X and D Y Calculate the second total loss function L total2 The second total loss function is the fourth sub-loss function L. cyclic (X,Y) and the fifth sub-loss function L isometric The sum of (X,Y) is L total2 =L cyclic (X,Y)+L isometric (X,Y).

[0074] Fourth sub-loss function L cyclic The formula for (X,Y) is:

[0075]

[0076] In the formula, |X| represents the total number of points in point cloud X, and |Y| represents the total number of points in point cloud Y.

[0077] Based on the fourth sub-loss function L cyclic (X,Y) increases the bijectivity of the mappings estimated by the Neural Intrinsic Mapping Network (NIM) along different directions. Therefore, this embodiment assumes that the shapes of interest are approximately equidistant from each other, and thus further considers the fifth sub-loss function L. isometric (X,Y), specifically, the fifth sub-loss function L isometric The formula for (X,Y) is:

[0078]

[0079] S3: Input the two point clouds to be registered into the trained first neural network to obtain two high-dimensional feature embedding matrices, and then input the two point clouds to be registered into the trained second neural network to obtain two high-dimensional feature descriptor matrices.

[0080] S4: Based on the two high-dimensional feature embedding matrices and two high-dimensional feature descriptor matrices obtained in step S3, obtain the spectral function mapping, and then obtain the correspondence mapping of the two point clouds to be registered based on the spectral function mapping.

[0081] The Neural Intrinsic Embedding Network (NIE) used in Embodiment 1 of this invention meets the following requirements: (1) it is aware of the intrinsic geometry of the base surface; (2) it is computationally efficient; and (3) it is robust to the numerous typical influencing factors in point clouds. Furthermore, based on the Neural Intrinsic Embedding Network (NIE), a weakly supervised learning framework of Neural Intrinsic Mapping Network (NIM) is proposed for non-rigid point cloud matching. Although closely related to the Depth Function Mapping (DFM) framework, the method of this invention replaces the spectral embedding with a trained Neural Intrinsic Embedding Network (NIE) and further learns to extract optimal features based on self-supervised loss. Finally, a weakly supervised non-rigid point cloud matching process is established, requiring only that all point clouds be rigidly aligned, and that the geodesic distance matrix of the training point clouds is accessible. Therefore, the overall process of this invention is simple and geometrically informative.

[0082] In the non-rigid point cloud registration method proposed in Embodiment 1 of this invention, a Neural Intrinsic Embedding Network (NIE) is constructed, and a weakly supervised matching network—Neural Intrinsic Mapping Network (NIM)—is proposed based on the NIE. During inference, both NIE and NIM only require approximately rigidly aligned point clouds from training, without any other structural information. Furthermore, the high-dimensional feature embeddings learned on the point cloud in the proposed NIE can be used to replace the Laplacian operator. Based on the NIM, the dependence on the true labels is eliminated. Combining these two networks achieves robustness against noise, incompleteness, and connectivity issues.

[0083] Compared with existing related technologies: Function mapping and deep function mapping both require the calculation of Laplace-Beltramm feature bases, which is very time-consuming and requires a mesh structure as input; Linear invariant embedding methods require a large amount of data for training and also require the training set to have real correspondence labels. In contrast, the method proposed in this invention does not require the calculation of feature bases, real labels, or a large amount of training data.

[0084] The following detailed description of the effects of the non-rigid point cloud registration method of Embodiment 1 of the present invention will further illustrate these effects.

[0085] In a specific implementation, a first neural network (Neural Intrinsic Embedding Network) and a second neural network (Neural Intrinsic Mapping Network) were implemented using PyTorch. Both the first and second neural networks were modified based on the implementation of the DGCNN network. The network contains three EdgeConv layers, mapping the input dimension from 3 to 64, then to 512, followed by three convolutional layers that reduce the dimension from 512 to the output dimension. The first and second neural networks use the same backbone network (a modified DGCNN) to generate high-dimensional feature embedding matrices and high-dimensional feature descriptor matrices, respectively, differing only in the output feature dimension; for example, the output high-dimensional feature embedding matrix has a dimension of 20, and the output high-dimensional feature descriptor matrix has a dimension of 40, for training.

[0086] Specifically, given a point cloud / mesh with approximately 5000 points, the Neural Intrinsic Embedding Network (NIE) takes 4.9 milliseconds to generate basis functions, which is comparable to LIE (3.0 milliseconds) and faster than computing LBO basis functions (10 milliseconds).

[0087] Furthermore, training was conducted on the FAUST_r, SCAPE_R, and SURREAL_r datasets, and the learned embeddings were evaluated. Ablation studies were provided to demonstrate the design proposed in this invention. The method of this invention outperforms even MDS (multidimensional scaling, a method for generating high-dimensional features based on the geodesic distance matrix) with an input ground truth geodesic matrix. This is because MDS, based on absolute geodesic error regression embeddings, naturally tends to preserve long-range data. From the perspective of OPT (Optical Point Adjustment), MDS20 is also outperformed by the method of this invention, demonstrating its effectiveness.

[0088] The following presents the matching results of the proposed neural intrinsic mapping and compares them with several competing baselines; it also demonstrates the robustness of the neural intrinsic embedding network and neural intrinsic mapping network of the present invention to noise and various partialities.

[0089] like Figure 5 As shown, from left to right, the geometric information displayed by Euclidean coordinates, LIE, GPS, the method of this invention (OURS), and the real label (Geodesic) is as follows: The method of this invention only uses point clouds and generates geometrically aware segmentation. It can be seen that the geometric information displayed by the method of this invention is most similar to the real label. The method of this invention can effectively and reasonably segment the various parts of the human body, which reflects that our method contains surface information of human body geometry.

[0090] like Figure 6 As shown, this is the qualitative result of noise-free instances based on the FAUST_r dataset. Each row represents one method, and from top to bottom, they are the results of CorrNet-3D, LIE, and the method of this invention (OURS). The left side shows the original point cloud (Source), and the dotted boxes indicate the mismatched parts. By comparing the color correspondence, the qualitative effect of the method of this invention is the best.

[0091] like Figure 7 As shown, this is the qualitative result of noisy instances based on the SCAPE_r dataset. Each row represents one method, and from top to bottom, they are the results of CorrNet-3D, LIE, and the method of this invention (OURS). The left side shows the original point cloud (Source), and the mismatched parts are highlighted in the dashed box. By comparing the color correspondence, the method of this invention still performs the best even when the point cloud is noisy.

[0092] like Figure 8The image shows a qualitative example of partial point cloud matching. Each row represents one method, and three examples are listed: cases with holes, cuts, and half of the point cloud. From top to bottom, the results are LIE and the method of this invention (OURS). The left side shows the original point cloud (Source), with the non-corresponding parts highlighted in the dashed boxes. By comparing the color correspondence, the method of this invention performs better in all three cases of missing information (holes, cuts, and half of the point cloud).

[0093] The Neural Intrinsic Embedding Network (NIE) proposed in this invention effectively learns intrinsic information, thus allowing for structured map encoding; moreover, the Neural Intrinsic Mapping Network (NIM) exhibits good matching performance and excellent generalization ability; furthermore, both the Neural Intrinsic Embedding Network (NIE) and the Neural Intrinsic Mapping Network (NIM) are robust to common artifacts, including noise and various localities.

[0094] Unlike existing methods, this invention does not require extensive and sensitive offline infrastructure construction (e.g., eigenvalue decomposition of the Laplacian matrix) or supervision based on correspondence labels of ground truth values. The entire process in this invention is simple and geometrically informative, and has the following advantages: (1) This method performs comparable to or even better than competing benchmark methods in almost equivalent point cloud matching, while these benchmark methods typically require more supervision and / or more structural geometric inputs; (2) This method achieves reasonable generalization performance by reducing the bias in point sampling density through tailored design; (3) This method is robust in handling noise and various incompletenesses.

[0095] Embodiment 2 of the present invention discloses a storage medium storing a computer program, wherein the computer program is configured to be run by a processor to perform the steps of the non-rigid point cloud registration method described in Embodiment 1. Optionally, the storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0096] In other embodiments, SO(3) invariant or isovariant networks can also be incorporated into the Neural Intrinsic Mapping Network (NIM) framework, which may handle external pose changes more robustly, thereby improving the accuracy and generalization performance of the matching results.

[0097] Furthermore, the aforementioned non-rigid point cloud matching method can also be applied to dynamic 3D reconstruction problems.

[0098] The background section of this invention may include background information about the problems or circumstances surrounding the invention, rather than a description of prior art by others. Therefore, the content included in the background section is not an admission of prior art by the applicant.

[0099] The above description provides a further detailed explanation of the present invention in conjunction with specific / preferred embodiments, and it should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various substitutions or modifications can be made to these described embodiments without departing from the concept of the present invention, and all such substitutions or modifications should be considered within the scope of protection of the present invention. In the description of this specification, the reference to terms such as "an embodiment," "some embodiments," "preferred embodiment," "example," "specific example," or "some examples," etc., indicates that the specific features, structures, materials, or characteristics described in connection with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Furthermore, those skilled in the art can combine and integrate different embodiments or examples and features of different embodiments or examples described in this specification without contradiction. Although the embodiments of the present invention and their advantages have been described in detail, it should be understood that various changes, substitutions, and modifications can be made herein without departing from the scope defined by the appended claims.

Claims

1. A non-rigid point cloud registration method, characterized in that, Includes the following steps: S1: Establish a first neural network for generating a high-dimensional feature embedding matrix, and train the first neural network according to a first objective, the first objective being to make the Euclidean distance between every two high-dimensional feature embeddings in the generated high-dimensional feature embedding matrix equal to the geodesic distance between the corresponding two points in the input point cloud. S2: Establish a second neural network for generating a high-dimensional feature descriptor matrix, and train the second neural network based on the high-dimensional feature embedding matrix generated by the first neural network; S3: Input the two point clouds to be registered into the trained first neural network to obtain two high-dimensional feature embedding matrices, and then input the two point clouds to be registered into the trained second neural network to obtain two high-dimensional feature descriptor matrices. S4: Based on the two high-dimensional feature embedding matrices and two high-dimensional feature descriptor matrices obtained in step S3, obtain the spectral function mapping, and then obtain the correspondence mapping of the two point clouds to be registered based on the spectral function mapping; Step S2 specifically includes the following steps: S21: Input point cloud X and point cloud Y into the trained first neural network respectively to obtain the high-dimensional feature embedding matrix Φ. X and Φ Y ; S22: Establish a second neural network for generating a high-dimensional feature descriptor matrix. Input point clouds X and Y into the second neural network to obtain the high-dimensional feature descriptor matrix G. X and G Y ; S23: Calculate the spectral function C: in and Φ X and Φ Y The result of the inverse operation, A represents Y The result of the inverse operation; S24: Calculate the mapping P between point cloud X and point cloud Y: P = softmax(-α||Φ) X C-Φ Y ||2), where α is a hyperparameter; S25: Input the correspondence mapping into the second total loss function to train the second neural network.

2. The non-rigid point cloud registration method according to claim 1, characterized in that, The generation of a high-dimensional feature embedding matrix through the first neural network includes a local feature aggregation step, wherein the local feature aggregation step includes: sampling the farthest point of the input point cloud to obtain a uniformly distributed subset, finding the neighboring points corresponding to the preset point in the subset, and assigning the local feature set of the corresponding neighboring points in the subset to the preset point.

3. The non-rigid point cloud registration method according to claim 1, characterized in that, The first total loss function for training the first neural network in step S1 includes the first sub-loss function L. G The first sub-loss function L G The formula is: In the formula, i represents the i-th point cloud, and v p and v q S represents two points in a point cloud. i It is the shape of the i-th point cloud. In the i-th point cloud, v p Point and v q The Euclidean distance between the high-dimensional feature embeddings corresponding to the points, d S (v p ,v q ) represents v in the point cloud p Point and v q The geodesic distance between points.

4. The non-rigid point cloud registration method according to claim 1, characterized in that, The first total loss function for training the first neural network in step S1 includes the second sub-loss function L. KL The second sub-loss function L KL The formula is: In the formula, i represents the i-th point cloud, and v p Let KL represent the points in the point cloud, and let KL represent the loss based on information divergence. The high-dimensional features of all points in the i-th point cloud are embedded into v. p The distance matrix of high-dimensional feature embedding of points. It is all points in the i-th point cloud that reach v p The geodesic distance matrix of a point.

5. The non-rigid point cloud registration method according to claim 1, characterized in that, The first total loss function for training the first neural network in step S1 includes the third sub-loss function L. B The third sub-loss function L B The formula is: in, I is the identity matrix, i represents the i-th point cloud, a represents the set of the first M points of the total set obtained by sampling the farthest point of the point cloud according to odd indices, and b represents the set of the first M points of the total set obtained by sampling the farthest point of the point cloud according to even indices. M takes the value of 0.2 to 0.5 times the total number of points in the point cloud. This represents the high-dimensional feature embedding matrix corresponding to set a. This represents the high-dimensional feature embedding matrix corresponding to set b; express The result of the inverse operation, express The result of the inverse operation, Π ab Π represents the permutation matrix of points in set a to points in set b. ba This represents the permutation matrix of points in set b to points in set a.

6. The non-rigid point cloud registration method according to claim 1, characterized in that, The second total loss function includes a fourth sub-loss function L cyclic (X,Y), fourth sub-loss function L cyclic The formula for (X,Y) is: In the formula, |X| represents the total number of points in point cloud X, |Y| represents the total number of points in point cloud Y, and D X and D Y Let X and Y represent the geodesic distance matrices, respectively. This is the inverse mapping calculated based on mapping P.

7. The non-rigid point cloud registration method according to claim 1, characterized in that, The second total loss function includes the fifth sub-loss function L. isometric (X,Y), the fifth sub-loss function L isometric The formula for (X,Y) is: In the formula, |X| represents the total number of points in point cloud X, |Y| represents the total number of points in point cloud Y, and D X and D Y Let X and Y represent the geodesic distance matrices, respectively. This is the inverse mapping calculated based on mapping P.

8. The non-rigid point cloud registration method according to any one of claims 1 to 7, characterized in that, The first neural network and the second neural network each use the DGCNN network as their backbone network.

9. A storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program is configured to be run by a processor to perform the steps of the non-rigid point cloud registration method according to any one of claims 1 to 8.

Images