A point cloud shape completion method based on diffusion probability model

Abstract

The present invention relates to the field of robot vision and discloses a point cloud shape completion method based on a diffusion probability model, comprising: S1: randomly deleting a certain proportion of points from a complete point cloud in a data set to generate a residual point cloud p for training; S2: inputting the residual point cloud into a trained encoder to obtain a feature vector z corresponding to the residual point cloud; S3: inputting the residual point cloud into a neural network, and generating a noise point cloud based on the parameters μ and σ of the input initialized noise point cloud; S4: designing a loss function and training a Markov chain model; S5: gradually denoising each point in the noise point cloud according to the Markov chain model, and generating a completed complete point cloud when the output of the neural network no longer changes significantly. The present invention extracts features from the residual point cloud to perform inverse diffusion on the noise point cloud, completing the point cloud completion work with high accuracy and restoration.

Description

Technical Field

[0001] The present invention relates to the field of robot vision, and in particular to a point cloud shape completion method based on a diffusion probability model. Background Art

[0002] A point cloud is a computer-generated method for representing the surface shape of an object. It typically uses a three-dimensional coordinate system, discretizing the object's surface into numerous points and assigning coordinates to each point. This allows the object's three-dimensional shape to be constructed using a computer. Point cloud technology is widely used in fields such as 3D modeling and robotic navigation, enabling rapid creation of 3D models and measurement and analysis of objects. During data acquisition, 3D laser scanners are affected by the characteristics of the object being measured, the measurement method, and the environment, inevitably leading to point cloud loss. The stability of the 3D scanner during scanning also has a particular impact on the scanned point cloud. Scaffolding, mechanical structures, and the continuous rotation of the scanner inevitably cause mechanical vibration, which can affect the echo and deviation between the captured point cloud positions and the actual object being measured. However, many tasks, such as object classification and robotic grasping, rely on the complete 3D shape of an object. Therefore, 3D point cloud completion technology is a fundamental and important aspect of 3D perception. Point cloud completion involves using computer technology to complete missing parts of a point cloud model. Its goal is to predict and recover the complete 3D shape based on incomplete scanned point cloud fragments.

[0003] There are four main technical solutions for point cloud completion:

[0004] 1. Fitting-based method: Generate a complete point cloud model by fitting the point cloud to a surface or polyhedron; 2. Interpolation-based method: Fill in the missing areas in the point cloud by interpolating the point cloud data to generate a complete point cloud model; 3. Learning-based method: Train the point cloud data using a machine learning algorithm to learn the features of the point cloud and generate a complete point cloud model; 4. Optimization-based method: Define the objective function of point cloud completion and use an optimization algorithm to solve the optimal point cloud completion result.

[0005] When the scene complexity is high and the noise and sampling rate are uneven, the fitting-based method may have problems such as inaccurate fitting, oversimplified or complex fitting results; when the data noise and sampling rate are uneven, the interpolation-based method may have problems such as discontinuous interpolation results, over-smoothing or not smooth enough; when the training data quality is low, the data distribution is uneven, and the network structure is inappropriate, the learning-based method may have problems such as overfitting, underfitting, and insufficient training data; since optimization problems usually have the difficult properties of high dimensionality and non-convexity and the special properties of point cloud data, the optimization-based method may have problems such as strong dependence on the initial point cloud, high computational complexity, and difficulty in parameter adjustment. Summary of the Invention

[0006] In response to the shortcomings of the existing point cloud completion methods in the background technology, the present invention provides a point cloud shape completion method based on a diffusion probability model, which solves the problems of inaccurate fitting of the point cloud completion work in the existing technology proposed in the above background technology, such as overly simplified or complex fitting results, insufficient training data, strong dependence on the initial point cloud and high computational complexity. The present invention can generate point clouds with high precision and restore more details, thereby ensuring the quality of subsequent point cloud reconstruction, three-dimensional model reconstruction, local spatial information extraction and subsequent processing.

[0007] The present invention provides the following technical solutions: S1: randomly deleting a certain proportion of points from a complete point cloud in a data set to generate a residual point cloud p for training; S2: inputting the residual point cloud into a trained encoder to obtain a feature vector z corresponding to the residual point cloud; S3: inputting the residual point cloud into a neural network to generate a noise point cloud based on the parameters μ and σ of the input initialized noise point cloud; wherein the noise point cloud obeys a Gaussian distribution, μ represents the mean of the distribution, and σ represents the standard deviation of the distribution; S4: designing a loss function to train a Markov chain model; S5: gradually reducing the noise of each point in the noise point cloud according to the Markov chain model, and generating a complete point cloud after completion when the output of the neural network no longer changes significantly.

[0008] Preferably, the datasets are ModelNet40 and ShapeNetPart.

[0009] Preferably, the point cloud shape completion method based on the diffusion probability model, S2 includes:

[0010] The residual defect cloud is input into the trained encoder φ, which maps the residual defect cloud from the high-dimensional sample space to the low-dimensional feature space. One residual defect cloud generates a one-dimensional feature vector z. The mathematical expression of the feature extraction by encoder φ is as follows:

[0011] z=φ(p),

[0012] Among them, z is a one-dimensional feature vector, φ is the encoder, and p is the residual defect cloud.

[0013] Preferably, the point cloud shape completion method based on the diffusion probability model, S3 includes:

[0014] Random sampling is performed from the Gaussian distributed point cloud to generate a random initial noise point cloud, and the neural network δ is used to learn how to obtain the parameters μ and σ of the noise point cloud through the residual point cloud.

[0015]

[0016] Where μ is the mean of the distribution, σ is the standard deviation of the distribution, δ is the neural network, and p is the residual defect cloud.

[0017] Preferably, the point cloud shape completion method based on the diffusion probability model, S4 includes:

[0018] Chamfer Distance Loss and Hausdorff Distance Loss are used as loss functions for model training to limit the distance between the original complete point cloud and the completed point cloud.

[0019] Preferably, the process of gradually generating the completed complete point cloud based on the obtained features of the residual point cloud and the noise point cloud is an inverse diffusion process, and the Markov chain model is an inverse diffusion process;

[0020] Preferably, the mathematical expression of the chamfer distance loss is as follows:

[0021]

[0022] Among them, X and Y are two point cloud sets, x is any point in point cloud set X, and y is any point in point cloud set Y. Represents the distance from point x to the nearest point in point set Y, It represents the distance from point y to the nearest point in point set X;

[0023] The mathematical expression of the Hausdorff distance is as follows:

[0024]

[0025] Among them, X and Y are two point cloud sets, x and y are any point in the point cloud sets X and Y respectively. Represents the distance from point x to the nearest point in point set Y, It represents the maximum distance between all points in point set X and the nearest point in point set Y. Represents the distance from point y to the nearest point in point set X, It represents the maximum distance between all points in point set Y and the nearest point in point set X.

[0026] Preferably, the point cloud shape completion method based on the diffusion probability model, S5 includes:

[0027] The completed point cloud is gradually generated based on the characteristics of the obtained residual point cloud and the noise point cloud. This process is an inverse diffusion process. The whole process is represented by the Markov chain model. Each point x in the noise point cloud is denoised by the following transition probability:

[0028]

[0029] Among them, t represents the number of steps of inverse diffusion, Xi represents each point in the point cloud, and x i (t) represents the point cloud at the t-th step of inverse diffusion, θ is the parameter learned from the neural network, and z is the eigenvector of the residual point cloud.

[0030] The present invention has the following beneficial effects:

[0031] 1. This method proposes a method to complete the incomplete point cloud through a diffusion probability model. By extracting features from the incomplete point cloud, the noise point cloud is inversely diffused, completing the point cloud completion with higher accuracy and restoration.

[0032] 2. Compared with existing shape completion work for three-dimensional point clouds, the point cloud generated by the completion method of the present invention is highly accurate and restores more details, thereby ensuring the quality of subsequent point cloud reconstruction, three-dimensional model reconstruction, local spatial information extraction and subsequent processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a working framework diagram of the present invention. DETAILED DESCRIPTION

[0034] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0035] The core of the point cloud shape completion method based on the diffusion probability model is inspired by the diffusion process in non-equilibrium thermodynamics, and regards the points in the point cloud as particles in a thermodynamic system in contact with a heat reservoir, which diffuses from the original distribution to a noise distribution. Therefore, the point cloud generation and completion process is equivalent to a reverse diffusion process that converts the noise distribution into the required distribution. This reverse process is modeled as a Markov chain conditioned on a certain implicit shape. When the present invention introduces noise into the point cloud, the point cloud gradually changes from a complex distribution related to the shape to Gaussian distributed noise. This process is called a forward diffusion process. The shape completion model of the point cloud can be regarded as a reverse diffusion process. Through a trained deep neural network, some shape features in the residual point cloud are extracted and learned, and the extracted features are used to gradually reduce the noise of a noisy point cloud sampled from a Gaussian distribution until the point cloud restores its complete shape.

[0036] See also Figure 1 The present invention provides the following technical solution: a point cloud shape completion method based on a diffusion probability model, comprising:

[0037] S1: Randomly delete a certain proportion (e.g., 40%) of the points in the complete point cloud in the dataset to generate a residual point cloud p for training; the dataset is ModelNet40 and ShapeNetPart;

[0038] S2: Input the residual defect cloud into the trained encoder to obtain the feature vector z corresponding to the residual defect cloud;

[0039] Specifically, the point cloud shape completion method based on the diffusion probability model, S2 includes:

[0040] The residual defect cloud is input into the trained encoder φ, which maps the residual defect cloud from the high-dimensional sample space to the low-dimensional feature space. One residual defect cloud generates a one-dimensional feature vector z. The mathematical expression of the feature extraction by encoder φ is as follows:

[0041] z=φ(p),

[0042] Among them, z is a one-dimensional feature vector, φ is the encoder, and p is the residual defect cloud.

[0043] S3: Input the residual point cloud into the neural network and generate a noise point cloud based on the parameters μ and σ of the input initialized noise point cloud; where the noise point cloud obeys a Gaussian distribution, μ represents the mean of the distribution, and σ represents the standard deviation of the distribution;

[0044] Specifically, a random initial noise point cloud is generated by random sampling from the Gaussian distributed point cloud. The neural network δ is used to learn how to obtain the parameters μ and σ of the noise point cloud through the residual point cloud. The mathematical expression of the learning process is as follows:

[0045] μ,σ=δ(p),

[0046] Where μ is the mean of the distribution, σ is the standard deviation of the distribution, δ is the neural network, and p is the residual defect cloud.

[0047] S4: Design loss function and train the Markov chain model;

[0048] Specifically, the model training mainly uses Chamfer Distance Loss and Hausdorff Distance Loss as the training loss function, mainly to limit the distance between the original complete point cloud and the completed point cloud;

[0049] Specifically, the process of gradually generating the completed point cloud based on the characteristics of the obtained residual point cloud and the noise point cloud is an inverse diffusion process, and the Markov chain model is an inverse diffusion process;

[0050] Specifically, the mathematical expression of the chamfer distance loss is as follows:

[0051]

[0052] Among them, X and Y are two point cloud sets, x is any point in point cloud set X, and y is any point in point cloud set Y. Represents the distance from point x to the nearest point in point set Y, It represents the distance from point y to the nearest point in point set X.

[0053] The mathematical expression of the Hausdorff distance is as follows:

[0054]

[0055] Among them, X and Y are two point cloud sets, x and y are any point in the point cloud sets X and Y respectively. Represents the distance from point x to the nearest point in point set Y, It represents the maximum distance between all points in point set X and the nearest point in point set Y. Represents the distance from point y to the nearest point in point set X, It represents the maximum distance between all points in point set Y and the nearest point in point set X.

[0056] S5: Each point in the noisy point cloud is gradually de-noised using the Markov chain model. After a sufficient number of steps, a complete point cloud is generated. When the Hausdorff distance between the point cloud generated in this round and the point cloud generated in the previous round is less than 0.0001, it can be considered that the output of the neural network no longer changes significantly, and a complete point cloud can be generated.

[0057] Specifically, each point x in the noisy point cloud is denoised by the following transition probability:

[0058]

[0059] Among them, t represents the number of steps of inverse diffusion, Xi represents each point in the point cloud, and x i (t) represents the point cloud at the t-th step of inverse diffusion, θ is the parameter learned from the neural network, and z is the eigenvector of the residual point cloud.

[0060] The present invention is a point cloud shape completion method based on a diffusion probability model. Inspired by the diffusion process in non-equilibrium thermodynamics, the entire model is modeled based on a Markov chain, and forward diffusion is performed by adding noise to the original point cloud to train the network model. After the training is completed, the residual point cloud is input into the trained encoder and neural network to obtain the feature vector of the point cloud and the initial parameters of the noise point cloud, and then the completed point cloud is generated through the inverse diffusion process. The present invention completes the residual point cloud through a diffusion probability model, that is, the encoder extracts the features of the point cloud and learns how to complete the point cloud through forward diffusion, and the noise point cloud is restored to a complete point cloud through reverse diffusion, thereby ensuring the high precision of the generated point cloud and restoring more details.

[0061] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A point cloud shape completion method based on a diffusion probability model, comprising: S1: Randomly delete a certain proportion of points from the complete point cloud in the dataset to generate a residual point cloud p for training; S2: Input the residual defect cloud into the trained encoder to obtain the feature vector z corresponding to the residual defect cloud; S3: Input the residual point cloud into the neural network and generate a noise point cloud based on the parameters μ and σ of the input initialized noise point cloud; where the noise point cloud obeys a Gaussian distribution, μ represents the mean of the distribution, and σ represents the standard deviation of the distribution; S4: Design loss functions to train the Markov chain model; including: using chamfer distance loss and Hausdorff distance loss as loss functions for model training to limit the distance between the original complete point cloud and the completed point cloud; The mathematical expression of the chamfer distance loss is as follows: ， Among them, X and There are two point cloud sets, x is any point of point cloud set X, y is any point of point cloud set Y Any point, Represents the distance from point x to the nearest point in point set Y, It represents the distance from point y to the nearest point in point set X; The mathematical expression of the Hausdorff distance loss is as follows: ， Among them, X and Y are two point cloud sets, x and y are any point in the point cloud sets X and Y respectively. Represents the distance from point x to the nearest point in point set Y, Represents the maximum distance between all points in point set X and the nearest point in point set Y; similarly, Represents the distance from point y to the nearest point in point set X, Represents the maximum distance between all points in point set Y and the nearest point in point set X; S5: Each point in the noisy point cloud is gradually de-noised according to the Markov chain model. When the output of the neural network no longer changes significantly, a complete point cloud is generated.

2. The point cloud shape completion method based on the diffusion probability model according to claim 1, characterized in that: The datasets are ModelNet40 and ShapeNetPart.

3. The point cloud shape completion method based on the diffusion probability model according to claim 1, characterized in that S2 include: Feed the residual defect cloud into the trained encoder , encoder The residual defect cloud is mapped from the high-dimensional sample space to the low-dimensional feature space, and a residual defect cloud corresponds to a one-dimensional feature vector z; encoder The mathematical expression for feature extraction is as follows: ， Among them, z is a one-dimensional eigenvector, is the encoder, and p is the residual defect cloud.

4. The point cloud shape completion method based on the diffusion probability model according to claim 1, characterized in that: S3 includes: Random sampling is performed from the Gaussian distributed point cloud to generate a random initial noise point cloud, and the neural network δ is used to learn how to obtain the parameters μ and σ of the noise point cloud through the residual point cloud. The mathematical expression of the learning process is as follows: ， in, is the mean of the distribution, is the standard deviation of the distribution, is a neural network, and p is a residual defect cloud.

5. The point cloud shape completion method based on a diffusion probability model according to claim 1, characterized in that: The process of gradually generating the completed complete point cloud based on the characteristics of the obtained residual point cloud and the noise point cloud is an inverse diffusion process, and the Markov chain model is an inverse diffusion process.

6. The method for point cloud shape completion based on a diffusion probability model according to claim 1, wherein S5 include: Each point x in the noisy point cloud is denoised by the following transition probability: ， Where t represents the number of steps of inverse diffusion, Xi represents each point in the point cloud, represents the point cloud at the t-th step of inverse diffusion, θ is the parameter learned from the neural network, and z is the eigenvector of the residual point cloud.

Images