Density prior guided unsupervised point cloud depth denoising method

Abstract

The application discloses a density prior guided unsupervised point cloud depth denoising method, and has the following advantages compared with the prior art: firstly, a density prior is designed, and the probability of each point being located on a real underlying surface can be described through the distribution of a noise point cloud; when discussing the conditional probability distribution of a clean point, not only the distance between a neighborhood point and the current point is considered, but also the probability of the neighborhood point being located on the underlying sampling surface is considered, thereby effectively avoiding the problems existing in the prior art, improving the quality of the predicted true value, and reducing the influence of outliers; secondly, multi-scale features of points in the noise point cloud are extracted, a score of the points is learned through an MLP, and low-noise points are obtained; the low-noise points are pre-filtered; finally, the pre-filtered point set is up-sampled to obtain a denoised point cloud, and the denoised point cloud is constrained by the density prior, thereby effectively improving the unsupervised point cloud denoising quality.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of point cloud denoising in computer graphics, and particularly relates to a density prior guided unsupervised point cloud depth denoising method. BACKGROUND

[0002] The point cloud obtained by a three-dimensional scanner or through three-dimensional reconstruction of two-dimensional images usually has a large amount of noise and outliers due to interference of external factors (such as light, vibration, noise, etc.) and limitations of equipment and algorithms, which directly affects the quality of subsequent processing of the point cloud data. Therefore, point cloud denoising is crucial to improving the performance of various tasks.

[0003] Traditional point cloud denoising methods usually approximate the point cloud as a smooth surface, and then project the points in the noisy point cloud onto the fitted surface to obtain a clean point cloud. This method works well at low noise levels, but when the noise is large or the edges are too sharp, it will appear to be over-smoothed. With the advent of point-based deep neural networks, supervised point cloud denoising methods based on deep learning have achieved leading results in many tasks, but their effectiveness is due to the architecture of deep learning and a large number of noisy and clean point pairs available for training.

[0004] Existing unsupervised point cloud denoising algorithms based on deep learning do not require paired noisy points and clean data in training. Typical methods such as Total Denoising describe the conditional probability of a clean point given a noisy point as a Gaussian noise centered on the current noisy point, guiding the noisy point to converge to the underlying surface. However, the conditional probability distribution of the clean point given by this method is relatively simple and only related to the position of the current noisy point, ignoring the overall distribution of the point cloud, so the quality of the predicted true value is low, and the network framework is sensitive to outliers, which can easily produce over-smoothing, and the denoising effect is not ideal. DMR (Differentiable Manifold Reconstruction) filters out high-noise points by sampling low-noise points through a differentiable pool, thereby reducing the influence of outliers on the results. However, the unsupervised learning version of this method provides the same conditional probability distribution of the clean point given the noisy point and loss function as Total Denoising, only considering the position of the current point with respect to the neighborhood points, without considering the probability of the point being on the underlying surface. SUMMARY

[0005] The present application is to solve the above technical problems existing in the prior art, and provides a density prior guided unsupervised point cloud depth denoising method.

[0006] The technical solution of the present application is: a density prior guided unsupervised point cloud depth denoising method, which is performed according to the following steps:

[0007] Step 1. Input a point cloud P containing noise, the number of points in the point cloud P is N;

[0008] Step 2. Based on the noise points p in the noise point cloud P, the probability distribution of the points located on the real underlying surface is constructed to construct a density prior; i

[0009] Step 3. Initialize the dynamic graph convolution network, construct the score network, the pre-filtering network and the up-sampling network using the multi-layer perception and initialize;

[0010] Step 4. Extract the multi-scale features of each point in the noise point cloud P using the dynamic graph convolution network to form a feature matrix

[0011] Step 5. Learn the multi-scale feature matrix through the score network to obtain a point score vector s, retain M points with higher scores, M=N / 2, and obtain a low-noise point set

[0012] Step 6. Pre-filter the low-noise point set through the pre-filtering network;

[0013] Step 7. Upsample the pre-filtered low-noise point set through the up-sampling network to generate a denoised point cloud;

[0014] Step 8. Determine whether the denoised point cloud meets the requirements, if yes, end; if not, proceed to Step 9;

[0015] Step 9. Randomly select a neighborhood to construct an unsupervised loss function and optimize the dynamic graph convolution network, the score network, the pre-filtering network and the up-sampling network, and return to Step 4.

[0016] The step 2 is to construct a density prior according to formula (1);

[0017]

[0018] The is the neighborhood of p i , containing k1 nearest neighbor points; the sigma is a hyperparameter, the value of sigma is predicted by the average distance between points, calculated according to formula (2);

[0019]

[0020] The |·| is the number of elements contained in the set, is the neighborhood composed of k2 nearest neighbor points of p i , sigma k is a set parameter;

[0021] The feature matrix ​is constructed according to formula (3):

[0022]

[0023] The is the feature of the input of the l+1 layer, is the point p i The output feature of the l layer, H θ is a multi-layer perception parameterized by θ, denotes the neighborhood of the point p i , and max denotes the maximum pooling;

[0024] The step 5 is performed according to formula (4), (5):

[0025] s = MLP (X) (4)

[0026] S = [p i1 ,p i2 ,…,p iM ], p ij ∈P, ij∈arg top M (s) (5)

[0027] The MLP is a multi-layer perception, and arg top M (s) is the subscript corresponding to the M maximum values in s;

[0028] The step 6 is performed according to formula (6), (7), (8):

[0029] Y = X s ⊙sigmoid(s s ·1 1×F ) (6)

[0030] ΔS = MLP (Y) (7)

[0031]

[0032] The is a feature matrix of the low-noise point set S, is a score vector of the points in S, ⊙ denotes element-wise multiplication, and · denotes matrix multiplication;

[0033] The step 7 is specifically as follows:

[0034] Two two-dimensional vectors (u i1 ,v i1 ) are randomly generated, (u i2 ,v i2 ) ∈ [-1, 1] 2 , the two-dimensional vector is concatenated with the feature vector y i ∈Y to obtain [ui1 ,v i1 ,y i ], [u i2 ,v i2 ,y i ], using [u i1 ,v i1 ,y i ], [u i2 ,v i2 ,y i ]right The above data was collected, and a noise-reduced point cloud was generated according to the formula (9).

[0035]

[0036] Step 9 is as follows:

[0037] Step 9.1 Construct constraints for each point by randomly selecting a neighborhood each time according to formulas (10) and (11);

[0038]

[0039]

[0040] The ω j λ is the weight, and λ is the normalization coefficient. For p i A randomly selected neighborhood;

[0041] Step 9.2 By minimizing energy, construct the bijection f between the noisy point cloud and the denoised point cloud, the bijection f being calculated according to formula (12);

[0042]

[0043] The For noise reduction of point clouds;

[0044] Step 9.3 Minimize the energy function L as shown in Equation (13) U Optimize the dynamic graph convolutional network, scoring network, pre-filtering network, and upsampling network;

[0045]

[0046] Compared with existing technologies, this invention has the following advantages: First, a density prior is designed, which can characterize the probability of each point being located on the true underlying surface by the distribution of the noisy point cloud. When discussing the conditional probability distribution of clean points, not only the distance between the current point and the neighboring points is considered, but also the probability of the neighboring points being located on the underlying sampled surface. This effectively avoids the problems existing in existing methods, improves the quality of the predicted true value, and reduces the influence of outliers. Second, multi-scale features are extracted from the points in the noisy point cloud. The scores of the points are learned through MLP to obtain low-noise points, which are then pre-filtered. Finally, the pre-filtered point set is sampled to obtain the denoised point cloud. The density prior is used to constrain the denoised point cloud, which effectively improves the quality of unsupervised point cloud denoising. Attached Figure Description

[0047] Figure 1 This is a flowchart illustrating an embodiment of the present invention.

[0048] Figure 2 This is a comparison chart of the noise reduction effects of the embodiments of the present invention and the prior art. Detailed Implementation

[0049] This invention provides a density-prior-guided unsupervised point cloud depth denoising method, such as... Figure 1 As shown, proceed as follows:

[0050] Step 1. Input a noisy point cloud P, where the number of points in point cloud P is N;

[0051] Step 2. Based on the noise point p in the noise point cloud P i The probability distribution located on the true underlying surface is used to construct a density prior, as follows:

[0052] Construct a density prior according to formula (1);

[0053]

[0054] The For p i The neighborhood contains k1 = 180 nearest neighbor points; σ is a hyperparameter, and the value of σ is predicted by the average distance between points and calculated according to formula (2);

[0055]

[0056] The |·| represents the number of elements contained in the set. For p i The neighborhood formed by k2 = 7 nearest neighbors, σ k To set parameters, σ k =0.65;

[0057] Step 3. Initialize the dynamic graph convolution network, construct the score network, pre-filter network and up-sampling network using the multi-layer perception and initialize;

[0058] Step 4. Extract the multi-scale features of each point of the noisy point cloud P using the dynamic graph convolution network to form a feature matrix

[0059] The feature matrix is constructed according to formula (3):

[0060]

[0061] The input feature of the l+1 layer is is the output feature of the l layer, i is the multi-layer perception parameterized by θ, θ denotes the neighborhood of point p i , and max denotes the maximum pooling;

[0062] Step 5. Learn the multi-scale feature matrix through the score network to obtain a point score vector s, retain the M points with higher scores, M=N / 2, and obtain a low-noise point set

[0063] Specifically, formula (4) and (5) are used for calculation:

[0064] s=MLP(X) (4)

[0065] S=[p i1 ,p i2 ,…,p iM ],p ij ∈P,ij∈arg top M (s) (5)

[0066] The MLP is a multi-layer perception, and arg top M (s) is the subscript corresponding to the M maximum values in s;

[0067] Step 6. Pre-filter the low-noise point set through the pre-filter network:

[0068] Specifically, formula (6), (7) and (8) are used:

[0069] Y=X s ⊙sigmoid(s s ·1 1×F ) (6)

[0070] ΔS=MLP(Y) (7) ​​

[0071]

[0072] The is a feature matrix of the low-noise point set S, is a score vector of the points in S, and represents element-wise multiplication, and represents matrix multiplication;

[0073] Step 7. Up-sampling the pre-filtered low-noise point set through an up-sampling network to generate a denoised point cloud, specifically as follows:

[0074] Randomly generate two two-dimensional vectors (u i1 ,v i1 ) and (u i2 ,v i2 ) ∈ [-1, 1] 2 , concatenate the two-dimensional vectors with the feature vector y i ∈ Y to obtain [u i1 ,v i1 ,y i ] and [u i2 ,v i2 ,y i ], and up-sample using [u i1 ,v i1 ,y i ] and [u i2 ,v i2 ,y i ] to generate a denoised point cloud

[0075]

[0076] Step 8. Determine whether the denoised point cloud meets the requirements. If yes, end; otherwise, proceed to step 9.

[0077] Step 9. Randomly select a neighborhood to construct an unsupervised loss function and optimize the dynamic graph convolution network, the score network, the pre-filtering network, and the up-sampling network, and return to step 4.

[0078] The step 9 is specifically as follows:

[0079] Step 9.1. Construct the constraint of each point according to formulas (10) and (11) by randomly selecting a neighborhood each time.

[0080]

[0081]

[0082] The ω j is a weight, λ is a normalization coefficient, is p irandomly selected neighborhood;

[0083] Step 9.2 constructing a bijection f between the noisy point cloud and the denoised point cloud by minimizing the energy, the bijection f being calculated according to formula (12);

[0084]

[0085] The for the denoised point cloud;

[0086] Step 9.3 constructing a bijection f between the noisy point cloud and the denoised point cloud by minimizing the energy function L U optimizing the dynamic graph convolution network, the scoring network, the pre-filtering network and the up-sampling network;

[0087]

[0088] The denoising result graphs and quantitative comparisons of the embodiments of the present application and other methods (bilateral filtering, average filtering, Total Denoising, DMR) are shown in Figure 2 and Table 1.

[0089] For the test data, the present application collects 20 different classes from the ModelNet-40 database, each class containing 3 different meshes, and generates point clouds by randomly sampling 20,000 and 50,000 points. In the case of a given clean model, the corresponding noisy model is synthesized by adding central Gaussian noise, and the standard deviation is defined as 1%, 2%, 2.5% and 3% of the diagonal of the original model bounding box, finally generating a test database containing 480 models.

[0090] Figure 2 are the visualization results of four denoising algorithms under different noise scales, and the noise intensity is 2%, 2.5% and 3% of the diagonal of the original model bounding box. The distance of each point to the true underlying surface is distinguished by color, and the points with closer distance are rendered as gray, otherwise as black. a~e correspond to the original noisy point cloud input, bilateral filtering, average filtering, DMR and the present application respectively.

[0091] Table 1

[0092]

[0093]

[0094] The comparison results show that the denoising quality of the method of the present application is better than that of other methods.

Claims

1. A density prior guided unsupervised point cloud depth denoising method, characterized in that According to the following steps: Step 1. Input a point cloud containing noise , the number of points in the point cloud ;​ Step 2. Noise-based point cloud Mid noise points The probability distribution of lying on the real base surface constructs the density prior Step 3. Initialize the dynamic graph convolution network, construct the scoring network, the pre-filtering network and the up-sampling network using the multi-layer perception and initialize; Step 4. Dynamic graph convolutional network for noisy point clouds Extracting multi-scale features of each point to form a feature matrix ; Step 5. Learning the multi-scale feature matrix by a scoring network to obtain a point score vector , retaining points with higher scores , , obtaining a low-noise point set ; Step 6. Pre-filtering the low-noise point set by a pre-filtering network performing pre-filtering; Step 7. Generate the denoised point cloud by up-sampling the pre-filtered low-noise point set through the up-sampling network; Step 8. Determine whether the denoised point cloud meets the requirements, if yes, end; Otherwise, proceed to step 9; Step 9. Randomly select a neighborhood to construct an unsupervised loss function and optimize the dynamic graph convolution network, the scoring network, the pre-filtering network and the up-sampling network, and return to step 4; The step 2 is to construct the density prior according to formula (1); （1） The For The neighborhood of The The value of the hyperparameter The value of the hyperparameter is predicted by the average distance between points, calculated according to formula (2). （2） the is the number of elements contained in the set, is the set of is the is the neighborhood of is the set of parameters; The feature matrix of step 4 is constructed according to equation (3): （3） The For the first Features of the layer input, For point exist The output features of the layer For the reason Parameterized multilayer perceptron, Point The neighborhood, Indicates the maximum pool; The step 5 is according to formula (4), (5): （4） （5） MLP is a multi-layer perceptron, is in the subscript corresponding to the maximum value; The step 6 is according to formula (6), (7), (8): （6） （7） （8） The is a feature matrix of a low-noise point set , is a score vector of the midpoint, denotes element-wise multiplication, denotes matrix multiplication; The step 7 is specifically as follows: Randomly generate two two-dimensional vectors , and the characteristic vector , in series, to obtain , using on , and according to the public (9) to generate a denoising point cloud ; （9）； The step 9 is specifically as follows: Step 9.1 construct the constraint of each point according to formula (10), (11) by randomly selecting a neighborhood each time; （10） （11） The is a weight, is a normalization coefficient, is randomly selected neighborhood; Step 9.2 Constructing a bijection between the noise point cloud and the denoised point cloud by minimizing energy , the bijection is calculated according to formula (12); （12） The , is a denoised point cloud; Step 9.

3. Optimize the dynamic graph convolution network, the scoring network, the pre-filter network, and the up-sampling network by minimizing an energy function as shown in equation (13) optimizing the dynamic graph convolution network, the scoring network, the pre-filter network, and the up-sampling network （13）。

Images