Deep learning point cloud completion method under noise interference

Abstract

The invention discloses a deep learning point cloud completion method under noise interference. According to the method, for incomplete point cloud data containing noise and acquired by point cloud acquisition equipment such as a laser radar, firstly, outliers in original point cloud data are screened, neighbor points of the outliers are selected, weights are distributed according to the distances between the neighbor points and the outliers, coordinates of the outliers are corrected by using coordinate weighted values of the neighbor points, and the optimal point cloud data are obtained. And the outliers approach the main point cloud as much as possible. The method comprises the following steps of: weighting point cloud data, encoding the weighted and corrected point cloud data in a fusion pooling mode, inputting the encoded point cloud data into a multi-scale decoder, down-sampling an encoding result into three global features with different scales, and sequentially decoding, arranging high-resolution feature points by taking a low-resolution decoding feature as a center, and outputting a coarse output point cloud. And finally, performing up-sampling on the coarse output point cloud to obtain a dense point cloud, and completing the completion of the original point cloud data. According to the method, a good completion effect on the point cloud data can be realized under noise interference.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a point cloud completion method in three-dimensional reconstruction, and in particular to a deep learning point cloud completion method under noise interference. Background Art

[0002] The Metaverse is a persistent virtual environment that integrates virtual reality, augmented reality, artificial intelligence and other technologies. It has the characteristics of convergence, physical persistence, connection perception and sharing, and provides users with a diversified and highly interactive virtual world. As the foundation of the construction of the Metaverse, three-dimensional technology provides it with key functions such as virtual world modeling, virtual reality and augmented reality experience, interactive experience, real-time rendering and animation, and multi-platform compatibility. However, to achieve a truly realistic virtual experience, more advanced three-dimensional data processing technology is required, and point cloud technology is one of them. The equipment for obtaining point clouds is mainly point cloud acquisition equipment such as laser radar and RGB-D camera. However, due to the influence of equipment resolution and environmental factors, the collected point cloud data is often missing or contains noise, which brings certain difficulties to the subsequent processing of point clouds.

[0003] Point cloud completion is an important research direction in the field of computer vision. By performing a series of intelligent processing on an incomplete point set, a complete point set with obvious shape and easy to identify is output. Traditional point cloud completion methods can be divided into two types: geometry-based methods and alignment-based methods. Among them, geometry-based methods usually complete the point cloud data by modeling and analyzing the geometric features. Alignment-based methods usually use known geometric models or reference point cloud data as templates, align the point cloud to be completed to the template, and complete it according to the alignment relationship. However, these methods can only be limited to some objects with relatively simple shapes, require the input incomplete point cloud to have high integrity, and the results are not ideal when processing large-scale point clouds due to high computational complexity.

[0004] With the rapid development of computer vision and the enrichment of point cloud datasets, deep learning-based point cloud completion networks have gradually become mainstream with their powerful capabilities. Deep learning models such as convolutional neural networks (CNNs) and generative adversarial networks (GANs) can learn relevant features from a large amount of complete point cloud data and infer missing areas in incomplete data.

[0005] However, existing completion models are usually trained based on noise-free ideal data sets, which makes the models highly sensitive to noise. However, in practical applications, point clouds acquired by point cloud acquisition devices not only have missing parts, but also are accompanied by a lot of noise. Summary of the invention

[0006] In view of the shortcomings of the existing technology, the present invention proposes a deep learning point cloud completion method under noise interference, which reduces the impact of noise by screening outliers and weighting them by distance, fusion pooling and multi-scale decoding, and achieves high-quality completion under noise interference.

[0007] A deep learning point cloud completion method under noise interference specifically includes the following steps: Step 1: Use point cloud acquisition equipment to obtain point cloud data.

[0008] Step 2: Filter outliers and perform weighted correction on their positions to make them as close to the main point cloud as possible: s2.1. Sample M points from the original point cloud through farthest point sampling (FPS).

[0009] s2.2. Screen the M points obtained by sampling and calculate the number S of the M points within the radius range d. When S is less than the set value, the point is identified as an outlier.

[0010] s2.3. For outliers, use the KNN algorithm to select the N points closest to them (X1, X2, …, X N ), and then assign weights to these N points according to their distance from the outliers , i=1,2,…N, and The larger the distance, the lower the weight. The weighted values ​​of the coordinates of N points are used as the new positions of the outliers. : in, Represents the coordinates of the i-th point.

[0011] Step 3: Encode the weighted corrected point cloud data by fusion pooling to obtain the point cloud feature v.

[0012] Step 4: Input the point cloud feature v into the multi-scale decoder, and downsample the global features into three different scale global features through the fully connected layer and the MLP layer, and decode them in turn. Centering on the low-resolution decoded features, the high-resolution feature points are arranged to output the coarse output point cloud. .

[0013] Step 5: Output the rough point cloud Upsampling is performed through the folding operation to obtain a dense point cloud , completing the completion of the original point cloud data containing noise and partial missing.

[0014] The present invention has the following beneficial effects: By weighting the distance and correcting the coordinates of the outliers, the outliers outside the main point cloud are made to be as close to the main point cloud as possible to reduce the interference of noise; then, by fusing the encoding method of the pooling module, mean pooling is introduced to reduce the loss of detail features and high noise sensitivity caused by using only maximum pooling; finally, through multi-scale decoding, by outputting point clouds of different resolutions, with the low-resolution point cloud as the center, combined with contextual information, the low-resolution features are retained, and the stability of the network and the ability to recognize noise are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of a deep learning point cloud completion method under noise interference according to the present invention; Figure 2 A schematic diagram of weighted correction of the position of outliers; Figure 3 Schematic diagram of the multi-scale decoder structure. DETAILED DESCRIPTION

[0016] The present invention will be further explained below with reference to the accompanying drawings; This example takes the ShapeNet dataset as an example to introduce Figure 1 A deep learning point cloud completion method under noise interference is shown: Step 1: There are 30,974 samples in the ShapeNet dataset. Each sample includes a corresponding complete point cloud and a residual defect cloud. Noise is added to the residual defect cloud of the original sample, and random points are generated using a random function to simulate the actual collected residual defect cloud data with noise for completion. The complete point cloud is used as the label of the processed residual defect cloud to evaluate the effectiveness of the method.

[0017] Step 2: Filter outliers from the residual cloud data with noise and Figure 2 The method shown makes a weighted correction to the position of the outlier points so that the outlier points are as close to the main point cloud as possible: s2.1. Sample M points from the original point cloud through farthest point sampling (FPS), where M is 70% of the total number of points in the original point cloud.

[0018] s2.2. Screen the M points obtained by sampling, and calculate the number S of each point within the radius range d=0.1. When S<20, the point is identified as an outlier.

[0019] s2.3. For outliers, use the KNN algorithm to select the 50 points closest to them, and then assign weights according to their distances from the outliers. , i=1,2,…50, the closer the distance, the greater the weight assigned, and . Use the weighted values ​​of the coordinates of these points as the new positions of the outliers :

[0020] in, Represents the coordinates of the i-th point.

[0021] Step 3: Encode the weighted corrected point cloud data by fusion pooling to obtain the point cloud feature v. The specific method is: first, the dimension of the input point data with a shape of (M×3) is increased to 256 dimensions through the multi-layer perceptron MLP, and then maximum pooling and mean pooling are performed respectively to obtain two pooling features of size 1×256. Then, through the cross-attention mechanism, a fused pooling feature of size 1×256 is obtained, which is copied and spliced ​​to the features before pooling for feature enhancement, and then the dimension is increased to 1024 dimensions. The maximum pooling, mean pooling and fusion operations are repeated to obtain a point cloud feature v of size 1×1024.

[0022] Step 4: Use Figure 3 The multi-scale decoder shown in the figure performs preliminary decoding on the point cloud feature v to obtain a rough output point cloud : s4.1. Downsample the 1024-dimensional point cloud feature v into 512-dimensional and 256-dimensional global features through a fully connected layer.

[0023] s4.2. Input the 256-dimensional global features into the fully connected layer and the MLP layer for decoding to obtain a 128-point ; s4.3, decode the 512-dimensional global features to obtain 256 points, and use these 256 points as Arrange as the center, and get .

[0024] s4.4. Finally, decode the 1024-dimensional point cloud feature v to obtain 1024 points, and store these 1024 points in Arrange the center to get the rough output point cloud .

[0025] Step 5: Fold the coarse output point cloud The t points of the two-dimensional grid around each point in the t matrix form a t*2 matrix, which is the rough output point cloud. The features of and the point cloud features v output in step 3 are copied and spliced, and then t upsampled points are output through MLP to obtain a dense point cloud .

[0026] In order to guide the completion process of the point cloud, the loss function is calculated according to the chamfer distance (CD) , optimize the model parameters during encoding and decoding: in, represents the real point cloud, is the weight parameter. Chamfer distance To generate the average minimum square distance between the point cloud S1 and the real point cloud S2:

[0027] in, and They represent the number of points in the generated point cloud S1 and the real point cloud S2 respectively. The first part represents the average minimum square distance of the points in the generated point cloud to the real point cloud, and the second part represents the average minimum square distance of the points in the real point cloud to the generated point cloud. The smaller the value, the closer the generated point cloud is to the real point cloud, and the better the completion effect of the model.

Claims

1. A deep learning point cloud completion method under noise interference, characterized by: The specific steps include: Step 1: Use point cloud acquisition equipment to obtain point cloud data; Step 2: Correct the position of outliers: s2.1, sample M points from the original point cloud through farthest point sampling; s2.2, calculate the number S of neighboring points of M points within the radius range d. When S is less than the set threshold, the point is identified as an outlier; s2.

3. For outliers, use the KNN algorithm to select the N points closest to them, and then assign weights according to the distance from the outliers. , i=1,2,…N, the larger the distance, the lower the weight, and the weighted values ​​of the coordinates of N points are used as the new positions of the outliers : in, Represents the coordinates of the i-th point; Step 3: Encode the weighted corrected point cloud data by fusion pooling to obtain the point cloud feature v; Step 4: Input the point cloud feature v into the multi-scale decoder, downsample it into global features of three different scales, and decode them in turn. Centering on the low-resolution decoded features, arrange the high-resolution feature points, and finally output the coarse output point cloud. ; Step 5: Output the rough point cloud Upsampling is performed through the folding operation to obtain a dense point cloud , completing the completion of the original point cloud data containing noise and partial missing.

2. The deep learning point cloud completion method under noise interference as claimed in claim 1, characterized in that: The M is 70% of the total number of points in the original point cloud.

3. The deep learning point cloud completion method under noise interference as claimed in claim 1, characterized in that: Set the radius d=0.1 and the threshold of the number of points S to 20.

4. The deep learning point cloud completion method under noise interference as claimed in claim 1, characterized in that: The encoding method of the fusion pooling is: the dimension of the input data is increased by a multi-layer perceptron MLP, and then maximum pooling and mean pooling are performed respectively, and then the two pooling features are fused through a cross-attention mechanism, and the obtained fusion pooling features are spliced ​​with the features before pooling. After the dimension is increased again, the maximum pooling, mean pooling and fusion operations are repeated to obtain the point cloud feature v.

5. The deep learning point cloud completion method under noise interference as claimed in claim 1, characterized in that: The specific steps of the multi-scale decoding are: s4.

1. Downsample the point cloud feature v to 1 / 4 and 1 / 2 of the original dimension through the fully connected layer to obtain the first dimension global feature and the second dimension global feature; s4.

2. Input the first dimension global features into the fully connected layer and MLP layer for decoding to obtain the first resolution decoding features ; s4.

3. Decode the second dimension global features and surround the decoded points with a resolution decoding feature Arrange and obtain the second resolution decoding feature ; s4.

4. Finally, decode the point cloud feature v and surround the decoded points with the second resolution decoding feature Arrange and get the rough output point cloud .

6. The deep learning point cloud completion method under noise interference as claimed in claim 1, characterized in that: Calculate the loss function based on the chamfer distance , optimize the model parameters during encoding and decoding: in, represents the real point cloud, is the weight parameter; chamfer distance () is the average minimum square distance between the generated point cloud S1 and the real point cloud S2: in, and Represent the number of points of the generated point cloud S1 and the real point cloud S2 respectively.

7. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 6.

Images