A deep learning point cloud completion method under noise interference
By filtering outliers and weighting them by distance, using fusion pooling and multi-scale decoding, the problem of point cloud completion under noise interference was solved, and high-quality point cloud data reconstruction was achieved.
Patent Information
- Application Number
- CN202411301545.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-18
- Publication Date
- 2025-12-12
- Estimated Expiration
- 2044-09-18
AI Technical Summary
Existing deep learning point cloud completion models perform poorly under noise interference and cannot effectively process noisy point cloud data acquired by acquisition devices.
By filtering outliers and weighting them by distance, using fusion pooling, and multi-scale decoding, the impact of noise is reduced, and high-quality point cloud completion is achieved.
It effectively reduces noise interference, improves the stability and recognition ability of point cloud completion, and outputs high-quality dense point clouds.
Smart Images
Figure CN119167017B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of computer vision, and relates to a point cloud completion method in three-dimensional reconstruction, in particular to a deep learning point cloud completion method under noise interference. BACKGROUND
[0002] The metaverse is a persistent virtual environment that combines virtual reality, augmented reality, artificial intelligence, and other technologies, with characteristics such as convergence, physical persistence, connection perception, and sharing, providing users with a diverse and interactive virtual world. Three-dimensional technology, as the foundation of the metaverse, provides key functions such as virtual world modeling, virtual reality and augmented reality experience, interactive experience, real-time rendering and animation, and multi-platform compatibility. However, to achieve a truly realistic virtual experience, more advanced three-dimensional data processing technology is needed, and point cloud technology is one of them. The devices that obtain point clouds are mainly laser radars and RGB-D cameras and other point cloud acquisition devices. However, due to the influence of device resolution and environmental factors, the collected point cloud data often has missing or contains noise, which brings certain difficulties to the subsequent processing of point clouds.
[0003] Point cloud completion is an important research direction in the field of computer vision. Through a series of intelligent processing of a defective point set, a complete point set with obvious shape and easy to identify is output. Traditional point cloud completion methods can be divided into two categories: geometry-based methods and alignment-based methods. The geometry-based method usually models and analyzes the geometric features of point cloud data for completion. The alignment-based method usually uses known geometric models or reference point cloud data as a template to align the point cloud to be completed to the template, and completes it according to the alignment relationship. However, these methods can only be limited to some simple-shaped objects, and require the input defective point cloud to have high integrity. Moreover, due to the high computational complexity, the results are not ideal when dealing with large-scale point clouds.
[0004] With the rapid development of the field of computer vision and the richness of point cloud datasets, point cloud completion networks based on deep learning have gradually become mainstream due to their powerful capabilities. Convolutional neural networks (CNNs) and generative adversarial networks (GANs) and other deep learning models can learn relevant features from a large amount of complete point cloud data and infer the missing areas in incomplete data.
[0005] However, existing completion models are usually trained based on ideal noise-free datasets, resulting in high sensitivity to noise. However, in actual applications, point clouds obtained through point cloud acquisition devices not only have missing parts, but also contain a large amount of noise. SUMMARY
[0006] In view of the deficiencies of the prior art, the present application provides a deep learning point cloud completion method under noise interference, which reduces the influence of noise by screening outliers and weighting by distance, fusing pooling and multi-scale decoding, and realizes high-quality completion under noise interference.
[0007] A deep learning point cloud completion method under noise interference, specifically comprising the following steps:
[0008] Step one, use a point cloud acquisition device to obtain point cloud data.
[0009] Step two, screen outliers and correct the position of the outliers by weighting, so that the outliers are as close to the main point cloud as possible:
[0010] s2.1, sample M points from the original point cloud by farthest point sampling (Farthest Point Sampling, FPS).
[0011] s2.2, screen the M points obtained by sampling, calculate the number of points S in the radius range d, and when S is less than the set value, the point is identified as an outlier.
[0012] s2.3, for outliers, use KNN algorithm to select the nearest N points (X1, X2, …, X N ) from the outliers, and then assign weights to the N points according to the distance from the outliers , i=1,2,…N, and the greater the distance, the lower the weight, and the weighted value of the coordinates of the N points is used as the new position of the outlier :
[0013]
[0014] wherein, represents the coordinates of the i-th point.
[0015] Step three, encode the point cloud data after weighting correction by fusion pooling to obtain point cloud feature v.
[0016] Step four, input the point cloud feature v into the multi-scale decoder, pass it through the full connection layer and the MLP layer, downsample the global feature into three different scale global features, and decode them in turn, arrange the high-resolution feature points around the low-resolution decoding feature, and output the rough output point cloud .
[0017] Step five, fold the rough output point cloud to upsample to get the dense point cloud , and complete the completion of the original point cloud data containing noise and partial loss.
[0018] The present application has the following beneficial effects:
[0019] By distance weighting, the outlier coordinates are corrected, so that the outliers outside the main point cloud are as close to the main point cloud as possible, reducing the interference of noise; then by the encoding method of the fusion pooling module, the mean pooling is introduced, which reduces the loss of detailed features and high noise sensitivity caused by only using the maximum pooling; finally, through multi-scale decoding, different resolution point clouds are output, taking low resolution point cloud as the center, combining context information, retaining low resolution features, and improving the stability and noise recognition ability of the network. BRIEF DESCRIPTION OF DRAWINGS
[0020] Figure 1 Figure 1 is a schematic diagram of a deep learning point cloud completion method under noise interference according to the present application;
[0021] Figure 2 Figure 2 is a schematic diagram of weighting correction of the position of an outlier;
[0022] Figure 3 Figure 3 is a schematic diagram of a multi-scale decoder structure. DETAILED DESCRIPTION
[0023] The present application will be further explained and described below in conjunction with the accompanying drawings;
[0024] This embodiment takes ShapeNet dataset as an example to introduce a deep learning point cloud completion method under noise interference as shown in Figure 1: Figure 1
[0025] Step one, there are 30974 samples in the ShapeNet dataset, each sample includes a corresponding complete point cloud and a defective point cloud, noise is added to the original sample defective point cloud, and random points are generated using a random function to simulate the actual acquisition of defective point cloud data with noise for completion processing. The complete point cloud is used as the label of the processed defective point cloud to evaluate the effectiveness of the method.
[0026] Step two, outliers are selected from the defective point cloud data with noise, and the position of the outliers is corrected by weighting according to the method shown in Figure 2, so that the outliers are as close to the main point cloud as possible: Figure 2
[0027] s2.1, M points are sampled from the original point cloud by Farthest Point Sampling (FPS), and M is 70% of the total number of points in the original point cloud.
[0028] s2.2, the M points obtained by sampling are screened, and the number of points S within a radius range d=0.1 for each point is calculated, when S<20, the point is identified as an outlier.
[0029] s2.3, for outliers, select the nearest 50 points to it using KNN algorithm, then assign weights to them according to the distance between the outlier and the points , i = 1, 2, … 50, the closer the distance, the greater the weight assigned, and . Use the weighted value of the coordinates of these points as the new position of the outlier :
[0030]
[0031] wherein, represents the coordinates of the i-th point.
[0032] Step three, encode the weighted corrected point cloud data by fusion pooling method to obtain the point cloud feature v, the specific method is: first, the dimension of the input point data with the shape of (Mx3) is improved to 256 dimensions by multi-layer perception MLP, then maximum pooling and mean pooling are performed respectively to obtain two pooling features with the size of 1x256, then cross attention mechanism is used to obtain a fusion pooling feature with the size of 1x256, which is copied and spliced to the feature before pooling for feature enhancement, and then the dimension is improved to 1024 dimensions, and the maximum value pooling, mean value pooling and fusion operation are repeated to obtain a point cloud feature v with the size of 1x1024.
[0033] Step four, use the multi-scale decoder as shown in Figure 3 to preliminarily decode the point cloud feature v to obtain a coarse output point cloud :
[0034] s4.1, downsample the 1024-dimensional point cloud feature v to 512-dimensional and 256-dimensional global features through a fully connected layer.
[0035] s4.2, input the 256-dimensional global feature into a fully connected layer and an MLP layer for decoding to obtain a containing 128 points.
[0036] s4.3, decode the 512-dimensional global feature to obtain 256 points, and arrange these 256 points around the to obtain .
[0037] s4.4, finally, decode the 1024-dimensional point cloud feature v to obtain 1024 points, and arrange these 1024 points around the to obtain a coarse output point cloud .
[0038] Step five, fold the coarse output point cloud For each point, t points around it in the 2D grid form a t*2 matrix, which is used to copy and paste the features of the coarse output point cloud and the features of the point cloud output in step 3, and then output t up-sampled points through a MLP to get the dense point cloud .
[0039] To guide the completion process of the point cloud, a loss function is calculated according to the Chamfer Distance (CD) , and the model parameters in the encoding and decoding processes are optimized:
[0040]
[0041] wherein, represents the real point cloud, is a weight parameter. The Chamfer Distance is the average least square distance between the generated point cloud S1 and the real point cloud S2:
[0042]
[0043] wherein, and respectively represent the number of points in the generated point cloud S1 and the real point cloud S2, the first part represents the average least square distance of the points in the generated point cloud to the real point cloud, and the second part represents the average least square distance of the points in the real point cloud to the generated point cloud, The smaller the value is, the closer the generated point cloud is to the real point cloud, and the better the completion effect of the model is.
Claims
1. A deep learning point cloud completion method under noise interference, characterized in that: Specifically, the following steps are included: Step 1: Acquire point cloud data using point cloud acquisition equipment; Step 2: Correct the location of outliers: s2.
1. Sample M points from the original point cloud using the farthest point sampling method; s2.2 Calculate the number S of the number of neighboring points of M points within the radius range d. When S is less than a set threshold, the point is identified as an outlier. s2.3 For outliers, use the KNN algorithm to select the N nearest points, and then assign weights based on their distance from the outlier. For each point i = 1, 2, ..., N, the greater the distance, the lower the weight. The new location of the outlier is determined by the weighted sum of the coordinates of the N points. : in, Represents the coordinates of the i-th point; Step 3: Encode the weighted and corrected point cloud data using a fusion pooling method to obtain point cloud features v; Step 4: Input the point cloud feature v into the multi-scale decoder, downsample it into global features of three different scales, and decode them sequentially. Centering on the low-resolution decoded features, arrange the high-resolution feature points to finally output a coarse point cloud. ; Step 5: Output coarse point cloud Upsampling is performed through a folding operation to obtain a dense point cloud. This completed the restoration of the original point cloud data, which contained noise and some missing data.
2. The deep learning point cloud completion method under noise interference as described in claim 1, characterized in that: M represents 70% of the total number of points in the original point cloud.
3. The deep learning point cloud completion method under noise interference as described in claim 1, characterized in that: Set the radius d=0.1 and the threshold for the number of points S to 20.
4. The deep learning point cloud completion method under noise interference as described in claim 1, characterized in that: The encoding method of the fusion pooling is as follows: the dimension of the input data is increased by multilayer perceptron (MLP), then max pooling and mean pooling are performed respectively, and the two pooled features are fused by cross attention mechanism. The resulting fusion pooled feature is concatenated with the feature before pooling, and the dimension is increased again. Max pooling, mean pooling and fusion operations are repeated to obtain point cloud feature v.
5. The deep learning point cloud completion method under noise interference as described in claim 1, characterized in that: The specific steps of the multi-scale decoding are as follows: s4.
1. Downsample the point cloud features v to 1 / 4 and 1 / 2 of the original dimensions through a fully connected layer to obtain the first-dimensional global features and the second-dimensional global features. s4.
2. Input the first-dimensional global features into the fully connected layer and the MLP layer for decoding to obtain the first-resolution decoded features. ; s4.3 Decode the second-dimensional global features, and then use the decoded points to surround the one-resolution decoded features. Arrange them to obtain the second-resolution decoding features. ; s4.4 Finally, the point cloud feature v is decoded, and the decoded points are arranged around the two-resolution decoded feature. Arrange the points to obtain a coarse output point cloud. .
6. The deep learning point cloud completion method under noise interference as described in claim 1, characterized in that: Calculate the loss function based on the chamfer distance. Optimize model parameters during encoding and decoding: in, Represents a true point cloud. These are weighting parameters; chamfer distance. ( ) represents the average minimum squared distance between the generated point cloud S1 and the real point cloud S2: in, and These represent the number of points in the generated point cloud S1 and the actual point cloud S2, respectively.
7. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method of any one of claims 1 to 6.