A point cloud completion method based on low-quality denoising and multi-stage upsampling

Abstract

The application discloses a point cloud completion method based on low-quality denoising and multi-stage upsampling, which comprises the following steps: firstly, an auto-encoder module, a neighborhood adaptive denoising module and a multi-stage upsampling module are designed, built and trained; in the completion process, the incomplete point cloud is firstly generated into a sparse and rough complete point cloud through the auto-encoder module, the point cloud only has a global shape and lacks fine geometric details; then the sparse and rough complete point cloud is denoised through the neighborhood adaptive module, most of the noise points are removed, and the geometric structure is clearer; finally, the denoised point cloud is input into the multi-stage upsampling module to obtain a dense complete point cloud. Compared with the prior art, the point cloud completion method of the application can retain the geometric structure of the incomplete point cloud while generating a dense complete point cloud with fine geometric details, and there are no noise points or a small amount of noise points at the slender structure.

Description

Technical Field

[0001] This invention relates to the field of 3D data reconstruction technology in computer vision, specifically to a point cloud completion method based on low-quality denoising and multi-level upsampling. Background Technology

[0002] Point clouds are collections of points sampled from the surface of an object in three-dimensional space, containing rich geometric and spatial information. Due to their advantages such as small data size, ease of storage and acquisition, and strong representational capabilities, point cloud data is widely used in fields such as 3D vision and computer graphics. Common point cloud data scanning devices include laser scanners, binocular cameras, structured light sensors, and LiDAR. However, in the real world, due to limitations in resolution, viewpoint occlusion, and sensor distance of point cloud scanning devices, the acquired point clouds are often sparse and incomplete, meaning they contain holes or missing parts of their shape. Directly using such point cloud data for downstream tasks leads to severe performance degradation. Therefore, point cloud completion—the task of inferring a complete point cloud from a sparse and incomplete one—is urgently needed for downstream tasks.

[0003] Existing point cloud completion methods can be broadly classified into two categories: (1) traditional methods; and (2) deep learning-based methods. Traditional methods, such as symmetry-driven, surface reconstruction, and retrieval matching methods, typically require objects to have good symmetry and can only handle point clouds with small holes. They are also very sensitive to noise. In addition, retrieval matching methods require optimization during the retrieval process, resulting in high time complexity and making them unsuitable for online applications. Early deep learning-based methods typically employed an autoencoder architecture. However, these methods only supervised the final generated point cloud, resulting in the generation of a complete point cloud with a rough shape. In recent years, popular deep learning-based methods have typically adopted a coarse-to-fine architecture, introducing an additional refinement network on top of the original autoencoder to optimize the generated point cloud. The refinement network in existing methods usually lacks denoising capabilities for the generated coarse complete point cloud, resulting in the completed point cloud being filled with noisy points in complex or slender structures. Furthermore, the refinement network in existing methods usually lacks information exchange between the neighborhoods of the point cloud, resulting in the completed point cloud lacking fine geometric details. Summary of the Invention

[0004] The purpose of this invention is to overcome the above-mentioned defects in the prior art and provide a point cloud completion method based on low-quality denoising and multi-level upsampling.

[0005] The objective of this invention can be achieved by adopting the following technical solutions:

[0006] A point cloud completion method based on low-quality denoising and multi-level upsampling, the point cloud completion method includes the following steps:

[0007] S1. Construct a point cloud completion network based on low-quality denoising and multi-level upsampling. The network includes an autoencoder module, a neighborhood adaptive denoising module, and a multi-level upsampling module connected in sequence. The autoencoder module consists of an encoder and a decoder connected in sequence, and the multi-level upsampling module consists of two partition-based upsampling Transformer modules connected in sequence.

[0008] S2. Train a point cloud completion network based on low-quality denoising and multi-level upsampling using a point cloud completion dataset. Iterate the training in an end-to-end manner by minimizing the loss function. Calculate the loss function value during each iteration and use backpropagation to update the parameters in the point cloud completion network until the network converges.

[0009] S3. Input the residual point cloud P, use the encoder in the autoencoder module to encode the residual point cloud to obtain the global feature g, and use the decoder in the autoencoder module to decode the global feature g to obtain the sparse coarse complete point cloud P0.

[0010] S4. The sparse, coarse, complete point cloud P0 is denoised using the neighborhood adaptive denoising module to obtain the denoised point cloud P1.

[0011] S5. After mixing the denoised point cloud P1 with the residual point cloud P, downsample the point cloud. Input the downsampled point cloud into the multi-level upsampling module. The multi-level upsampling module performs multiple consecutive upsamplings on the downsampled point cloud to obtain a dense and complete point cloud C.

[0012] Furthermore, the encoder operates as follows:

[0013] Given a residual cloud Let P be the i-th point in the point cloud, and N be the number of points in the residual point cloud. Construct a geometric context C from the residual point cloud P. geo The process is as follows: For each point in the residual cloud P, find its neighborhood and obtain the neighborhood tensor. k is the size of the neighborhood corresponding to each point in the residual point cloud P; at the same time, the residual point cloud P is expanded with a new dimension and the point coordinates of the point cloud are copied k times along this dimension to form the original coordinate tensor. The relative coordinates ΔP = N are obtained by using the neighborhood tensor and the original coordinate tensor. geo -o geo The geometric context C is obtained by concatenating relative coordinates and the original coordinate tensor. geo =concat[O geo ;△P], concat[;] indicates a tensor concatenation operation along a specified dimension;

[0014] Constructing the feature context C using the defect cloud P. featThe process is as follows: Point-by-point feature extraction is performed on the residual defect cloud P using Shared MLP to obtain the point-by-point features of the residual defect cloud P. d represents the number of channels for the pointwise features of the residual cloud P, f i For the i-th point p in the defective cloud P i The corresponding features; Shared MLP, proposed by Qi et al. in the paper "PointNet: Deep learning on point sets for 3D classification and segmentation", consists of one-dimensional convolutional layers, ReLU activation function layers, and batch normalization layers, and is used for point-by-point feature extraction and feature dimensionality reduction of point clouds; for each point of the residual point cloud P, the features f i Find the neighborhood and obtain the feature neighborhood tensor. Simultaneously, the pointwise features F of the residual feature cloud P are extended to a new dimension, and the features of each point are copied k times along this dimension to obtain the original feature tensor. The relative feature tensor ΔF = N is obtained by comparing the feature neighborhood tensor with the original feature tensor. feat -O fea t, the feature context C is obtained by concatenating the relative feature tensor and the original feature tensor. feat =concat[o feat ;△F];

[0015] Geometric context C geo and feature context C feat F is obtained by fusion. f The process is as follows: First, the two are passed through two independent multilayer perceptrons. and Obtain feature encoding and Next, the two are concatenated along the channel dimension, and finally, max pooling is performed on the neighborhood to obtain F. f , This indicates that a max pooling operation is performed on a specified dimension.

[0016] For F f F is obtained by performing feature enhancement through hierarchical feature extraction operations. f The hierarchical feature extraction process is as follows: For F f F is obtained by performing the first cross-attention operation. c1 , for F c1 F was obtained by performing the first self-attention operation. s1 , for F s1 Perform a second cross-attention operation to obtain F c2Cross-attention and self-attention operations, as described by Wang et al. in their paper "PointAttN: You only need attention for point cloud completion," involve cross-attention operations using features from a downsampled point cloud to perform multi-head attention on features from a point cloud before downsampling. This enables downsampling of the point cloud, increases the interactivity between point-by-point features, and increases the number of feature channels. Self-attention operations use point cloud features to perform multi-head attention on themselves, which also increases the interactivity between point-by-point features. For F... s1 Perform a second cross-attention operation to obtain F c2 , for F c2 F is obtained by performing a second self-attention operation. s2 ; For F s2 The third cross-attention operation yields F. c3 , for F c3 F is obtained by performing the third self-attention operation. f ′, for the enhanced feature F f Global features are obtained through max pooling. cg is the dimension of the global feature g.

[0017] Furthermore, the construction and fusion of the geometric context and feature context mainly consider two types of contexts. The first type is the geometric context, which is the spatial context formed by points sampled from the synthesized polygon mesh or obtained by scanning the real scene, containing the geometric distribution of points in the original 3D space. The second type is the feature context, which is the encoding space formed after preliminary feature extraction, containing rich potential representations for visual analysis. Fusing the two types of contexts can fully explore the potential spatial and semantic relationships in the point cloud, which helps to improve the performance of point cloud completion.

[0018] Furthermore, the decoder operates as follows: It expands the global feature g using a 1D transposed convolution to obtain the feature... N0 is the number of features after expansion, c t The number of channels for the expanded feature; then, the global feature g is copied N0 times and stacked to form a feature matrix, which is then compared with F. G Features are obtained by concatenating them along the channel dimension. Finally, further feature extraction is performed using Res-Shared MLP, and Shared MLP is used to reduce the number of extracted feature channels to 3, thus obtaining a sparse coarse point cloud. Res-Shared MLP, proposed by Xiang et al. in the paper "Snowflakenet: Point cloud completion by snowflake pointdeconvolution with skip-transformer", consists of two layers of Shared MLP and residual connections, and is used for point-by-point feature extraction or feature enhancement of point clouds.

[0019] Furthermore, the denoising process of the neighborhood adaptive denoising module is as follows:

[0020] A mixed point cloud is obtained by mixing the sparse coarse point cloud P0 with the residual point cloud P. Extracting point-by-point features from the hybrid point cloud P0′ D is a characteristic The number of channels; the feature matrix formed by stacking the global feature g after copying it N0+N times, and the point-by-point feature. The point-by-point features are obtained by stitching them together along the channel dimension to achieve a fused global shape. For the i-th point in the point cloud * Features corresponding to each point;

[0021] Downsampling of the mixed point cloud P0′ yields P0″; for each point p in P0″... j Find the neighborhood in P0′ p jk For point p j For a given point within its corresponding neighborhood, K+1 represents the size of that neighborhood; for p jk Local coordinates are constructed through local normalization operations, and then compared with p. i p ik p is obtained by concatenating and extracting point-by-point features within the neighborhood. jk The corresponding pointwise feature r jk ;

[0022] p j Corresponding point-by-point features of the fused global shape With r jk Enhanced point-by-point features within the neighborhood are obtained by stitching along the channel dimension. Calculate p j With neighboring regions Each point p inside jk Corresponding weights softmax(·) is the normalization function. φ(·) and φ(·) correspond to two independent multilayer perceptrons; a new point coordinate is obtained by weighted summation of the point coordinates in each neighborhood and their corresponding weights. All generated new coordinates constitute a denoised point cloud. N1 represents the number of points in the denoised point cloud P1, and N1 = N0.

[0023] Furthermore, in the denoising process of the neighborhood adaptive denoising module, the sparse coarse point cloud P0 is mixed with the residual point cloud P to increase the resolution of the point cloud and expand the search range for the subsequent search neighborhood. At the same time, the mixing process also retains the existing geometric information in the input residual point cloud, which to a certain extent provides significant structural information in the input residual point cloud for the completion result, thereby improving the fidelity of the completed point cloud.

[0024] Furthermore, the neighborhood size in the neighborhood adaptive denoising module is K+1 during the denoising process because when processing p... j The first point when searching for the neighborhood is point p. j itself; point p j It is obtained by downsampling from P0′. This point itself may be a noise point. Therefore, it is more accurate and has a better denoising effect by using only all points in the neighborhood except this point to generate new coordinates.

[0025] Furthermore, the operation of the multi-level upsampling module is as follows:

[0026] The input point cloud P0 is mixed with the denoised point cloud P1 and downsampled to obtain P1′; P1′ is upsampled for the first level by the partition-based upsampling Transformer module to obtain the first-level upsampled point cloud P2; P2 is upsampled again for the second level by the partition-based upsampling Transformer module to obtain the dense complete point cloud C.

[0027] Furthermore, during the operation of the multi-level upsampling module, the input point cloud P0 and the denoised point cloud P1 are mixed and downsampled, which preserves the existing geometric information in the input residual point cloud for subsequent upsampling operations, thereby improving the fidelity of the upsampled point cloud to a certain extent.

[0028] Furthermore, the partition-based upsampling Transformer module operates as follows:

[0029] Given a sparse point cloud that needs upsampling N down To determine the number of points in a sparse point cloud, first consider P... down Constructing geometric context and feature context and fusing them to obtain point-by-point features c1 is the number of channels for this point-by-point feature; for the point-by-point feature F down Enhanced pointwise features are obtained by performing three consecutive self-attention operations. c2 represents the number of channels for the enhanced pointwise feature;

[0030] For F down Perform feature expansion operation to obtain Fup The workflow of feature expansion operation is as follows: First, for F... down Features are obtained by expanding using 1D transposed convolution. r is the upsampling factor. The number of channels for this feature is r, and the kernel size and stride of the transposed convolution are both r; F down The characteristic matrix formed by repeating point by point r times and F t Feature F is obtained by splicing along the channel direction. up ;

[0031] For feature F up Perform coordinate reconstruction to obtain the upsampled point cloud to complete the upsampling operation P. up The workflow of coordinate reconstruction is as follows: First, F is reconstructed using Shared MLP. up Dimensionality reduction to 3 yields point-by-point displacement Then F down The upsampled point cloud is obtained by repeating the process point by point r times and adding it to the point-by-point displacement. N up For upsampled point cloud P up The number of midpoints, and there are N up =rN down .

[0032] Furthermore, during the operation of the partition-based upsampling Transformer module, feature expansion is performed using 1D transposed convolution. This operation is a learning-based feature expansion operation, which is more flexible and consumes less time and space compared to using a multilayer perceptron to increase the channel dimension and using tensor deformation operations to expand the number of points to reduce the channel dimension. This improves performance while reducing time and space complexity.

[0033] Furthermore, during the operation of the partition-based upsampling Transformer module, F... down The characteristic matrix formed by repeating point by point r times and F t Feature F is obtained by splicing along the channel direction. up This operation aims to constrain feature expansion by utilizing the original point-by-point features, preventing points from being divided too far apart, resulting in outliers, or too close together, causing points to cluster together, thus effectively improving the quality of the upsampled point cloud.

[0034] Furthermore, during the operation of the partition-based upsampling Transformer module, the extended features are processed into displacements. This operation learns multiple displacements for a point and adds them to the original point to generate multiple points, which is equivalent to partitioning the point. Since the neighborhood adaptive denoising module may not remove all noise points, some noise points may still remain in the denoised point cloud P1. This displacement-based approach performs both upsampling and denoising, i.e., further denoising, making it more suitable for completion tasks and effectively improving the quality of the upsampled point cloud.

[0035] Furthermore, the loss function is defined as follows:

[0036]

[0037] in, For the overall loss function, To reconstruct the loss term, For noise reduction loss term, This is the first-level upsampling loss term. This is the second-level upsampling loss term;

[0038] Reconstructing the loss term Calculate using the following formula:

[0039]

[0040] Where Y0 is from the reference point cloud P GT The point cloud obtained by downsampling has the same number of points as the sparse, coarse, complete point cloud P0. The chamfer distance is calculated as follows:

[0041]

[0042] Where S1 is the predicted point cloud, S2 is the reference point cloud corresponding to S1, x is any point in S1, y is any point in S2, |·| represents the number of points in a point cloud, and ‖·‖2 represents the Euclidean distance.

[0043] The denoising loss term is calculated using the following formula:

[0044] Where Y1 is from the reference point cloud P GT The point cloud obtained by downsampling has the same number of points as the denoised point cloud P1.

[0045] The first-level upsampling loss term is calculated using the following formula:

[0046] Where Y2 is from the reference point cloud P GT The point cloud obtained by downsampling has the same number of points as the first-level upsampled point cloud P2.

[0047] The second-level upsampling loss term is calculated using the following formula:

[0048]

[0049] Among them, P GT For reference point cloud, the number of points is the same as that of dense complete point cloud C.

[0050] Furthermore, the loss function uses chamfered distance. Chamfered distance calculates the average distance from all points in one point cloud to the nearest point in another point cloud, and it has more reasonable time and space complexity compared to other distance metrics.

[0051] The present invention has the following advantages and effects compared with the prior art:

[0052] 1. The completed point cloud has fewer noise points near slender structures. This invention proposes a neighborhood-adaptive denoising method. Compared to existing denoising methods based on point-by-point displacement, this method adaptively learns the weights of point coordinates within the neighborhood based on point features. Furthermore, considering that the search point itself is a noise point, it uses all points in the neighborhood except the search point to generate new coordinates for denoising. This method is more effective at removing noise points near slender structures, further improving the quality of the completed point cloud. In addition, because the input point cloud is mixed during the denoising process, it retains some of the existing geometric information of the input residual point cloud, improving the fidelity of the completed point cloud.

[0053] 2. It can generate fine geometric details. Points in a point cloud should not be independent; there should be a certain correlation between them, and the set of points should reflect the semantic information of a certain part of an object. Therefore, increasing the interaction between neighborhoods plays a key role in improving the quality of the completed point cloud. This invention proposes a partitioned point cloud upsampling Transformer. By aggregating neighborhood features of each point and using an attention mechanism to enable spatial and semantic interaction between neighborhood features, the quality of the completed point cloud is improved. Furthermore, feature expansion is performed using transposed convolution and feature concatenation, which improves the flexibility of upsampling while also limiting the partitioning process, resulting in upsampled point clouds with fine geometric details. Attached Figure Description

[0054] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0055] Figure 1 This is a flowchart of the point cloud completion method disclosed in this invention, which includes a training process and a testing process;

[0056] Figure 2 This is a schematic diagram of the composition of the autoencoder module in the point cloud completion method disclosed in this invention;

[0057] Figure 3 This is a schematic diagram of the composition of the multi-level upsampling module in the point cloud completion method disclosed in this invention;

[0058] Figure 4 This is a comparison diagram of the point cloud completion method disclosed in this invention and existing methods when the point cloud resolution is 16384.

[0059] Figure 5 This is a comparison diagram of the point cloud completion method disclosed in this invention and existing methods when the point cloud resolution is 2048. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] Example 1

[0062] This embodiment discloses a point cloud completion method based on low-quality denoising and multi-level upsampling, which specifically includes the following steps:

[0063] S1. Construct a point cloud completion network based on low-quality denoising and multi-level upsampling. The network includes an autoencoder module, a neighborhood adaptive denoising module, and a multi-level upsampling module connected in sequence. The autoencoder module consists of an encoder and a decoder connected in sequence, and the multi-level upsampling module consists of two partition-based upsampling Transformer modules connected in sequence.

[0064] The first partition-based upsampling Transformer module has an upsampling factor of 4, and the second partition-based upsampling Transformer module has an upsampling factor of 8.

[0065] S2. Train a point cloud completion network based on low-quality denoising and multi-level upsampling using a point cloud completion dataset. Iterate the training in an end-to-end manner by minimizing the loss function. Calculate the loss function value during each iteration and use backpropagation to update the parameters in the point cloud completion network until the network converges.

[0066] The point cloud completion dataset uses the PCN dataset proposed by Yuan et al. in the paper "PCN: Point Completion Network". The dataset contains 2048 points in the residual point cloud and 16384 points in the reference point cloud, for a total of 28974 training reference point clouds. Each reference point cloud corresponds to 8 residual point clouds from random views.

[0067] The loss function is defined as follows:

[0068]

[0069] in, For the overall loss function, To reconstruct the loss term, For noise reduction loss term, This is the first-level upsampling loss term. This is the second-level upsampling loss term;

[0070] Reconstructing the loss term Calculate using the following formula:

[0071]

[0072] Where Y0 is from the reference point cloud P GT The point cloud obtained by downsampling has the same number of points as the sparse, coarse, complete point cloud P0, both being 256. The chamfer distance is calculated as follows:

[0073]

[0074] Where S1 is the predicted point cloud, S2 is the reference point cloud corresponding to S1, x is any point in S1, y is any point in S2, |·| represents the number of points in a certain point cloud, and ‖·‖2 represents the Euclidean distance;

[0075] The denoising loss term is calculated using the following formula:

[0076]

[0077] Where Y1 is from the reference point cloud P GT The point cloud obtained by downsampling has the same number of points as the denoised point cloud P1, both being 256.

[0078] The first-level upsampling loss term is calculated using the following formula:

[0079]

[0080] Where Y2 is from the reference point cloud P GTThe point cloud obtained by downsampling has the same number of points as the first-level upsampled point cloud P2, both being 2048.

[0081] The second-level upsampling loss term is calculated using the following formula:

[0082]

[0083] Among them, P GT For reference point cloud, the number of points is the same as that of dense complete point cloud C, which is 16384.

[0084] During training, the initial learning rate was 0.0001, and the learning rate decreased to 0.7 times the original value every 5 rounds, for a total of 50 rounds of training.

[0085] S3. Input the residual point cloud P, and use the encoder in the autoencoder module to encode the residual point cloud to obtain the global feature g, which has a dimension of 512. Use the decoder in the autoencoder module to decode the global feature g to obtain the sparse coarse complete point cloud P0.

[0086] S4. The sparse, coarse, complete point cloud P0 is denoised using the neighborhood adaptive denoising module to obtain the denoised point cloud P1.

[0087] S5. After mixing the denoised point cloud P1 with the residual point cloud P, downsample the point cloud. Input the downsampled point cloud into the multi-level upsampling module. The multi-level upsampling module performs multiple consecutive upsamplings on the downsampled point cloud to obtain a dense and complete point cloud C.

[0088] like Figure 4 As shown, the first column is the residual point cloud, the second column is the completed point cloud of the SnowflakeNet method proposed in the paper "Snowflakenet: Point cloud completion by snowflake point deconvolution with skip-transformer", the third column is the completed point cloud of the FBNet method proposed in the paper "FBNet: Feedback network for point cloud completion", the fourth column is the dense and complete point cloud obtained in step S5, which is the result of this invention, and the fifth column is the reference point cloud. As can be seen from the figure, for the aircraft in the first row, the engine on the side wing completed by this invention has better geometric details, while the engine shape on the side wing completed by other methods is blurry and has obvious noise points. For the chair in the second column, the hole between the chair back and seat board completed by this invention is clearer and almost noise-free.

[0089] Table 1 presents a quantitative comparison of the performance of the currently popular methods SnowflakeNet and FBNet with that of the present invention on the PCN dataset test set. The evaluation metric is the chamfer distance, with a smaller value being better. The results in the table show that the present invention outperforms existing methods in the vast majority of categories, demonstrating the effectiveness of the present invention.

[0090] Table 1. Quantitative Comparison Results of the Method Disclosed in this Invention and Other Methods (×10) -3 )

[0091]

[0092] Example 2

[0093] This embodiment discloses a point cloud completion method based on low-quality denoising and multi-level upsampling, which specifically includes the following steps:

[0094] S1. Construct a point cloud completion network based on low-quality denoising and multi-level upsampling. The network includes an autoencoder module, a neighborhood adaptive denoising module, and a multi-level upsampling module connected in sequence. The autoencoder module consists of an encoder and a decoder connected in sequence, and the multi-level upsampling module consists of two partition-based upsampling Transformer modules connected in sequence.

[0095] The first partition-based upsampling Transformer module has an upsampling factor of 2, and the second partition-based upsampling Transformer module also has an upsampling factor of 2.

[0096] S2. Train a point cloud completion network based on low-quality denoising and multi-level upsampling using a point cloud completion dataset. Iterate the training in an end-to-end manner by minimizing the loss function. Calculate the loss function value during each iteration and use backpropagation to update the parameters in the point cloud completion network until the network converges.

[0097] The point cloud completion dataset uses the Completion3D dataset proposed by Tchapmi et al. in the paper "TopNet: Structural pointcloud decoder". The dataset contains 2048 points in the residual point cloud and 2048 points in the reference point cloud. The resolution is different from the PCN dataset. There are a total of 28974 training reference point clouds, and each reference point cloud corresponds to a residual point cloud from a random viewpoint.

[0098] The loss function is defined as follows:

[0099]

[0100] in, For the overall loss function, To reconstruct the loss term, For noise reduction loss term, This is the first-level upsampling loss term. This is the second-level upsampling loss term;

[0101] Reconstructing the loss term Calculate using the following formula:

[0102]

[0103] Where Y0 is from the reference point cloud P GT The point cloud obtained by downsampling has the same number of points as the sparse, coarse, complete point cloud P0, both being 256. The chamfer distance is calculated as follows:

[0104]

[0105] Where S1 is the predicted point cloud, S2 is the reference point cloud corresponding to S1, x is any point in S1, y is any point in S2, |·| represents the number of points in a point cloud, and ‖·‖2 represents the Euclidean distance.

[0106] The denoising loss term is calculated using the following formula:

[0107]

[0108] Where Y1 is from the reference point cloud P GT The point cloud obtained by downsampling has the same number of points as the denoised point cloud P1, both being 256.

[0109] The first-level upsampling loss term is calculated using the following formula:

[0110]

[0111] Where Y2 is from the reference point cloud P GT The point cloud obtained by downsampling has the same number of points as the first-level upsampled point cloud P2, both being 1024.

[0112] The second-level upsampling loss term is calculated using the following formula:

[0113]

[0114] Among them, P GT For reference point cloud, the number of points is the same as that of dense complete point cloud C, which is 2048.

[0115] During training, the initial learning rate was 0.0001, and it decreased to 0.7 times the original value every 20 rounds, for a total of 150 rounds.

[0116] S3. Input the residual point cloud P, and use the encoder in the autoencoder module to encode the residual point cloud to obtain the global feature g, which has a dimension of 512. Use the decoder in the autoencoder module to decode the global feature g to obtain the sparse coarse complete point cloud P0.

[0117] S4. The sparse, coarse, complete point cloud P0 is denoised using the neighborhood adaptive denoising module to obtain the denoised point cloud P1.

[0118] S5. After mixing the denoised point cloud P1 with the residual point cloud P, downsample the point cloud. Input the downsampled point cloud into the multi-level upsampling module. The multi-level upsampling module performs multiple consecutive upsamplings on the downsampled point cloud to obtain a dense and complete point cloud C.

[0119] like Figure 5 As shown, the first column is the incomplete point cloud, the second column is the completed point cloud obtained by the Snowflakenet method, the third column is the completed point cloud obtained by the FBNet method, the fourth column is the dense and complete point cloud obtained in step S5, and the fifth column is the reference point cloud. It can be seen from the figure that, for the sofa in the first row, the sofa backrest completed by this invention is closer to the backrest of the reference point cloud, and the holes on the backrest are obvious, while the sofa backrests completed by other methods lack corresponding holes. For the table lamp in the second column, the table lamp completed by this invention has a more complete global shape, while the table lamps completed by other methods have larger deformations and obvious noise points.

[0120] Table 2 presents a quantitative comparison of the methods SnowflakeNet, FBNet, and the present invention on the test set of the Completion3D dataset. The evaluation metric is the chamfer distance, with a smaller value indicating better performance. The results in the table show that the present invention outperforms existing methods in most categories, demonstrating its effectiveness.

[0121] Table 2. Quantitative comparison results between the point cloud completion method disclosed in this invention and other methods (×10) -3 )

[0122]

[0123] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A point cloud completion method based on low-quality denoising and multi-stage upsampling, characterized in that, The point cloud completion method comprises the following steps: S1, constructing a point cloud completion network based on low-quality denoising and multi-level upsampling, the network comprising a self-encoder module, a neighborhood adaptive denoising module and a multi-level upsampling module connected in sequence, wherein the self-encoder module is composed of an encoder and a decoder connected in sequence, and the multi-level upsampling module is composed of two partition-based upsampling Transformer modules connected in sequence; the working process of the encoder is as follows: Given a residual cloud i∈{1,2,…,N}, Let P be the i-th point in the point cloud, and N be the number of points in the residual point cloud. Construct a geometric context C from the residual point cloud P. geo The process is as follows: For each point in the residual cloud P, find its neighborhood and obtain the neighborhood tensor N. geo ∈ k is the size of the neighborhood corresponding to each point in the residual point cloud P; at the same time, the residual point cloud P is expanded with a new dimension and the point coordinates of the point cloud are copied k times along this dimension to form the original coordinate tensor. The relative coordinates ΔP = N are obtained by using the neighborhood tensor and the original coordinate tensor. geo -O geo The geometric context C is obtained by concatenating relative coordinates and the original coordinate tensor. geo =concat[O geo ;△[], concat[;] indicates a tensor concatenation operation along a specified dimension; By the incomplete point cloud [constructing feature context C feat , the process is as follows: the point feature of the incomplete point cloud [is obtained by performing point-by-point feature extraction on the incomplete point cloud P d represents the number of channels of the point feature of the incomplete point cloud [; f i is the feature corresponding to the i-th point p i of the incomplete point cloud P; the feature f i of each point of the incomplete point cloud P is searched for a neighborhood to obtain a feature neighborhood tensor At the same time, the point feature F of the incomplete point cloud [is expanded by one new dimension, and the feature of each point is copied k times along the dimension to obtain an original feature tensor The relative feature tensor △F=N feat -O feat is obtained by the feature neighborhood tensor and the original feature tensor; and the feature context C feat is obtained by splicing the relative feature tensor and the original feature tensor feat ; The geometric context C geo and the feature context C feat are fused to obtain F f , the process is as follows: first, the two are respectively passed through two independent multi-layer perceptrons and to obtain feature encodings and Then, the two are spliced in the channel dimension, and finally, the neighborhood is maximum-pooled to obtain F f , represents the maximum pooling operation on a specified dimension. F f Feature enhancement is performed on F f by a hierarchical feature extraction operation to obtain F f A first cross-attention operation is performed on F c1 to obtain F c1 A first self-attention operation is performed on F s1 to obtain F s1 A second cross-attention operation is performed on F c2 to obtain F c2 A second self-attention operation is performed on F s2 to obtain F s2 A third cross-attention operation is performed on F c3 to obtain F c3 A third self-attention operation is performed on F f to obtain F f Global feature g is obtained from the enhanced feature F c g is the dimension of the global feature g. S2, training the point cloud completion network based on low-quality denoising and multi-level upsampling through a point cloud completion dataset, iteratively training in an end-to-end manner by minimizing a loss function, calculating the loss function value in each iteration process, and updating the parameters in the point cloud completion network using a back propagation technique until the network converges; S3, inputting the incomplete point cloud P, obtaining global features g by feature encoding of the incomplete point cloud through the encoder in the self-encoder module, and obtaining a sparse and rough complete point cloud P0 by decoding operation of the global features g through the decoder in the self-encoder module; S4, denoising the sparse and rough complete point cloud P0 through the neighborhood adaptive denoising module to obtain a denoised point cloud P1; S5, mixing the denoised point cloud P1 with the incomplete point cloud P and performing down-sampling, inputting the down-sampled point cloud into the multi-level upsampling module, and obtaining a dense complete point cloud C by continuously upsampling the down-sampled point cloud multiple times through the multi-level upsampling module.

2. The point cloud completion method based on low-quality denoising and multi-stage upsampling according to claim 1, characterized in that, The working process of the decoder is as follows: The feature expansion is performed on the global feature g using a 1-dimensional transpose convolution to obtain a feature N0 is the number of expanded features, c t is the number of channels of the expanded feature; The feature matrix stacked after the global feature g is replicated N0 times is F G The features are spliced together in the channel dimension to obtain features Finally, feature extraction is performed, and the extracted feature channel number is reduced to 3 to obtain a sparse rough point cloud 3. The point cloud completion method based on low-quality denoising and multi-stage upsampling according to claim 1, characterized in that, The denoising process of the neighborhood adaptive denoising module is as follows: A mixed point cloud is obtained by mixing the sparse coarse point cloud P0 with the residual point cloud P. Extracting point-by-point features from the hybrid point cloud P0′ D is a characteristic The number of channels; the feature matrix formed by stacking the global feature g after copying it N0+N times, and the point-by-point feature. The point-by-point features are obtained by stitching them together along the channel dimension to achieve a fused global shape. For the i-th point in the point cloud * Features corresponding to each point; Downsample the mixed point cloud P0' to obtain P0"; find each point p j Find the neighborhood in P0' p jk For the point p j A certain point in the corresponding neighborhood, K+1 is the size of the corresponding neighborhood; for p jk Construct a local coordinate through a local normalization operation, and compare the local coordinate with p i , p ik Splice and extract point-by-point features in the neighborhood to obtain p jk The corresponding point-by-point feature r jk ; p j Point-wise features corresponding to the fused global shape With r jk Concatenate along the channel dimension to get enhanced point-wise features within the neighborhood Calculate p j With the neighborhood Each point p jk Corresponding weights Softmax(·) is a normalization function, And φ(·) correspond to two independent multi-layer perceptrons respectively; for each point within the neighborhood, the weighted sum of the point coordinates and their corresponding weights gives a new point coordinate All generated new coordinates constitute the denoised point cloud N1 represents the number of points in the denoised point cloud P1, and N1=N0.

4. The point cloud completion method based on low-quality denoising and multi-stage upsampling according to claim 1, characterized in that, The working process of the multi-level upsampling module is as follows: Mixing and down-sampling the input point cloud P0 and the de-noised point cloud P1 to obtain P1 ′ ; P1 ′ First-level up-sampling is performed on P1 by using a division-based up-sampling Transformer module to obtain a first-level up-sampled point cloud P2; second-level up-sampling is performed on P2 by using the division-based up-sampling Transformer module again to obtain a dense complete point cloud C; the working process of the division-based up-sampling Transformer module is as follows: Sparse point cloud needing upsampling is given N down For the number of points in the sparse point cloud, first, P down Construct geometric context and feature context and fuse to obtain point-wise features c1 is the number of channels of the point-wise features; the point-wise features F down Perform three consecutive self-attention operations to obtain enhanced point-wise features c2 is the number of channels of the enhanced point-wise features; F down is obtained by performing a feature expansion operation on F up The workflow of the feature expansion operation is as follows: F down is expanded using a 1-dimensional transpose convolution to obtain a feature F r is an up-sampling ratio, is the number of channels of the feature, and the convolution kernel size and the step of the transpose convolution are both r; a feature matrix formed after F down is repeated r times point by point is spliced with F t along the channel direction to obtain a feature F up ; For feature F up Perform coordinate reconstruction operation to get up-sampled point cloud to complete up-sampling operation P up The workflow of coordinate reconstruction operation is as follows: first, reduce F up to 3 by Shared MLP to get point-wise displacement Then add point-wise displacement to point-wise repeated r times to get up-sampled point cloud P down N up is the number of points in up-sampled point cloud P up , and N up = rN down .​ 5. The point cloud completion method based on low-quality denoising and multi-stage upsampling according to claim 1, characterized in that, The loss function is defined as follows: wherein, is a total loss function, is a reconstruction loss term, is a denoising loss term, is a first-stage up-sampling loss term, is a second-stage up-sampling loss term; reconstruction loss term is calculated by the following equation: wherein Y0 is the reference point cloud P GT The down-sampled point cloud has the same number of points as the sparse coarse complete point cloud P0, is the chamfer distance, which is calculated as follows: Wherein, S1 is a predicted point cloud, S2 is a reference point cloud corresponding to S1, x is any point in S1, y is any point in S2, |·| represents the number of points of a certain point cloud, and ‖·‖2 represents the Euclidean distance; The denoising loss term is calculated by the following formula: wherein Y1 is a point cloud from the reference point cloud P GT The down-sampled point cloud has the same number of points as the denoised point cloud P1. The first-level upsampling loss term is calculated by the following formula: wherein Y2 is a point cloud from the reference point cloud P GT The down-sampled point cloud has the same number of points as the first up-sampled point cloud P2. The second-level upsampling loss term is calculated by the following formula: where P GT is the reference point cloud, with the same number of points as the dense complete point cloud C.

Images