A point cloud denoising processing method, system, computer device and medium

Abstract

The application belongs to the technical field of point cloud processing, and specifically discloses a point cloud denoising processing method and system, computer equipment and a medium. The application uses an alternating direction multiplier method (ADMM) to solve an optimization model constructed by the application, and fuses a deep algorithm expansion network to expand the ADMM algorithm into a deep neural network, so that a point cloud denoising process with model-data double driving, consideration of interpretability and strong representation capability is realized, and the denoising effect and robustness of the method of the application are ensured. In the first iteration layer, a generalized bilateral filter is used to initialize a measurement matrix required for feature map learning, the initialization process is implicitly embedded into an iteration framework of the ADMM, a stable initial denoising result is provided for the network, the instability of random initialization in traditional deep learning is avoided, and a foundation is laid for feature map learning of subsequent layers, so that the denoising process is stable and excellent.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of point cloud processing, and relates to a point cloud denoising processing method and system, computer equipment and a medium. BACKGROUND

[0002] Point cloud is composed of discrete 3D points irregularly sampled from continuous surfaces. With the development of three-dimensional scanning technology, sensor technology and computer computing power, it has become feasible to acquire and process large-scale point cloud data. At present, point cloud has been widely used in computer vision, autonomous driving, robots and immersive remote presentation fields. However, in the process of acquiring and processing point cloud data, due to the limitations of the acquisition equipment, environmental interference, data loss or damage during acquisition, transmission and storage and other factors, the point cloud is often disturbed by noise. Due to the deformation of the underlying structure, the noise in the point cloud can seriously damage the quality and accuracy of the data, not only interfering with subsequent data analysis and processing, significantly affecting the downstream tasks such as rendering, reconstruction and analysis, but also possibly leading to a decrease in model accuracy. Therefore, point cloud denoising is crucial for related 3D vision applications.

[0003] Current point cloud denoising methods mainly fall into two categories: model-driven methods and data-driven methods. Model-driven point cloud denoising methods are designed based on specific assumptions and prior knowledge, and are solved by interpretable optimization algorithms, so they are extremely dependent on geometric priors, and need to balance between detail preservation and denoising effect, but it is difficult to accurately predict the prior information in complex scenes, and it is difficult to obtain good detail preservation and denoising effect in complex scenes. Data-driven point cloud denoising methods focus on data and have strong representation ability, and often achieve good restoration effect, but have poor generalization ability for complex application scenarios, and deep neural networks lack mathematical interpretability, and need to prepare a large amount of related data for network deep learning training, and if the amount of training data is not enough, there is a high probability of overfitting phenomenon, that is, some features in the data are over-trained and learned, and the pure data-driven point cloud denoising method is randomly initialized at the beginning of training, which is unstable, and a large number of parameter sets in the deep learning network also need to be adjusted, the calculation amount is huge, and the practicability is poor. SUMMARY

[0004] The purpose of this invention is to propose a point cloud denoising method. In the process of point cloud denoising, an optimized model is constructed by solving the Alternating Direction Multiplier Method (ADMM), and a deep algorithm unfolding network is integrated to unfold the ADMM algorithm into a deep neural network, which helps to improve the point cloud denoising performance. Even with limited data, it can still effectively remove noise and outliers in noisy point clouds and accurately restore clean point clouds. At the same time, in the first iteration layer of ADMM, a generalized bilateral filter is used to initialize the metric matrix required for feature map learning. This initialization process is embedded in the iterative framework of ADMM and is completed implicitly, which can provide stable initial denoising results and ensure that the denoising has a stable and good effect.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A point cloud denoising method includes the following steps:

[0007] Step 1. Obtain noisy point cloud data and divide it into multiple patches, each containing multiple points;

[0008] Step 2. Solve using the Alternating Direction Multiplier Method (ADMM), and transform the ADMM iterative optimization process into an end-to-end trainable network by unfolding, which contains multiple iterative layers, each corresponding to one iteration of ADMM;

[0009] Input each patch from step 1 into the network to begin the iterative optimization of ADMM. Then, for each iteration, execute steps 3 to 5 below until the iteration ends, and then proceed to step 6.

[0010] Step 3. Construct a connection graph based on the K-nearest neighbor algorithm, and establish a connection relationship between each point in the patch and its K nearest neighbors to obtain point pairs;

[0011] In the first iterative layer, for each patch extracted in step 1, the three-dimensional coordinates and curvature of the points in each patch are first extracted as point features. Then, the point features are processed using a generalized bilateral filter to construct a metric matrix and further calculate the Mahalanobis distance between point pairs to complete the initialization process.

[0012] In other iterative layers, the patches output from the previous iteration are used as input. First, the 3D coordinates of the points in each patch are input into the feature extraction model. Then, the feature vectors output by the feature extraction model are used to form new point features. Finally, the feature map is used to learn the metric matrix and calculate the Mahalanobis distance between point pairs.

[0013] Step 4. Calculate the distance between nodes in each subgraph based on Mahalanobis distance, obtain the edge weights, and obtain the adjacency matrix and degree matrix of the subgraph based on the edge weights. Then, construct the graph Laplacian matrix using the definition of the graph Laplacian matrix.

[0014] Step 5. Optimize each patch based on the graph Laplacian matrix and the 3D coordinates of all points, and then input each optimized patch into the next iteration layer;

[0015] Step 6. Process all patches using the farthest point sampling method to obtain the final denoised point cloud.

[0016] Furthermore, based on the aforementioned point cloud denoising method, this invention also proposes a point cloud denoising system corresponding to the aforementioned point cloud denoising method. Both are based on the same inventive concept and employ the following technical solution:

[0017] A point cloud denoising system includes the following modules:

[0018] The preprocessing module is used to acquire noisy point cloud data and divide it into multiple patches, each containing multiple points;

[0019] The network building module is used to solve ADMM using the Alternating Direction Multiplier Method. It transforms the ADMM iterative optimization process into an end-to-end trainable network by unfolding, which contains multiple iterative layers, each corresponding to one iteration of ADMM. Each patch is input into the network to start the iterative optimization of ADMM.

[0020] Specifically, a connection graph is constructed based on the K-nearest neighbor algorithm, and each point in the patch is connected to its K nearest neighbors to obtain point pairs;

[0021] In the first iterative layer, for each patch extracted in step 1, the three-dimensional coordinates and curvature of the points in each patch are first extracted as point features. Then, the point features are processed using a generalized bilateral filter to construct a metric matrix and further calculate the Mahalanobis distance between point pairs to complete the initialization process.

[0022] In other iterative layers, the patches output from the previous iteration are used as input. First, the 3D coordinates of the points in each patch are input into the feature extraction model. Then, the feature vectors output by the feature extraction model are used to form new point features. Finally, the feature map is used to learn the metric matrix and calculate the Mahalanobis distance between point pairs.

[0023] The distance between nodes in each subgraph is calculated based on Mahalanobis distance, the edge weights are obtained, and the adjacency matrix and degree matrix of the subgraph are obtained based on the edge weights. Then, the graph Laplacian matrix is ​​constructed using the definition of the graph Laplacian matrix.

[0024] Based on the graph Laplacian matrix and the 3D coordinates of all points, each patch is optimized, and then each optimized patch is fed into the next iteration layer;

[0025] And the aggregation module, which uses the farthest point sampling to process all patches, finally obtains the denoised point cloud.

[0026] Furthermore, based on the aforementioned point cloud denoising method, this invention also proposes a computer device comprising a memory and one or more processors. Executable code is stored in the memory. When the processor executes the executable code, it implements the steps of the aforementioned point cloud denoising method.

[0027] Furthermore, based on the above-mentioned point cloud denoising processing method, the present invention also proposes a computer-readable storage medium storing a program thereon, which, when executed by a processor, is used to implement the steps of the above-mentioned point cloud denoising processing method.

[0028] The present invention has the following advantages:

[0029] As described above, this invention discloses a point cloud denoising method. This method utilizes the Alternating Direction Multiplier Method (ADMM) to solve an optimized model during point cloud denoising, and integrates deep algorithm unrolling to unfold the ADMM algorithm into a deep neural network. This achieves a point cloud denoising method that balances interpretability and powerful representation capabilities through a model-data dual-driven approach. The point cloud denoising problem is essentially a convex quadratic programming (QP) problem with an l2-norm graphical smoothing prior, known as graph Laplacian regularization, which facilitates piecewise planar signal reconstruction. To solve the proposed unconstrained QP problem, this invention does not directly use inversion algorithms or conjugate gradient algorithms with fewer parameters. Instead, it introduces auxiliary variables to expand the degrees of freedom of the parameters and designs a set of ADMM algorithms accordingly. These algorithms are then unfolded into a feedforward network of variable complexity, and parameters are adjusted through backpropagation. The unfolded network has periodically inserted graph learning modules, similar to the self-attention mechanism in the transformer architecture of deep learning, to learn the inherent pairwise similar structures in the data. This allows for dynamic adjustment of the balance between the data fidelity term and the graph regularization term in the objective function during ADMM iterations. The expansion constructs the iterative algorithm of the mathematical model into a neural network structure, organically combining traditional optimization methods with deep learning methods, while ensuring the denoising effect and robustness of the proposed method. Simultaneously, in the first iteration layer, the invention initializes the metric matrix required for feature map learning using a generalized bilateral filter. This initialization process is embedded in the ADMM iterative framework and performed implicitly, providing stable initial denoising results and avoiding the instability of random initialization in traditional deep learning. It also provides a high-quality starting point for subsequent feature map structure learning, ensuring stable and effective denoising. Furthermore, the invention introduces graph structure learning (i.e., feature map learning), using deep features extracted by the feature extraction network and a learnable metric matrix to calculate weights, and then constructing a Laplacian matrix for training the parameters and metric matrix. Training the metric matrix reduces the number of parameters, and the metric matrix can learn to filter features, shortening training time and allowing the parameters to reach their optimal values ​​more quickly. In addition, this invention also designs a lightweight feature extraction model based on PointNet++. Due to the lightweight design, the number of parameters required for training is far less than that required for most current pure data-driven methods, and the required computing resources are minimal, which significantly improves the computational efficiency. Attached Figure Description

[0030] Figure 1 This is a flowchart of the point cloud denoising processing method in Embodiment 1 of the present invention;

[0031] Figure 2This is a network structure diagram of the lightweight feature extraction model based on PointNet++ improvement in Embodiment 1 of the present invention;

[0032] Figure 3 A visual diagram showing the comparison between the method of this invention and the comparative method;

[0033] in Figure 3 In the middle, (a) and (h) are noisy point clouds and clean point clouds, respectively; (b), (c), (d), (e), (f), and (g) are schematic diagrams of BF, MRPCA, LDMMGLR, FGL, SBD methods and the processed point cloud of the present invention, respectively. Detailed Implementation

[0034] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0035] Example 1

[0036] like Figure 1 As shown in the figure, this embodiment describes a point cloud denoising method, which includes the following steps:

[0037] Step 1. Obtain noisy point cloud data and divide the described point cloud into multiple patches, each containing multiple points.

[0038] For a given noisy point cloud .

[0039] in For the first The three-dimensional coordinates of each point , This indicates the number of points contained in the noisy point cloud.

[0040] The goal of point cloud denoising is to remove outliers and restore a clean point cloud. To reduce the computational cost of sparse graph processing, the noisy point cloud is divided into multiple patches. The data is fed into the network model described in step 2 below for learning.

[0041] make The coordinate matrix representing the point cloud, and the matrix affected by noise. Represented as:

[0042] .

[0043] In the formula The coordinate matrix representing the clean, original point cloud. This represents the zero-mean noise matrix.

[0044] For noisy point clouds Random sampling method is used to select points from the point cloud. One point serves as the patch center, among which Then use The nearest neighbor algorithm is used to construct patches, and the process is as follows:

[0045] For each patch center In noisy point clouds Search for it in The nearest neighbor points are used to form a noise patch;

[0046] In conjunction with noisy point clouds Corresponding clean original point cloud In the middle, find the same patch center of The nearest neighbor points are used to form a clean patch; among them This represents a scaling factor, or multiple, used to control the size of the clean patch relative to the noisy patch. All values ​​are greater than or equal to 1, with the aim of obtaining broader local structural information in clean point clouds.

[0047] Each patch All contain The three-dimensional coordinates of each point constitute The characteristic matrix of .

[0048] Step 2. Solve using the Alternating Direction Multiplier Method (ADMM) and expand the network using a deep algorithm, transforming the ADMM iterative optimization process into an end-to-end trainable network.

[0049] This network is an Expanded Plug-and-Play Graph Laplacian Regularized (UPnPGLR) network. The UPnPGLR network contains multiple iterative layers, each corresponding to one iteration of ADMM.

[0050] Building an optimization model, the denoising problem of point cloud blocks is a maximum a posteriori problem:

[0051] (1)

[0052] In the formula matrix Represents the observed noisy point cloud, matrix This represents a clean, raw point cloud. Represents the graph Laplace matrix. The regularization parameter is and , This represents the Frobenius norm.

[0053] In the above formula (1), the first term is the data fidelity term, which is used to measure the difference between the observed point cloud and the recovered point cloud, and the second term is the graph Laplace regularization term, which is used to promote the piecewise constant reconstruction of the point cloud.

[0054] To solve the problem in equation (1) above, an auxiliary variable is introduced. and constraints The ADMM algorithm is used to solve this problem.

[0055] Therefore, equation (1) above can be rewritten using the augmented Lagrange method as follows:

[0056] (2)

[0057] In the formula, It is a Lagrange multiplier matrix. This is the penalty parameter; the ADMM iteration steps are as follows:

[0058] (3)

[0059] (4)

[0060] (5)

[0061] In the formula, , , ;

[0062] express No. The result of X in the nth iteration, i.e. the nth The input for the next iteration;

[0063] express No. The Z result of the nth iteration, i.e. The input for the next iteration;

[0064] express No. The Z result of the nth iteration, i.e. The input for the next iteration;

[0065] express No. The result of the Λ iteration, i.e. the Λ of the . The input for the next iteration;

[0066] express No. The result of the Λ iteration, i.e. the Λ of the . The input for the next iteration.

[0067] Unfold ADMM into a neural network and set... Each iteration layer solves equations (3), (4), and (5) in each iteration, and uses the denoised point cloud output of the current iteration layer as the input of the next iteration layer until the iteration ends.

[0068] Specifically, each patch from step 1 is input into the network to begin iterative optimization of ADMM. Then, steps 3 through 5 are performed in each iteration until the iteration ends, and then proceed to step 6.

[0069] The Expanded Graph Laplacian Regularized Network (UPnPGLR) expands the iterative optimization algorithm into deep network layers and embeds Graph Laplacian Regularized (GLR) into a deep learning framework. This network leverages the powerful expressiveness of deep neural networks to enhance the prior modeling of GLR, while incorporating GLR, an effective denoising prior, into the network structure. This limits the network's solution space, avoids overfitting, and yields the final denoised point cloud. UPnPGLR maintains both the theoretical robustness and interpretability of model-driven methods and the powerful learning capabilities of data-driven methods, significantly improving point cloud denoising performance. Even with limited data, it can effectively remove noise and outliers from noisy point clouds, accurately restoring clean point clouds.

[0070] Step 3. Construct a connectivity graph based on the K-Nearest Neighbors (KNN) algorithm, and establish connections between each point in the patch and its K nearest neighbors to obtain point pairs.

[0071] In the first iterative layer, for each patch extracted in step 1, the three-dimensional coordinates and curvature of the points in each patch are first extracted as point features; then, these point features are processed using a generalized bilateral filter to construct a metric matrix, and the Mahalanobis distance between point pairs is further calculated to complete the initialization process.

[0072] For each patch, in each iteration layer, the problem of equation (3) is solved first, which has a closed solution:

[0073] (6)

[0074] The solution is obtained Then, the above formula is used to obtain... .

[0075] In the first iteration layer, The three-dimensional coordinates and corresponding curvature of each patch are combined to form a feature vector. , Let the coordinates of the point be 3D coordinates. Let be the curvature of the point.

[0076] Then, a generalized bilateral filter is used for initialization, and the initialization method is as follows:

[0077] According to the generalized bilateral filter, the definition of point cloud denoising is as shown in formula (7):

[0078] (7)

[0079] In the formula These are the weights of the generalized bilateral filter. Represents the spatial domain weight function, Represents the range weight function. , They represent the first , The coordinates of the points , It is the first , The curvature at a point, It is the standard deviation of the spatial domain. It is the standard deviation of the range.

[0080] Equation (7) can be expressed in the form of equation (8), that is... Includes coordinates and curvature. The feature weights, composed of corresponding coordinates and curvature, implicitly initialize the bilateral filter, thereby calculating the point pairs in each patch. and Mahalanobis distance between :

[0081] (8)

[0082] In the formula, and They represent the first The point and the first points 3D feature vectors.

[0083] structure metric matrix .

[0084] In the first iterative layer, a generalized bilateral filter is used to initialize the metric matrix required for feature map learning. This initialization process is implicitly embedded in the iterative framework of the alternating direction multiplier method, which can provide stable initial denoising results, avoid the instability of random initialization in traditional deep learning, and provide a good starting point for feature map structure learning, ensuring stable and good denoising performance.

[0085] In other iterative layers, the patch output from the previous iteration is used as input. First, the three-dimensional coordinates of the midpoint are input into the lightweight feature extraction model. Then, the feature vectors output by the feature extraction model are used to construct new point features. Finally, the feature map is used to learn the metric matrix and calculate the Mahalanobis distance between point pairs.

[0086] Taking any iteration layer other than the first iteration layer as an example, the three-dimensional coordinates of each patch in the result of equation (5) are... ,in Representing three-dimensional coordinates, used as input for lightweight feature extraction models.

[0087] In this embodiment, the lightweight feature extraction model adopts a network structure based on an improved PointNet++ lightweight feature extraction model, as follows: Figure 2 As shown, it adopts a dual-branch structure, and the specific processing flow is as follows:

[0088] In the local feature branch, the local neighborhood of each point is first constructed, then the coordinates of the neighboring points of each point relative to the point are calculated, and then the local information is aggregated through two convolutional layers and max pooling to obtain local features.

[0089] In the global context branch, two convolutional layers are applied to the 3D coordinates of the input points to extract global features.

[0090] In the local feature branch, the output channels in each convolutional layer are 32 and 64, respectively.

[0091] In the global context branch, the output channels for each convolutional layer are 64 and 128, respectively.

[0092] The local features extracted in this embodiment capture the local geometry of the point cloud by considering each point and its neighbors. This helps to identify noise points and preserve local details. Local features are invariant to global transformations, which helps the network maintain stability when processing point clouds in different poses. Local features also help to recover the fine structure of the point cloud.

[0093] Global features can capture the global structure and shape context of point clouds; global features help maintain the overall shape of point clouds and avoid distortion of the overall structure of point clouds after denoising; global features can provide guidance for local processing.

[0094] By combining local and global features, the network can simultaneously utilize local details and global context, thereby maintaining the authenticity of the local structure without destroying the overall shape during denoising, resulting in higher quality denoising results.

[0095] The local and global features are then concatenated and fed into the feature fusion layer and feature enhancement layer. Finally, the learned deep features are output as feature vectors based on the lightweight feature extraction model improved by PointNet++.

[0096] Similarly, using the Mahalanobis distance calculation expression shown in formula (8), we obtain the connection point pairs for each group. and Distance between In the formula, and They represent the first The point and the first points 3D feature vectors.

[0097] Then set a learnable metric matrix. .

[0098] ;in The feature weights are the corresponding features of the learned deep features.

[0099] In each iterative layer of the present invention, the metric matrix of the first iterative layer... It only includes feature weights corresponding to the coordinates and curvature, and other iterative layers It only includes the feature weights of the corresponding learned deep features.

[0100] In this embodiment, a graph (i.e., a feature map, which is graph structure information) is constructed based on points in the point cloud, which can preserve finer geometric details, achieve lower computational overhead, and support end-to-end learning.

[0101] This invention designs a learnable angle quantity matrix. and adaptive ADMM parameters (parameters in equation (2)) and The graph Laplacian regularization strength is automatically optimized through end-to-end training, thereby achieving a progressive denoising process guided by coordinate features.

[0102] The first iterative layer uses only coordinate and curvature features and is initialized with a bilateral filter for coarse denoising. Subsequent iterative layers, except for the first iterative layer, combine the learned deep features for fine optimization, forming a hierarchical denoising.

[0103] The method of this invention is based on a lightweight feature extraction model and a hierarchical processing mechanism (i.e., multiple iterative layers). It effectively preserves geometric structure information during the feature extraction process, achieves more reliable original point cloud recovery, and the performance degradation is negligible.

[0104] In this invention, the Alternating Direction Multiplier Method (ADMM) solves the optimization problem by alternately updating variables, and reliable initial values ​​help the ADMM algorithm start optimization from a good starting point, thereby improving the denoising performance and stability.

[0105] Step 4. Calculate the distance between nodes in each subgraph based on Mahalanobis distance, obtain the edge weights, and obtain the adjacency matrix and degree matrix of the subgraph based on the edge weights. Then, construct the graph Laplacian matrix using the definition of the graph Laplacian matrix.

[0106] Distance for each given pair of points Points can be calculated and Similarity measure between them, i.e., connection points and The edge weights of a graph can be defined as: In the formula This represents a similarity measure.

[0107] Based on the adjacency matrix and degree matrix, the graph Laplacian matrix is ​​constructed using the definition of the graph Laplacian matrix. Specifically, in this embodiment, the constructed subgraph G is defined by each vertex as a node and the connections between vertices as edges.

[0108] Given a pair of connection points and The edge weight is defined as The adjacency matrix of subgraph G It is The matrix whose first... Item for The degree matrix of the subgraph G It is a diagonal matrix, and its first... Item for .

[0109] Combinatorial Laplace Matrix Depend on get.

[0110] Step 5. Optimize each patch based on the graph Laplacian matrix and the 3D coordinates of all points, and then input each optimized patch into the next iteration layer.

[0111] Solve equation (4) using equation (9). :

[0112] (9)

[0113] in The identity matrix is ​​represented; equation (9) is solved using the conjugate gradient method (CG) iteratively, specifically as follows:

[0114] For each iteration step t, the solution is updated as follows: residual and search direction :

[0115] (10)

[0116] (11)

[0117] (12)

[0118] Among them, step size and momentum The calculation is as follows:

[0119] , ;

[0120] in It is the Graph Laplace matrix. The patch represents the three-dimensional coordinates of the point. Then, the results obtained from equations (3) and (4) are used to solve equation (5), and the resulting denoising result is input into the next iteration layer.

[0121] Step 6. Process all patches using the farthest point sampling method to obtain the final denoised point cloud.

[0122] Since the optimization of Graph Laplacian Regularized (GLR) involves large-scale sparse graph computation, this invention proposes a subgraph-based denoising strategy to reduce computational complexity.

[0123] Specifically, the complete point cloud is divided into patch Each patch is composed of a random seed point and its spatial neighborhood points, resulting in a denoised point cloud patch. Defined in a similar manner.

[0124] However, subgraph-based denoising methods inevitably introduce errors when constructing subgraphs.

[0125] To achieve effective denoising, a sub-graph G must be constructed based on the entire point cloud to fully capture global structural features and avoid information loss, especially in the boundary regions between patches.

[0126] This invention employs a patch aggregation strategy based on farthest point sampling to mitigate the impact of boundary discontinuities.

[0127] Specifically, after denoising is completed, each denoised point cloud patch is... Perform farthest point sampling, iteratively select the point farthest from the existing sampling points to uniformly cover the point cloud surface, thereby obtaining the final denoised point cloud.

[0128] Furthermore, the expanded graph Laplacian regularized network UPnPGLR uses chamfered distance as the loss function. Specifically:

[0129] For a given noisy point cloud patch Define its corresponding clean point cloud as And its denoised point cloud blocks For point cloud denoising tasks, chamfer distance (CD) is used as the loss function to evaluate each... The denoising results.

[0130] Where the chamfer distance loss function Represented as:

[0131] ;

[0132] In the formula The chamfer distance between each denoised point cloud block and its corresponding clean point cloud block is represented by the following formula:

[0133] (13)

[0134] definition for From any point in the middle The formula for the sum of minimum distances is as follows:

[0135] (14)

[0136] in , ;definition for any point in the middle The formula for the sum of the minimum distances is as follows:

[0137] (15).

[0138] This invention directly optimizes the final target by using the cd (chamfer distance) of the denoised point cloud and the clean point cloud, making the training process more stable and efficient, and producing visually smoother and geometrically more reasonable results.

[0139] Meanwhile, this invention also employs a multi-resolution training strategy and a block processing technique based on farthest point sampling, which significantly reduces the computational complexity of graph construction. Specifically, the multi-resolution training strategy refers to using point clouds with different numbers of points during the experiment, such as 10,000, 30,000, and 50,000 points.

[0140] In addition, the following experimental verification is provided to verify the effectiveness of the method of the present invention.

[0141] Figure 3The results of this invention's method and several comparative methods are shown in the test results on the PUNet dataset with 2% Gaussian noise. It is clear that the method of this invention outperforms the comparative methods in removing overall noise and better preserves the geometric structure of flat regions in the point cloud. The relevant comparative methods are as follows: BF (Bilateral Filter), MRPCA (Moving Robust Principal Compent Analysis), LDMMGLR (Low Dimensional Manifold Model Graph Laplacian Regularization) for 3D point cloud denoising based on a low-dimensional manifold model, FGL (Feature Graph Learning) for 3D point cloud denoising based on feature graph learning, and SBD (Score-based Point Cloud Denoising) for score-based point cloud denoising.

[0142] Table 1 shows a comparison of the point cloud denoising performance and parameter quantities between the method of the present invention and the comparative methods, using CD and P2M as comparison indicators. The CD and P2M indicators in Table 1 are all multiplied by 10. 4 In Table 1, M is a unit of quantity, representing millions.

[0143] Table 1. Comparison of noise reduction and parameter quantity between the method of the present invention and the comparative method.

[0144]

[0145] In Table 1, CD represents the chamfer distance. A larger distance indicates a greater difference between the two point cloud sets; a smaller distance indicates better denoising. P2M represents the distance from the point to the mesh. The point cloud was normalized to a unit sphere and tested under Gaussian noise with a standard deviation of 1%, 2%, and 2.5% of the boundary sphere radius.

[0146] As shown in Table 1, the point cloud denoising results of the method of the present invention on the PUNet dataset are better than those of the comparison method. At the same time, the method of the present invention uses very few parameters, which are fewer than those of the comparison method (SBD method), thus significantly improving the computational efficiency.

[0147] Example 2

[0148] This embodiment 2 describes a point cloud denoising system, which is based on the same inventive concept as the point cloud denoising method in embodiment 1 above.

[0149] The point cloud denoising system in this embodiment includes the following modules:

[0150] The preprocessing module is used to acquire noisy point cloud data and divide it into multiple patches, each containing multiple points;

[0151] The network building module is used to solve ADMM using the Alternating Direction Multiplier Method. It transforms the ADMM iterative optimization process into an end-to-end trainable network by unfolding, which contains multiple iterative layers, each corresponding to one iteration of ADMM. Each patch is input into the network to start the iterative optimization of ADMM.

[0152] Specifically, a connection graph is constructed based on the K-nearest neighbor algorithm, and each point in the patch is connected to its K nearest neighbors to obtain point pairs;

[0153] In the first iterative layer, for each patch extracted in step 1, the three-dimensional coordinates and curvature of the points in each patch are first extracted as point features. Then, the point features are processed using a generalized bilateral filter to construct a metric matrix and further calculate the Mahalanobis distance between point pairs to complete the initialization process.

[0154] In other iterative layers, the patches output from the previous iteration are used as input. First, the 3D coordinates of the points in each patch are input into the feature extraction model. Then, the feature vectors output by the feature extraction model are used to form new point features. Finally, the feature map is used to learn the metric matrix and calculate the Mahalanobis distance between point pairs.

[0155] The distance between nodes in each subgraph is calculated based on Mahalanobis distance, the edge weights are obtained, and the adjacency matrix and degree matrix of the subgraph are obtained based on the edge weights. Then, the graph Laplacian matrix is ​​constructed using the definition of the graph Laplacian matrix.

[0156] Based on the graph Laplacian matrix and the 3D coordinates of all points, each patch is optimized, and then each optimized patch is fed into the next iteration layer;

[0157] And the aggregation module, which uses the farthest point sampling to process all patches, finally obtains the denoised point cloud.

[0158] It should be noted that any content not mentioned in the above-described functional modules of the point cloud denoising system described in this embodiment can be referred to the step description of the corresponding method in Embodiment 1 above, and will not be elaborated here.

[0159] Example 3

[0160] This embodiment 3 describes a computer device including a memory and one or more processors. Executable code is stored in the memory. When the processor executes the executable code, it implements the steps of the point cloud denoising processing method in embodiment 1 above.

[0161] In this embodiment, the computer device can be any device or apparatus with data processing capabilities, and will not be described in detail here.

[0162] Example 4

[0163] This embodiment 4 describes a computer-readable storage medium storing a program that, when executed by a processor, is used to implement the steps of the point cloud denoising processing method in embodiment 1 above.

[0164] The computer-readable storage medium can be an internal storage unit of any device or apparatus with data processing capabilities, such as a hard disk or memory, or an external storage device of any device with data processing capabilities, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc.

[0165] Of course, the above description is only a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. It should be noted that any equivalent substitutions or obvious modifications made by those skilled in the art under the guidance of this specification fall within the scope of this specification and should be protected by the present invention.

Claims

1. A point cloud denoising method, characterized in that, Includes the following steps: Step 1. Obtain noisy point cloud data and divide it into multiple patches, each containing multiple points; Step 2. Solve using the Alternating Direction Multiplier Method (ADMM), and transform the ADMM iterative optimization process into an end-to-end trainable network by unfolding, which contains multiple iterative layers, each corresponding to one iteration of ADMM; Input each patch from step 1 into the network to begin the iterative optimization of ADMM. Then, for each iteration, execute steps 3 to 5 below until the iteration ends, and then proceed to step 6. Step 3. Construct a connection graph based on the K-nearest neighbor algorithm, and establish a connection relationship between each point in the patch and its K nearest neighbors to obtain point pairs; In the first iterative layer, for each patch extracted in step 1, the three-dimensional coordinates and curvature of the points in each patch are first extracted as point features. Then, the point features are processed using a generalized bilateral filter to construct a metric matrix and further calculate the Mahalanobis distance between point pairs to complete the initialization process. In other iterative layers, the patches output from the previous iteration are used as input. First, the 3D coordinates of the points in each patch are input into the feature extraction model. Then, the feature vectors output by the feature extraction model are used to form new point features. Finally, the feature map is used to learn the metric matrix and calculate the Mahalanobis distance between point pairs. The feature extraction model adopts a lightweight feature extraction model based on PointNet++; this lightweight feature extraction model based on PointNet++ adopts a dual-branch structure; In the local feature branch, the local neighborhood of each point is first constructed, then the coordinates of the neighboring points of each point relative to the point are calculated, and then the local information is aggregated through two convolutional layers and max pooling to obtain local features. In the global context branch, two convolutional layers are applied to the 3D coordinates of the input points to extract global features; In the local feature branch, the output channels in each convolutional layer are 32 and 64, respectively; In the global context branch, the output channels of each convolutional layer are 64 and 128, respectively; The local and global features are then concatenated and fed into the feature fusion layer and feature enhancement layer. Finally, the learned deep features are output as feature vectors based on the lightweight feature extraction model improved by PointNet++. Learn each pair of connection points using feature maps and Distance between The formula is as follows: ； In the formula and They represent the first The point and the first points 1. Diffuse eigenvectors; then set a learnable metric matrix. , ;in The feature weights corresponding to the learned deep features; Step 4. Calculate the distance between nodes in each subgraph based on Mahalanobis distance, obtain the edge weights, and obtain the adjacency matrix and degree matrix of the subgraph based on the edge weights. Then, construct the graph Laplacian matrix using the definition of the graph Laplacian matrix. Step 5. Optimize each patch based on the graph Laplacian matrix and the 3D coordinates of all points, and then input each optimized patch into the next iteration layer; Step 6. Process all patches using the farthest point sampling method to obtain the final denoised point cloud.

2. The point cloud denoising method according to claim 1, characterized in that, Step 1 specifically involves: For a given noisy point cloud ,in For the first The three-dimensional coordinates of each point , This indicates the number of points contained in the noisy point cloud, dividing the noisy point cloud into multiple patches. The data is fed into the network from step 2 for learning. For noisy point clouds Random sampling method is used to select points from the point cloud. One point serves as the patch center, among which Then use The nearest neighbor algorithm is used to construct patches, and the process is as follows: For each patch center In noisy point clouds Search for it in The nearest neighbor points are used to form a noise patch; In conjunction with noisy point clouds Corresponding clean original point cloud In the middle, find the same patch center of The nearest neighbor points are used to form a clean patch; among them This represents a scaling factor. Greater than or equal to 1; Each patch All include The three-dimensional coordinates of each point constitute The characteristic matrix of .

3. The point cloud denoising method according to claim 1, characterized in that, Step 2 specifically involves: Building an optimization model, the denoising problem of point cloud blocks is a maximum a posteriori problem: (1) In the formula matrix Represents the observed noisy point cloud, matrix This represents a clean, raw point cloud. Represents the graph Laplace matrix. Indicates the number of points contained in the noisy point cloud; The regularization parameter and , Denotes the Frobenius norm; The first term in the above formula is the data fidelity term, which measures the difference between the observed point cloud and the reconstructed point cloud. The second term is the graph Laplacian regularization term, which promotes piecewise constant reconstruction of the point cloud. To solve the problem in equation (1), an auxiliary variable is introduced. and constraints Solve using the ADMM algorithm; Therefore, equation (1) can be rewritten using the augmented Lagrange method as follows: (2) In the formula, It is a Lagrange multiplier matrix. This is the penalty parameter; the ADMM iteration steps are as follows: (3) (4) (5) In the formula, , , ; express No. The result of X in the nth iteration, i.e. the nth The input for the next iteration; express No. The Z result of the nth iteration, i.e. The input for the next iteration; express No. The Z result of the nth iteration, i.e. The input for the next iteration; express No. The result of the Λ iteration, i.e. the Λ of the . The input for the next iteration; express No. The result of the Λ iteration, i.e. the Λ of the . The input for the next iteration; Unfold ADMM into a neural network and set... Each iteration layer solves equations (3), (4), and (5) in each iteration, and uses the denoised point cloud output of the current iteration layer as the input of the next iteration layer until the iteration ends.

4. The point cloud denoising method according to claim 3, characterized in that, In step 3, the process of using a generalized bilateral filter is as follows: For each patch, in each iteration layer, the problem of equation (3) is solved first, which has a closed-form solution: (6) The solution is obtained After that, it was obtained In the first iteration layer, The three-dimensional coordinates and corresponding curvature of each patch are combined to form a feature vector. ; in Let the coordinates of the point be 3D coordinates. The curvature of a point; According to the generalized bilateral filter, the definition of point cloud denoising is as shown in formula (7): (7) In the formula These are the weights of the generalized bilateral filter; Represents the spatial domain weight function, Represents the range weight function. , They represent the first , The coordinates of the points , It is the first , The curvature at a point, It is the standard deviation of the spatial domain. It is the standard deviation of the range; Equation (7) is expressed in the form of Equation (8) below, implicitly implementing the bilateral filter, completing the initialization process, and then calculating the point pairs in each patch. and Mahalanobis distance between ; (8) In the formula, and They represent the first The point and the first points 3D feature vector; structure metric matrix .

5. The point cloud denoising method according to claim 3, characterized in that, Step 5 specifically involves: Solve equation (4) using equation (9). The formula is as follows: (9) in The identity matrix is ​​represented; equation (9) is solved using the conjugate gradient method (CG) iteratively, specifically as follows: For each iteration step t, the solution is updated according to the following formulas (10) to (12). residual and search direction : (10) (11) (12) in Indicates the step size. Indicates momentum. and The calculation is as follows: ， ； in This represents a patch, which contains the three-dimensional coordinates of the point; Then, using the results obtained from equations (3) and (4), equation (5) is solved, and the resulting denoising result is input into the next iteration layer.

6. The point cloud denoising method according to claim 1, characterized in that, In step 2, the expanded network is trained using the chamfer distance loss function, as follows: For a given noisy point cloud patch Define its corresponding clean point cloud as And its denoised point cloud blocks ; Where the chamfer distance loss function Represented as: ； In the formula The chamfer distance between each denoised point cloud block and its corresponding clean point cloud block is represented by the following formula: (13) definition for From any point in the middle The formula for the sum of minimum distances is as follows: (14) in , ; definition for any point in the middle The formula for the sum of the minimum distances is as follows: (15) 。 7. A point cloud denoising system for implementing the point cloud denoising method as described in claim 1, characterized in that, The point cloud denoising system includes the following modules: The preprocessing module is used to acquire noisy point cloud data and divide it into multiple patches, each containing multiple points; The network building module is used to solve ADMM using the Alternating Direction Multiplier Method. It transforms the ADMM iterative optimization process into an end-to-end trainable network by unfolding, which contains multiple iterative layers, each corresponding to one iteration of ADMM. Each patch is input into the network to start the iterative optimization of ADMM. Specifically, a connection graph is constructed based on the K-nearest neighbor algorithm, and each point in the patch is connected to its K nearest neighbors to obtain point pairs; In the first iterative layer, for each patch extracted in step 1, the three-dimensional coordinates and curvature of the points in each patch are first extracted as point features. Then, the point features are processed using a generalized bilateral filter to construct a metric matrix and further calculate the Mahalanobis distance between point pairs to complete the initialization process. In other iterative layers, the patches output from the previous iteration are used as input. First, the 3D coordinates of the points in each patch are input into the feature extraction model. Then, the feature vectors output by the feature extraction model are used to form new point features. Finally, the feature map is used to learn the metric matrix and calculate the Mahalanobis distance between point pairs. The distance between nodes in each subgraph is calculated based on Mahalanobis distance, the edge weights are obtained, and the adjacency matrix and degree matrix of the subgraph are obtained based on the edge weights. Then, the graph Laplacian matrix is ​​constructed using the definition of the graph Laplacian matrix. Based on the graph Laplacian matrix and the 3D coordinates of all points, each patch is optimized, and then each optimized patch is fed into the next iteration layer; And the aggregation module, which uses the farthest point sampling to process all patches, finally obtains the denoised point cloud.

8. A computer device, comprising a memory and one or more processors; characterized in that, The memory stores executable code, which, when executed by the processor, is used to implement the steps of the point cloud denoising processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a program stored thereon; characterized in that, When the program is executed by the processor, it is used to implement the steps of the point cloud denoising processing method according to any one of claims 1 to 6.

Images