A three-dimensional point cloud denoising method
By employing a Laplacian regularized subspace nonlocal low-rank learning method, the problems of over-smoothing and over-sharpening in 3D point cloud denoising are solved, achieving better visual feature preservation and quantization metrics, which is superior to existing technologies.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NANJING UNIV OF POSTS & TELECOMM
- Filing Date
- 2023-02-22
- Publication Date
- 2026-05-29
AI Technical Summary
Existing 3D point cloud denoising methods tend to over-smooth or over-sharpen when dealing with noise, making it difficult to effectively preserve visually significant structural features, and they perform poorly under high noise conditions.
We employ a Laplacian regularized subspace nonlocal low-rank learning method, which combines subspace representation, low-dimensional manifold module, subspace nonlocal low-rank factorization, and denoising model module with graph Laplacian regularizer and alternating iterative minimization strategy to compute denoised 3D point cloud images.
Under different types of noise, it significantly improves the visual feature preservation ability and quantization index of the denoised image point cloud, which is superior to existing methods.
Smart Images
Figure CN116205815B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image denoising, specifically a three-dimensional point cloud denoising method. The device and method are based on Laplacian regularized subspace nonlocal low-rank learning. Background Technology
[0002] 3D point clouds are a novel signal representation of three-dimensional objects, consisting of discrete sets of triples that mark the positions of external object surfaces in three-dimensional space. 3D point clouds have become an important and popular signal representation of three-dimensional objects. 3D point clouds can be directly acquired using low-cost depth sensors such as Microsoft Kinect, or high-resolution 3D scanners such as LiDAR. Furthermore, in recent years, multi-view stereo matching techniques have been extensively studied to recover 3D models from images or videos, with point clouds being a typical output format. However, in any case, the output point cloud contains noise, which can significantly impact subsequent computer vision research. Therefore, many denoising methods exist for noisy image point clouds.
[0003] Current point cloud denoising efforts can be categorized into four types: moving least squares (MLS) based methods, locally optimal projection (LOP) based methods, sparsity-based methods, and nonlocal similarity-based methods. MLS-based methods approximate the input samples as a smooth surface and project the points onto the generated surface. To construct the surface, the method proposed by Alexa et al. was the first moving least squares approach, best suited for finding a local reference domain for each neighboring point, and then defining a reference domain-based function by fitting a polynomial function to the neighboring data. Several extensions have also been proposed to address the unstable reconstruction problem in high curvature cases, such as Algebraic Point Set Surfaces (APSS). These methods can robustly generate smooth surfaces from noisy input images but are often criticized for oversmoothing. Unlike MLS-based methods, LOP-based methods do not compute explicit parameters of the surface. For example, the LOP-based method proposed by Lipman et al. outputs a set of points representing the underlying surface while using a repulsion term in the optimization to force a uniform distribution across the point cloud. Modifications include Weighted LOP (WLOP), which provides a more uniform output by adapting the repulsion term to the local density, and Anisotropic WLOP (AWLOP), which modifies WLOP to use anisotropic weighting functions to maintain sharp features. LOP-based methods can suffer from oversmoothing due to the use of local operators, or the generation of additional features due to noise. Sparsity-based methods are based on the assumption of local planarity and optimize for sparse representations of certain geometric features. Methods based on surface normal sparsity first use... or Regularization solves a global minimization problem to obtain a sparse reconstruction of surface normals, and then another global minimization problem is solved based on the local plane assumption, updating point positions with surface normals. A recent method called Moving Robust Principal Component Analysis (MRPCA) uses the deviation of points from a local reference plane. Minimization is used to preserve sharp features. Sparse-based methods achieve state-of-the-art performance, but in high-noise conditions, the estimation of normals or local planes can be very poor, leading to over-smoothing or over-sharpening. Nonlocal methods are widely used for image denoising. This method extends the concepts of nonlocal means and nonlocal self-similarity from the BM3D image denoising algorithm to point cloud denoising and shows better preservation of structural features at high noise levels. However, extending nonlocal image denoising schemes to point clouds is difficult due to the lack of regular structure in point clouds. Curvature-based similarity can be used for nonlocal filtering, allowing the filtering to consider the neighborhood geometry and better preserve fine shape features. Alternatively, polynomial coefficients of local MLS surfaces can be used as neighborhood descriptors to compute point similarity. Summary of the Invention
[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a device and method for denoising three-dimensional point clouds, which can obtain denoised image point clouds that retain visually significant structural features well and have good quantization indicators. This method is based on Laplacian regularized subspace nonlocal low-rank learning.
[0005] The specific technical solution adopted in this invention is as follows:
[0006] A 3D point cloud denoising device, comprising:
[0007] 3D Point Cloud Acquisition Module: Used to acquire 3D point clouds of 3D image models containing different levels of noise and their corresponding clean 3D image models using a Poisson disk.
[0008] Subspace module: used to perform subspace representation of 3D point cloud data based on the 3D point cloud of the image model, and obtain the subspace basis E;
[0009] Low-dimensional manifold module: Used to transform the problem of similarity between patches into solving patch manifolds. The issue of dimensions;
[0010] Subspace Nonlocal Low-Rank Factorization Module: Used to study the nonlocal self-similarity problem of subspace based on the principle that the high-dimensional information of image point cloud is located in the low-dimensional subspace;
[0011] The 3D Tensor Acquisition Module is used to calculate the feature image Z from non-locally similar 3D patches and generate a 3D tensor from the image composed of the feature images.
[0012] Denoising Model Module: Used for sampling alternating iterative minimization strategy to iteratively calculate and obtain the denoised 3D point cloud image.
[0013] In addition, the device includes a processor and a storage medium: the storage medium is used to store instructions; the processor is used to operate according to the instructions to perform the steps of the method described in the first aspect.
[0014] This invention also discloses a three-dimensional point cloud denoising method, which uses the above-mentioned three-dimensional point cloud denoising apparatus and includes the following steps:
[0015] Step 1: Obtain 3D image models containing different levels of noise and their corresponding clean 3D image models;
[0016] Step 2: Use a Poisson disk to sample each grid of the 3D image model to obtain a 3D point cloud;
[0017] Step 3: Perform subspace representation on the 3D point cloud data obtained in Step 2 to obtain the subspace basis E;
[0018] Step 4: Based on the low-dimensional manifold model, construct the center point set and patches of the 3D point cloud image, and transform the study of the similarity between patches into the problem of solving the dimension of the patch manifold M(U);
[0019] Step 5: Approximate the manifold dimension in the continuous domain using a patch-based graph Laplacian regularizer to calculate the manifold dimension of the 3D point cloud image;
[0020] Step 6: Based on the principle that high-dimensional information of image point clouds resides in low-dimensional subspaces, the nonlocal self-similarity of subspaces is studied using nonlocal low-rank factors.
[0021] Step 7: Calculate the feature image Z from the non-locally similar 3D patches, and generate a 3D tensor from the image composed of the feature images;
[0022] Step 8: Input the data obtained in the above steps into the denoising model, use the sampling alternating iterative minimization strategy, and iteratively calculate to obtain the denoised 3D point cloud image.
[0023] In the above technical solution, step 3, the method for subspace representation of 3D point cloud data specifically includes: setting the subspace dimension k of the subspace basis E; performing singular value decomposition on the 3D point cloud data; and obtaining the subspace basis E from the singular vectors of the 3D point cloud data.
[0024] In step 4, the study of the similarity between patches is transformed into solving the dimension problem of the patch manifold M(U). The formula for the maximum a posteriori problem is formulated using prior and fidelity terms as follows:
[0025] ,
[0026] in, It is the prior parameter of the fidelity term that is exchanged, and It is the Frobenius norm.
[0027] The specific process for calculating the manifold dimension using graph Laplacian regularization in step 5 is as follows:
[0028] First, construct a discrete graph G on the manifold, whose vertex set is an observed surface patch on M(U). ,Right now Two of the dough pieces and The edge weights between them are:
[0029] ,
[0030] in, It is a threshold Gaussian function, expressed as:
[0031]
[0032] It is two pieces of dough and The Euclidean distance between them, i.e. , It is an edge set. It is a regularization term; then the graph Laplacian regularizer GLR is used to calculate the manifold dimension of the discrete domain, while avoiding sorting the global coordinates;
[0033] The expression for the Graph Laplacian Regularizer (GLR):
[0034] ,
[0035] coordinate function on M Discrete points are obtained by sampling at position P: The expression for the graph Laplacian regularizer (GLR):
[0036]
[0037] The discrete manifold dimension is approximated using a discrete graph Laplacian regularizer, and the formula for the value of the discrete manifold dimension is as follows:
[0038] ;
[0039] The method for calculating the manifold dimension based on GLR priors is as follows:
[0040] The objective function for optimization is as follows:
[0041]
[0042] make ,but Convert to , The objective function for optimization is as follows:
[0043] ;
[0044] Regularization terms are calculated based on the local correspondences between adjacent faces, thereby eliminating the problem of global sorting. The expression is as follows:
[0045] .
[0046] A further improvement to this invention is that the specific process for constructing the 3D tensor for the denoising model in step 7 is as follows:
[0047] Step 1: Perform global low-rank factor decomposition on the point cloud;
[0048] Step 2: Define a binary operator from a mathematical perspective. This is used to find nonlocally similar overlapping 3D blocks from RCI, and to generate a 3D tensor by stitching together the 3D blocks. The formula for forming the 3D tensor is as follows:
[0049]
[0050] Where s and b are the spatial size and number of 3-D blocks, respectively. This represents each 3-D block in RCI, where i is the index of the top-left pixel. ;
[0051] Step 3: Transform each 3D block Expand to a size of 2-D matrix;
[0052] Step 4: Stack b 2-D matrices together to form a matrix of size b. 3-D tensors;
[0053] Among them, from tensor Estimating the clean tensor The expression is as follows:
[0054]
[0055] in, It is a 3D tensor along the modulus -q( Expand the matrix to find the sum of the singular values. yes The i-th singular value, and It is the noise variance of the current 3-D tensor and is used as a regularization parameter for the Frobenius norm.
[0056] A further improvement to this invention is that step 8 employs an alternating iterative minimization strategy, and the process for iteratively calculating the denoised 3D point cloud image is as follows:
[0057] Step 1: Obtain the estimated low-rank tensor Then, the denoising model becomes:
[0058] ;
[0059] Step 2: Solve the model using the augmented Lagrange multiplier method:
[0060] ;
[0061] Step 3: Update sparse noise A and optimize A: ,
[0062] Update feature image Z, optimize Z: ,
[0063] Update subspace E, optimize E:
[0064] The final denoised 3D point cloud image is ;
[0065] The denoising model expression for step 8 is as follows:
[0066] .
[0067] The beneficial effects of this invention are as follows: This invention proposes a device and method for denoising 3D point clouds. This device and method are based on Laplacian regularized subspace nonlocal low-rank learning, extending the previously proposed low-dimensional flow model of pixel blocks to surface blocks in point clouds. A graph Laplacian regularizer for the patches is used to obtain the manifold dimension of the 3D point cloud image. Then, based on the principle that the high-dimensional information of the image point cloud resides in a low-dimensional subspace, a nonlocal low-rank factor is used to study the nonlocal self-similarity of the subspace, estimating the 3D tensor generated by the nonlocally similar 3D patches. Finally, the 3D tensor is used to construct a denoising model, resulting in a denoised image with superior visual and quantization metrics. This method can be applied to denoised images containing different types of noise and other interferences, and it outperforms other existing 3D point cloud denoising methods on various datasets. Furthermore, the denoised image point cloud obtained by this method better preserves visually significant structural features, and also exhibits excellent quantization metrics. Attached Figure Description
[0068] Figure 1 This is the network framework diagram of the present invention.
[0069] Figure 2 This is a comparison image of the 3D point cloud data before and after noise removal in an embodiment of the present invention. Detailed Implementation
[0070] To enhance understanding of the present invention, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. These embodiments are only used to explain the invention and do not limit the scope of protection of the invention.
[0071] Examples, such as Figure 1 As shown, a three-dimensional point cloud denoising device includes:
[0072] 3D Point Cloud Acquisition Module: Used to acquire 3D point clouds of 3D image models containing different levels of noise and their corresponding clean 3D image models using a Poisson disk.
[0073] Subspace module: used to perform subspace representation of 3D point cloud data based on the 3D point cloud of the image model, and obtain the subspace basis E;
[0074] Low-dimensional manifold module: Used to transform the problem of similarity between patches into solving patch manifolds. The issue of dimensions;
[0075] Subspace Nonlocal Low-Rank Factorization Module: Used to study the nonlocal self-similarity problem of subspace based on the principle that the high-dimensional information of image point cloud is located in the low-dimensional subspace;
[0076] The 3D Tensor Acquisition Module is used to calculate the feature image Z from non-locally similar 3D patches and generate a 3D tensor from the image composed of the feature images.
[0077] Denoising Model Module: Used for sampling alternating iterative minimization strategy to iteratively calculate and obtain the denoised 3D point cloud image.
[0078] In addition, it includes a processor and a storage medium: the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the method.
[0079] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0080] The specific noise reduction method of this invention includes the following steps:
[0081] Step 1: Obtain 3D image models containing different levels of noise and their corresponding clean 3D image models;
[0082] Step 2: Use a Poisson disk to sample each grid of the 3D image model to obtain a 3D point cloud;
[0083] Step 3: Perform subspace representation on the 3D point cloud data obtained in Step 2 to obtain the subspace basis E;
[0084] Step 4: Based on the low-dimensional manifold model, construct the center point set and patches of the 3D point cloud image, and transform the study of the similarity between patches into the problem of solving the dimension of the patch manifold M(U);
[0085] Step 5: Approximate the manifold dimension in the continuous domain using a patch-based graph Laplacian regularizer to calculate the manifold dimension of the 3D point cloud image;
[0086] Step 6: Based on the principle that high-dimensional information of image point clouds resides in low-dimensional subspaces, the nonlocal self-similarity of subspaces is studied using nonlocal low-rank factors.
[0087] Step 7: Calculate the feature image Z from the non-locally similar 3D patches, and generate a 3D tensor from the image composed of the feature images;
[0088] Step 8: Input the data obtained in the above steps into the denoising model of the present invention, use the sampling alternating iterative minimization strategy, and iteratively calculate to obtain the denoised 3D point cloud image.
[0089] The noise reduction method in this embodiment mainly involves the following contents.
[0090] 1. Construct a network based on nonlocal low-rank learning of Laplace regularized subspaces.
[0091] Based on the low-dimensional manifold model, a set of center points and patches of 3D point cloud images are constructed. The study of the similarity between patches is transformed into solving the dimension problem of the patch manifold M(U). Then, a patch-based graph Laplacian regularizer is used to approximate the manifold dimension in the continuous domain, and the manifold dimension of the 3D point cloud image is calculated.
[0092] 1.1 Graph Laplacian Regularization Based on Low-Dimensional Manifold Model
[0093] 1.1.1 Patch Manifold
[0094] A patch manifold is a (roughly uniform) discrete sample of the 2D surface of a 3D object. Position matrix of image point cloud A noisy image point cloud can be represented as:
[0095] (1)
[0096] Where U represents the 3D point cloud of the real, noise-free image, N represents zero-mean Gaussian noise independent of the image, and A represents sparse noise independent of the image. .
[0097] In order to obtain the point cloud U of the denoised real image, this invention uses the low-dimensional manifold model prior (LDMM) as a regularization term.
[0098] (1) Surface patch
[0099] From point cloud Select M points as the center point set of the patch, i.e. Given Center of the face For point clouds Zhongyu The set of the k closest points by Euclidean distance. It is a patch of coordinates consisting of k points. The joint of the patches should cover the entire point cloud, i.e. Different noodle centers can be selected. Degrees of freedom can be used to balance computational cost and noise reduction performance.
[0100] (2) Patch manifold
[0101] Based on the assumption that patches are samples of low-dimensional manifolds in high-dimensional space, patch pairs are embedded in... The low-dimensional smooth manifold in the point cloud U is sampled; this manifold is called the patch manifold associated with the point cloud U. To assess the similarity between patches, the patches first need to be aligned, for example... It should be relative to the center Translate it so that it is located at the origin (0,0,0).
[0102] (3) Priors of low-dimensional patch manifolds
[0103] The prior assumptions of low-dimensional manifold models (LDMMs) include patches that minimize the manifold dimension. Therefore, this invention uses prior and fidelity terms to formulate the maximum a posteriori (MAP) problem, as follows:
[0104] (2)
[0105] in, It is the prior parameter of the fidelity term that is exchanged, and It is the Frobenius norm. Since the patch is determined by the point cloud U, and the patch itself defines the patch manifold. ,so, It is a function of the point cloud U.
[0106] The patches can be different and can be sampled from different manifolds of different dimensions. The dimension of the patch manifold. It is also a function of the point cloud U, and exist The integral over is used as a regularization term, and formula (2) can be expressed as:
[0107] (3)
[0108] in, It is at point place Dimensions yes The point above is calculated according to the above formula. .
[0109] 1.1.2 Graph Laplace Regularization
[0110] (1) Calculate the dimension of a manifold in a continuous domain
[0111] Based on existing research on the computational dimension of manifolds in continuous domains, we first assume... For embedding in The coordinate function on the manifold M, i.e. At point place The dimension expression is as follows:
[0112] (4)
[0113] in, Indicates at point coordinate function on M The gradient of equation (4) is integrated over M:
[0114] (5)
[0115] This expression is an integral along different dimensions over a continuous manifold M. The usual solution is to perform the integral separately for each dimension. The partial differential equation (PDE) provides a dimension-minimizing solution. This equation is discretized using patch observations from PIM and solved via a linear system, but PIM requires patch coordinates. To define by sorting However, the surface patches in a 3D point cloud are unstructured, and there is no specific method to achieve global coordinate sorting of all patches. This invention uses a Graph Laplacian Regularizer (GLR) to discretize the manifold dimensions, eliminating the need for global sorting.
[0116] (2) Discretization of manifold dimension based on GLR
[0117] First, a discrete graph is constructed on the manifold. Then, a graph Laplacian regularizer (GLR) is used to compute the manifold dimension of the discrete domain, while avoiding global coordinate sorting. (Constructing a discrete graph on the manifold...) Its vertex set is Observed surface patch ,Right now Two of the dough pieces and The edge weights between them are:
[0118] (6)
[0119] in It is a threshold Gaussian function, expressed as:
[0120] (7)
[0121] It's two pieces of dough. and The Euclidean distance between them, i.e. , It is an edge set. This is a regularization term. The constructed graph is an r-neighborhood graph, meaning the distance between any two faces is less than r. , It is a constant.
[0122] Use edge weights Define a symmetric adjacency matrix , can be The conclusion is The value of . D represents the angle matrix, Therefore, the Laplace matrix of the constructed graph is... The Graph Laplace regularizer (GLR) can be expressed as follows:
[0123] (8)
[0124] coordinate function on M Discrete points are obtained by sampling at position P: The formula for the Graph Laplace regularizer GLR is transformed from (8) into the following formula:
[0125] (9)
[0126] The discrete manifold dimension is approximated using a discrete graph Laplace regularizer, and the formula for the value of the discrete manifold dimension is as follows:
[0127] (10)
[0128] in, It is the volume of manifold M. It is the dimension of the manifold, and the approximation holds if M is true. and A constant. Formula (10) represents the neighborhood distance as M increases. convergence, It approximates the dimension of a continuous manifold. Furthermore, if the dimension of the manifold... The lower the dimensionality, the better the approximation of the continuous regularization function, even when a low-dimensional manifold is embedded in a high-dimensional space.
[0129] Therefore, given a point cloud coordinate... At that time, a discrete graph can be constructed using formula (10), and the dimension of M can be approximated using a discrete graph Laplace regularizer:
[0130] (11)
[0131] 1.1.3 Obtaining the Graph Laplacian Matrix Based on Local Correspondences of Adjacent Faces
[0132] By approximating the dimension of a continuous manifold using a discrete graph Laplacian regularizer, an approximate value for the dimension of a discrete manifold can be obtained, but it is still necessary to consider the face coordinates. Sort in order to define And based on the distance of the facets Determine the similarity of the facets.
[0133] This invention selects a regularization term that can be calculated based on the local correspondence between adjacent faces, thereby eliminating the problem of global sorting. Therefore, the manifold dimension in formula (11) can also be expressed as follows:
[0134] (12)
[0135] According to this formula, we can derive Laplacian regularization is not necessary for computational graphs, therefore global coordinate sorting is not required, and the weights are obtained through (6). yes The function is such that we only need to find the local correspondence between adjacent patches to give the result. Then the dimension of the manifold can be calculated.
[0136] To transform formula (12) into matrix form, the adjacent face pairs forming the subgraph are first re-formulated. Then, the weights between the face pairs are added together to obtain the final Graph Laplacian Regularized (GLR).
[0137] For adjacent faces and ,set up yes and Concatenation of coordinates, where Represents 3D coordinates. Given and The local correspondences between them are used to construct a subgraph by connecting the corresponding points, and then linked to the edge weights. Multiplying them together yields the Laplace matrix of the subgraph. . The expression can be converted into the following formula:
[0138] (13)
[0139] make From Extract The sampling matrix, where, It is the coordinate vector of the points in all the patches, which can be denoted as: ,but It can be converted into the following formula:
[0140] (14)
[0141] The manifold dimension of formula (12) can be transformed into the following formula:
[0142] (15)
[0143] (16)
[0144] in, It is the Laplacian matrix of the overall point cloud.
[0145] 1.1.4 Using GLR priors to calculate manifold dimensions
[0146] For formula (16) Optimization objective function:
[0147] (17)
[0148] make ,but Can be converted , 3D samples of real images The relationship is as follows:
[0149] (18)
[0150] in, It is a sampling matrix used to sample point clouds. Select points to form M facets, each facet containing k 3D points. The center of the surface. Therefore, for The optimization formula can be written as follows:
[0151] (19)
[0152] 1.2 Subspace Nonlocal Low-Rank and Sparse Regularization Model
[0153] Under the assumptions of Gaussian noise and sparse noise, the general point cloud image denoising model can be written as follows:
[0154] (20)
[0155] in Describing the Frobenius norm, This represents a data consistency constraint, measures the approximation between the 3D point cloud of a real, noise-free image and the point cloud of a blurred image, and removes Gaussian noise W. Regularization related to U, such as nuclear norm and non-convexity. - Norms, etc. Used to suppress sparse noise. Parameters The role of this is to combine these three factors and balance their impact on the final result. The performance of the denoising model depends to a large extent on the choice of regularization for the prior information of the 3D point cloud of the real noise-free image.
[0156] 1.2.1 Global Low-Rank Factor Decomposition of Point Clouds
[0157] Because point cloud data of images have strong correlations, this invention assumes that the 3D point cloud of a real, noise-free image is... (Each row corresponds to a point cloud center) located in a k-dimensional subspace Among them Therefore, the 3D point cloud matrix of a true noise-free image can be decomposed in the following form:
[0158] (twenty one)
[0159] in, It is a subspace The basis, matrix These are the representation coefficients of U with respect to E. Assume the subspace E is orthogonal, i.e. (It is an identity matrix of size k). Furthermore, the subspace E can learn data using the HySime algorithm or singular value decomposition (SVD). Based on existing literature on spatial basis research, the orthogonality of E can reduce model complexity and accelerate algorithm convergence. According to formula (20), the noisy image point cloud can be expressed as:
[0160] (twenty two)
[0161] Add orthogonal constraint terms to subspaces Then, the general point cloud image denoising model can be transformed into the following formula:
[0162] (twenty three)
[0163] in It is a data consistency constraint that measures the approximation between the 3D point cloud of a real, noise-free image and the point cloud of a blurred image, and removes Gaussian noise W. It is a regularization term related to the coefficient matrix Z. Used to remove sparse noise A. The advantage of this model is that it can simultaneously uncover the correlations between point cloud data and reduce computational costs.
[0164] 1.2.2 Spatial Nonlocal Low-Rank Factorization
[0165] coefficient matrix This can be obtained by projecting the subspace E onto the image point cloud U, i.e. Rearranging each row of matrix Z into a matrix of size m×n in the image space, this matrix is called the feature image. All feature images together form the Representation Coefficients Image (RCI). The RCI has two important prior information that is beneficial for denoising: the nonlocal similarity in each feature image and the correlation between feature images. This invention uses nonlocal techniques to utilize both of these properties simultaneously.
[0166] Next, we will explain in detail the construction of the 3D tensor used for the denoising model. The specific process of constructing the 3D tensor includes the following four steps:
[0167] 1) Subspace Identification (SI);
[0168] 2) Finding similar 3D blocks (Grouping);
[0169] 3) Unfold each 3-D block;
[0170] 4) Stack all 3-D blocks.
[0171] Step 1: Perform global low-rank factor decomposition on the point cloud;
[0172] Step 2: Define a binary operator from a mathematical perspective This is used to find nonlocally similar overlapping 3D blocks from RCI, and to generate a 3D tensor by stitching together the 3D blocks. The formula for forming the 3D tensor is as follows:
[0173] (twenty four)
[0174] Where s and b are the spatial size and number of 3-D blocks, respectively. This represents each 3-D block in RCI, where i is the index of the top-left pixel. Operator The operation involves two steps: First, all regions on the RCI are scanned with step sizes of 1 and 3 respectively, and all overlapping blocks and reference blocks are extracted from the RCI. Second, for each given reference block, a set of overlapping blocks that are most similar to it in its neighborhood are found based on Euclidean distance. The purpose of extracting overlapping blocks is to avoid artifacts in the reconstructed image.
[0175] Step 3: Divide each 3-D block Expand to a size of 2-D matrix;
[0176] Step 4: Stack the b 2-D matrices together to form a matrix of size b. 3-D tensor.
[0177] From a mathematical perspective, the goal is to estimate a clean tensor from a tensor by solving the following formula. :
[0178] (25)
[0179] in, It is a 3D tensor along the modulus -q( Expand the matrix to find the sum of the singular values. yes The i-th singular value, and It is the noise variance of the current 3-D tensor and is used as a regularization parameter for the Frobenius norm.
[0180] 1.2.3 Subspace Nonlocal Low-Rank and Sparse Regularization Model
[0181] Based on the above analysis, the point cloud image denoising model with subspace nonlocal low-rank and sparse regularization terms proposed in this invention can be expressed as:
[0182] (26)
[0183] in, It can be seen that noise reduction consists of three parts:
[0184] (1) Utilize low-rank decomposition of the image in the subspace;
[0185] (2) Utilize nonlocal low-rank decomposition to simultaneously mine nonlocal similarity in RCI and correlation between feature images;
[0186] (3) Use norm constraints to regularize the sparse noise.
[0187] This invention employs an alternating iterative minimization strategy to solve the denoising model. The entire optimization process includes the following two sub-problems:
[0188] (1) Low-rank decomposition of 3-D tensors
[0189] optimization With other variables fixed, the expression is as follows:
[0190] (27)
[0191] The purpose of this model: From The middle estimate optimal value Among them, binary operators The method involves extracting non-local similar blocks from the feature image Z to form a 3D tensor. In the experiments, a low-rank decomposition algorithm based on SVD is used. For ease of representation, details are omitted in this algorithm. The subscript i, and Tensor The matrix expanded along the q-th direction. The SVD of matrix U is... Therefore, we can obtain ,in , It is a shrinkage / threshold function.
[0192] (2) RCI noise reduction
[0193] Obtain the estimated low-rank tensor Then, the denoising model becomes:
[0194] (28)
[0195] Solve this model using the augmented Lagrange multiplier method, which involves minimizing the following model:
[0196] (29)
[0197] in, It is an indicator function. Solving this model is equivalent to iteratively solving the following three subproblems until the result converges. The three subproblems are:
[0198] Update sparse noise A: Optimize A
[0199] (30)
[0200] in, It is a soft threshold operator;
[0201] Update feature image Z: Optimize Z
[0202] (31)
[0203] This is a quadratic optimization problem. We can take the derivative with respect to Z. When the derivative is 0, it can be simplified to:
[0204] (32)
[0205] Update subspace E: Optimize E
[0206] (33)
[0207] in, and Separate matrices The left singular vector and the right singular vector.
[0208] Final denoised image point cloud .
[0209] 1.2 A 3D Point Cloud Denoising Method Based on Laplacian Regularized Subspace Nonlocal Low-Rank Learning
[0210] The graph-based low-dimensional manifold model extends the proposed low-dimensional manifold model of image patches to point clouds. It uses block manifold priors to find self-similar patches, ensuring that structural features are well preserved even when removing noise from images at high noise levels. Furthermore, the original PIM method is replaced with a graph Laplacian regularizer that eliminates the need for ordering coordinate functions.
[0211] The subspace-based nonlocal low-rank sparse factor decomposition model explores the principle of placing the point cloud features of pixels in a low-dimensional subspace. It considers the nonlocal self-similarity of space and uses the continuous singular value decomposition low-rank factor decomposition algorithm to estimate the three-dimensional (3-D) tensor generated by nonlocal similar three-dimensional patches. The augmented Lagrangian method is used to effectively solve various noises in the image.
[0212] Combining the two models above, namely formula (19) and formula (26), we obtain the overall denoising model:
[0213]
[0214] The manifold dimension is calculated using the Graph Laplacian Regularizer (GLR) prior, and then each facet is denoised to obtain the denoised image.
[0215] Using the low-rank decomposition algorithm of SVD, the tensor data is projected to the point cloud dimension for denoising, projected to the spatial dimension for denoising within and between image patches, and finally projected back to the original dimension to obtain the estimated tensor.
[0216] The estimated values are then incorporated into the denoising model, and the augmented Lagrangian method is used to iteratively solve for the optimal solutions for the sparse noise A, the feature image Z, and the subspace E. Finally... The denoised 3D point cloud image is obtained.
[0217] 2. Training a super-resolution model for light field images based on flow prior and polarity compensation.
[0218] 2.1 3D Point Cloud Image Dataset
[0219] To train the created model, this invention chose to use the ShapeNetCore dataset and learn the parameter settings. The ShapeNetCore dataset is a subset of the ShapeNet dataset, containing 55 object categories and 52,491 unique 3D models. The denoising model created in this invention was trained using data from the 10 most common categories, with 10 models randomly selected from each category for testing, thus using a total of 100 models. Furthermore, the 3D models in the ShapeNetCore dataset are low-polygon meshes, and point clouds were obtained by sampling approximately 30,000 points on each mesh using a Poisson disk.
[0220] 2.2 Training the Model
[0221] The proposed 3D point cloud denoising method based on subspace nonlocal low-rank and graph low-dimensional flow models is compared with existing methods for 3D point cloud denoising: APSS, RIMLS, AWLOP, nonlocal denoising algorithms, and state-of-the-art MRPCA and LR. APSS and RIMLS are implemented using MeshLab software, AWLOP using EAR software, MRPCA source code provided by the authors, and NLD and LR implemented in MATLAB. First, the parameters of seven models are empirically tuned on a small dataset. Then, the parameter settings learned from the small dataset are generalized to a larger dataset, namely 100 models in the ShapeNetCore dataset, for validation. A detailed comparison with existing methods on both datasets is provided below.
[0222] Three evaluation metrics are used to evaluate point cloud denoising models. Assume the clean image point cloud and the denoised image point cloud are... , ,in Point clouds can have different sizes, i.e. and They don't have to be equal.
[0223] The evaluation indicators are defined as follows.
[0224] 1) Mean Squared Error (MSE): First, the average of the squared Euclidean distances between the clean image point cloud and the denoised image point cloud is calculated. Then, the average of the two measurements is taken to calculate the MSE. The expression for MSE is:
[0225] (34)
[0226] 2) Signal-to-noise ratio (SNR): The unit of measurement for SNR is dB, and the expression is as follows:
[0227] (35)
[0228] 3) Mean Patch Distance (MCD): MCD is similar to MSE, which will... norm replacement Norm, expressed as follows:
[0229] (36)
[0230] like Figure 2 The figure shows a comparison of the denoising effect of three-dimensional point cloud data before and after the denoising in an embodiment of the present invention. It can be seen that the denoised image point cloud obtained by using this method can better preserve the visually significant structural features, and the quantization index is also very good.
[0231] The above description is an exemplary embodiment of the present invention and does not limit the scope of patent protection of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.
Claims
1. A method for denoising three-dimensional point clouds, characterized in that, Includes the following steps: Step 1: Obtain 3D image models containing different levels of noise and their corresponding clean 3D image models; Step 2: Use a Poisson disk to sample each grid of the 3D image model to obtain a 3D point cloud; Step 3: Perform subspace representation on the 3D point cloud data obtained in Step 2 to obtain the subspace basis E; Step 4: Based on the low-dimensional manifold model, construct the center point set and patches of the 3D point cloud image, and transform the study of the similarity between patches into the problem of solving the dimension of the patch manifold M(U); Step 5: Approximate the manifold dimension in the continuous domain using a patch-based graph Laplacian regularizer to calculate the manifold dimension of the 3D point cloud image; Step 6: Based on the principle that high-dimensional information of image point clouds resides in low-dimensional subspaces, the nonlocal self-similarity of subspaces is studied using nonlocal low-rank factors. Step 7: Calculate the feature image Z from the non-locally similar 3D patches, and generate a 3D tensor from the image composed of the feature images; Step 8: Input the data obtained in Step 7 into the denoising model, and use an alternating iterative minimization strategy to iteratively calculate and obtain the denoised 3D point cloud image; In step 4, the study of similarity between patches is transformed into solving the dimension problem of the patch manifold M(U). The formula for the maximum a posteriori problem is formulated using prior and fidelity terms as follows: ， in, It is the prior parameter of the fidelity term that is exchanged, and It is the Frobenius norm; The specific process for calculating the manifold dimension using graph Laplacian regularization in step 5 is as follows: First, construct a discrete graph G on the manifold, whose vertex set is an observed surface patch on M(U). ,Right now Two of the dough pieces and The edge weights between them are: ， in, It is a threshold Gaussian function, expressed as: ， It's two pieces of dough. and The Euclidean distance between them, i.e. , It is an edge set. It is a regularization term; then the graph Laplacian regularizer GLR is used to calculate the manifold dimension of the discrete domain, while avoiding sorting the global coordinates; The expression for the Graph Laplacian Regularizer (GLR) is: ， coordinate function on M Discrete points are obtained by sampling at position P: The expression for the graph Laplacian regularizer (GLR): ， The discrete manifold dimension is approximated using a discrete graph Laplacian regularizer, and the formula for the value of the discrete manifold dimension is as follows: ； The method for calculating manifold dimensions based on GLR priors is as follows: The objective function for optimization is as follows: ， make ,but Convert to , The objective function for optimization is as follows: ； The specific process for constructing the 3D tensor for the denoising model in step 7 is as follows: Step 1: Perform global low-rank factor decomposition on the point cloud; Step 2: Define a binary operator from a mathematical perspective. This is used to find nonlocally similar overlapping 3D blocks from RCI, and to generate a 3D tensor by stitching together the 3D blocks. The formula for forming the 3D tensor is as follows: ， Where s and b are the spatial size and number of 3-D blocks, respectively. This represents each 3-D block in RCI, where i is the index of the top-left pixel. ; Step 3: Transform each 3D block Expand to a size of 2-D matrix; Step 4: Stack b 2-D matrices together to form a matrix of size b. 3-D tensors; Among them, from tensor Estimating the clean tensor The expression is as follows: ， in, It is a 3D tensor along the modulus -q( Expand the matrix to find the sum of the singular values. yes The i-th singular value, and It is the noise variance of the current 3-D tensor and is used as a regularization parameter for the Frobenius norm; The denoising model expression for step 8 is as follows: 。 2. The three-dimensional point cloud denoising method according to claim 1, characterized in that, In step 3, the method for subspace representation of 3D point cloud data specifically includes: setting the subspace dimension k of the subspace basis E; performing singular value decomposition on the 3D point cloud data; and obtaining the subspace basis E from the singular vectors of the 3D point cloud data.
3. The three-dimensional point cloud denoising method according to claim 2, characterized in that, The regularization term is calculated based on the local correspondence between adjacent faces, thereby eliminating the problem of global sorting. The expression is as follows: 。 4. The three-dimensional point cloud denoising method according to claim 3, characterized in that, In step 8, an alternating iterative minimization strategy is used. The process of iteratively calculating the denoised 3D point cloud image is as follows: Step 1: Obtain the estimated low-rank tensor Then, the denoising model becomes: ； Step 2: Solve the model using the augmented Lagrange multiplier method: ； Step 3: Update sparse noise A and optimize A: , Update feature image Z, optimize Z: , Update subspace E, optimize E: , The final denoised 3D point cloud image is .