A directed point cloud denoising method based on self-learning

Abstract

The present invention relates to the technical field of point cloud denoising, and more specifically to a self-learning-based directed point cloud denoising method. Based on a differentiable Poisson solver, the directed point cloud is first mapped to a grid domain. Self-training and optimization are performed on the grid domain using more sufficient geometric information. The loss function is then transmitted back to the point cloud domain based on its differentiable properties, and the points in the point cloud domain are moved and optimized to achieve denoising. Denoising is achieved by mapping the denoising problem in the point cloud domain to the self-adjustment problem in the grid domain. During the self-learning training process, resampling steps are used to optimize the local normal consistency of the directed point cloud. At the same time, the multi-resolution loss function is redefined so that denoising can achieve a balance between local and global information.

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud denoising, and more specifically to a directed point cloud denoising method based on self-learning. Background Art

[0002] With the continuous development of 3D acquisition equipment and 3D reconstruction technology, various types of 3D input data are becoming increasingly accessible, and 3D point cloud data is being widely used in a variety of fields. Non-contact point cloud data acquisition methods, due to their high precision, high resolution, and sampling speed, have very important applications in fields such as autonomous driving, industrial production, robotics, and medicine. Point cloud data obtained using 3D laser scanning equipment is disordered and scattered. Furthermore, due to factors such as the surface roughness of the target object, equipment accuracy, and ambient lighting, the obtained 3D point cloud is inevitably affected by noise points. Noise points not only seriously affect subsequent feature extraction, matching, and reconstruction accuracy, but also affect the recognition rate of point cloud data. Therefore, how to effectively remove noise points from point cloud data while obtaining a complete and usable point cloud has been a focus of many researchers.

[0003] Traditional point cloud denoising methods usually use filters. By manually defining the parameters of the filter, the filter can effectively remove point cloud noise. However, defining the parameters requires a considerable amount of knowledge, and the complexity of point cloud data and the unknown nature of noise make the definition of filter parameters very difficult, resulting in poor generalization performance. With the development of artificial intelligence, many methods based on deep neural networks have been proposed. In point cloud denoising technology based on deep neural networks, the neural network takes a noisy point cloud as input and directly outputs a denoised point cloud. This simple way of using neural networks makes the performance of the method rely on the network's own learning ability, which puts tremendous pressure on the training of the neural network and leads to a large demand for training data.

[0004] If we can eliminate the reliance on manually defined parameters and large training data sets and fully exploit the information of the 3D point cloud data itself, we can obtain more application scenarios.

[0005] The present invention seeks to address these and other unresolved needs in the art. Summary of the Invention

[0006] In order to solve at least one of the technical problems mentioned in the above background technology, the purpose of the present invention is to provide a directed point cloud denoising method based on self-learning, which maps the denoising problem in the point cloud domain to the self-adjustment problem in the grid domain to achieve denoising. During the self-learning training process, the resampling step is used to optimize the local normal consistency of the directed point cloud, and at the same time, the multi-resolution loss function is redefined so that denoising can achieve a balance between local and global information.

[0007] In one aspect, the present invention is directed to a directed point cloud denoising method based on self-learning. On the basis of a differentiable Poisson solver, the directed point cloud is first mapped to a grid domain, and more sufficient geometric information is used on the grid domain for self-training and optimization. The loss function is then transferred back to the point cloud domain based on the differentiable property, and the points in the point cloud domain are moved and optimized to achieve denoising. The method of the present invention proposes a point cloud denoising method based on self-learning based on a differentiable Poisson solver. The method maps the denoising problem in the point cloud domain to the self-adjustment problem in the grid domain. The method does not require pre-training. The directed point cloud is first mapped to a grid domain, and more sufficient geometric information is used on the grid domain for self-training and optimization. The loss function is then transferred back to the point cloud domain based on the differentiable property, and the points in the point cloud domain are moved and optimized to achieve the purpose of denoising.

[0008] In a specific embodiment, the method utilizes a grid domain resampling algorithm to ensure the local consistency of point cloud normals, and combines multi-resolution point clouds in the loss function, so that the denoising work can achieve a balance between local and global information.

[0009] In a specific embodiment, the directed point cloud denoising method based on self-learning specifically includes the following steps:

[0010] Step 1: Use the Shape As Points algorithm as a differentiable Poisson equation solver to map the directed point cloud to the grid domain;

[0011] Step 2: Copy the input noisy directed point cloud P to obtain P gt , P to P gt The learning process is called self-training. gt Use the differentiable Poisson equation solver to transform to the grid domain, and obtain the grids M and M gt , the chamfer distance is used to calculate the loss function between the mesh vertices in the two mesh domains, and the obtained loss function is back-propagated to the point cloud domain, so as to optimize the points in the point cloud domain, and perform denoising after several self-training.

[0012] Using a concept similar to the domain Fourier transform, the point cloud problem is transformed into a grid domain for solution, transforming the denoising problem of directed point clouds into a self-adjustment problem in the grid domain. Specifically, a differentiable Poisson equation solver is used to solve the problem of converting directed point clouds from the point cloud domain to the grid domain. The differentiability of this process is exploited to optimize the midpoints in the point cloud domain using changes in the grid domain, and denoising is achieved through a self-training process.

[0013] In a specific embodiment, the step one specifically includes: using the Shape As Points algorithm as a differentiable Poisson equation solver, first establishing the Poisson equation, then using the differentiable Poisson surface reconstruction solver (DPSR) to solve the implicit field, and then converting the implicit field into a specific grid through Marching Cubes.

[0014] In a specific embodiment, the step 1 further includes: using Fourier transform to accelerate the solution in a differentiable Poisson surface reconstruction solver;

[0015] Each step in the entire transformation process is differentiable. It is necessary to find the loss in the grid domain. The purpose of learning is achieved by finding the partial derivative of the loss for the point p in the grid domain M. Suppose the grid vertex in the grid domain is p mesh , the implicit field is χ, then according to the chain rule:

[0016]

[0017] in Solved by a differentiable Poisson solver, Approximate mesh surface normals solved using Marching Cubes.

[0018] In this way, we can transfer the adjustment of the grid domain to the point cloud domain in a differentiable form, thereby driving the denoising work of the entire point cloud.

[0019] In a specific embodiment, the loss function in step 2 is:

[0020]

[0021] Where p is a point in the grid domain M, q is a point in the grid domain M gt For points in , dis(p,q) is the distance between p and q.

[0022] In a specific embodiment, the method further includes step three, periodically using a resampling strategy from the grid domain during the self-training rounds of step two to constrain the neighborhood normal consistency of the directed point cloud.

[0023] In a specific embodiment, the resampling strategy in step three is: interspersing the resampling method in the middle of a certain training round, first converting the current point cloud to the grid domain, and then considering uniformly sampling a number of points from the grid in the grid domain and returning them to the point cloud domain to continue the training round.

[0024] However, during the denoising process, due to the randomness of the noise, the directed point cloud will lose its local normal consistency. In this process, the resampling step is used to optimize the local normal consistency of the directed point cloud, thereby bringing good denoising effect.

[0025] In a specific embodiment, the method further includes step 4 of multi-resolution loss calculation, wherein for the noise point cloud P in training, downsampling is performed to P1, P2, P3, ... at several ratios r1, r2, r3, ... We redefine the loss function as:

[0026] loss(P,P gt ,P1,P2,P3,...)=w0*chamfer_distance(P,P gt )

[0027] +w1*chamfer_distance(P,P1)

[0028] +w2*chamfer_distance(P,P2)

[0029] +w3*chamfer_distance(P,P3)

[0030] +...

[0031] The weight sequence w0, w1, w2, w3, ... and the downsampling ratio are parameters that we need to obtain through experiments. In our method, we choose r1 = 2, r2 = 4, r3 = 8, ...

[0032] During the self-training denoising process, we use the resampling step to optimize the local normal consistency of the directed point cloud and redefine the multi-resolution loss function. We hope that the denoising process can achieve a good effect by balancing the local and global information, so that the denoising work can achieve a balance between local and global information.

[0033] A computer storage medium stores at least one computer instruction, and when the computer instruction is executed, at least one step of the above method is executed.

[0034] A computer device includes a processor, a memory, and a computer program, wherein the computer program is stored in the memory and executes at least one step of the aforementioned method when the computer program is executed by the processor.

[0035] On the basis of conforming to the common sense in this field, the above-mentioned preferred conditions can be combined with each other to obtain a specific implementation method.

[0036] The beneficial effects of the present invention are:

[0037] 1) The method of the present invention maps the denoising problem in the point cloud domain to the self-adjustment problem in the grid domain. This method does not require pre-training. It first maps the directed point cloud to the grid domain, where it performs self-training and optimization using more comprehensive geometric information. The loss function, based on its differentiable properties, is then propagated back to the point cloud domain, where points are moved and optimized to achieve denoising.

[0038] 2) During the denoising process, due to the randomness of the noise, the directed point cloud will lose its local normal consistency. In this process, the resampling step is used to optimize the local normal consistency of the directed point cloud, thereby achieving good denoising effect;

[0039] 3) During the self-training denoising process, the resampling step is used to optimize the local normal consistency of the directed point cloud and redefine the multi-resolution loss function. It is hoped that the denoising process can achieve a good effect by balancing the local and global information, so that the denoising work can achieve a balance between local and global information.

[0040] The present invention adopts the above technical solution to achieve the above purpose, which makes up for the shortcomings of the existing technology and has reasonable design and convenient operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Some drawings are provided to enable those skilled in the art to more quickly and clearly understand the above and / or other purposes, features, advantages and examples of the present application. It should be pointed out that the drawings, schematic embodiments and descriptions constituting the specification of the present application are used to provide a further understanding of the present application and do not constitute an improper limitation on the present application.

[0042] Figure 1 It is a flow chart of the method of the present invention;

[0043] Figure 2 It is the original noisy cloud image;

[0044] Figure 3 This is the effect picture after self-learning denoising. DETAILED DESCRIPTION

[0045] Those skilled in the art may refer to the contents herein and appropriately replace and / or modify the process parameters to achieve the desired effect. However, it should be noted that all such replacements and / or modifications are obvious to those skilled in the art and are considered to be included in the present invention. The present invention has been described through preferred embodiments, and it is obvious that relevant persons can modify or appropriately change and combine the contents described herein to implement and apply the technology of the present invention without departing from the content, spirit, and scope of the present invention.

[0046] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0047] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the technical solutions of this application. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0048] The present invention is described in detail below.

[0049] Example 1:

[0050] Provide a directed point cloud denoising method based on self-learning, the method flow chart is as follows Figure 1 As shown, we first determine the differentiable Poisson equation solver: we use the Shape As Points algorithm as the differentiable Poisson equation solver. We first establish the Poisson equation, then use the Differentiable Poisson Surface Reconstruction Solver (DPSR) to solve the implicit field, and then use MarchingCubes to convert the implicit field into a specific mesh. During this process, the size of the implicit field can be selected at will. Larger implicit fields provide better accuracy but result in slower speed. In the Differentiable Poisson Surface Reconstruction Solver, Fourier transforms are used to accelerate the solution.

[0051] Each step in the entire transformation process is differentiable. We need to find the loss in the grid domain, and we want to know the partial derivative of the loss at point p in the grid domain to achieve the purpose of learning. Suppose the grid vertex in the grid domain is p mesh , the implicit field is χ, then according to the chain rule:

[0052]

[0053] in Solved by a differentiable Poisson solver, We utilize Marching Cubes to approximate the mesh surface normal solution.

[0054] In this way, we can transfer the adjustment of the grid domain to the point cloud domain in a differentiable form, thereby driving the denoising work of the entire point cloud.

[0055] Example 2:

[0056] On the basis of the above embodiment, we continue to improve the directed point cloud denoising method based on self-learning, and the denoising method based on self-training: we copy the input directed point cloud P with noise to obtain P gt . Use P to P gt The learning process is called self-training. Consider P and P gt Use the differentiable Poisson equation solver to transform to the grid domain, and obtain the grids M and M gt Consider using chamfer distance to find the loss function between mesh vertices in two mesh domains:

[0057]

[0058] Where p is a point in the grid domain M, q is a point in the grid domain M gt The point in .

[0059] Due to the differentiability of the Poisson equation solver, we can backpropagate the loss function to the point cloud domain, thereby optimizing the points in the point cloud domain. After several training sessions, the self-training method has a good effect on handling noise.

[0060] Example 3:

[0061] On the basis of the above-mentioned embodiment, we continue to improve the directed point cloud denoising method based on self-learning, and perform resampling on the grid domain: considering that our loss function has a strong constraint on the points in the point cloud domain, but a weak constraint on the normal vector of each point in the point cloud domain. This will result in the local neighborhood normal consistency of the entire directed point cloud being very poor after a certain round of training. Specifically, although the overall graphics can be seen in the point cloud domain, the grid in the grid domain will be severely distorted. In order to pursue the local optimal solution, a large number of points will converge to a region rich in details, which will lead to a reduction in the number of points in the point cloud domain in the sparser smooth area. After multiple iterations, this is reflected in the grid domain as the absence of the grid here due to too few points.

[0062] To address this issue, we considered periodically resampling the grid domain to enforce neighborhood normal consistency in directed point clouds. Resampling offers many advantages, such as controlling the number of sampled points and ensuring uniform distribution across the entire point cloud. However, this also increases the neighborhood consistency of noise in the point cloud domain, complicating further denoising efforts.

[0063] We intersperse resampling between certain training rounds. We first consider converting the current point cloud to the grid domain, then evenly sampling a certain number of points from the grid in the grid domain back to the point cloud domain, and continue training rounds.

[0064] Example 4:

[0065] Based on the above examples, we continue to improve the directed point cloud denoising method based on self-learning and perform multi-resolution loss calculation:

[0066] For the noise point cloud P in training, we perform downsampling P1, P2, P3, ... at several ratios r1, r2, r3, .... We redefine the loss function as:

[0067] loss(P,P gt ,P1,P2,P3,...)=w0*chamfer_distance(P,P g t)

[0068] +w1*chamfer_distance(P,P1)

[0069] +w2*chamfer_distance(P,P2)

[0070] +w3*chamfer_distance(P,P3)

[0071] +...

[0072] The weight sequence w0, w1, w2, w3, ... and the downsampling ratio are parameters we need to obtain through experimentation. In our method, we choose r1 = 2, r2 = 4, r3 = 8, .... When the sampling ratio is lower, the sampled point cloud has richer details than the real object and is more affected by noise. When the sampling ratio is higher, the sampled point cloud focuses more on the overall information than the real object and is less affected by noise. Therefore, the selection of the sampling ratio and weights is particularly critical. Appropriate parameter selection can ensure good noise removal efficiency while preserving the rich details of the point cloud.

[0073] Finally, the result of denoising the noise point cloud is output.

[0074] The method described in the above embodiment is Figure 2 The noisy cloud shown is the input, and the output Figure 3 From the denoising effect diagram shown, it can be seen that the self-learning-based directed point cloud denoising method provided in this application can achieve a balance between local and overall information, taking into account both the completeness of the overall information and the richness of the detailed information, effectively removing noise points in the point cloud data, and obtaining a complete and usable point cloud at the same time.

[0075] The conventional techniques in the above embodiments are prior arts known to those skilled in the art, and thus will not be described in detail here.

[0076] The specific embodiments described herein are merely illustrative of the spirit of the present invention. Persons skilled in the art may make various modifications, additions, or substitutions to the described specific embodiments without departing from the spirit of the present invention or exceeding the scope defined by the appended claims.

[0077] Although the present invention has been described in detail and certain specific embodiments have been cited, it will be apparent to those skilled in the art that various changes and modifications can be made without departing from the spirit and scope of the invention.

[0078] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

[0079] Matters not covered in the present invention are all known technologies.

Claims

1. A directed point cloud denoising method based on self-learning, characterized by: Based on the differentiable Poisson solver, the directed point cloud is first mapped to the grid domain. Self-training and optimization are performed on the grid domain using more comprehensive geometric information. The loss function is then propagated back to the point cloud domain based on its differentiable properties. The points in the point cloud domain are moved and optimized to achieve denoising. The specific steps include the following: Step 1: Use the Shape As Points algorithm as a differentiable Poisson equation solver to map the directed point cloud to the grid domain; Step 2: Copy the input noisy directed point cloud P to obtain P gt , P to P gt The learning process is called self-training. gt Use the differentiable Poisson equation solver to transform to the grid domain, and obtain the grids M and M gt , use the chamfer distance to find the loss function between the mesh vertices in the two mesh domains, back-propagate the loss function to the point cloud domain, and optimize the points in the point cloud domain. After several self-training sessions, perform denoising. The loss function in step 2 is: where p is a point in the grid domain M and q is a point in the grid domain M. gt The point in is the distance between p and q.

2. The method according to claim 1, wherein: The method uses a grid domain resampling algorithm to ensure the local consistency of point cloud normals and combines multi-resolution point clouds in the loss function, so that the denoising work can achieve a balance between local and global information.

3. The method according to claim 1, wherein: The step 1 specifically includes: The Shape As Points algorithm is used as a differentiable Poisson equation solver. First, the Poisson equation is established, and then the implicit field is solved using the Differentiable Poisson Surface Reconstruction Solver (DPSR). The implicit field is then converted into a specific grid through Marching Cubes. In the differentiable Poisson surface reconstruction solver, Fourier transform is used to accelerate the solution; Each step in the entire transformation process is differentiable. It is necessary to find the loss in the grid domain. The purpose of learning is achieved by finding the partial derivative of the loss for the point p in the grid domain M. Suppose the grid vertex in the grid domain is p mesh , the implicit field is , then according to the chain rule: in Solved by a differentiable Poisson solver, Approximate mesh surface normals solved using Marching Cubes.

4. The method according to claim 1, characterized in that Also includes step three: In the middle of the self-training rounds in step 2, a resampling strategy is periodically used from the grid domain to constrain the neighborhood normal consistency of the directed point cloud.

5. The method according to claim 4, characterized in that The specific resampling strategy is: The resampling method is interspersed between certain training rounds. First, the current point cloud is converted to the grid domain, and then a certain number of points are evenly sampled from the grid in the grid domain and returned to the point cloud domain to continue the training round.

6. The method according to claim 1, characterized in that Also includes step four: Multi-resolution calculation loss, for the noisy point cloud P in training, several ratios r1, r2, r3, ... are downsampled to P1, P2, P3, ...; Redefine the loss function as: The weight sequence w0, w1, w2, w3, ... and the downsampling ratio are parameters that need to be obtained through experiments.

7. A computer storage medium storing at least one computer instruction, characterized in that When the computer instructions are executed, at least one step of the method according to any one of claims 1 to 6 is performed.

8. A computer device comprising a processor, a memory and a computer program, wherein the computer program is stored in the memory, When the computer program is executed by the processor, at least one step of the method according to any one of claims 1 to 6 is performed.

Images