Point cloud denoising method based on graph convolution and external attention mechanism

Abstract

The present invention discloses a point cloud denoising method based on graph convolution and external attention mechanism, which relates to the field of artificial intelligence and three-dimensional data processing technology, and includes the following steps: Step 1, obtaining noisy point cloud data and dividing the noisy point cloud data into local point cloud blocks; Step 2, using three-dimensional graph convolution to extract features of the local point cloud blocks to obtain a feature matrix; Step 3, using an external attention module to update the feature matrix to obtain updated features; Step 4, decoding the updated features and splicing the point cloud blocks to generate a complete denoised point cloud. By utilizing three-dimensional graph convolution technology, this application can process disordered point cloud data in parallel, can capture the local geometric structure of disordered point clouds, and combines with an external attention mechanism to reduce computational complexity and improve the accuracy of point cloud denoising. It is suitable for scenarios that require high-quality point cloud data, such as autonomous driving, robot navigation, and three-dimensional modeling.

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence and three-dimensional data processing technology, and in particular to a point cloud denoising method based on graph convolution and external attention mechanism. Background Art

[0002] Point clouds, as an important form of 3D data representation, are widely used in fields such as autonomous driving, robotic navigation, and 3D modeling. However, point cloud data often contains noise and outliers, which poses numerous challenges for downstream tasks such as point cloud registration and surface reconstruction. Point cloud denoising, a key preprocessing step, aims to effectively remove noise while preserving the geometric structure of the point cloud.

[0003] Traditional point cloud denoising methods are mainly divided into two categories: filtering-based and optimization-based. F. Zhang et al. proposed a filtering-based method in "Point cloud denoising with principal component analysis and a novel bilateral filter," Traitement du Signal, vol. 36, pp. 393–398, 2019. [Online]. This method designs filters based on point positions and normal vectors, but it has poor adaptability to complex geometric structures. M. Alexa proposed an optimization-based moving least squares method in "Computing and rendering point set surfaces," IEEE Trans. Vis. Comput. Graph., vol. 9, pp. 3–15, 2003. [Online]. This method reconstructs smooth surfaces through iterative projection, but it has high computational complexity and is sensitive to noise distribution. In recent years, deep learning-based methods have gradually emerged. For example, M.-J. Rakotosaona used the PointNet architecture to estimate displacement in the paper "Pointcleannet: Learning to denoise and remove outliers from dense point clouds," 2019. [Online]. Shitong Luo removed noise by reconstructing local surface structure in the paper "Differentiable manifold reconstruction for point cloud denoising," Proceedings of the 28th ACM International Conference on Multimedia, 2020. [Online].

[0004] However, the above-mentioned existing technologies have the following shortcomings: first, they cannot adequately capture the local geometric structure of disordered point clouds; second, the feature update relying on the self-attention mechanism has a high computational complexity (O(N²)), resulting in poor point cloud denoising accuracy and unable to meet the needs of scenarios that require high-quality point cloud data. Summary of the Invention

[0005] This application provides a point cloud denoising method based on graph convolution and external attention mechanism to solve the problems that existing point cloud denoising technology is insufficient in capturing the local geometric structure of disordered point clouds, the computational complexity of feature updates relying on the self-attention mechanism is high, the point cloud denoising accuracy is poor, and it cannot meet the needs of scenarios that require high-quality point cloud data.

[0006] On the one hand, the present application provides a point cloud denoising method based on graph convolution and external attention mechanism, comprising the following steps:

[0007] Step 1: Acquire noisy point cloud data and divide the noisy point cloud data into local point cloud blocks.

[0008] Step 2: Use three-dimensional graph convolution to extract features of the local point cloud block to obtain a feature matrix.

[0009] Step three: Use an external attention module to update the feature matrix to obtain updated features.

[0010] Step 4: Decode the updated features and perform point cloud block splicing to generate a complete denoised point cloud.

[0011] In a possible implementation, in step 1, dividing the noisy point cloud data into local point cloud blocks includes:

[0012] A farthest point sampling algorithm is used to select several reference points from the noisy point cloud data.

[0013] With each reference point as the center, a preset number of nearest neighboring points are selected based on the Euclidean distance to form a local point cloud block.

[0014] In a possible implementation, in step 1, after obtaining the local point cloud block, a Gaussian weight is assigned to each point in the local point cloud block.

[0015] In one possible implementation, step 2 includes:

[0016] The receptive field of each point in the local point cloud block is defined.

[0017] Designing deformable convolution kernels.

[0018] Perform graph convolution calculation on the receptive field and the deformable convolution kernel.

[0019] The graph convolution calculation is performed by stacking multiple layers of convolution to obtain a feature matrix.

[0020] In one possible implementation, step three includes:

[0021] Define two external memory units as the key and value of the external attention module.

[0022] The external attention module is used to perform feature update calculation on the feature matrix to obtain updated features.

[0023] In a possible implementation, in step three, multi-scale parallel calculations are performed on the convolution kernels with different numbers of support points of the three-dimensional graph convolution in step two, and multi-scale fusion is performed.

[0024] In one possible implementation, in step 4, the feature decoding includes:

[0025] The updated features are input into a fully connected layer, and the three-dimensional coordinate offset of each point in the updated features is output to obtain a denoised point.

[0026] In a possible implementation, in step 4, the point cloud block stitching includes:

[0027] The denoised points are spliced ​​to generate a complete denoised point cloud.

[0028] In one possible implementation, the point cloud denoising model constructed based on steps one to four uses CD distance and P2M distance to construct a joint loss function.

[0029] The point cloud denoising method based on graph convolution and external attention mechanism in this application has the following advantages:

[0030] By utilizing 3D graph convolution technology, disordered point cloud data can be processed in parallel, and the local geometric structure of disordered point clouds can be captured. Combined with the external attention mechanism, the computational complexity is reduced and the accuracy of point cloud denoising is improved. It is suitable for scenarios that require high-quality point cloud data, such as autonomous driving, robot navigation, and 3D modeling.

[0031] By leveraging the parallel processing capabilities of 3D graph convolution and the low-complexity nature of the external attention mechanism, this approach provides a scalable foundation for large-scale point cloud processing. Furthermore, by optimizing the network structure (such as reducing the number of convolutional layers or adjusting the number of support points), real-time denoising can be achieved on embedded devices, meeting the time-sensitive requirements of industrial applications.

[0032] The proposed farthest point sampling algorithm selects several reference points from noisy point cloud data. With each reference point as the center, a preset number of nearest neighboring points are selected based on the Euclidean distance to form a local point cloud block, thereby improving the distribution uniformity of the point cloud block.

[0033] After obtaining a local point cloud block, the proposed method assigns a Gaussian weight to each point in the block. This weight allocation mechanism, based on a Gaussian distribution, dynamically adjusts the contribution of each point based on its distance from a reference point, optimizing the block splicing process. This design not only improves the continuity of the denoising results but also reduces error accumulation in boundary areas, making it suitable for point cloud data of varying density and distribution.

[0034] The proposed deformable convolution kernel can dynamically adapt to the different scales and shape characteristics of the point cloud while maintaining translation and scale invariance; the proposed multi-layer convolution stacking further enhances the modeling ability of complex surfaces and edge areas, enabling the denoising process to better preserve the intrinsic geometric characteristics of the point cloud.

[0035] The proposed method defines two external memory units as the key and value of the external attention module, calculates the affinity between input features and the external memory, and achieves efficient feature updates. Compared with the self-attention mechanism, this module reduces the computational complexity to O(LSN), significantly reducing memory and computational overhead. At the same time, it captures global context information through external memory, improving the robustness and accuracy of denoising.

[0036] The proposed method performs multi-scale parallel computation on convolution kernels with different numbers of support points for 3D graph convolution, and performs multi-scale fusion to further improve the model's adaptability to noise levels and point cloud density. The proposed method also uses CD distance and P2M distance to construct a joint loss function to comprehensively optimize the geometric consistency and surface fidelity of the point cloud. This multi-scale fusion and multi-metric optimization approach improves the robustness and universality of denoising. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0038] Figure 1 A flowchart of a point cloud denoising method based on graph convolution and external attention mechanism provided in an embodiment of the present application;

[0039] Figure 2A comparison chart of the point cloud denoising results of the SCORE point cloud denoising method, DMR point cloud denoising method, PCN point cloud denoising method, MODNet point cloud denoising method, Pointfilter point cloud denoising method provided in the embodiments of this application and the method of this application. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0041] like Figure 1 As shown, the embodiment of the present application provides a point cloud denoising method based on graph convolution and external attention mechanism, including the following steps:

[0042] Step 1: Acquire noisy point cloud data and divide the noisy point cloud data into local point cloud blocks.

[0043] Step 2: Use three-dimensional graph convolution to extract features of the local point cloud block to obtain a feature matrix.

[0044] Step three: Use an external attention module to update the feature matrix to obtain updated features.

[0045] Step 4: Decode the updated features and perform point cloud block splicing to generate a complete denoised point cloud.

[0046] Specifically, in this embodiment, the noisy point cloud data is obtained from the PU-Net public point cloud database and is denoted as , Represents the total number of points in the noisy cloud data, where each point Contains 3D coordinates .

[0047] Exemplarily, in step 1, dividing the noisy point cloud data into local point cloud blocks includes:

[0048] A farthest point sampling algorithm is used to select several reference points from the noisy point cloud data.

[0049] With each reference point as the center, a preset number of nearest neighboring points are selected based on the Euclidean distance to form a local point cloud block.

[0050] Specifically, in this embodiment, the farthest point sampling algorithm is used to extract the noisy point cloud data { Select Reference points , ensuring that the reference points are evenly distributed on the surface of the point cloud. In the specific implementation, a point is randomly selected initially, and then the point farthest from the existing reference point set is iteratively selected until the point is reached. indivual. The size of can be adjusted dynamically according to the point cloud scale, for example Point Cloud, .

[0051] At each reference point As the center, select its nearest neighbor points, forming a local point cloud block . Usually set to 32 or 64, and the specific value is adjusted according to the point cloud density.

[0052] Exemplarily, in step one, after obtaining the local point cloud block, a Gaussian weight is assigned to each point in the local point cloud block.

[0053] Specifically, in order to solve the problem of multiple point filtering in overlapping areas, for local point cloud blocks Each point in Assign Gaussian weights , the calculation formula is:

[0054] .

[0055] in, , is the block radius, usually set to 0.1 times the radius of the point cloud bounding sphere. This weighting mechanism prioritizes points close to the reference point to reduce the asymmetric influence of boundary points.

[0056] Exemplarily, step 2 includes:

[0057] The receptive field of each point in the local point cloud block is defined.

[0058] Designing deformable convolution kernels.

[0059] Perform graph convolution calculation on the receptive field and the deformable convolution kernel.

[0060] The graph convolution calculation is performed by stacking multiple layers of convolution to obtain a feature matrix.

[0061] Specifically, in this embodiment, for a local point cloud block Each point in , and define its receptive field as:

[0062] .

[0063] in is the number of neighborhood points (recommended ). Represents distance-based The M nearest neighbors of . Direction vector Used for subsequent convolution calculations. represent Any point in the receptive field.

[0064] Define the convolution kernel as ,in As the center point, The number of support points (recommended ). Each support point Offset Through dynamic adjustment during training, a deformable convolution kernel is formed to adapt to the diverse geometric structures of point clouds.

[0065] Receptive field and deformable convolution kernels Perform graph convolution calculation, the calculation formula is:

[0066] .

[0067] Among them, the similarity function Based on the inner product and cosine similarity calculation, it is defined as:

[0068] .

[0069] in, is the initial feature of the point (the first layer is set to 1), Represents the kernel weight vector, which is learned through training.

[0070] Design multi-layer convolution stack as Layer convolution (recommended ), the output feature dimensions of each layer are , 128, 256, 512, generate the feature matrix of the point cloud block .

[0071] Exemplarily, step three includes:

[0072] Define two external memory units as the key and value of the external attention module.

[0073] The external attention module is used to perform feature update calculation on the feature matrix to obtain updated features.

[0074] Specifically, in this embodiment, two external memory units are defined. and , representing keys and values ​​respectively, which are implemented through one-dimensional convolution, and the initial weights are randomly generated.

[0075] For the feature matrix The formula for feature update calculation is as follows:

[0076] .

[0077] .

[0078] in, For normalization operations (such as Softmax), ensure that the sum of attention weights is 1. The complexity is O(LSN), which is much lower than the O(N²) of self-attention.

[0079] Exemplarily, in step three, multi-scale parallel calculations are performed on the convolution kernels with different numbers of support points of the three-dimensional graph convolution in step two, and multi-scale fusion is performed.

[0080] Specifically, in order to enhance the adaptability to different noise levels, multiple numbers of support points S (such as ) convolution kernel, integrating multi-scale features to further improve robustness.

[0081] Exemplarily, in step 4, the feature decoding includes:

[0082] The updated features are input into a fully connected layer, and the three-dimensional coordinate offset of each point in the updated features is output to obtain a denoised point.

[0083] Specifically, in this embodiment, the feature Input the fully connected layer and output the 3D coordinate offset of each point in the updated feature , the denoised points are:

[0084] .

[0085] Exemplarily, in step 4, the point cloud block stitching includes:

[0086] The denoised points are spliced ​​to generate a complete denoised point cloud.

[0087] Specifically, in this embodiment, for the points in the overlapping area, according to the Gaussian weights in step 1, Perform weighted averaging to generate a complete denoised point cloud .

[0088] Exemplarily, the point cloud denoising model constructed based on steps 1 to 4 uses CD distance and P2M distance to construct a joint loss function.

[0089] Specifically, the calculation formulas for CD distance and P2M distance are as follows:

[0090] .

[0091] .

[0092] Among them, CD distance is a symmetrical distance indicator that comprehensively measures the proximity and coverage between the denoised point cloud and the real point cloud. The smaller the value, the better the denoising effect. The first item in CD represents the distance from each point in the denoised point cloud to the real point cloud. To the real point cloud The average square distance of the nearest point in the denoised point cloud is used to measure whether the denoised point cloud is close to the real point cloud. The second term represents the distance from each point in the real point cloud to the real point cloud. To denoised point cloud The average squared distance between the nearest points in the denoised point cloud measures whether the denoised point cloud covers all areas of the true point cloud. The P2M distance measures the proximity between the denoised point cloud and the mesh surface reconstructed from the true point cloud, reflecting whether the denoised point cloud adheres to the true geometry. Smaller values ​​indicate that the denoised point cloud is closer to the true surface. Represents the grid, Representative Points To Grid The minimum distance.

[0093] The joint loss function is as follows:

[0094] .

[0095] in, and Respectively represent the weight coefficients of CD distance and P2M distance. In this embodiment, Set to 1.0, Set to 0.5.

[0096] The network is trained using the Adam optimizer with a learning rate of The training time is gradually reduced with the increase of epochs, and 100 epochs are trained. The hardware configuration is RTX 3090Ti GPU.

[0097] Specifically, in one possible embodiment, the method of this application was verified through simulation experiments. A test point cloud was input into a trained model to generate denoising results. Testing was conducted at different noise levels (Gaussian noise standard deviation 0.05-0.2), and compared with existing PCN point cloud denoising methods and DMRNet point cloud denoising methods. The results showed that the method of this application outperformed existing methods in both CD and P2M metrics, verifying its efficiency and denoising capabilities. The specific simulation experiment process is as follows:

[0098] Simulation Conditions: This application was simulated using Python and other related toolkits on an Intel(R) Xeon(R) E5-2697 2.60GHz CPU, 4GB of RAM, a 3090Ti GPU, and the Ubuntu 14 operating system. The training and testing database used in the experiment is the PU-Net public point cloud database.

[0099] Simulation content: First, the network is trained using the training data and tested on the test set to generate denoised point cloud data. Next, the denoised point cloud is compared with the real point cloud without noise, and the scores of relevant indicators are calculated and recorded. The comparison results with other methods at 10k points and 50k points are shown in Tables 1 and 2 respectively:

[0100] Table 1 Comparison results of CD and P2M indicators of point cloud with 10k points

[0101]

[0102] Table 2 Comparison of CD and P2M indicators of point cloud with 50k points

[0103]

[0104] In the table above, five methods are selected: SCORE point cloud denoising method, DMR point cloud denoising method, PCN point cloud denoising method, MODNet point cloud denoising method, and Pointfilter point cloud denoising method, and compared with the method of this application. Figure 2 The following is a comparison of the point cloud denoising results. Figure 2 Noisy in represents the noisy point cloud data, and Ours represents the point cloud denoising result of the method of this application.

[0105] The SCORE point cloud denoising method was proposed in the paper “S. Luo and W. Hu, “Score-based point clouddenoising,” in 2021 IEEE / CVF International Conference on Computer Vision (ICCV). IEEE, Oct. 2021, p. 4563–4572. [Online].”

[0106] The DMR point cloud denoising method was proposed in the paper “Differentiable manifold reconstruction for point cloud denoising,” Proceedings of the 28th ACM International Conference on Multimedia, 2020. [Online].”

[0107] The PCN point cloud denoising method was proposed in the paper “M.-J. Rakotosaona, VL Barbera, P. Guerrero,NJ Mitra, and M. Ovsjanikov, “Pointcleannet: Learning to denoise andremove outliers from dense point clouds,” 2019. [Online].”

[0108] The MODNet point cloud denoising method was proposed in the paper “A. Huang, Q. Xie, Z. Wang, D. Lu, M. Wei,and J. Wang, “Modnet: Multi-offset point cloud denoising network customizedfor multi-scale patches,” Computer Graphics Forum, vol. 41, no. 7, p. 109–119, Oct. 2022. [Online].”

[0109] The Pointfilter point cloud denoising method was proposed in the paper “D. Zhang, X. Lu, H. Qin, and Y. He, “Pointfilter: Point cloud filtering via encoder-decoder modeling,” 2020. [Online].”

[0110] The embodiment of the present application utilizes three-dimensional graph convolution technology to process disordered point cloud data in parallel, capture the local geometric structure of disordered point clouds, and combine with external attention mechanisms to reduce computational complexity and improve the accuracy of point cloud denoising. It is suitable for scenarios that require high-quality point cloud data, such as autonomous driving, robot navigation, and three-dimensional modeling.

[0111] By leveraging the parallel processing capabilities of 3D graph convolution and the low-complexity nature of the external attention mechanism, this approach provides a scalable foundation for large-scale point cloud processing. Furthermore, by optimizing the network structure (such as reducing the number of convolutional layers or adjusting the number of support points), real-time denoising can be achieved on embedded devices, meeting the time-sensitive requirements of industrial applications.

[0112] The proposed farthest point sampling algorithm selects several reference points from noisy point cloud data. With each reference point as the center, a preset number of nearest neighboring points are selected based on the Euclidean distance to form a local point cloud block, thereby improving the distribution uniformity of the point cloud block.

[0113] After obtaining a local point cloud block, the proposed method assigns a Gaussian weight to each point in the block. This weight allocation mechanism, based on a Gaussian distribution, dynamically adjusts the contribution of each point based on its distance from a reference point, optimizing the block splicing process. This design not only improves the continuity of the denoising results but also reduces error accumulation in boundary areas, making it suitable for point cloud data of varying density and distribution.

[0114] The proposed deformable convolution kernel can dynamically adapt to the different scales and shape characteristics of the point cloud while maintaining translation and scale invariance; the proposed multi-layer convolution stacking further enhances the modeling ability of complex surfaces and edge areas, enabling the denoising process to better preserve the intrinsic geometric characteristics of the point cloud.

[0115] The proposed method defines two external memory units as the key and value of the external attention module, calculates the affinity between input features and the external memory, and achieves efficient feature updates. Compared with the self-attention mechanism, this module reduces the computational complexity to O(LSN), significantly reducing memory and computational overhead. At the same time, it captures global context information through external memory, improving the robustness and accuracy of denoising.

[0116] The proposed method performs multi-scale parallel computation on convolution kernels with different numbers of support points for 3D graph convolution, and performs multi-scale fusion to further improve the model's adaptability to noise levels and point cloud density. The proposed method also uses CD distance and P2M distance to construct a joint loss function to comprehensively optimize the geometric consistency and surface fidelity of the point cloud. This multi-scale fusion and multi-metric optimization approach improves the robustness and universality of denoising.

[0117] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0118] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A point cloud denoising method based on graph convolution and external attention mechanism, characterized by: The following steps are involved: Step 1: obtaining noisy point cloud data and dividing the noisy point cloud data into local point cloud blocks; Step 2: Using three-dimensional graph convolution to extract features of the local point cloud block to obtain a feature matrix; Step 3: Use an external attention module to update the feature matrix to obtain updated features; Step 4: decoding the updated features and concatenating the point cloud blocks to generate a complete denoised point cloud; Step 2 includes: Defining the receptive field of each point in the local point cloud block; Design deformable convolution kernels; Performing graph convolution calculation on the receptive field and the deformable convolution kernel; Perform multi-layer convolution stacking on the graph convolution calculation to obtain a feature matrix; For local point cloud blocks Each point in , and define its receptive field as: ； in is the number of neighborhood points, Represents distance-based The M nearest neighbors, direction vector For subsequent convolution calculations, represent Any point in the receptive field; Define the convolution kernel as ,in As the center point, is the number of support points, each support point Offset Through dynamic adjustment during training, a deformable convolution kernel is formed to adapt to the diverse geometric structures of point clouds; Receptive field and deformable convolution kernels Perform graph convolution calculation, the calculation formula is: ； Among them, the similarity function Based on the inner product and cosine similarity calculation, it is defined as: ； in, is the initial feature of the point, is the kernel weight vector, learned through training; Design multi-layer convolution stack as Layer convolution, each layer output feature dimensions are , 128, 256, 512, generate the feature matrix of the point cloud block ; Step three includes: Define two external memory units as the key and value of the external attention module; Using the external attention module to perform feature update calculation on the feature matrix to obtain updated features; Define two external memory units and , representing keys and values ​​respectively, implemented through one-dimensional convolution, and the initial weights are randomly generated; For the feature matrix The formula for feature update calculation is as follows: ； ； in, This is a normalization operation to ensure that the sum of attention weights is 1, and the complexity is O(LSN).

2. The point cloud denoising method based on graph convolution and external attention mechanism according to claim 1, characterized in that In step 1, dividing the noisy point cloud data into local point cloud blocks includes: Selecting a number of reference points from the noisy point cloud data using a farthest point sampling algorithm; With each reference point as the center, a preset number of nearest neighboring points are selected based on the Euclidean distance to form a local point cloud block.

3. The point cloud denoising method based on graph convolution and external attention mechanism according to claim 2, characterized in that In step 1, after obtaining the local point cloud block, a Gaussian weight is assigned to each point in the local point cloud block.

4. The point cloud denoising method based on graph convolution and external attention mechanism according to claim 1, characterized in that In step three, the convolution kernels with different numbers of support points of the three-dimensional graph convolution in step two are calculated in parallel at multiple scales and fused at multiple scales.

5. The point cloud denoising method based on graph convolution and external attention mechanism according to claim 1, characterized in that In step 4, the feature decoding includes: The updated features are input into a fully connected layer, and the three-dimensional coordinate offset of each point in the updated features is output to obtain a denoised point.

6. The point cloud denoising method based on graph convolution and external attention mechanism according to claim 5, characterized in that In step 4, the point cloud block stitching includes: The denoised points are spliced ​​to generate a complete denoised point cloud.

7. The point cloud denoising method based on graph convolution and external attention mechanism according to claim 1, characterized in that The point cloud denoising model constructed based on steps 1 to 4 uses CD distance and P2M distance to construct a joint loss function.

Images