Point cloud denoising method based on graph convolution and external attention mechanism

Abstract

The invention discloses a point cloud denoising method based on graph convolution and an external attention mechanism, and relates to the technical field of artificial intelligence and three-dimensional data processing, and the method comprises the following steps: 1, obtaining noisy point cloud data, and dividing the noisy point cloud data into local point cloud blocks; 2, performing feature extraction on the local point cloud blocks by adopting three-dimensional graph convolution to obtain a feature matrix; 3, updating the feature matrix by adopting an external attention module to obtain updated features; and step 4, performing feature decoding and point cloud block splicing on the updated features to generate a complete de-noised point cloud. By using the three-dimensional graph convolution technology, the disordered point cloud data can be processed in parallel, the local geometric structure of the disordered point cloud can be captured, the calculation complexity is reduced in combination with an external attention mechanism, the point cloud denoising precision is improved, and the method is suitable for automatic driving, robot navigation, three-dimensional modeling and other scenes needing high-quality point cloud data.

Description

Technical Field

[0001] This application relates to the technical fields of artificial intelligence and 3D data processing, and particularly to a point cloud denoising method based on graph convolution and external attention mechanism. Background Art

[0002] Point cloud, as an important form of 3D data representation, is widely used in fields such as autonomous driving, robot navigation, and 3D modeling. However, point cloud data usually contains noise and outliers, which bring many difficulties to subsequent downstream tasks (such as point cloud registration, surface reconstruction, etc.). Point cloud denoising, as a key preprocessing step, aims to effectively remove noise while preserving the geometric structure features of the point cloud.

[0003] Traditional point cloud denoising methods are mainly divided into two categories: filter-based and optimization-based. F. Zhang et al. proposed a filter-based method in the literature "Point cloud denoising with principal component analysis and a novel bilateral filter," Traitement du Signal, vol. 36, pp. 393–398, 2019. [Online]. which designs a filter through the positions and normal vectors of points, but has poor adaptability to complex geometric structures; M. Alexa proposed an optimization-based moving least squares method in the literature "Computing and rendering point set surfaces," IEEE Trans. Vis. Comput. Graph., vol. 9, pp. 3–15, 2003. [Online], which reconstructs a smooth surface through iterative projection, but has high computational complexity and is sensitive to noise distribution. In recent years, deep learning-based methods have gradually emerged. For example, M.-J. Rakotosaona estimated displacements using the PointNet architecture in the literature "Pointcleannet: Learning to denoise and remove outliers from dense point clouds," 2019. [Online]. and Shitong Luo denoised by reconstructing the local surface structure in the literature "Differentiable manifold reconstruction for point cloud denoising," Proceedings of the 28th ACM International Conference on Multimedia, 2020. [Online].

[0004] However, the above-mentioned existing technologies have the following shortcomings: first, they cannot adequately capture the local geometric structure of disordered point clouds; second, the feature update relying on the self-attention mechanism has a high computational complexity (O(N²)), resulting in poor point cloud denoising accuracy and unable to meet the needs of scenarios that require high-quality point cloud data. Summary of the Invention

[0005] This application provides a point cloud denoising method based on graph convolution and external attention mechanism to solve the problems that existing point cloud denoising technology is insufficient in capturing the local geometric structure of disordered point clouds, the computational complexity of feature updates relying on the self-attention mechanism is high, the point cloud denoising accuracy is poor, and it cannot meet the needs of scenarios that require high-quality point cloud data.

[0006] On the one hand, the present application provides a point cloud denoising method based on graph convolution and external attention mechanism, comprising the following steps: Step 1: Acquire noisy point cloud data and divide the noisy point cloud data into local point cloud blocks.

[0007] Step 2: Use three-dimensional graph convolution to extract features of the local point cloud block to obtain a feature matrix.

[0008] Step three: Use an external attention module to update the feature matrix to obtain updated features.

[0009] Step 4: Decode the updated features and perform point cloud block splicing to generate a complete denoised point cloud.

[0010] In a possible implementation, in step 1, dividing the noisy point cloud data into local point cloud blocks includes: A farthest point sampling algorithm is used to select several reference points from the noisy point cloud data.

[0011] With each reference point as the center, a preset number of nearest neighboring points are selected based on the Euclidean distance to form a local point cloud block.

[0012] In a possible implementation, in step 1, after obtaining the local point cloud block, a Gaussian weight is assigned to each point in the local point cloud block.

[0013] In one possible implementation, step 2 includes: The receptive field of each point in the local point cloud block is defined.

[0014] Designing deformable convolution kernels.

[0015] Perform graph convolution calculation on the receptive field and the deformable convolution kernel.

[0016] Perform multi-layer convolution stacking on the graph convolution calculation to obtain a feature matrix.

[0017] In a possible implementation, step three includes: Define two external memory units as the key and value of the external attention module.

[0018] Use the external attention module to perform feature update calculation on the feature matrix to obtain updated features.

[0019] In a possible implementation, in step three, perform multi-scale parallel calculation on the convolution kernels with different numbers of support points of the three-dimensional graph convolution in step two, and perform multi-scale fusion.

[0020] In a possible implementation, in step four, the feature decoding includes: Input the updated features into a fully connected layer, output the three-dimensional coordinate offsets of each point in the updated features, and obtain denoised points.

[0021] In a possible implementation, in step four, the point cloud block stitching includes: Stitch the denoised points to generate a complete denoised point cloud.

[0022] In a possible implementation, the point cloud denoising model constructed based on steps one to four uses the CD distance and the P2M distance to construct a joint loss function.

[0023] The point cloud denoising method based on graph convolution and external attention mechanism in this application has the following advantages: By using the three-dimensional graph convolution technology, unordered point cloud data can be processed in parallel, the local geometric structure of the unordered point cloud can be captured, the computational complexity is reduced by combining the external attention mechanism, the point cloud denoising accuracy is improved, and it is applicable to scenarios such as autonomous driving, robot navigation, and 3D modeling that require high-quality point cloud data.

[0024] By using the parallel processing characteristics of the three-dimensional graph convolution technology and the low-complexity mechanism of the external attention mechanism, a scalability foundation is provided for large-scale point cloud processing. At the same time, by optimizing the network structure (such as reducing the number of convolution layers or adjusting the number of support points), real-time denoising can be achieved on embedded devices to meet the time sensitivity requirements of industrial applications.

[0025] The proposed method uses the farthest point sampling algorithm to select several reference points from the noisy point cloud data, and based on each reference point, a preset number of nearest points are selected based on the Euclidean distance to form local point cloud blocks, which improves the distribution uniformity of the point cloud blocks.

[0026] After obtaining the local point cloud patch, a Gaussian weight is assigned to each point in the local point cloud patch, introducing a weight assignment mechanism based on Gaussian distribution, dynamically adjusting the contribution degree according to the distance between the point and the reference point, and optimizing the patch stitching process. This design not only improves the continuity of the denoising result, but also reduces the error accumulation in the boundary region, and is applicable to point cloud data with different densities and distributions.

[0027] The proposed deformable convolution kernel can dynamically adapt to different scale and shape features of the point cloud while maintaining translational and scale invariance; the proposed multi-layer convolution stacking further enhances the modeling ability for complex surfaces and edge regions, enabling the denoising process to better preserve the intrinsic geometric characteristics of the point cloud.

[0028] The proposed method defines two external memory units as the key and value of the external attention module, calculates the affinity between the input features and the external memory, and realizes the efficient update of the features. Compared with the self-attention mechanism, this module reduces the computational complexity to O(LSN), significantly reducing the memory and computational overhead. At the same time, it captures global context information through the external memory, improving the robustness and accuracy of denoising.

[0029] The proposed method performs multi-scale parallel computing on convolution kernels with different numbers of support points for three-dimensional graph convolution and conducts multi-scale fusion, further enhancing the adaptability of the model to the noise level and point cloud density; the proposed method constructs a joint loss function using CD distance and P2M distance to comprehensively optimize the geometric consistency and surface fidelity of the point cloud. This multi-scale fusion and multi-index optimization method improves the robustness and universality of denoising. Brief Description of the Drawings

[0030] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0031] Figure 1 It is a schematic flowchart of the point cloud denoising method based on graph convolution and external attention mechanism provided by the embodiments of the present application; Figure 2 It is a comparison chart of the point cloud denoising results of the SCORE point cloud denoising method, DMR point cloud denoising method, PCN point cloud denoising method, MODNet point cloud denoising method, Pointfilter point cloud denoising method and the method of the present application provided by the embodiments of the present application. Detailed Embodiments

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0033] like Figure 1 As shown, the embodiment of the present application provides a point cloud denoising method based on graph convolution and external attention mechanism, including the following steps: Step 1: Acquire noisy point cloud data and divide the noisy point cloud data into local point cloud blocks.

[0034] Step 2: Use three-dimensional graph convolution to extract features of the local point cloud block to obtain a feature matrix.

[0035] Step three: Use an external attention module to update the feature matrix to obtain updated features.

[0036] Step 4: Decode the updated features and perform point cloud block splicing to generate a complete denoised point cloud.

[0037] Specifically, in this embodiment, the noisy point cloud data is obtained from the PU-Net public point cloud database and is denoted as , Represents the total number of points in the noisy point cloud data, where each point Contains 3D coordinates .

[0038] Exemplarily, in step 1, dividing the noisy point cloud data into local point cloud blocks includes: A farthest point sampling algorithm is used to select several reference points from the noisy point cloud data.

[0039] With each reference point as the center, a preset number of nearest neighboring points are selected based on the Euclidean distance to form a local point cloud block.

[0040] Specifically, in this embodiment, the farthest point sampling algorithm is used to extract the noisy point cloud data { Select Reference points , ensuring that the reference points are evenly distributed on the surface of the point cloud. In the specific implementation, a point is randomly selected initially, and then the point farthest from the existing reference point set is iteratively selected until the point is reached. indivual. The size of can be adjusted dynamically according to the point cloud scale, for example Point Cloud, .

[0041] At each reference point Centered on it, select its nearest neighbor points based on the Euclidean distance to form a local point cloud block . It is usually set to 32 or 64, and the specific value is adjusted according to the point cloud density.

[0042] Exemplarily, in step one, after obtaining the local point cloud block, assign Gaussian weights to each point in the local point cloud block.

[0043] Specifically, to solve the problem of multi-point and multi-time filtering in the overlapping area, for the local point cloud block each point in assign Gaussian weights , and the calculation formula is: .

[0044] Among them, , is the block radius, usually set to 0.1 times the radius of the point cloud boundary sphere. This weight mechanism preferentially retains the points close to the reference point and reduces the asymmetric influence of the boundary points.

[0045] Exemplarily, step two includes: Define the receptive field of each point in the local point cloud block.

[0046] Design a deformable convolutional kernel.

[0047] Perform graph convolutional calculation on the receptive field and the deformable convolutional kernel.

[0048] Perform multi-layer convolutional stacking on the graph convolutional calculation to obtain a feature matrix.

[0049] Specifically, in this embodiment, for each point in the local point cloud block , define its receptive field as: .

[0050] Among them is the number of neighborhood points (recommended ). represents the M nearest neighbors based on distance. The direction vector is used for subsequent convolutional calculations. represents any point in the receptive field.

[0051] Define the convolutional kernel as , where is the center point, is the number of support points (recommended ). Each support point offset is dynamically adjusted through training to form a deformable convolution kernel to adapt to the diverse geometric structures of the point cloud.

[0052] For the receptive field and the deformable convolution kernel perform graph convolution calculations, and the calculation formula is: .

[0053] Among them, the similarity function is calculated based on the inner product and cosine similarity and is defined as: .

[0054] Among them, is the initial feature of the point (set to 1 for the first layer), represents the kernel weight vector and is learned through training.

[0055] Design multiple layers of convolution stacked as layers of convolution (recommended ), and the output feature dimensions of each layer are , 128, 256, 512 respectively, to generate the feature matrix of the point cloud block.

[0056] Exemplarily, step three includes: Define two external memory units as the key and value of the external attention module.

[0057] Use the external attention module to perform feature update calculations on the feature matrix to obtain updated features.

[0058] Specifically, in this embodiment, two external memory units and are defined, representing the key and value respectively, and are implemented through one-dimensional convolution, and the initial weights are randomly generated.

[0059] The formula for performing feature update calculations on the feature matrix is as follows: .

[0060] .

[0061] Among them, is a normalization operation (such as Softmax) to ensure that the sum of the attention weights is 1. The complexity is O(LSN), which is much lower than O(N²) of self-attention.

[0062] Exemplarily, in step three, the convolution kernels with different numbers of support points in the three-dimensional graph convolution in step two are calculated in parallel on multiple scales and multi-scale fusion is performed.

[0063] Specifically, to enhance the adaptability to different noise levels, convolution kernels with multiple numbers of support points S (such as ) can be calculated in parallel, and multi-scale features are fused to further improve robustness.

[0064] Exemplarily, in step four, the feature decoding includes: Inputting the updated features into a fully connected layer, outputting the three-dimensional coordinate offsets of each point in the updated features, and obtaining denoised points.

[0065] Specifically, in this embodiment, the updated features are input into a fully connected layer, and the three-dimensional coordinate offsets of each point in the updated features are output , and the denoised points are: .

[0066] Exemplarily, in step four, the point cloud block stitching includes: Stitching the denoised points to generate a complete denoised point cloud.

[0067] Specifically, in this embodiment, for the points in the overlapping region, weighted averaging is performed according to the Gaussian weights in step one to generate a complete denoised point cloud .

[0068] Exemplarily, the point cloud denoising model constructed based on steps one to four uses the CD distance and the P2M distance to construct a joint loss function.

[0069] Specifically, the calculation formulas of the CD distance and the P2M distance are shown as follows: .

[0070] .

[0071] Among them, the CD distance is a symmetric distance metric that comprehensively measures the closeness and coverage between the denoised point cloud and the real point cloud. The smaller the value, the better the denoising effect. The first term in CD represents the average squared distance from each point in the denoised point cloud to the nearest point in the real point cloud , measuring whether the denoised point cloud is close to the real point cloud. The second term represents the average squared distance from each point in the real point cloud to the denoised point cloud The mean squared distance of the nearest points in the denoised point cloud measures whether the denoised point cloud covers all regions of the real point cloud. The P2M distance measures the proximity between the denoised point cloud and the mesh surface reconstructed from the real point cloud, reflecting whether the denoised point cloud fits the real geometry. The smaller the value, the closer the denoised point cloud is to the real surface. represents the mesh, represents the point to the mesh the minimum distance.

[0072] The combined loss function is as follows: .

[0073] Among them, and respectively represent the weight coefficients of the CD distance and the P2M distance. In this embodiment, is set to 1.0, is set to 0.5.

[0074] The network is trained using the Adam optimizer, and the learning rate gradually decays as the epoch increases from for 100 epochs. The hardware configuration is an RTX 3090Ti GPU.

[0075] Specifically, in a possible embodiment, the method of this application is verified through simulation experiments. The test point cloud is input into the trained model to generate a denoising result. Tests are conducted on different noise levels (Gaussian noise standard deviation 0.05 - 0.2), and comparisons are made with existing PCN point cloud denoising methods, DMRNet point cloud denoising methods, etc. The results show that the method of this application is superior to existing methods in both CD and P2M metrics, verifying its efficiency and denoising ability. The specific simulation experiment process is as follows: Simulation conditions: This application conducts simulations using Python and other related toolkits on a computer with an Intel(R) Xeon(R) E5 - 2697 2.60GHZ central processing unit, 4G of memory, a 3090Ti GPU, and an Ubuntu 14 operating system. The training and test databases used in the experiment are the publicly available PU - Net point cloud database.

[0076] Simulation content: First, the network is trained with the training data and tested on the test set to generate the denoised point cloud data. Second, the denoised point cloud is compared with the real point cloud without noise, and the scores of relevant metrics are calculated and recorded. The comparison results of the metrics with other methods at 10k points and 50k points are shown in Table 1 and Table 2 respectively: Table 1 Comparison results of CD and P2M metrics of point clouds at 10k points

[0077] Table 2 Comparison results of CD and P2M metrics of point cloud under 50k points

[0078] In the above table, five methods are selected: SCORE point cloud denoising method, DMR point cloud denoising method, PCN point cloud denoising method, MODNet point cloud denoising method, Pointfilter point cloud denoising method, and compared with the method of this application. As Figure 2 shown in the comparison chart of point cloud denoising results, Figure 2 Noisy in it represents the noisy point cloud data, and Ours represents the point cloud denoising result of the method of this application.

[0079] The SCORE point cloud denoising method was proposed in the literature "S. Luo and W. Hu, “Score-based point cloud denoising,” in 2021 IEEE / CVF International Conference on Computer Vision (ICCV). IEEE, Oct. 2021, p. 4563–4572. [Online]."

[0080] The DMR point cloud denoising method was proposed in the literature "Differentiable manifold reconstruction for point cloud denoising," Proceedings of the 28th ACM International Conference on Multimedia, 2020. [Online]."

[0081] The PCN point cloud denoising method was proposed in the literature "M.-J. Rakotosaona, V. L. Barbera, P. Guerrero, N. J. Mitra, and M. Ovsjanikov, “Pointcleannet: Learning to denoise and remove outliers from dense point clouds,” 2019. [Online]."

[0082] The MODNet point cloud denoising method was proposed in the literature “A. Huang, Q. Xie, Z. Wang, D. Lu, M. Wei, and J. Wang, “Modnet: Multi-offset point cloud denoising network customized for multi-scale patches,” Computer Graphics Forum, vol. 41, no. 7, p. 109–119, Oct. 2022. [Online].”

[0083] The Pointfilter point cloud denoising method was proposed in the literature “D. Zhang, X. Lu, H. Qin, and Y. He, “Pointfilter: Point cloud filtering via encoder-decoder modeling,” 2020. [Online].”

[0084] By using 3D graph convolution technology, the embodiments of the present application can process unordered point cloud data in parallel, capture the local geometric structure of unordered point clouds, combine with an external attention mechanism to reduce the computational complexity, improve the point cloud denoising accuracy, and are applicable to scenarios such as autonomous driving, robot navigation, and 3D modeling that require high-quality point cloud data.

[0085] By leveraging the parallel processing characteristics of 3D graph convolution technology and the low-complexity mechanism of the external attention mechanism, a scalability foundation for large-scale point cloud processing is provided. At the same time, by optimizing the network structure (such as reducing the number of convolutional layers or adjusting the number of support points), real-time denoising can be achieved on embedded devices to meet the time-sensitive requirements of industrial applications.

[0086] The proposed method selects a number of reference points from the noisy point cloud data using the farthest point sampling algorithm, and based on each reference point, a preset number of nearest points are selected based on the Euclidean distance to form local point cloud patches, improving the distribution uniformity of the point cloud patches.

[0087] After obtaining the local point cloud patches, the proposed method assigns Gaussian weights to each point in the local point cloud patches, introduces a weight assignment mechanism based on the Gaussian distribution, dynamically adjusts the contribution degree according to the distance between the point and the reference point, and optimizes the block stitching process. This design not only improves the continuity of the denoising results but also reduces the error accumulation in the boundary region, and is applicable to point cloud data with different densities and distributions.

[0088] The proposed deformable convolutional kernel can dynamically adapt to the different scale and shape features of point clouds while maintaining translational and scale invariance; the proposed multi-layer convolutional stacking further enhances the modeling ability for complex surfaces and edge regions, enabling the denoising process to better preserve the intrinsic geometric properties of point clouds.

[0089] The proposed method defines two external memory units as the key and value of the external attention module, calculates the affinity between the input features and the external memory, and realizes the efficient update of features. Compared with the self-attention mechanism, this module reduces the computational complexity to O(LSN), significantly reducing the memory and computational overhead. At the same time, it captures global context information through the external memory, improving the robustness and accuracy of denoising.

[0090] The proposed method performs multi-scale parallel computation on convolutional kernels with different numbers of support points for 3D graph convolution and conducts multi-scale fusion, further enhancing the model's adaptability to noise levels and point cloud densities; the proposed method constructs a joint loss function using CD distance and P2M distance to comprehensively optimize the geometric consistency and surface fidelity of point clouds. This multi-scale fusion and multi-index optimization method improves the robustness and universality of denoising.

[0091] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to these embodiments once they learn the basic creative concept. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0092] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these changes and modifications.

Claims

1. A point cloud denoising method based on graph convolution and external attention mechanism, characterized in that, It includes the following steps: Step 1: Obtain noisy point cloud data and divide the noisy point cloud data into local point cloud blocks; Step 2: Use 3D graph convolution to extract features from the local point cloud blocks to obtain a feature matrix; Step 3: Use an external attention module to update the feature matrix to obtain updated features; Step 4: Perform feature decoding and point cloud block stitching on the updated features to generate a complete denoised point cloud.

2. The point cloud denoising method based on graph convolution and external attention mechanism according to claim 1, wherein In Step 1, dividing the noisy point cloud data into local point cloud blocks includes: Selecting a number of reference points from the noisy point cloud data using the farthest point sampling algorithm; Taking each reference point as the center and selecting a preset number of nearest points based on the Euclidean distance to form local point cloud blocks.

3. The point cloud denoising method based on graph convolution and external attention mechanism according to claim 2, wherein In Step 1, after obtaining the local point cloud blocks, assign Gaussian weights to each point in the local point cloud blocks.

4. The point cloud denoising method based on graph convolution and external attention mechanism according to claim 1, characterized in that, Step 2 includes: Defining the receptive field of each point in the local point cloud block; Designing a deformable convolution kernel; Performing graph convolution calculations on the receptive field and the deformable convolution kernel; Performing multi-layer convolution stacking on the graph convolution calculations to obtain a feature matrix.

5. The point cloud denoising method based on graph convolution and external attention mechanism according to claim 1, characterized in that, Step 3 includes: Defining two external memory units as the key and value of the external attention module; Using the external attention module to perform feature update calculations on the feature matrix to obtain updated features.

6. The point cloud denoising method based on graph convolution and external attention mechanism according to claim 5, characterized in that In Step 3, perform multi-scale parallel calculations on the convolution kernels with different support point numbers of the 3D graph convolution in Step 2 and perform multi-scale fusion.

7. The point cloud denoising method based on graph convolution and external attention mechanism according to claim 1, characterized in that In Step 4, the feature decoding includes: Inputting the updated features into a fully connected layer and outputting the 3D coordinate offsets of each point in the updated features to obtain denoised points.

8. The point cloud denoising method based on graph convolution and external attention mechanism according to claim 7, characterized in that In Step 4, the point cloud block stitching includes: Stitching the denoised points to generate a complete denoised point cloud.

9. The point cloud denoising method based on graph convolution and external attention mechanism according to claim 1, characterized in that, The point cloud denoising model constructed based on Steps 1 to 4 constructs a joint loss function using the CD distance and the P2M distance.

Images