A 3D point cloud denoising method based on multi-stage information interaction

Abstract

The present invention provides a 3D point cloud denoising method based on multi-stage information interaction, comprising: step S1, obtaining a data set for a 3D object, and dividing the noise point cloud into a training set and a test set according to the clean point cloud in the data set, and randomly extracting noise blocks; step S2, building a 3D point cloud denoising network using a multi-stage framework, each stage including an encoder module and a decoder module; step S3, extracting multiple training noise blocks from the training set, training the 3D point cloud denoising network to obtain a multi-stage denoising result; step S4, training the 3D point cloud denoising network for multiple rounds according to step S3 to obtain a 3D point cloud denoising network model; step S5, extracting multiple test noise blocks from the test set for testing, and outputting the multi-stage denoising result of each test noise block as the 3D point cloud denoising result. The beneficial effect is that the present invention can effectively and accurately denoise the noise point cloud.

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud denoising, and in particular to a three-dimensional point cloud denoising method based on multi-stage information interaction. Background Art

[0002] With the rapid development of 3D sensing technology, point cloud has become the preferred format for representing 3D data and has been widely used in 3D vision fields such as autonomous driving, augmented reality and 3D motion recognition. However, during the acquisition process, 3D point cloud data will be interfered by factors such as inherent errors of hardware equipment, human operation, and environmental conditions, thereby introducing noise. These noises will cause the underlying structure of the 3D point cloud to deform, inevitably affecting the subsequent data processing and analysis process. Therefore, how to effectively process and eliminate noise in 3D point cloud data has become an important research topic in the current 3D vision field.

[0003] Traditional 3D point cloud denoising methods have achieved certain results. However, these methods often rely on geometric prior knowledge and lack robustness when dealing with 3D point clouds with high noise or non-uniform sampling. In recent years, with the successful application of network architectures designed specifically for 3D point clouds, deep learning-based denoising methods have made significant progress. Although these deep learning-based data-driven methods have shown excellent performance, they still face challenges in generalization ability. For example, when faced with 3D point clouds contaminated by unknown high levels of noise, existing methods tend to iteratively denoise the 3D point clouds during testing, while not considering the differences in residual noise at different stages during training. This limitation may lead to excessive or insufficient denoising in the denoising process. Therefore, there is an urgent need to study a method that can transcend the limitations of existing technologies and achieve effective and accurate denoising of noisy point clouds. Summary of the invention

[0004] The technical problem to be solved by the present invention is to achieve effective and accurate denoising of noisy point clouds. In order to overcome the defects of the above-mentioned prior art (or related technology), the present invention provides a three-dimensional point cloud denoising method based on multi-stage information interaction.

[0005] The present invention provides a three-dimensional point cloud denoising method based on multi-stage information interaction, comprising the following steps:

[0006] Step S1, for at least one three-dimensional object, obtain a data set of the three-dimensional object, the data set includes a plurality of clean point clouds, and pre-process each of the clean point clouds to obtain a corresponding noise point cloud, and randomly extract a point cloud of size 1 from each of the clean point clouds and each of the noise point clouds. Noise blocks, then mixing each of the clean point clouds and each of the noise point clouds and randomly dividing them into a training set and a test set;

[0007] Step S2, using a deep learning framework to build a 3D point cloud denoising network, the 3D point cloud denoising network adopts a multi-stage framework and each stage includes an encoder module and a decoder module, and a cross-stage cross-encoding module is provided between two adjacent stages, the cross-stage cross-encoding module is respectively connected to the encoder module and the decoder module of the two adjacent stages, and the noise block is received through the encoder module and the feature map is output , receiving the feature map through the decoder module And output point cloud displacement , receiving the point cloud displacement at each stage through the output module And combined with the noise block to obtain a multi-stage denoising result, where the point cloud displacement output in the previous stage is said noise block as input to the next stage;

[0008] Step S3, extracting a plurality of training noise blocks from the training set, and training the three-dimensional point cloud denoising network using the training set according to step S2 to obtain the multi-stage denoising result corresponding to each of the training noise blocks;

[0009] Step S4, training the three-dimensional point cloud denoising network for multiple rounds according to step S3 to obtain a three-dimensional point cloud denoising network model;

[0010] Step S5, extracting multiple test noise blocks from the test set, testing each of the test noise blocks through the three-dimensional point cloud denoising network model, and outputting the multi-stage denoising result corresponding to each of the test noise blocks as the three-dimensional point cloud denoising result.

[0011] Compared with the prior art, the 3D point cloud denoising method based on multi-stage information interaction of the present invention has the following advantages:

[0012] In the present invention, step S1 is used to acquire a data set and divide a training set and a test set, step S2 is used to build a three-dimensional point cloud denoising network, step S3 is used to train the three-dimensional point cloud denoising network, step S4 is used to generate a three-dimensional point cloud denoising network model, and step S5 is used to test the three-dimensional point cloud denoising network model. A three-dimensional point cloud denoising network based on multi-stage information interaction is built using a deep learning framework. The network follows a multi-stage framework and mainly includes an encoder module, a decoder module, a cross-stage cross-encoding module, and an output module. The encoder module uses an adaptive edge graph convolution module to capture local parts of the clean point cloud. The decoder module integrates the multi-scale features extracted by the encoder module and learns to predict the point cloud displacement required for denoising, ensuring that the geometric structure and details of the clean point cloud are maintained during denoising. The cross-stage and cross-encoding module enhances the expression ability and learning efficiency of the multi-stage model through multi-stage information interaction, ensuring that the information of the previous stage is fully disseminated and utilized in the subsequent stages. The output module receives and integrates the point cloud displacement predicted by the 3D point cloud denoising network to obtain the final multi-stage denoising result, which can use multi-stage information interaction to more effectively denoise different noise point clouds and achieve effective and accurate denoising of noise point clouds.

[0013] In a possible implementation manner, in step S1, the preprocessing method adopted is:

[0014] Each of the clean point clouds is sequentially normalized, Gaussian noise with different noise levels is added, randomly scaled up or down, and randomly rotated to obtain the corresponding noisy point cloud.

[0015] In a possible implementation, in step S2, each of the encoder modules includes three adaptive edge graph convolution modules with the same structure connected in sequence, and are defined as the first adaptive edge graph convolution module, the second adaptive edge graph convolution module and the third adaptive edge graph convolution module according to the information transmission direction, and the input end of the first adaptive edge graph convolution module is used as the input end of the encoder module, and the noise block is received through the first adaptive edge graph convolution module and the feature map is output. , receiving the feature map through the second adaptive edge map convolution module And output feature map , receiving the feature map through the third adaptive edge map convolution module And output feature map , the feature map , the feature map And the feature map Splice into the feature map ; wherein the feature map The size is , the feature map The size is , the feature map The size is , the feature map The size is .

[0016] In a possible implementation, each of the adaptive edge graph convolution modules includes a graph building block, an adaptive edge attention block, and an edge graph convolution block connected in sequence, the first adaptive edge graph convolution module includes the first graph building block, the first adaptive edge attention block, and the first edge graph convolution block, the second adaptive edge graph convolution module includes the second graph building block, the second adaptive edge attention block, and the second edge graph convolution block, the third adaptive edge graph convolution module includes the third graph building block, the third adaptive edge attention block, and the third edge graph convolution block, the noise block is received through the first graph building block and the graph structure is output , receiving the graph structure through the first adaptive edge attention block And output the graph structure , receiving the graph structure through the first edge graph convolution block And output the feature map , receiving the feature map through the second map building block and the noise block and output graph structure , receiving the graph structure through the second adaptive edge attention block And output the graph structure , receiving the graph structure through the second edge graph convolution block And output the feature map , receiving the feature map through the third map building block and the noise block and output graph structure , receiving the graph structure through the third adaptive edge attention block And output the graph structure , receiving the graph structure through the third edge graph convolution block And output the feature map .

[0017] In a possible implementation, in step S2, each of the graph building blocks receives the noise block and a high-dimensional noise block obtained by extracting semantic features from the noise block, takes each point in the noise block and the high-dimensional noise block as a centroid, defines neighboring points of the centroid based on geometric information and semantic information according to the Euclidean algorithm, connects the centroid with the neighboring points, and constructs a geometric graph based on the geometric information and the semantic information. and semantic graph , and then the geometric figure and the semantic graph The graph structure is obtained by aggregating different neighborhoods of Or the graph structure Or the graph structure Output.

[0018] In a possible implementation, in step S2, the cross-stage cross-coding module between the adjacent T-th stage and the T+1-th stage updates the feature map output by each of the adaptive edge graph convolution modules and the first fully connected layer in the T+1-th stage through four branches, wherein the execution steps of the first branch include:

[0019] The feature map output by the first adaptive edge graph convolution module in the Tth stage And the feature map output by the fourth fully connected layer As input, two layers of size 2 are obtained through a fully connected layer with an output channel number of 64 and a ReLU activation function layer. Feature map and feature map , and the feature map And the feature map Perform addition operation to obtain feature map , and then the feature map The feature map output by the first adaptive edge graph convolution module in the T+1 stage Add, and after performing information interaction, the feature map output by the first adaptive edge graph convolution module in the T+1 stage Updated to ;

[0020] The execution steps of the second branch include:

[0021] The feature map output by the second adaptive edge graph convolution module in the Tth stage And the feature map output by the third fully connected layer As input, two layers of size 128 are obtained through a fully connected layer with an output channel number of 128 and a ReLU activation function layer. Feature map and feature map , and the feature map And the feature map Perform addition operation to obtain feature map , and then the feature map The feature map output by the second adaptive edge graph convolution module in the T+1 stage Add, and after performing information interaction, the feature map output by the second adaptive edge graph convolution module in the T+1 stage Updated to ;

[0022] The execution steps of the third branch include:

[0023] The feature map output by the third adaptive edge graph convolution module in the Tth stage And the feature map output by the second fully connected layer As input, two layers of size 256 are obtained through a fully connected layer with an output channel number of 256 and a ReLU activation function layer. Feature map and feature map , and the feature map And the feature map Perform addition operation to obtain feature map , and then the feature map The feature map output by the third adaptive edge graph convolution module in the T+1 stage Add, and after performing information interaction, the feature map output by the third adaptive edge graph convolution module in the T+1 stage Updated to ;

[0024] The execution steps of the fourth branch include:

[0025] The feature map output by the first fully connected layer in the Tth stage As input, a fully connected layer with an output channel number of 512 and a ReLU activation function layer are used to obtain a size of Feature map , and the feature map And the feature map output by the first fully connected layer in the T+1 stage Channel splicing is performed to obtain a size of Feature map , and then the feature map Through a fully connected layer with an output channel number of 512 and a ReLU activation function layer, the size is Feature map , and the feature map The feature map output by the first fully connected layer in the T+1 stage Add, and after performing information interaction, the feature map output by the first fully connected layer in the T+1 stage Updated to .

[0026] In a possible implementation, in step S2, each of the decoder modules includes five fully connected layers connected in sequence, and are defined as a first fully connected layer, a second fully connected layer, a third fully connected layer, a fourth fully connected layer, and a fifth fully connected layer according to the information transmission direction, and the feature map is received through the first fully connected layer. And output feature map , receiving the feature map through the second fully connected layer And output feature map , receiving the feature map through the third fully connected layer And output feature map , receiving the feature map through the fourth fully connected layer And output feature map , receiving the feature map through the fifth fully connected layer And output the point cloud displacement ; wherein the feature map The size is , the feature map The size is , the feature map The size is , the feature map The size is , the point cloud displacement The size is .

[0027] In a possible implementation, in step S2, each of the adaptive edge attention blocks receives the graph structure output by the graph building block. Or the graph structure Or the graph structure After that, according to the graph structure Or the graph structure Or the graph structure The centroid feature and the edge feature are calculated, and the edge feature is passed through a fully connected layer with an output channel number of 1 and a Sigmod activation function layer to obtain the edge attention weight, and the edge feature is multiplied by the edge attention weight to obtain the weighted edge feature, and then the weighted edge feature and the centroid feature are added to obtain the corresponding feature map or feature map or feature map Output.

[0028] In a possible implementation, in step S2, each of the edge graph convolution blocks includes two graph convolution layers, two batch normalization layers, two ReLU activation function layers, a fully connected layer and a maximum pooling layer, and the two graph convolution layers are defined as the first graph convolution layer and the second graph convolution layer according to the information transmission direction, the two batch normalization layers are defined as the first batch normalization layer and the second batch normalization layer, the two ReLU activation function layers are defined as the first ReLU activation function layer and the second ReLU activation function layer, the first graph convolution layer is connected to the first batch normalization layer, the first batch normalization layer is connected to the first ReLU activation function layer, the first ReLU activation function layer is connected to the second graph convolution layer, the second graph convolution layer is connected to the second batch normalization layer, the second batch normalization layer is connected to the second ReLU activation function layer, the second ReLU activation function layer is connected to the fully connected layer, and the fully connected layer is connected to the maximum pooling layer.

[0029] In a possible implementation, the convolution kernel size of the first graph convolution layer in the first edge graph convolution block is 1, the step size is 1, the number of input channels is 6, and the number of output channels is 64. The convolution kernel size of the second graph convolution layer in the first edge graph convolution block is 1, the step size is 1, the number of input channels is 64, and the number of output channels is 64. The number of input channels of the fully connected layer in the first edge graph convolution block is 32, and the number of output channels is 32. The convolution kernel size of the first graph convolution layer in the second edge graph convolution block is 1, the step size is 1, the number of input channels is 128, and the number of output channels is 128. The second edge graph convolution block has a second The convolution kernel size of the graph convolution layer is 1, the stride size is 1, the number of input channels is 128, and the number of output channels is 128. The number of input channels of the fully connected layer in the second edge graph convolution block is 32, and the number of output channels is 32. The convolution kernel size of the first graph convolution layer in the third edge graph convolution block is 1, the stride size is 1, the number of input channels is 256, and the number of output channels is 256. The convolution kernel size of the second graph convolution layer in the third edge graph convolution block is 1, the stride size is 1, the number of input channels is 256, and the number of output channels is 256. The number of input channels of the fully connected layer in the third edge graph convolution block is 32, and the number of output channels is 32. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a flow chart of the steps of the present invention;

[0031] Figure 2 It is a structural schematic diagram of the three-dimensional point cloud denoising network of the present invention;

[0032] Figure 3 It is a structural schematic diagram of the cross-stage and cross-coding module of the present invention;

[0033] Figure 4 It is a structural schematic diagram of the output module of the present invention;

[0034] Figure 5 It is a structural schematic diagram of a graph building block of the present invention;

[0035] Figure 6 Schematic diagram of the structure of the adaptive edge attention block of the present invention;

[0036] Figure 7 Schematic diagram of the structure of the edge graph convolution block of the present invention;

[0037] Figure 8 The figure is a schematic diagram showing the denoising results of some point cloud data on the PUNet test set using the method of the present invention and the existing point cloud denoising method. DETAILED DESCRIPTION

[0038] First, those skilled in the art should understand that these implementations are only used to explain the technical principles of the embodiments of the present invention, and are not intended to limit the protection scope of the embodiments of the present invention. Those skilled in the art can make adjustments to them as needed to adapt to specific application scenarios.

[0039] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] See also Figure 1 The embodiment of the present invention discloses a three-dimensional point cloud denoising method based on multi-stage information interaction, comprising the following steps:

[0041] Step S1: for at least one three-dimensional object, a data set of the three-dimensional object is obtained, the data set includes multiple clean point clouds; then each clean point cloud is normalized, Gaussian noise of different noise levels is added, randomly scaled up or down, and randomly rotated to obtain a corresponding noisy point cloud; and the clean point cloud and the corresponding noisy point cloud are divided into a training set and a test set, and then a fixed number of points are randomly extracted from the clean point cloud and the noisy point cloud, respectively. The noise block is used as the input of the network, where in this embodiment ;

[0042] Step S2: Use the deep learning framework to build a deep neural network as a 3D point cloud denoising network based on multi-stage information interaction, such as Figure 2 As shown in Figure 1, the 3D point cloud denoising network follows a multi-stage framework. The number of network stages is S. Each stage includes an encoder module and a decoder module. There are cross-stage and cross-encoding modules between different stages. Finally, the output module integrates the multi-stage denoising results.

[0043] Step S3, using the training set to train the 3D point cloud denoising network based on multi-stage information interaction. After each round of training, the 3D point cloud denoising network based on multi-stage information interaction outputs the multi-stage predicted displacement corresponding to the input noise block in the training set, which is recorded as , and then calculate the loss of the network, recorded as , ; In this embodiment, S=4, , represents the actual displacement of the i-th stage in the training set, and the initial learning rate of the Adam optimizer is , every 10 rounds, the learning rate is decayed to 0.5 times of the original value, and the batch size is 8;

[0044] Step S4: training for 100 rounds in total according to the process of step S3, and obtaining a 3D point cloud denoising network model based on multi-stage information interaction;

[0045] Step S5: Use the 3D point cloud denoising network model based on multi-stage information interaction to test each noise point cloud in the test set. The 3D point cloud denoising network model based on multi-stage information interaction outputs the multi-stage predicted displacement corresponding to each noise point cloud in the test set to obtain the final denoising result.

[0046] Continue to see Figure 1 ,In step S1, the process of preprocessing a clean point cloud is as follows: first, the clean point cloud is normalized, the coordinate value of each point is subtracted from the average of the maximum and minimum coordinate values, and then the maximum radius of the point cloud bounding sphere is calculated as the scaling factor, so as to normalize the range of the point cloud to the standard range of [-1,1]; secondly, Gaussian noise with a standard deviation of 0.5%~2% of the bounding sphere radius is added to the normalized point cloud.

[0047] Continue to see Figure 1 Taking the single-stage network as an example, the encoder module is mainly composed of three adaptive edge graph convolution modules with the same structure; each adaptive edge graph convolution module contains a graph building block, an adaptive edge attention block and an edge graph convolution block connected in sequence. The input end of the first graph building block receives a size of The input of the first adaptive edge attention block receives the graph structure output by the output of the first graph building block. , the input of the first edge graph convolution block receives the graph structure output by the output of the first adaptive edge attention block The output of the first edge graph convolution block is used as the feature map output by the first adaptive edge graph convolution module and is recorded as The input of the second graph building block receives the feature map output by the output of the first edge graph convolution block as the input of the second adaptive edge graph convolution module. and the original noise block, the input of the second adaptive edge attention block receives the graph structure output by the output of the second graph building block , the input of the second edge graph convolution block receives the graph structure output by the output of the second adaptive edge attention block The output of the second edge graph convolution block is used as the feature map output by the second adaptive edge graph convolution module and is recorded as The input of the third graph building block receives the feature map output by the output of the second edge graph convolution block as the input of the third adaptive edge graph convolution module. and the original noise block, the input of the third adaptive edge attention block receives the graph structure output by the output of the third graph building block , the input of the third edge graph convolution block receives the graph structure output by the output of the third adaptive edge attention block The output of the third edge graph convolution block is used as the feature map output by the output of the third adaptive edge graph convolution module and is recorded as ; The multi-scale feature map spliced ​​from the output of the three adaptive edge graph convolution modules is denoted as ; Among them, the feature map The size is , feature map The size is , feature map The size is , feature map The size is .

[0048] Continue to see Figure 1 Taking the single-stage network as an example, the decoder module includes five fully connected layers. The first fully connected layer receives the multi-scale feature map output by the output of the decoder module as the input of the decoder module. The input of the second fully connected layer receives the feature map output by the output of the first fully connected layer The input of the third fully connected layer receives the feature map output by the output of the second fully connected layer , the input of the fourth fully connected layer receives the feature map output by the output of the third fully connected layer , the input of the fifth fully connected layer receives the feature map output by the output of the fourth fully connected layer , the output of the fifth fully connected layer outputs the final predicted point cloud displacement ; Among them, the feature map The size is , feature map The size is , feature map The size is , feature map The size is , feature map The size is .

[0049] See also Figure 3 The cross-stage and cross-coding modules between the two stages are executed in four branches. Taking the processing between the Tth stage and the T+1th stage as an example, the details are as follows:

[0050] (1) The execution steps of the first branch include: taking the feature map output from the output end of the first adaptive edge graph convolution module of the encoder module in the Tth stage The feature map output from the output of the fourth fully connected layer of the decoder module As input, it then passes through a fully connected layer with 64 output channels and a ReLU activation function layer to obtain two layers of size Feature map and ; Then perform the addition operation on the two feature maps in sequence to obtain the feature map , and then the feature map The feature map output from the output of the first adaptive edge graph convolution module of the encoder module in the T+1 stage After adding and performing information interaction, the feature map outputted from the output of the first adaptive edge graph convolution module of the encoder module in the T+1 stage is updated to ;

[0051] (2) The execution steps of the second branch include: taking the feature map outputted from the output end of the second adaptive edge graph convolution module of the encoder module in the Tth stage The feature map output from the output of the third fully connected layer of the decoder module As input, it then passes through a fully connected layer with an output channel number of 128 and a ReLU activation function layer to obtain two layers of size Feature map and ; Then perform the addition operation on the two feature maps in sequence to obtain the feature map , and then the feature map The feature map output from the output of the second adaptive edge graph convolution module of the encoder module in the T+1 stage After adding and performing information interaction, the feature map outputted from the output of the second adaptive edge graph convolution module of the encoder module in the T+1 stage is updated to ;

[0052] (3) The execution steps of the third branch include: taking the feature map outputted from the output end of the third adaptive edge graph convolution module of the encoder module of the Tth stage The feature map output from the output of the second fully connected layer of the decoder module As input, it then passes through a fully connected layer with an output channel number of 256 and a ReLU activation function layer to obtain two layers of size Feature map and ; Then perform the addition operation on the two feature maps in sequence to obtain the feature map , and then the feature map The feature map output from the output of the third adaptive edge graph convolution module of the T+1 stage encoder module After adding and performing information interaction, the feature map outputted from the output of the third adaptive edge graph convolution module of the encoder module in the T+1 stage is updated to ;

[0053] (4) The execution steps of the fourth branch include: taking the feature map output from the output end of the first fully connected layer of the decoder module in the Tth stage As input, it then passes through a fully connected layer with an output channel number of 512 and a ReLU activation function layer to obtain a size of Feature map ; Then for the feature map And the feature map output by the decoder module at stage T+1 After channel splicing, the obtained feature map is recorded as , The size is ; Through a fully connected layer with an output channel number of 512 and a ReLU activation function layer, the size is Feature map , and then the feature map The feature map output from the output of the first fully connected layer of the decoder module in the T+1 stage After adding and performing information interaction, the feature map output from the output of the first fully connected layer of the decoder module in the T+1 stage is updated to .

[0054] See also Figure 4 The output module receives the predicted point cloud displacement of the 3D point cloud denoising network , the point cloud displacement is combined with the input noise block in turn to obtain the final multi-stage denoising result; where the size of each point cloud displacement and the final multi-stage denoising result is .

[0055] See also Figure 5 In step S2, the graph building block receives an original noise block and a high-dimensional noise block after semantic feature extraction, and then takes each point in the noise block as the centroid, searches for Euclidean neighbors based on geometric information and semantic information, and connects the centroid with the neighbor points to construct a geometric graph. and semantic graph ; Then aggregate the different neighborhoods of the two graph structures, and the result is used as the graph structure output at the output end of the graph building block.

[0056] See also Figure 6In step S2, the adaptive edge attention block receives the graph structure outputted from the output end of the graph building block, and calculates the centroid feature and the edge feature according to the graph structure; the edge feature passes through a fully connected layer with an output channel number of 1 and a Sigmod activation function layer to obtain the edge attention weight, and then the edge feature is multiplied by the edge attention weight and then added to obtain the weighted edge feature; the result of combining the weighted edge feature and the centroid feature is used as the feature map outputted from the output end of the adaptive edge attention block.

[0057] See also Figure 7 In step S2, the edge graph convolution block includes two graph convolution layers, two batch normalization layers, two ReLU activation function layers, a fully connected layer and a maximum pooling layer. The input end of the first graph convolution layer receives the feature map output by the output end of the adaptive edge attention block as the input end of the edge convolution block where it is located. The first batch normalization layer receives the feature map output by the output end of the first graph convolution layer. The first ReLU activation function layer receives the feature map output by the output end of the first batch normalization layer. The input end of the second graph convolution layer receives the feature map output by the output end of the first ReLU activation function layer. The second batch normalization layer receives the feature map output by the output end of the second graph convolution layer. The second ReLU activation function layer receives the feature map output by the output end of the second batch normalization layer. The fully connected layer receives the feature map output by the output end of the second ReLU activation function layer. The maximum pooling layer receives the feature map output by the output end of the fully connected layer. The pooling result is the feature map output by the output end of the edge graph convolution block where it is located. The convolution kernel size of the first graph convolution layer in the convolution block is 1, the stride size is 1, the number of input channels is 6, and the number of output channels is 64. The convolution kernel size of the second graph convolution layer is 1, the stride size is 1, the number of input channels is 64, and the number of output channels is 64. The number of input channels of the fully connected layer is 32 and the number of output channels is 32. The convolution kernel size of the first graph convolution layer in the second edge graph convolution block is 1, the stride size is 1, the number of input channels is 128, and the number of output channels is 128. The convolution kernel size of the second graph convolution layer is 1, the stride size is 1, the number of input channels is 128, and the number of output channels is 128. The kernel size is 1, the stride size is 1, the number of input channels is 128, the number of output channels is 128, and the number of input channels of the fully connected layer is 32, and the number of output channels is 32; the kernel size of the first graph convolution layer in the third edge graph convolution block is 1, the stride size is 1, the number of input channels is 256, and the number of output channels is 256, the kernel size of the second graph convolution layer is 1, the stride size is 1, the number of input channels is 256, and the number of output channels is 256, and the number of input channels of the fully connected layer is 32, and the number of output channels is 32.

[0058] Continue to see Figure 1 In order to further verify the feasibility and effectiveness of the method of the present invention, the following experiments were conducted:

[0059] In the experiment, the public PUNet point cloud denoising dataset was directly selected for training and testing. Training was performed on the PUNet training set, and testing was performed on the PUNet test set.

[0060] Two widely used evaluation indicators are adopted, including chamfer distance , point to grid distance Specifically, the chamfer distance is a measure of the distance between two point sets. It calculates the sum of the average distance from the denoised point cloud to the original point cloud and the average distance from the original point cloud to the denoised point cloud. It is often used to evaluate point cloud registration and denoising effects. The point-to-grid distance is an important indicator for measuring the error between point cloud data and a three-dimensional mesh model. It can directly measure the degree of geometric matching between the point cloud and the mesh, and has a good evaluation capability for the retention of surface details. Figure 8 The denoising results of some point cloud data of the method of the present invention and the existing point cloud denoising method on the training set of PUNet are shown in the figure. The following Table 1 gives a quantitative comparison of the evaluation indicators of the method of the present invention and the common point cloud denoising methods on the test set of PUNet:

[0061] Table 1 Quantitative comparison of the evaluation indicators of the proposed method and common point cloud denoising methods on the PUNet test set

[0062] ;

[0063] from Figure 8 As can be seen from Table 1 above, the method of the present invention has better denoising performance than the common point cloud denoising methods.

[0064] In the description of the present invention, the description with reference to the terms "one embodiment", "some embodiments", "in the present embodiment", "specific example", or "some examples" etc. means that the specific features, mechanisms, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, mechanisms, materials or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0065] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A three-dimensional point cloud denoising method based on multi-stage information interaction, characterized in that: The following steps are involved: Step S1, for at least one three-dimensional object, obtain a data set of the three-dimensional object, the data set includes a plurality of clean point clouds, and pre-process each of the clean point clouds to obtain a corresponding noise point cloud, and randomly extract a point cloud of size 1 from each of the clean point clouds and each of the noise point clouds. Noise blocks, then mixing each of the clean point clouds and each of the noise point clouds and randomly dividing them into a training set and a test set; Step S2, using a deep learning framework to build a 3D point cloud denoising network, the 3D point cloud denoising network adopts a multi-stage framework and each stage includes an encoder module and a decoder module, and a cross-stage cross-encoding module is provided between two adjacent stages, the cross-stage cross-encoding module is respectively connected to the encoder module and the decoder module of the two adjacent stages, and the noise block is received through the encoder module and the feature map is output , receiving the feature map through the decoder module And output point cloud displacement , receiving the point cloud displacement at each stage through the output module And combined with the noise block to obtain a multi-stage denoising result, where the point cloud displacement output in the previous stage is said noise block as input to the next stage; Step S3, extracting a plurality of training noise blocks from the training set, and training the three-dimensional point cloud denoising network using the training set according to step S2 to obtain the multi-stage denoising result corresponding to each of the training noise blocks; Step S4, training the three-dimensional point cloud denoising network for multiple rounds according to step S3 to obtain a three-dimensional point cloud denoising network model; Step S5, extracting multiple test noise blocks from the test set, testing each of the test noise blocks through the three-dimensional point cloud denoising network model, and outputting the multi-stage denoising result corresponding to each of the test noise blocks as the three-dimensional point cloud denoising result.

2. The three-dimensional point cloud denoising method based on multi-stage information interaction according to claim 1, characterized in that: In step S1, the preprocessing method adopted is: Each of the clean point clouds is sequentially normalized, Gaussian noise with different noise levels is added, randomly scaled up or down, and randomly rotated to obtain the corresponding noisy point cloud.

3. The three-dimensional point cloud denoising method based on multi-stage information interaction according to claim 1, characterized in that: In step S2, each of the encoder modules includes three adaptive edge graph convolution modules with the same structure connected in sequence, and are defined as the first adaptive edge graph convolution module, the second adaptive edge graph convolution module and the third adaptive edge graph convolution module according to the information transmission direction. The input end of the first adaptive edge graph convolution module is used as the input end of the encoder module, and the noise block is received through the first adaptive edge graph convolution module and the feature map is output. , receiving the feature map through the second adaptive edge map convolution module And output feature map , receiving the feature map through the third adaptive edge map convolution module And output feature map , the feature map , the feature map and the characteristic graph Splice into the feature map ; wherein the feature map The size is , the feature map The size is , the feature map The size is , the feature map The size is .

4. The three-dimensional point cloud denoising method based on multi-stage information interaction according to claim 3 is characterized in that: Each of the adaptive edge graph convolution modules includes a graph building block, an adaptive edge attention block and an edge graph convolution block connected in sequence. The first adaptive edge graph convolution module includes the first graph building block, the first adaptive edge attention block and the first edge graph convolution block. The second adaptive edge graph convolution module includes the second graph building block, the second adaptive edge attention block and the second edge graph convolution block. The third adaptive edge graph convolution module includes the third graph building block, the third adaptive edge attention block and the third edge graph convolution block. The noise block is received through the first graph building block and the graph structure is output. , receiving the graph structure through the first adaptive edge attention block And output the graph structure , receiving the graph structure through the first edge graph convolution block And output the feature map , receiving the feature map through the second map building block and the noise block and output graph structure , receiving the graph structure through the second adaptive edge attention block And output the graph structure , receiving the graph structure through the second edge graph convolution block And output the feature map , receiving the feature map through the third map building block and the noise block and output graph structure , receiving the graph structure through the third adaptive edge attention block And output the graph structure , receiving the graph structure through the third edge graph convolution block And output the feature map .

5. The three-dimensional point cloud denoising method based on multi-stage information interaction according to claim 4 is characterized in that: In step S2, each of the graph building blocks receives the noise block and the high-dimensional noise block obtained by extracting semantic features from the noise block, takes each point in the noise block and the high-dimensional noise block as the centroid, defines the neighboring points of the centroid based on the geometric information and the semantic information according to the Euclidean algorithm, connects the centroid with the neighboring points, and constructs a geometric graph based on the geometric information and the semantic information. and semantic graph , and then the geometric figure and the semantic graph The graph structure is obtained by aggregating different neighborhoods of Or the graph structure Or the graph structure Output.

6. The three-dimensional point cloud denoising method based on multi-stage information interaction according to claim 3, characterized in that: In step S2, each of the decoder modules includes five fully connected layers connected in sequence, and are defined as the first fully connected layer, the second fully connected layer, the third fully connected layer, the fourth fully connected layer and the fifth fully connected layer according to the information transmission direction, and the feature map is received through the first fully connected layer And output feature map , receiving the feature map through the second fully connected layer And output feature map , receiving the feature map through the third fully connected layer And output feature map , receiving the feature map through the fourth fully connected layer And output feature map , receiving the feature map through the fifth fully connected layer And output the point cloud displacement ; wherein the feature map The size is , the feature map The size is , the feature map The size is , the feature map The size is , the point cloud displacement The size is .

7. The three-dimensional point cloud denoising method based on multi-stage information interaction according to claim 6, characterized in that: In step S2, the cross-stage and cross-coding modules between the adjacent T-th stage and the T+1-th stage update the feature maps output by each of the adaptive edge graph convolution modules and the first fully connected layer in the T+1-th stage through four branches, wherein the execution steps of the first branch include: The feature map output by the first adaptive edge graph convolution module in the Tth stage And the feature map output by the fourth fully connected layer As input, two layers of size 2 are obtained through a fully connected layer with an output channel number of 64 and a ReLU activation function layer. Feature map and feature map , and the feature map and the characteristic graph Perform addition operation to obtain feature map , and then the feature map The feature map output by the first adaptive edge graph convolution module in the T+1 stage Add, and after performing information interaction, the feature map output by the first adaptive edge graph convolution module in the T+1 stage Updated to ; The execution steps of the second branch include: The feature map output by the second adaptive edge graph convolution module in the Tth stage And the feature map output by the third fully connected layer As input, two layers of size 128 are obtained through a fully connected layer with an output channel number of 128 and a ReLU activation function layer. Feature map and feature map , and the feature map and the characteristic graph Perform addition operation to obtain feature map , and then the feature map The feature map output by the second adaptive edge graph convolution module in the T+1 stage Add, and after performing information interaction, the feature map output by the second adaptive edge graph convolution module in the T+1 stage Updated to ; The execution steps of the third branch include: The feature map output by the third adaptive edge graph convolution module in the Tth stage And the feature map output by the second fully connected layer As input, two layers of size 256 are obtained through a fully connected layer with an output channel number of 256 and a ReLU activation function layer. Feature map and feature map , and the feature map and the characteristic graph Perform addition operation to obtain feature map , and then the feature map The feature map output by the third adaptive edge graph convolution module in the T+1 stage Add, and after performing information interaction, the feature map output by the third adaptive edge graph convolution module in the T+1 stage Updated to ; The execution steps of the fourth branch include: The feature map output by the first fully connected layer in the Tth stage As input, a fully connected layer with an output channel number of 512 and a ReLU activation function layer are used to obtain a size of Feature map , and the feature map And the feature map output by the first fully connected layer in the T+1 stage Channel splicing is performed to obtain a size of Feature map , and then the feature map Through a fully connected layer with an output channel number of 512 and a ReLU activation function layer, the size is Feature map , and the feature map The feature map output by the first fully connected layer in the T+1 stage Add, and after performing information interaction, the feature map output by the first fully connected layer in the T+1 stage Updated to .

8. The three-dimensional point cloud denoising method based on multi-stage information interaction according to claim 4, characterized in that: In step S2, each of the adaptive edge attention blocks receives the graph structure output by the graph building block. Or the graph structure Or the graph structure After that, according to the graph structure Or the graph structure Or the graph structure The centroid feature and the edge feature are calculated, and the edge feature is passed through a fully connected layer with an output channel number of 1 and a Sigmod activation function layer to obtain the edge attention weight, and the edge feature is multiplied by the edge attention weight to obtain the weighted edge feature, and then the weighted edge feature and the centroid feature are added to obtain the corresponding feature map or feature map or feature map Output.

9. The three-dimensional point cloud denoising method based on multi-stage information interaction according to claim 4, characterized in that: In step S2, each of the edge graph convolution blocks includes two graph convolution layers, two batch normalization layers, two ReLU activation function layers, a fully connected layer and a maximum pooling layer, and the two graph convolution layers are defined as the first graph convolution layer and the second graph convolution layer according to the information transmission direction, the two batch normalization layers are defined as the first batch normalization layer and the second batch normalization layer, the two ReLU activation function layers are defined as the first ReLU activation function layer and the second ReLU activation function layer, the first graph convolution layer is connected to the first batch normalization layer, the first batch normalization layer is connected to the first ReLU activation function layer, the first ReLU activation function layer is connected to the second graph convolution layer, the second graph convolution layer is connected to the second batch normalization layer, the second batch normalization layer is connected to the second ReLU activation function layer, the second ReLU activation function layer is connected to the fully connected layer, and the fully connected layer is connected to the maximum pooling layer.

10. The three-dimensional point cloud denoising method based on multi-stage information interaction according to claim 9, characterized in that: The convolution kernel size of the first graph convolution layer in the first edge graph convolution block is 1, the stride size is 1, the number of input channels is 6, and the number of output channels is 64. The convolution kernel size of the second graph convolution layer in the first edge graph convolution block is 1, the stride size is 1, the number of input channels is 64, and the number of output channels is 64. The number of input channels of the fully connected layer in the first edge graph convolution block is 32, and the number of output channels is 32. The convolution kernel size of the first graph convolution layer in the second edge graph convolution block is 1, the stride size is 1, the number of input channels is 128, and the number of output channels is 128. The second graph convolution layer in the second edge graph convolution block The convolution kernel size is 1, the stride size is 1, the number of input channels is 128, and the number of output channels is 128. The number of input channels of the fully connected layer in the second edge graph convolution block is 32, and the number of output channels is 32. The convolution kernel size of the first graph convolution layer in the third edge graph convolution block is 1, the stride size is 1, the number of input channels is 256, and the number of output channels is 256. The convolution kernel size of the second graph convolution layer in the third edge graph convolution block is 1, the stride size is 1, the number of input channels is 256, and the number of output channels is 256. The number of input channels of the fully connected layer in the third edge graph convolution block is 32, and the number of output channels is 32.

Images