A point cloud completion method based on attention mechanism

Abstract

A point cloud completion method based on an attention mechanism, firstly obtains the data set required by the point cloud completion algorithm, sends the residual point cloud to the encoder to extract the feature vector of the residual point cloud, and then sends the feature vector to the decoder to generate a complete point cloud; the quality of the generated point cloud is evaluated by using the chamfer distance between the generated point cloud and the real point cloud, which is used as a loss function to guide the update of the neural network parameters; finally, the actual scanned point cloud data is sent to the trained encoder and decoder to generate a complete point cloud. The present invention introduces the AGFA module to extract features from the existing geometric structure, ensuring the authenticity of the shape of the completed point cloud and better detailed structure. The CGFA module is used to better refer to the features of the previous rough point cloud in the point cloud generation stage, and generate a higher quality high-resolution point cloud based on the low-resolution point cloud. The surface of the completed point cloud is smooth, with fewer noise points, which conforms to the distribution of the real point cloud.

Description

Technical Field

[0001] The present invention relates to a shape completion method for three-dimensional point cloud data, and in particular to a method for completing incomplete point cloud shapes using an attention mechanism. Background Art

[0002] 3D data has increasingly important applications in fields such as autonomous driving, robotics, remote sensing, and medicine. With the rapid development of data acquisition technology, people have more extensive use of 3D sensors, including various types of 3D scanners, lidars, and RGB-D cameras (such as Kinect, RealSense, and Apple Depth cameras). Three-dimensional data can usually be represented in data formats such as depth images, point clouds, meshes, and voxels. Among them, point clouds have become the preferred data format for many three-dimensional scene understanding tasks (target classification, target detection, target segmentation, etc.) due to their advantage of being able to maintain the original geometric information of the three-dimensional space. However, due to the limitations of the viewing angle and resolution of the 3D sensor and the occlusion of interference objects, the acquired point clouds are often highly sparse and incomplete point clouds, resulting in the loss of object geometry and semantic information, affecting the effect of subsequent point cloud scene recognition and understanding. Therefore, reconstructing a complete point cloud from the incomplete point cloud, that is, point cloud completion, such as Figure 1 As shown in (a) and (b), it is of great significance in practical applications.

[0003] Traditional 3D shape completion mainly includes two methods: geometric regularity completion and template matching completion (Liu Xinpu, Ma Yanxin, et al. Multi-scale point cloud completion embedded in Transformer structure [J]. Journal of Image and Graphics, 2022, 27(02): 538-549). In the geometric regularity completion method, Zhao et al. used a smooth interpolation algorithm to complete the missing parts of the 3D shape ("A robust hole-filling algorithm for triangular mesh." Visual Computer 23, 987–997.2007, doi: 10.1007 / s00371-007-0167-y). Mitra et al. identified the symmetry axis and repeated structure in the input shape ("Symmetry in 3D geometry: Extraction and applications." in Computer Graphics Forum, 32: 1-23.2013, doi: 10.1111 / cgf.12010) in order to use symmetry and repetitive rules for shape completion. In the template matching completion method, Li et al. achieve the completion operation by matching the input shape with the model in the shape database ("Database-Assisted Object Retrieval for Real-Time 3D Reconstruction." Computer Graphics Forum, 34(2):435-446.2015, doi:10.1111 / cgf.12573). These traditional shape completion methods require that the input must be as complete as possible, that is, there is a high lower limit requirement for the incompleteness of the shape, and the robustness to new objects and environmental noise is poor.

[0004] In recent years, with the development of neural network technology and the improvement of computer computing power, point cloud completion algorithms based on deep learning technology have begun to emerge. They are mainly divided into three categories: voxel grid-based methods, point-based multi-layer perceptron methods, and graph convolution-based methods.

[0005] Voxel is short for volume pixel, which is similar to the smallest unit of two-dimensional space - pixel. As its name suggests, volume pixel is the smallest unit of digital data in three-dimensional space partition, and is used in fields such as three-dimensional imaging, scientific data and medical video. Dividing point cloud data into voxels makes it easier to apply three-dimensional convolution (3D-CNN) operations on point cloud data. GRNet (Gridding Residual Network) is a representative of voxel grid-based point cloud completion methods (Xie HZ, Yao HX,: Gridding residual network for dense point cloud completion [C] / / Proceedings of European Conference on Computer Vision. 2020). In the voxelization layer, it first uses the interpolation function to weight the eight vertices of the three-dimensional grid unit where the point is located to measure the geometric relationship of the point cloud. Then, the three-dimensional convolution operation is used to learn the shape and spatial features of the point cloud, allowing the network to complete the missing parts of the incomplete point cloud. In the devoxelization layer, the output 3D grid is converted into a complete point cloud by replacing each 3D grid cell with a new point whose coordinates are the weighted sum of the eight vertices of the grid cell. Although the network introduces a three-dimensional convolution operation to extract the detailed features of the point cloud, the voxelization process brings a large computational overhead, which limits the generation of higher-resolution point clouds and is not conducive to the practical application of the point cloud completion algorithm.

[0006] The pioneer of the point-based multilayer perceptron method is the PointNet network, which directly takes the point cloud as input, uses multiple fully connected networks (MLP: Multilayer Perceptron) to independently learn the point cloud features point by point, and finally uses the maximum pooling operation to extract the global features. Since the maximum function is a symmetric function, PointNet has a good solution to the permutation invariance problem of point cloud arrangement. The point completion network (PCN: PointCompletion Network) proposed by Wentao Yuan et al. is one of the first algorithms to apply the PointNet idea (W.Yuan, T.Khot, "PCN: PointCompletion Network," 2018 International Conference on 3D Vision (3DV), 2018, pp.728-737, doi:10.1109 / 3DV.2018.00088). PCN operates directly on the original point cloud without any structural assumptions or prior information of the point cloud data. It adopts the design of codec network. On the encoder side, PCN uses a two-layer PointNet structure connected in series to extract the feature information of the residual point cloud. On the decoder side, PCN adopts a coarse-first-fine point cloud completion mode. The coarse point cloud is first generated by the fully connected layer, and then the refined point cloud is generated by the manifold fitting operation. Although the PointNet network directly uses the neural network algorithm at the point cloud level for the first time, which improves the computational efficiency, it only considers the global structural features and ignores the local detail features of the point cloud. In addition, the maximum pooling function causes a certain amount of information loss, which restricts the effectiveness of the algorithm.

[0007] Graph neural network is a recent emerging neural network algorithm. Its topological structure of points and edges can naturally be applied to point clouds and their neighborhood structures. DGCNN is a representative work on the use of graph neural networks in point clouds (Wang, Yue; Sun,. Dynamic Graph CNN for Learning on Point Clouds [J]. ACM Transactions on Graphics, 2019, Vol. 38 (5): 146). Its network structure is similar to that of PointNet, except that the structure for extracting features is modified to edge convolution (EdgeConv). In the feature dimension, the neighborhood of the point cloud is selected according to the Euclidean distance for feature aggregation, solving the problem of processing local features that PointNet does not have. However, it does not get rid of the limitation of Euclidean distance, making it difficult to fully query the neighborhood of the point cloud, limiting the ability to extract local features of the point cloud. Summary of the invention

[0008] The purpose of this invention is to provide a point cloud completion method based on attention mechanism.

[0009] The present invention proposes a point cloud completion algorithm based on the attention mechanism, which is divided into two stages: training and implementation. The training stage includes four parts, namely, obtaining the point cloud completion data set, extracting the residual point cloud feature vector, generating the complete point cloud, and generating the point cloud quality assessment. The implementation stage includes three parts, namely, obtaining the actual scanned point cloud data, extracting the residual point cloud feature vector, and generating the complete point cloud.

[0010] Training phase: First, obtain the data set required by the point cloud completion algorithm, which contains several point cloud pairs (incomplete / complete). Then send the incomplete point cloud to the encoder to extract the feature vector of the incomplete point cloud. Then send the obtained feature vector to the decoder to generate a complete point cloud. Finally, the quality of the generated point cloud is evaluated, and the chamfer distance between the generated point cloud and the real point cloud is calculated. It is used as a loss function to guide the update of the neural network parameters in the algorithm until the quality of the generated point cloud meets the requirements or the maximum number of training rounds is reached.

[0011] Implementation phase: First, obtain the actual scanned point cloud data. Generally speaking, the scanned point cloud data is incomplete. Send this incomplete point cloud into the algorithm that has been trained in the training phase, and finally generate a complete point cloud by extracting the feature vector of the incomplete point cloud.

[0012] The encoder extracts the feature vector of the residual point cloud, including: 1. Obtaining the point cloud completion dataset

[0013] The data set used by the algorithm proposed in the present invention is a public data set of the point completion network, which can be publicly downloaded on the Internet.

[0014] 2 Extraction of residual cloud feature vector

[0015] After obtaining the residual cloud data P in After that, it is sent to the encoder. The structure of the encoder is a position encoding module, four series-connected adaptive global feature aggregation modules and a maximum pooling module. The position encoding module consists of two series-connected multi-layer perceptron MLP layers to enhance the feature dimension of the point cloud. The output of the position encoding module is sent to the adaptive global feature aggregation module (AGFA: Adaptive global feature augmentation module). There are four adaptive global feature aggregation modules in total, which are series structures and are used to extract and aggregate the feature information of the residual point cloud. The output result of the last adaptive global feature aggregation module 4 is divided into two paths. One path is sent to the complete point cloud generation module for subsequent processing as an input of the cross-global feature aggregation module. The other path is subjected to the maximum pooling operation to obtain the residual point cloud feature vector f. The residual point cloud feature vector f is expressed as follows:

[0016] f=Max-pool(AGFA (4) (MLP(P in )))

[0017] Max-pool in the formula represents the maximum pooling operation, and the superscript (4) represents the iteration through 4 series-connected AGFA modules.

[0018] The structure of the adaptive global feature aggregation module is as follows: the output of the second multi-layer perceptron MLP of the position encoding module is used as the "value" matrix and the "key" matrix, and the down-sampling of the output of the second multi-layer perceptron MLP of the position encoding module is used as the "query" matrix, and the "value" matrix, the "key" matrix and the "query" matrix are sent to the multi-head attention module 1 together, and the multi-head attention module 1 is implemented by the Pytorch deep learning toolkit. Then the output result of the multi-head attention module 1 and the "query" matrix form a cross connection, and the output after the cross connection is normalized and sent to the multi-layer perceptron MLP2. The output of the multi-layer perceptron MLP2 and the normalized output constitute the second cross connection again, and the output result of the second cross connection replaces the output of the second multi-layer perceptron MLP of the position encoding module last time. After repeating this process 3 times, the aggregate feature matrix is ​​obtained. The aggregate feature matrix is ​​sent to the average pooling and multi-layer perceptron to obtain the channel attention weight matrix, and the channel attention weight matrix is ​​multiplied by the aggregate feature matrix to obtain the output of the adaptive global feature aggregation module, that is, the residual cloud feature vector.

[0019] The decoder generates a complete point cloud, including the following process: 3 Generate a complete point cloud

[0020] After obtaining the residual point cloud feature vector, it is sent to the decoder. The structure of the decoder is the rough point cloud generation module, the point cloud refinement module 1 and the point cloud refinement module 2. The output of the decoder is the final complete point cloud P out .

[0021] In the rough point cloud generation module, the residual point cloud feature vector is aggregated with geometric features through the multi-layer perceptron and the deconvolution layer in turn, and the output of the deconvolution layer and the output of the adaptive global feature aggregation module 4 are sent to the cross global feature aggregation module 1 (CGFA: Cross global feature augmentation module), wherein the structure of the cross global feature aggregation module 1 is to use the output of the deconvolution layer as the "value" matrix, the "key" matrix and the "query" matrix, and send them to the multi-head attention module 2, which is implemented by the Pytorch deep learning toolkit. Then the output of the multi-head attention module 2 and the "query" matrix form a cross connection. The output after the cross connection is normalized and sent to the multi-layer perceptron MLP3. The output of the multi-layer perceptron MLP3 and the normalized output constitute a second cross connection again, and the output of the second cross connection replaces the output of the deconvolution layer, and then the process is repeated 3 times to obtain the aggregated feature matrix. Then the aggregate feature matrix is ​​sent to the average pooling and multi-layer perceptron MLP4 to obtain the channel attention weight matrix. The channel attention weight matrix is ​​multiplied by the aggregate feature matrix to obtain the output of the cross-global feature aggregation module. The output result of the cross-global feature aggregation module is sent to the multi-layer perceptron MLP5 and connected in parallel with the original residual point cloud according to the number of points. That is, the two point clouds are fused into one point cloud, and then the coarse point cloud P is obtained by downsampling. coarse The output of the cross-global feature aggregation module 1 is also used as the pre-module input of the cross-global feature aggregation module 2 sent to the point cloud refinement module.

[0022] The point cloud refinement module sends the obtained coarse point cloud and the obtained residual point cloud feature vectors into the multilayer perceptron respectively. The outputs of the two multilayer perceptrons are connected in parallel. The parallel output and the output of the cross-global feature aggregation module 1 are sent to the cross-global feature aggregation module 2 together. The structure of the cross-global feature aggregation module 2 is exactly the same as that of the cross-global feature aggregation module 1. Its output is then connected in parallel with the multilayer perceptron output result of the coarse point cloud, and then sent to the multilayer perceptron MLP6 for feature fusion. The output of the multilayer perceptron MLP6 after feature fusion forms a cross-layer connection with the coarse point cloud to output a refined point cloud.

[0023] Two or more point cloud refinement modules can be connected in series.

[0024] 4 Quality assessment of generated point clouds

[0025] The present invention adopts chamfer distance to calculate the difference between the generated point cloud and the real point cloud, which is used as a loss function to guide the updating of the neural network parameters in the second and third part algorithms of the training stage until the maximum training rounds are reached.

[0026] The loss function is calculated as follows:

[0027] L = d CD (P coarse ,GT coarse )+d CD (P mid ,GT mid )+d CD (P out ,GT out )

[0028] Where P coarse , P mid and P out They correspond to the generated coarse point cloud and two higher resolution fine point clouds (the specific resolution depends on the task), GT coarse GT mid and GT out It is the result of downsampling the real point cloud in the data set according to the same number of points as the generated point cloud. All downsampling operations in the present invention adopt the farthest point sampling operation.

[0029] The calculation formula of chamfer distance is as follows:

[0030]

[0031] Where P and Q represent two point clouds, and p and q are points in the P point cloud and the Q point cloud respectively.

[0032] In the implementation phase, we first obtain the actual scanned point cloud data, downsample the number of points in the residual point cloud, and then send the residual point cloud to the algorithm model that has been trained in the training phase to obtain the complete point cloud after output completion.

[0033] Due to the introduction of the adaptive global feature aggregation module AGFA, the present invention pays more attention to feature extraction of existing geometric structures while completing the point cloud, ensuring the authenticity of the completed point cloud shape and completing better detail structures. The cross-global feature aggregation module CGFA is adopted, so that the algorithm can better refer to the features of the previous rough point cloud in the point cloud generation stage, and generate higher quality high-resolution point cloud based on the low-resolution point cloud, so that the surface of the completed point cloud result is smoother and has fewer noise points; the extraction of channel attention weights in this algorithm makes the density of the completed point cloud more uniform and more in line with the distribution of the real point cloud. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Schematic diagram of reconstructing a complete point cloud from an incomplete point cloud, where (a) is before completion and (b) is after completion;

[0035] Figure 2 This is a flow chart of the algorithm training phase of the present invention;

[0036] Figure 3 It is a flow chart of the implementation phase of the algorithm of the present invention;

[0037] Figure 4 A flow chart of extracting residual defect cloud feature vectors according to the present invention;

[0038] Figure 5 This is a structural diagram of the adaptive global feature aggregation module of the present invention;

[0039] Figure 6 Generate a complete point cloud flow chart for the present invention;

[0040] Figure 7 This is a structural diagram of the rough point cloud generation module of the present invention;

[0041] Figure 8 This is a structural diagram of the point cloud refinement module of the present invention;

[0042] Fig. 9 This is a structural diagram of the cross-global feature aggregation module of the present invention;

[0043] Fig.10 This is a visual comparison of the point cloud completion effect of the algorithm proposed in the present invention and the existing algorithm. (a) is an example of a residual point cloud map, (b) is an example of a point cloud map obtained by an existing algorithm, (c) is an example of a point cloud map obtained by the algorithm of the present invention, and (d) is an example of a real point cloud map. DETAILED DESCRIPTION

[0044] In order to explain the present invention more clearly, the present invention is further described in detail below in conjunction with the accompanying drawings. The accompanying drawings in the following description are only specific embodiments of the present invention and do not constitute a limitation to the present invention.

[0045] The present invention is divided into two stages: training and implementation. The training stage includes four parts, namely, obtaining point cloud completion data set, extracting residual point cloud feature vectors, generating complete point cloud and generating point cloud quality assessment. The process is as follows: Figure 2 The implementation phase consists of three parts: obtaining the actual scanned point cloud data, extracting the residual point cloud feature vector, and generating a complete point cloud. Figure 3 shown.

[0046] The present invention comprises the following specific process:

[0047] 1. Get the point cloud completion dataset

[0048] The data set used by the algorithm proposed in the present invention is a public data set of the point completion network, which can be publicly downloaded on the Internet.

[0049] 2 Extraction of residual cloud feature vector

[0050] The process of extracting residual defect cloud feature vector is as follows: Figure 4 shown.

[0051] After obtaining the residual cloud data P in (size is 2048,3), and then sent to the encoder. The structure of the encoder is a position encoding module, 4 series-connected adaptive global feature aggregation modules and a maximum pooling module. The position encoding module consists of two series-connected multi-layer perceptron MLP layers. The output of the position encoding module is sent to the adaptive global feature aggregation module (AGFA: Adaptive global feature augmentation module). There are four adaptive global feature aggregation modules in total, which are series structures and are used to extract and aggregate the feature information of the residual point cloud. The output result of the last adaptive global feature aggregation module 4 is divided into two paths. One path is sent to the complete point cloud generation module for subsequent processing as an input of the cross global feature aggregation module. The other path is subjected to the maximum pooling operation to obtain the residual point cloud feature vector f (size is 1×512). The residual point cloud feature vector f is expressed as follows:

[0052] f=Max-pool(AGFA (4) (MLP(P in )))

[0053] Max-pool in the formula represents the maximum pooling operation, and the superscript (4) represents the iteration through 4 series-connected AGFA modules.

[0054] The structure of the adaptive global feature aggregation module is as follows: Figure 5 As shown in the figure, the output of the second multilayer perceptron MLP of the position encoding module is used as the "value" matrix and the "key" matrix, and the down-sampling of the output of the second multilayer perceptron MLP of the position encoding module is used as the "query" matrix. The "value" matrix, the "key" matrix and the "query" matrix are sent to the multi-head attention module 1 together, and the multi-head attention module 1 is implemented by the Pytorch deep learning toolkit. Then the output result of the multi-head attention module 1 and the "query" matrix form a cross connection. The output after the cross connection is normalized and sent to the multilayer perceptron MLP2. The output of the multilayer perceptron MLP2 and the normalized output constitute the second cross connection again. The output result of the second cross connection replaces the output of the second multilayer perceptron MLP of the position encoding module last time. After repeating this process 3 times, the aggregate feature matrix is ​​obtained. The aggregate feature matrix is ​​sent to the average pooling and multilayer perceptron to obtain the channel attention weight matrix. The channel attention weight matrix is ​​multiplied by the aggregate feature matrix to obtain the output of the adaptive global feature aggregation module, that is, the residual cloud feature vector.

[0055] 3 Generate a complete point cloud

[0056] The process of generating a complete point cloud is as follows Figure 6 As shown in the figure, after obtaining the residual point cloud feature vector, it is sent to the decoder. The structure of the decoder is the rough point cloud generation module, the point cloud refinement module 1 and the point cloud refinement module 2. The output of the decoder is the final complete point cloud P out (The size is 16384,3).

[0057] The rough point cloud generation module structure is as follows Figure 7 As shown, the residual cloud feature vector is aggregated with geometric features through the multi-layer perceptron and the deconvolution layer in turn, and the output of the deconvolution layer and the output of the adaptive global feature aggregation module 4 are sent to the cross global feature aggregation module 1 (CGFA: Cross global feature augmentation module), wherein the structure of the cross global feature aggregation module 1 is that the output of the deconvolution layer is used as the "value" matrix, the "key" matrix and the "query" matrix, and sent to the multi-head attention module 2, which is implemented by the Pytorch deep learning toolkit. Then the output of the multi-head attention module 2 and the "query" matrix form a cross connection. The output after the cross connection is normalized and sent to the multi-layer perceptron MLP3. The output of the multi-layer perceptron MLP3 and the normalized output constitute the second cross connection again. The output of the second cross connection replaces the output of the last deconvolution layer. Repeat this process 3 times to obtain the aggregated feature matrix. Subsequently, the aggregated feature matrix is ​​sent to the average pooling and the multi-layer perceptron MLP4 to obtain the channel attention weight matrix. The channel attention weight matrix is ​​multiplied by the aggregated feature matrix to obtain the output of the cross global feature aggregation module. The output result of the cross-global feature aggregation module is then sent to the multi-layer perceptron MLP5 and connected in parallel with the original residual point cloud according to the number of points. That is, the two point clouds are fused into one point cloud, and then the rough point cloud P is obtained by downsampling. coarse (Size is 512,3). The output of cross-global feature aggregation module 1 is also used as the pre-module input to cross-global feature aggregation module 2 sent to the point cloud refinement module.

[0058] The point cloud refinement module structure is as follows: Figure 8 As shown, the obtained coarse point cloud and the obtained residual point cloud feature vectors are respectively sent to the multilayer perceptron, and the outputs of the two multilayer perceptrons are connected in parallel. The parallel output and the output of the cross-global feature aggregation module 1 are sent to the cross-global feature aggregation module 2 together. The structure of the cross-global feature aggregation module 2 is exactly the same as that of the cross-global feature aggregation module 1. Its output is then connected in parallel with the multilayer perceptron output result of the coarse point cloud, and then sent to the multilayer perceptron MLP6 for feature fusion. The output of the multilayer perceptron MLP6 after feature fusion forms a cross-layer connection with the coarse point cloud, and a fine point cloud is output. Fig. 9 This is the structural diagram of the cross-global feature aggregation module.

[0059] Two or more point cloud refinement modules can be connected in series.

[0060] In this embodiment, there are two point cloud refinement modules in the training, and the sizes of the refined point clouds output by them are P mid (2048,3) and P out (16384,3).

[0061] 4 Quality assessment of generated point clouds

[0062] The present invention uses chamfer distance to calculate the difference between the generated point cloud and the real point cloud, which is used as a loss function to guide the update of the MLP neural network parameters in the training phase until the accuracy requirement or the maximum number of training rounds is reached.

[0063] The loss function is calculated as follows:

[0064] L = d CD (P coarse ,GT coarse )+d CD (P mid ,GT mid )+d CD (P out ,GT out )

[0065] Where P coarse , P mid and P out They correspond to the generated coarse point cloud and two higher resolution fine point clouds (the specific resolution depends on the task), GT coarse GT mid and GT out It is the result of downsampling the real point cloud in the data set according to the same number of points as the generated point cloud. All downsampling operations in the present invention adopt the farthest point sampling operation.

[0066] The calculation formula of chamfer distance is as follows:

[0067]

[0068] Where P and Q represent two point clouds, and p and q are points in the P point cloud and the Q point cloud respectively.

[0069] After 300 rounds of training, the quantitative and visual comparison results of the proposed algorithm and the existing optimal algorithm are shown in Tables 1 and 2. Fig.10 As shown, the comparison result is measured by the average chamfer distance of the category. The smaller the chamfer distance, the better the algorithm completion effect.

[0070] Table 1 Quantitative comparison of the point cloud completion effect of the proposed algorithm and the existing algorithm (×10 -3 )

[0071] algorithm average value airplane cupboard car Chair desk lamp sofa table Ship Existing Algorithms 7.21 4.29 9.16 8.08 7.89 6.07 9.23 6.55 6.40 Algorithm of the present invention 6.76 3.89 9.03 7.68 7.18 5.52 8.72 6.18 5.91

[0072] exist Fig.10 The advantages of the algorithm proposed in the present invention can be seen from the following: 1. Due to the introduction of the adaptive global feature aggregation module AGFA, the algorithm pays more attention to feature extraction of existing geometric structures while completing the point cloud, thereby ensuring the authenticity of the shape of the completed point cloud. For example, the engine of an airplane, the raised structures at both ends of the chair backrest, and the lampshade of a desk lamp have all been completed with better detail structures. 2. Due to the introduction of the cross-global feature aggregation module CGFA, the algorithm can better refer to the features of the previous rough point cloud during the point cloud generation stage, and generate a higher quality high-resolution point cloud based on the low-resolution point cloud, so that the surface of the completed point cloud result is smoother and has fewer noise points. 3. The extraction of channel attention weights in this algorithm makes the density of the completed point cloud more uniform and more consistent with the distribution of the real point cloud.

[0073] In the implementation phase, we first need to obtain the actual scanned point cloud data, which can be obtained using 3D scanning devices such as depth cameras or lidars, and then convert it into a point cloud format, and downsample the number of points in the residual point cloud to the same number of points as the algorithm input. Then, the residual point cloud is fed into the algorithm model that has been trained in the training phase, and then the completed point cloud can be output.

Claims

1. A point cloud completion method based on attention mechanism, characterized by It consists of two stages: 1) Training phase: First, the data set required by the point cloud completion algorithm is obtained, which contains several incomplete and complete point cloud pairs. The incomplete point cloud is sent to the encoder to extract the feature vector of the incomplete point cloud, and then the feature vector is sent to the decoder to generate a complete point cloud. The quality of the generated point cloud is evaluated, and the chamfer distance between the generated point cloud and the real point cloud is calculated as the loss function to guide the update of the neural network parameters in the encoder and decoder. 2) Implementation phase: Obtain the actual scanned point cloud data, send the point cloud data to the encoder and decoder that have been trained in the training phase, and generate a complete point cloud by extracting the feature vector of the residual point cloud; The encoder is used to generate a residual defect cloud feature vector: After obtaining the residual cloud data P in After that, it is sent to the encoder. The structure of the encoder is a position encoding module, four series-connected adaptive global feature aggregation modules and a maximum pooling module. The position encoding module is composed of two series-connected multi-layer perceptron MLP layers. The output of the second multi-layer perceptron MLP layer of the position encoding module is sent to the adaptive global feature aggregation module AGFA. There are four adaptive global feature aggregation modules in total, forming a series structure. The output result of the last adaptive global feature aggregation module 4 is divided into two paths, one of which is used as an input of the cross-global feature aggregation module 1, and the other is obtained after the maximum pooling operation to obtain the residual defect cloud feature vector f. The residual defect cloud feature vector f is expressed as follows: f=Max-pool(AGFA (4) (MLP(P in ))); Max-pool in the formula represents the maximum pooling operation, and the superscript (4) represents the iteration through 4 serially connected AGFA modules; The structure of the adaptive global feature aggregation module AGFA is as follows: the output of the second multilayer perceptron MLP of the position encoding module is used as the "value" matrix and the "key" matrix, and the down-sampling of the output of the second multilayer perceptron MLP of the position encoding module is used as the "query" matrix, and the "value" matrix, the "key" matrix and the "query" matrix are sent to the multi-head attention module 1 together. The multi-head attention module 1 is implemented by the Pytorch deep learning toolkit; then the output result of the multi-head attention module 1 is straddled with the "query" matrix, and the output after the straddle connection is normalized and sent to the multilayer perceptron. MLP2, the output of the multilayer perceptron MLP2 and the normalized output constitute the second cross connection again, and the output result of the second cross connection replaces the output of the second multilayer perceptron MLP of the last position encoding module. After repeating this process three times, the aggregated feature matrix is ​​obtained; the aggregated feature matrix is ​​sent to the average pooling and multilayer perceptron to obtain the channel attention weight matrix, and the channel attention weight matrix is ​​multiplied by the aggregated feature matrix to obtain the output of the adaptive global feature aggregation module, and then the output of the adaptive global feature aggregation module is sent to the maximum pooling layer. After maximum pooling, the residual cloud feature vector f is obtained; The decoder is used to generate a complete point cloud; After obtaining the residual point cloud feature vector, it is sent to the decoder. The structure of the decoder is the rough point cloud generation module, the point cloud refinement module 1 and the point cloud refinement module 2. The output of the decoder is the final complete point cloud P out ; In the rough point cloud generation module, the residual point cloud feature vector is aggregated with geometric features through a multi-layer perceptron and a deconvolution layer in turn, and the output of the deconvolution layer and the output of the adaptive global feature aggregation module 4 are sent to a cross-global feature aggregation module 1 together. The structure of the cross-global feature aggregation module 1 is as follows: the output of the deconvolution layer is used as a "value" matrix, a "key" matrix and a "query" matrix, and is sent to a multi-head attention module 2. The head attention module 2 is implemented by the Pytorch deep learning toolkit; then the output result of the multi-head attention module 2 is straddled with the "query" matrix, and the output after the straddle connection is normalized and sent to a multi-layer perceptron MLP3. The output of the layer perceptron MLP3 and the normalized output constitute the second cross connection again, and the output result of the second cross connection replaces the output of the deconvolution layer, and then the process is repeated 3 times to obtain the aggregated feature matrix; the aggregated feature matrix is ​​sent to the average pooling and multi-layer perceptron MLP4 to obtain the channel attention weight matrix, and the channel attention weight matrix is ​​multiplied by the aggregated feature matrix to obtain the output of the cross-global feature aggregation module; the output result of the cross-global feature aggregation module is sent to the multi-layer perceptron MLP5, and the output of the multi-layer perceptron MLP5 is connected in parallel with the original residual point cloud according to the number of points, that is, the two point clouds are fused into one point cloud, and then the coarse point cloud P is obtained by downsampling. coarse ; The output of the cross-global feature aggregation module 1 is also used as the pre-module input of the cross-global feature aggregation module 2 sent to the point cloud refinement module; The point cloud refinement module sends the obtained coarse point cloud and the obtained residual point cloud feature vectors to the multilayer perceptron respectively. The outputs of the two multilayer perceptrons are connected in parallel. The parallel output and the output of the cross-global feature aggregation module 1 are sent to the cross-global feature aggregation module 2 together. The structure of the cross-global feature aggregation module 2 is exactly the same as that of the cross-global feature aggregation module 1. Its output is then connected in parallel with the multilayer perceptron output result of the coarse point cloud, and then sent to the multilayer perceptron MLP6 for feature fusion. The output of the multilayer perceptron MLP6 after feature fusion forms a cross-layer connection with the coarse point cloud, and outputs a refined point cloud. The structure of the point cloud refinement module 2 is exactly the same as that of the point cloud refinement module 1.

2. A point cloud completion method based on an attention mechanism as claimed in claim 1, characterized in that: The quality assessment of the generated point cloud uses the chamfer distance to calculate the difference between the generated point cloud and the real point cloud, and the loss function is used to guide the update of the neural network parameters in the encoder and decoder in the training phase until the maximum training round is reached; The loss function is calculated as follows: L=d CD (P coarse ,GT coarse )+d CD (P mid ,GT mid )+d CD (P out ,GT out ); Where P coarse , P mid and P out Corresponding to the generated coarse point cloud and two fine point clouds, GT coarse GT mid and GT out The real point cloud in the dataset is downsampled according to the same number of points as the generated point cloud; The calculation formula of chamfer distance is as follows: Where P and Q represent two point clouds, and p and q are points in the P point cloud and the Q point cloud respectively.

Images