A large receptive field point cloud repairing method
By constructing a point cloud restoration network, identifying close points in the point cloud and extracting features using a large receptive field convolutional layer, the problem of inaccurate restoration caused by a small receptive field is solved, achieving more accurate point cloud restoration.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- QINGDAO UNIV OF TECH
- Filing Date
- 2023-09-05
- Publication Date
- 2026-05-01
AI Technical Summary
The existing technology suffers from inaccurate point cloud repair due to the small receptive field.
By constructing a point cloud restoration network, we first identify points that are close to each other in the point cloud, use convolutional layers with large receptive fields to extract highly abstract point features, and combine them with global features for restoration. The network includes an offset-attention embedding module, a rearrangement module, convolutional layers with convolutional kernels of different sizes, and transposed convolutional layers.
It achieves accurate extraction of highly abstract features from point cloud data, improving the accuracy of point cloud restoration.
Smart Images
Figure CN117237663B_ABST
Abstract
Description
A point cloud restoration method with a large receptive field Technical Field
[0001] This invention relates to the field of point cloud restoration technology, and more specifically, to a point cloud restoration method with a large receptive field. Background Technology
[0002] Point cloud classification, semantic segmentation, and object detection are fundamental and crucial perception tasks for intelligent vehicles. These tasks require a complete point cloud as input. However, in reality, due to differences in viewpoint, occlusion, and noise, the points acquired by 3D sensors may be incomplete. Therefore, repairing partial point clouds is an important task for intelligent vehicles.
[0003] Most point cloud inpainting methods can be categorized into three types based on whether they directly process point cloud data: voxel-based methods, point-based methods, and transformer-based methods. Voxel-based methods first convert unordered point clouds into ordered data through voxelization, and then inpaint them. For example, GRNet first converts the point cloud into a 3D mesh, uses a 3D convolutional neural network (CNN) for inpainting, and then converts the inpainted 3D mesh back into a point cloud. Conversely, point-based methods directly inpaint parts of the point cloud using a CNN with a kernel size of 1 and max-pooling operations. Besides the methods described above, transformers are being used in research on point cloud tasks. Similar to point-based methods, transformers also use convolutions with a kernel size of 1 to extract point cloud features.
[0004] However, a convolutional kernel of size 1 can only extract features from a single point, resulting in a small receptive field. When max pooling is applied to it, the obtained highly abstract features are not precise, and therefore, the final point cloud obtained is also inaccurate. Summary of the Invention
[0005] This invention provides a point cloud restoration method with a large receptive field, which solves the technical problem in the prior art where the point cloud obtained after restoration is inaccurate due to the small receptive field.
[0006] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:
[0007] A point cloud restoration method with a large receptive field includes the following steps:
[0008] S1: Acquire point cloud data;
[0009] S2: Construct a point cloud restoration network. After identifying the relative positions between points in the point cloud data, the point cloud restoration network puts the points that are close to each other together, uses a convolutional layer with a large receptive field to extract highly abstract point features of the point cloud data, then extracts the global features of the point cloud data, and obtains the restored point cloud based on the highly abstract point features and global features.
[0010] S3: Input the point cloud data obtained in step S1 into the point cloud repair network in step S2, and the point cloud repair network outputs the repaired point cloud.
[0011] In the aforementioned techniques, due to the disordered nature of point clouds, point cloud inpainting networks cannot simply use larger kernels to extract highly abstract point features. More specifically, several points that are close to each other can be abstracted into a single feature, while several points that are far apart cannot. Therefore, it cannot be guaranteed that points entering a larger kernel are close to each other. To address this issue, this scheme first groups closer points together, and then uses a larger convolutional kernel to process these points, extracting highly abstract point features and resolving the technical problem of inaccurate point clouds obtained after inpainting due to a small receptive field.
[0012] Furthermore, the point cloud inpainting network includes an offset-attention embedding module, a rearrangement module, pooling and repetition layers, convolutional layers with kernels of different sizes, and transposed convolutional layers with kernels of different sizes, wherein:
[0013] After the offset-attention embedding module identifies the importance of points in the point cloud data, the rearrangement module groups points that are close to each other together. The convolutional layers with different kernel sizes extract highly abstract point features of different dimensions from the point cloud data. The pooling and repetition layers extract global features from the point cloud data based on the output of the convolutional layers with different kernel sizes. The transposed convolutional layers with different kernel sizes obtain the repaired point cloud based on the highly abstract point features and global features of the point cloud data.
[0014] Furthermore, the offset-attention embedding module specifically comprises:
[0015] The input to the offset-attention embedding module is the position of the point, and the query matrix, key matrix, and value matrix are calculated based on the position of the point:
[0016] Q = P in ·W q
[0017] K = P in ·W k
[0018] V = P in·W v
[0019] In the formula, Q, K, and V represent the query matrix, key matrix, and value matrix, respectively, and P in W represents the position of a point. q W k W v Let each represent a weight matrix, and · denote matrix multiplication. The parameters satisfy the following:
[0020]
[0021]
[0022] Calculate the attention map AM based on the query matrix and key matrix:
[0023] AM=K T ·Q
[0024]
[0025] Calculate the attention feature AF based on the attention map AM and the value matrix:
[0026] AF=(l1_norm(softmaxAM)·(V))
[0027] In the formula, softmax represents the softmax operator, and l1_norm represents the l1 norm;
[0028] Subtract P from the attention feature AF in The results are then fed into a linear + batch normalized + rectified linear unit layer to calculate the offset attention P. out As the output of the offset-attention embedding module:
[0029] P out =αP in +β(LBR(P in -AF))
[0030] In the formula, α and β are weights, and LBR is a linear + batch normalized + rectified linear unit layer.
[0031] Furthermore, the rearrangement module specifically comprises:
[0032] Calculate the spatial relative distance of all points, use a clustering algorithm to find the k nearest points for each point, record the indices of the k nearest points, and arrange each point with its k nearest points in turn according to the index until all points are rearranged.
[0033] Furthermore, the clustering algorithm is K-means clustering, Mean-Shift clustering, or KNN.
[0034] Furthermore, the pooling operation in the pooling and repeating layers is max pooling.
[0035] Furthermore, the convolutional layers with different kernel sizes specifically include several convolutional layers with a kernel size of 1 and several convolutional layers with a kernel size of n, where n is greater than 1.
[0036] Furthermore, the transposed convolutional layers with different kernel sizes specifically include several transposed convolutional layers with a kernel size of 1 and several convolutional layers with a kernel size of m, wherein m is greater than 1.
[0037] Furthermore, the point cloud repair network also includes an optimization network, which performs high-precision optimization and repair on the repaired point cloud.
[0038] Furthermore, the optimized network includes several convolutional layers and upsampling and downsampling modules. The repaired point cloud is input to the upsampling and downsampling modules after passing through several convolutional layers. The output of the upsampling and downsampling modules is then passed through several convolutional layers to obtain a high-precision optimized and repaired complete point cloud.
[0039] The upsampling and downsampling module includes two downsampling units and two upsampling units. The upsampling and downsampling module passes through the two downsampling units and the two upsampling units in sequence, and finally outputs from the upsampling and downsampling module. In the downsampling unit, the feature map first obtains key points through the farthest point sampling operation, and then obtains features using two convolutional layers and pooling and repetition layers. In the upsampling unit, repetition and shaping operations are performed to ensure that the input and output sizes of the upsampling and downsampling module are equal.
[0040] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:
[0041] This invention proposes a point cloud restoration method with a large receptive field. First, points that are relatively close in the point cloud are grouped together. Then, a convolutional layer with a large receptive field is used to extract highly abstract point features from the point cloud data. This achieves accurate extraction of highly abstract point features from the point cloud data and improves the accuracy of point cloud restoration. Attached Figure Description
[0042] Figure 1 is a schematic flowchart of a point cloud restoration method with a large receptive field provided by an embodiment of the present invention.
[0043] Figure 2 is a schematic diagram of the point cloud repair network provided in an embodiment of the present invention.
[0044] Figure 3 is a schematic diagram of the offset-attention embedding module provided in an embodiment of the present invention.
[0045] Figure 4 is a schematic diagram of the rearrangement module provided in an embodiment of the present invention.
[0046] Figure 5 is a schematic diagram of the point cloud repair network including the optimized network provided in an embodiment of the present invention.
[0047] Figure 6 is a comparison of the restoration quality of a point cloud restoration method with a large receptive field provided in this embodiment of the invention with other point cloud restoration methods.
[0048] Figure 7 is a comparison of the restoration quality of a point cloud restoration method with a large receptive field provided by the present invention on the KITTI dataset, compared with other point cloud restoration methods. Detailed Implementation
[0049] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent.
[0050] To better illustrate this embodiment, some parts in the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions;
[0051] It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings.
[0052] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.
[0053] Example 1
[0054] A point cloud restoration method with a large receptive field, as shown in Figure 1, includes the following steps:
[0055] S1: Acquire point cloud data;
[0056] S2: Construct a point cloud restoration network. After identifying the relative positions between points in the point cloud data, the point cloud restoration network puts the points that are close to each other together, uses a convolutional layer with a large receptive field to extract highly abstract point features of the point cloud data, then extracts the global features of the point cloud data, and obtains the restored point cloud based on the highly abstract point features and global features.
[0057] S3: Input the point cloud data obtained in step S1 into the point cloud repair network in step S2, and the point cloud repair network outputs the repaired point cloud.
[0058] Based on the disordered nature of point clouds, this invention first identifies the relative positions of points in the point cloud, then groups points that are close to each other together, and then uses a convolutional layer with a large receptive field to extract precise, highly abstract point features, thereby accurately repairing the point cloud.
[0059] Example 2
[0060] Based on Example 1, this embodiment continues to provide the following specific embodiments:
[0061] The point cloud inpainting network, as shown in Figure 2, includes an Offset-Attention Embedding Module (ROA), a Rearrangement Module (RM), a Pooling & Repeat Layer (PR), Convolutional Layers with Kernels of Different Sizes (RC), and Transposed Convolutional Layers with Kernels of Different Sizes (RT), wherein:
[0062] After the offset-attention embedding module identifies the importance of points in the point cloud data, the rearrangement module groups points that are close to each other together. The convolutional layers with different kernel sizes extract highly abstract point features of different dimensions from the point cloud data. The pooling and repetition layers extract global features from the point cloud data based on the output of the convolutional layers with different kernel sizes. The transposed convolutional layers with different kernel sizes obtain the repaired point cloud based on the highly abstract point features and global features of the point cloud data.
[0063] In a further embodiment, a single self-attention mechanism module and an offset attention mechanism module take the embedded point features as input to obtain the correlation between them. Compared with the self-attention mechanism module, the offset-attention mechanism module may achieve better performance. The offset-attention embedding module can achieve better performance by obtaining the importance of each point, thus determining which points in the point cloud need to be repaired and which do not. The offset-attention embedding module is shown in Figure 3, and specifically:
[0064] The input to the offset-attention embedding module is the position of the point, and the query matrix, key matrix, and value matrix are calculated based on the position of the point:
[0065] Q = P in ·W q
[0066] K = P in ·W k
[0067] V = P in ·W v
[0068] In the formula, Q, K, and V represent the query matrix, key matrix, and value matrix, respectively, and P in W represents the position of a point. q W k W v Let each represent a weight matrix, and · denote matrix multiplication. The parameters satisfy the following:
[0069]
[0070]
[0071] The attention map AM is calculated based on the query matrix and key matrix. The attention map shows the positional relationship between a point and other points in the same point cloud.
[0072] AM=K T ·Q
[0073]
[0074] To visually demonstrate the contribution of each point, we apply the softmax operator and the l1 norm to AM. Then, we multiply it by V to obtain the attention feature AF:
[0075] AF=(l1-norm(softmaxAM)·(V))
[0076] In the formula, softmax represents the softmax operator, and l1-norm represents the l1 norm;
[0077] Subtract P from the attention feature AF in The results are then fed into a linear + batch normalized + rectified linear unit layer to calculate the offset attention P. out As the output of the offset-attention embedding module, the ultimate goal of this module is to identify regions in the point cloud that require and do not require restoration. To achieve this, when the input consists of points that do not require restoration, the output is equal to the input. When the output consists of points that require restoration, its output is a mapping of the input in 3D space. Therefore, weights are added to the results obtained above, and P is... in Add to the corresponding addition operation:
[0078] P out =αP in +β(LBR(P in -AF))
[0079] In the formula, α and β are weights, and LBR is a linear + batch normalized + rectified linear unit layer.
[0080] In a further embodiment, due to the disordered nature of point clouds, the input order of adjacent points may differ significantly, resulting in them lacking similar features and making it impossible to use convolutional kernel sizes larger than 1. To apply larger convolutional kernel sizes to point clouds, a rearrangement module is proposed. Figure 4 illustrates an example of the rearrangement process of this module, specifically:
[0081] Calculate the spatial relative distance of all points, use a clustering algorithm to find the k nearest points for each point, record the indices of the k nearest points, and arrange each point with its k nearest points in turn according to the index until all points are rearranged.
[0082] In a further embodiment, the clustering algorithm is K-means clustering, Mean-Shift clustering, or KNN. Since the algorithm is offline, the focus is primarily on its effectiveness, rather than excessively on its time consumption.
[0083] In a further embodiment, the pooling and pooling operations in the repeated layers are max pooling.
[0084] In a further embodiment, the convolutional layers with different kernel sizes specifically include several convolutional layers with a kernel size of 1 and several convolutional layers with a kernel size of n, wherein n is greater than 1.
[0085] In a further embodiment, the transposed convolutional layer with different kernel sizes specifically includes several transposed convolutional layers with a kernel size of 1 and several convolutional layers with a kernel size of m, wherein m is greater than 1.
[0086] Example 3
[0087] Based on Examples 1 and 2, this embodiment continues to provide the following specific embodiments:
[0088] The point cloud repair networks in Examples 1 and 2 can complete the structure of some point clouds, but their accuracy is still low. In order to improve the accuracy, an optimization network is designed based on the point cloud repair networks in Examples 1 and 2. The optimization network performs high-precision optimization and repair on the repaired point cloud. The point cloud repair network after adding the optimization network is shown in Figure 5.
[0089] The optimized network includes several convolutional layers (CC) and downsampling and upsampling modules (CDU). The repaired point cloud is input to the upsampling and upsampling modules after passing through several convolutional layers. The output of the upsampling and downsampling modules is then passed through several convolutional layers to obtain a high-precision optimized and repaired complete point cloud.
[0090] The upsampling and downsampling module includes two downsampling units and two upsampling units (UU). The upsampling and downsampling module passes through the two downsampling units and the two upsampling units in sequence, and finally outputs from the upsampling and downsampling module. In the downsampling unit, the feature map first undergoes farthest point sampling (FPS) to obtain key points, and then uses two convolutional layers and pooling and repetition layers to obtain features. In the upsampling unit, repetition and shaping operations are performed to ensure that the input and output sizes of the upsampling and downsampling module are equal.
[0091] In a further embodiment, the point cloud inpainting network with the optimized network is trained in an end-to-end manner. To control the training results, a weighted reconstruction network loss (Li) is used. rec ) and complete network loss (L com ) parameters. Therefore, the loss function Loss can be expressed as:
[0092] Loss=λL rec +γL com =λL EMD (Y rec ,Y)+γL EMD (Y com Y)
[0093] Among them, Y rec and Y com The points are obtained by the point cloud restoration network without optimization and the point cloud restoration network with optimization, respectively, where Y is the ground truth point, and λ and γ are L... rec and L com The weighted parameter, L EMD () represents the EMD distance, and the specific calculation formula is as follows:
[0094]
[0095] P and Q are the coordinate matrices of the two point clouds. This represents the mapping from P to Q.
[0096] In a specific embodiment, α and β are set to 1 and 0.1 respectively, k is 2, n is 3, and m is 4.
[0097] The point cloud inpainting network presented in this embodiment was trained and tested on the MVP dataset and compared with existing state-of-the-art methods. The MVP dataset, proposed by Pan, includes 62,400 training pairs and 41,600 test pairs. Unlike previous datasets such as PCN and Completion3D, MVP comprises high-quality synthetic partial scans of 16 categories (airplane, cabinet, car, chair, lamp, sofa, table, boat, bed, bench, bookshelf, bus, guitar, motorcycle, pistol, and skateboard) for 3D computer-aided design (CAD) models, making it one of the most challenging datasets available.
[0098] The point cloud inpainting network provided in this embodiment uses ReLU as its activation function and Adam as its optimization algorithm. The architecture parameters of the point cloud inpainting network are shown in Table 1. The point cloud inpainting network provided in this embodiment was tested using various resolutions (number of points = 2048, 4096) and compared with state-of-the-art work (PMP, PCN, TopNet, Wang et al., SnowflakeNet, SpareNet, and MSN). All models were trained on an NVIDIA TITAN X GPU with a batch size of 32 and 100 epochs.
[0099] Table 1. Network parameters for point cloud repair
[0100]
[0101] The qualitative comparison results (2048 points) are shown in Figure 6. It can be seen that all models can complete a partial point cloud, indicating that they have all effectively learned the structure of the MVP data. However, the accuracy of the repair results varies significantly. Specifically, SpareNet can repair the contours; however, the repaired point cloud has large holes. PCN and TopNet can fill in part of the missing point cloud, but they cannot preserve the input point cloud. The method proposed by Wang et al. can preserve most of the input details and achieves better results in the comparison. The point cloud repair network of this embodiment can perfectly fill in the missing parts and preserve the details of the input, most closely resembling the actual situation.
[0102] The overall quantitative comparison results are shown in Table 2. When the input point cloud has 2048 points and 4096 points, the EMD loss of the point cloud restoration network in this embodiment is 0.0357 and 0.0364, respectively, while the suboptimal model (MSN) is 0.0398 and 0.0396, respectively. Regardless of whether the input point cloud has 2048 or 4096 points, the loss of the point cloud restoration network in this embodiment is the smallest. The EMD loss results obtained for each category are shown in Table 3. Compared with other models, the EMD loss of the point cloud restoration network in this embodiment is the smallest, indicating that the point cloud restoration network in this embodiment achieves the best results in each category. Therefore, the point cloud restoration network in this embodiment outperforms the aforementioned state-of-the-art methods on the MVP dataset.
[0103] Table 2. EMD loss for each method (lower is better)
[0104]
[0105] Table 3 EMD loss for each category (EMD loss multiplied by 10) 3 )
[0106]
[0107] Ablation studies were conducted on the offset-attention embedding module, downsampling, and upsampling modules. The results are shown in Table 4, demonstrating that point cloud inpainting networks with offset-attention embedding, downsampling, and upsampling modules achieve better point cloud inpainting integrity. Notably, the point cloud inpainting network can be tested after removing the offset-attention embedding module because the reconstruction network includes pooling and repetition layers, regardless of the order of the point cloud. This also indicates that convolutional layers and transposed convolutional layers with different kernel sizes can use max pooling operations to extract and represent point features.
[0108] Table 4 Ablation studies of the offset attention embedding module, downsampling, and upsampling modules (2048 points)
[0109]
[0110] Secondly, experiments were designed to compare the effects of different convolutional kernel sizes. More specifically, the kernel size of the convolutional layers in the point cloud inpainting network was changed to 1, and the network was trained again. The results are shown in Table 5. The EMD loss of the convolutional layer with a kernel size of 3 was lower than that of the convolutional layer with a kernel size of 1. This indicates that the convolutional layer with a kernel size of 3 can extract features from surrounding points.
[0111] Table 5 Ablation studies with different kernel sizes
[0112]
[0113] Since the MVP dataset is composed of synthetic scans from 3D models, further evaluation uses the KITTI dataset, which is collected from the real world. Because the KITTI dataset does not contain repaired point clouds, the results can only be evaluated qualitatively, as shown in Figure 7. It can be seen that PCN, TopNet, and Wang et al. can complete the car structure regardless of the number of points. However, these methods cannot complete the car's details, such as the wheels. Conversely, MSN and SparNet can achieve better repaired details, but the overall structure is poor. The point cloud repair network of this embodiment achieves optimal performance in terms of both structure and detail.
[0114] In the above text, the PMP method comes from X. Wen, P. Xiang, Z. Han, Y.-P. Cao, P. Wan, and W. Zheng, “Pmpnet: Point cloud completion by learning multi-step point moving paths,” IEEE Conference on Computer Vision and Pattern Recognition, 2022.5, 6;
[0115] The PCN method comes from W.Yuan, T.Khot, D.Held, C.Mertz, and M.Hebert, "Pcn: Pointcompletion network," in Proceedings of the International Conference on 3DVision, 2018, pp.728–737.2,5,6,7;
[0116] The TopNet method comes from LPTchapmi, V.Kosaraju, H.Rezatofighi, I.Reid, andS.Savarese, "Topnet: Structural point cloud decoder," in Proceedings of the IEEEConference on Computer Vision and Pattern Recognition, 2019, pp.383–392.2,5,6,7;
[0117] The Wang et al. method is from X. Wang, M. H. Ang Jr, and G. H. Lee, “Cascaded refinement network for point cloud completion,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020, pp. 790–799. 1, 2, 5, 6, 7;
[0118] The SnowflakeNet method is from P. Xiang, X. Wen, Y.-S. Liu, Y.-P. Cao, P. Wan, W. Zheng, and Z. Han, “Snowflakenet: Point cloud completion by snowflake point deconvolution with skip-transformer,” in Proceedings of the IEEE International Conference on Computer Vision, 2021, pp. 5499–5509. 2, 5, 6, 7;
[0119] The SpareNet method is from H. Xie, H. Yao, S. Zhou, J. Mao, S. Zhang, and W. Sun, “Grnet: Gridding residual network for dense point cloud completion,” in Proceedings of the European Conference on Computer Vision, 2020, pp. 365–381. 1, 2, 5, 6, 7;
[0120] The MSN method is from M. Liu, L. Sheng, S. Yang, J. Shao, and S.-M. Hu, “Morphing and sampling network for dense point cloud completion,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 07, 2020, pp. 11 596–11 603. 2, 5, 6, 7, 8;
[0121] The same or similar labels correspond to the same or similar parts;
[0122] The terms used to describe positional relationships in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this patent.
[0123] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.
Claims
1. A method for point cloud restoration with a large receptive field, characterized in that, Includes the following steps: S1: Acquire point cloud data; S2: Construct a point cloud restoration network. After identifying the relative positions between points in the point cloud data, the point cloud restoration network puts the points that are close to each other together, uses a convolutional layer with a large receptive field to extract highly abstract point features of the point cloud data, then extracts the global features of the point cloud data, and obtains the restored point cloud based on the highly abstract point features and global features. S3: Input the point cloud data obtained in step S1 into the point cloud restoration network in step S2. The point cloud restoration network outputs the restored point cloud. The point cloud restoration network includes an offset-attention embedding module, a rearrangement module, pooling and repetition layers, convolutional layers with different kernel sizes, and transposed convolutional layers with different kernel sizes. The offset-attention embedding module identifies the importance of points in the point cloud data, and the rearrangement module groups points that are close to each other together. The convolutional layers with different kernel sizes extract highly abstract point features of different dimensions of the point cloud data. The pooling and repetition layers extract global features of the point cloud data based on the output of the convolutional layers with different kernel sizes. The transposed convolutional layers with different kernel sizes obtain the restored point cloud based on the highly abstract point features and global features of the point cloud data.
2. The point cloud restoration method with a large receptive field according to claim 1, characterized in that, The offset-attention embedding module specifically works as follows: the input to the offset-attention embedding module is the position of the point, and the query matrix, key matrix, and value matrix are calculated based on the position of the point. In the formula, 、 、 These represent the query matrix, key matrix, and value matrix, respectively. Indicates the position of the point. 、 、 These represent the corresponding weight matrices. To represent matrix multiplication, the parameters satisfy: Calculate the attention map AM based on the query matrix and key matrix: Calculate the attention feature AF based on the attention map AM and the value matrix: In the formula, express Operators, express Norm; subtracted from the attention feature AF The results are then fed into a linear + batch normalized + rectified linear unit layer to calculate the offset attention. As the output of the offset-attention embedding module: In the formula, 、 As weight, It is a linear + batch normalized + rectified linear unit layer.
3. The point cloud restoration method with a large receptive field according to claim 1, characterized in that, The rearrangement module specifically involves: calculating the spatial relative distance of all points, using a clustering algorithm to find the k nearest points for each point, recording the indices of the k nearest points, and arranging each point with its k nearest points in turn according to the indices until all points are rearranged.
4. The point cloud restoration method with a large receptive field according to claim 3, characterized in that, The clustering algorithm is K-means clustering, Mean-Shift clustering, or KNN.
5. The point cloud restoration method with a large receptive field according to claim 1, characterized in that, The pooling and pooling operations in the repeated layers are max pooling.
6. The point cloud restoration method with a large receptive field according to claim 5, characterized in that, The convolutional layers with different kernel sizes specifically include several convolutional layers with a kernel size of 1 and several convolutional layers with a kernel size of n, where n is greater than 1.
7. The point cloud restoration method with a large receptive field according to claim 6, characterized in that, The transposed convolutional layers with different kernel sizes specifically include several transposed convolutional layers with a kernel size of 1 and several convolutional layers with a kernel size of m, where m is greater than 1.
8. The point cloud restoration method with a large receptive field according to any one of claims 1 to 7, characterized in that, The point cloud repair network also includes an optimization network, which performs high-precision optimization and repair on the repaired point cloud.
9. The point cloud restoration method with a large receptive field according to claim 8, characterized in that, The optimized network includes several convolutional layers and upsampling and downsampling modules. The repaired point cloud is input to the upsampling and downsampling modules after passing through several convolutional layers. The output of the upsampling and downsampling modules is then passed through several convolutional layers to obtain a high-precision optimized and repaired complete point cloud. The upsampling and downsampling modules include two downsampling units and two upsampling units. The upsampling and downsampling modules pass through the two downsampling units and the two upsampling units in sequence and are finally output from the upsampling and downsampling modules. In the downsampling unit, the feature map first obtains key points through farthest point sampling, and then obtains features using two convolutional layers, pooling, and repetition layers. In the upsampling unit, repetition and shaping operations are performed to ensure that the input and output sizes of the upsampling and downsampling modules are equal.