A feature extraction and point cloud generation method for point cloud completion

Abstract

The application relates to the technical field of networks, in particular to a feature extraction and point cloud generation method for point cloud completion. The steps are as follows: a model is formed by an encoder-decoder structure, and the model comprises a feature extractor F, a missing point cloud generator G and a missing point cloud discriminator D. The feature extraction and point cloud generation method for point cloud completion provided by the application effectively enhances the extraction capability of the model for local and global features of point clouds by combining an offset attention feature extraction module; the extracted features are fused and connected from different scales and different dimensions, the representation capability of point cloud data is improved, and the features of interest are more abundant; and the missing point cloud generator module combining a seed generation module and a hierarchical feature pyramid can guarantee that the overall architecture of the predicted missing point cloud is complete and clear.

Description

Technical Field

[0001] This invention relates to the field of network technology, and in particular to a method for feature extraction and point cloud generation for point cloud completion. Background Technology

[0002] 3D representations contain three-dimensional spatial information about objects and their surrounding environment, enabling a more realistic representation of target objects. Point clouds are a type of 3D representation, characterized by non-uniformity, unstructured nature, and disorder. During point cloud data acquisition, issues such as partial occlusion of the target object, relative motion, and multiple reflections often arise, resulting in incomplete raw point clouds that differ from the real object. Because point cloud data lacks topological relationships and deep engineering semantic information between points, it requires denoising, smoothing, registration, and fusion processing. These operations can also lead to missing data and topological errors, affecting the quality of the point cloud and the processing performance of subsequent tasks. Therefore, to improve the performance of downstream point cloud tasks, point cloud completion operations are necessary to address these issues.

[0003] Point cloud completion tasks are primarily achieved through traditional methods and deep learning methods. Traditional methods are simple and efficient, but they require the input point cloud to be almost complete or exhibit significant symmetry. When the missing area of ​​an object is large or its shape is complex and lacks symmetry, these methods become ineffective and are difficult to process in real-time or port to different applications. Deep learning-based point cloud completion algorithms, on the other hand, can automatically learn the features of point cloud data, resulting in better completion performance and are widely used in point cloud completion tasks. Using deep learning technology, end-to-end automatic point cloud completion can be achieved by training neural networks.

[0004] PointNet was the first application of deep learning technology in point cloud processing. This model not only enabled direct processing of point clouds, but also solved the problem of point arrangement invariance, providing a foundation for improvement of subsequent algorithms.

[0005] PointNet++ is implemented by introducing a local feature extraction module on the basis of PointNet, which improves the ability to extract global and local features of point clouds.

[0006] PF-Net is a novel network that uses missing point clouds to achieve cloud completion. This network only takes the missing point cloud as input, predicts the incomplete point cloud through the network, aggregates features using multiple scales, and introduces a GAN network to distinguish between real and fake generated point clouds, thereby improving the accuracy and efficiency of automatic point cloud completion.

[0007] PF-Net provides a new approach to point cloud completion tasks, but it also exposes some shortcomings, such as the limited feature extraction capability of the completion network and the possibility of incomplete or distorted predicted point cloud frameworks.

[0008] SnowflakeNet is a point cloud completion network proposed by Zhou et al., which simulates the generation process of incomplete point clouds as the growth process of snowflakes, greatly improving the performance of point cloud completion.

[0009] SnowflakeNet proposes a snowflake-point deconvolution module that gradually increases the number of points and introduces a module that combines self-attention and skip connections to learn the splitting patterns in the snowflake-point deconvolution module.

[0010] The SnowflakeNet algorithm can generate the approximate outline of point clouds and complete them by simulating snowflakes, achieving good completion results. However, it suffers from problems such as distortion of the generated point cloud's detailed shape and increases computational complexity. Therefore, improving the network's feature extraction capabilities and addressing issues like point cloud detail distortion are pressing problems that current point cloud completion algorithms need to solve.

[0011] To address this issue, we designed a feature extraction and point cloud generation method for point cloud completion, providing an alternative technical solution. Summary of the Invention

[0012] Therefore, it is necessary to provide a feature extraction and point cloud generation method for point cloud completion to address the aforementioned technical problems.

[0013] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0014] A method for feature extraction and point cloud generation for point cloud completion, characterized by the following steps:

[0015] The model consists of an encoder-decoder structure, including a feature extractor F, a missing point cloud generator G, and a missing point cloud discriminator D.

[0016] By introducing offset attention into the feature extractor F to obtain more robust relative coordinates, a multi-scale feature vector V is obtained.

[0017] The decoder reconstructs incomplete point cloud blocks in coordinate space using the missing point cloud generator G;

[0018] The encoder-decoder structure is used as the generator of the GAN network, and a missing point cloud discriminator D is added after the output point cloud.

[0019] The original complete point cloud is input into the missing point cloud discriminator D. The missing point cloud discriminator D performs a binary classification task, adjusting the generator to generate a more realistic point cloud by distinguishing whether the generated point cloud is the original point cloud.

[0020] As a preferred embodiment of the feature extraction and point cloud generation method for point cloud completion provided by the present invention, the step of introducing offset attention into the feature extractor F to obtain more robust relative coordinates and obtain a multi-scale feature vector V is as follows:

[0021] The corresponding scale is mapped to three separate combined latent feature vectors F′ using three independent feature extractors F. i and multiple F i Connecting them together forms a potential feature map M' of size 1920×3;

[0022] By concatenating features of the same dimension obtained from convolution at different scales, a multidimensional feature vector f is obtained. i "The new feature F formed by connection" i ", will multiple F i "Connecting them together forms a potential feature map M of size 1920×3";

[0023] Aggregate and connect two latent feature maps M' and M” with different scales and dimensions;

[0024] The final processed feature vector V is obtained through convolution operations.

[0025] As a preferred embodiment of the feature extraction and point cloud generation method for point cloud completion provided by the present invention, the latent feature maps M' and M" of different scales and dimensions obtained by convolution and offset attention are aggregated and connected as follows:

[0026] The first three channels are formed by splicing together the dimensions from largest to smallest;

[0027] The last three channels are formed by splicing together the dimensions from smallest to largest.

[0028] As a preferred embodiment of the feature extraction and point cloud generation method for point cloud completion provided by the present invention, the feature vector V is expressed as follows:

[0029] V = Conv(CON(P(OA(MLP(V)) in )))))

[0030] Among them, V in Indicates the input point cloud features; MLP represents V inPerform dimensionality increase operation; OA represents offset attention operation; P represents max pooling operation; CON represents concatenating features at different levels; and Conv represents convolution operation.

[0031] As a preferred embodiment of the feature extraction and point cloud generation method for point cloud completion provided by the present invention, the step of reconstructing incomplete point cloud blocks in coordinate space using the missing point cloud generator G by the decoder is as follows:

[0032] First, the three sub-feature layers FC are obtained by passing the feature vector V to the fully connected layer. i Each feature layer is responsible for predicting point clouds at different resolutions.

[0033] The seed generator is introduced into the missing point cloud generator G. By utilizing the extracted 256-dimensional feature FC3, the seed generator first generates region features and captures existing and missing shapes through transposed convolution.

[0034] Each region feature is concatenated with the shape feature, and a coarse point cloud P of size M0×3 is generated through a multilayer perceptron and residual operations.

[0035] As a preferred embodiment of the feature extraction and point cloud generation method for point cloud completion provided by the present invention, a basic framework point cloud is generated through a seed generation structure.

[0036] The seed architecture points are input with 256-dimensional features, resulting in M0×3 points, which are then compared with the main center point Y. primary Combined, with the main center point Y primary Predicting from FC3 with dimension 256;

[0037] After deformation and convolution operations, FC3 is obtained with dimension [value missing]. The points are then added to the M0×1×3 points of the seed architecture after dimensionality upgrade to obtain the new Y. primary The size is M1×3.

[0038] As a preferred embodiment of the feature extraction and point cloud generation method for point cloud completion provided by the present invention, according to Y primary The production method of Y generates Y secondary and Y detail Y secondary and Y detail The sizes are M2×3 and M×3, Y detail This provides more detailed predictions for the final generated forecast.

[0039] As a preferred embodiment of the feature extraction and point cloud generation method for point cloud completion provided by the present invention, the model is trained using the benchmark ShapeNet-Part dataset, and a multi-scale completion loss L is adopted. completion and adversarial loss L oppose As an optimization loss.

[0040] It is clear without a doubt that the technical solution described above in this application can solve the technical problem that this application aims to address.

[0041] Meanwhile, through the above technical solutions, the present invention has at least the following beneficial effects:

[0042] This invention provides a feature extraction and point cloud generation method for point cloud completion. By combining a feature extraction module with offset attention, it effectively enhances the model's ability to extract local and global features of point clouds. The extracted features are fused and connected from different scales and dimensions to improve the representation ability of point cloud data and make the features of interest richer. The missing point cloud generator module, which combines a seed generation module with a hierarchical feature pyramid, can ensure the integrity and clarity of the overall architecture of the predicted missing point cloud. Attached Figure Description

[0043] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a schematic diagram of the feature extraction and point cloud generation method for point cloud completion according to the present invention;

[0045] Figure 2 This is a structural diagram of the feature extractor F of the present invention;

[0046] Figure 3 This is a structural diagram of the missing point cloud generator G of the present invention;

[0047] Figure 4 This is a structural diagram of the seed generator of the present invention;

[0048] Figure 5 This is a schematic diagram comparing the completion results of PF-Net and the algorithm of this invention on Shapenet-Part;

[0049] Figure 6 This is a schematic diagram comparing the completion results of PF-Net and the algorithm of this invention on ModelNet10;

[0050] Figure 7 This is a schematic diagram showing the comparison of the ablation experiment completion results of the present invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0052] To enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0053] It should be noted that, unless otherwise specified, the embodiments and features and technical solutions in the embodiments of the present invention can be combined with each other.

[0054] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0055] Reference Figures 1-7 A method for feature extraction and point cloud generation for point cloud completion.

[0056] 1 component

[0057] This invention proposes a feature extraction and point cloud generation method for point cloud completion. The overall model framework is as follows: Figure 1 As shown, the model consists of an encoder-decoder structure, specifically divided into three parts: a feature extractor F, a missing point cloud generator G, and a missing point cloud discriminator D.

[0058] The autoencoder learns high-dimensional implicit features from incomplete point clouds. Here, the feature extractor F introduces offset attention to obtain more robust relative coordinates, resulting in a multi-scale feature vector V. This vector is then fused with multi-dimensional features at the same scale and features at different scales of the same dimension. Since features at different scales and dimensions focus on different details, higher-quality features are obtained. The decoder reconstructs incomplete point cloud blocks in coordinate space using a missing point cloud generator G. Here, seed generation and a feature pyramid layered with fully connected layers and projection deformation are combined to obtain complete point clouds at different scales. The encoder-decoder structure is used as the generator in a GAN network. A discriminator is added after the output point cloud, and the original complete point cloud is also input into the discriminator. The discriminator performs a binary classification task, thereby adjusting the generator to produce more realistic point clouds by distinguishing whether the generated point cloud is the original point cloud.

[0059] 1.1 Feature Extractor F

[0060] In point clouds, the overall structure can be represented by taking a small number of points. This invention applies Iterative Farthest Point Sampling (IFPS) to obtain a set of skeleton points. The input of the multi-scale encoder is an N×3 unordered point cloud. IFPS is used to downsample the input point cloud to obtain two other scales with different resolutions.

[0061] In the feature extractor F, this invention proposes a method to combine offset attention features to capture low-level, mid-level, and high-level features rich in local information.

[0062] The feature extractor F structure is as follows: Figure 2 As shown, convolution is used to encode each point into multiple dimensions. The output of the last four layers of the MLP is used as the input to the offset attention, and then max pooling is performed to obtain the multidimensional feature vector f. i ′。 All f i The combinations are connected to form a combined latent feature vector F', which contains multi-level feature information.

[0063] Three independent feature extractors F are used to map these three scales to three separate combined latent feature vectors F. i Each vector represents a feature extracted from a point cloud at a specific resolution. Then all F... i Connecting them together forms a potential feature map M' with a size of 1920×3.

[0064] Since different scales and dimensions of features focus on different details of point clouds, this invention proposes an efficient way to utilize different features of point clouds. This method can fuse lower-level high-resolution features with higher-level semantic features, thereby avoiding information loss and enhancing the expressive power of the features. Features of the same dimension obtained by convolution at different scales are concatenated to obtain a multi-dimensional feature vector f. i "The new feature F formed by connection" i "。All F i "Connecting them forms a latent feature map M of size 1920×3." The two latent feature maps M' and M'" of different scales and dimensions are aggregated and connected. The first three channels are concatenated by decreasing dimensions, while the last three channels are concatenated by increasing dimensions. Finally, a convolution operation is performed to obtain the final processed feature vector V. Since features obtained through convolution and attention each have their advantages—convolutional features focus more on modeling local relationships, while attention features emphasize focusing on local details—combining the two yields higher-quality features. Furthermore, by concatenating higher-level features to lower-level features in reverse, the model can incorporate a wider range of contextual information into the lower-level features.

[0065] Traditional self-attention mechanisms primarily capture the relationships between points in a point cloud. Inspired by the PCT paper, offset attention utilizes offset vectors to capture the relative positional relationships between a point and its neighbors, thereby modeling and learning the local structure of the point cloud. Offset attention leverages the robustness of relative coordinates in mappings and the effectiveness of the Laplacian matrix in graph convolution. It calculates the offset difference between the self-attention feature and the input feature through element-wise subtraction, automatically adjusting the focus based on the shape of the point cloud, concentrating attention on important points. This invention improves the ability to extract key local features by introducing offset attention and feature fusion, maximizing the preservation of features at different scales and multi-level feature information. The final feature vector V is calculated as follows:

[0066] V = Conv(CON(P(OA(MLP(V)) in ))))), (1)

[0067] In the above formula, V in Indicates the input point cloud features; MLP represents V in Perform dimensionality increase operation; OA represents offset attention operation; P represents max pooling operation; CON represents concatenating features at different levels; and Conv represents convolution operation.

[0068] 1.2 Missing Point Cloud Generator G

[0069] The decoder takes the final feature vector V as input and aims to predict the missing point cloud region, which is a point cloud of size N×3. The point cloud shape predicted by the LGAN-AE algorithm using a fully connected decoder often results in the loss of local features.

[0070] Previous work combined with folding-based methods, where a folding-based decoder deforms a regular 2D mesh onto the 3D object surface of a point cloud, enabling better prediction of local features of the point cloud. However, if the point cloud surface is complex, this method is not good at handling convoluted surfaces and has difficulty recovering the original point cloud contour.

[0071] Inspired by FPN and PF-Net, this invention designs the decoder as a hierarchical structure based on feature points. It generates a basic framework point cloud through a seed generation structure and combines this with the two-layer progressive construction of a more detailed point cloud structure proposed in PF-Net, proposing a three-layer hierarchical structure for recovering missing point clouds. The structure proposed in this invention addresses the limitation of traditional algorithms in effectively recovering point cloud contour information. The missing point cloud generator structure is as follows: Figure 3 As shown.

[0072] First, the three sub-feature layers FC are obtained by passing the feature vector V to the fully connected layer. i Each feature layer is responsible for predicting point clouds at different resolutions.

[0073] The seed architecture points are input with 256-dimensional features, resulting in M0×3 points, which are then compared with the main center point Y. primary Combined, with the main center point Y primary Predict from a 256-dimensional FC3. Use "Expand" and "Add" operations to transform the dimensions of each point in the seed architecture, making it the center point, and then correlate it with Y. primary Combine to generate a new Y primary The size is M1×3.

[0074] The specific operation involves transforming and convolutionally converting FC3 to obtain a dimension of... The points are then added to the M0×1×3 points of the seed architecture after dimensionality increase, completing the transfer from low-level features to high-level features. FC2 will be used to predict the secondary center point Y. secondary The relative coordinates. FC1 will be used to predict the detailed point Y. detail The relative coordinates.

[0075] Y secondary and Y detail The generation of Y primary Similarly, its size is M2×3 and M×3. Y detail Y secondary Y primary The points are matched with the ground truth and the sampled point cloud. In this multi-scale generative architecture, high-level features influence the representation of low-level features, while low-resolution feature points can propagate local geometric information to high resolution to aid prediction. The generative process in point cloud completion aims to generate a new set of points, preserving data fidelity by retaining local geometry, or inferring missing parts based on existing shape and structure.

[0076] Inspired by SnowflakeNet, this invention proposes a seed generator that combines transposed convolution, MLP, and residuals. It filters features obtained from a feature extractor to obtain domain features and local information containing a clear skeletal structure and different regions. Each seed covers a small region around that point, and the seed features contain semantic cues about that region. By providing rich information through seed features, the seed generator can recover blurred, missing details.

[0077] Seed generator structure as follows Figure 4As shown, using the extracted 256-dimensional feature FC3, the seed generator first generates region features, capturing existing and missing shapes through transposed convolution. In point clouds, transposed convolution can expand features and propagate information in the spatial dimension by learning an invertible transposed convolution kernel. This helps capture the contextual relationships and global structure in the point cloud, thereby improving the accuracy and continuity of point cloud completion. Each region feature is concatenated with the shape features, and a coarse point cloud P of size M0×3 is generated through multilayer perceptron (MLP) and residual operations. This structure helps convey contextual information, enabling the model to utilize global and local relationships in the point cloud, better capture the complex features and structure of the point cloud data, and improve the accuracy and frame recovery ability of point cloud completion.

[0078] 1.3 Missing Point Cloud Discriminator D

[0079] Through the continuous interaction between the generator and discriminator networks, GAN networks can generate higher-quality, more realistic point cloud data and capture the distribution characteristics of the data, ensuring that the completed point cloud is consistent with the original point cloud data as a whole. The discriminator attempts to distinguish between predicted and real missing regions. The discriminator uses a similar processing method to the feature extractor F to extract features from the generated and real point clouds respectively, and then uses a classification network with sequential MLP layers to obtain the predicted value through a binary classifier.

[0080] 2.4 Loss Function and Evaluation Metrics

[0081] definition For realistic point clouds, Let n be the predicted point cloud to be evaluated. ψ and n φ These represent the number of points ψ and φ, respectively. Chamfer distance has the advantages of being differentiable and computationally efficient; therefore, it is used as a quantitative evaluation metric in the experiment. Following the considerations in PF-Net, the chamfer distance consists of two parts: d CD (Pred→GT) / d CD (GT→Pred), the distance between ψ and φ is defined as:

[0082]

[0083] During training, a multi-scale completion loss L was used. completion and adversarial loss L oppose As an optimization loss, the completion loss attempts to compare the predicted and the actual missing point cloud Y. gt Matching. Adversarial loss attempts to make the predictions closer to reality by optimizing the generator. Similar to PF-Net, the total training loss is L. completion and L oppose The weighted sum of λ, where λcompletion and λ oppose These are the weights of the completion loss and the adversarial loss, satisfying λ. completion +λ oppose =1. The specific definition is as follows:

[0084] L=λ completion L completion +λ oppose L oppose (3)

[0085] L completion Combined with different resolutions of the generator output: Y, Y1, Y2. As shown below, formula (4) is used to calculate the true missing point cloud Y. gt and the true missing point cloud of the sample The CD distance between it and its predicted value is defined as follows: Weighted by hyperparameter α:

[0086]

[0087] Adversarial loss L oppose Following the GAN model. Partial input... Mapping to the predicted missing region Then, the missing point cloud discriminator D attempts to distinguish the predicted missing regions. and the actual missing area The missing point cloud discriminator D is a classification network with a serial MLP layer.

[0088]

[0089] in It is a dataset and Size.

[0090] 2. Dataset and Network Training

[0091] To train the model, the benchmark ShapeNet-Part dataset is used, which is a subset of the ShapeNetCore model and provides part segmentation.

[0092] It contains 16 different models, with a total of 17,775 point cloud models. Real-world point clouds are sampled, and 512 points within a certain radius are removed from random viewpoints. ModelNet10 is a commonly used point cloud dataset containing 4,899 point cloud models from 10 object categories. To further verify the robustness and generalization of the algorithm, experiments were also conducted on ModelNet10. The experimental parameters were set identically to those on the ShapeNet-Part dataset.

[0093] The network was implemented using PyTorch and CUDA. All algorithms were optimized using the ADAM optimizer with an initial learning rate of 0.0001. For point cloud predictions at different scales, the learner size was set to 64, 128, and 512. On the ShapeNet-Part dataset, the network was trained on a single NVIDIA Tesla A100 GPU with a batch size of 30, and the optimization was stopped after 200 epochs. On the ModelNet10 dataset, the network was trained with a batch size of 8, and other experimental settings were the same as those for the ShapeNet-Part dataset.

[0094] 3. Analysis and Discussion

[0095] 3.1 Comparison of completion effects of different methods

[0096] 3.1.1 ShapeNet-Part Dataset

[0097] The algorithm of this invention is compared with several point cloud completion algorithms: LGAN-AE algorithm, 3D-Capsule algorithm, PFNet algorithm, MSTCN algorithm and CP-Net algorithm.

[0098] The evaluation metrics mentioned above are used to evaluate the performance of each algorithm, specifically including two metrics: d CD (Pred→GT) Error calculation measures the distance from each point in the prediction to the nearest point in the true value, and it measures the degree of difference between the prediction and the actual situation; d CD The (GT→Pred) error calculates the distance from each point in the true value to the nearest point in the prediction, representing the extent to which the predicted shape covers the true point cloud surface.

[0099] Table 1 shows the experimental results of different algorithms on ShapeNet-Part. All metrics are scaled up to a ratio of 1000, with smaller values ​​indicating better performance. The algorithm proposed in this invention performs well in 16 categories. CD (Pred→GT) and d CD The mean (GT→Pred) score is superior to other algorithms in all categories, specifically decreasing by approximately 1.55% and 19.29%. Furthermore, in six categories—airplane, bag, chair, guitar, pistol, and table—the algorithm of this invention shows significant improvements in both metrics, while in the remaining ten categories, only one metric is slightly lower than other algorithms. Table 1 shows that the algorithm of this invention can better predict the framework and local parts of missing point clouds. The complete point cloud is obtained by concatenating the input point cloud and the predicted missing point cloud. Because the algorithm of this invention only needs to predict the missing part of the point cloud through the network, it preserves the input point cloud from prediction distortion by the network.

[0100] Table 1. Performance comparison of different algorithms on Shapenet-Part

[0101]

[0102] Figure 5 The diagram shows point clouds reconstructed using the PF-Net algorithm and the method of this invention. The network of this invention produces a significantly well-reconstructed general point cloud while preserving the realistic details of the original point cloud. For example, in the hat point cloud, the PF-Net algorithm generates discrete point clouds in the brim area, while the algorithm of this invention reconstructs the straight brim well. In the cup point cloud, the PF-Net algorithm generates curved angles on the sides of the cup, while the algorithm of this invention reconstructs the rim of the cup well. In the pistol point cloud, the muzzle generated by the PF-Net algorithm is somewhat diffused, while the algorithm of this invention reconstructs the straight and regular muzzle well. In the table point cloud, the missing parts generated by the PF-Net algorithm are discrete and messy, while the algorithm of this invention reconstructs the structure of the table well. In summary, the illustrated results demonstrate the advantages of the algorithm of this invention in terms of completion performance.

[0103] The effectiveness of the offset attention fusion module and the seed generation module can be intuitively observed in the qualitative visualization results. The core of the seed generation module is to generate the overall contour of the predicted missing region, while the core of the offset attention fusion module is to extract higher-quality features and better recover the local features of the point cloud through fusion connections. This allows for a direct evaluation of the visualization results of PF-Net and the algorithm of this invention. The algorithm of this invention can reconstruct the overall contour of the missing region and capture relevant details. The algorithm of this invention successfully predicted the approximate contours of missing regions for all categories, especially for airplanes, hats, cups, pistols, and tables.

[0104] 3.1.2 ModelNet10 dataset

[0105] Table 2 shows the experimental results of different algorithms on ModelNet10. The algorithm proposed in this invention performs well in 10 categories. CD (Pred→GT) and d CD The mean (GT→Pred) score is superior to other algorithms in all categories, specifically decreasing by approximately 22.16% and 43.42%. Furthermore, the algorithm of this invention exhibits a significant advantage in both metrics across all categories. The results in Table 2 demonstrate that the algorithm of this invention has good generalization ability and achieves better performance with smaller datasets.

[0106] Table 2 compares the performance of PF-Net and the algorithm of this invention on ModelNet10.

[0107]

[0108]

[0109] Figure 6 Point clouds reconstructed using the PF-Net algorithm and the algorithm of this invention are shown. The network of this invention recovers the general outline of the missing point cloud and makes an effort to recover local features. For example, in the chair point cloud, the PF-Net algorithm generates an irregular and dense point cloud in the back of the chair, while the algorithm of this invention reconstructs the outline of the back of the chair very well. In the display point cloud, the PF-Net algorithm generates a sparse point cloud randomly in the upper part of the display, while the algorithm of this invention recovers the regular and straight structure of the display.

[0110] 3.2 Ablation Experiment

[0111] Tables 3 and 4 list the ablation experiment results of the algorithm of this invention, including all proposed modules: the offset attention fusion module and the seed generation module. The experimental results show a performance decline when one of the proposed modules is removed, which verifies the effectiveness of the proposed module.

[0112] To demonstrate the effectiveness of the offset attention fusion module proposed in this invention, the results of the point cloud completion algorithm are analyzed by removing the offset attention fusion module. As shown in Table 3, the algorithm proposed in this invention performs well in 16 categories of point cloud completion. CD (Pred→GT) and d CD The mean values ​​of (GT→Pred) are both better than the algorithm without this module, specifically decreasing by approximately 14.08% and 18.66%. The visualization results after removing the offset attention fusion module are shown below. Figure 7 As shown, removing the offset attention fusion module from the generated point clouds of cars, chandeliers, computers, and tables will cause the loss or distortion of local features, while the algorithm proposed in this invention can reconstruct the local features of the missing point clouds.

[0113] Table 3 Comparison results of ablation experiments

[0114]

[0115] To demonstrate the effectiveness of the seed generation module proposed in this invention, the results of the point cloud completion algorithm are analyzed by removing the seed generation module. As shown in Table 4, the algorithm proposed in this invention performs well in 16 categories of point cloud completion. CD (Pred→GT) and d CD The mean values ​​of (GT→Pred) are superior to those of algorithms that remove this module, specifically decreasing by approximately 12.74% and 4.23% respectively. The visualization results for removing the seed generation module are shown below. Figure 7 As shown. Removing the structure of the seed generation module to generate point clouds of cars, chandeliers, computers, and tables will result in distorted general outlines or generate discrete point clouds. However, the algorithm proposed in this invention can reconstruct the general outline of the missing point cloud very well.

[0116] Table 4 Comparison Results of Ablation Experiments

[0117]

[0118]

[0119] 4. Conclusion

[0120] This invention proposes a novel network architecture for end-to-end point cloud completion tasks, which takes a partial point cloud as input to predict its missing parts. The model can smoothly and efficiently generate point clouds by utilizing local information and the overall structure of the target object. The model outperforms existing baseline algorithms on the ShapeNet-Part and ModelNet10 datasets. Furthermore, experiments demonstrate the effectiveness of the proposed modules. The offset attention fusion module captures feature information in a more meaningful way and with less redundant information. The missing point cloud generator, combined with seed generation and feature pyramid, results in a point cloud with a complete overall architecture and better prediction of local features in the missing region.

[0121] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for feature extraction and point cloud generation for point cloud completion, characterized in that, The steps are as follows: The model consists of an encoder-decoder structure, including a feature extractor F, a missing point cloud generator G, and a missing point cloud discriminator D. By introducing offset attention into the feature extractor F to obtain more robust relative coordinates, a multi-scale feature vector V is obtained. The decoder reconstructs incomplete point cloud blocks in coordinate space using the missing point cloud generator G; The encoder-decoder structure is used as the generator of the GAN network, and a missing point cloud discriminator D is added after the output point cloud. The original complete point cloud is input into the missing point cloud discriminator D. The missing point cloud discriminator D performs a binary classification task. By distinguishing whether the generated point cloud is the original point cloud, the generator is adjusted to generate a more realistic point cloud. The steps to introduce offset attention into the feature extractor F to obtain more robust relative coordinates and thus a multi-scale feature vector V are as follows: The corresponding scale is mapped to three separate combined latent feature vectors through three independent feature extractors F. and multiple Connect them to form a size of Latent feature map ; By concatenating features of the same dimension obtained from convolution at different scales, a multidimensional feature vector is obtained. New features formed by connection , will multiple Connect them to form a size of Latent feature map ; Two latent feature maps of different scales and dimensions and Perform aggregation joins; The final processed feature vector is obtained through convolution operations. ; The feature vector The expression is as follows: in, Represents the input point cloud features; express Perform a dimensional upgrade operation; This indicates a shift in attention operation; This represents the max pooling operation; This indicates an operation that connects features at different levels. Indicates the convolution operation; The steps for reconstructing incomplete point cloud blocks in coordinate space using the missing point cloud generator G are as follows: First, by using the feature vector Passing to the fully connected layer to obtain three sub-feature layers Each feature layer is responsible for predicting point clouds at different resolutions. A seed generator is introduced into the missing point cloud generator G, utilizing the extracted 256-dimensional features. The seed generator first generates region features, and then captures existing and missing shapes through transposed convolution; Each region feature is concatenated with shape features, and then a multilayer perceptron and residual operation are used to generate a product of size [size missing]. coarse dot clouds ; Generate a basic framework point cloud by generating a structure from seeds; The seed architecture points are taken as input with 256-dimensional features, and the results are obtained. Each point is compared with the main center point. To combine, the main central point From dimension 256 predict; Will After deformation and convolution operations, the dimension is obtained as follows: Each point, and then the seed architecture after dimensional upgrading. Adding the points together, we get a new... Size is .

2. The method for feature extraction and point cloud generation for point cloud completion according to claim 1, characterized in that, Latent feature maps of different scales and dimensions obtained by convolution and offset attention and To perform an aggregation join, follow these steps: The first three channels are formed by splicing together the dimensions from largest to smallest; The last three channels are formed by splicing together the dimensions from smallest to largest.

3. The method for feature extraction and point cloud generation for point cloud completion according to claim 1, characterized in that, according to The production method generates and , and The size is and , This provides more detailed predictions for the final generated forecast.

4. The method for feature extraction and point cloud generation for point cloud completion according to claim 1, characterized in that, The model was trained using the benchmark ShapeNet-Part dataset, employing a multi-scale completion loss. and adversarial losses As an optimization loss.

Images