A point cloud completion method based on local covariance optimization

Abstract

The present invention discloses a point cloud completion method based on local covariance optimization, including the following steps: S1, data set acquisition; S2, data preprocessing; S3, neural network model construction; S4, neural network model loss construction; S5, neural network model training and optimization; S6, save the model and model parameters, adopt the above technical scheme, take the incomplete point cloud as input, and can output a complete point cloud with a complete shape and more refined. In the feature encoding stage, covariance is used to analyze the disordered and complex topological relationship between local points, and different convolution kernels are used to extract local geometric structure information; considering the integrity and structural similarity of the shape, the characteristics of the missing structure are inferred by fusing multi-scale hierarchical features to obtain the global feature vector of the point cloud under the complete shape; in the decoding stage, not only the expansion of the number of point clouds can be achieved, but also the local geometric structure of the point cloud can be optimized, thereby generating a more refined complete point cloud.

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud completion, and in particular to a point cloud completion method based on local covariance optimization. Background Art

[0002] With the improvement of the availability of scanning equipment, people can obtain a large amount of high-precision point cloud data from the surface of 3D objects in a short time. As a collection of points in three-dimensional space, point cloud retains more details and rich semantics. It has gradually become a popular way to represent 3D models and is widely used in autonomous driving, robot control, 3D design, AR / VR and other fields. However, due to the complexity of the shape and the influence of the shooting environment, such as light intensity, occlusion, etc., the point cloud obtained by scanning is usually incomplete. The incomplete point cloud model not only fails to represent the actual shape of the object and affects recognition, but also affects subsequent point cloud processing tasks, such as point cloud classification, segmentation and point cloud matching. Therefore, completing the incomplete point cloud is a very meaningful task.

[0003] Point cloud completion usually has two goals. The first is to recover the complete shape from the incomplete point cloud, and the second is to recover the local shape details of the point cloud. Traditional completion methods can complete smaller missing parts through surface fitting and grid interpolation, but are often powerless for large missing parts. In recent years, with the development of deep learning technology and the emergence of large 3D model databases, learning-based completion methods have gradually been proposed and achieved satisfactory results. The early strategy was to convert the discretized point cloud into structured grids and voxels, and then use regularized 3DCNN to complete point cloud reconstruction and completion. However, the conversion is very time-consuming, and the converted data is prone to information redundancy and loss. Later, thanks to the proposed multi-layer perceptron MLP and symmetric functions with shared parameters in PoinetNet, deep neural networks can directly act on point cloud data. Subsequently, some point cloud completion methods based on FlodingNet and PCN have also been proposed one after another. They directly encode the incomplete point cloud to obtain a global feature vector, and then use the decoder to reconstruct the vector into a point cloud with a complete shape. However, since these methods only consider reconstructing point clouds from global feature vectors and ignore local structure and shape features, they can only restore the basic shape of the point cloud, but cannot complete the second task of point cloud completion and cannot restore the fine local structure. Summary of the invention

[0004] In order to solve the problem that the existing technology cannot restore local details, this paper proposes a point cloud completion method based on local covariance optimization. Based on covariance analysis and self-attention mechanism, a network structure with local enhancement and geometric optimization is designed, which can effectively extract local structural information and enrich local details.

[0005] In order to solve the above technical problems, the technical solution of the present invention is:

[0006] A point cloud completion method based on local covariance optimization includes the following steps:

[0007] S1. Dataset acquisition: obtain the public dataset ShapetNet and complete the construction of the point cloud dataset for 3D model training;

[0008] S2, data preprocessing, surface sampling of each 3D model in the data set to obtain a complete point cloud, then simulating the shape loss process to generate a corresponding incomplete point cloud for each complete point cloud, and standardizing all point cloud data;

[0009] S3. Construction of neural network model. Based on covariance analysis and self-attention mechanism, local feature embedding module and local refinement module are designed, and an automatic encoding network CovNet is constructed based on them.

[0010] S4. Construction of neural network model loss, definition of the two-stage loss function of the neural network, and calculation of the difference between the completed point cloud output by the network and the real complete point cloud at different levels of refinement;

[0011] S5. Training and optimization of neural network models, setting network parameters, and training the network end-to-end using an optimizer;

[0012] S6. Save the model and model parameters. When the training reaches the maximum number of iterations or the objective function tends to be stable, save the model and model parameters.

[0013] Preferably, the data set in step S1 contains 55 different objects, and in step S2, 16 common objects are selected from the 55 different objects in the data set.

[0014] As an optimal solution, in step S2, the 16 common objects selected are uniformly sampled with a specified number of points on the surface of the 3D model using the Poisson disk sampling method as a true complete point cloud.

[0015] Preferably, in step S2, a random hole method is used to randomly select a point in the complete point cloud, delete points within a radius r to simulate point cloud loss, and sample to obtain a specified number of incomplete point clouds.

[0016] Preferably, in step S2, all point cloud data are subjected to standardization processing, including centralization processing and normalization processing.

[0017] Preferably, in step S3, the automatic encoding network includes an encoder and a decoder, the encoder includes multiple cascaded local feature embedding modules and a multi-level feature fusion and reasoning operation module, and the decoder consists of two parts: generation and refinement.

[0018] Preferably, the encoder performs feature encoding on an incomplete point cloud with an input size of N×3. The encoder first generates a coarse point cloud using feature expansion, and then refines the local geometric structure based on covariance analysis and self-attention mechanism to generate a complete fine point cloud with a size of N×3.

[0019] Preferably, in step S4, the definition of the first stage loss function comes from the rough reconstruction stage of the encoder, and is obtained by calculating the difference between the rough point cloud and the real point cloud through the chamfer distance CD, which can be specifically expressed as:

[0020]

[0021] Where S1 and S2 represent the rough point cloud and the real point cloud respectively;

[0022] The definition of the second-stage loss function comes from the refinement stage of the encoder. It is obtained by calculating the difference between the completed point cloud output by the network and the real point cloud through the bulldozer distance EMD. It can be specifically expressed as:

[0023]

[0024] Where S1 and S2 represent the completed point cloud and the real point cloud respectively; φ represents the mapping function;

[0025] Combining the first-stage loss function and the second-stage loss function, the total loss is set to

[0026]

[0027] Among them, the parameters λ1 and λ2 represent the weights of the two adjustable losses respectively.

[0028] Preferably, step S5 includes continuously training the neural network model for 200 cycles, with a BatchSize of 32, and every 70 cycles, the learning rate is reduced to 30% of the original, the loss weight λ1 is fixed to 1, and the loss weight λ2 is initialized to 0.01, and is expanded by 10 times every 50 cycles during training until it reaches 1.

[0029] Preferably, the optimizer adopts Adam with a learning rate of 0.0005, the entire neural network model for completion is implemented by Pytorch, the activation function of the neural network model is the Relu function, the number of points in the coarse point cloud in the neural network model is 512, and the number of points in the input and output point clouds are both 2048.

[0030] The present invention has the following characteristics and beneficial effects:

[0031] By adopting the above technical solution, it takes an incomplete point cloud as input and can output a complete point cloud with a complete shape and more refined. In the feature encoding stage, covariance is used to analyze the disordered and complex topological relationship between local points, and different convolution kernels are used to extract local geometric structure information; considering the integrity and structural similarity of the shape, the present invention also infers the characteristics of the missing structure by fusing multi-scale hierarchical features to obtain the global feature vector of the point cloud under the complete shape; in the decoding stage, the present invention designs a refinement module by introducing covariance and local self-attention mechanism, which can not only achieve the expansion of the number of point clouds, but also optimize the local geometric structure of the point cloud, thereby generating a more refined complete point cloud. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0033] Figure 1 This is a point cloud completion flow chart of an embodiment of the present invention.

[0034] Figure 2 Schematic diagram of the structure of the neural network model in the embodiment of the present invention.

[0035] Figure 3 It is a local feature embedding module of the neural network model in the embodiment of the present invention.

[0036] Figure 4 It is a refinement module of the neural network model in the embodiment of the present invention.

[0037] Figure 5 This is the completion effect of the neural network for different numbers of input points in the embodiment of the present invention.

[0038] Figure 6 This is the effect of the neural network completing different input missing ratios in the embodiment of the present invention. DETAILED DESCRIPTION

[0039] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0040] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and the like are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", and the like may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0041] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood by specific circumstances.

[0042] Example 1

[0043] The present invention provides a point cloud completion method based on local covariance optimization, such as Figure 1 As shown, the following steps are included:

[0044] S1. Dataset acquisition: obtain the public dataset ShapetNet and complete the construction of the point cloud dataset for 3D model training, where the point cloud dataset contains 55 different objects.

[0045] S2, data preprocessing, surface sampling of each 3D model in the data set to obtain a complete point cloud, and then simulate the shape loss process to generate a corresponding incomplete point cloud for each complete point cloud, and standardize all point cloud data. The data set used for preprocessing is 16 common objects selected from 55 different objects in the data set, which greatly reduces the amount of calculation;

[0046] It can be understood that all point cloud data include complete point clouds and incomplete point clouds.

[0047] S3. Construction of neural network model. Based on covariance analysis and self-attention mechanism, local feature embedding module and local refinement module are designed, and an automatic encoding network CovNet is constructed based on them.

[0048] S4. Construction of neural network model loss, definition of the two-stage loss function of the neural network, and calculation of the difference between the completed point cloud output by the network and the real complete point cloud at different levels of refinement;

[0049] S5. Training and optimization of neural network models, setting network parameters, and training the network end-to-end using an optimizer;

[0050] S6. Maintain the model and model parameters. When the training reaches the maximum number of iterations or the objective function tends to be stable, save the model and model parameters.

[0051] In the above technical solution, it takes an incomplete point cloud as input and can output a complete point cloud with a complete shape and more refined. In the feature encoding stage, covariance is used to analyze the disordered and complex topological relationship between local points, and different convolution kernels are used to extract local geometric structure information; considering the integrity and structural similarity of the shape, this embodiment also infers the characteristics of the missing structure by fusing multi-scale hierarchical features to obtain the global feature vector of the point cloud under the complete shape; in the decoding stage, this embodiment designs a refinement module by introducing covariance and local self-attention mechanism, which can not only achieve the expansion of the number of point clouds, but also optimize the local geometric structure of the point cloud, thereby generating a more refined complete point cloud.

[0052] Specifically, in step S2, the selected 16 common objects are uniformly sampled on the surface of the 3D model using the Poisson disk sampling method to obtain a specified number of points as a true complete point cloud, and a random hole method is used to randomly select a point in the complete point cloud, delete points within a radius of r to simulate point cloud loss, and sample a specified number of incomplete point clouds. In addition, all point cloud data are standardized, including centering and normalization.

[0053] The centering process is to translate the center of the point cloud to the origin for centering process, and the normalization process is to scale it into a sphere with a radius of 1 for normalization process to obtain a standardized point cloud.

[0054] The present invention is further configured as follows: Figure 2As shown, in step S3, the automatic encoding network includes an encoder for hierarchical feature extraction and a decoder for generation and refinement, wherein the encoder is composed of multiple cascaded local feature embedding modules and multi-level feature fusion and reasoning operation modules, and is mainly responsible for feature encoding of an incomplete point cloud with an input size of N×3 to obtain a global feature vector describing the complete shape of the point cloud, and its size is 1024. The decoder consists of two parts, generation and refinement, and is responsible for decoding the global features and reconstructing a point cloud with a complete shape.

[0055] It can be understood that the encoder first generates a coarse point cloud using feature expansion, and then refines the local geometric structure based on covariance analysis and self-attention mechanism to generate a complete fine point cloud of size N×3.

[0056] Specifically, the encoder for hierarchical feature extraction takes a point cloud of size N×3 as input, and outputs a global feature vector of size 1024 through multiple stacked local feature embedding modules and multi-level feature fusion operations.

[0057] Among them, the local feature embedding module, such as Figure 3 As shown in Figure 1, it first determines each local center point by farthest point sampling (FPS), then uses ball query or K-nearest neighbor (KNN) to determine the local area, and then uses graph convolution to aggregate local point features p f , using covariance analysis to extract local shape features p cov Finally, at each local fusion point feature and shape feature, the enhanced local embedding feature is obtained. where φ is a multilayer perceptron (MLP), It is the concatenation operation of the feature dimension.

[0058] In detail, the extraction formula of local point features can be expressed as

[0059]

[0060] Among them, p f Represents point features, represents the features of the kth neighboring point of point p. h represents a multi-layer perceptron (MLP) with shared parameters, used to extract point features. g represents a symmetric function, such as MaxPooling. Local shape feature p cov The calculation formula is

[0061]

[0062] Among them, P local is a k×3 local point set used to calculate the 3×3 local covariance, and mlp is a 1×1 convolution used to extract shape features.

[0063] As mentioned above, each time a local feature embedding module is passed, the number of points in the point cloud will be reduced, and the local features will be abstracted to the center point. As the level increases, each KNN query will form a larger local, which also means that the encoder can extract a larger local structure and more easily find the missing area. To this end, the present invention predicts the missing feature information by fusing multi-scale local features, and adds it to the features of the existing structure to obtain a complete global feature vector, so that it can better describe the shape of the point cloud. The entire calculation process can be expressed as:

[0064]

[0065] in is the local feature of the point cloud after the i-th local embedding; P g is the global latent vector. θ is a multi-layer perceptron (MLP) with shared weights; g is the symmetric function MaxPooling.

[0066] Further, such as Figure 4 As shown in Figure 1, the decoder is responsible for reconstructing the global feature vector extracted from the encoder into a point cloud with a complete shape, which is mainly divided into two parts: generation and refinement:

[0067] In the generation phase, the encoder takes a global feature vector P of size 1024 g As input, it is then mapped to a vector of size 3M using a linear layer, and finally deformed to M×3, which represents a coarse point cloud containing only XYZ coordinates. The coarse point cloud can be regarded as a low-resolution point cloud, which has the complete shape of the point cloud. coarse Merge with the input point cloud X to preserve the existing structure in the incomplete point cloud. Considering that the density of the two point clouds may be different, the farthest point sampling (FPS) is used to downsample the merged point cloud to obtain a uniformly distributed point cloud.

[0068] In the refinement stage, the local geometry optimization is mainly performed on the merged uniform point cloud, so that the reconstructed point cloud has a more refined structure. To this end, the present invention designs a refinement module with local geometry optimization. The specific structure is shown in the attached figure. Figure 4 As shown. Given a merged point cloud and expansion rate r, first Split into r parts to generate sub-point clouds and spread it to Neighbor points of point cloud; then by combining the local point features of point cloud and shape features Predicting sub-point clouds Coordinate offset of Finally, the original point cloud Copy r times and update the coordinates of the sub-point cloud Among them, the local shape features can be extracted through operations similar to the local feature embedding module, which provides local structural information for the distribution of sub-points. The local point features are extracted through the local self-attention mechanism, which smoothes the local structure by fusing the features of adjacent points. The self-attention mechanism is used to As a query, As a key, calculate the attention score of each neighbor point and the center point Specifically, it can be expressed as:

[0069]

[0070] in, Represents the operation of calculating the difference of point features, such as cosine distance, element subtraction, etc. δ represents the multi-layer perceptron MLP. At this time, each point feature after the attention mechanism is optimized

[0071]

[0072] in Add the elements separately, and ⊙ represents element-by-element multiplication.

[0073] In a further configuration of the present invention, in step S4, the definition of the loss function in the first stage comes from the rough reconstruction stage of the encoder, and is obtained by calculating the difference between the rough point cloud and the real point cloud by the chamfer distance CD, that is, the CD distance calculates the average value of the distances from all points in one set to the nearest point of another set, which can be specifically expressed as:

[0074]

[0075] It can be understood that in the above technical solution, the proposed point cloud completion network belongs to supervised learning, and the loss of network training comes from the completion loss, which is calculated by the chamfer distance CD and the bulldozer distance EMD. S1 and S2 represent the rough point cloud and the real point cloud respectively.

[0076] Furthermore, the definition of the second-stage loss function comes from the refinement stage of the encoder, and is obtained by calculating the difference between the completed point cloud output by the network and the real point cloud through the bulldozing distance EMD, that is, the bulldozing distance EMD calculates the minimum cost required to transform one set into another set, requiring the two sets to have the same size. Its essence is to find a bijection that minimizes the average distance difference between the mappings, which can be specifically expressed as:

[0077]

[0078] Where S1 and S2 represent the completed point cloud and the real point cloud respectively; φ represents the mapping function.

[0079] Since the decoder uses a coarse-to-fine strategy to output a complete point cloud, and the number of points in the coarse point cloud is different from that in the real point cloud, the present invention uses the CD distance to calculate the completion loss of the coarse point cloud and the EMD distance to calculate the completion loss of the refined point cloud. Combining the first-stage loss function and the second-stage loss function, the total loss is set as

[0080]

[0081] Among them, the parameters λ1 and λ2 represent the weights of the two adjustable losses respectively; P coarse represents the coarse point cloud; P fine represents the completed point cloud predicted by the network; P gt Indicates the true complete point cloud corresponding to the input incomplete point cloud.

[0082] A further arrangement of the present invention is that step S5 includes continuously training the neural network model for 200 cycles, with a batch size (BatchSize) of 32, and every 70 cycles, the learning rate is reduced to 30% of the original, the loss weight λ1 is fixed to 1, and the loss weight λ2 is initialized to 0.01, and is expanded by 10 times every 50 cycles during training until it is 1.

[0083] Specifically, 16 common objects in the public dataset ShapeNet were used, including 14,473 point cloud models, of which 11,705 were in the training set and 2,768 were in the test set. The entire point cloud completion network was implemented in Pytorch, and all activation functions except the output layer and the residual layer were Relu. In the decoder, the number of points in the coarse point cloud was 512, and the number of points in the refined completion point cloud was 2,048. Adam with a learning rate of 0.0005 was used for optimization.

[0084] In order to demonstrate the effectiveness and superiority of the point cloud completion method based on local covariance optimization, this embodiment also conducted comparative analysis with several other representative completion methods, including FCAE, PCN, MSN, and TopNet. In order to ensure the fairness and consistency of the experimental comparison, this experiment used the same data set and data preprocessing method, and also used two metrics for evaluation: CD and F1 Scores. CD measures the similarity of point clouds by calculating the distance deviation between the predicted point cloud and the actual point cloud. F1 Scores measures the similarity of point clouds by calculating the number of overlapping points between two point clouds. The experimental comparison results are shown in the following table, and the values ​​shown in the table are [CD / F1 Scores]. The magnification ratio of CD is 10 4 .

[0085] FCAE <![CDATA[ TopNet ]]> PCN MSN <![CDATA[ CovNet ]]> Airplane 14.20 / 0.85 12.68 / 0.86 11.26 / 0.89 6.73 / 0.93 6.07 / 0.95 Bag 60.73 / 0.33 46.55 / 0.42 45.44 / 0.43 25.33 / 0.67 22.17 / 0.70 Cap 77.43 / 0.40 56.64 / 0.43 55.87 / 0.45 21.56 / 0.75 19.63 / 0.75 Car 35.52 / 0.47 33.55 / 0.49 31.46 / 0.53 19.11 / 0.69 18.25 / 0.71 Chair 27.57 / 0.63 21.50 / 0.70 22.62 / 0.68 12.70 / 0.82 11.24 / 0.85 Earphone 68.99 / 0.43 59.16 / 0.44 47.03 / 0.49 25.58 / 0.74 25.36 / 0.75 Guitar 6.99 / 0.94 5.11 / 0.97 5.35 / 0.96 13.51 / 0.94 2.49 / 0.99 Knife 8.89 / 0.91 6.76 / 0.94 7.26 / 0.94 2.95 / 0.98 2.90 / 0.98 Lamp 59.75 / 0.52 45.40 / 0.59 47.83 / 0.57 24.24 / 0.80 23.06 / 0.82 Laptop 17.96 / 0.71 15.12 / 0.75 14.35 / 0.79 8.63 / 0.86 9.68 / 0.86 Motorbike 30.32 / 0.54 29.91 / 0.55 25.09 / 0.61 14.22 / 0.77 13.86 / 0.81 Mug 51.30 / 0.30 47.43 / 0.33 43.29 / 0.33 19.03 / 0.67 24.57 / 0.58 Pistol 19.75 / 0.74 17.36 / 0.77 16.41 / 0.79 9.44 / 0.88 9.92 / 0.90 Rocket 13.41 / 0.83 10.78 / 0.87 9.95 / 0.89 4.72 / 0.95 3.92 / 0.97 Skateboard 19.63 / 0.78 14.81 / 0.83 15.41 / 0.84 7.70 / 0.92 8.43 / 0.92 Table 32.38 / 0.62 26.35 / 0.69 26.73 / 0.68 13.61 / 0.83 14.04 / 0.84 avg 34.05 / 0.63 28.07 / 0.66 26.58 / 0.68 14.31 / 0.83 13.47 / 0.84

[0086] Table 1 Comparison of point cloud completion results

[0087] As shown in Table 1, although the complexity of the objects is different, resulting in obvious differences in the results of the repair. However, in the evaluation of indicators in most categories, the completion method CovNet proposed in this embodiment still achieved the best results. Since the local geometric structure of the point cloud is extracted through local covariance, the missing features are inferred through multi-level feature fusion, and the local part is refined using the self-attention mechanism, the network can generate a complete point cloud with real and fine texture. In general, the method of the present invention is superior to other methods in terms of results and can better complete the completion task.

[0088] The present invention uses Examples 1-2 to intuitively demonstrate the completion results of the neural network model CovNet and the robustness of the method.

[0089] Embodiment 2:

[0090] The difference between this embodiment and embodiment 1 is that after the neural network model CovNet is trained, when the aircraft point cloud is selected with a missing ratio of 35%, the number of points in the incomplete point cloud is reduced from N=2048 to 256. Figure 5 As shown, the method proposed in the present invention still has excellent completion results.

[0091] Embodiment 3:

[0092] The difference between this embodiment and embodiment 1 is that after the training of the neural network model CovNet is completed, when the number of points in the incomplete point cloud of the aircraft point cloud is fixed (N=2048), the missing ratio is gradually increased from 0.25 to 0.75. Figure 6 As shown, the method proposed in the present invention still has a good completion result.

[0093] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions and variations of these embodiments including components are made without departing from the principles and spirit of the present invention, and still fall within the scope of protection of the present invention.

Claims

1. A point cloud completion method based on local covariance optimization, characterized in that: The following steps are involved: S1. Dataset acquisition: obtain the public dataset ShapetNet and complete the construction of the point cloud dataset for 3D model training; S2, data preprocessing, surface sampling of each 3D model in the data set to obtain a complete point cloud, then simulating the shape loss process to generate a corresponding incomplete point cloud for each complete point cloud, and standardizing all point cloud data; S3. Construction of neural network model. Based on covariance analysis and self-attention mechanism, local feature embedding module and local refinement module are designed, and an automatic encoding network CovNet is constructed based on them. The automatic encoding network includes an encoder for hierarchical feature extraction and a decoder for generating refinement, wherein the encoder includes a plurality of cascaded local feature embedding modules and a multi-level feature fusion and reasoning operation module, and the decoder consists of two parts: generation and refinement; The encoder performs feature encoding on an incomplete point cloud of input size N×3. The encoder first generates a coarse point cloud by feature expansion, and then refines the local geometric structure based on covariance analysis and self-attention mechanism to generate a complete fine point cloud of size N×3. S4. Construction of neural network model loss, definition of the two-stage loss function of the neural network, and calculation of the difference between the completed point cloud output by the network and the real complete point cloud at different levels of refinement; The definition of the first-stage loss function in the two-stage loss function comes from the rough reconstruction stage of the encoder, which is obtained by calculating the difference between the rough point cloud and the real point cloud through the chamfer distance CD, which can be specifically expressed as: Where S1 and S2 represent the rough point cloud and the real point cloud respectively; The definition of the second-stage loss function in the two-stage loss function comes from the refinement stage of the encoder, and is obtained by calculating the difference between the completed point cloud output by the bulldozer distance EMD network and the real point cloud. It can be specifically expressed as: Where S1 and S2 represent the completed point cloud and the real point cloud respectively; φ represents the mapping function; Combining the first-stage loss function and the second-stage loss function, the total loss is set to Among them, the parameters λ1 and λ2 represent the weights of the two adjustable losses respectively; S5. Training and optimization of neural network models, setting network parameters, and training the network end-to-end using an optimizer; S6. Save the model and model parameters. When the training reaches the maximum number of iterations or the objective function tends to be stable, save the model and model parameters.

2. The point cloud completion method based on local covariance optimization according to claim 1, characterized in that: The data set in step S1 contains 55 different objects. In step S2, 16 common objects are selected from the 55 different objects in the data set.

3. The point cloud completion method based on local covariance optimization according to claim 2, characterized in that: In step S2, the selected 16 common objects are uniformly sampled with a specified number of points on the surface of the 3D model using the Poisson disk sampling method as a true complete point cloud.

4. The point cloud completion method based on local covariance optimization according to claim 2, characterized in that: In step S2, a random hole method is used to randomly select a point in the complete point cloud, delete points within a radius r to simulate point cloud loss, and sample a specified number of incomplete point clouds.

5. The point cloud completion method based on local covariance optimization according to claim 1, characterized in that: In the step S2, all point cloud data are standardized, including centralization and normalization.

6. The point cloud completion method based on local covariance optimization according to claim 1, characterized in that: In the step S5, the neural network model is continuously trained for 200 cycles, the BatchSize is 32, the learning rate is reduced to 30% of the original every 70 cycles, the loss weight λ1 is fixed to 1, and the loss weight λ2 is initialized to 0.01, and is expanded by 10 times every 50 cycles during training until it is 1.

7. The point cloud completion method based on local covariance optimization according to claim 6, characterized in that: The optimizer uses Adam with a learning rate of 0.0005. The entire neural network model for completion is implemented by Pytorch. The activation function of the neural network model is the Relu function. The number of points in the coarse point cloud in the neural network model is 512, while the number of points in the input and output point clouds are both 2048.

Images