Three-dimensional point cloud completion method and system based on multi-scale structured knowledge distillation

Abstract

The application discloses a three-dimensional point cloud completion method and system based on multi-scale structured knowledge distillation. The method comprises the following steps: constructing an incomplete three-dimensional point cloud dataset; introducing a multi-scale hierarchical knowledge self-distillation encoding module and a reconstruction decoding module to construct a three-dimensional point cloud data completion network model; and performing three-dimensional point cloud completion on the incomplete three-dimensional point cloud dataset based on the three-dimensional point cloud data completion network model to obtain a complete three-dimensional point cloud dataset. According to the embodiment of the application, the differences between different local features can be learned by maximizing the mutual information therebetween, the distribution of the deepest layer module is specified as supervision on the previous module, and the completeness of data completion is improved. The application can be widely applied to the technical field of point cloud data completion.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of point cloud data completion, in particular to a three-dimensional point cloud completion method and system based on multi-scale structured knowledge distillation. BACKGROUND

[0002] At present, the research on point cloud data is mainly divided into two categories: reconstruction and understanding. PointNet and PointNet++ establish the basic model of point cloud understanding, while PCN and PU-Net formulate the process of point cloud reconstruction (including completion and upsampling). Their common feature is that before the subsequent understanding or reconstruction module, the point cloud data, whether partial or complete, should always be encoded into implicit encoding with high-order information. And these basic models mainly use simple aggregation operations for encoding, because this is a prerequisite for encoding unordered point clouds. Aggregation usually needs to be implemented with a symmetric function, such as max pooling, summation, and global set abstraction. Therefore, due to the existence of information loss, these methods inevitably cannot accurately understand or reconstruct the shape. At the same time, simply encoding the point cloud, especially for incomplete point clouds, may result in the loss of high-order shape details. In real-world scanning, the quality of point clouds varies greatly, which may be sparse, discontinuous, or incomplete. This greatly increases the need to encode a consistent and robust representation from point clouds with missing areas.

[0003] In summary, the technical problems existing in the related art need to be improved. SUMMARY

[0004] The main purpose of the embodiments of the present application is to propose a three-dimensional point cloud completion method and system based on multi-scale structured knowledge distillation, which can learn the differences between different local features by maximizing the mutual information between them, and specify the distribution of the deepest layer module as supervision to the previous module, thereby improving the completeness of data completion.

[0005] To achieve the above-mentioned purpose, one aspect of the embodiments of the present application proposes a three-dimensional point cloud completion method based on multi-scale structured knowledge distillation, which comprises:

[0006] Constructing a non-complete three-dimensional point cloud dataset;

[0007] Introducing a multi-scale hierarchical knowledge self-distillation encoding module and a reconstruction decoding module to construct a three-dimensional point cloud data completion network model;

[0008] Based on the three-dimensional point cloud data completion network model, the non-complete three-dimensional point cloud dataset is completed to obtain a complete three-dimensional point cloud dataset.

[0009] In some embodiments, the three-dimensional point cloud data completion network model comprises a multi-scale hierarchical knowledge self-distillation encoding module, a multi-layer perceptron module, and a reconstruction decoding module, which are sequentially connected, wherein the expression of the loss function of the three-dimensional point cloud data completion network model is specifically as follows:

[0010] ;

[0011] In the above formula, represents the loss function of the three-dimensional point cloud data completion network model, represents an empirical coefficient, represents a reconstruction loss function, represents a self-distillation loss function, represents a corresponding completion result of the target set, represents a complete ground truth point cloud, represents a predicted point cloud output by the reconstruction decoding module, represents a corresponding resolution ground truth output by the reconstruction decoding module.

[0012] In some embodiments, the multi-scale hierarchical knowledge self-distillation encoding module comprises a plurality of self-distillation modules, each of which comprises a k-nearest neighbor layer, a farthest point sampling layer, a multi-layer perceptron layer, and a max-pooling layer, and the reconstruction decoding module comprises a PointNet layer, a local attention mechanism layer, a one-dimensional deconvolution layer, a first multi-layer perceptron layer, a second multi-layer perceptron layer, and an up-sampling layer.

[0013] In some embodiments, the three-dimensional point cloud data completion network model is used to complete the non-complete three-dimensional point cloud data set to obtain a complete three-dimensional point cloud data set, comprising:

[0014] inputting the non-complete three-dimensional point cloud data set into the three-dimensional point cloud data completion network model;

[0015] based on the multi-scale hierarchical knowledge self-distillation encoding module of the three-dimensional point cloud data completion network model, performing self-distillation loss encoding on the non-complete three-dimensional point cloud data set to obtain sparse point cloud data and neighborhood features;

[0016] based on the multi-layer perceptron module of the three-dimensional point cloud data completion network model, performing data dimension reduction processing on the sparse point cloud data and the neighborhood features to obtain a rough point cloud data set;

[0017] based on the reconstruction decoding module of the three-dimensional point cloud data completion network model, performing data reconstruction on the rough point cloud data set to obtain the complete three-dimensional point cloud data set.

[0018] In some embodiments, the multi-scale hierarchical knowledge self-distillation encoding module based on the three-dimensional point cloud data completion network model encodes the non-complete three-dimensional point cloud data set by a self-distillation loss to obtain sparse point cloud data and neighborhood features, including:

[0019] The non-complete three-dimensional point cloud data set is input into the multi-scale hierarchical knowledge self-distillation encoding module;

[0020] The self-distillation module based on the multi-scale hierarchical knowledge self-distillation encoding module extracts features from the non-complete three-dimensional point cloud data set to obtain a plurality of preliminary neighborhood features;

[0021] The preliminary neighborhood features are calculated by a softmax activation function to obtain probability distributions corresponding to the preliminary neighborhood features;

[0022] The probability distribution corresponding to the last preliminary neighborhood feature is taken as a supervision signal, and knowledge backflow is realized by KL divergence to output the sparse point cloud data and the neighborhood features.

[0023] In some embodiments, the self-distillation module based on the multi-scale hierarchical knowledge self-distillation encoding module extracts features from the non-complete three-dimensional point cloud data set to obtain a plurality of preliminary neighborhood features, including:

[0024] The non-complete three-dimensional point cloud data set is input into the self-distillation module of the multi-scale hierarchical knowledge self-distillation encoding module;

[0025] The k-nearest neighbor layer based on the self-distillation module performs k-nearest neighbor search on the non-complete three-dimensional point cloud data set to construct local neighbor features of point cloud data;

[0026] The farthest point sampling layer based on the self-distillation module samples center points from the local neighbor features of point cloud data to obtain center points of local neighbor features;

[0027] The multi-layer perception layer based on the self-distillation module maps features of the center points of local neighbor features to obtain mapped local neighbor features;

[0028] The max-pooling layer based on the self-distillation module performs aggregation operation processing on the mapped local neighbor features to obtain a plurality of preliminary neighborhood features.

[0029] In some embodiments, the loss function of the self-distillation module is specifically as follows:

[0030] ;

[0031] In the above formula, represents the loss function of the self-distillation module, denotes the KL divergence, denotes the predicted label distribution, denotes the real label distribution, denotes the distribution of the last layer, .

[0032] In some embodiments, the reconstruction decoding module based on the three-dimensional point cloud data completion network model, the data reconstruction is carried out on the rough point cloud data set, and the complete three-dimensional point cloud data set is obtained, comprising:

[0033] The rough point cloud data set is input into the reconstruction decoding module of the three-dimensional point cloud data completion network model;

[0034] Based on the PointNet layer of the reconstruction decoding module, the point cloud registration processing is carried out on the rough point cloud data set, and the current point cloud neighborhood feature is obtained;

[0035] Based on the local attention mechanism layer of the reconstruction decoding module, the local feature extraction processing is carried out on the current point cloud neighborhood feature, and the local feature is obtained;

[0036] The global feature of the rough point cloud data set is obtained, and the point cloud correlation feature is obtained by combining the local feature;

[0037] Based on the one-dimensional deconvolution layer of the reconstruction decoding module, the point cloud correlation feature is copied upwards to obtain the copied point cloud correlation feature;

[0038] Based on the first and second multi-layer perceptron layers of the reconstruction decoding module, the copied point cloud correlation feature is mapped to obtain the point-by-point displacement of the point cloud feature;

[0039] Based on the up-sampling layer of the reconstruction decoding module, the rough point cloud data set is up-sampled to obtain the up-sampled rough point cloud data set;

[0040] The point-by-point displacement of the point cloud feature and the up-sampled rough point cloud data set are added to obtain the complete three-dimensional point cloud data set.

[0041] In some embodiments, the loss function of the reconstruction decoding module is specifically as follows:

[0042] ;

[0043] In the above formula, denotes the loss function of the reconstruction decoding module, denotes the incomplete three-dimensional point cloud data set, denotes the complete three-dimensional point cloud data set, represents a point cloud data point in a non-complete three-dimensional point cloud data set, represents a point cloud data point in a complete three-dimensional point cloud data set, represents an L2 norm.

[0044] To achieve the above object, another aspect of the embodiment of the present application provides a three-dimensional point cloud completion system based on multi-scale structured knowledge distillation, which comprises:

[0045] A first module is configured to construct a non-complete three-dimensional point cloud data set;

[0046] A second module is configured to introduce a multi-scale hierarchical knowledge self-distillation encoding module and a reconstruction decoding module to construct a three-dimensional point cloud data completion network model;

[0047] A third module is configured to complete the three-dimensional point cloud based on the three-dimensional point cloud data completion network model to obtain a complete three-dimensional point cloud data set.

[0048] The embodiment of the present application at least has the following beneficial effects: the present application provides a three-dimensional point cloud completion method and system based on multi-scale structured knowledge distillation. The scheme constructs a non-complete three-dimensional point cloud data set, further introduces a multi-scale hierarchical knowledge self-distillation encoding module and a reconstruction decoding module to construct a three-dimensional point cloud data completion network model, proposes a hierarchical structure, focuses on more effective coding of point clouds, uses multi-scale hierarchical knowledge distillation to learn the difference between different local features by maximizing their mutual information, specifies the distribution of the deepest layer module as supervision of the previous module, and the plug-and-play module can be integrated into most point cloud processing networks as an encoder, recovers the point cloud from coarse to fine through the reconstruction decoding module, and finally completes the three-dimensional point cloud based on the three-dimensional point cloud data completion network model to improve the completeness of data completion. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 is a flowchart of the three-dimensional point cloud completion method based on multi-scale structured knowledge distillation provided by the embodiment of the present application;

[0050] Figure 2 is a structural schematic diagram of the three-dimensional point cloud completion system based on multi-scale structured knowledge distillation provided by the embodiment of the present application;

[0051] Figure 3 is a structural schematic diagram of the three-dimensional point cloud data completion network model provided by the embodiment of the present application;

[0052] Figure 4 is a structural schematic diagram of the multi-scale hierarchical knowledge self-distillation encoding module provided by the embodiment of the present application;

[0053] Figure 5 FIG. 1 is a structural schematic diagram of a reconstruction decoding module provided by an embodiment of the present application;

[0054] Figure 6 FIG. 6 is a schematic diagram of visualization of the completion effect of different models on different objects on the data set PCN. DETAILED DESCRIPTION

[0055] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not intended to limit the present application. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementation described in the following exemplary embodiments does not represent all the implementations consistent with the embodiments of the present application, but is only an example of the system and method consistent with some aspects of the embodiments of the present application as described in the appended claims.

[0056] It can be understood that the terms "first", "second", and the like used in the present application can be used herein to describe various concepts, but unless specifically stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "when" or "when" or "in response to determining".

[0057] The terms "at least one", "multiple", "each", "any" and the like used in the present application include one, two or more than two, multiple includes two or more than two, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as understood by a person skilled in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0059] Before the embodiments of the present application are described in detail, first, some of the terms and terms involved in the embodiments of the present application are explained, point cloud as a kind of 3D data form with accurate geographic coordinates and rich semantic information, plays a crucial role in many fields. Point cloud application range is wide, including but not limited to virtual reality, automatic driving system, medical modeling and building maintenance and many other fields. In practical application, point cloud data needs to be processed in real time, or needs to be stored in large storage device for subsequent analysis. When the cost of data processing and storage exceeds the transmission cost, in order to the economic benefit under the actual application, the reconstruction technology of point cloud becomes particularly important. Especially in those time-limited or cross-distance scenarios, the reconstruction of point cloud becomes the most feasible method.

[0060] In the related art, some deficiencies exist, such as the existing completion methods such as SnowflakeNet and SeedFormer, which focus on the decoding process of the missing part, i.e. restoration and upsampling, and ignore the function of the encoding process in understanding the intrinsic semantic information of the model.

[0061] Therefore, in the embodiments of the present application, a three-dimensional point cloud completion method based on multi-scale structured knowledge distillation is provided, which adds a plug-and-play hierarchical self-distillation (HSD) framework to maximize the mutual information from multiple coverage scales, and transfers the most unique implicit features in the deepest level of hyperspace to other scales. The last layer classifier in the series of different scales is designated as the teacher network (the deepest layer) to supervise the classifiers in the previous network layers, so as to transfer important knowledge to the hyperspace under different perception scales.

[0062] Reference Figure 1 , Figure 1 A flowchart of a three-dimensional point cloud completion method based on multi-scale structured knowledge distillation provided by the embodiments of the present application is shown in Figure 1 The method comprises the following steps:

[0063] S100, constructing an incomplete three-dimensional point cloud data set;

[0064] In some specific embodiments, the embodiments of the present application can use any three-dimensional point cloud data set. In this embodiment, a common PCN completion data set is used, which is taken from ShapeNet, and the pairing of incomplete input and complete output has been completed. By default, each object uses 2048 points of input and 16384 points of output.

[0065] S200, introducing multi-scale hierarchical knowledge self-distillation encoding module and reconstruction decoding module, constructing three-dimensional point cloud data completion network model;

[0066] It should be noted that in some embodiments, such asFigure 3 As shown, the three-dimensional point cloud data completion network model comprises a multi-scale hierarchical knowledge self-distillation encoding module, a multi-layer perceptron module and a reconstruction decoding module, the multi-scale hierarchical knowledge self-distillation encoding module, the multi-layer perceptron module and the reconstruction decoding module are sequentially connected, the multi-scale hierarchical knowledge self-distillation encoding module comprises a plurality of self-distillation modules, each of the plurality of self-distillation modules comprises a k-nearest neighbor layer, a farthest point sampling layer, a multi-layer perceptron layer and a max-pooling layer, and the reconstruction decoding module comprises a PointNet layer, a local attention mechanism layer, a one-dimensional deconvolution layer, a first multi-layer perceptron layer, a second multi-layer perceptron layer and an up-sampling layer.

[0067] In some embodiments, the expression of the loss function of the three-dimensional point cloud data completion network model is specifically as follows:

[0068] ;

[0069] In the above formula, denotes the loss function of the three-dimensional point cloud data completion network model, denotes an empirical coefficient, denotes a reconstruction loss function, denotes a self-distillation loss function, denotes a corresponding completion result of the target set, denotes a complete ground truth point cloud, denotes a predicted point cloud output by the reconstruction decoding module, denotes a corresponding resolution ground truth output by the reconstruction decoding module.

[0070] S300, based on the three-dimensional point cloud data completion network model, performing three-dimensional point cloud completion on the incomplete three-dimensional point cloud data set to obtain a complete three-dimensional point cloud data set;

[0071] It should be noted that in some embodiments, step S300 can comprise: S310, inputting the incomplete three-dimensional point cloud data set into the three-dimensional point cloud data completion network model;

[0072] S320, based on the multi-scale hierarchical knowledge self-distillation encoding module of the three-dimensional point cloud data completion network model, performing self-distillation loss encoding on the incomplete three-dimensional point cloud data set to obtain sparse point cloud data and neighborhood features;

[0073] Specifically, the incomplete 3D point cloud dataset is input into the multi-scale hierarchical knowledge self-distillation encoding module; based on the self-distillation module of the multi-scale hierarchical knowledge self-distillation encoding module, feature extraction is performed on the incomplete 3D point cloud dataset to obtain several preliminary neighborhood features; the preliminary neighborhood features are calculated using the softmax activation function to obtain the probability distributions corresponding to the preliminary neighborhood features; the probability distribution corresponding to the last preliminary neighborhood feature is used as a supervision signal, and knowledge backflow is achieved through KL divergence to output the sparse point cloud data and the neighborhood features.

[0074] More specifically, the incomplete 3D point cloud dataset is input into the self-distillation module of the multi-scale hierarchical knowledge self-distillation encoding module; based on the k-nearest neighbor layer of the self-distillation module, a k-nearest neighbor search is performed on the incomplete 3D point cloud dataset to construct local nearest neighbor features of the point cloud data; based on the farthest point sampling layer of the self-distillation module, center point sampling is performed on the local nearest neighbor features of the point cloud data to obtain the center points of the local nearest neighbor features; based on the multilayer perceptron layer of the self-distillation module, feature mapping is performed on the center points of the local nearest neighbor features to obtain the mapped local nearest neighbor features; based on the max pooling layer of the self-distillation module, the mapped local nearest neighbor features are aggregated to obtain several preliminary neighborhood features.

[0075] In some specific embodiments, such as Figure 4 As shown, it should first be noted that traditionally, various aggregation operations are used to encode global or local shapes, such as max pooling and set abstraction. Therefore, these methods inevitably ignore specific low-order geometric features. This phenomenon can be further amplified, especially when the input is incomplete. To mitigate the feature performance degradation caused by aggregation, this invention utilizes multi-scale hierarchical knowledge distillation to learn the differences between different local features by maximizing their mutual information. Since they can be integrated into any hierarchical model, they can replace the encoder in various point cloud completion methods. Specifically, this invention applies self-distillation modules to SnowflakeNet and PMP-Net++ to replace their set abstraction modules. Specifically, with incomplete point clouds... As input, the encoder integrates three ensemble abstraction modules (SA, including k-nearest neighbor kNN, farthest point sampling FPS, MLP, and max pooling AgG) to cloud-compute the neighborhood features of the input points. The probability distributions y of the three SAs are obtained by applying softmax activation. The last SA is then subjected to self-distillation loss. The signal is passed back to the two students in front, SA ( and ). Although no real probability distribution is available, only a geometric prior is available, but its implicit encoding can still be used to represent a particular object and to represent its virtual probability distribution. Thus, we map the encodings from different aggregation operations to a probability space, through a softmax activation, making the different distributions act as virtual objective functions. We designate the distribution of the deepest layer module as supervision to the previous modules. Let be the KL divergence, then the self-distillation loss is expressed as

[0076] ;

[0077] In the above equation, denotes the loss function of the self-distillation module, denotes the KL divergence, denotes the predicted label distribution, denotes the real label distribution, denotes the distribution of the last layer, .

[0078] where is the predicted value, . Note that in the self-distillation module, the predicted label distribution is also compared with the ground truth distribution . Knowledge can be backflowed in the forward training step through this formula to provide stronger supervision.

[0079] S330, based on the multi-layer perception module of the three-dimensional point cloud data completion network model, the data dimension reduction processing is performed on the sparse point cloud data and the neighborhood feature, to obtain a rough point cloud data set;

[0080] S340, based on the reconstruction decoding module of the three-dimensional point cloud data completion network model, data reconstruction is performed on the rough point cloud data set, to obtain the complete three-dimensional point cloud data set.

[0081] Specifically, the coarse point cloud dataset is input to a reconstruction decoding module of the three-dimensional point cloud data completion network model; based on a PointNet layer of the reconstruction decoding module, point cloud registration processing is performed on the coarse point cloud dataset to obtain current point cloud neighborhood features; based on a local attention mechanism layer of the reconstruction decoding module, local feature extraction processing is performed on the current point cloud neighborhood features to obtain local features; global features of the coarse point cloud dataset are obtained, and the local features are combined to obtain point cloud correlation features; based on a one-dimensional deconvolution layer of the reconstruction decoding module, the point cloud correlation features are copied upwards to obtain copied point cloud correlation features; based on a first multi-layer perceptron layer and a second multi-layer perceptron layer of the reconstruction decoding module, feature mapping is performed on the copied point cloud correlation features to obtain point-by-point displacement of point cloud features; based on an up-sampling layer of the reconstruction decoding module, up-sampling processing is performed on the coarse point cloud dataset to obtain an up-sampled coarse point cloud dataset; the point-by-point displacement of the point cloud features and the up-sampled coarse point cloud dataset are added to obtain the complete three-dimensional point cloud dataset.

[0082] In some embodiments, as shown in FIG. 2, based on the sparse points Figure 5 sampled by the SA in the encoder and their neighborhood features , a MLP (multi-layer perceptron) dimensionality reduction mapping can obtain a coarse point cloud , and the present application uses a decoder to complete the reconstruction of . In the decoding process, the present application regards the maximum-pooled neighborhood features obtained by the encoder as global features G. Thereafter, the present application integrates three SPD (splitting-based deconvolution) modules to recover the point cloud from coarse to fine. Each SPD inputs the current coarse point cloud from the previous step to a PointNet to obtain the current point cloud neighborhood features , combines the global features G and local attention to obtain the correlation between the neighborhood features and their surrounding points and the global features G , and copies the correlation upwards through a one-dimensional deconvolution , and finally maps these features to point-by-point displacement based on a MLP. At the same time, based on the point cloud obtained by the previous SPD (here, taking the coarse point cloud input to the first SPD as an example), the number of points is doubled through simple up-sampling (coordinate copying), and finally these up-sampled points are added to the point-by-point displacement to obtain the complete point cloud output , where is the output of the final SPD. This procedure is equivalent to rearranging the generated points based on upsampled original coordinates, rather than simply shuffling high-order representations like PU-Net, or grid patching like PCN. Overall, the network aims to reconstruct a complete point cloud from an incomplete input reconstructing a complete point cloud wherein and are the size of the ground truth point cloud and the incomplete input point cloud, respectively. Specifically, the reconstruction loss can be formulated as:

[0083]

[0084] In the above formula, denotes the loss function of the reconstruction decoding module, denotes the incomplete three-dimensional point cloud dataset, denotes the complete three-dimensional point cloud dataset, denotes the point cloud data points in the incomplete three-dimensional point cloud dataset, denotes the point cloud data points in the complete three-dimensional point cloud dataset, denotes the L2 norm.

[0085] Finally, as shown in Figure 6 , the comparison of the completion error of the model and other models for the dataset PCN , through more effective encoding (MSG and HSD), the completion accuracy is improved, and the data is as shown in Table 1.

[0086] Table 1 Comparison of completion error data of various models

[0087]

[0088] Please refer to Figure 2 , the application embodiment further provides a three-dimensional point cloud completion system based on multi-scale structured knowledge distillation, which can realize the above-mentioned three-dimensional point cloud completion method based on multi-scale structured knowledge distillation. The system comprises:

[0089] A first module 201 is configured to construct an incomplete three-dimensional point cloud dataset.

[0090] A second module 202 is configured to introduce a multi-scale hierarchical knowledge self-distillation encoding module and a reconstruction decoding module to construct a three-dimensional point cloud data completion network model.

[0091] A third module 203 is configured to perform three-dimensional point cloud completion on the incomplete three-dimensional point cloud dataset based on the three-dimensional point cloud data completion network model to obtain a complete three-dimensional point cloud dataset.

[0092] ​It can be understood that the contents in the method embodiments are applicable to the system embodiments, the system embodiments specifically implement the functions same as those of the method embodiments, and achieve the beneficial effects same as those of the method embodiments.

[0093] The preferred embodiments of the application are described above with reference to the accompanying drawings, and are not intended to limit the scope of the application. Any modification, equivalent replacement and improvement made by those skilled in the art without departing from the scope and essence of the application shall be within the scope of the application.

Claims

1. A 3D point cloud completion method based on multi-scale structured knowledge distillation, characterized in that, The method includes the following steps: Constructing an incomplete 3D point cloud dataset; A multi-scale hierarchical knowledge self-distillation encoding module and a reconstruction decoding module are introduced to construct a 3D point cloud data completion network model. The 3D point cloud data completion network model includes a multi-scale hierarchical knowledge self-distillation encoding module, a multilayer perceptron module, and a reconstruction decoding module, which are sequentially connected. The multi-scale hierarchical knowledge self-distillation encoding module includes several self-distillation modules, each of which includes a k-nearest neighbor layer, a farthest point sampling layer, a multilayer perceptron layer, and a max pooling layer. The reconstruction decoding module includes a PointNet layer, a local attention mechanism layer, a one-dimensional deconvolution layer, a first multilayer perceptron layer, a second multilayer perceptron layer, and an upsampling layer. The incomplete 3D point cloud dataset is input into the 3D point cloud data completion network model; Based on the k-nearest neighbor layer of the self-distillation module of the multi-scale hierarchical knowledge self-distillation encoding module in the 3D point cloud data completion network model, a k-nearest neighbor search is performed on the incomplete 3D point cloud dataset to construct local nearest neighbor features of the point cloud data. Based on the farthest point sampling layer of the self-distillation module, the center point of the local nearest neighbor feature of the point cloud data is sampled to obtain the center point of the local nearest neighbor feature. Based on the multilayer perceptron layer of the self-distillation module, feature mapping is performed on the center point of the local nearest neighbor feature to obtain the mapped local nearest neighbor feature; Based on the max pooling layer of the self-distillation module, the mapped local nearest neighbor features are aggregated to obtain several preliminary neighborhood features. The preliminary neighborhood features are calculated using the softmax activation function to obtain the probability distributions corresponding to several preliminary neighborhood features; The probability distribution corresponding to the last preliminary neighborhood feature is used as a supervision signal. Knowledge feedback is achieved through KL divergence, and sparse point cloud data and neighborhood features are output. The multilayer perceptron module based on the three-dimensional point cloud data completion network model performs data dimensionality reduction processing on the sparse point cloud data and the neighborhood features to obtain a coarse point cloud dataset. The reconstruction and decoding module based on the 3D point cloud data completion network model reconstructs the coarse point cloud dataset to obtain the complete 3D point cloud dataset.

2. The method according to claim 1, characterized in that, The specific expression for the loss function of the 3D point cloud data completion network model is as follows: ； In the above formula, This represents the loss function for a 3D point cloud data completion network model. Represents the empirical coefficient. Represents the reconstruction loss function. This represents the self-distillation loss function. This represents the corresponding completion result of the target set. Representing a complete truth point cloud, This represents the predicted point cloud output by the reconstruction decoding module. This indicates the actual resolution value output by the reconstruction decoding module.

3. The method according to claim 1, characterized in that, The loss function of the self-distillation module is as follows: ； In the above formula, This represents the loss function of the self-distillation module. Denotes KL divergence, This represents the predicted label distribution. Represents the true label distribution. Indicates the distribution of the last layer. .

4. The method according to claim 1, characterized in that, The reconstruction and decoding module based on the 3D point cloud data completion network model reconstructs the coarse point cloud dataset to obtain the complete 3D point cloud dataset, including: The coarse point cloud dataset is input into the reconstruction and decoding module of the 3D point cloud data completion network model; Based on the PointNet layer of the reconstruction decoding module, point cloud registration processing is performed on the coarse point cloud dataset to obtain the current point cloud neighborhood features; Based on the local attention mechanism layer of the reconstruction decoding module, local feature extraction processing is performed on the current point cloud neighborhood features to obtain local features; Obtain the global features of the coarse point cloud dataset and combine them with the local features to obtain the point cloud association features; Based on the one-dimensional deconvolution layer of the reconstruction decoding module, the point cloud association features are copied upwards to obtain the copied point cloud association features. Based on the first and second multilayer perceptron layers of the reconstruction decoding module, feature mapping is performed on the copied point cloud associated features to obtain the point-by-point displacement of the point cloud features. Based on the upsampling layer of the reconstruction decoding module, the coarse point cloud dataset is upsampled to obtain the upsampled coarse point cloud dataset. The point-by-point displacement of the point cloud features is added to the upsampled coarse point cloud dataset to obtain the complete 3D point cloud dataset.

5. The method according to claim 4, characterized in that, The loss function of the reconstruction decoding module is as follows: ； In the above formula, This represents the loss function of the reconstruction decoding module. This represents a non-complete 3D point cloud dataset. Represents a complete 3D point cloud dataset. This represents point cloud data points in an incomplete 3D point cloud dataset. This represents the point cloud data points in a complete 3D point cloud dataset. This represents the L2 norm.

6. A 3D point cloud completion system based on multi-scale structured knowledge distillation, characterized in that, The system includes: The first module is used to construct an incomplete 3D point cloud dataset; The second module is used to introduce a multi-scale hierarchical knowledge self-distillation encoding module and a reconstruction decoding module to construct a 3D point cloud data completion network model. The 3D point cloud data completion network model includes a multi-scale hierarchical knowledge self-distillation encoding module, a multilayer perceptron module, and a reconstruction decoding module, which are connected sequentially. The multi-scale hierarchical knowledge self-distillation encoding module includes several self-distillation modules, each of which includes a k-nearest neighbor layer, a farthest point sampling layer, a multilayer perceptron layer, and a max pooling layer. The reconstruction decoding module includes a PointNet layer, a local attention mechanism layer, a one-dimensional deconvolution layer, a first multilayer perceptron layer, a second multilayer perceptron layer, and an upsampling layer. The third module is used to perform the following steps: The incomplete 3D point cloud dataset is input into the 3D point cloud data completion network model; Based on the k-nearest neighbor layer of the self-distillation module of the multi-scale hierarchical knowledge self-distillation encoding module in the 3D point cloud data completion network model, a k-nearest neighbor search is performed on the incomplete 3D point cloud dataset to construct local nearest neighbor features of the point cloud data. Based on the farthest point sampling layer of the self-distillation module, the center point of the local nearest neighbor feature of the point cloud data is sampled to obtain the center point of the local nearest neighbor feature. Based on the multilayer perceptron layer of the self-distillation module, feature mapping is performed on the center point of the local nearest neighbor feature to obtain the mapped local nearest neighbor feature; Based on the max pooling layer of the self-distillation module, the mapped local nearest neighbor features are aggregated to obtain several preliminary neighborhood features. The preliminary neighborhood features are calculated using the softmax activation function to obtain the probability distributions corresponding to several preliminary neighborhood features; Using the probability distribution corresponding to the last preliminary neighborhood feature as a supervision signal, knowledge feedback is achieved through KL divergence, outputting sparse point cloud data and neighborhood features. The multilayer perceptron module based on the three-dimensional point cloud data completion network model performs data dimensionality reduction processing on the sparse point cloud data and the neighborhood features to obtain a coarse point cloud dataset. The reconstruction and decoding module based on the 3D point cloud data completion network model reconstructs the coarse point cloud dataset to obtain the complete 3D point cloud dataset.

Images