An improved method, apparatus and processing device for three-dimensional point cloud completion

Abstract

The application provides an improved three-dimensional point cloud completion method, device and processing equipment, a novel three-dimensional point cloud completion network structure is specially designed, a mapping relationship is established between input and output through the design and training of a high-precision completion network, various correlations in the point cloud are deeply analyzed, the correlations are maximally utilized, various levels of features extracted in the point cloud completion process and different granularity point clouds generated are timely corrected, and finally a high-precision, high-stability and efficient three-dimensional point cloud completion effect is formed.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of three-dimensional point cloud completion, in particular to an improved three-dimensional point cloud completion method, device and processing equipment. BACKGROUND

[0002] With the popularization of various three-dimensional scanning devices such as laser radar and depth camera, the acquisition of point cloud data has become more convenient, and has been widely applied in the fields of robots, autonomous driving and three-dimensional modeling.

[0003] However, the original point cloud data directly collected by these three-dimensional scanning devices often presents sparsity and incompleteness due to occlusion, device resolution or acquisition distance limitations. Therefore, how to generate complete point cloud from partial observation data is of great significance to promote the development of downstream applications.

[0004] Point Completion Network (PCN) is one of the pioneering works in the field of point cloud completion. This method adopts an encoder-decoder structure, in which the encoder extracts global features from the incomplete point cloud using a shared perception machine, and the decoder adopts a top-down approach, i.e., first generating a complete coarse-grained point cloud using a multi-layer perception machine, and then using a shared perception machine to complete the local details. The advantage of this method is that the calculation is simple, and the completion accuracy is very stable under various changes of the target point cloud. The disadvantage is that the completion process is very rough, and the various correlations in the point cloud are not fully utilized to timely correct the extracted features at different levels and the generated point clouds at different granularities, resulting in very low completion accuracy.

[0005] In order to improve the completion accuracy, various attention mechanisms are introduced on the basis of the above-mentioned encoder-decoder structure. Representative works include Point Cloud Completion with Geometry-Aware Transformers (PointTr), Patch Seeds based Point Cloud Completion with Upsample Transformer (SeedFormer) and Complementing Point Cloud via Self-view Augmentation and Self-structure Dual-generator (SVDFormer).

[0006] PointTr introduces a self-attention mechanism in a missing point cloud encoder based on a dynamic graph convolutional neural network (DGCNN), uses the local neighborhood feature correlation and global feature correlation of the points in the missing point cloud to correct the features of the points layer by layer; SeedFormer uses a self-attention mechanism between the abstraction layers of a missing point cloud encoder based on PointNet++, uses the global feature correlation of the points in the missing point cloud at different scales to correct the features of the points layer by layer; meanwhile, a cross-attention mechanism is introduced in the medium-granularity and fine-granularity point cloud decoders, and the mutual correlation of the coarse-granularity point cloud features and other granularity point cloud features is used to correct the features of other granularity point cloud layer by layer; SVDFormer introduces a self-attention mechanism in a missing point cloud encoder based on three-view depth maps, uses the global feature correlation of the depth maps to correct the global features of the depth maps, and meanwhile introduces a cross-attention mechanism in the medium-granularity and fine-granularity point cloud decoders, and uses the mutual correlation of the missing point cloud features and the coarse-granularity point cloud features to correct the coarse-granularity point cloud features. These methods effectively improve the completion accuracy.

[0007] However, the inventors of the present application found that whether the various correlations on which the above methods are based exist in actual applications, and if so, how strong or weak they are, have not been thoroughly analyzed, which leads to the fact that after using certain correlations, the above methods not only waste a lot of computing resources, but also do not improve the overall completion accuracy.

[0008] Therefore, how to make the most of the various correlations in the point cloud based on in-depth analysis of the various correlations in the point cloud, and timely correct the features extracted at each level and the different granularity point clouds generated in the point cloud completion process, so as to further improve the overall completion accuracy, is still a problem to be solved. SUMMARY

[0009] The present application provides an improved three-dimensional point cloud completion method, device and processing equipment, a novel three-dimensional point cloud completion network structure is specially designed, a mapping relationship is established between the input and the output through the design and training of the high-precision completion network, the various correlations in the point cloud are maximized based on in-depth analysis of the various correlations in the point cloud, and the features extracted at each level and the different granularity point clouds generated in the point cloud completion process are timely corrected, and finally a high-precision, high-stability and efficient three-dimensional point cloud completion effect is formed.

[0010] In a first aspect, the present application provides an improved three-dimensional point cloud completion method, which comprises:

[0011] obtaining a missing point cloud to be completed, wherein the missing point cloud to be completed is specifically a three-dimensional point cloud;

[0012] inputting the incomplete point cloud to be completed into a pre-configured three-dimensional point cloud completion network, wherein the three-dimensional point cloud completion network sequentially comprises an incomplete point cloud encoder based on a global feature fusion point completion network, a coarse-grained point cloud decoder based on residual fine-tuning, a medium-grained point cloud decoder based on feature layer-by-layer fusion correction, and a fine-grained point cloud decoder based on feature layer-by-layer fusion correction, the incomplete point cloud encoder extracts incomplete point cloud features according to the input point cloud of the network, the coarse-grained point cloud decoder generates a coarse-grained complete point cloud according to the incomplete point cloud features, the medium-grained point cloud decoder generates a medium-grained complete point cloud according to the input point cloud of the network, the incomplete point cloud features, and the coarse-grained complete point cloud, and the fine-grained point cloud decoder generates a fine-grained complete point cloud according to the input point cloud of the network, the incomplete point cloud features, and the medium-grained complete point cloud;

[0013] extracting the fine-grained complete point cloud output by the three-dimensional point cloud completion network as a point cloud completion result.

[0014] In a second aspect, the present application provides an improved three-dimensional point cloud completion device, which comprises:

[0015] an acquisition unit configured to acquire an incomplete point cloud to be completed, wherein the incomplete point cloud to be completed is a three-dimensional point cloud;

[0016] a completion unit configured to input the incomplete point cloud to be completed into a pre-configured three-dimensional point cloud completion network, wherein the three-dimensional point cloud completion network sequentially comprises an incomplete point cloud encoder based on a global feature fusion point completion network, a coarse-grained point cloud decoder based on residual fine-tuning, a medium-grained point cloud decoder based on feature layer-by-layer fusion correction, and a fine-grained point cloud decoder based on feature layer-by-layer fusion correction, the incomplete point cloud encoder extracts incomplete point cloud features according to the input point cloud of the network, the coarse-grained point cloud decoder generates a coarse-grained complete point cloud according to the incomplete point cloud features, the medium-grained point cloud decoder generates a medium-grained complete point cloud according to the input point cloud of the network, the incomplete point cloud features, and the coarse-grained complete point cloud, and the fine-grained point cloud decoder generates a fine-grained complete point cloud according to the input point cloud of the network, the incomplete point cloud features, and the medium-grained complete point cloud;

[0017] an extraction unit configured to extract the fine-grained complete point cloud output by the three-dimensional point cloud completion network as a point cloud completion result.

[0018] In a third aspect, the present application provides a processing device comprising a memory and a processor, the memory stores a computer program, and the processor executes the computer program in the memory to perform the method provided in the first aspect of the present application.

[0019] In a fourth aspect, the present application provides a computer readable storage medium, which stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the method provided in the first aspect of the present application.

[0020] From the above, the present application has the following beneficial effects:

[0021] For the three-dimensional point cloud completion target, the present application specially designs a novel three-dimensional point cloud completion network structure, through the design and training of the high-precision completion network, a mapping relationship is established between the input and the output, on the basis of in-depth analysis of various correlations in the point cloud, the correlations are maximally utilized, and each level feature extracted in the point cloud completion process and different granularity point clouds generated are timely corrected, finally forming a high-precision, high-stability and efficient three-dimensional point cloud completion effect. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0023] Figure 1 A flowchart of the improved three-dimensional point cloud completion method of the present application;

[0024] Figure 2 A working diagram of the three-dimensional point cloud completion network of the present application;

[0025] Figure 3 Another working diagram of the three-dimensional point cloud completion network of the present application;

[0026] Figure 4 A working diagram of the incomplete point cloud encoder of the global feature fusion point completion network of the present application;

[0027] Figure 5 A working diagram of the coarse-grained point cloud decoder based on residual fine-tuning of the present application;

[0028] Figure 6 A working diagram of the medium / fine-grained point cloud decoder based on feature layer-by-layer fusion correction of the present application;

[0029] Figure 7 A structural diagram of the feature correction module of the present application;

[0030] Figure 8 A structural diagram of the improved three-dimensional point cloud completion device of the present application;

[0031] Figure 9 A structural diagram of the processing device of the present application. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be described clearly and completely in the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0033] The terms "first", "second", and the like in the specification and claims of the present application and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or modules does not necessarily have to be limited to those steps or modules clearly listed, but can include other steps or modules that are not clearly listed or inherent to the process, method, product or device. The naming or numbering of the steps appearing in the present application does not mean that the steps in the method flow must be executed in the order / time sequence indicated by the naming or numbering. The execution order of the steps that have been named or numbered can be changed according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0034] The division of modules appearing in the present application is a logical division, and in actual application, there can be another division manner, for example, a plurality of modules can be combined or integrated in another system, or some features can be ignored or not executed, in addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be through some interface, the indirect coupling or communication connection between the modules can be electrical or other similar forms, which are not limited in the present application. In addition, the modules or sub-modules described as separate components can or can not be physically separate, can or can not be physical modules, or can be distributed into a plurality of circuit modules, and some or all of the modules can be selected to achieve the purpose of the present application according to actual needs.

[0035] Before introducing the improved three-dimensional point cloud completion method provided by the present application, the background content involved in the present application will be introduced first.

[0036] The application provides an improved three-dimensional point cloud completion method, device, processing equipment and computer readable storage medium. The application specially designs a novel three-dimensional point cloud completion network structure, establishes a mapping relationship between the input and output through the design and training of a high-precision completion network, and maximizes the use of various correlations in the point cloud on the basis of in-depth analysis of the correlations, timely corrects the features extracted at each level and the different granularity point clouds generated in the point cloud completion process, and finally forms a high-precision, high-stability and efficient three-dimensional point cloud completion effect.

[0037] The improved three-dimensional point cloud completion method mentioned in the application can be executed by an improved three-dimensional point cloud completion device, or a server, a physical host or a user equipment (UE) or other types of processing equipment integrated with the improved three-dimensional point cloud completion device. The improved three-dimensional point cloud completion device can be realized in hardware or software, the UE can be a terminal device such as a smart phone, a tablet computer, a notebook computer, a desktop computer or a personal digital assistant (PDA), and the processing equipment can be set in the form of a device cluster.

[0038] It can be understood that, considering that the scheme of the application is mainly based on the point cloud completion processing of the pre-configured three-dimensional point cloud completion network, the processing equipment executing the improved three-dimensional point cloud completion method of the application or carrying the corresponding application service of the improved three-dimensional point cloud completion method of the application usually meets the required data processing capability, and the specific device type and device deployment form are flexible and can be flexibly configured according to actual conditions.

[0039] If the collection of three-dimensional point clouds or the display of three-dimensional point cloud completion results and other specific data applications are involved, it is easy to understand that the processing equipment needs to be further configured in terms of software and hardware.

[0040] As an example, the processing equipment can display the processing process / result of the three-dimensional point cloud through a display screen (including a touch screen) configured by itself, an external display device or other devices with a display screen.

[0041] As another example, the processing equipment can include a first device part for executing the training work of the three-dimensional point cloud completion network, a second device part for executing the application work of the three-dimensional point cloud completion network and a third device part for executing the three-dimensional point cloud collection work.

[0042] Next, the improved three-dimensional point cloud completion method provided by the application will be introduced.

[0043] Firstly, refer toFigure 1 , Figure 1 A flowchart of the improved three-dimensional point cloud completion method of the present application is shown. The improved three-dimensional point cloud completion method provided by the present application can specifically include the following steps S101 to S103:

[0044] Step S101, obtaining a to-be-completed incomplete point cloud, wherein the to-be-completed incomplete point cloud is specifically a three-dimensional point cloud;

[0045] It can be understood that the present application scheme starts from a pre-configured three-dimensional point cloud completion network, and in the specific application of the three-dimensional point cloud completion network, the corresponding three-dimensional point cloud completion processing is usually initiated in the form of a work task. Correspondingly, it can also involve obtaining three-dimensional point cloud completion task processing, which can be initiated manually, received from other devices, or initiated autonomously according to the corresponding autonomous initiation strategy. These are all possible.

[0046] And for the three-dimensional point cloud to be completed, that is, the to-be-completed incomplete point cloud, in the case of a three-dimensional point cloud completion task, it can be carried in the task information, or it can be extracted from the corresponding storage location according to the task information, or it can be real-time collected and processed according to the task information, or it can be entered manually.

[0047] In addition, for the obtained three-dimensional point cloud, it is usually assumed to be incomplete and needs to be completed. Of course, in some cases, it can also be determined to be incomplete and needs to be completed after detection, and then the specific three-dimensional point cloud completion processing is promoted by the present application scheme.

[0048] Among them, the three-dimensional point cloud is usually collected by various three-dimensional scanning devices such as laser radar and depth camera.

[0049] Step S102, inputting the to-be-completed incomplete point cloud into a pre-configured three-dimensional point cloud completion network, wherein the three-dimensional point cloud completion network sequentially includes an incomplete point cloud encoder based on a global feature fusion point completion network, a coarse-grained point cloud decoder based on residual fine-tuning, a medium-grained point cloud decoder based on feature layer-by-layer fusion correction, and a fine-grained point cloud decoder based on feature layer-by-layer fusion correction. The incomplete point cloud encoder extracts incomplete point cloud features according to the input point cloud of the network, the coarse-grained point cloud decoder generates coarse-grained complete point cloud according to the incomplete point cloud features, the medium-grained point cloud decoder generates medium-grained complete point cloud according to the input point cloud of the network, incomplete point cloud features and coarse-grained complete point cloud, and the fine-grained point cloud decoder generates fine-grained complete point cloud according to the input point cloud of the network, incomplete point cloud features and medium-grained complete point cloud;

[0050] It can be seen that the application designs a novel three-dimensional point cloud completion network structure, which is specifically composed of four parts of a defective point cloud encoder based on a global feature fusion point completion network, a coarse-grained point cloud decoder based on residual fine-tuning, a medium-grained point cloud decoder based on feature layer-by-layer fusion correction, and a fine-grained point cloud decoder based on feature layer-by-layer fusion correction, which are sequentially connected, and also correspond to the data flow between the model structures. In addition, there is an additional connection relationship between the interval modules.

[0051] Specifically, in combination with Figure 2 A working schematic diagram of the three-dimensional point cloud completion network of the application can be seen. In the three-dimensional point cloud completion network, the defective point cloud encoder based on the global feature fusion point completion network corresponds to the network input side, the fine-grained point cloud decoder based on the feature layer-by-layer fusion correction corresponds to the network output side, and the processing idea of defective point cloud-coarse-grained point cloud-medium-grained point cloud-fine-grained point cloud is followed to gradually complete the completion / reconstruction work of the input network of the incomplete point cloud to be completed. Here, reference can be made to Figure 2 A logical schematic diagram of the three-dimensional point cloud completion processing of the application is shown.

[0052] Corresponding to the application of the three-dimensional point cloud completion network, the application can also involve related network training in the early stage, which also corresponds to the case that high-precision, high-stability and high-efficiency three-dimensional point cloud completion effect can be obtained under the network structure design of the application.

[0053] For this, the method of the application can further include, in specific operation:

[0054] Obtaining a defective point cloud sample and its corresponding fine-grained complete point cloud true value;

[0055] Training the three-dimensional point cloud completion network based on the defective point cloud sample and its corresponding fine-grained complete point cloud true value.

[0056] Among them, in addition to the acquisition method of the above incomplete point cloud to be completed, the acquisition of the defective point cloud sample can also involve the configuration of the virtual defective point cloud, that is, the virtual defective point cloud that can be put into the network training work can be obtained by modifying the real defective point cloud, or the virtual defective point cloud that can be put into the network training work can be directly constructed, so as to enrich the diversity of training samples and guarantee the generalization ability of the network after training.

[0057] The fine-grained complete point cloud true value corresponding to the defective point cloud sample is usually completed by manual close-range multi-view point cloud graph cutting and splicing, or is the fine-grained complete point cloud sampled in the process of constructing the virtual defective point cloud, or is obtained by searching the defective point cloud sample in a large-scale fine-grained complete data set.

[0058] In the specific network training process, it can be understood that the specific network training scheme (for example, 80% of the training set and 20% of the validation set) can be adopted, and the loss function (for example, the chamfer distance loss function and the earth moving distance loss function) used in the training process. Considering that the focus of the present application is not the focus of the present application, the related ready-made scheme can be used, and therefore, no further expansion is made here.

[0059] Of course, it can also be understood that both of them can be further optimized and improved on the basis of the existing scheme or even adopt a brand-new self-developed scheme, which is also possible in some cases.

[0060] Step S103, extracting the fine-grained complete point cloud output by the three-dimensional point cloud completion network as the point cloud completion result.

[0061] As can be easily understood, after the current incomplete point cloud to be completed is processed by the three-dimensional point cloud completion network, the point cloud completion result output by the three-dimensional point cloud completion network can be extracted, that is, the fine-grained complete point cloud generated by the fine-grained point cloud decoder based on feature layer-by-layer fusion correction in the last layer of the three-dimensional point cloud completion network, completing a three-dimensional point cloud completion task.

[0062] Further, it can be understood that the three-dimensional point cloud can be collected and configured for related point cloud application services. For this, the point cloud completion can also be a data preprocessing means for improving the quality of digital twinning, target recognition, detection and tracking, and other specific point cloud application services, and the background can involve three-dimensional modeling, robots and autonomous driving, and other specific application scenarios.

[0063] Correspondingly, after the fine-grained complete point cloud output by the three-dimensional point cloud completion network is extracted, it can be locally stored, stored remotely, displayed, pushed, output to complete the completion prompt or used for further point cloud application services.

[0064] From Figure 1 As shown in the embodiments, for the three-dimensional point cloud completion target, the present application specially designs a novel three-dimensional point cloud completion network structure. Through the design and training of the high-precision completion network, a mapping relationship is established between the input and the output, the correlation in the point cloud is deeply analyzed, and the correlation is maximally utilized to timely correct the features extracted at each level and the different granularity point clouds generated in the point cloud completion process, and finally a high-precision, high-stability and efficient three-dimensional point cloud completion effect is formed.

[0065] Next, the three-dimensional point cloud completion network specially designed by the present application will be further expanded.

[0066] From the overall level, the input of the high-precision point cloud completion network is the three-dimensional coordinate matrix of the incomplete point cloud to be completed, and the output is the three-dimensional coordinate matrix of the complete point cloud in fine granularity, that is, the goal of the high-precision three-dimensional point cloud completion network is to establish a mapping relationship between the input and the output through the design and training of the high-precision completion network, so as to realize the high-precision, high-stability and high-efficiency completion of the incomplete point cloud.

[0067] To this end, in combination with Figure 3 Another working schematic diagram of the three-dimensional point cloud completion network of the application is shown, in the process of point cloud completion, there are:

[0068] Firstly, in the first stage, the dimensional incomplete point cloud three-dimensional coordinate matrix (that is, the incomplete point cloud three-dimensional coordinate matrix containing points) is input into the incomplete point cloud encoder based on the global feature fusion point completion network, to obtain dimensional incomplete point cloud feature vector (that is, incomplete point cloud feature);

[0069] Then, in the second stage, the dimensional incomplete point cloud feature vector is continuously input into the coarse-grained point cloud decoder based on residual fine-tuning, to generate dimensional coarse-grained point cloud three-dimensional coordinate matrix (that is, coarse-grained complete point cloud);

[0070] Next, in the third stage, the dimensional coarse-grained point cloud three-dimensional coordinate matrix, dimensional incomplete point cloud feature vector and dimensional incomplete point cloud three-dimensional coordinate matrix are input into the medium-grained point cloud decoder based on feature layer-by-layer fusion correction, to obtain dimensional medium-grained point cloud three-dimensional coordinate matrix (that is, medium-grained complete point cloud);

[0071] Finally, in the fourth stage, the dimensional medium-grained point cloud three-dimensional coordinate matrix, dimensional incomplete point cloud feature vector and dimensional incomplete point cloud three-dimensional coordinate matrix are input into the fine-grained point cloud decoder based on feature layer-by-layer fusion correction, to obtain dimensional fine-grained point cloud three-dimensional coordinate matrix (that is, fine-grained complete point cloud).

[0072] For the overall level operation processing, it can be learned that compared with the existing completion network, the high-precision three-dimensional point cloud completion network designed by itself has the following two main features:

[0073] (1) The incomplete point cloud encoder directly uses PCN as the backbone network instead of PointNet++ or DGCNN, and does not use an attention mechanism to ensure the stability of the point features, and further improves the accuracy of the incomplete point cloud features by fusing the low-level global features and the high-level global features;

[0074] (2) The medium / fine-grained point cloud decoder directly reuses the incomplete point cloud features and the coarse / medium-grained point cloud features for layer-by-layer fusion to maximize the correction of the deviation of the point features in the dimension increasing process, and uses a feature correction network based on transformation-fine-tuning to correct the fused point features layer by layer to minimize the error of the point features in the fusion process.

[0075] The above measures effectively improve the completion accuracy. Experiments on multiple benchmark test sets show that the scheme of the present application exceeds the existing mainstream methods in multiple accuracy indicators, while the training and reasoning speed and the memory occupation are significantly reduced, meeting the needs of users for cost reduction and quality improvement.

[0076] Next, the incomplete point cloud encoder based on the global feature fusion point completion network, the coarse-grained point cloud decoder based on residual fine-tuning, the medium-grained point cloud decoder based on feature layer-by-layer fusion correction, and the fine-grained point cloud decoder based on feature layer-by-layer fusion correction are separately described in detail according to the above overall processing architecture related to specific specification parameters.

[0077] Reference Figure 4 The working schematic diagram of the incomplete point cloud encoder based on the global feature fusion point completion network of the present application is shown. For the incomplete point cloud encoder based on the global feature fusion point completion network corresponding to the first stage, it can specifically include a one-dimensional convolution module , a max pooling module , a replication module , a splicing module , a one-dimensional convolution module , a max pooling module , a splicing module , and a one-dimensional convolution module ;

[0078] Based on the above component modules, the working process of the incomplete point cloud encoder based on the global feature fusion point completion network can be:

[0079] 1.1) inputting a three-dimensional coordinate matrix of a three-dimensional incomplete point cloud to the one-dimensional convolution module to obtain a three-dimensional low-level feature matrix (i.e., 256-dimensional low-level features of 1024 points of the incomplete point cloud), wherein the network input point cloud is specifically denoted as ​​​a matrix of three-dimensional coordinates of the incomplete point cloud, is the number of incomplete point cloud center points;

[0080] 1.2) input the low-level feature matrix of dimension into the max pooling module to obtain the low-level feature vector of dimension ;

[0081] 1.3) replicate the low-level feature vector of dimension times in the row direction by the replication module , and then splice the replication result of the replication module with the low-level feature matrix of dimension obtained in 1.1) in the column direction by the splicing module to obtain the spliced feature matrix of dimension ;

[0082] 1.4) input the spliced feature matrix of dimension into the one-dimensional convolution module to obtain the high-level feature matrix of dimension ;

[0083] 1.5) input the fusion feature matrix of dimension into the max pooling module to obtain the high-level feature vector of dimension ;

[0084] 1.6) splice the high-level feature vector of dimension with the low-level feature vector of dimension obtained in 1.2) in the column direction by the splicing module to obtain the spliced feature vector of dimension ;

[0085] 1.7) input the spliced feature vector of dimension into the one-dimensional convolution module to obtain the incomplete point cloud feature vector of dimension , which is the incomplete point cloud feature extracted by the incomplete point cloud encoder based on the global feature fusion point completion network of the first layer of the three-dimensional point cloud completion network.

[0086] Under the above very specific network structure setting, it can be understood that, compared with the existing completion network, the incomplete point cloud encoder based on the global feature fusion point completion network specially designed for the first layer of the three-dimensional point cloud completion network has the following characteristics:

[0087] ​(1) Due to factors such as the resolution of the acquisition device, acquisition distance and acquisition angle, the local neighborhood distribution of the actual acquired point cloud changes drastically and the neighborhood correlation is weak. Using the neighborhood features of the points to correct the point features often leads to larger errors in the point features. Therefore, the residual point cloud encoder uses a point completion network encoder based on global features as the backbone network, instead of using PointNet++ based on multi-scale neighborhood features or DGCNN based on dynamic neighborhood features, in order to ensure the stability of the point features and lay the foundation for high-precision global feature extraction.

[0088] (2) In addition to the acquisition factors, the actual acquisition of point clouds is also affected by occlusion factors, resulting in various missing parts and details in the acquired point clouds. The global and local correlations are weak. Using the attention mechanism to correct the point features often leads to greater errors in the point features. Therefore, the residual point cloud encoder avoids using the attention mechanism to eliminate the impact of the above errors on the feature extraction accuracy.

[0089] (3) The low-level feature vector contains the basic geometric information of the point cloud, and the high-level feature vector contains the global structural information of the point cloud. The splicing and fusion of the two is conducive to better balancing the basic geometric information and the global structural information, and further improving the accuracy of residual point cloud feature extraction.

[0090] Next, refer to Figure 5 The diagram shown illustrates one operation of the coarse-grained point cloud decoder based on residual fine-tuning of this application. Specifically, the coarse-grained point cloud decoder based on residual fine-tuning, corresponding to the second stage, may include a transposed convolution module. Copy module splicing module One-dimensional convolution module Shape reshaping module Residual module and one-dimensional convolutional modules .

[0091] Based on the above-mentioned components, the working process of the coarse-grained point cloud decoder based on residual fine-tuning can be summarized as follows:

[0092] 2.1) The result obtained from 1.7) 3D residual cloud feature vector input transpose convolution module ,get Low-level transposed convolutional feature matrix;

[0093] 2.2) By the copy module The result obtained from 1.7) The feature vector of the defective cloud is copied 128 times in the row direction, and then the concatenation module performs the stitching. Copy module the replication result of 2.1), and the replication result of 2.2) is obtained The low-dimensional transposed convolution feature matrix is spliced in the column direction to obtain a dimensional spliced feature matrix.

[0094] 2.3) inputting the dimensional spliced feature matrix into a one-dimensional convolution module to obtain a dimensional fusion feature matrix.

[0095] 2.4) performing shape remodeling on the dimensional fusion feature matrix by a shape remodeling module to obtain a dimensional remodeled feature matrix.

[0096] 2.5) inputting the dimensional remodeled feature matrix into a residual module to obtain a dimensional remodeled residual feature matrix.

[0097] 2.6) inputting the dimensional remodeled residual feature matrix into a one-dimensional convolution module to obtain a dimensional coarse-grained point cloud three-dimensional coordinate matrix as a coarse-grained complete point cloud generated by the coarse-grained point cloud decoder based on residual fine-tuning.

[0098] And under the above very specific network structure setting, it can be understood that, compared with the existing completion network, the residual fine-tuning-based incomplete point cloud encoder specially designed for the second layer of the three-dimensional point cloud completion network has the following characteristics:

[0099] (1) In the process of generating a coarse-grained complete point cloud, the points are distributed sparsely, and the correlation between the point features is weak. Therefore, using an attention mechanism to correct the point features in the coarse-grained point cloud generation process often leads to greater errors in the point features, so the coarse-grained point cloud decoder based on residual fine-tuning avoids using an attention mechanism to eliminate the influence of the above errors on the feature extraction accuracy.

[0100] (2) The low-dimensional transposed convolution feature matrix contains the basic geometric information of the point cloud, and the incomplete point cloud feature vector contains the global structural information of the point cloud. Splicing and fusing the two is conducive to better balancing the basic geometric information and the global structural information, and improving the accuracy of generating a coarse-grained complete point cloud.

[0101] (3) Shape remodeling causes changes in the feature dimension space, which will introduce dimension conversion errors, so a residual module is introduced to the remodeled The dimensionality of the feature remodeling matrix is fine-tuned to reduce dimensionality conversion errors, further improving the generation accuracy of the coarse-grained complete point cloud.

[0102] Next, referring to Figure 6 The application shown is based on a working schematic diagram of a feature layer-by-layer fusion correction-based medium / fine-grained point cloud decoder, where the subscripted letters after the forward slash and the dimension numbers in the brackets correspond to the feature layer-by-layer fusion correction-based fine-grained point cloud decoder. When focusing on the feature layer-by-layer fusion correction-based medium-grained point cloud decoder, the content after the forward slash and in the brackets is ignored.

[0103] In the above case, for the feature layer-by-layer fusion correction-based medium-grained point cloud decoder corresponding to the third stage, it can specifically include a merging module , a farthest point sampling module , a one-dimensional convolution module , a feature correction module , a one-dimensional convolution module , a one-dimensional convolution module , a one-dimensional convolution module , a copying module , a splicing module , a one-dimensional convolution module , a feature correction module , a copying module , a splicing module , a one-dimensional convolution module , a feature correction module , a copying module , a splicing module , a one-dimensional convolution module , a shape remodeling module , a one-dimensional convolution module , and a copying module .

[0104] Based on the above constituent modules, the working process of the feature layer-by-layer fusion correction-based medium-grained point cloud decoder can be:

[0105] 3.1) The merging module merges the dimensional incomplete point cloud three-dimensional coordinate matrix, which is also input as 1.1), with the dimensional coarse-grained point cloud three-dimensional coordinate matrix obtained in 2.6), to obtain a dimensional merged point cloud three-dimensional coordinate matrix;

[0106] 3.2) input the dimensional merged point cloud three-dimensional coordinate matrix into the farthest point sampling module , to obtain a The 3D coordinate matrix of the sampled point cloud, where, Choose 512 or 1024;

[0107] 3.3) will 3D sampling point cloud 3D coordinate matrix input to 1D convolution module ,get dimensional initial feature matrix;

[0108] 3.4) will 3D initial feature matrix input feature correction module ,get Initialized corrected feature matrix;

[0109] 3.5) The result obtained from 1.7) 3D residual feature vector input to 1D convolution module ,get The feature vector of the residual cloud in the middle layer of the dimension is then... The feature vector of the mid-layer residual cloud is input into the one-dimensional convolution module. ,get The low-level residual cloud feature vector is then... Low-level residual cloud feature vector input to one-dimensional convolution module ,get Initial residual feature vector;

[0110] 3.6) By the copy module Will The initial residual feature vector of the dimensional cloud is copied in the row direction. Next, the splicing module Copy module The replication results are the same as those obtained in 3.4). The initial modified feature matrix is ​​concatenated along the column direction to obtain... Low-level concatenated feature matrix;

[0111] 3.7) will Low-level concatenation of feature matrices into a one-dimensional convolutional module ,get Low-dimensional feature matrix;

[0112] 3.8) will Low-dimensional feature matrix input feature correction module ,get Low-level modified feature matrix;

[0113] 3.9) By the copy module The result obtained from 3.5) The low-dimensional residual cloud feature vector is copied in the row direction. Next, the splicing module Copy module The replication results are the same as those obtained in 3.8). The lower-level modified feature matrices are concatenated along the column direction to obtain... Mid-level concatenation of feature matrices;

[0114] 3.10) will The feature matrix is ​​concatenated in the middle layer and input into the one-dimensional convolution module. ,get Mid-level feature matrix;

[0115] 3.11) will 2D mid-layer feature matrix input feature correction module ,get Mid-level modified feature matrix;

[0116] 3.12) By the copy module The result obtained from 3.5) The feature vector of the residual cloud in the middle layer is copied in the row direction. Next, the splicing module Copy module The replication results are the same as those obtained in 3.10). The modified feature matrices in the middle layer are concatenated along the column direction to obtain... High-level splicing feature matrix;

[0117] 3.13) will High-level concatenated feature matrix input to one-dimensional convolution module ,get High-level feature matrix;

[0118] 3.14) will High-dimensional feature matrix input shape reshaping module ,get Reshape the feature matrix in 3D;

[0119] 3.15) will 3D reshaping feature matrix input one-dimensional convolution module This yields the three-dimensional incremental coordinates of 2048 points;

[0120] 3.16) By the copy module The result obtained in 3.2) Copying the 3D coordinate matrix of the 3D sampled point cloud in the row direction This is repeated and added to the three-dimensional incremental coordinates of 2048 points to obtain... The three-dimensional coordinate matrix of the medium-granularity point cloud serves as the medium-granularity complete point cloud generated by the medium-granularity point cloud decoder based on feature-layer fusion correction.

[0121] And in the above-mentioned very specific network structure setting, it can be understood that, compared with the existing completion network, the third layer of the three-dimensional point cloud completion network specially designed based on the feature layer-by-layer fusion correction of the medium-grained point cloud decoder has the following characteristics:

[0122] (1) In the process of generating medium-grained complete point cloud, the low-dimensional initial feature of the point to the high-dimensional high-level feature extraction adopts the way of layer-by-layer splicing and fusing the corresponding level of incomplete point cloud feature, which integrates the fusion of global features throughout the feature extraction, which helps to strengthen the continuous guidance of global features to local features and correct the deviation caused in the feature dimensioning process in time.

[0123] (2) In the process of converting point features from low-dimensional space to high-dimensional space, conversion errors will be introduced, and the point distribution in the medium-grained complete point cloud is dense, and the correlation between point features is strong. Therefore, in the process of extracting low-dimensional initial features of points to high-dimensional high-level features, the way of layer-by-layer correction of features is adopted, which integrates the use of point feature correlation throughout the feature extraction, which helps to eliminate conversion errors in time and further improve the accuracy of point feature extraction.

[0124] Continue to pay attention to the fourth stage based on the feature layer-by-layer fusion correction of the fine-grained point cloud decoder, still refer to Figure 6 But in detail, the subscripted letter after the forward slash and the dimension number in the brackets shall prevail, that is, the subscripted letter after the forward slash is replaced by the subscripted letter before the forward slash, and the dimension number in the brackets is replaced by the dimension number outside the brackets, for example, it should be dimensional medium-grained point cloud three-dimensional coordinate matrix, not dimensional coarse-grained point cloud three-dimensional coordinate matrix; the input is the merging module , not the merging module .

[0125] Specifically, the fine-grained point cloud decoder based on feature layer-by-layer fusion correction includes merging module , farthest point sampling module , one-dimensional convolution module , feature correction module , one-dimensional convolution module , one-dimensional convolution module , one-dimensional convolution module , copy module , splicing module , one-dimensional convolution module , feature correction module , copy module , splicing module , one-dimensional convolution module , feature correction module , copy module , concatenation module , one-dimensional convolution module , shape remodeling module , one-dimensional convolution module , and copy module .

[0126] Based on the above constituent modules, the working process of the feature-based layer-by-layer fusion correction fine-grained point cloud decoder can be:

[0127] 4.1) by the merging module merge the dimensional incomplete point cloud three-dimensional coordinate matrix, which is also input in 1.1), with the dimensional medium-grained point cloud three-dimensional coordinate matrix obtained in 3.16), to obtain a dimensional merged point cloud three-dimensional coordinate matrix;

[0128] 4.2) input the dimensional merged point cloud three-dimensional coordinate matrix into the farthest point sampling module , to obtain a dimensional sampled point cloud three-dimensional coordinate matrix;

[0129] 4.3) input the dimensional sampled point cloud three-dimensional coordinate matrix into the one-dimensional convolution module , to obtain a dimensional initial feature matrix;

[0130] 4.4) input the dimensional initial feature matrix into the feature correction module , to obtain a dimensional initial corrected feature matrix;

[0131] 4.5) input the dimensional incomplete point cloud feature vector obtained in 1.7) into the one-dimensional convolution module , to obtain a dimensional medium-layer incomplete point cloud feature vector, and then input the dimensional medium-layer incomplete point cloud feature vector into the one-dimensional convolution module , to obtain a dimensional low-layer incomplete point cloud feature vector, and then input the dimensional low-layer incomplete point cloud feature vector into the one-dimensional convolution module , to obtain a dimensional initial incomplete point cloud feature vector;

[0132] 4.6) by the copy module copy the dimensional initial incomplete point cloud feature vector in the row direction Next, the splicing module Copy module The replication results are the same as those obtained in 4.4). The initial modified feature matrix is ​​concatenated along the column direction to obtain... Low-level concatenated feature matrix;

[0133] 4.7) will Low-level concatenation of feature matrices into a one-dimensional convolutional module ,get Low-dimensional feature matrix;

[0134] 4.8) will Low-dimensional feature matrix input feature correction module ,get Low-level modified feature matrix;

[0135] 4.9) By the copy module The result obtained from 4.5) The low-dimensional residual cloud feature vector is copied in the row direction. Next, the splicing module Copy module The replication results are the same as those obtained in 4.8). The lower-level modified feature matrices are concatenated along the column direction to obtain... Mid-level concatenation of feature matrices;

[0136] 4.10) will The feature matrix is ​​concatenated in the middle layer and input into the one-dimensional convolution module. ,get Mid-level feature matrix;

[0137] 4.11) will 2D mid-layer feature matrix input feature correction module ,get Mid-level modified feature matrix;

[0138] 4.12) By the copy module The result obtained from 4.5) The feature vector of the residual cloud in the middle layer of the dimension is copied in the row direction. Next, the splicing module Copy module The replication results are the same as those obtained in 4.11). The modified feature matrices in the middle layer are concatenated along the column direction to obtain... High-level splicing feature matrix;

[0139] 4.13) will High-level concatenated feature matrix input to one-dimensional convolution module ,get high-level feature matrix;

[0140] 4.14) inputting the high-level feature matrix into a shape remodeling module , to obtain a 3D remodeling feature matrix; 4.15) inputting the 3D remodeling feature matrix into a one-dimensional convolution module , to obtain 16384-point 3D incremental coordinates;

[0141] 4.16) copying the 3D coordinate matrix of the 3D sampling point cloud obtained in 4.2) in the row direction times, and adding the 16384-point 3D incremental coordinates, to obtain a 3D fine-grained point cloud coordinate matrix, as a fine-grained complete point cloud generated by the fine-grained point cloud decoder based on feature layer-by-layer fusion correction.

[0142] 4.16) copying the 3D coordinate matrix of the 3D sampling point cloud obtained in 4.2) in the row direction times, and adding the 16384-point 3D incremental coordinates, to obtain a 3D fine-grained point cloud coordinate matrix, as a fine-grained complete point cloud generated by the fine-grained point cloud decoder based on feature layer-by-layer fusion correction.

[0143] And under the above very specific network structure setting, it can be understood that, compared with the existing completion network, the fine-grained point cloud decoder based on feature layer-by-layer fusion correction specially designed for the fourth layer of the three-dimensional point cloud completion network has the following characteristics, similar to the medium-grained point cloud decoder based on feature layer-by-layer fusion correction of the third layer:

[0144] (1) In the process of generating a fine-grained complete point cloud, the low-dimensional initial feature of the point to the high-dimensional high-level feature extraction adopts a layer-by-layer splicing fusion of the corresponding level of incomplete point cloud feature, which integrates the fusion of global features throughout the feature extraction process, helps to strengthen the continuous guidance of global features to local features, and corrects the deviation generated in the feature dimensioning process in time;

[0145] (2) In the process of converting point features from low-dimensional space to high-dimensional space, conversion errors will be introduced, and the point distribution in the fine-grained complete point cloud is dense, and the correlation between point features is strong. Therefore, in the process of extracting low-dimensional initial features of points to high-dimensional high-level features, the layer-by-layer feature correction method is adopted, which utilizes the correlation of point features throughout the feature extraction process, helps to eliminate conversion errors in time, and further improves the accuracy of point feature extraction.

[0146] In addition, from the above content, it can be seen that both the medium-grained point cloud decoder based on feature layer-by-layer fusion correction and the fine-grained point cloud decoder based on feature layer-by-layer fusion correction contain corresponding feature correction modules based on transformation fine-tuning to improve feature extraction accuracy.

[0147] For this purpose, reference is made to Figure 7 ​​​​A structural schematic diagram of the feature correction module of the application is shown, and the module structure of the feature correction module in both the medium-granularity point cloud decoder based on feature layer-by-layer fusion correction and the fine-granularity point cloud decoder based on feature layer-by-layer fusion correction can specifically include a self-attention sub-module and a residual sub-module ;

[0148] On the basis of the above constituent modules, the working process of the feature correction module can be:

[0149] 5.1) inputting the original feature matrix of dimensions into the self-attention sub-module for transformation to obtain transformed features of dimensions;

[0150] 5.2) inputting the transformed features of dimensions into the residual sub-module for fine-tuning to obtain a corrected feature matrix of dimensions.

[0151] And under the above very specific network structure setting, it can be understood that the feature correction module adopts a transformation-fine-tuning idea to eliminate the conversion error of features from a low-dimensional space to a high-dimensional space.

[0152] On the one hand, the use of the self-attention mechanism can fully utilize the global correlation to transform the features and eliminate the conversion error as a whole.

[0153] On the other hand, the use of the residual mechanism can further utilize the global correlation to fine-tune the features and further eliminate the residual error.

[0154] In this way, the transformation and fine-tuning complement each other and jointly improve the accuracy of feature extraction.

[0155] Finally, as for the specific configuration content of the three-dimensional point cloud completion network at each level, in general, it can be learned that, based on in-depth analysis of the point feature correlation in the medium / fine-granularity complete point cloud, the application designs the medium / fine-granularity point cloud decoder by using the feature layer-by-layer fusion correction method, maximally utilizes the correlation of point features at each level of the medium / fine-granularity, and realizes high-precision point cloud completion.

[0156] Meanwhile, based on in-depth analysis of the point feature correlation in the incomplete point cloud and the coarse-granularity complete point cloud, the application carefully selects and improves the incomplete point cloud encoder and the coarse-granularity point cloud decoder backbone network, and pre-excludes the influence of improper module selection on the accuracy of point feature extraction, thereby improving the accuracy of point cloud completion while ensuring the stability of point cloud completion performance.

[0157] Therefore, compared with the existing completion network, the application avoids time-consuming neighborhood calculation throughout, and avoids using time-consuming attention mechanism in the incomplete point cloud encoding and coarse-grained point cloud decoding stages, so that the training, inference time and memory consumption are significantly reduced, and the user's requirement for cost reduction and quality improvement is well met.

[0158] The above is an introduction to the improved three-dimensional point cloud completion method provided by the application. In order to better implement the improved three-dimensional point cloud completion method provided by the application, the application also provides an improved three-dimensional point cloud completion device from the perspective of functional modules.

[0159] Reference Figure 8 , Figure 8 is a structural schematic diagram of the improved three-dimensional point cloud completion device of the application. In the application, the improved three-dimensional point cloud completion device 800 can specifically include the following structures:

[0160] The acquisition unit 801 is configured to acquire an incomplete point cloud to be completed, wherein the incomplete point cloud to be completed is specifically a three-dimensional point cloud;

[0161] The completion unit 802 is configured to input the incomplete point cloud to be completed into a pre-configured three-dimensional point cloud completion network, wherein the three-dimensional point cloud completion network sequentially includes an incomplete point cloud encoder based on global feature fusion point completion network, a coarse-grained point cloud decoder based on residual fine-tuning, a medium-grained point cloud decoder based on feature layer-by-layer fusion correction, and a fine-grained point cloud decoder based on feature layer-by-layer fusion correction. The incomplete point cloud encoder extracts incomplete point cloud features according to the input point cloud of the network, the coarse-grained point cloud decoder generates a coarse-grained complete point cloud according to the incomplete point cloud features, the medium-grained point cloud decoder generates a medium-grained complete point cloud according to the input point cloud of the network, the incomplete point cloud features and the coarse-grained complete point cloud, and the fine-grained point cloud decoder generates a fine-grained complete point cloud according to the input point cloud of the network, the incomplete point cloud features and the medium-grained complete point cloud;

[0162] The extraction unit 803 is configured to extract the fine-grained complete point cloud output by the three-dimensional point cloud completion network as a point cloud completion result.

[0163] In an exemplary embodiment, the incomplete point cloud encoder includes a one-dimensional convolution module , a max pooling module , a copy module , a splicing module , a one-dimensional convolution module , a max pooling module , a splicing module and a one-dimensional convolution module ;

[0164] The working process of the incomplete point cloud encoder is as follows:

[0165] 1.1) inputting a three-dimensional coordinate matrix of a residual point cloud into a one-dimensional convolution module , to obtain a three-dimensional low-level feature matrix, wherein the point cloud inputted into the network is denoted as , and the three-dimensional coordinate matrix of the residual point cloud is denoted as , wherein n is the number of points in the residual point cloud;

[0166] 1.2) inputting the three-dimensional low-level feature matrix into a max-pooling module , to obtain a three-dimensional low-level feature vector;

[0167] 1.3) replicating the three-dimensional low-level feature vector in the row direction for times by a replication module , and then splicing the replication result of the replication module with the three-dimensional low-level feature matrix in the column direction by a splicing module , to obtain a three-dimensional spliced feature matrix;

[0168] 1.4) inputting the three-dimensional spliced feature matrix into a one-dimensional convolution module , to obtain a three-dimensional high-level feature matrix;

[0169] 1.5) inputting the three-dimensional high-level feature matrix into a max-pooling module , to obtain a three-dimensional high-level feature vector;

[0170] 1.6) splicing the three-dimensional high-level feature vector with the three-dimensional low-level feature vector in the column direction by a splicing module , to obtain a three-dimensional spliced feature vector;

[0171] 1.7) inputting the three-dimensional spliced feature vector into a one-dimensional convolution module , to obtain a three-dimensional residual point cloud feature vector as the residual point cloud feature.

[0172] In another exemplary embodiment, the coarse-grained point cloud decoder comprises a transpose convolution module , a replication module , a splicing module , a one-dimensional convolution module , and a shape remodeling module ​​​​​​​​​​​​​​​​​, residual module and one-dimensional convolution module ;

[0173] The working process of the coarse-grained point cloud decoder is as follows:

[0174] 2.1) input the incomplete point cloud feature vector of dimension into the transpose convolution module to obtain the low-level transpose convolution feature matrix of dimension ;

[0175] 2.2) duplicate the incomplete point cloud feature vector of dimension in the row direction 128 times by the duplication module , and then splice the duplication result of the duplication module with the low-level transpose convolution feature matrix of dimension in the column direction by the splicing module to obtain the spliced feature matrix of dimension ;

[0176] 2.3) input the spliced feature matrix of dimension into the one-dimensional convolution module to obtain the fusion feature matrix of dimension ;

[0177] 2.4) reshape the fusion feature matrix of dimension by the shape reshaping module to obtain the reshaped feature matrix of dimension ;

[0178] 2.5) input the reshaped feature matrix of dimension into the residual module to obtain the reshaped residual feature matrix of dimension ;

[0179] 2.6) input the reshaped residual feature matrix of dimension into the one-dimensional convolution module to obtain the coarse-grained point cloud three-dimensional coordinate matrix of dimension , which is taken as the coarse-grained complete point cloud.

[0180] In another exemplary embodiment, the medium-grained point cloud decoder includes a merging module , a farthest point sampling module , a one-dimensional convolution module , a feature correction module , a one-dimensional convolution module , a one-dimensional convolution module , a one-dimensional convolution module , a duplication module , and a splicing module One-dimensional convolution module Feature correction module Copy module Splicing module One-dimensional convolution module Feature correction module Copy module Splicing module One-dimensional convolution module Shape remodeling module One-dimensional convolution module Copy module ;

[0181] The working process of the medium-grained point cloud decoder is as follows:

[0182] 3.1) The three-dimensional coordinate matrix of the incomplete point cloud in the three-dimensional space is combined with the three-dimensional coordinate matrix of the coarse-grained point cloud in the three-dimensional space by the combination module to obtain the three-dimensional coordinate matrix of the combined point cloud in the three-dimensional space;

[0183] 3.2) The three-dimensional coordinate matrix of the combined point cloud in the three-dimensional space is input into the farthest point sampling module to obtain the three-dimensional coordinate matrix of the sampled point cloud in the three-dimensional space, wherein 512 or 1024 is taken; 3.3) The three-dimensional coordinate matrix of the sampled point cloud in the three-dimensional space is input into the one-dimensional convolution module to obtain the initial feature matrix in the three-dimensional space;

[0184] 3.4) The initial feature matrix in the three-dimensional space is input into the feature correction module to obtain the initial corrected feature matrix in the three-dimensional space; 3.5) The incomplete point cloud feature vector in the three-dimensional space is input into the one-dimensional convolution module

[0185] to obtain the medium-layer incomplete point cloud feature vector in the three-dimensional space, and then the medium-layer incomplete point cloud feature vector in the three-dimensional space is input into the one-dimensional convolution module to obtain the low-layer incomplete point cloud feature vector in the three-dimensional space, and then the low-layer incomplete point cloud feature vector in the three-dimensional space is input into the one-dimensional convolution module to obtain the initial incomplete point cloud feature vector in the three-dimensional space;

[0186] ​​​​​​​​​​​​​​​

[0187] 3.6) by the replication module replicate the dimensional initial incomplete point cloud feature vector in the row direction times, and then by the splicing module splice the replication result of the replication module with the dimensional initial correction feature matrix in the column direction to obtain a dimensional low-level spliced feature matrix;

[0188] 3.7) input the dimensional low-level spliced feature matrix into a one-dimensional convolution module to obtain a dimensional low-level feature matrix;

[0189] 3.8) input the dimensional low-level feature matrix into a feature correction module to obtain a dimensional low-level correction feature matrix;

[0190] 3.9) by the replication module replicate the dimensional low-level incomplete point cloud feature vector in the row direction times, and then by the splicing module splice the replication result of the replication module with the dimensional low-level correction feature matrix in the column direction to obtain a dimensional middle-level spliced feature matrix;

[0191] 3.10) input the dimensional middle-level spliced feature matrix into a one-dimensional convolution module to obtain a dimensional middle-level feature matrix;

[0192] 3.11) input the dimensional middle-level feature matrix into a feature correction module to obtain a dimensional middle-level correction feature matrix;

[0193] 3.12) by the replication module replicate the dimensional middle-level incomplete point cloud feature vector in the row direction times, and then by the splicing module splice the replication result of the replication module with the dimensional middle-level correction feature matrix in the column direction to obtain a dimensional high-level spliced feature matrix;

[0194] 3.13) input the High-level concatenated feature matrix input to one-dimensional convolution module ,get High-level feature matrix;

[0195] 3.14) will High-dimensional feature matrix input shape reshaping module ,get Reshape the feature matrix in 3D;

[0196] 3.15) will 3D reshaping feature matrix input one-dimensional convolution module This yields the three-dimensional incremental coordinates of 2048 points;

[0197] 3.16) By the copy module Will Copying the 3D coordinate matrix of the 3D sampled point cloud in the row direction This is repeated and added to the three-dimensional incremental coordinates of 2048 points to obtain... The three-dimensional coordinate matrix of the medium-grained point cloud is used as the medium-grained complete point cloud.

[0198] In yet another exemplary embodiment, the fine-grained point cloud decoder includes a merging module. Farthest point sampling module One-dimensional convolution module Feature correction module One-dimensional convolution module One-dimensional convolution module One-dimensional convolution module Copy module splicing module One-dimensional convolution module Feature correction module Copy module splicing module One-dimensional convolution module Feature correction module Copy module splicing module One-dimensional convolution module Shape reshaping module One-dimensional convolution module and copy module ;

[0199] The working process of a fine-grained point cloud decoder includes:

[0200] 4.1) By the merging module Will 3D coordinate matrix of defect cloud and The three-dimensional coordinate matrix of the merged point cloud is obtained The three-dimensional coordinate matrix of the merged point cloud is obtained

[0201] 4.2) The three-dimensional coordinate matrix of the merged point cloud is input into the farthest point sampling module The three-dimensional coordinate matrix of the sampled point cloud is obtained

[0202] 4.3) The three-dimensional coordinate matrix of the sampled point cloud is input into the one-dimensional convolution module The initial feature matrix is obtained

[0203] 4.4) The initial feature matrix is input into the feature correction module The initial corrected feature matrix is obtained

[0204] 4.5) The incomplete point cloud feature vector is input into the one-dimensional convolution module The middle layer incomplete point cloud feature vector is obtained, and the middle layer incomplete point cloud feature vector is input into the one-dimensional convolution module The low layer incomplete point cloud feature vector is obtained, and the low layer incomplete point cloud feature vector is input into the one-dimensional convolution module The initial incomplete point cloud feature vector is obtained 4.6) The initial incomplete point cloud feature vector is copied in the row direction by the copy module

[0205] 4.6) The initial incomplete point cloud feature vector is copied in the row direction by the copy module The copy result of the copy module is spliced with the initial corrected feature matrix in the column direction by the splicing module The low layer splicing feature matrix is obtained

[0206] 4.7) The low layer splicing feature matrix is input into the one-dimensional convolution module The low layer feature matrix is obtained

[0207] 4.8) The low layer feature matrix is input into the feature correction module The low layer corrected feature matrix is obtained ​​​​​​​​​​​​​​​​​​​​

[0208] 4.9) Copy the 3D low-level incomplete point cloud feature vector in the row direction for 3 times, and then splice the copied results with the 3D low-level correction feature matrix in the column direction to obtain a 3D middle-level spliced feature matrix. Copy the 3D low-level incomplete point cloud feature vector in the row direction for 3 times, and then splice the copied results with the 3D low-level correction feature matrix in the column direction to obtain a 3D middle-level spliced feature matrix. Copy the 3D low-level incomplete point cloud feature vector in the row direction for 3 times, and then splice the copied results with the 3D low-level correction feature matrix in the column direction to obtain a 3D middle-level spliced feature matrix. Copy the 3D low-level incomplete point cloud feature vector in the row direction for 3 times, and then splice the copied results with the 3D low-level correction feature matrix in the column direction to obtain a 3D middle-level spliced feature matrix. 4.10) Input the 3D middle-level spliced feature matrix into a one-dimensional convolution module to obtain a 3D middle-level feature matrix.

[0209] 4.10) Input the 3D middle-level spliced feature matrix into a one-dimensional convolution module to obtain a 3D middle-level feature matrix. 4.10) Input the 3D middle-level spliced feature matrix into a one-dimensional convolution module to obtain a 3D middle-level feature matrix.

[0210] 4.11) Input the 3D middle-level feature matrix into a feature correction module to obtain a 3D middle-level correction feature matrix. 4.11) Input the 3D middle-level feature matrix into a feature correction module to obtain a 3D middle-level correction feature matrix. 4.11) Input the 3D middle-level feature matrix into a feature correction module to obtain a 3D middle-level correction feature matrix.

[0211] 4.12) Copy the 3D middle-level incomplete point cloud feature vector in the row direction for 3 times, and then splice the copied results with the 3D middle-level correction feature matrix in the column direction to obtain a 3D high-level spliced feature matrix. 4.12) Copy the 3D middle-level incomplete point cloud feature vector in the row direction for 3 times, and then splice the copied results with the 3D middle-level correction feature matrix in the column direction to obtain a 3D high-level spliced feature matrix. 4.12) Copy the 3D middle-level incomplete point cloud feature vector in the row direction for 3 times, and then splice the copied results with the 3D middle-level correction feature matrix in the column direction to obtain a 3D high-level spliced feature matrix. 4.12) Copy the 3D middle-level incomplete point cloud feature vector in the row direction for 3 times, and then splice the copied results with the 3D middle-level correction feature matrix in the column direction to obtain a 3D high-level spliced feature matrix.

[0212] 4.13) Input the 3D high-level spliced feature matrix into a one-dimensional convolution module to obtain a 3D high-level feature matrix. 4.13) Input the 3D high-level spliced feature matrix into a one-dimensional convolution module to obtain a 3D high-level feature matrix. 4.13) Input the 3D high-level spliced feature matrix into a one-dimensional convolution module to obtain a 3D high-level feature matrix.

[0213] 4.14) Input the 3D high-level feature matrix into a shape remodeling module to obtain a 3D remodeled feature matrix. 4.14) Input the 3D high-level feature matrix into a shape remodeling module to obtain a 3D remodeled feature matrix.

[0214] 4.15) Input the 3D remodeled feature matrix into a one-dimensional convolution module to obtain a three-dimensional incremental coordinate of 16384 points. 4.15) Input the 3D remodeled feature matrix into a one-dimensional convolution module to obtain a three-dimensional incremental coordinate of 16384 points.

[0215] 4.16) Copy the 3D sampled point cloud three-dimensional coordinate matrix in the row direction for 3 times, and then add it to the three-dimensional incremental coordinate of 16384 points to obtain a three-dimensional coordinate matrix of 49152 points. 4.16) Copy the 3D sampled point cloud three-dimensional coordinate matrix in the row direction for 3 times, and then add it to the three-dimensional incremental coordinate of 16384 points to obtain a three-dimensional coordinate matrix of 49152 points. 4.16) Copy the 3D sampled point cloud three-dimensional coordinate matrix in the row direction for 3 times, and then add it to the three-dimensional incremental coordinate of 16384 points to obtain a three-dimensional coordinate matrix of 49152 points.​​​​​​​​​​​​​​​The fine-grained point cloud three-dimensional coordinate matrix is taken as a fine-grained complete point cloud.

[0216] In yet another exemplary embodiment, the module structure of the feature correction module in both the medium-grained point cloud decoder and the fine-grained point cloud decoder includes a self-attention sub-module and a residual sub-module .

[0217] The working process of the feature correction module is as follows:

[0218] 5.1) inputting the dimensional original feature matrix into the self-attention sub-module for transformation to obtain dimensional transformed features;

[0219] 5.2) inputting the dimensional transformed features into the residual sub-module for fine-tuning to obtain a dimensional corrected feature matrix.

[0220] In yet another exemplary embodiment, the apparatus further includes a training unit 804 configured to:

[0221] obtain a partial point cloud sample and its corresponding fine-grained complete point cloud ground truth;

[0222] train the three-dimensional point cloud completion network based on the partial point cloud sample and its corresponding fine-grained complete point cloud ground truth.

[0223] The present application also provides a processing device from the hardware structure perspective. Referring to Figure 9 , Figure 9 a structural schematic diagram of the processing device of the present application is shown. Specifically, the processing device of the present application can include a processor 901, a memory 902, and an input / output device 903. The processor 901 is configured to execute a computer program stored in the memory 902 to implement the steps of the improved three-dimensional point cloud completion method in the corresponding embodiments; or the processor 901 is configured to execute a computer program stored in the memory 902 to implement the functions of the units in the corresponding embodiments. The memory 902 is configured to store the computer program required by the improved three-dimensional point cloud completion method in the corresponding embodiments. Figure 1 Figure 8 Figure 1

[0224] ​​​For example, the computer program can be divided into one or more modules / units, one or more modules / units are stored in the memory 902 and executed by the processor 901 to complete the present application. One or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the computer device.

[0225] The processing device can include, but is not limited to, the processor 901, the memory 902, and the input / output device 903. Those skilled in the art can understand that the diagram is only an example of the processing device and does not constitute a limitation on the processing device, which can include more or fewer components than the diagram, or combine certain components, or different components, for example, the processing device can also include a network access device, a bus, etc., and the processor 901, the memory 902, and the input / output device 903 are connected through the bus.

[0226] The processor 901 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or other conventional processors, and the processor is the control center of the processing device, which connects various parts of the device through various interfaces and lines.

[0227] The memory 902 can be used to store computer programs and / or modules, and the processor 901 realizes various functions of the computer device by running or executing the computer programs and / or modules stored in the memory 902, and calling the data stored in the memory 902. The memory 902 mainly includes a storage program area and a storage data area, wherein the storage program area can store operating systems, application programs required to realize at least one function, etc.; the storage data area can store data created during use of the processing device, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory card, a smart media card (SMC), a secure digital (SD) card, a flash card, a memory device with at least one disk, a flash memory device, or other volatile solid-state memory device.

[0228] The processor 901 is used to execute the computer program stored in the memory 902, and can specifically implement the following functions:

[0229] Obtain a to-be-completed incomplete point cloud, wherein the to-be-completed incomplete point cloud is specifically a three-dimensional point cloud;

[0230] Input the to-be-completed incomplete point cloud into a pre-configured three-dimensional point cloud completion network, wherein the three-dimensional point cloud completion network sequentially includes an incomplete point cloud encoder based on a global feature fusion point completion network, a coarse-grained point cloud decoder based on residual fine-tuning, a medium-grained point cloud decoder based on feature layer-by-layer fusion correction, and a fine-grained point cloud decoder based on feature layer-by-layer fusion correction, the incomplete point cloud encoder extracts incomplete point cloud features according to the input point cloud of the network, the coarse-grained point cloud decoder generates a coarse-grained complete point cloud according to the incomplete point cloud features, the medium-grained point cloud decoder generates a medium-grained complete point cloud according to the input point cloud of the network, the incomplete point cloud features, and the coarse-grained complete point cloud, and the fine-grained point cloud decoder generates a fine-grained complete point cloud according to the input point cloud of the network, the incomplete point cloud features, and the medium-grained complete point cloud;

[0231] Extract the fine-grained complete point cloud output by the three-dimensional point cloud completion network as a point cloud completion result.

[0232] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the improved three-dimensional point cloud completion device, the processing device and the corresponding units thereof described above can be referred to as Figure 1 The description of the improved three-dimensional point cloud completion method in the corresponding embodiments will not be repeated here.

[0233] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by related hardware controlled by the instructions, which can be stored in a computer readable storage medium and loaded and executed by a processor.

[0234] Therefore, the present application provides a computer readable storage medium, which stores a plurality of instructions capable of being loaded by a processor to execute the improved three-dimensional point cloud completion method of the present application as Figure 1 The steps of the improved three-dimensional point cloud completion method in the corresponding embodiments can be specifically implemented by referring to the above description. Figure 1 The description of the improved three-dimensional point cloud completion method in the corresponding embodiments will not be repeated here.

[0235] The computer readable storage medium can include a read only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0236] Due to the instructions stored in the computer readable storage medium, the improved three-dimensional point cloud completion method provided in the present application can be implemented as Figure 1 The steps of the improved three-dimensional point cloud completion method in the corresponding embodiments, therefore, the present application can be implemented as Figure 1 Figure 1 The beneficial effects that the improved three-dimensional point cloud completion method in the corresponding embodiments can achieve are described in detail in the foregoing description, which will not be repeated here.

[0237] The improved three-dimensional point cloud completion method, device, processing equipment and computer readable storage medium provided in the present application are described in detail above, and the principles and implementation modes of the present application are described by applying specific examples. The above description of the embodiments is only used to help understand the core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed; in view of the above, the content of the specification should not be understood as a limitation of the present application.

Claims

1. An improved method for completing three-dimensional point clouds, characterized in that, The method includes: Obtain the defect cloud to be completed, wherein the defect cloud to be completed is specifically a three-dimensional point cloud; The incomplete point cloud to be completed is input into a pre-configured 3D point cloud completion network. The 3D point cloud completion network sequentially includes an incomplete point cloud encoder based on a global feature fusion point completion network, a coarse-grained point cloud decoder based on residual fine-tuning, a medium-grained point cloud decoder based on feature layer-by-layer fusion correction, and a fine-grained point cloud decoder based on the feature layer-by-layer fusion correction. The incomplete point cloud encoder extracts incomplete point cloud features based on the input point cloud of the network. The coarse-grained point cloud decoder generates a coarse-grained complete point cloud based on the incomplete point cloud features. The medium-grained point cloud decoder generates a medium-grained complete point cloud based on the input point cloud of the network, the incomplete point cloud features, and the coarse-grained complete point cloud. The fine-grained point cloud decoder generates a fine-grained complete point cloud based on the input point cloud of the network, the incomplete point cloud features, and the medium-grained complete point cloud. Extract the fine-grained complete point cloud output by the 3D point cloud completion network as the point cloud completion result; The defect cloud encoder includes a one-dimensional convolution module. Maximum pooling module Copy module splicing module One-dimensional convolution module Maximum pooling module splicing module and one-dimensional convolutional modules ; The working process of the defective cloud encoder is as follows: Will The three-dimensional coordinate matrix of the defect cloud is input into the one-dimensional convolution module. ,get The low-level feature matrix is ​​denoted as the input point cloud of the network. The three-dimensional coordinate matrix of the defect cloud. It is the number of points in the defective cloud; The The low-level feature matrix is ​​input into the max pooling module. ,get Low-dimensional feature vectors; By the copy module The The low-level feature vectors are copied in the row direction. Next, the splicing module The copy module The replication result, and the... The low-level feature matrices are concatenated along the column direction to obtain... 3D concatenation of feature matrices; The The 3D concatenated feature matrix is ​​input into the 1D convolution module. ,get 3D high-level feature matrix; The The high-level feature matrix is ​​input into the max pooling module. ,get High-dimensional feature vectors; The splicing module The High-level feature vectors, and the aforementioned The low-level feature vectors are concatenated in the column direction to obtain 3D concatenation of feature vectors; The The concatenated feature vectors are input into the one-dimensional convolution module. ,get The feature vector of the residual defect cloud is used as the feature of the residual defect cloud.

2. The method according to claim 1, characterized in that, The coarse-grained point cloud decoder includes a transposed convolution module. Copy module splicing module One-dimensional convolution module Shape reshaping module Residual module and one-dimensional convolutional modules ; The working process of the coarse-grained point cloud decoder includes: The 3D residual cloud feature vector input transpose convolution module ,get Low-level transposed convolutional feature matrix; By the copy module The The feature vector of the defective cloud is copied 128 times in the row direction, and then the splicing module is used. The copy module The replication result, and the... The low-level transposed convolutional feature matrices are concatenated along the column direction to obtain... 3D concatenation of feature matrices; The The 3D concatenated feature matrix is ​​input into the 1D convolution module. ,get 3D fusion feature matrix; By the shape reshaping module Regarding the Shape reshaping is performed on the 3D fusion feature matrix to obtain Reshape the feature matrix in 3D; The The feature matrix is ​​reshaped and input into the residual module. ,get Reconstruct the residual feature matrix in dimensionality; The The 3D reconstructed residual feature matrix is ​​input into the 1D convolution module. ,get The three-dimensional coordinate matrix of the coarse-grained point cloud is used as the coarse-grained complete point cloud.

3. The method according to claim 2, characterized in that, The medium-granularity point cloud decoder includes a merging module. Farthest point sampling module One-dimensional convolution module Feature correction module One-dimensional convolution module One-dimensional convolution module One-dimensional convolution module Copy module splicing module One-dimensional convolution module Feature correction module Copy module splicing module One-dimensional convolution module Feature correction module Copy module splicing module One-dimensional convolution module Shape reshaping module One-dimensional convolution module and copy module ; The working process of the medium-granularity point cloud decoder includes: By the merging module The The three-dimensional coordinate matrix of the defect cloud and the said The three-dimensional coordinate matrices of coarse-grained point clouds are merged to obtain... 3D coordinate matrix of merged point cloud; The The three-dimensional coordinate matrix of the merged point cloud is input to the farthest point sampling module. ,get The 3D coordinate matrix of the sampled point cloud, where, Choose 512 or 1024; The The 3D coordinate matrix of the sampled point cloud is input into the 1D convolution module. ,get dimensional initial feature matrix; The The initial feature matrix is ​​input into the feature correction module. ,get Initialized corrected feature matrix; Will The feature vector of the defect cloud is input into the one-dimensional convolution module. ,get The feature vector of the residual cloud in the middle layer of the dimension, and then the... The feature vector of the residual cloud in the middle layer is input into the one-dimensional convolution module. ,get The low-dimensional residual cloud feature vector, and then the... The low-level residual cloud feature vector is input into the one-dimensional convolution module. ,get Initial residual feature vector; By the copy module The The initial residual feature vector of the dimensional cloud is copied in the row direction. Next, the splicing module The copy module The replication results are consistent with the above. The initial modified feature matrix is ​​concatenated along the column direction to obtain... Low-level concatenated feature matrix; The The low-level concatenated feature matrix is ​​input into the one-dimensional convolutional module. ,get Low-level feature matrix; The The low-level feature matrix is ​​input into the feature correction module. ,get Low-level modified feature matrix; By the copy module The The low-dimensional residual cloud feature vector is copied in the row direction. Next, the splicing module The copy module The replication results are consistent with the above. The lower-level modified feature matrices are concatenated along the column direction to obtain... Mid-level concatenation of feature matrices; The The feature matrix concatenated in the middle layer is input into the one-dimensional convolution module. ,get Mid-level feature matrix; The The feature correction module inputs the mid-level feature matrix. ,get Mid-level modified feature matrix; By the copy module The The feature vector of the residual cloud in the middle layer is copied in the row direction. Next, the splicing module The copy module The replication results are consistent with the above. The modified feature matrices in the middle layer are concatenated along the column direction to obtain... High-level splicing feature matrix; The The high-level concatenated feature matrix is ​​input into the one-dimensional convolution module. ,get 3D high-level feature matrix; The The high-dimensional feature matrix is ​​input to the shape reshaping module. ,get Reshape the feature matrix in 3D; The The 3D reshaped feature matrix is ​​input into the 1D convolution module. This yields the three-dimensional incremental coordinates of 2048 points; By the copy module The Copying the 3D coordinate matrix of the 3D sampled point cloud in the row direction This is repeated and then added to the three-dimensional incremental coordinates of the 2048 points to obtain... The three-dimensional coordinate matrix of the medium-grained point cloud is used as the medium-grained complete point cloud.

4. The method according to claim 3, characterized in that, The fine-grained point cloud decoder includes a merging module. Farthest point sampling module One-dimensional convolution module Feature correction module One-dimensional convolution module One-dimensional convolution module One-dimensional convolution module Copy module splicing module One-dimensional convolution module Feature correction module Copy module splicing module One-dimensional convolution module Feature correction module Copy module splicing module One-dimensional convolution module Shape reshaping module One-dimensional convolution module and copy module ; The working process of the fine-grained point cloud decoder includes: By the merging module The The three-dimensional coordinate matrix of the defect cloud and the said The 3D coordinate matrices of the mid-grained point clouds are merged to obtain... 3D coordinate matrix of merged point cloud; The The three-dimensional coordinate matrix of the merged point cloud is input to the farthest point sampling module. ,get 3D coordinate matrix of sampled point cloud; The The 3D coordinate matrix of the sampled point cloud is input into the 1D convolution module. ,get dimensional initial feature matrix; The The initial feature matrix is ​​input into the feature correction module. ,get Initialized corrected feature matrix; Will The feature vector of the defect cloud is input into the one-dimensional convolution module. ,get The feature vector of the residual cloud in the middle layer of the dimension, and then the... The feature vector of the residual cloud in the middle layer is input into the one-dimensional convolution module. ,get The low-dimensional residual cloud feature vector, and then the... The low-level residual cloud feature vector is input into the one-dimensional convolution module. ,get Initial residual feature vector; By the copy module The The initial residual feature vector of the dimensional cloud is copied in the row direction. Next, the splicing module The copy module The replication results are consistent with the above. The initial modified feature matrix is ​​concatenated along the column direction to obtain... Low-level concatenated feature matrix; The The low-level concatenated feature matrix is ​​input into the one-dimensional convolutional module. ,get Low-level feature matrix; The The low-level feature matrix is ​​input into the feature correction module. ,get Low-level modified feature matrix; By the copy module The The low-dimensional residual cloud feature vector is copied in the row direction. Next, the splicing module The copy module The replication results are consistent with the above. The lower-level modified feature matrices are concatenated along the column direction to obtain... Mid-level concatenation of feature matrices; The The feature matrix concatenated in the middle layer is input into the one-dimensional convolution module. ,get Mid-level feature matrix; The The feature correction module inputs the mid-level feature matrix. ,get Mid-level modified feature matrix; By the copy module The The feature vector of the residual cloud in the middle layer is copied in the row direction. Next, the splicing module The copy module The replication results are consistent with the above. The modified feature matrices in the middle layer are concatenated along the column direction to obtain... High-level splicing feature matrix; The The high-level concatenated feature matrix is ​​input into the one-dimensional convolution module. ,get 3D high-level feature matrix; The The high-dimensional feature matrix is ​​input to the shape reshaping module. ,get Reshape the feature matrix in 3D; The The 3D reshaped feature matrix is ​​input into the 1D convolution module. This yields 16,384 points of incremental three-dimensional coordinates. By the copy module The Copying the 3D coordinate matrix of the 3D sampled point cloud in the row direction This is repeated and then added to the three-dimensional incremental coordinates of the 16384 points to obtain... The three-dimensional coordinate matrix of the fine-grained point cloud is used as the fine-grained complete point cloud.

5. The method according to claim 4, characterized in that, The feature correction module in both the medium-granularity point cloud decoder and the fine-granularity point cloud decoder includes a self-attention submodule. and residual submodule ; The working process of the feature correction module includes: Will The original feature matrix is ​​input into the self-attention submodule. Perform the transformation to obtain 3D transformation characteristics; The 3D transformation feature input to the residual submodule Fine-tuning was performed to obtain... Repair positive characteristic matrix.

6. The method according to claim 1, characterized in that, The method further includes: Obtain incomplete point cloud samples and their corresponding fine-grained complete point cloud ground truth values; The 3D point cloud completion network is trained based on the incomplete point cloud samples and their corresponding fine-grained complete point cloud ground truth values.

7. An improved three-dimensional point cloud completion device, characterized in that, The device includes: An acquisition unit is used to acquire a defect cloud to be completed, wherein the defect cloud to be completed is specifically a three-dimensional point cloud; The completion unit is used to input the incomplete point cloud to be completed into a pre-configured 3D point cloud completion network. The 3D point cloud completion network sequentially includes an incomplete point cloud encoder based on a global feature fusion point completion network, a coarse-grained point cloud decoder based on residual fine-tuning, a medium-grained point cloud decoder based on feature layer-by-layer fusion correction, and a fine-grained point cloud decoder based on the feature layer-by-layer fusion correction. The incomplete point cloud encoder extracts incomplete point cloud features based on the point cloud input to the network. The coarse-grained point cloud decoder generates a coarse-grained complete point cloud based on the incomplete point cloud features. The medium-grained point cloud decoder generates a medium-grained complete point cloud based on the point cloud input to the network, the incomplete point cloud features, and the coarse-grained complete point cloud. The fine-grained point cloud decoder generates a fine-grained complete point cloud based on the point cloud input to the network, the incomplete point cloud features, and the medium-grained complete point cloud. The extraction unit is used to extract the fine-grained complete point cloud output by the three-dimensional point cloud completion network as the point cloud completion result; The defect cloud encoder includes a one-dimensional convolution module. Maximum pooling module Copy module splicing module One-dimensional convolution module Maximum pooling module splicing module and one-dimensional convolutional modules ; The working process of the defective cloud encoder is as follows: Will The three-dimensional coordinate matrix of the defect cloud is input into the one-dimensional convolution module. ,get The low-level feature matrix is ​​denoted as the input point cloud of the network. The three-dimensional coordinate matrix of the defect cloud. It is the number of points in the defective cloud; The The low-level feature matrix is ​​input into the max pooling module. ,get Low-dimensional feature vectors; By the copy module The The low-level feature vectors are copied in the row direction. Next, the splicing module The copy module The replication result, and the... The low-level feature matrices are concatenated along the column direction to obtain... 3D concatenation of feature matrices; The The 3D concatenated feature matrix is ​​input into the 1D convolution module. ,get 3D high-level feature matrix; The The high-level feature matrix is ​​input into the max pooling module. ,get High-dimensional feature vectors; The splicing module The High-level feature vectors, and the aforementioned The low-level feature vectors are concatenated in the column direction to obtain 3D concatenation of feature vectors; The The concatenated feature vectors are input into the one-dimensional convolution module. ,get The feature vector of the residual defect cloud is used as the feature of the residual defect cloud.

8. A processing apparatus, characterized in that, The method includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the method as described in any one of claims 1 to 6 when it invokes the computer program in the memory.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to perform the method of any one of claims 1 to 6.

Images