Point cloud data completion model training method and device, point cloud data completion method and device, electronic equipment and computer readable storage medium
By introducing feature enhancement modules in the point cloud data completion model, the feature information of incomplete and complete point cloud data is used as supervision signals to enhance the model training effect, and the insufficient feature representation ability of point cloud data completion in the existing technology is solved. The problem has improved the quality and accuracy of the complete data.
Patent Information
- Application Number
- CN202410965145.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-11-12
AI Technical Summary
The existing point cloud data completion model has poor feature representation capabilities in autonomous driving scenarios and is difficult to meet data quality requirements, resulting in uneven point cloud data quality after completion processing.
The training method of point cloud data completion model is adopted to obtain incomplete and complete point cloud data, and the feature enhancement module is used to extract feature information, which is deeply integrated into the model training process as an additional supervised signal, including the category compilation module and feature information. Memory module enhances the model's ability to characterize point cloud data characteristics.
The accuracy and effect of the point cloud data completion model is improved, the model's ability to characterize point cloud data characteristics is enhanced, and the cost of point cloud data annotation is reduced.
Smart Images

Figure CN118941889A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of point cloud data completion, and in particular to a training method and device for a point cloud data completion model, a point cloud data completion method and device, an electronic device, and a computer-readable storage medium. Background Art
[0002] With the continuous development of autonomous driving perception technology, deep learning technology has become an important implementation algorithm in autonomous driving perception. LiDAR is one of the important sensors in autonomous driving scenarios, and the point cloud data collected by LiDAR is also an important data source for training deep learning models.
[0003] However, the cost of labeling point cloud data currently used for model training is high, while the feature representation capability of the algorithm model for completing point cloud data is poor. The quality of the point cloud data after completion is uneven, making it difficult to meet the data quality requirements for training point cloud data perception models in autonomous driving scenarios. Summary of the invention
[0004] The embodiments of the present application provide a training method and device for a point cloud data completion model, a point cloud data completion method and device, an electronic device and a computer-readable storage medium to improve the point cloud data completion model's ability to characterize point cloud data features, thereby improving the accuracy and effect of point cloud data completion.
[0005] The present application embodiment adopts the following technical solutions:
[0006] In a first aspect, an embodiment of the present application provides a method for training a point cloud data completion model, wherein the method for training the point cloud data completion model comprises:
[0007] Acquire training sample data of a point cloud data completion model, wherein the training sample data includes incomplete point cloud data and corresponding complete point cloud data;
[0008] According to the incomplete point cloud data and the complete point cloud data, a feature enhancement module of the point cloud data completion model is used to perform feature enhancement processing to obtain feature enhancement information of the point cloud data;
[0009] The point cloud data completion model is trained according to the training sample data and the feature enhancement information of the point cloud data to obtain a trained point cloud data completion model.
[0010] Optionally, the feature enhancement module includes a category compilation module, and the feature enhancement processing is performed using the feature enhancement module of the point cloud data completion model according to the incomplete point cloud data and the complete point cloud data, and the feature enhancement information of the point cloud data obtained includes:
[0011] Processing the incomplete point cloud data using the category compilation module to obtain feature information of the incomplete point cloud data output by the category compilation module;
[0012] The feature information of the incomplete point cloud data output by the category compilation module is used as first feature enhancement information of the point cloud data.
[0013] Optionally, the feature enhancement module further includes a feature information memory module, and the feature enhancement processing is performed using the feature enhancement module of the point cloud data completion model according to the incomplete point cloud data and the complete point cloud data, and the feature enhancement information of the point cloud data obtained includes:
[0014] storing the feature information of the incomplete point cloud data output by the category compilation module into the feature information memory module;
[0015] The feature information memory module is used to process the feature information of the complete point cloud data and the feature information of the incomplete point cloud to obtain second feature enhancement information of the point cloud data.
[0016] Optionally, the using the feature information memory module to process the feature information of the complete point cloud data and the feature information of the incomplete point cloud to obtain the second feature enhancement information of the point cloud data includes:
[0017] Using a preset clustering strategy to perform clustering processing on the feature information of the complete point cloud data and the feature information of the incomplete point cloud, to obtain a clustering result of the point cloud data;
[0018] The clustering result of the point cloud data is used as second feature enhancement information of the point cloud data.
[0019] Optionally, the step of training the point cloud data completion model according to the training sample data and the feature enhancement information of the point cloud data to obtain the trained point cloud data completion model comprises:
[0020] The point cloud data completion model is trained according to the training sample data, the first feature enhancement information and the second feature enhancement information of the point cloud data to obtain a trained point cloud data completion model.
[0021] Optionally, the step of training the point cloud data completion model according to the training sample data and the feature enhancement information of the point cloud data to obtain the trained point cloud data completion model comprises:
[0022] Processing the incomplete point cloud data using an encoder of the point cloud data completion model to obtain encoding features output by the encoder;
[0023] Fusing the feature enhancement information with the encoding feature output by the encoder to obtain a fused feature;
[0024] Inputting the fused features into a decoder of the point cloud data completion model to obtain decoding features of the decoder;
[0025] Upsampling the decoded features of the decoder to obtain upsampled point cloud data;
[0026] The loss value of the point cloud data completion model is determined according to the upsampled point cloud data and the corresponding complete point cloud data, and the loss value is used to update the parameters of the point cloud data completion model to obtain the trained point cloud data completion model.
[0027] In a second aspect, an embodiment of the present application further provides a point cloud data completion method, wherein the point cloud data completion method comprises:
[0028] Obtain incomplete point cloud data;
[0029] Processing the incomplete point cloud data using a point cloud data completion model to obtain complete point cloud data;
[0030] The point cloud data completion model is trained based on any of the training methods for the point cloud data completion model described above.
[0031] In a third aspect, an embodiment of the present application further provides a training device for a point cloud data completion model, wherein the training device for the point cloud data completion model comprises:
[0032] A first acquisition unit is used to acquire training sample data of a point cloud data completion model, wherein the training sample data includes incomplete point cloud data and corresponding complete point cloud data;
[0033] A feature enhancement processing unit, configured to perform feature enhancement processing using a feature enhancement module of the point cloud data completion model according to the incomplete point cloud data and the complete point cloud data, so as to obtain feature enhancement information of the point cloud data;
[0034] A training unit is used to train the point cloud data completion model according to the training sample data and the feature enhancement information of the point cloud data to obtain a trained point cloud data completion model.
[0035] In a fourth aspect, an embodiment of the present application further provides a point cloud data completion device, wherein the point cloud data completion device comprises:
[0036] A second acquisition unit, used for acquiring incomplete point cloud data;
[0037] A completion unit, used for processing the incomplete point cloud data using a point cloud data completion model to obtain complete point cloud data;
[0038] Wherein, the point cloud data completion model is trained based on the training device of the aforementioned point cloud data completion model.
[0039] In a fifth aspect, an embodiment of the present application further provides an electronic device, including:
[0040] Processor; and
[0041] A memory arranged to store computer executable instructions, which, when executed, cause the processor to execute any of the aforementioned training methods for the point cloud data completion model, or to execute the aforementioned point cloud data completion method.
[0042] In a sixth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple applications, the electronic device executes any of the aforementioned training methods for the point cloud data completion model, or executes the aforementioned point cloud data completion method.
[0043] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: the training method of the point cloud data completion model in the embodiments of the present application first obtains the training sample data of the point cloud data completion model, and the training sample data includes incomplete point cloud data and corresponding complete point cloud data; then the feature enhancement module of the point cloud data completion model is used to process the incomplete point cloud data and the complete point cloud data respectively to obtain the feature enhancement information of the point cloud data; finally, the point cloud data completion model is trained according to the feature enhancement information of the training sample data and the point cloud data to obtain the trained point cloud data completion model. The training method of the point cloud data completion model in the embodiments of the present application extracts feature enhancement information from the incomplete point cloud data and the complete point cloud data, and deeply integrates it into the training process of the point cloud data completion model as an additional supervisory signal, thereby improving the model's ability to characterize the features of the point cloud data without affecting the model processing speed and the amount of calculation, thereby improving the accuracy and effect of the subsequent model on the point cloud data completion. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0045] Figure 1 A flowchart of a training method for a point cloud data completion model in an embodiment of the present application;
[0046] Figure 2 A schematic diagram of a training process of a point cloud data completion model in an embodiment of the present application;
[0047] Figure 3 A schematic diagram of a process of a point cloud data completion method in an embodiment of the present application;
[0048] Figure 4 A schematic diagram of the structure of a training device for a point cloud data completion model in an embodiment of the present application;
[0049] Figure 5 This is a schematic diagram of the structure of a point cloud data completion device in an embodiment of the present application;
[0050] Figure 6 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION
[0051] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.
[0052] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.
[0053] Conventional point cloud data completion models usually first input incomplete point cloud data, which is then processed by network structures such as Encoder, Maxpool, and Decoder to finally obtain the completed point cloud data. In such a model structure, the connection between the incomplete point cloud data and the true value of the complete point cloud data is weak, and a strong supervision signal cannot be effectively formed to allow the deep network model to fit the two types of data. As a result, in the model inference stage, when an incomplete point cloud data is input, the output result is often incomplete and of low quality.
[0054] Based on this, the present application embodiment provides a training method for a point cloud data completion model, such as Figure 1 As shown, a flow chart of a training method for a point cloud data completion model in an embodiment of the present application is provided, and the training method for the point cloud data completion model at least includes the following steps S110 to S130:
[0055] Step S110, obtaining training sample data of the point cloud data completion model, wherein the training sample data includes incomplete point cloud data and corresponding complete point cloud data.
[0056] When training a point cloud data completion model, the embodiment of the present application needs to first obtain sample data for training the model. The training sample data specifically includes sample pairs consisting of incomplete point cloud data and corresponding complete point cloud data. Incomplete point cloud data refers to data with missing point clouds, which serves as the input for training the point cloud data completion model. The complete point cloud data is the true value corresponding to the incomplete point cloud data, and is used to provide a supervisory signal for model training. In addition to the original point cloud data, the complete point cloud data will also be annotated with some basic feature information in advance, such as the category of the point cloud data and other information. The specific method of collecting and annotating the training sample data can be flexibly determined by those skilled in the art in combination with the prior art, and is not specifically limited here.
[0057] Step S120, performing feature enhancement processing using a feature enhancement module of the point cloud data completion model according to the incomplete point cloud data and the complete point cloud data, to obtain feature enhancement information of the point cloud data.
[0058] Since the training process of the point cloud data completion model usually uses back-propagation for iterative training, the true value information as a supervisory signal will gradually weaken during this process. Therefore, relying solely on this supervisory signal for training will ultimately lead to poor model training results and make it difficult to meet the requirements of point cloud data completion.
[0059] Based on this, the embodiment of the present application adopts a feature enhancement processing strategy, and an additional feature enhancement module is set in the point cloud data completion model. The incomplete point cloud data and the complete point cloud data are processed by the additional feature enhancement module during the training phase, thereby providing additional and stronger supervision signals for the training of the point cloud data completion model. The enhancement processing is mainly to further capture other feature information in the incomplete point cloud data and the complete point cloud data, thereby enhancing the connection between the incomplete point cloud data and the corresponding complete point cloud data true value, thereby improving the effect of model training.
[0060] Step S130: training the point cloud data completion model according to the training sample data and the feature enhancement information of the point cloud data to obtain a trained point cloud data completion model.
[0061] After obtaining the feature enhancement information of the point cloud data, the feature enhancement information of the point cloud data is used as an additional supervision signal in the model training process, and combined with the training sample data to train the point cloud data completion model, thereby compensating for the problem that the true value information is gradually weakened during the training process.
[0062] It should be noted that the basic network structure of the above-mentioned point cloud data completion model can be implemented based on the network structure of the existing training point cloud data completion model. On the basis of the existing network structure, an additional feature enhancement module of the present application is set to enhance the original network structure's ability to characterize point cloud data features.
[0063] The training method of the point cloud data completion model in the embodiment of the present application extracts feature enhancement information from incomplete point cloud data and complete point cloud data, and deeply integrates it into the training process of the point cloud data completion model as an additional supervisory signal. Without affecting the processing speed and computational complexity of the model, the model's ability to characterize the features of point cloud data is improved, thereby improving the accuracy and effect of subsequent model completion of point cloud data.
[0064] In some embodiments of the present application, the feature enhancement module includes a category compilation module, and the feature enhancement processing is performed using the feature enhancement module of the point cloud data completion model based on the incomplete point cloud data and the complete point cloud data to obtain feature enhancement information of the point cloud data, including: using the category compilation module to process the incomplete point cloud data to obtain feature information of the incomplete point cloud data output by the category compilation module; and using the feature information of the incomplete point cloud data output by the category compilation module as the first feature enhancement information of the point cloud data.
[0065] The feature enhancement module provided in the embodiment of the present application may include a category compilation module. The main function of the category compilation module is to extract category features and other information of incomplete point cloud data. In the embodiment of the present application, the category compilation module can be used to perform additional category compilation processing on the incomplete point cloud data, thereby obtaining the category and other feature information of the incomplete point cloud data as the first feature enhancement information. The first feature enhancement information is applied as an additional supervisory signal to the training process of the point cloud data completion model.
[0066] The category compilation module of the embodiment of the present application can be implemented by using algorithms such as SVM (Support Vector Machine). The specific form of the category compilation algorithm can be flexibly selected by technicians in this field according to actual needs, and no specific limitation is made here.
[0067] In some embodiments of the present application, the feature enhancement module also includes a feature information memory module, and the feature enhancement processing is performed using the feature enhancement module of the point cloud data completion model based on the incomplete point cloud data and the complete point cloud data to obtain the feature enhancement information of the point cloud data, including: storing the feature information of the incomplete point cloud data output by the category compilation module in the feature information memory module; and using the feature information memory module to process the feature information of the complete point cloud data and the feature information of the incomplete point cloud to obtain second feature enhancement information of the point cloud data.
[0068] Considering that the complete true value of point cloud data is generally only labeled with large category labels, and no sub-category labels are labeled, the existing category feature information is not sufficient for the training accuracy requirements of the model. Based on this, the feature enhancement module of the embodiment of the present application also includes a feature information memory module. The main function of the feature information memory module is to provide more refined category feature information of point cloud data and perform maintenance and updates. For example, point cloud data with the same or similar features can be clustered into a refined sub-category to provide the model with more correlation information between point cloud features.
[0069] The feature information memory module of the embodiment of the present application can process the feature information of all complete and incomplete point clouds at the same time, so as to capture the correlation information between these features as the second feature enhancement information, and apply the second feature enhancement information as an additional supervisory signal to the training process of the point cloud data completion model. Of course, the specific implementation of the feature information memory module can be flexibly set by those skilled in the art according to actual needs, and is not specifically limited here.
[0070] In some embodiments of the present application, the feature information memory module is used to process the feature information of the complete point cloud data and the feature information of the incomplete point cloud to obtain the second feature enhancement information of the point cloud data, including: using a preset clustering strategy to cluster the feature information of the complete point cloud data and the feature information of the incomplete point cloud to obtain the clustering results of the point cloud data; and using the clustering results of the point cloud data as the second feature enhancement information of the point cloud data.
[0071] When the embodiment of the present application utilizes the feature information memory module to process the feature information of complete point cloud data and the feature information of incomplete point clouds, a certain clustering strategy can be adopted to implement it. For example, the correlation between the feature information can be measured based on the similarity calculation between the feature information, and then the feature information of all point clouds can be clustered into multiple subcategories.
[0072] It should be noted that since the true value of the complete point cloud data does not provide the true value label of the subcategory, the ultimate purpose of the above clustering using the feature information memory module is not to determine the specific subcategory of the point cloud data, but to provide additional stronger supervision signals for model training by capturing the correlation information between features.
[0073] In some embodiments of the present application, training the point cloud data completion model according to the training sample data and the feature enhancement information of the point cloud data to obtain the trained point cloud data completion model includes: training the point cloud data completion model according to the training sample data, the first feature enhancement information and the second feature enhancement information of the point cloud data to obtain the trained point cloud data completion model.
[0074] Based on the aforementioned embodiments, the category compilation module and the feature information memory module are two independent modules, both of which can provide additional supervisory signals for the training of the point cloud data completion model. Therefore, when training the point cloud data completion model, the first feature enhancement information and the second feature enhancement information can be used simultaneously as additional supervisory signals for enhanced training. Since the first feature enhancement information comes from the category compilation module and the second feature enhancement information comes from the feature information memory module, considering the feature enhancement information output by these two modules at the same time can also have the effect of mutual verification and constraint, because the output information of a single module is not 100% correct.
[0075] In some embodiments of the present application, the point cloud data completion model is trained according to the feature enhancement information of the training sample data and the point cloud data to obtain the trained point cloud data completion model, including: using the encoder of the point cloud data completion model to process the incomplete point cloud data to obtain the encoded features output by the encoder; fusing the feature enhancement information with the encoded features output by the encoder to obtain fused features; inputting the fused features into the decoder of the point cloud data completion model to obtain decoded features of the decoder; upsampling the decoded features of the decoder to obtain upsampled point cloud data; determining the loss value of the point cloud data completion model according to the upsampled point cloud data and the corresponding complete point cloud data, and using the loss value to update the parameters of the point cloud data completion model to obtain the trained point cloud data completion model.
[0076] In order to facilitate the understanding of the above embodiments, Figure 2As shown, a training flow diagram of a point cloud data completion model in an embodiment of the present application is provided. The basic network structure of the point cloud data completion model in the embodiment of the present application can be implemented using the existing Encoder (encoder) -Maxpool (maximum pool) -Decoder (decoder) network structure, on this basis, two feature enhancement modules, a class compilation module Class tokenizer and a feature information memory module Adaptive MemoryBank, are additionally set.
[0077] During training, the incomplete point cloud data is input into the category compilation module for processing, and the feature information of the incomplete point cloud data is obtained as the first feature enhancement information. At the same time, the feature information of the incomplete point cloud data and the feature information of the complete point cloud data are input into the feature information memory module. This process can be regarded as an Assimilation process. The feature information memory module processes the feature information of the incomplete point cloud data and the feature information of the complete point cloud data to obtain the second feature enhancement information. In addition, the incomplete point cloud data is also input into the encoder and the maximum pooling layer in turn to obtain the encoded feature Latent Feature of the incomplete point cloud data.
[0078] Then, the first feature enhancement information and the second feature enhancement information are used as additional supervisory signals to perform feature fusion with the encoding features of the incomplete point cloud data, and the extracted incomplete point cloud features are fine-tuned to improve the feature representation ability of the encoder, thereby improving the reconstruction ability of the model. This process can be regarded as an Accommodation process.
[0079] The above fused features are then input into the decoder and upsampled to obtain the upsampled point cloud data.
[0080] Finally, the loss value of the model is calculated based on the upsampled point cloud and the true value of the corresponding complete point cloud data, and the network parameters of the point cloud data completion model are updated until the model converges, and the trained point cloud data completion model is output.
[0081] To summarize, in order to reduce the information gap between incomplete point cloud data and complete point cloud data, the embodiment of the present application proposes an Assimilation-Accommodation mechanism, and deploys a category compilation module and a feature information memory module, which enhances the supervision signal of the model, more effectively utilizes the information of incomplete point cloud and complete point cloud, improves the model's mining efficiency for incomplete point cloud data and the model's reconstruction ability, thereby optimizing the completion effect of incomplete point cloud data and reducing the cost of point cloud data annotation.
[0082] The present application also provides a method for completing point cloud data. Figure 3 As shown, a schematic flow chart of a point cloud data completion method in an embodiment of the present application is provided, and the point cloud data completion method at least includes the following steps S310 to S320:
[0083] Step S310, obtaining incomplete point cloud data;
[0084] Step S320, using the point cloud data completion model to process the incomplete point cloud data to obtain complete point cloud data;
[0085] The point cloud data completion model is trained based on any of the training methods for the point cloud data completion model described above.
[0086] When the embodiment of the present application uses a trained point cloud data completion model to complete incomplete point cloud data, the incomplete point cloud data can be directly input into the trained point cloud data completion model to obtain complete point cloud data output by the model.
[0087] The above-trained point cloud data completion model can better represent the point cloud features in the process of completing incomplete point cloud data, thereby improving the quality of the completed point cloud data.
[0088] The present application also provides a training device 400 for a point cloud data completion model. Figure 4 As shown, a schematic diagram of the structure of a training device for a point cloud data completion model in an embodiment of the present application is provided, wherein the training device 400 for the point cloud data completion model comprises: a first acquisition unit 410, a feature enhancement processing unit 420 and a training unit 430, wherein:
[0089] A first acquisition unit 410 is used to acquire training sample data of a point cloud data completion model, wherein the training sample data includes incomplete point cloud data and corresponding complete point cloud data;
[0090] A feature enhancement processing unit 420 is used to perform feature enhancement processing using a feature enhancement module of the point cloud data completion model according to the incomplete point cloud data and the complete point cloud data to obtain feature enhancement information of the point cloud data;
[0091] The training unit 430 is used to train the point cloud data completion model according to the training sample data and the feature enhancement information of the point cloud data to obtain a trained point cloud data completion model.
[0092] In some embodiments of the present application, the feature enhancement module includes a category compilation module, and the feature enhancement processing unit 420 is specifically used to: use the category compilation module to process the incomplete point cloud data to obtain feature information of the incomplete point cloud data output by the category compilation module; and use the feature information of the incomplete point cloud data output by the category compilation module as the first feature enhancement information of the point cloud data.
[0093] In some embodiments of the present application, the feature enhancement module also includes a feature information memory module, and the feature enhancement processing unit 420 is specifically used to: store the feature information of the incomplete point cloud data output by the category compilation module into the feature information memory module; use the feature information memory module to process the feature information of the complete point cloud data and the feature information of the incomplete point cloud to obtain second feature enhancement information of the point cloud data.
[0094] In some embodiments of the present application, the feature enhancement processing unit 420 is specifically used to: use a preset clustering strategy to cluster the feature information of the complete point cloud data and the feature information of the incomplete point cloud to obtain a clustering result of the point cloud data; and use the clustering result of the point cloud data as the second feature enhancement information of the point cloud data.
[0095] In some embodiments of the present application, the training unit 430 is specifically used to: train the point cloud data completion model according to the training sample data, the first feature enhancement information and the second feature enhancement information of the point cloud data to obtain a trained point cloud data completion model.
[0096] In some embodiments of the present application, the training unit 430 is specifically used to: use the encoder of the point cloud data completion model to process the incomplete point cloud data to obtain the encoded features output by the encoder; fuse the feature enhancement information with the encoded features output by the encoder to obtain fused features; input the fused features into the decoder of the point cloud data completion model to obtain decoded features of the decoder; upsample the decoded features of the decoder to obtain upsampled point cloud data; determine the loss value of the point cloud data completion model based on the upsampled point cloud data and the corresponding complete point cloud data, and use the loss value to update the parameters of the point cloud data completion model to obtain the trained point cloud data completion model.
[0097] It can be understood that the training device of the above-mentioned point cloud data completion model can implement the various steps of the training method of the point cloud data completion model provided in the aforementioned embodiments. The relevant explanations on the training method of the point cloud data completion model are applicable to the training device of the point cloud data completion model and will not be repeated here.
[0098] The present application also provides a point cloud data completion device 500. Figure 5 As shown, a schematic diagram of the structure of a point cloud data completion device in an embodiment of the present application is provided, wherein the point cloud data completion device 500 comprises: a second acquisition unit 510 and a completion unit 520, wherein:
[0099] A second acquisition unit 510 is used to acquire incomplete point cloud data;
[0100] A completion unit 520, configured to process the incomplete point cloud data using a point cloud data completion model to obtain complete point cloud data;
[0101] Wherein, the point cloud data completion model is trained based on the training device of the aforementioned point cloud data completion model.
[0102] It can be understood that the above-mentioned point cloud data completion device can implement the various steps of the point cloud data completion method provided in the aforementioned embodiment. The relevant explanations about the point cloud data completion method are applicable to the training device of the point cloud data completion model and will not be repeated here.
[0103] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 6At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.
[0104] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0105] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.
[0106] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a training device or a point cloud data completion device for the point cloud data completion model at the logical level. The processor executes the program stored in the memory.
[0107] The above application Figure 1 The method for executing the training device of the point cloud data completion model disclosed in the embodiment shown and Figure 1The method performed by the point cloud data completion device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or instructions in software form. The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in a decoding processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.
[0108] The present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 1 The method executed by the training device of the point cloud data completion model in the embodiment shown, or the method executed Figure 1 The method performed by the point cloud data completion device in the illustrated embodiment.
[0109] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0110] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0111] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0112] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0113] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0114] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0115] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0116] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0117] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0118] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A training method for a point cloud data completion model, wherein: The training method of the point cloud data completion model includes: Acquire training sample data of a point cloud data completion model, wherein the training sample data includes incomplete point cloud data and corresponding complete point cloud data; According to the incomplete point cloud data and the complete point cloud data, a feature enhancement module of the point cloud data completion model is used to perform feature enhancement processing to obtain feature enhancement information of the point cloud data; The point cloud data completion model is trained according to the training sample data and the feature enhancement information of the point cloud data to obtain a trained point cloud data completion model.
2. The method for training a point cloud data completion model according to claim 1, wherein: The feature enhancement module includes a category compilation module, and the feature enhancement module of the point cloud data completion model is used to perform feature enhancement processing according to the incomplete point cloud data and the complete point cloud data, and the feature enhancement information of the point cloud data obtained includes: Processing the incomplete point cloud data using the category compilation module to obtain feature information of the incomplete point cloud data output by the category compilation module; The feature information of the incomplete point cloud data output by the category compilation module is used as first feature enhancement information of the point cloud data.
3. The method for training a point cloud data completion model according to claim 2, wherein: The feature enhancement module further includes a feature information memory module. The feature enhancement module of the point cloud data completion model is used to perform feature enhancement processing according to the incomplete point cloud data and the complete point cloud data, and the feature enhancement information of the point cloud data obtained includes: storing the feature information of the incomplete point cloud data output by the category compilation module into the feature information memory module; The feature information memory module is used to process the feature information of the complete point cloud data and the feature information of the incomplete point cloud to obtain second feature enhancement information of the point cloud data.
4. The method for training a point cloud data completion model according to claim 3, wherein: The using the feature information memory module to process the feature information of the complete point cloud data and the feature information of the incomplete point cloud to obtain the second feature enhancement information of the point cloud data includes: Using a preset clustering strategy to perform clustering processing on the feature information of the complete point cloud data and the feature information of the incomplete point cloud, to obtain a clustering result of the point cloud data; The clustering result of the point cloud data is used as second feature enhancement information of the point cloud data.
5. The method for training a point cloud data completion model according to claim 4, wherein: The step of training the point cloud data completion model according to the training sample data and the feature enhancement information of the point cloud data to obtain the trained point cloud data completion model comprises: The point cloud data completion model is trained according to the training sample data, the first feature enhancement information and the second feature enhancement information of the point cloud data to obtain a trained point cloud data completion model.
6. The method for training a point cloud data completion model according to any one of claims 1 to 5, wherein: The step of training the point cloud data completion model according to the training sample data and the feature enhancement information of the point cloud data to obtain the trained point cloud data completion model comprises: Processing the incomplete point cloud data using an encoder of the point cloud data completion model to obtain encoding features output by the encoder; Fusing the feature enhancement information with the encoding feature output by the encoder to obtain a fused feature; Inputting the fused features into a decoder of the point cloud data completion model to obtain decoding features of the decoder; Upsampling the decoded features of the decoder to obtain upsampled point cloud data; The loss value of the point cloud data completion model is determined according to the upsampled point cloud data and the corresponding complete point cloud data, and the loss value is used to update the parameters of the point cloud data completion model to obtain the trained point cloud data completion model.
7. A point cloud data completion method, wherein: The point cloud data completion method comprises: Obtain incomplete point cloud data; Processing the incomplete point cloud data using a point cloud data completion model to obtain complete point cloud data; Wherein, the point cloud data completion model is trained based on the training method of the point cloud data completion model described in any one of claims 1 to 6.
8. A training device for a point cloud data completion model, wherein: The training device of the point cloud data completion model includes: A first acquisition unit is used to acquire training sample data of a point cloud data completion model, wherein the training sample data includes incomplete point cloud data and corresponding complete point cloud data; A feature enhancement processing unit, configured to perform feature enhancement processing using a feature enhancement module of the point cloud data completion model according to the incomplete point cloud data and the complete point cloud data, so as to obtain feature enhancement information of the point cloud data; A training unit is used to train the point cloud data completion model according to the training sample data and the feature enhancement information of the point cloud data to obtain a trained point cloud data completion model.
9. A point cloud data completion device, wherein: The point cloud data completion device comprises: A second acquisition unit, used for acquiring incomplete point cloud data; A completion unit, used for processing the incomplete point cloud data using a point cloud data completion model to obtain complete point cloud data; Wherein, the point cloud data completion model is trained based on the training device of the point cloud data completion model according to claim 8.
10. An electronic device, comprising: processor; as well as A memory arranged to store computer executable instructions, which, when executed, cause the processor to execute the training method for the point cloud data completion model of any one of claims 1 to 6, or execute the point cloud data completion method of claim 7.
11. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, enables the electronic device to execute the training method of the point cloud data completion model according to any one of claims 1 to 6, or Execute the point cloud data completion method described in claim 7.
