Feature enhancement method and device of point cloud data, electronic equipment and computer readable storage medium

The preset category coding model extracts the category information of point cloud data and integrates it with the encoding features to generate category feature enhancement information, which solves the problem of insufficient feature representation ability of point cloud data completion model in the existing technology, and improves the completion effect and Reduced labeling costs.

CN118941888APending Publication Date: 2024-11-12ZHIDAO NETWORK TECH (BEIJING) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410965138.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The existing point cloud data annotation cost is high, while the algorithm model that completes point cloud data has poor feature representation capabilities, which is difficult to meet the requirements for data quality in autonomous driving scenarios.

Method used

The preset category encoding model is used to process incomplete point cloud data, extract its category information, and merge it with the encoding features through attention mechanism to generate category feature enhancement information, which is used to train the point cloud data completion model and provide additional Supervise signals to improve the model's ability to characterize point cloud features.

Benefits of technology

The training effect of the point cloud data completion model has been improved, the quality of completion of incomplete point cloud data has been improved, and the cost of point cloud data annotation has been reduced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941888A_ABST
    Figure CN118941888A_ABST
Patent Text Reader

Abstract

The invention discloses a feature enhancement method and device for point cloud data, electronic equipment and a computer readable storage medium, and the method comprises the steps: obtaining training sample data of a point cloud data completion model and corresponding coding features, the training sample data of the point cloud data completion model comprising incomplete point cloud data; processing the incomplete point cloud data by using a preset category coding model to obtain category information of the incomplete point cloud data; and generating category feature enhancement information of the incomplete point cloud data according to the coding feature corresponding to the incomplete point cloud data and the category information of the incomplete point cloud data. According to the method, the global information of the incomplete point cloud, especially the category information of the incomplete point cloud, is extracted with the minimum information loss by adopting the preset category coding model, so that an additional and stronger supervision signal is provided for training of a point cloud data completion model, the characterization capability of the model on the point cloud features is enhanced, and the accuracy of the point cloud data completion model is improved. And the complementation effect of the model on the incomplete point cloud data is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of point cloud data processing, and in particular to a method, device, electronic device, and computer-readable storage medium for feature enhancement of point cloud data. Background Art

[0002] With the continuous development of autonomous driving perception technology, deep learning technology has become an important implementation algorithm in autonomous driving perception. LiDAR is one of the important sensors in autonomous driving scenarios, and the point cloud data collected by LiDAR is also an important data source for training deep learning models.

[0003] However, the cost of labeling point cloud data currently used for model training is high, while the feature representation capability of the algorithm model for completing point cloud data is poor. The quality of the point cloud data after completion is uneven, making it difficult to meet the data quality requirements for training point cloud data perception models in autonomous driving scenarios. Summary of the invention

[0004] The embodiments of the present application provide a feature enhancement method, device, electronic device, and computer-readable storage medium for point cloud data to provide additional and stronger supervisory signals for the training of a point cloud data completion model, thereby improving the model's ability to represent point cloud features.

[0005] The present application embodiment adopts the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a method for enhancing features of point cloud data, wherein the method for enhancing features of point cloud data comprises:

[0007] Acquire training sample data of a point cloud data completion model and corresponding encoding features, wherein the training sample data of the point cloud data completion model includes incomplete point cloud data;

[0008] Processing the incomplete point cloud data using a preset category coding model to obtain category information of the incomplete point cloud data;

[0009] According to the coding features corresponding to the incomplete point cloud data and the category information of the incomplete point cloud data, category feature enhancement information of the incomplete point cloud data is generated.

[0010] Optionally, the step of obtaining training sample data of the point cloud data completion model and corresponding encoding features includes:

[0011] Acquire training sample data of a point cloud data completion model, wherein the training sample data of the point cloud data completion model includes incomplete point cloud data;

[0012] The incomplete point cloud data is input into the encoder of the point cloud data completion model to obtain encoding features corresponding to the incomplete point cloud data.

[0013] Optionally, the preset category coding model is trained in the following manner:

[0014] Acquire training sample data of a category coding model, wherein the training sample data of the category coding model is annotated with subcategory information of incomplete point cloud data;

[0015] The class coding model is trained using the training sample data of the class coding model to obtain the preset class coding model.

[0016] Optionally, generating category feature enhancement information of the incomplete point cloud data according to the coding features corresponding to the incomplete point cloud data and the category information of the incomplete point cloud data includes:

[0017] Inputting the encoded features corresponding to the incomplete point cloud data into the maximum pooling layer of the point cloud data completion model to obtain the pooled features corresponding to the incomplete point cloud data;

[0018] Based on the attention mechanism, the pooled features corresponding to the incomplete point cloud data are fused with the category information of the incomplete point cloud data to obtain the fused features as the category feature enhancement information of the incomplete point cloud data.

[0019] Optionally, the training sample data of the point cloud data completion model also includes complete point cloud data. After generating category feature enhancement information of the incomplete point cloud data according to the encoding features corresponding to the incomplete point cloud data and the category information of the incomplete point cloud data, the feature enhancement method of the point cloud data also includes:

[0020] The point cloud data completion model is trained according to the category feature enhancement information of the complete point cloud data and the incomplete point cloud data to obtain a trained point cloud data completion model.

[0021] Optionally, the step of training the point cloud data completion model according to the category feature enhancement information of the complete point cloud data and the incomplete point cloud data to obtain the trained point cloud data completion model comprises:

[0022] Inputting the category feature enhancement information of the incomplete point cloud data into a decoder of the point cloud data completion model to obtain decoding features of the decoder;

[0023] Upsampling the decoded features of the decoder to obtain upsampled point cloud data;

[0024] The loss value of the point cloud data completion model is determined according to the upsampled point cloud data and the corresponding complete point cloud data, and the loss value is used to update the parameters of the point cloud data completion model to obtain the trained point cloud data completion model.

[0025] Optionally, the category feature enhancement information of the point cloud data is first feature enhancement information of the point cloud data, and the step of training the point cloud data completion model according to the category feature enhancement information of the complete point cloud data and the incomplete point cloud data to obtain the trained point cloud data completion model further includes:

[0026] Obtain complete feature information of point cloud data;

[0027] Storing the category feature enhancement information of the incomplete point cloud data and the feature information of the complete point cloud data in a feature information memory module of the point cloud data completion model;

[0028] Using the feature information memory module to process the category feature enhancement information of the incomplete point cloud data and the feature information of the complete point cloud data, to obtain second feature enhancement information of the point cloud data;

[0029] The point cloud data completion model is trained according to the first feature enhancement information and the second feature enhancement information of the point cloud data to obtain a trained point cloud data completion model.

[0030] In a second aspect, an embodiment of the present application further provides a feature enhancement device for point cloud data, wherein the feature enhancement device for point cloud data comprises:

[0031] An acquisition unit, used to acquire training sample data of a point cloud data completion model and corresponding encoding features, wherein the training sample data of the point cloud data completion model includes incomplete point cloud data;

[0032] A category coding unit, used for processing the incomplete point cloud data using a preset category coding model to obtain category information of the incomplete point cloud data;

[0033] The feature enhancement unit is used to generate category feature enhancement information of the incomplete point cloud data according to the encoding features corresponding to the incomplete point cloud data and the category information of the incomplete point cloud data.

[0034] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0035] Processor; and

[0036] A memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor performs any of the aforementioned feature enhancement methods for point cloud data.

[0037] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple applications, the electronic device executes any of the aforementioned feature enhancement methods for point cloud data.

[0038] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: the feature enhancement method of point cloud data in the embodiments of the present application first obtains the training sample data and the corresponding coding features of the point cloud data completion model, and the training sample data of the point cloud data completion model includes incomplete point cloud data; then the feature information of the incomplete point cloud data is processed using a preset category coding model to obtain the category information of the incomplete point cloud data; finally, the category feature enhancement information of the incomplete point cloud data is generated according to the coding features corresponding to the incomplete point cloud data and the category features of the incomplete point cloud data. The feature enhancement method of point cloud data in the embodiments of the present application adopts a preset category coding model to extract the global information of the incomplete point cloud, especially the category information of the incomplete point cloud, with minimal information loss, thereby providing additional and stronger supervisory signals for the training of the point cloud data completion model, enhancing the model's ability to characterize point cloud features, and improving the model's completion effect on incomplete point cloud data. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0040] Figure 1 A schematic diagram of a process flow of a feature enhancement method for point cloud data in an embodiment of the present application;

[0041] Figure 2 A schematic diagram of a training process of a point cloud data completion model in an embodiment of the present application;

[0042] Figure 3 This is a schematic diagram of the structure of a feature enhancement device for point cloud data in an embodiment of the present application;

[0043] Figure 4 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0045] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0046] The present application embodiment provides a method for enhancing the features of point cloud data, such as Figure 1 As shown, a flow chart of a method for enhancing the features of point cloud data in an embodiment of the present application is provided, and the method for enhancing the features of point cloud data at least includes the following steps S110 to S130:

[0047] Step S110, obtaining training sample data of a point cloud data completion model and corresponding encoding features, wherein the training sample data of the point cloud data completion model includes incomplete point cloud data.

[0048] The feature enhancement method for point cloud data of the embodiment of the present application is mainly used to provide feature enhancement information for the training process of the point cloud data completion model as an additional supervision signal to improve the training effect of the model. Therefore, it is necessary to first obtain the training sample data of the point cloud data completion model. The feature enhancement method for point cloud data of the embodiment of the present application is mainly used in the feature enhancement processing stage. Therefore, the training sample data targeted here are mainly incomplete point cloud data. Incomplete point cloud data refers to data with missing point clouds. In the subsequent actual training stage of the point cloud completion model, the complete point cloud data corresponding to the incomplete point cloud data will also be used as true value information.

[0049] In addition, it is also necessary to obtain the encoding features corresponding to the incomplete point cloud data. The encoding features here can be understood as the original potential features corresponding to the incomplete point cloud data obtained by processing the backbone network based on the point cloud data completion model. The backbone network structure of the point cloud data completion model can be implemented based on network structures such as Transformer or CNN (Convolutional Neural Networks). The specific backbone network to be used can be flexibly selected by technical personnel in this field according to actual needs, and no specific limitation is made here.

[0050] Step S120: Process the incomplete point cloud data using a preset category coding model to obtain category information of the incomplete point cloud data.

[0051] Based on the point cloud data completion model, the embodiment of the present application additionally adopts a preset category encoding model to process incomplete point cloud data, extracting the global information of the incomplete point cloud data with minimal information loss, especially the category information of the incomplete point cloud data, which can be regarded as a learnable category marker Class Token. The category information of the embodiment of the present application is not only the division of large categories, but also includes more detailed sub-category divisions. For example, cars, trucks, buses, etc. can be regarded as the division of larger categories, and each large category can be further subdivided into multiple sub-categories. For example, cars can be further divided into sedans, sports cars, off-road vehicles, etc., thereby capturing more detailed and accurate feature information in incomplete point cloud data.

[0052] The preset category encoding model can be regarded as a model independent of the point cloud data completion model. The incomplete point cloud data is processed separately through the pre-trained category encoding model. Of course, the preset category encoding model can also be regarded as a module embedded in the point cloud data completion model. The specific setting can be flexibly set by technical personnel in this field according to actual needs, and no specific limitation is made here.

[0053] Step S130: generating category feature enhancement information of the incomplete point cloud data according to the coding features corresponding to the incomplete point cloud data and the category information of the incomplete point cloud data.

[0054] By fusing the category information of the incomplete point cloud data obtained in the previous steps with the original encoded features output by the point cloud data completion model, features with enhanced category information can be obtained, thereby providing additional and stronger supervision information for the subsequent training of the point cloud data completion model.

[0055] The feature enhancement method for point cloud data in the embodiment of the present application adopts a preset category encoding model to extract the global information of the incomplete point cloud with minimal information loss, especially the category information of the incomplete point cloud, thereby providing additional and stronger supervision signals for the training of the point cloud data completion model, enhancing the model's ability to represent point cloud features, and improving the model's completion effect on incomplete point cloud data.

[0056] In some embodiments of the present application, the method of obtaining training sample data of a point cloud data completion model and corresponding encoding features includes: obtaining training sample data of a point cloud data completion model, wherein the training sample data of the point cloud data completion model includes incomplete point cloud data; inputting the incomplete point cloud data into an encoder of the point cloud data completion model to obtain encoding features corresponding to the incomplete point cloud data.

[0057] The backbone network of the point cloud data completion model of the embodiment of the present application includes an encoder. The acquired incomplete point cloud data is input into the encoder for encoding processing to obtain the original encoding features output by the encoder of the point cloud data completion model.

[0058] Traditional point cloud data completion models usually first input incomplete point cloud data, which is then processed by network structures such as Encoder, Maxpooling, and Decoder to finally obtain the completed point cloud data. Such a model structure is difficult to capture the global information of incomplete point cloud data and cannot effectively form a strong supervision signal. As a result, when incomplete point cloud data is input during the model inference stage, the output result is often incomplete and of low quality.

[0059] The embodiment of the present application forms a stronger supervision signal by further integrating the category information of incomplete point cloud data on the basis of the original encoding features output by the above-mentioned encoder, thereby enhancing the point cloud data completion model's ability to characterize the features of incomplete point cloud data and improving the completion effect.

[0060] In some embodiments of the present application, the preset category coding model is trained in the following manner: obtaining training sample data of the category coding model, the training sample data of the category coding model being annotated with subcategory information of incomplete point cloud data; training the category coding model using the training sample data of the category coding model to obtain the preset category coding model.

[0061] The preset category coding model of the embodiment of the present application can be pre-trained. The training of the category coding model is mainly to train the model's ability to perceive and extract sub-category feature information of incomplete point cloud data. Therefore, when annotating training samples, the sub-category information corresponding to the incomplete point cloud data can be annotated as a supervisory signal to train the category coding model.

[0062] The preset category coding model of the embodiment of the present application can be trained based on a model structure such as a multilayer perceptron (MLP for short). The MLP can learn to extract global features from the input point cloud data and use these features to predict the category of the point cloud. Of course, it can also be implemented using algorithms such as SVM (Support Vector Machine). The specific form of the category compilation algorithm can be flexibly selected by those skilled in the art according to actual needs, and is not specifically limited here.

[0063] In some embodiments of the present application, generating category feature enhancement information of incomplete point cloud data based on the encoded features corresponding to the incomplete point cloud data and the category information of the incomplete point cloud data includes: inputting the encoded features corresponding to the incomplete point cloud data into the maximum pooling layer of the point cloud data completion model to obtain pooled features corresponding to the incomplete point cloud data; based on the attention mechanism, fusing the pooled features corresponding to the incomplete point cloud data with the category information of the incomplete point cloud data to obtain fused features as the category feature enhancement information of the incomplete point cloud data.

[0064] The point cloud data completion model of the embodiment of the present application also includes a maximum pooling layer, which is arranged after the encoder and is used to perform maximum pooling processing on the original encoded features output by the encoder. It is mainly used to achieve the purpose of feature dimensionality reduction. Through the pooling operation, the size of the feature map can be reduced, thereby reducing the amount of calculation and the number of parameters, preventing overfitting, and retaining important features.

[0065] Based on the attention mechanism, the pooled features corresponding to the incomplete point cloud data are fused with the category information of the incomplete point cloud data to obtain the fused features. The fused features can be regarded as the category feature enhancement information corresponding to the incomplete point cloud data. The attention mechanism can focus on more important feature information in the feature fusion process and assign a higher fusion weight. Of course, the specific attention mechanism to be adopted can be flexibly selected by technicians in this field in combination with the existing technology, and no specific limitation is made here.

[0066] In some embodiments of the present application, the training sample data of the point cloud data completion model also includes complete point cloud data. After generating category feature enhancement information of the incomplete point cloud data based on the encoding features corresponding to the incomplete point cloud data and the category information of the incomplete point cloud data, the feature enhancement method of the point cloud data also includes: training the point cloud data completion model based on the complete point cloud data and the category feature enhancement information of the incomplete point cloud data to obtain a trained point cloud data completion model.

[0067] Based on the above embodiments, the category feature enhancement information of incomplete point cloud data can provide additional supervisory signals for the training of the point cloud data completion model. Therefore, when training the point cloud data completion model, the category feature enhancement information of the incomplete point cloud data and the true value of the complete power data corresponding to the incomplete point cloud data can be combined to perform enhanced training on the point cloud data completion model, thereby improving the point cloud data completion model's ability to characterize point cloud features, and further improving the model's completion effect on incomplete point cloud data.

[0068] In some embodiments of the present application, the point cloud data completion model is trained according to the category feature enhancement information of the complete point cloud data and the incomplete point cloud data to obtain the trained point cloud data completion model, including: inputting the category feature enhancement information of the incomplete point cloud data into the decoder of the point cloud data completion model to obtain the decoded features of the decoder; upsampling the decoded features of the decoder to obtain upsampled point cloud data; determining the loss value of the point cloud data completion model according to the upsampled point cloud data and the corresponding complete point cloud data, and using the loss value to update the parameters of the point cloud data completion model to obtain the trained point cloud data completion model.

[0069] The backbone network of the point cloud data completion model also includes a decoder, which is set after the maximum pooling layer and is used to decode the category feature enhancement information obtained by fusing the features after pooling in the maximum pooling layer with the category information of the incomplete point cloud data to obtain the decoded features. Since the feature dimension reduction is performed through the maximum pooling process, the decoded features can be further upsampled through the upsampling module to obtain the upsampled point cloud data.

[0070] Finally, the loss value of the point cloud data completion model is calculated based on the upsampled point cloud data and the corresponding complete point cloud data true value, and the network parameters of the point cloud data completion model are updated until the model converges, and the trained point cloud data completion model is output.

[0071] In some embodiments of the present application, the category feature enhancement information of the point cloud data is the first feature enhancement information of the point cloud data, and the training of the point cloud data completion model according to the category feature enhancement information of the complete point cloud data and the incomplete point cloud data to obtain the trained point cloud data completion model also includes: acquiring the feature information of the complete point cloud data; storing the category feature enhancement information of the incomplete point cloud data and the feature information of the complete point cloud data in a feature information memory module of the point cloud data completion model; processing the category feature enhancement information of the incomplete point cloud data and the feature information of the complete point cloud data using the feature information memory module to obtain the second feature enhancement information of the point cloud data; training the point cloud data completion model according to the first feature enhancement information and the second feature enhancement information of the point cloud data to obtain the trained point cloud data completion model.

[0072] In addition to the original point cloud data, the complete point cloud data will be annotated with some basic feature information in advance, such as the category of the point cloud data. However, the categories here are mainly relatively coarse-grained categories, and the true value labels of the subclasses are not annotated.

[0073] Considering that the complete point cloud data true value does not have a label for the sub-category true value, the existing category feature information is not sufficient for the training accuracy requirements of the point cloud data completion model. Based on this, the embodiment of the present application additionally sets a feature information memory module based on the network structure of the existing point cloud data completion model. The main function of the feature information memory module is to store and process the category feature enhancement information of the incomplete point cloud data obtained based on the preset category encoding model and the feature information of the complete point cloud data, thereby providing more detailed sub-category feature information of the point cloud data, and updating it as the data is continuously introduced.

[0074] The feature information memory module of the embodiment of the present application can process the feature information of all complete and incomplete point clouds at the same time, so as to capture the correlation information between these features as the second feature enhancement information, and apply the second feature enhancement information as an additional supervisory signal to the training process of the point cloud data completion model. Of course, the specific implementation of the feature information memory module can be flexibly set by those skilled in the art according to actual needs, and is not specifically limited here.

[0075] When the embodiment of the present application utilizes the feature information memory module to process the feature information of complete point cloud data and the feature information of incomplete point clouds, a certain clustering strategy can be adopted to implement it. For example, the correlation between the feature information can be measured based on the similarity calculation between the feature information, and then the feature information of all point clouds can be clustered into multiple subcategories.

[0076] It should be noted that since the true value of the complete point cloud data does not provide the true value label of the subcategory, the ultimate purpose of the above clustering using the feature information memory module is not to determine the specific subcategory of the point cloud data, but to provide additional stronger supervision signals for model training by capturing the correlation information between features.

[0077] Based on the aforementioned embodiment, the preset category encoding model and the feature information memory module are two independent parts, both of which can provide additional supervisory signals for the training of the point cloud data completion model. Therefore, when training the point cloud data completion model, the above-mentioned first feature enhancement information and the second feature enhancement information can be used simultaneously as additional supervisory signals for enhanced training. Since the first feature enhancement information comes from the preset category encoding model and the second feature enhancement information comes from the feature information memory module, considering the feature enhancement information of these two parts at the same time can also have the effect of mutual verification and constraint, because the single output information is not 100% correct.

[0078] In order to facilitate the understanding of the above embodiments, Figure 2As shown, a training process diagram of a point cloud data completion model in an embodiment of the present application is provided. Taking the point cloud data completion model based on the Transformer architecture as an example, the backbone network structure of the point cloud data completion model in the embodiment of the present application can be implemented using the existing Encoder-Maxpool-Decoder network structure. On this basis, a module Class tokenizer and a feature information memory module for implementing a preset category encoding model are additionally set as two feature enhancement modules.

[0079] During training, the incomplete point cloud data is first input into the encoder of the backbone network to obtain the original encoding features, and the original encoding features are input into the maximum pooling layer for pooling processing to obtain the pooled features. In addition, the incomplete point cloud data will also be input into a preset category encoding model such as MLP Head for processing to obtain the category information of the incomplete point cloud data, and the pooled features of the incomplete point cloud data are fused with the category information of the incomplete point cloud data to obtain the category feature enhancement information as the first feature enhancement information.

[0080] The category feature enhancement information of the incomplete point cloud data and the feature information of the complete point cloud data are input into the feature information memory module, and the feature information memory module processes the category feature enhancement information of the incomplete point cloud data and the feature information of the complete point cloud data to obtain second feature enhancement information.

[0081] Then, the first feature enhancement information and the second feature enhancement information are used as additional supervisory signals to fine-tune the incomplete point cloud features extracted by the encoder of the backbone network, thereby improving the feature representation capability of the encoder and thus improving the reconstruction capability of the model.

[0082] The above fused features are then input into the decoder and upsampled to obtain upsampled point cloud data.

[0083] Finally, the loss value of the model is calculated based on the true value of the upsampled point cloud data and the corresponding complete point cloud data, and the network parameters of the point cloud data completion model are updated until the model converges, and the trained point cloud data completion model is output.

[0084] To summarize, in order to reduce the information gap between incomplete point cloud data and complete point cloud data, the embodiment of the present application deploys a preset category encoding model based on the backbone network of the point cloud data completion model, enhances the supervision signal of the model, and more effectively utilizes the information of incomplete point cloud and complete point cloud, thereby improving the model's mining efficiency for incomplete point cloud data and the model's reconstruction ability, thereby optimizing the completion effect of incomplete point cloud data and reducing the cost of point cloud data annotation.

[0085] The present application embodiment also provides a feature enhancement device 300 for point cloud data, such as Figure 3 As shown, a schematic diagram of the structure of a feature enhancement device for point cloud data in an embodiment of the present application is provided, wherein the feature enhancement device 300 for point cloud data comprises: an acquisition unit 310, a category encoding unit 320 and a feature enhancement unit 330, wherein:

[0086] An acquisition unit 310 is used to acquire training sample data of a point cloud data completion model and corresponding encoding features, wherein the training sample data of the point cloud data completion model includes incomplete point cloud data;

[0087] A category coding unit 320, configured to process the incomplete point cloud data using a preset category coding model to obtain category information of the incomplete point cloud data;

[0088] The feature enhancement unit 330 is used to generate category feature enhancement information of the incomplete point cloud data according to the encoding features corresponding to the incomplete point cloud data and the category information of the incomplete point cloud data.

[0089] In some embodiments of the present application, the acquisition unit 310 is specifically used to: acquire training sample data of a point cloud data completion model, wherein the training sample data of the point cloud data completion model includes incomplete point cloud data; input the incomplete point cloud data into an encoder of the point cloud data completion model to obtain encoding features corresponding to the incomplete point cloud data.

[0090] In some embodiments of the present application, the preset category coding model is trained in the following manner: obtaining training sample data of the category coding model, the training sample data of the category coding model being annotated with subcategory information of incomplete point cloud data; training the category coding model using the training sample data of the category coding model to obtain the preset category coding model.

[0091] In some embodiments of the present application, the feature enhancement unit 330 is specifically used to: input the encoded features corresponding to the incomplete point cloud data into the maximum pooling layer of the point cloud data completion model to obtain the pooled features corresponding to the incomplete point cloud data; based on the attention mechanism, fuse the pooled features corresponding to the incomplete point cloud data with the category information of the incomplete point cloud data to obtain the fused features as the category feature enhancement information of the incomplete point cloud data.

[0092] In some embodiments of the present application, the training sample data of the point cloud data completion model also includes complete point cloud data, and the feature enhancement device of the point cloud data also includes: a training unit, which is used to train the point cloud data completion model according to the complete point cloud data and the category feature enhancement information of the incomplete point cloud data after generating category feature enhancement information of the incomplete point cloud data according to the encoding features corresponding to the incomplete point cloud data and the category information of the incomplete point cloud data, so as to obtain a trained point cloud data completion model.

[0093] In some embodiments of the present application, the training unit is specifically used to: input the category feature enhancement information of the incomplete point cloud data into the decoder of the point cloud data completion model to obtain the decoding features of the decoder; upsample the decoding features of the decoder to obtain upsampled point cloud data; determine the loss value of the point cloud data completion model based on the upsampled point cloud data and the corresponding complete point cloud data, and use the loss value to update the parameters of the point cloud data completion model to obtain the trained point cloud data completion model.

[0094] In some embodiments of the present application, the category feature enhancement information of the point cloud data is the first feature enhancement information of the point cloud data, and the training unit is specifically used to: obtain the feature information of the complete point cloud data; store the category feature enhancement information of the incomplete point cloud data and the feature information of the complete point cloud data in the feature information memory module of the point cloud data completion model; use the feature information memory module to process the category feature enhancement information of the incomplete point cloud data and the feature information of the complete point cloud data to obtain the second feature enhancement information of the point cloud data; train the point cloud data completion model according to the first feature enhancement information and the second feature enhancement information of the point cloud data to obtain the trained point cloud data completion model.

[0095] It can be understood that the above-mentioned feature enhancement device for point cloud data can implement the various steps of the feature enhancement method for point cloud data provided in the aforementioned embodiments, and the relevant explanations on the feature enhancement method for point cloud data are all applicable to the feature enhancement device for point cloud data, which will not be repeated here.

[0096] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 4At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0097] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0098] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0099] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a feature enhancement device for point cloud data at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0100] Acquire training sample data of a point cloud data completion model and corresponding encoding features, wherein the training sample data of the point cloud data completion model includes incomplete point cloud data;

[0101] Processing the incomplete point cloud data using a preset category coding model to obtain category information of the incomplete point cloud data;

[0102] According to the coding features corresponding to the incomplete point cloud data and the category information of the incomplete point cloud data, category feature enhancement information of the incomplete point cloud data is generated.

[0103] The above application Figure 1The method performed by the feature enhancement device of point cloud data disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software. The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in a decoding processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0104] The present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 1 The method performed by the feature enhancement device of point cloud data in the embodiment shown is specifically used to perform:

[0105] Acquire training sample data of a point cloud data completion model and corresponding encoding features, wherein the training sample data of the point cloud data completion model includes incomplete point cloud data;

[0106] Processing the incomplete point cloud data using a preset category coding model to obtain category information of the incomplete point cloud data;

[0107] According to the coding features corresponding to the incomplete point cloud data and the category information of the incomplete point cloud data, category feature enhancement information of the incomplete point cloud data is generated.

[0108] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0109] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0110] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0111] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0112] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0113] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0114] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0115] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0116] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0117] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A feature enhancement method for point cloud data, wherein: The feature enhancement method of the point cloud data comprises: Acquire training sample data of a point cloud data completion model and corresponding encoding features, wherein the training sample data of the point cloud data completion model includes incomplete point cloud data; Processing the incomplete point cloud data using a preset category coding model to obtain category information of the incomplete point cloud data; According to the coding features corresponding to the incomplete point cloud data and the category information of the incomplete point cloud data, category feature enhancement information of the incomplete point cloud data is generated.

2. The feature enhancement method of point cloud data according to claim 1, wherein: The acquisition of training sample data of the point cloud data completion model and corresponding encoding features includes: Acquire training sample data of a point cloud data completion model, wherein the training sample data of the point cloud data completion model includes incomplete point cloud data; The incomplete point cloud data is input into the encoder of the point cloud data completion model to obtain encoding features corresponding to the incomplete point cloud data.

3. The feature enhancement method of point cloud data according to claim 1, wherein: The preset category encoding model is trained in the following way: Acquire training sample data of a category coding model, wherein the training sample data of the category coding model is annotated with subcategory information of incomplete point cloud data; The class coding model is trained using the training sample data of the class coding model to obtain the preset class coding model.

4. The feature enhancement method of point cloud data according to claim 1, wherein: The generating the category feature enhancement information of the incomplete point cloud data according to the coding features corresponding to the incomplete point cloud data and the category information of the incomplete point cloud data comprises: Inputting the encoded features corresponding to the incomplete point cloud data into the maximum pooling layer of the point cloud data completion model to obtain the pooled features corresponding to the incomplete point cloud data; Based on the attention mechanism, the pooled features corresponding to the incomplete point cloud data are fused with the category information of the incomplete point cloud data to obtain the fused features as the category feature enhancement information of the incomplete point cloud data.

5. The feature enhancement method of point cloud data according to claim 1, wherein: The training sample data of the point cloud data completion model also includes complete point cloud data. After generating category feature enhancement information of the incomplete point cloud data according to the encoding features corresponding to the incomplete point cloud data and the category information of the incomplete point cloud data, the feature enhancement method of the point cloud data also includes: The point cloud data completion model is trained according to the category feature enhancement information of the complete point cloud data and the incomplete point cloud data to obtain a trained point cloud data completion model.

6. The feature enhancement method of point cloud data according to claim 5, wherein: The step of training the point cloud data completion model according to the category feature enhancement information of the complete point cloud data and the incomplete point cloud data to obtain the trained point cloud data completion model comprises: Inputting the category feature enhancement information of the incomplete point cloud data into a decoder of the point cloud data completion model to obtain decoding features of the decoder; Upsampling the decoded features of the decoder to obtain upsampled point cloud data; The loss value of the point cloud data completion model is determined according to the upsampled point cloud data and the corresponding complete point cloud data, and the loss value is used to update the parameters of the point cloud data completion model to obtain the trained point cloud data completion model.

7. The feature enhancement method of point cloud data according to claim 5, wherein: The category feature enhancement information of the point cloud data is first feature enhancement information of the point cloud data, and the point cloud data completion model is trained according to the category feature enhancement information of the complete point cloud data and the incomplete point cloud data to obtain the trained point cloud data completion model, and further comprises: Obtain complete feature information of point cloud data; Storing the category feature enhancement information of the incomplete point cloud data and the feature information of the complete point cloud data in a feature information memory module of the point cloud data completion model; Using the feature information memory module to process the category feature enhancement information of the incomplete point cloud data and the feature information of the complete point cloud data, to obtain second feature enhancement information of the point cloud data; The point cloud data completion model is trained according to the first feature enhancement information and the second feature enhancement information of the point cloud data to obtain a trained point cloud data completion model.

8. A feature enhancement device for point cloud data, wherein: The feature enhancement device of the point cloud data comprises: An acquisition unit, used to acquire training sample data of a point cloud data completion model and corresponding encoding features, wherein the training sample data of the point cloud data completion model includes incomplete point cloud data; A category coding unit, used for processing the incomplete point cloud data using a preset category coding model to obtain category information of the incomplete point cloud data; The feature enhancement unit is used to generate category feature enhancement information of the incomplete point cloud data according to the encoding features corresponding to the incomplete point cloud data and the category information of the incomplete point cloud data.

9. An electronic device, comprising: processor; as well as A memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor executes the feature enhancement method for point cloud data according to any one of claims 1 to 7.

10. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, enables the electronic device to execute the feature enhancement method of point cloud data of any one of claims 1 to 7.