Feature enhancement method and device of point cloud data, electronic equipment and computer readable storage medium

Through the adaptive clustering strategy, the feature information of point cloud data is processed and subcategory feature information is generated, which solves the problems of high annotation cost and poor feature representation ability in the existing technology, and improves the point cloud data in autonomous driving scenarios Complete quality.

CN118941887APending Publication Date: 2024-11-12ZHIDAO NETWORK TECH (BEIJING) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410965133.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The existing point cloud data annotation cost is high, and the algorithm model that completes point cloud data has poor feature representation capabilities, making it difficult to meet the data quality requirements in autonomous driving scenarios.

Method used

Adaptive clustering strategy is used to process the feature information of point cloud data. The feature information memory module stores incomplete and complete feature information of point cloud data, clusters, and generates subcategory feature information as an additional supervision signal to enhance The model's ability to characterize point cloud features.

Benefits of technology

The training effect of the point cloud data completion model has been improved, the quality of completion of incomplete point cloud data has been improved, and the cost of point cloud data annotation has been reduced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941887A_ABST
    Figure CN118941887A_ABST
Patent Text Reader

Abstract

The invention discloses a feature enhancement method and device for point cloud data, electronic equipment and a computer readable storage medium, and the method comprises the steps: obtaining feature information of training sample data of a point cloud data completion model, including feature information of incomplete point cloud data and corresponding complete point cloud data; storing the feature information of the incomplete point cloud data and the feature information of the complete point cloud data into a feature information memory module of a point cloud data completion model; the feature information of the incomplete point cloud data and the feature information of the complete point cloud data in the feature information memory module are clustered through an adaptive clustering strategy, the clustering result of the point cloud data is obtained and serves as feature enhancement information, and the clustering result comprises sub-cluster clustering results. According to the method, the adaptive clustering strategy is adopted to realize the division of the point cloud feature sub-categories, additional and stronger supervision signals are provided for the training of the point cloud data completion model, the characterization capability of the model on the point cloud features is enhanced, and the completion effect of the model is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of point cloud data processing, and in particular to a method, device, electronic device, and computer-readable storage medium for feature enhancement of point cloud data. Background Art

[0002] With the continuous development of autonomous driving perception technology, deep learning technology has become an important implementation algorithm in autonomous driving perception. LiDAR is one of the important sensors in autonomous driving scenarios, and the point cloud data collected by LiDAR is also an important data source for training deep learning models.

[0003] However, the cost of labeling point cloud data currently used for model training is high, while the feature representation capability of the algorithm model for completing point cloud data is poor. The quality of the point cloud data after completion is uneven, making it difficult to meet the data quality requirements for training point cloud data perception models in autonomous driving scenarios. Summary of the invention

[0004] The embodiments of the present application provide a feature enhancement method, device, electronic device, and computer-readable storage medium for point cloud data to provide additional and stronger supervisory signals for the training of a point cloud data completion model, thereby improving the model's ability to represent point cloud features.

[0005] The present application embodiment adopts the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a method for enhancing features of point cloud data, wherein the method for enhancing features of point cloud data comprises:

[0007] Acquire feature information of training sample data of a point cloud data completion model, wherein the feature information of the training sample data includes feature information of incomplete point cloud data and feature information of corresponding complete point cloud data;

[0008] The feature information of the incomplete point cloud data and the feature information of the complete point cloud data are stored in a feature information memory module of the point cloud data completion model;

[0009] An adaptive clustering strategy is used to cluster the feature information of the incomplete point cloud data and the feature information of the complete point cloud data stored in the feature information memory module to obtain a clustering result of the point cloud data as feature enhancement information of the point cloud data. The clustering result of the point cloud data includes a subclass clustering result of the point cloud data.

[0010] Optionally, the feature information of the incomplete point cloud data is obtained by:

[0011] Acquire training sample data of a point cloud data completion model, wherein the training sample data includes incomplete point cloud data;

[0012] The incomplete point cloud data is processed using the category compilation module of the point cloud data completion model to obtain feature information of the incomplete point cloud data output by the category compilation module.

[0013] Optionally, the step of clustering the feature information of the incomplete point cloud data and the feature information of the complete point cloud data stored in the feature information memory module using an adaptive clustering strategy to obtain a clustering result of the point cloud data includes:

[0014] For each feature information stored in the feature information memory module, a preset positive and negative sample division strategy is used to perform positive and negative sample division to obtain a positive and negative sample division result;

[0015] The clustering result of the point cloud data is determined according to the positive and negative sample division result.

[0016] Optionally, for each feature information stored in the feature information memory module, performing positive and negative sample division using a preset positive and negative sample division strategy, and obtaining a positive and negative sample division result includes:

[0017] Taking any one of the feature information stored in the feature information memory module as an anchor point, determining the similarity between the feature information of the anchor point and the remaining feature information stored in the feature information memory module;

[0018] Determining feature information most similar to the feature information of the anchor point according to the similarity;

[0019] The feature information most similar to the feature information of the anchor point is used as a basic positive sample, and the remaining feature information stored in the feature information memory module is divided into positive and negative samples to obtain a positive and negative sample division result corresponding to the feature information of the anchor point.

[0020] Optionally, taking the feature information most similar to the feature information of the anchor point as a basic positive sample, performing positive and negative sample division on the remaining feature information stored in the feature information memory module, and obtaining the positive and negative sample division result corresponding to the feature information of the anchor point includes:

[0021] Determine whether the feature information of the anchor point and the basic positive sample are the most similar feature information to each other, and whether the remaining feature information and the basic positive sample have the same category label;

[0022] If yes, then determining that the remaining feature information is a positive sample;

[0023] Otherwise, it is determined that the remaining feature information is a negative sample.

[0024] Optionally, after clustering the feature information of the incomplete point cloud data and the feature information of the complete point cloud data stored in the feature information memory module using an adaptive clustering strategy, the feature enhancement method of the point cloud data further includes:

[0025] Using the feature information of the incomplete point cloud data output by the category compilation module as first feature enhancement information of the point cloud data, and using the clustering result of the point cloud data as second feature enhancement information of the point cloud data;

[0026] The point cloud data completion model is trained according to the training sample data of the point cloud data completion model, the first feature enhancement information and the second feature enhancement information of the point cloud data, to obtain a trained point cloud data completion model.

[0027] Optionally, the step of training the point cloud data completion model according to the training sample data of the point cloud data completion model, the first feature enhancement information and the second feature enhancement information of the point cloud data to obtain the trained point cloud data completion model comprises:

[0028] Processing the incomplete point cloud data using the encoder of the point cloud data completion model to obtain encoding features output by the encoder;

[0029] Fusing the first feature enhancement information and the second feature enhancement information of the point cloud data with the encoding feature output by the encoder to obtain a fused feature;

[0030] Inputting the fused features into a decoder of the point cloud data completion model to obtain decoding features of the decoder;

[0031] Upsampling the decoded features of the decoder to obtain upsampled point cloud data;

[0032] The loss value of the point cloud data completion model is determined according to the upsampled point cloud data and the corresponding complete point cloud data, and the loss value is used to update the parameters of the point cloud data completion model to obtain the trained point cloud data completion model.

[0033] In a second aspect, an embodiment of the present application further provides a feature enhancement device for point cloud data, wherein the feature enhancement device for point cloud data comprises:

[0034] An acquisition unit, used to acquire feature information of training sample data of a point cloud data completion model, wherein the feature information of the training sample data includes feature information of incomplete point cloud data and feature information of corresponding complete point cloud data;

[0035] A storage unit, used to store the feature information of the incomplete point cloud data and the feature information of the complete point cloud data in a feature information memory module of the point cloud data completion model;

[0036] The feature enhancement processing unit is used to cluster the feature information of the incomplete point cloud data and the feature information of the complete point cloud data stored in the feature information memory module using an adaptive clustering strategy to obtain a clustering result of the point cloud data as feature enhancement information of the point cloud data, wherein the clustering result of the point cloud data includes a subclass clustering result of the point cloud data.

[0037] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0038] Processor; and

[0039] A memory arranged to store computer executable instructions which, when executed, cause the processor to perform any of the methods described above.

[0040] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple application programs, the electronic device executes any of the aforementioned methods.

[0041] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: the feature enhancement method of point cloud data in the embodiments of the present application first obtains the feature information of the training sample data of the point cloud data completion model, and the feature information of the training sample data includes the feature information of the incomplete point cloud data and the feature information of the corresponding complete point cloud data; then the feature information of the incomplete point cloud data and the feature information of the complete point cloud data are stored in the feature information memory module of the point cloud data completion model; finally, the feature information of the incomplete point cloud data and the feature information of the complete point cloud data stored in the feature information memory module are clustered using an adaptive clustering strategy to obtain the clustering results of the point cloud data as the feature enhancement information of the point cloud data, and the clustering results of the point cloud data include the subclass clustering results of the point cloud data. The feature enhancement method of point cloud data in the embodiments of the present application adopts an adaptive clustering strategy to realize the division of point cloud feature subcategories, thereby more effectively providing additional and stronger supervisory signals for the training of the point cloud data completion model, enhancing the model's ability to characterize point cloud features, and improving the model's completion effect on incomplete point cloud data. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0043] Figure 1 A schematic diagram of a process flow of a feature enhancement method for point cloud data in an embodiment of the present application;

[0044] Figure 2 A schematic diagram of a training process of a point cloud data completion model in an embodiment of the present application;

[0045] Figure 3 This is a schematic diagram of the structure of a feature enhancement device for point cloud data in an embodiment of the present application;

[0046] Figure 4 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0048] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0049] The present application embodiment provides a method for enhancing the features of point cloud data, such as Figure 1 As shown, a flow chart of a method for enhancing the features of point cloud data in an embodiment of the present application is provided, and the method for enhancing the features of point cloud data at least includes the following steps S110 to S130:

[0050] Step S110, acquiring feature information of training sample data of the point cloud data completion model, wherein the feature information of the training sample data includes feature information of incomplete point cloud data and feature information of corresponding complete point cloud data.

[0051] The feature enhancement method of point cloud data in the embodiment of the present application is mainly used to provide feature enhancement information for the training process of the point cloud data completion model as an additional supervisory signal to improve the training effect of the model. Therefore, it is necessary to first obtain the feature information of the training sample data of the point cloud data completion model. The training sample data specifically includes a sample pair consisting of incomplete point cloud data and corresponding complete point cloud data. The incomplete point cloud data refers to data with missing point clouds. As the input of the training point cloud data completion model, the complete point cloud data is the true value corresponding to the incomplete point cloud data, which is used to provide a supervisory signal for model training. In addition to the original point cloud data, the complete point cloud data will also be annotated with some basic feature information in advance, such as the category of the point cloud data. The categories here are mainly relatively coarse-grained categories, and there are no true value labels for the subclasses. The feature information of incomplete point cloud data can be obtained by feature extraction and other methods. The specific method of obtaining the training sample data and its feature data can be flexibly determined by those skilled in the art in combination with the prior art, and no specific limitation is made here.

[0052] Step S120: storing the feature information of the incomplete point cloud data and the feature information of the complete point cloud data in a feature information memory module of the point cloud data completion model.

[0053] Considering that the complete point cloud data truth value is generally only labeled with the large category truth value label, and no sub-category truth value label is marked, the existing category feature information is not enough for the training accuracy requirements of the point cloud data completion model. Based on this, the embodiment of the present application additionally sets a feature information memory module based on the network structure of the existing point cloud data completion model. The main function of the feature information memory module is to store and process the feature information of incomplete point cloud data and the feature information of complete point cloud data, thereby providing more detailed sub-category feature information of point cloud data, and updating it as data is continuously introduced.

[0054] Step S130, using an adaptive clustering strategy to cluster the feature information of the incomplete point cloud data and the feature information of the complete point cloud data stored in the feature information memory module to obtain a clustering result of the point cloud data as feature enhancement information of the point cloud data, wherein the clustering result of the point cloud data includes a subclass clustering result of the point cloud data.

[0055] Normally, when completing point cloud data, the point cloud data completion model needs to fit a series of features of the point cloud data, such as the number of point clouds, point cloud density, and categories. The feature information memory module set in the embodiment of the present application is precisely to enhance the supervision signal in the model training process and strengthen the connection between incomplete point cloud data and true value data. Category information is an important feature of point cloud data. Effectively establishing the connection between the category information of incomplete point clouds and complete point clouds is very important to improve model performance. An inappropriate number of subclasses will hinder the model's fitting of the data distribution. Manually set subclasses are often not accurate enough and may damage model performance.

[0056] Based on this, the embodiment of the present application further introduces an adaptive subclass clustering strategy in the feature information memory module, and uses the adaptive clustering strategy to cluster the feature information of the stored incomplete point cloud data and the feature information of the complete point cloud data, thereby clustering the point cloud data with the same or similar features into more detailed subcategories, enhancing the embedding of point cloud information, reducing intra-class deviations, and providing more effective correlation information between point cloud features for model training.

[0057] It should be noted here that since the true value of the complete point cloud data does not provide the true value label of the subcategory, the ultimate purpose of the above-mentioned clustering using the feature information memory module is not to determine the specific subcategory of the point cloud data, but to provide additional stronger supervision signals for model training by capturing the correlation information between features.

[0058] The feature enhancement method for point cloud data in the embodiment of the present application adopts an adaptive clustering strategy to realize the division of point cloud feature subcategories, thereby more effectively providing additional and stronger supervision signals for the training of the point cloud data completion model, enhancing the model's ability to represent point cloud features, and improving the model's completion effect on incomplete point cloud data.

[0059] In some embodiments of the present application, the feature information of the incomplete point cloud data is obtained in the following manner: obtaining training sample data of a point cloud data completion model, the training sample data including incomplete point cloud data; processing the incomplete point cloud data using a category compilation module of the point cloud data completion model to obtain the feature information of the incomplete point cloud data output by the category compilation module.

[0060] In order to provide feature information of incomplete point cloud data, the point cloud data completion model of the embodiment of the present application is additionally provided with a category compilation module, which uses the category compilation module to perform additional category compilation processing on the incomplete point cloud data, thereby obtaining feature information such as the category of the incomplete point cloud data and storing it in the feature information memory module for clustering processing by the feature information memory module.

[0061] The category compilation module of the embodiment of the present application can be implemented by using algorithms such as SVM (Support Vector Machine). The specific form of the category compilation algorithm can be flexibly selected by technicians in this field according to actual needs, and no specific limitation is made here.

[0062] In some embodiments of the present application, the feature information of the incomplete point cloud data and the feature information of the complete point cloud data stored in the feature information memory module are clustered using an adaptive clustering strategy to obtain a clustering result of the point cloud data, including: for each feature information stored in the feature information memory module, positive and negative samples are divided into positive and negative samples using a preset positive and negative sample division strategy to obtain a positive and negative sample division result; and the clustering result of the point cloud data is determined based on the positive and negative sample division result.

[0063] Since there is no label of the true value of the subclass of the point cloud data, the embodiment of the present application adopts an implicit method to perform subclass memory clustering, shortening the distance between features of the same class and increasing the distance between features of different classes. In this way, subclass memory features can be adaptively clustered together to achieve the ultimate goal of adaptive subclass clustering.

[0064] Specifically, in order to shorten the distance between sample features within a class and increase the distance between sample features between classes, the embodiment of the present application can first adopt a preset positive and negative sample division strategy to divide each feature information stored in the feature information memory module into positive and negative samples, clarify the positive sample set S+ and negative sample set S- corresponding to each feature sample, and then adaptively cluster the feature information through the positive and negative sample division.

[0065] It should be noted that the above positive and negative samples are a relative concept. Here is a simple example. Assume that the feature information memory module currently stores four feature information a1, a2, a3, and a4. Let the currently processed feature a1 be a positive sample. Then the feature information a2, a3, and a4 other than a1 can be regarded as negative samples. If the currently processed feature a2 is a positive sample, then the feature information a1, a3, and a4 other than a2 can be regarded as negative samples. Of course, the specific strategy for dividing positive and negative samples can be determined by further considering the distance between features.

[0066] In some embodiments of the present application, each feature information stored in the feature information memory module is divided into positive and negative samples using a preset positive and negative sample division strategy to obtain the positive and negative sample division results, including: taking any one feature information stored in the feature information memory module as an anchor point, and determining the similarity between the feature information of the anchor point and the remaining feature information stored in the feature information memory module; determining the feature information most similar to the feature information of the anchor point based on the similarity; taking the feature information most similar to the feature information of the anchor point as the basic positive sample, and performing positive and negative sample division on the remaining feature information stored in the feature information memory module to obtain the positive and negative sample division results corresponding to the feature information of the anchor point.

[0067] Each memory feature (m) in the feature information memory module is used as an anchor point, and the remaining features (f r ) uses algorithms such as cosine similarity to find the point cloud feature (f) that is closest to feature m. s ), as shown in the following formula (1):

[0068]

[0069] Take fs as the basic positive sample, divide the remaining feature samples into positive and negative samples, and obtain the positive and negative sample division results corresponding to the feature information of the anchor point.

[0070] It should be noted that the specific algorithm to be adopted to measure the distance between features can be flexibly selected by those skilled in the art in combination with the existing technology, and is not specifically limited here.

[0071] In some embodiments of the present application, the feature information most similar to the feature information of the anchor point is used as a basic positive sample, and the remaining feature information stored in the feature information memory module is divided into positive and negative samples to obtain the positive and negative sample division results corresponding to the feature information of the anchor point, including: determining whether the feature information of the anchor point and the basic positive sample are the most similar feature information to each other, and whether the remaining feature information and the basic positive sample have the same category label; if so, determining that the remaining feature information is a positive sample; otherwise, determining that the remaining feature information is a negative sample.

[0072] In the embodiment of the present application, when dividing positive and negative samples, the division can be performed based on the following two conditions:

[0073] 1)f s Whether and m are the most similar features to each other;

[0074] 2)f r and f s Whether there is the same category label, as shown in the following formula (2):

[0075]

[0076] Among them, c r is the category label of fr, c s is the category label of fs.

[0077] If the above two conditions are met at the same time, it means that the sample is similar to the current feature sample m, and the sample can be divided into the positive sample set corresponding to the feature sample m. After searching all positive samples, the positive sample set is constructed, and the remaining samples constitute the negative sample set. Similarly, the above processing can be performed for each memory feature entering the feature information memory module, thereby continuously updating the association information between the features memorized in the feature information memory module.

[0078] This adaptive method can better improve the division of point cloud feature subcategories, provide more effective supervision signals for the model, enhance the model's ability to fit point cloud data, and improve the model's ability to complete incomplete point cloud data.

[0079] In some embodiments of the present application, after clustering the feature information of the incomplete point cloud data and the feature information of the complete point cloud data stored in the feature information memory module using an adaptive clustering strategy, the feature enhancement method of the point cloud data also includes: using the feature information of the incomplete point cloud data output by the category compilation module as the first feature enhancement information of the point cloud data, and using the clustering result of the point cloud data as the second feature enhancement information of the point cloud data; training the point cloud data completion model according to the training sample data of the point cloud data completion model, the first feature enhancement information and the second feature enhancement information of the point cloud data, to obtain a trained point cloud data completion model.

[0080] Since the training process of the point cloud data completion model usually uses back-propagation for iterative training, the true value information as a supervisory signal will gradually weaken during this process. Therefore, relying solely on this supervisory signal for training will ultimately lead to poor model training results and make it difficult to meet the requirements of point cloud data completion.

[0081] Since the clustering results of point cloud data include the clustering results of subcategories, the clustering results of subcategories reflect the more fine-grained associations between all point cloud data. Therefore, the embodiment of the present application can use this association information as a feature enhancement information and as an additional supervisory signal to supervise the training of the model and improve the model's ability to characterize point cloud features. In addition, the feature information of the incomplete point cloud data output by the category compilation module can be used not only for storage in the feature information memory module for clustering, but also directly as a feature enhancement information and as an additional supervisory signal for model training.

[0082] Therefore, the category compilation module and feature information memory module of the embodiment of the present application can be regarded as two independent modules, both of which can provide additional supervisory signals for the training of the point cloud data completion model. When training the point cloud data completion model, the feature enhancement information output by these two modules can be used as additional supervisory signals to enhance the model training, thereby improving the training effect of the model. In addition, since the two modules provide feature enhancement information independently of each other, the use of the feature enhancement information provided by these two modules for supervised training can also have the effect of mutual verification and constraint, because the output information of a single module is not 100% correct.

[0083] In some embodiments of the present application, the point cloud data completion model is trained according to the training sample data of the point cloud data completion model, the first feature enhancement information and the second feature enhancement information of the point cloud data to obtain the trained point cloud data completion model, including: using the encoder of the point cloud data completion model to process incomplete point cloud data to obtain the encoded features output by the encoder; fusing the first feature enhancement information and the second feature enhancement information of the point cloud data with the encoded features output by the encoder to obtain fused features; inputting the fused features into the decoder of the point cloud data completion model to obtain the decoded features of the decoder; upsampling the decoded features of the decoder to obtain upsampled point cloud data; determining the loss value of the point cloud data completion model according to the upsampled point cloud data and the corresponding complete point cloud data, and using the loss value to update the parameters of the point cloud data completion model to obtain the trained point cloud data completion model.

[0084] In order to facilitate the understanding of the above embodiments, Figure 2 As shown, a training flow diagram of a point cloud data completion model in an embodiment of the present application is provided. The basic network structure of the point cloud data completion model in the embodiment of the present application can be implemented using the existing Encoder (encoder) -Maxpool (maximum pool) -Decoder (decoder) network structure, on this basis, two feature enhancement modules, a class compilation module Class tokenizer and a feature information memory module Adaptive MemoryBank, are additionally set.

[0085] During training, the incomplete point cloud data is input into the category compilation module for processing, and the feature information of the incomplete point cloud data is obtained as the first feature enhancement information. At the same time, the feature information of the incomplete point cloud data and the feature information of the complete point cloud data are input into the feature information memory module. This process can be regarded as an Assimilation process. The feature information memory module processes the feature information of the incomplete point cloud data and the feature information of the complete point cloud data to obtain the second feature enhancement information. In addition, the incomplete point cloud data is also input into the encoder and the maximum pooling layer in turn to obtain the encoded feature Latent Feature of the incomplete point cloud data.

[0086] Then, the first feature enhancement information and the second feature enhancement information are used as additional supervisory signals to perform feature fusion with the encoding features of the incomplete point cloud data, and the extracted incomplete point cloud features are fine-tuned to improve the feature representation ability of the encoder, thereby improving the reconstruction ability of the model. This process can be regarded as an Accommodation process.

[0087] The above fused features are then input into the decoder and upsampled to obtain the upsampled point cloud data.

[0088] Finally, the loss value of the model is calculated based on the upsampled point cloud and the true value of the corresponding complete point cloud data, and the network parameters of the point cloud data completion model are updated until the model converges, and the trained point cloud data completion model is output.

[0089] To summarize, in order to reduce the information gap between incomplete point cloud data and complete point cloud data, the embodiment of the present application proposes an Assimilation-Accommodation mechanism, and deploys a category compilation module and a feature information memory module, which enhances the supervision signal of the model, more effectively utilizes the information of incomplete point cloud and complete point cloud, improves the model's mining efficiency for incomplete point cloud data and the model's reconstruction ability, thereby optimizing the completion effect of incomplete point cloud data and reducing the cost of point cloud data annotation.

[0090] The present application embodiment also provides a feature enhancement device 300 for point cloud data, such as Figure 3 As shown, a schematic diagram of the structure of a feature enhancement device for point cloud data in an embodiment of the present application is provided, wherein the feature enhancement device 300 for point cloud data comprises: an acquisition unit 310, a storage unit 320 and a feature enhancement processing unit 330, wherein:

[0091] An acquisition unit 310 is used to acquire feature information of training sample data of a point cloud data completion model, wherein the feature information of the training sample data includes feature information of incomplete point cloud data and feature information of corresponding complete point cloud data;

[0092] A storage unit 320, used to store the feature information of the incomplete point cloud data and the feature information of the complete point cloud data in a feature information memory module of the point cloud data completion model;

[0093] The feature enhancement processing unit 330 is used to cluster the feature information of the incomplete point cloud data and the feature information of the complete point cloud data stored in the feature information memory module using an adaptive clustering strategy to obtain a clustering result of the point cloud data as feature enhancement information of the point cloud data. The clustering result of the point cloud data includes a subclass clustering result of the point cloud data.

[0094] In some embodiments of the present application, the feature information of the incomplete point cloud data is obtained in the following manner: obtaining training sample data of a point cloud data completion model, the training sample data including incomplete point cloud data; processing the incomplete point cloud data using a category compilation module of the point cloud data completion model to obtain the feature information of the incomplete point cloud data output by the category compilation module.

[0095] In some embodiments of the present application, the feature enhancement processing unit 330 is specifically used to: perform positive and negative sample division on each feature information stored in the feature information memory module using a preset positive and negative sample division strategy to obtain positive and negative sample division results; and determine the clustering results of the point cloud data based on the positive and negative sample division results.

[0096] In some embodiments of the present application, the feature enhancement processing unit 330 is specifically used to: take any one feature information stored in the feature information memory module as an anchor point, determine the similarity between the feature information of the anchor point and the remaining feature information stored in the feature information memory module; determine the feature information that is most similar to the feature information of the anchor point according to the similarity; take the feature information that is most similar to the feature information of the anchor point as the basic positive sample, perform positive and negative sample division on the remaining feature information stored in the feature information memory module, and obtain the positive and negative sample division result corresponding to the feature information of the anchor point.

[0097] In some embodiments of the present application, the feature enhancement processing unit 330 is specifically used to: determine whether the feature information of the anchor point and the basic positive sample are the most similar feature information to each other, and whether the remaining feature information and the basic positive sample have the same category label; if so, determine that the remaining feature information is a positive sample; otherwise, determine that the remaining feature information is a negative sample.

[0098] In some embodiments of the present application, the feature enhancement device for point cloud data also includes: a training unit, which is used to cluster the feature information of the incomplete point cloud data and the feature information of the complete point cloud data stored in the feature information memory module using an adaptive clustering strategy, and then use the feature information of the incomplete point cloud data output by the category compilation module as the first feature enhancement information of the point cloud data, and use the clustering result of the point cloud data as the second feature enhancement information of the point cloud data; train the point cloud data completion model according to the training sample data of the point cloud data completion model, the first feature enhancement information and the second feature enhancement information of the point cloud data, to obtain the trained point cloud data completion model.

[0099] In some embodiments of the present application, the training unit is specifically used to: use the encoder of the point cloud data completion model to process incomplete point cloud data to obtain encoded features output by the encoder; fuse the first feature enhancement information and the second feature enhancement information of the point cloud data with the encoded features output by the encoder to obtain fused features; input the fused features into the decoder of the point cloud data completion model to obtain decoded features of the decoder; upsample the decoded features of the decoder to obtain upsampled point cloud data; determine the loss value of the point cloud data completion model based on the upsampled point cloud data and the corresponding complete point cloud data, and use the loss value to update the parameters of the point cloud data completion model to obtain a trained point cloud data completion model.

[0100] It can be understood that the above-mentioned feature enhancement device for point cloud data can implement the various steps of the feature enhancement method for point cloud data provided in the aforementioned embodiments, and the relevant explanations on the feature enhancement method for point cloud data are all applicable to the feature enhancement device for point cloud data, which will not be repeated here.

[0101] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 4 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0102] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0103] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0104] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a feature enhancement device for point cloud data at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0105] Acquire feature information of training sample data of a point cloud data completion model, wherein the feature information of the training sample data includes feature information of incomplete point cloud data and feature information of corresponding complete point cloud data;

[0106] The feature information of the incomplete point cloud data and the feature information of the complete point cloud data are stored in a feature information memory module of the point cloud data completion model;

[0107] An adaptive clustering strategy is used to cluster the feature information of the incomplete point cloud data and the feature information of the complete point cloud data stored in the feature information memory module to obtain a clustering result of the point cloud data as feature enhancement information of the point cloud data. The clustering result of the point cloud data includes a subclass clustering result of the point cloud data.

[0108] The above application Figure 1The method performed by the feature enhancement device of point cloud data disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software. The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in a decoding processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0109] The present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 1 The method performed by the feature enhancement device of point cloud data in the embodiment shown is specifically used to perform:

[0110] Acquire feature information of training sample data of a point cloud data completion model, wherein the feature information of the training sample data includes feature information of incomplete point cloud data and feature information of corresponding complete point cloud data;

[0111] The feature information of the incomplete point cloud data and the feature information of the complete point cloud data are stored in a feature information memory module of the point cloud data completion model;

[0112] An adaptive clustering strategy is used to cluster the feature information of the incomplete point cloud data and the feature information of the complete point cloud data stored in the feature information memory module to obtain a clustering result of the point cloud data as feature enhancement information of the point cloud data. The clustering result of the point cloud data includes a subclass clustering result of the point cloud data.

[0113] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0114] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0115] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0116] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0117] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0118] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0119] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0120] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0121] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0122] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A feature enhancement method for point cloud data, in, The feature enhancement method of the point cloud data comprises: Acquire feature information of training sample data of a point cloud data completion model, wherein the feature information of the training sample data includes feature information of incomplete point cloud data and feature information of corresponding complete point cloud data; The feature information of the incomplete point cloud data and the feature information of the complete point cloud data are stored in a feature information memory module of the point cloud data completion model; An adaptive clustering strategy is used to cluster the feature information of the incomplete point cloud data and the feature information of the complete point cloud data stored in the feature information memory module to obtain a clustering result of the point cloud data as feature enhancement information of the point cloud data. The clustering result of the point cloud data includes a subclass clustering result of the point cloud data.

2. The feature enhancement method of point cloud data according to claim 1, in, The feature information of the incomplete point cloud data is obtained in the following manner: Acquire training sample data of a point cloud data completion model, wherein the training sample data includes incomplete point cloud data; The incomplete point cloud data is processed using the category compilation module of the point cloud data completion model to obtain feature information of the incomplete point cloud data output by the category compilation module.

3. The feature enhancement method of point cloud data according to claim 1, in, The adaptive clustering strategy is used to perform clustering processing on the feature information of the incomplete point cloud data and the feature information of the complete point cloud data stored in the feature information memory module to obtain the clustering result of the point cloud data, including: For each feature information stored in the feature information memory module, respectively, a positive and negative sample division is performed using a preset positive and negative sample division strategy to obtain a positive and negative sample division result; The clustering result of the point cloud data is determined according to the positive and negative sample division result.

4. The feature enhancement method of point cloud data according to claim 3, in, The step of dividing each feature information stored in the feature information memory module into positive and negative samples using a preset positive and negative sample division strategy to obtain a positive and negative sample division result includes: Taking any one of the feature information stored in the feature information memory module as an anchor point, determining the similarity between the feature information of the anchor point and the remaining feature information stored in the feature information memory module; Determining feature information most similar to the feature information of the anchor point according to the similarity; The feature information most similar to the feature information of the anchor point is used as a basic positive sample, and the remaining feature information stored in the feature information memory module is divided into positive and negative samples to obtain a positive and negative sample division result corresponding to the feature information of the anchor point.

5. The feature enhancement method of point cloud data according to claim 4, in, The step of taking the feature information most similar to the feature information of the anchor point as a basic positive sample, dividing the remaining feature information stored in the feature information memory module into positive and negative samples, and obtaining the positive and negative sample division result corresponding to the feature information of the anchor point includes: Determine whether the feature information of the anchor point and the basic positive sample are the most similar feature information to each other, and whether the remaining feature information and the basic positive sample have the same category label; If yes, determining that the remaining feature information is a positive sample; Otherwise, it is determined that the remaining feature information is a negative sample.

6. The feature enhancement method of point cloud data according to claim 2, in, After clustering the feature information of the incomplete point cloud data and the feature information of the complete point cloud data stored in the feature information memory module using an adaptive clustering strategy, the feature enhancement method of the point cloud data further includes: Using the feature information of the incomplete point cloud data output by the category compilation module as first feature enhancement information of the point cloud data, and using the clustering result of the point cloud data as second feature enhancement information of the point cloud data; The point cloud data completion model is trained according to the training sample data of the point cloud data completion model, the first feature enhancement information and the second feature enhancement information of the point cloud data, to obtain a trained point cloud data completion model.

7. The feature enhancement method of point cloud data according to claim 6, in, The step of training the point cloud data completion model according to the training sample data of the point cloud data completion model, the first feature enhancement information and the second feature enhancement information of the point cloud data to obtain the trained point cloud data completion model comprises: Processing the incomplete point cloud data using the encoder of the point cloud data completion model to obtain encoding features output by the encoder; Fusing the first feature enhancement information and the second feature enhancement information of the point cloud data with the encoding feature output by the encoder to obtain a fused feature; Inputting the fused features into a decoder of the point cloud data completion model to obtain decoding features of the decoder; Upsampling the decoded features of the decoder to obtain upsampled point cloud data; The loss value of the point cloud data completion model is determined according to the upsampled point cloud data and the corresponding complete point cloud data, and the loss value is used to update the parameters of the point cloud data completion model to obtain the trained point cloud data completion model.

8. A feature enhancement device for point cloud data, in, The feature enhancement device of the point cloud data comprises: An acquisition unit, used to acquire feature information of training sample data of a point cloud data completion model, wherein the feature information of the training sample data includes feature information of incomplete point cloud data and feature information of corresponding complete point cloud data; A storage unit, used to store the feature information of the incomplete point cloud data and the feature information of the complete point cloud data in a feature information memory module of the point cloud data completion model; The feature enhancement processing unit is used to cluster the feature information of the incomplete point cloud data and the feature information of the complete point cloud data stored in the feature information memory module using an adaptive clustering strategy to obtain a clustering result of the point cloud data as feature enhancement information of the point cloud data, wherein the clustering result of the point cloud data includes a subclass clustering result of the point cloud data.

9. An electronic device, include: processor; as well as A memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor executes the feature enhancement method for point cloud data according to any one of claims 1 to 7.

10. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, enables the electronic device to execute the feature enhancement method of point cloud data of any one of claims 1 to 7.