Label iteration correction method and device of three-dimensional point cloud, medium and equipment

Abstract

The embodiment of the invention provides a three-dimensional point cloud label iteration correction method and device, a medium and equipment. The method comprises the following steps: performing at least one round of label iteration correction on an original point cloud by adopting a plurality of trained verifiers to obtain a target point cloud after label correction; wherein in each round of label iterative correction, each point in the original point cloud and a correction label determined in the previous round are respectively input into a plurality of verifiers, so that each verifier respectively outputs a judgment result that each point in the original point cloud is a clean sample or a polluted sample; and for the points which are determined to be polluted samples by the verifiers, respectively inputting the polluted samples into the deep neural networks contained in the verifiers, so that the deep neural networks output corresponding prediction labels to determine corresponding correction labels of the polluted samples in the current round. According to the technical scheme of the embodiment of the invention, the pollution label samples in the point cloud data set can be accurately separated and corrected, and the effectiveness of subsequent neural network training is ensured.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, medium and equipment for iterative label correction of a three-dimensional point cloud. Background Art

[0002] Semantic understanding of 3D point clouds is an important research direction in the 3D field. Although existing 3D deep neural networks have gradually achieved higher performance in the field of deep learning, especially in the fields of convolutional neural networks (CNNs), graph neural networks (GNNs) and deep neural networks (DNNs), a major factor in the effectiveness of these neural networks is the use of high-precision, carefully annotated and labeled data.

[0003] However, due to the accuracy limitations of the equipment used, obstacles or other environmental disturbances, noise will be introduced into the data set, making it difficult to obtain a high-quality point cloud data set. In the current technical solution, the method of removing outliers or noise points is often used to remove noise. Although it has a certain effect, excessive noise point discarding will lead to a large amount of data loss, and selective removal of all noise points will still make the learning process affected by the remaining noise data.

[0004] Therefore, how to accurately separate and correct the contaminated label samples in the point cloud dataset to ensure the effectiveness of subsequent neural network training has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The embodiments of the present application provide a method, apparatus, medium and device for iterative label correction of a three-dimensional point cloud, which can accurately separate and correct contaminated label samples in a point cloud data set to at least a certain extent, thereby ensuring the effectiveness of subsequent neural network training.

[0006] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0007] According to one aspect of an embodiment of the present application, a method for iterative label correction of a three-dimensional point cloud is provided, which uses a number of trained verifiers to perform at least one round of iterative label correction on an original point cloud to obtain a target point cloud after label correction;

[0008] In each round of label iterative correction, the following steps are included:

[0009] Input each point in the original point cloud and the correction label determined in the previous round into a plurality of the verifiers, so that each of the verifiers outputs a judgment result of whether each point in the original point cloud is a clean sample or a contaminated sample;

[0010] For the points determined as contaminated samples by each of the verifiers, the contaminated samples are respectively input into the deep neural networks included in each of the verifiers, so that each of the deep neural networks outputs a predicted label corresponding to the contaminated samples;

[0011] According to the quantitative relationship between the prediction labels of different categories corresponding to the contaminated samples, the correction labels corresponding to the contaminated samples in the current round are determined.

[0012] According to one aspect of an embodiment of the present application, a label iterative correction device for a three-dimensional point cloud is provided, the device being configured to: use a plurality of trained verifiers to perform at least one round of label iterative correction on an original point cloud to obtain a target point cloud after label correction;

[0013] In each round of label iterative correction, the following steps are included:

[0014] Input each point in the original point cloud and the correction label determined in the previous round into a plurality of the verifiers, so that each of the verifiers outputs a judgment result of whether each point in the original point cloud is a clean sample or a contaminated sample;

[0015] For the points determined as contaminated samples by each of the verifiers, the contaminated samples are respectively input into the deep neural networks included in each of the verifiers, so that each of the deep neural networks outputs a predicted label corresponding to the contaminated samples;

[0016] According to the quantitative relationship between the prediction labels of different categories corresponding to the contaminated samples, the correction labels corresponding to the contaminated samples in the current round are determined.

[0017] According to one aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for iterative label correction of a three-dimensional point cloud as described in the above embodiment is implemented.

[0018] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for iterative label correction of a three-dimensional point cloud as described in the above-mentioned embodiments.

[0019] According to one aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the label iterative correction method for a three-dimensional point cloud provided in the above embodiment.

[0020] In the technical solutions provided in some embodiments of the present application, several trained verifiers are used to perform at least one round of label iteration correction on the original point cloud to obtain a target point cloud after label correction, wherein, in each round of label iteration correction, each point in the original point cloud and its correction label determined in the previous round are respectively input into several verifiers, so that each verifier outputs the judgment result of whether each point in the original point cloud is a clean sample or a contaminated sample, and then, for the points determined as contaminated samples by each verifier, the contaminated samples are respectively input into the deep neural network contained in each verifier, so that each deep neural network outputs the predicted label corresponding to the contaminated sample, and then the corrected label corresponding to the contaminated sample in the current round is determined according to the quantitative relationship between the predicted labels of different categories corresponding to the contaminated samples. Therefore, by collaboratively identifying and correcting the contaminated samples in the original point cloud by multiple verifiers, not only the accuracy of contaminated sample identification can be improved, but also the accuracy of the labels after each correction can be guaranteed, thereby ensuring the effectiveness of subsequent neural network training.

[0021] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0023] Figure 1 A schematic diagram of the process of each round of iterative label correction according to an embodiment of the present application is shown;

[0024] Figure 2 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0025] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0026] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0027] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0028] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0029] The embodiment of the present application provides a method for iterative label correction of a three-dimensional point cloud, which can be applied to a terminal device or a server, wherein the terminal device may include but is not limited to one or more of a smart phone, a tablet computer, a portable computer, and a desktop computer, and the server may be a physical server or a cloud server. The following description is made by taking the application of the method to a terminal device as an example, hereinafter referred to as a "terminal".

[0030] In one embodiment, the label iterative correction method of the three-dimensional point cloud uses a plurality of trained verifiers to perform at least one round of label iterative correction on the original point cloud, thereby obtaining a target point cloud after label correction.

[0031] In this embodiment, the verifier can be a model used to verify whether the label corresponding to a point in the point cloud is a contaminated label. The terminal can call several pre-trained verifiers to perform at least one round of label iterative correction on the original point cloud, thereby separating and correcting the contaminated samples in the original point cloud.

[0032] It should be understood that points with contaminated labels may be inaccurately or incorrectly labeled points due to incorrect labeling, noise interference or other reasons. For example, some points in the point cloud are incorrectly labeled with category labels, such as a point cloud representing a tree, but is incorrectly labeled as a building; or during the data collection process, due to the accuracy limitations of the sensor, environmental factors or other technical problems, the point cloud data may contain noise, and these noise points may be incorrectly interpreted as having certain characteristics or belonging to a certain category, etc.

[0033] Thus, the original point cloud is subjected to multiple rounds of label iteration correction through several verifiers, so that the label information of the points in the original point cloud can be corrected to obtain a high-quality target point cloud after label correction. Researchers can subsequently use the target point cloud obtained by the above method to train the neural network (such as point cloud classification or semantic segmentation, etc.) to ensure the training effect.

[0034] In one embodiment, if Figure 1 As shown, in each round of label iterative correction, it includes:

[0035] S110, inputting each point in the original point cloud and the correction label determined in the previous round into a plurality of the verifiers, so that each of the verifiers outputs a judgment result of whether each point in the original point cloud is a clean sample or a contaminated sample;

[0036] S120, for the points determined as contaminated samples by each of the verifiers, inputting the contaminated samples into the deep neural networks included in each of the verifiers, so that each of the deep neural networks outputs a predicted label corresponding to the contaminated samples;

[0037] S130, determining the correction label corresponding to the contaminated sample in the current round according to the quantitative relationship between the prediction labels of different categories corresponding to the contaminated sample.

[0038] In this embodiment, the original point cloud may include several points and their initial labels. When the terminal performs each round of label iterative correction, each point in the original point cloud (such as the coordinate information and RGB information of the point) and the correction label determined in the previous round (if it is the first round of label iterative correction, it is the initial label) may be input into several verifiers respectively, so that each verifier outputs the judgment result of whether each point in the original point cloud is a clean sample or a contaminated sample. Among them, a clean sample indicates that the current label of the point is likely to be correct, and a contaminated sample indicates that the current label of the point is likely to be wrong.

[0039] Each verifier judges the points in the original point cloud respectively. Therefore, the judgment results output by each verifier may be the same or different. For example, a point is judged as a clean sample by verifier 1, while the point is judged as a contaminated sample by verifier 2, etc.

[0040] According to the judgment results output by each verifier, the terminal can perform label correction on each point. Specifically, for the points determined as contaminated samples by each verifier, the terminal can input the contaminated samples into the deep neural networks included in each verifier, so that each deep neural network outputs the predicted label corresponding to the contaminated sample. In other words, the verifier can include a deep neural network, which can predict the label of the point based on its attribute information and output the corresponding predicted label.

[0041] It should be understood that the prediction labels output by each deep neural network may be the same or different. The more times a certain category of prediction labels appears, the more likely it is that the contaminated sample corresponds to the prediction label of that category. Therefore, the terminal can determine the correction label corresponding to the contaminated sample in the current round based on the quantitative relationship between the prediction labels of different categories corresponding to the contaminated sample.

[0042] In one embodiment of the present application, determining the correction label corresponding to the contaminated sample in the current round according to the quantitative relationship between the predicted labels of different categories corresponding to the contaminated sample includes:

[0043] If the number of predicted labels of a certain category is greater than the number of predicted labels of other categories, the predicted labels of this category are determined as the correction labels corresponding to the contaminated samples in the current round;

[0044] If there are at least two categories whose number of predicted labels is the same and greater than the number of predicted labels of other categories, randomly select one of the predicted labels of the at least two categories as the correction label corresponding to the contaminated sample in the current round;

[0045] If the number of predicted labels of all categories is 1, the correction label determined for the contaminated sample in the previous round continues to be used as the correction label corresponding to the current round.

[0046] In this embodiment, based on the prediction labels output by each verifier for the same contaminated sample, the terminal can count the number of occurrences of the prediction labels of each category. If the number of prediction labels of a certain category (i.e., the number of occurrences) is greater than the number of prediction labels of other categories, it means that the contaminated sample is more likely to correspond to the prediction label. Therefore, the prediction label of the category with the largest number can be determined as the correction label corresponding to the contaminated sample in the current round.

[0047] If there are at least two categories of predicted labels with the same number of occurrences, and the number of occurrences is greater than the number of occurrences of predicted labels of other categories, the terminal can randomly select a category from the predicted labels of at least two categories with the same number and the largest number as the correction label corresponding to the contaminated sample in the current round.

[0048] If the number of predicted labels for all categories is 1, then the correction label determined for the contaminated sample in the previous round will not be changed, and it will continue to be used as the corresponding correction label in the current round.

[0049] Therefore, by voting on the predicted labels output by each validator to correct the labels corresponding to the contaminated samples, the validity of the determined corrected labels can be guaranteed.

[0050] In one embodiment, if the predicted labels output by each validator are different for a certain contaminated sample, then the contaminated sample will no longer be judged as a clean sample or a contaminated sample in the next round of label iterative correction, but will be directly label corrected (that is, directly input into each deep neural network for label prediction for correction in the next round of label iterative correction) until the label is corrected.

[0051] In one example, the terminal can perform multiple rounds of label iterative correction on the original point cloud through several verifiers until the proportion of the parts classified as clean samples tends to be stable (for example, in several consecutive rounds of label iterative correction, the proportion of clean samples is equal to or close to a certain proportion value). At this time, the terminal can stop the correction, thereby making full use of the data information and correcting the labels to the maximum extent.

[0052] In one embodiment of the present application, it is characterized in that, according to the quantitative relationship between the prediction labels of different categories corresponding to the contaminated samples, determining the correction label corresponding to the contaminated sample in the current round includes:

[0053] Clustering the points determined as contaminated samples by each of the validators to obtain at least one cluster;

[0054] Using the largest number of identical labels corresponding to the points contained in each cluster as the pre-corrected labels for all the points in the cluster;

[0055] According to the pre-correction label corresponding to the contaminated sample and the quantitative relationship between the predicted labels of different categories, the correction label corresponding to the contaminated sample in the current round is determined.

[0056] In this embodiment, it should be noted that for each point determined as a contaminated sample, its current corresponding label (i.e., the initial label or the corrected label determined in the previous round of label iterative correction) can be considered unreliable. Therefore, the terminal can pre-correct the point determined as a contaminated sample. Specifically, the terminal can use a clustering algorithm to cluster the points determined as contaminated samples by each verifier, so as to obtain at least one cluster. For each cluster, the terminal can use a majority voting strategy to update all points in the same cluster to the label with the most votes, that is, the same label with the largest number of points contained in each cluster is used as the pre-corrected label of all points in the cluster.

[0057] It is worth noting that the choice of clustering algorithm is mainly based on the data characteristics of the point cloud dataset. Generally, two clustering methods, K-means and DBSCAN, are used. Among them, the K-means algorithm is based on cluster centroids or partitions, and divides all points into K groups according to the Euclidean distance. The K-means algorithm only requires one parameter, which is the number of classes known in advance. DBSCAN is a density-based clustering algorithm that defines a cluster as the largest set of density-connected points, can identify and exclude noise points, and find clusters of arbitrary shapes. DBSCAN requires two parameters, namely the radius (Eps) and the minimum number of points (MinPts).

[0058] In addition, if the point cloud dataset (ModelNet40 dataset) contains multiple objects with complex geometries, each object has multiple views, this will affect the performance of the DBSCAN algorithm. Therefore, the DBSCAN clustering algorithm is not suitable for such datasets, and K-means is more suitable for processing datasets with complex geometries and noise. In contrast, DBSCAN is suitable for density-based clustering, which is more suitable for datasets from real-world scenes. Therefore, for synthetic datasets such as ModelNet40 and S3DIS where the number of labels is known, the K-means algorithm is more applicable. For real-world datasets such as ScanNetV2, DBSCAN can be used first.

[0059] After determining the pre-correction label corresponding to the contaminated sample, the terminal can determine the correction label corresponding to the contaminated sample in the current round according to the pre-correction label corresponding to the contaminated sample and the quantitative relationship between the prediction labels of different categories. For example, assuming that there are m deep neural networks, each contaminated sample corresponds to m prediction labels plus one pre-correction label, for a total of m+1 labels. The terminal can determine the correction label corresponding to the contaminated sample in the current round according to the voting method for determining the correction label corresponding to the contaminated sample in the current round according to the m+1 labels as described in the above embodiment.

[0060] In one embodiment of the present application, in each round of label iterative correction, the method further includes:

[0061] For the points determined as clean samples by each of the validators, the clean samples are respectively input into the deep neural networks included in other validators, so that each of the deep neural networks outputs the predicted labels corresponding to the clean samples;

[0062] The correction label corresponding to the clean sample in the current round is determined according to the correction label corresponding to the clean sample determined in the previous round and the quantitative relationship between the prediction labels of different categories.

[0063] In this embodiment, in each round of label iterative correction, in addition to performing label correction on points determined to be contaminated samples, the terminal can also perform label correction on points determined to be clean samples. Specifically, the terminal can input the clean samples determined by each verifier into the deep neural networks included in other verifiers, so that each deep neural network outputs the predicted label corresponding to the clean sample.

[0064] The terminal can determine the correction label corresponding to the clean sample in the current round according to the correction label corresponding to the clean sample determined in the previous round and the quantitative relationship between the prediction labels of different categories. It is worth noting that the method used to determine the correction label corresponding to the clean sample in the current round is the same as the method provided in the aforementioned embodiment to determine the correction label corresponding to the contaminated sample in the current round, and this application will not repeat it here.

[0065] Therefore, the iterative label correction method for three-dimensional point clouds provided in the embodiment of the present application not only performs label correction on points judged as contaminated samples, but also performs label correction on points judged as clean samples, thereby ensuring the validity of the target point cloud after label correction.

[0066] In one embodiment of the present application, the validator also includes a Gaussian mixture model (GMM), that is, each validator includes a deep neural network and a Gaussian mixture model. Each deep neural network has the same network architecture, but its respective parameters are updated independently during the training process of the validator, that is, one deep neural network updates its parameters during the training process, while the parameters of other deep neural networks remain unchanged for verification. When the validator is used to distinguish between clean samples and contaminated samples, the cross entropy loss l(θ) can be used to evaluate the degree of fit between the model and the sample, and to determine the loss value between the predicted label of the point output by the deep neural network and the corresponding input label, that is, whether the label of each point is a loss value of a clean label. The cross entropy loss l(θ) formula is as follows:

[0067]

[0068] Among them, pic is the deep neural network prediction probability that input i belongs to category c, y ic is the true label of sample i corresponding to category c.

[0069] Next, the corresponding distribution loss is fitted through the Gaussian mixture model, that is, the Gaussian mixture model is fitted to the loss distribution of each sample to calculate its clean probability τ:

[0070] τ is obtained through GMM, and l(θ) and τ are compared to determine clean samples and contaminated samples. Specifically, if l(θ) < τ, it is considered a clean sample, and if l(θ) ≥ τ, it is considered a contaminated sample. After this process, the trained validator has the ability to calculate the probability of clean labels. When the data set passes through m validators, the m validators output their divided clean samples and contaminated samples respectively.

[0071] In one embodiment of the present application, before training each verifier, the method further includes:

[0072] The deep neural network contained in each of the validators is preheated and trained, and optimized using the loss function shown below:

[0073] L=Loss CE +H

[0074] in, N is the total number of samples, C is the total number of label categories, and y ic is the true label of sample i corresponding to category c, p ic The deep neural network predicts the probability that input i belongs to category c; H = -∑ c p ic (i;θ)log p ic (i;θ), p ic A deep neural network predicts the probability that input i belongs to category c, where θ is a given parameter.

[0075] In this embodiment, based on the study of memory effects in deep learning, a simple pattern can be used for warm-up before learning noisy data, which helps to improve the generalization ability and adversarial robustness of the deep neural network. Warm-up training occurs at the beginning of model training, using labeled data samples to warm up the model. This warm-up phase focuses specifically on reducing the loss of the deep neural network on labeled data, and iteratively trains the deep neural network with gradually increasing periods to ensure that the model has sufficient learning and generalization capabilities before entering a more complex learning phase. In the warm-up training phase, a complete deep learning training process is established, including data preprocessing, forward propagation, loss calculation, back propagation, and parameter update.

[0076] In one example, the warm-up training phase uses a simple mode of the network to warm up for 40 epochs before the verifier training. In the warm-up training phase, only the basic cross entropy loss (CE-loss) is used to measure the difference between the predicted label and the true label. At the same time, for different noise modes, the loss function can be adjusted accordingly to adapt to different noise environments. The formula for the warm-up loss is as follows:

[0077]

[0078] Among them, N is the total number of samples, C is the total number of label categories, and y ic is the true label of sample i corresponding to category c, p ic The deep neural network predicts the probability that input i belongs to category c. Loss CE Label smoothing can be integrated to regularize the samples and make the model more robust to small changes in the input samples.

[0079] For asymmetric label noise, the network will quickly overfit the noise during the warm-up training phase, resulting in most samples having normalized loss close to zero. In this case, the Gaussian mixture model cannot effectively distinguish between clean samples and noisy samples based on the loss distribution. Therefore, during the warm-up training, the network is overfitted to the noise by adjusting the loss distribution. CE A negative entropy term H is added to penalize the network's confidence prediction. The negative entropy term H is defined as follows:

[0080]

[0081] Among them, p ic A deep neural network predicts the probability that input i belongs to category c, where θ is a given parameter.

[0082] Therefore, for asymmetric noise, the loss function L is calculated as:

[0083] L=Loss CE +H

[0084] Moreover, in real scene data sets, since noise is random and diverse, the loss function uses the L calculation method.

[0085] The following describes an embodiment of the device of the present application, which can be used to execute the label iterative correction method of the three-dimensional point cloud in the above embodiment of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the label iterative correction method of the three-dimensional point cloud in the above embodiment of the present application.

[0086] According to an embodiment of the present application, a three-dimensional point cloud label iterative correction device is configured to: use a plurality of trained verifiers to perform at least one round of label iterative correction on an original point cloud to obtain a target point cloud after label correction;

[0087] In each round of label iterative correction, the following steps are included:

[0088] Input each point in the original point cloud and the correction label determined in the previous round into a plurality of the verifiers, so that each of the verifiers outputs a judgment result of whether each point in the original point cloud is a clean sample or a contaminated sample;

[0089] For the points determined as contaminated samples by each of the verifiers, the contaminated samples are respectively input into the deep neural networks included in each of the verifiers, so that each of the deep neural networks outputs a predicted label corresponding to the contaminated samples;

[0090] According to the quantitative relationship between the prediction labels of different categories corresponding to the contaminated samples, the correction labels corresponding to the contaminated samples in the current round are determined.

[0091] In one embodiment of the present application, determining the correction label corresponding to the contaminated sample in the current round according to the quantitative relationship between the predicted labels of different categories corresponding to the contaminated sample includes:

[0092] If the number of predicted labels of a certain category is greater than the number of predicted labels of other categories, the predicted labels of this category are determined as the correction labels corresponding to the contaminated samples in the current round;

[0093] If there are at least two categories whose number of predicted labels is the same and greater than the number of predicted labels of other categories, randomly select one of the predicted labels of the at least two categories as the correction label corresponding to the contaminated sample in the current round;

[0094] If the number of predicted labels of all categories is 1, the correction label determined for the contaminated sample in the previous round continues to be used as the correction label corresponding to the current round.

[0095] In one embodiment of the present application, determining the correction label corresponding to the contaminated sample in the current round according to the quantitative relationship between the predicted labels of different categories corresponding to the contaminated sample includes:

[0096] Clustering the points determined as contaminated samples by each of the validators to obtain at least one cluster;

[0097] Using the largest number of identical labels corresponding to the points contained in each cluster as the pre-corrected labels for all the points in the cluster;

[0098] According to the pre-correction label corresponding to the contaminated sample and the quantitative relationship between the predicted labels of different categories, the correction label corresponding to the contaminated sample in the current round is determined.

[0099] In one embodiment of the present application, in each round of label iterative correction, it also includes:

[0100] For the points determined as clean samples by each of the validators, the clean samples are respectively input into the deep neural networks included in other validators, so that each of the deep neural networks outputs the predicted labels corresponding to the clean samples;

[0101] The correction label corresponding to the clean sample in the current round is determined according to the correction label corresponding to the clean sample determined in the previous round and the quantitative relationship between the prediction labels of different categories.

[0102] In one embodiment of the present application, before training each verifier, the apparatus is further configured to:

[0103] The deep neural network contained in each of the validators is preheated and trained, and optimized using the loss function shown below:

[0104] L=Loss CE +H

[0105] in, N is the total number of samples, C is the total number of label categories, and y ic is the true label of sample i corresponding to category c, p ic The deep neural network predicts the probability that input i belongs to category c; H = -∑ c p ic (i;θ)log p ic (i;θ), p ic A deep neural network predicts the probability that input i belongs to category c, where θ is a given parameter.

[0106] In one embodiment of the present application, the verifier also includes a Gaussian mixture model, which uses cross entropy loss to determine the loss value between the predicted label of the point output by the deep neural network and the corresponding input label, and fits the distribution loss through the Gaussian mixture model to determine the corresponding clean probability. When the loss value is less than the clean probability, the point is determined to be a clean sample.

[0107] Figure 2 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown.

[0108] It should be noted that Figure 2 The computer system of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0109] like Figure 2As shown, the computer system includes a central processing unit (CPU) 201, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 202 or the program loaded from the storage part 208 to the random access memory (RAM) 203, such as executing the method described in the above embodiment. In the RAM 203, various programs and data required for system operation are also stored. The CPU 201, ROM 202 and RAM 203 are connected to each other through a bus 204. An input / output (I / O) interface 205 is also connected to the bus 204.

[0110] The following components are connected to the I / O interface 205: an input section 206 including a keyboard, a mouse, etc.; an output section 207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 208 including a hard disk, etc.; and a communication section 209 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 209 performs communication processing via a network such as the Internet. A drive 210 is also connected to the I / O interface 205 as needed. A removable medium 211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 210 as needed so that a computer program read therefrom is installed into the storage section 208 as needed.

[0111] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 209, and / or installed from a removable medium 211. When the computer program is executed by a central processing unit (CPU) 201, various functions defined in the system of the present application are executed.

[0112] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0113] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0114] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0115] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

[0116] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0117] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0118] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0119] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for iterative label correction of a three-dimensional point cloud, characterized in that: Use several trained verifiers to perform at least one round of label iterative correction on the original point cloud to obtain a target point cloud after label correction; In each round of label iterative correction, the following steps are included: Input each point in the original point cloud and the correction label determined in the previous round into a plurality of the verifiers, so that each of the verifiers outputs a judgment result of whether each point in the original point cloud is a clean sample or a contaminated sample; For the points determined as contaminated samples by each of the verifiers, the contaminated samples are respectively input into the deep neural networks included in each of the verifiers, so that each of the deep neural networks outputs a predicted label corresponding to the contaminated samples; According to the quantitative relationship between the prediction labels of different categories corresponding to the contaminated samples, the correction labels corresponding to the contaminated samples in the current round are determined.

2. The method according to claim 1, characterized in that Determining the correction label corresponding to the contaminated sample in the current round according to the quantitative relationship between the prediction labels of different categories corresponding to the contaminated sample includes: If the number of predicted labels of a certain category is greater than the number of predicted labels of other categories, the predicted labels of this category are determined as the correction labels corresponding to the contaminated samples in the current round; If there are at least two categories whose number of predicted labels is the same and greater than the number of predicted labels of other categories, randomly select one of the predicted labels of the at least two categories as the correction label corresponding to the contaminated sample in the current round; If the number of predicted labels of all categories is 1, the correction label determined for the contaminated sample in the previous round continues to be used as the correction label corresponding to the current round.

3. The method according to claim 1, characterized in that Determining the correction label corresponding to the contaminated sample in the current round according to the quantitative relationship between the prediction labels of different categories corresponding to the contaminated sample includes: Clustering the points determined as contaminated samples by each of the validators to obtain at least one cluster; Using the largest number of identical labels corresponding to the points contained in each cluster as the pre-corrected labels for all the points in the cluster; According to the pre-correction label corresponding to the contaminated sample and the quantitative relationship between the predicted labels of different categories, the correction label corresponding to the contaminated sample in the current round is determined.

4. The method according to claim 1, characterized in that: In each round of label iterative correction, it also includes: For the points determined as clean samples by each of the validators, the clean samples are respectively input into the deep neural networks included in other validators, so that each of the deep neural networks outputs the predicted labels corresponding to the clean samples; The correction label corresponding to the clean sample in the current round is determined according to the correction label corresponding to the clean sample determined in the previous round and the quantitative relationship between the prediction labels of different categories.

5. The method according to claim 1, characterized in that Before training each validator, the method further includes: The deep neural network contained in each of the validators is preheated and trained, and optimized using the loss function shown below: L=Loss CE +H in, N is the total number of samples, C is the total number of label categories, and y ic is the true label of sample i corresponding to category c, p ic The deep neural network predicts the probability that input i belongs to category c; H=-∑ c p ic (i;θ)flog p ic (i;θ), p ic A deep neural network predicts the probability that input i belongs to category c, where θ is a given parameter.

6. The method according to claim 1, characterized in that The verifier also includes a Gaussian mixture model. The verifier uses cross entropy loss to determine the loss value between the predicted label of the point output by the deep neural network and the corresponding input label, and fits the distribution loss through the Gaussian mixture model to determine the corresponding clean probability. When the loss value is less than the clean probability, the point is determined to be a clean sample.

7. A three-dimensional point cloud label iterative correction device, characterized in that: The device is configured to: use a plurality of trained verifiers to perform at least one round of label iterative correction on the original point cloud to obtain a target point cloud after label correction; In each round of label iterative correction, the following steps are included: Input each point in the original point cloud and the correction label determined in the previous round into a plurality of the verifiers, so that each of the verifiers outputs a judgment result of whether each point in the original point cloud is a clean sample or a contaminated sample; For the points determined as contaminated samples by each of the verifiers, the contaminated samples are respectively input into the deep neural networks included in each of the verifiers, so that each of the deep neural networks outputs a predicted label corresponding to the contaminated samples; According to the quantitative relationship between the prediction labels of different categories corresponding to the contaminated samples, the correction labels corresponding to the contaminated samples in the current round are determined.

8. The device according to claim 7, characterized in that Determining the correction label corresponding to the contaminated sample in the current round according to the quantitative relationship between the prediction labels of different categories corresponding to the contaminated sample includes: If the number of predicted labels of a certain category is greater than the number of predicted labels of other categories, the predicted labels of this category are determined as the correction labels corresponding to the contaminated samples in the current round; If there are at least two categories whose number of predicted labels is the same and greater than the number of predicted labels of other categories, randomly select one of the predicted labels of the at least two categories as the correction label corresponding to the contaminated sample in the current round; If the number of predicted labels of all categories is 1, the correction label determined for the contaminated sample in the previous round continues to be used as the correction label corresponding to the current round.

9. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for iterative label correction of a three-dimensional point cloud according to any one of claims 1 to 6 is implemented.

10. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the label iterative correction method for a three-dimensional point cloud as described in any one of claims 1 to 6.

Images