Detection method and device

Abstract

The embodiment of the invention provides a detection method and device in the field of computer vision, and is used for taking less annotation data as supervision data, realizing annotation of unannotated data and realizing detection at lower annotation cost. The method comprises the steps that firstly, a first data set is obtained, and the first data set can comprise unlabeled data; the first data set is used as input of a detection model, a second data set is obtained, the second data set comprises a second label corresponding to the unlabeled data, the detection model is a model obtained by pre-training noise data and clean data, the detection model comprises at least one of a first module and a second module, and the detection model comprises a first label corresponding to the unlabeled data and a second label corresponding to the unlabeled data. The first module is used for searching in a feature map extracted based on the input data to obtain an output feature value, and the second module is used for obtaining a plurality of output results based on the output feature value and determining a second label corresponding to the input data according to the plurality of different output results.

Description

Technical Field

[0001] The present application relates to the field of computer vision, and in particular to a detection method and device. Background Art

[0002] With the widespread application of artificial intelligence in various fields, the demand for three-dimensional (3D) object detection is increasing in various intelligent scenarios, such as autonomous driving, robots, or drones.

[0003] However, for 3D object detection, models that perform these tasks typically rely on highly accurately labeled training data, and 3D labeling requires significant manual effort. Therefore, achieving object detection with less manual labeling or reducing labeling costs has become a pressing issue. Summary of the Invention

[0004] The present application provides a detection method and device for using less labeled data as supervision data to achieve labeling of unlabeled data and realize detection at a lower labeling cost.

[0005] In view of this, in a first aspect, the present application provides a detection method, comprising: first, obtaining a first data set, which may include unlabeled data, and a first label corresponding to the unlabeled data includes noise, and the unlabeled data can be understood as data that does not carry a label or data that carries noise in the label; then using the first data set as the input of a detection model to obtain a second data set, which includes the aforementioned unlabeled data and a second label corresponding to the unlabeled data, and the noise included in the second label is lower than the noise included in the first label. It can be understood that the second label is a label generated or updated by the output result of the detection model, and the detection model is used to output the label corresponding to the unlabeled data. The detection model A model obtained by pre-training using noisy data and clean data, where the noisy data is data without a label or data with a label that contains noise, and the clean data is labeled data, wherein the detection model includes at least one of a first module or a second module, the first module is used to search from a feature map extracted based on the input data to obtain an output feature value, the second module is used to obtain multiple output results based on the output feature value, and the first label of the input data is adjusted according to the multiple output results to obtain a second label. The multiple output results are different. In the initial stage, if the unlabeled data does not have a first label, the first label can be initialized for the unlabeled data to obtain an initial label for the unlabeled data.

[0006] In the implementation manner of the present application, a first module and / or a second module can be set in the detection model, and the first module and / or the second module can be used to obtain a more accurate label, thereby achieving more accurate labeling or detection of the input data. Specifically, the first module can search near the feature value corresponding to the first label, thereby searching for a feature value that better meets the requirements, such as a feature value with a higher confidence level, thereby obtaining a more accurate output feature; the second module can output multiple different output results, thereby adjusting the first label corresponding to the input data using multiple different output results, such as performing a noise assessment on the overall output result based on different output results, thereby adjusting the first label based on the noise assessment result to obtain a more accurate second label. Therefore, in the method provided by the present application, the detection model used can be obtained by training based on noise data and clean data, and the processing flow of the detection model can be used to achieve accurate detection or labeling, thereby reducing the labeling data required for training the detection model and reducing the labeling cost.

[0007] In one possible embodiment, the aforementioned second module may include multiple sub-modules, which can be used to output multiple output results. The multiple sub-modules are pre-trained sub-modules, and the data proportions used in the pre-training process of each of the multiple sub-modules are different. The data proportion is the proportion of labeled data or unlabeled data in the training set, and the labeled data is the labeled data.

[0008] In the embodiment of the present application, when pre-training the detection model, different amounts of noise data and / or clean data can be used for training for different submodules in the second module, so that the model weight parameters of each submodule are different. Therefore, when the same data is input, the multiple submodules can output different output results, so that the reasoning stage can adjust the label of the input data based on the differences between the output results of the multiple submodules to obtain more accurate labels. For example, noise evaluation can be performed based on the differences between the output results of the multiple submodules, thereby based on the evaluated noise. Therefore, through the method provided in the embodiment of the present application, more accurate labeling can be achieved at a lower labeling cost, reducing the cost of manual labeling.

[0009] In one possible implementation, the difference between the outputs of the aforementioned multiple submodules is used to determine the second label corresponding to the input data input to the detection model. In this embodiment of the present application, during the inference phase, the differences between the outputs of the multiple submodules can be used to adjust the label of the input data to obtain a more accurate label.

[0010] In one possible embodiment, the aforementioned multiple submodules include a first submodule and a second submodule; during the pre-training process, the proportion of labeled data corresponding to the first submodule is greater than the proportion of labeled data corresponding to the second submodule; the second label corresponding to the input data is obtained by weighted fusion based on the first label corresponding to the input data and the first divergence value, the first divergence value includes the difference between the output value of the first submodule and the output value of the second submodule, and the value of the second label corresponding to the input data is positively correlated with the first divergence value. In an embodiment of the present application, different data ratios can be used to train the first submodule and the second submodule. Compared with the second submodule, the difference in model weights between the first submodule and the second submodule may be the result of supervised learning using more clean data during training. Therefore, there is a difference between the prediction result output by the first submodule and the prediction result output by the second submodule, and there is also a difference between the true label corresponding to the input data and the first label. These two differences can be positively correlated, so the first label can be adjusted based on the difference between different modules to obtain a more accurate second label.

[0011] In one possible embodiment, the aforementioned multiple submodules also include a third submodule; during the pre-training process, the proportion of unlabeled data corresponding to the third submodule is smaller than the proportion of unlabeled data corresponding to the first submodule, and the proportion of unlabeled data corresponding to the third submodule is smaller than the proportion of unlabeled data corresponding to the second submodule; the second divergence value is used to update the learning weights, and the second divergence value is used to represent the difference between the output result of the third submodule and the output result of the first submodule, and the learning weights include the weights used to fuse the first label corresponding to the input data and the first divergence value. During pre-training, the third submodule and the first submodule are both trained using clean noise. Compared with the first submodule, the third submodule uses a smaller proportion of noise data for learning during training, which may result in different model weight parameters for the third submodule and the first submodule. Since both use clean data, the output results of the first submodule and the third submodule are more inclined to output predicted values that are closer to the labels of the clean data than the second submodule. Therefore, the confidence of the prediction result output by the first submodule can be measured by the similarity between the prediction result of the first submodule and the prediction result of the third submodule, so that the guidance weight corresponding to the first divergence value can be reversely updated based on the confidence of the prediction result output by the first submodule, thereby improving the degree of guidance of more accurate prediction results for adjusting labels.

[0012] In one possible implementation, the first module is specifically configured to extract features from input data to obtain a feature map, and then sample from the feature map based on the first label corresponding to the input data to obtain an output feature value. For example, a feature value with a higher confidence level may be searched from the feature map to replace or adjust the feature value corresponding to the first label in the feature map, thereby obtaining a more accurate output feature value.

[0013] In one possible implementation, the aforementioned first module is specifically configured to: obtain a first region corresponding to a first label in a feature map; and search a second region in the feature map based on the confidence level corresponding to each feature value in the second region to obtain an output feature value, where the second region is different from the first region. For example, the second region may be near the first region, thereby selecting feature values with higher confidence levels and improving the accuracy of downstream detection tasks.

[0014] In one possible implementation, the aforementioned detection model is specifically used for 3D object detection, thereby enabling 3D object detection on input data, and achieving 3D object detection or 3D object annotation with fewer annotations, thereby reducing labor costs.

[0015] In a possible implementation, the data in the aforementioned first data set includes images or point clouds.

[0016] In a second aspect, the present application provides a detection device, comprising:

[0017] An acquisition module is configured to acquire a first data set, where the first data set includes unlabeled data, and a first label corresponding to the unlabeled data includes noise;

[0018] A detection module is used to use a first data set as input to a detection model to obtain a second data set, where the second data set includes unlabeled data and a second label corresponding to the unlabeled data, where the noise included in the second label corresponding to the unlabeled data is lower than the noise included in the first label, and the detection model is used to output the label corresponding to the unlabeled data, wherein the detection model includes at least one of the first module or the second module, the first module is used to search from a feature map extracted based on the input data to obtain an output feature value, the second module is used to obtain multiple output results based on the output feature value, and the second label corresponding to the input data is determined according to the multiple output results, and the multiple output results are different.

[0019] Among them, the effects of the second aspect and any optional implementation of the second aspect can refer to the corresponding effects of the aforementioned first aspect or any optional implementation of the first aspect, and will not be repeated here.

[0020] In one possible embodiment, the second module includes multiple sub-modules, and the multiple sub-modules are used to output multiple output results. The multiple sub-modules are pre-training sub-modules. The data proportions used in the pre-training process of each of the multiple sub-modules are different. The data proportion is the proportion of labeled data or unlabeled data in the training set, and the labeled data is the labeled data.

[0021] In a possible implementation, the difference between the outputs of the multiple submodules is used to obtain a second label corresponding to the input data input to the detection model based on the first label corresponding to the input data.

[0022] In a possible implementation, the aforementioned multiple submodules include a first submodule and a second submodule; during the pre-training process, the proportion of labeled data corresponding to the first submodule is greater than the proportion of labeled data corresponding to the second submodule;

[0023] The second label corresponding to the input data is obtained by weighted fusion of the first label corresponding to the input data and the first divergence value. The first divergence value includes the difference between the output value of the first submodule and the output value of the second submodule. The value of the second label corresponding to the input data is positively correlated with the first divergence value.

[0024] In one possible implementation, the aforementioned multiple submodules further include a third submodule; during the pre-training process, the proportion of unlabeled data corresponding to the third submodule is smaller than the proportion of unlabeled data corresponding to the first submodule, and the proportion of unlabeled data corresponding to the third submodule is smaller than the proportion of unlabeled data corresponding to the second submodule;

[0025] The second divergence value is used to update the learning weight, and the second divergence value is used to represent the difference between the output result of the third submodule and the output result of the first submodule. The learning weight includes the weight used to fuse the first label corresponding to the input data and the first divergence value.

[0026] In a possible implementation, the aforementioned first module is specifically used to: extract features from input data to obtain a feature map, and sample from the feature map based on a first label corresponding to the input data to obtain an output feature value.

[0027] In a possible embodiment, the aforementioned first module is specifically used to: obtain a first area corresponding to the first label in the feature map; search the second area in the feature map based on the confidence corresponding to each feature value in the second area to obtain an output feature value, and the second area is a different area from the first area.

[0028] In a possible implementation, the aforementioned detection model is specifically used for performing 3D object detection.

[0029] In a possible implementation, the data in the aforementioned first data set includes images or point clouds.

[0030] In a third aspect, embodiments of the present application provide a detection device that has the functionality to implement the detection method of the first aspect described above. This functionality can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the aforementioned functionality.

[0031] In fourth aspect, an embodiment of the present application provides a detection device, which can also be called a digital processing chip or chip. The chip includes a processing unit and a communication interface. The processing unit obtains program instructions through the communication interface, and the program instructions are executed by the processing unit. The processing unit is used to perform functions related to processing as in the above-mentioned first aspect or any optional embodiment of the first aspect.

[0032] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium comprising instructions, which, when executed on a computer, enables the computer to execute the method in the above-mentioned first aspect or any optional embodiment of the first aspect.

[0033] In a sixth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method in the above-mentioned first aspect or any optional implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of the system architecture provided for this application;

[0035] Figure 2 Another system architecture diagram provided for this application;

[0036] Figure 3 A schematic diagram of an application scenario provided for this application;

[0037] Figure 4 A schematic diagram of another application scenario provided for this application;

[0038] Figure 5 A schematic diagram of a detection method provided in this application;

[0039] Figure 6 Another system architecture diagram provided for this application;

[0040] Figure 7 A schematic diagram of the structure of a detection model provided in this application;

[0041] Figure 8 A schematic diagram of the structure of another detection model provided in this application;

[0042] Figure 9 A schematic diagram of the structure of another detection model provided in this application;

[0043] Figure 10 A schematic structural diagram of a detection device provided in this application;

[0044] Figure 11 This is a schematic structural diagram of another detection device provided in this application. DETAILED DESCRIPTION

[0045] The following will describe the technical solutions in the embodiments of this application in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0046] First, the overall workflow of the artificial intelligence system is described. Figure 1 , Figure 1 The following diagram illustrates a structural diagram of the AI framework. This framework is explained below from two perspectives: the "intelligent information chain" (horizontal axis) and the "IT value chain" (vertical axis). The "intelligent information chain" reflects the entire process from data acquisition to processing. For example, it encompasses the general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. Throughout this process, data undergoes a condensed progression from "data-information-knowledge-wisdom." The "IT value chain," encompassing the entire process from the underlying infrastructure of human intelligence, information (provided and processed by technology), to the system's industrial ecosystem, reflects the value that AI brings to the information technology industry.

[0047] (1) Infrastructure

[0048] Infrastructure provides computing power for AI systems, enabling communication with the outside world and supporting this through a foundational platform. Communication with the outside world occurs through sensors; computing power is provided by intelligent chips, such as central processing units (CPUs), neural-network processing units (NPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), or hardware acceleration chips like field programmable gate arrays (FPGAs). The foundational platform includes distributed computing frameworks and networks, along with other related platform guarantees and support, including cloud storage and computing, and interconnected networks. For example, sensors communicate with the outside world to acquire data, which is then fed into the intelligent chips within the distributed computing system provided by the foundational platform for computation.

[0049] (2) Data

[0050] Data above the infrastructure layer represents data sources for AI. This data includes graphics, images, voice, and text, as well as IoT data from traditional devices. This includes business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0051] (3) Data processing

[0052] Data processing generally includes data training, machine learning, deep learning, search, reasoning, decision-making, etc.

[0053] Among them, machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, and training.

[0054] Reasoning refers to the process of simulating human intelligent reasoning in computers or intelligent systems, using formalized information to perform machine thinking and solve problems based on reasoning control strategies. Typical functions are search and matching.

[0055] Decision-making refers to the process of making decisions after intelligent information is reasoned, and usually provides functions such as classification, sorting, and prediction.

[0056] (4) General ability

[0057] After the data has undergone the data processing mentioned above, some general capabilities can be further formed based on the results of the data processing, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0058] (5) Smart products and industry applications

[0059] Smart products and industry applications refer to the products and applications of artificial intelligence systems in various fields. They are the encapsulation of the overall artificial intelligence solution, which productizes intelligent information decision-making and realizes practical application. Its application areas mainly include: smart terminals, smart transportation, smart medical care, autonomous driving, smart cities, etc.

[0060] For example, with the development of autonomous driving and robotics, as well as the accelerated mass production of autonomous vehicles, object detection, especially 3D object detection, is becoming increasingly important. The 3D annotated data required for training 3D object detection models is also very costly.

[0061] Therefore, the present application provides a detection method that can be used to learn or implement target detection based on less labeled data, and can be specifically applied to data labeling or online target detection scenarios. For example, in data labeling scenarios, fewer labels can be used as supervision to implement data labeling, such as the 3D position or target category of the target in the detection data as the data label, thereby achieving low-cost labeling. In particular, for 3D detection, higher-precision labels can be annotated at a lower cost.

[0062] Moreover, for autonomous driving, autonomous driving data is usually closed-loop platform business data, that is, a model method based on noisy labeled data reduces the cost and pressure of data labeling platforms and processes. In existing solutions, a certain amount of manual quality inspection is usually required to process and correct noisy labeled data, which limits the efficiency of data closure. The method provided in this application provides a robust 3D target detection method that can accommodate noise labels to a certain extent. By using a small amount of precise data + a large amount of noisy data, a robust 3D target detection model can still be obtained, which can be deployed in a data closed-loop platform. The model method based on noisy labeled data reduces the cost and pressure of data labeling platforms and processes. That is, there is no need for manual quality inspection and correction, which greatly reduces labor costs.

[0063] First, the method provided in this application involves some terms or concepts in the field of neural networks. For ease of understanding, some terms or concepts are introduced below.

[0064] (1) Neural Network

[0065] A neural network can be composed of neural units, which can be represented by xs The output of the operation unit with the intercept 1 as input can be shown as formula (1-1):

[0066]

[0067] Where, s = 1, 2, ... n, n is a natural number greater than 1, W s is x s The weight of the neural unit, b is the bias of the neural unit. f is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into the output signal. The output signal of the activation function can be used as the input of the next convolutional layer, and the activation function can be a sigmoid function. A neural network is a network formed by connecting multiple single neural units mentioned above, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field. The local receptive field can be an area composed of several neural units.

[0068] (2) Backbone network, also known as backbone network

[0069] Usually refers to the network used for feature extraction. Specific examples include convolutional neural networks (CNN), deep convolutional neural networks (DCNN), recurrent neural networks (RNN), region-based convolutional neural networks (RCNN), or faster RCNN.

[0070] In a neural network, in addition to the backbone network, other functional networks can also be included. For example, the output end of the backbone can also be connected to networks such as the region proposal network (RPN) and the feature pyramid network (FPN) to further process the features extracted by the backbone network, such as identifying the classification of features and performing semantic segmentation on features.

[0071] (3) Head (Head Network)

[0072] The Head is typically the output of the model, used to perform the required downstream prediction tasks. For example, for an object detection task, the Head may typically include a classification head (for predicting the object category) and a regression head (for predicting the location of the target box). In the embodiments of this application, the Head is typically placed after the output of the Backbone to perform further downstream tasks on the features output by the Backbone.

[0073] For example, in the neural network provided in this application, the detection model mentioned below may include Backbone and Head, Backbone may include the first module mentioned below, and Head may include the second module mentioned below. The first module is used for feature extraction, and the second module can be used to perform detection tasks based on the features extracted by the first module.

[0074] (4) Loss function

[0075] During the training of a deep neural network, because we want the output of the deep neural network to be as close as possible to the desired predicted value, we can compare the current network's predicted value with the desired target value and then update the weight vector of each layer of the neural network based on the difference between the two. (Of course, there is usually an initialization process before the first update, which is to pre-configure the parameters for each layer in the deep neural network.) For example, if the network's predicted value is too high, the weight vector is adjusted to make it predict a lower value. This adjustment is continued until the deep neural network can predict the desired target value or a value very close to the desired target value. Therefore, it is necessary to predefine "how to compare the difference between the predicted value and the target value." This is the loss function (or objective function), which is an important equation used to measure the difference between the predicted value and the target value. For example, the loss function output value (loss) indicates a greater difference, so training a deep neural network becomes a process of minimizing this loss as much as possible.

[0076] (5) Backpropagation algorithm

[0077] Neural networks can use the back propagation (BP) algorithm to correct the size of the parameters in the initial neural network model during training, reducing the reconstruction error loss of the neural network model. Specifically, the forward propagation of the input signal to the output generates error loss. This error loss information is then backpropagated to update the parameters in the initial neural network model, thereby converging the error loss. The BP algorithm is a backward propagation movement dominated by error loss, aiming to obtain the optimal parameters of the neural network model, such as the weight matrix.

[0078] (6) Supervised Learning

[0079] Supervised learning, also known as supervised learning, is a training method / learning method in machine learning. It refers to the process of training a model by letting the machine learn a large amount of labeled sample data, so that the model can obtain corresponding output based on the input.

[0080] Generally, supervised learning requires a large amount of labeled data. The more labeled data, the better the machine learning algorithm, and the more capable it is at selecting the correct answer from the input data. In other words, supervised learning involves finding the relationship between input and output. For example, taking an image or text as input and outputting classification features or segmentation features as output.

[0081] (7) Unsupervised Learning

[0082] Unsupervised learning is a training / learning method in machine learning. It can be understood as letting the model learn on its own, with its input being unlabeled samples, and learning the relationships between the input data through clustering or other methods.

[0083] Unsupervised learning is often used in data mining to learn relationships within large amounts of unlabeled data. The training data is unlabeled, and the goal is to classify or distinguish observations. For example, unsupervised learning should be able to distinguish images of "cats" from a large number of diverse images based solely on the characteristics of all images of "cats" without any additional prompts.

[0084] Compared with supervised learning, unsupervised learning does not require the input data to carry labels during the training process. It can be applied to more complex tasks and can be used to learn deeper relationships between data.

[0085] (8) Self-supervised learning

[0086] This is also known as contrastive learning, which constructs proxy tasks (pretext tasks). Self-supervised learning uses these pretext tasks to enable deep learning-based feature extractors (encoders) to encode deep features of point clouds without label information. This makes the encoder output effective for downstream tasks such as classification and segmentation.

[0087] Contrastive learning is a special self-supervised learning approach. Unlike the encoding-decoding approach used in generative learning, it focuses on learning the common features between instances of the same category and distinguishing the differences between instances of different categories. The core of contrastive learning is differentiation, and its encoder is more adept at learning holistic, discriminative features.

[0088] (9) Label

[0089] In machine learning, a label typically refers to the correct output or category of a sample, also known as the true value. In machine learning, labels are often the training objectives of supervised learning, where the model is trained so that its output is closer to or equal to the label.

[0090] In the embodiments of the present application, data carrying a label is referred to as labeled data, and data not carrying a label or carrying a label containing noise or containing noise greater than a preset noise value is referred to as unlabeled data or data to be labeled.

[0091] The method provided in the embodiments of the present application can be specifically executed on a server or on a terminal device. The server can be a distributed server or a centralized server; the terminal device can be a mobile phone with image processing capabilities, a tablet personal computer (TPC), a media player, a smart TV, a laptop computer (LC), a personal digital assistant (PDA), a personal computer (PC), a camera, a camcorder, a smart watch, a wearable device (WD), or an autonomous vehicle, etc., and the embodiments of the present application are not limited to this.

[0092] The following is an exemplary introduction to the system architecture of the detection method provided in this application.

[0093] See Figure 2 , a schematic diagram of a system architecture provided by the present application. The system may include a data acquisition device 201 and a detection device 202.

[0094] The data acquisition device 201 can be used for data acquisition. For example, the data acquisition device can be a laser radar, millimeter wave radar, image sensor, infrared sensor, etc. The acquisition device can transmit the collected data to the detection device 202. The data in the data sets (such as the first data set or the second data set, etc.) mentioned in the following embodiments of this application can include data collected by the data acquisition device 201.

[0095] The detection device 220 can be used to detect the data collected by the data collection device 201 and identify information about the target in the input data, such as the location and category of the target in the input data.

[0096] Specifically, the method provided in this application can be applied to online target detection scenarios, offline target detection scenarios, and dataset annotation scenarios. Online target detection involves performing real-time target detection on the data collected by the data acquisition device 201; offline target detection involves performing offline detection on the data collected by the data acquisition device 201. For example, after collecting data, the data acquisition device 201 can store the data and, when target detection is required, transmit the data to the detection device 202 for detection; and dataset annotation scenarios can involve, after collecting data, the data acquisition device 201 can upload the collected data to a cloud platform or other server, and the detection device 202 can perform target detection on the data stored in the cloud platform or other server, thereby performing data annotation.

[0097] The following introduces some possible application scenarios.

[0098] Scenario 1: Data Labeling Scenario

[0099] The method provided in this application can be applied to label unlabeled data in a dataset. That is, after the data to be labeled is tested using the method provided in this application, the test results are used as labels for the data.

[0100] like Figure 3 As shown, after the data acquisition device collects data, such as images or point clouds in the environment, the collected data is uploaded to the cloud server. The cloud server can directly use the data uploaded by the data acquisition device as the input of the detection device to annotate the data, such as annotating the one-dimensional or multi-dimensional coordinates of the target in the data, or can also identify the specific information of the target, such as the category of the target or the like. For example, if the data collected by the data acquisition device includes an image, the 2D coordinates of the target in the image can be identified. If the data collected by the data acquisition device includes a radar point cloud, the 3D coordinates of the target in the radar point cloud can be identified, or the specific information of the target can be further identified, such as the category of the target, the shape of the target, and other information. Compared with manual high-precision 3D annotation, the architecture provided by the present application can achieve lower-cost data annotation, and the annotated data can be used for model training, thereby reducing the cost of model training that requires the use of annotated data.

[0101] Scenario 2: Autonomous Driving Scenario

[0102] The method provided in this application can be applied to the perception module of a vehicle. For example, in implementation, it can be combined with the software and hardware systems of an autonomous driving vehicle. The hardware system may include a target detection sensor, or a processor, etc. The target detection sensor may include a lidar sensor for detecting targets in the surrounding environment of the vehicle. The processor can be used to receive data from the target detection sensor and process the data and output obstacle targets, such as a general-purpose processor, a graphics image processor, etc. The software system includes an operating system, a sensor driver, a sensor data processing program, etc. The present application can be deployed in the perception module of an autonomous driving software system. The present application can be used as a tracking management submodule of a multi-target tracking module in the perception module, which can maintain an internal tracking object sequence and output stable and reliable tracking object results to the outside, usually sent to other submodules in the perception module, such as a prediction module.

[0103] like Figure 4 As shown, the vehicle can be equipped with sensors such as laser radar, millimeter wave radar, image sensor, infrared sensor (i.e., as the aforementioned data acquisition device 201), which can be used to collect data about the environment in which the vehicle is located. Detection equipment can also be deployed in the vehicle's processor, or independent detection equipment can be set in the vehicle, or detection equipment can be deployed in the vehicle's onboard equipment. During the driving process of the vehicle, the laser radar, millimeter wave radar, image sensor, infrared sensor, etc. installed in the vehicle can collect real-time information about the environment in which the vehicle is traveling. The detection equipment installed in the vehicle performs 3D detection on the real-time collected data. For example, the position of the target in 3D space can be identified, thereby determining the position of the target relative to the vehicle, so as to facilitate the determination of corresponding driving decisions when the autonomous vehicle is driven. For example, if the target is too close to the vehicle, the vehicle can be determined to brake, change lanes, or turn to avoid the target that is too close to the vehicle. Alternatively, during the user's driving process, the user can be prompted for targets that are too close to the vehicle, so that the user can be notified of the targets that are too close to the vehicle in a timely manner, and the user can be warned in time to improve the safety of the user's driving.

[0104] Of course, in addition to the above scenarios, it can also be applied to other scenarios, such as behavioral decision control based on target detection results during robot operation or other scenarios that require data labeling or target detection. This application does not go into details about the applicable scenarios.

[0105] The following is an introduction to the method flow provided by this application.

[0106] See Figure 5 , a flow chart of a detection method provided in this application is as follows.

[0107] It should be noted that the method provided in this application can improve the accuracy of the output detection results through iterative execution, and one of the iterative processes is introduced below.

[0108] 501. Obtain a first data set, which includes unlabeled data.

[0109] The first data set may include unlabeled data, that is, data that has been manually labeled. In the initial iteration process, initial labels can be set for the unlabeled data, and in subsequent iterations, the initial labels can be iteratively updated until more accurate labels are obtained.

[0110] For ease of understanding, the label corresponding to the unlabeled data in this iteration is called the first label.

[0111] The first data set may specifically include data collected by the aforementioned data acquisition device, and may specifically include images or point cloud data, etc.

[0112] 502. Use the first data set as input to a detection model to obtain a second data set, and the detection model is used to output a detection result of the unlabeled data.

[0113] The first dataset can then be used as input for the detection model to generate the second dataset. The detection model can be used to detect the unlabeled data in the first dataset, output the detection results corresponding to the unlabeled data, and use the detection results as the labels corresponding to the unlabeled data.

[0114] The detection model can be used to perform target detection, specifically 3D target detection. Accordingly, the unlabeled data in the first data set can specifically include images or point clouds.

[0115] Specifically, the detection model may include at least one of the first module or the second module, the first module is used to search from the feature map extracted based on the input data to obtain the output feature value, the second module is used to obtain multiple output results based on the output feature value, and determine the second label corresponding to the input data according to the multiple output results, and the multiple output results are different.

[0116] The first module can be used to search the feature map extracted from the input data to output more accurate output feature values, thereby improving the accuracy of downstream prediction tasks. For example, a feature value with higher confidence can be searched from the feature map to replace or adjust the feature value corresponding to the first label in the feature map, thereby obtaining a more accurate output feature value.

[0117] The differences between the multiple different output results of the second module correspond to different noise levels or indicate different adjustment weights for the first label. The differences between the multiple different output results can be used to determine the second label corresponding to the input data. This is equivalent to adjusting the first label based on the differences between the multiple different output results to obtain a more accurate second label.

[0118] After obtaining the second label, the second label can be iteratively updated. For example, the second label is used as the first label corresponding to the input data, and steps 501 to 502 are repeated, thereby improving the accuracy of the final output result through iteration and obtaining a more accurate detection result.

[0119] In the case where it is necessary to iteratively update the label of the unlabeled data, the iteration can be stopped when the convergence condition is met. When the convergence condition is not met, the iteration can be continued, that is, step 501 is continued. The convergence condition may specifically include but is not limited to: the number of iterations reaches a preset number, the iteration time reaches a preset time, the change value of the output result is less than a preset value, or the accuracy corresponding to the output result is higher than a preset accuracy, etc., to adapt to various scenarios. When the convergence condition includes multiple conditions, the iteration can be terminated when one of the conditions is met, or it can be terminated after two or more of the conditions are met. For example, after the number of iterations reaches a preset number, the iteration can be terminated and the second label corresponding to the unlabeled data can be output; or, when the change value between the output results output by the recent multiple iterations is less than a preset value, the second label corresponding to the unlabeled data can be output; or, when the accuracy of the output result is higher than the preset accuracy value, the second label corresponding to the unlabeled data can be output.

[0120] Therefore, in the embodiments of the present application, new, more accurate labels can be obtained based on the aforementioned first module and / or second module. Specifically, the first module can obtain more accurate output features, thereby improving the accuracy of downstream detection tasks. The second module can adjust the first label based on the differences between different output results to obtain a more accurate second label.

[0121] In one possible implementation, the aforementioned first module can be specifically configured to extract features from the input data to obtain a feature map, and then sample from the feature map based on the first label corresponding to the input data to obtain an output feature value. That is, in the implementation of the present application, sampling from the feature map can be performed to obtain more accurate feature values, such as feature values with higher confidence levels.

[0122] In a possible embodiment, the aforementioned first module can be specifically used to obtain the first area corresponding to the first label in the feature map, for example, the first area can be the area where the coordinates of the target in the input data indicated by the first label are located, mapped to the area of the feature map; the second area in the feature map is searched based on the confidence corresponding to each eigenvalue in the second area to obtain the output eigenvalue, and the second area is a different area from the first area, for example, the second area can be an area near the first area, thereby screening out eigenvalues with higher confidence and improving the accuracy of downstream detection tasks.

[0123] In a possible embodiment, the aforementioned second module may include multiple submodules for outputting the aforementioned multiple output results. The multiple submodules are pre-training submodules, and the data ratios used in the pre-training process of each of the multiple submodules are different. The data ratio is the ratio of labeled data or unlabeled data in the training set, and the labeled data is the labeled data. It can be understood that different data ratios are used for training, so there are differences between the output results of each module. In other words, when pre-training, the amount of data corresponding to the labeled data and unlabeled data used by each module is not exactly the same. On this basis, the model weight parameters learned by each submodule may also be different, so there may be differences between the output results of each submodule. It can be understood that the differences between the submodules can be used to estimate the noise level of the label, or to evaluate the confidence of the output results of each submodule. Therefore, the differences based on the output results of each submodule can be used to adjust the first label to obtain a more accurate second label.

[0124] In one possible implementation, the difference between the outputs of multiple submodules is used to adjust the first label corresponding to the input data fed into the detection model to obtain the second label corresponding to the input data. Because each module is pre-trained using different data ratios, the differences between modules indicate different adjustment directions. This difference between modules can be used to adjust the first label corresponding to the input data, resulting in a more accurate second label.

[0125] In one possible embodiment, the aforementioned multiple sub-modules may include a first sub-module and a second sub-module; wherein, during the pre-training process of the multiple sub-modules, the proportion of labeled data corresponding to the first sub-module is greater than the proportion of labeled data corresponding to the second sub-module, and the proportion of unlabeled data corresponding to the first sub-module may be greater than or not greater than the proportion of unlabeled data corresponding to the second sub-module; that is, the proportion of clean data in the pre-training data corresponding to the first sub-module is greater than the proportion of clean data in the pre-training data corresponding to the second sub-module. The noise data or noisy data mentioned in the embodiment of the present application is data that is not labeled or data with noise in the label, and the clean data is labeled data.

[0126] Here, the difference between the output value of the first submodule and the output value of the second submodule is called the first divergence value. Therefore, when adjusting the first label corresponding to the input data, the first label and the first divergence value can be weighted and fused to obtain a new label, namely the second label. Specifically, during the weighted fusion process, the weight of the first divergence value is a positive number, that is, the value of the first divergence value is positively correlated with the value of the second label. For example, if the proportion of labeled data corresponding to the first submodule is greater than the proportion of labeled data corresponding to the second submodule, that is, the proportion of clean data used by the first submodule during pre-training is greater, the learned model weight parameters of the first and second submodules may be different. Therefore, when the same input data is input to the first and second submodules, the output prediction results may also be different. The difference in model weights between the first and second submodules may be the result of supervised learning using more clean data during training compared to the second submodule. It can be understood that the prediction result output by the first submodule is regarded as the more accurate prediction result, while the prediction result output by the second submodule is regarded as the more noisy prediction result. Therefore, there is a difference between the prediction result output by the first submodule and the prediction result output by the second submodule (called the first difference for easy distinction), and there is also a difference between the second label corresponding to the input data (that is, a more accurate label, which is a value to be determined) and the first label (that is, a label with a higher degree of noise) (called the second difference for easy distinction). The first difference and the second difference can be positively correlated, so the first label can be adjusted based on the differences between different modules to obtain a more accurate second label.

[0127] In one possible embodiment, the aforementioned multiple submodules may further include a third submodule; wherein, during the pre-training process, the proportion of labeled and unlabeled data corresponding to the third submodule is smaller than the proportion of unlabeled data corresponding to the first submodule, and the proportion of unlabeled data corresponding to the third submodule is smaller than the proportion of unlabeled data corresponding to the second submodule, that is, the third submodule uses more clean data than the second submodule, and less noise data than the first submodule, which is equivalent to using a higher proportion of clean data for supervised learning. The difference between the first submodule and the third submodule is called the second divergence value, and the second divergence value can be used to reversely update the learning weight, which is the weight used to fuse the first label and the first divergence value. Therefore, the fusion weight value of the first label and the first divergence value can be adjusted by the difference between the first submodule and the second submodule, so that the obtained weight value is more accurate.

[0128] Generally, the smaller the second divergence value, the higher the confidence level of the corresponding submodule's output results, and the higher the learning weight. That is, the higher the guidance weight provided by the first divergence value, the more accurate the resulting label. This can be understood as follows: during pre-training, the third submodule and the first submodule were both trained using clean noise. Compared to the first submodule, the third submodule used less noise data during training, which may result in different model weight parameters for the third submodule and the first submodule. Since both used clean data, the output results of the first and third submodules tend to output predictions that are closer to the labels of the clean data than the second submodule. Therefore, the confidence of the prediction result output by the first submodule can be measured by the similarity between the prediction result of the first submodule and the prediction result of the third submodule, so as to reversely update the guidance weight corresponding to the first divergence value based on the confidence of the prediction result output by the first submodule, that is, the weight corresponding to the first divergence value when updating the first label. For example, the higher the confidence, the higher the accuracy of the first divergence value for noise estimation, and the higher the guidance weight. Correspondingly, the lower the confidence, the lower the accuracy of the first divergence value for noise estimation, and the lower the guidance weight, thereby improving the degree of guidance of more accurate prediction results for adjusting labels.

[0129] Therefore, in the method provided in this application, the divergence values between each sub-module can be used to realize the confidence assessment and noise assessment of the output results of each sub-module, thereby using less clean data to complete the labeling of unlabeled data and reduce the cost of manual labeling.

[0130] The above describes the method flow provided by this application. The following describes the structure of the detection model and the method flow provided by this application in more detail in combination with specific application scenarios.

[0131] The following introduces the application architecture of the method provided in this application by taking the use of a 3D detection model to perform data annotation on a data annotation platform as an example.

[0132] See Figure 6 , a schematic diagram of a data annotation architecture provided in this application.

[0133] The data in the data annotation platform can include noisy data and clean data. Noisy data refers to data without labels or data with noise in the labels. Clean data refers to labeled data or data with a noise level below a preset noise threshold. The data in the data annotation platform can include RGB images, depth images, or point clouds.

[0134] Multiple modules can be set in the 3D detection model, such as the first module or the second module mentioned above. For ease of understanding, the first module can be called a multiple instance slacking module (MIS), and the second module can be called a discrepancy-based noise filtering module (DNF). In the implementation manner of the present application, the 3D detection model may include at least one of MIS or DNF.

[0135] For example, a 3D detection model including MIS and DNF can be Figure 7 As shown, MIS can be used to extract more accurate features, and DNF can be used to output multiple prediction results based on the features extracted by MIS, and adjust the labels of the input data based on the multiple prediction results to obtain new and more accurate labels.

[0136] The specific structures of MIS and DNF are introduced below.

[0137] 1. MIS

[0138] The MIS can be used to extract feature maps of one or more scales from input data. The first label of the input data can be used to identify the region corresponding to the target in the input data. Accordingly, after projecting this region onto the feature map, the feature value corresponding to the target indicated by the first label can be determined. The MIS can be used to search for regions in the feature map that are different from the region corresponding to the first label, and obtain feature values that are different from the feature values in the region corresponding to the first label as the output feature values of the MIS.

[0139] For example, the steps performed by the MIS may be as follows: Figure 8 As shown, MIS can extract feature maps of multiple scales from the input data, and the coordinates corresponding to the first label can be expressed as P k , taking one of the scales as an example, we can kSearch nearby and obtain the predicted value with higher confidence as the output feature value of the target. It can be understood that MIS can extract the feature map, P k The index k of the feature map is jittered and relaxed. For example, the feature map can be divided into multiple regions and searched in each region. For example, the feature value with the highest confidence can be searched, such as P k (max) is used as the feature value of the new label in the current scale, so as to output the predicted value pred in the downstream task bbox .

[0140] That is, this application proposes a 3D feature map multi-example jitter relaxation module for 3D weakly supervised target detection tasks. It can find more suitable position features for calculating predicted labels by index jitter on the feature map, thereby reducing the negative impact of noise labels and improving the accuracy of subsequent detection tasks.

[0141] 2. DNF

[0142] The DNF can set multiple heads. The ratio of noisy data to clean data used by these multiple heads during pre-training is different. Therefore, the prediction results output by these multiple heads are different when making predictions.

[0143] For example, the structure of the 3D detection model with the DNF module can be as follows: Figure 9 As shown in the figure, the 3D detection model can be BEVDet (High-Performance Multi-Camera 3D Object detection in Bird-Eye-View), and of course it can also be replaced by other 3D detection models, such as the BEVDepth 3D (depthfor multi-view 3D object detection) model for target detection.

[0144] First, based on BEVDet, multiple heads are set. Here, three heads are set as an example, such as head0, head1, and head2. In specific application scenarios, other numbers of heads can also be replaced, depending on the computing resources of the actual computing device or the efficiency of updating labels. For example, the more abundant the computing resources of the computing device, the more heads can be set, or the higher the efficiency requirement for updating labels, the more heads can be set. The number of heads can be determined according to the actual application scenario, and this application does not limit this.

[0145] The structures of head0, head1, and head2 can be the same or different. During pre-training, different proportions of data can be used for training, so that there are differences between the output results of each head, which facilitates label adjustment. For example, head0 can be trained with noisy data and clean data, head1 can be trained with noisy data, and head2 can be trained with clean data. For example, the training set used for pre-training can be set to 80% noisy data and 20% clean data. Head0 is pre-trained with 80% noisy data and 20% clean data, head1 is pre-trained with 80% noisy data, and head1 is trained with 20% clean data.

[0146] In the embodiments of the present application, the unlabeled data can be used as input to the 3D detection model. After feature extraction, the extracted features are input to each head. Because each head uses different data for supervised training, the prediction results output by each head will differ. The differences between the prediction results of each head can be used to update the first label of the input data.

[0147] For example, the divergence between head0 and head1 can be called 01 divergence, that is, the first divergence, and the divergence between head0 and head2 can be called 02 divergence, that is, the second divergence.

[0148] Regarding 01 disagreement:

[0149] Since head0 and head1 are trained using different data, the model weights of head0 and head1 may also be different, so there will be a disagreement between the output results of head0 and head1. Head0 is trained with more noise-free supervision data than head1, so the disagreement between the output results of head0 and head1 can be used to estimate the degree of noise in the label. Therefore, the predicted value of head0 minus the predicted value of head1, that is, the 01 divergence, is positively correlated with the difference between the true value of the data and the noise value. Based on this, the way to adjust the first label based on the 01 divergence can be expressed as: Y_fix=Y_noise+λ*dis_01(λ>0)

[0150] Here, Y_noise represents the first label, dis_01 represents the 01 disagreement, λ represents the weight corresponding to the 01 disagreement, and Y_fix represents the adjusted second label. That is, the larger the 01 disagreement value, the greater the impact on the adjusted label.

[0151] Therefore, in the embodiment of the present application, the output differences between different heads are used to adjust the labels of the noise data, so that the final labels are closer to the output results of the model obtained by training based on clean data.

[0152] Regarding 02 differences:

[0153] Due to the difference in the supervised data of head0 and head2, there will be a disagreement between head0 and head2. Since both heads are trained with clean data, the similarity between them also represents the confidence of the prediction results. The 02 divergence represents the similarity or alignment between the overall sample and the clean data, which also represents the credibility of the noise sample and the confidence of the corrected label. Therefore, in the method provided in the embodiment of the present application, the learning weights of the noise data and the corrected label can be adjusted through the 02 divergence, for example, it can be expressed as the following formula:

[0154] Loss_weight=λ*(1-sigmoid(|dis-02|))(λ>0)

[0155] Here, dis_02 represents the O2 divergence, and the sigmoid function is optional; for example, it can be omitted when the prediction value ranges between 0 and 1. This shows that the smaller the O2 divergence, the higher the confidence level of the label, and the higher the learned weight. The learned weight is then used to update the weight used when adjusting Y_noise using dis_01.

[0156] Therefore, in the embodiment of the present application, the differences between the output results of different heads in the label adjustment process can be utilized. For multiple heads that are trained with noise-free data, the similarity of the output results between the multiple heads can be used to represent the similarity or alignment degree between the overall sample and the noise-free data. Based on this 02 divergence, the learning weights of the noise data and the corrected labels can be adjusted, so that more accurate learning weights, or fusion weights, can be obtained in the next iteration.

[0157] That is, this application provides a noise data screening module based on 3D weakly supervised target detection divergence, which can correct and weight the noise data samples according to the degree of divergence of the prediction results between different branch models, thereby achieving the purpose of filtering noise data and improving data quality.

[0158] Furthermore, in the embodiments of this application, a robust 3D object detection model can be obtained by using a small amount of accurate data plus a large amount of noisy data. This can be deployed in a closed-loop data platform, and the model approach based on noisy annotated data can reduce the cost and pressure of the data annotation platform and process.

[0159] For ease of understanding, the following introduces the effects achieved in different data sets by implementing the method provided in the embodiments of the present application based on different 3D detection models as basic models.

[0160] For example, the method provided in this application can adjust the model structure based on BEVDet, such as adding the aforementioned MIS and DNF. As shown in Table 1, the effectiveness of the scheme is verified on the nuscene public set: the NDS (nuscenes detection score) indicator is significantly improved by more than 3%.

[0161] plan Training data parameter Average Mean Accuracy (mAP) NDS Upper-Bound 100% GT (True Value) - 29.6 36.9 Basic model (BevDet) 10% GT + 90% noise - 20.14 29.1 This application 10% GT + 90% noise DNF 20.69(+0.55) 30.22(+1.12) This application 10% GT + 90% noise DNF+MIS 21.15(+1.01) 30.28(+1.18) Basic model (BevDet) 20% GT + 80% noise - 21.75 30.86 This application 20% GT + 80% noise DNF 23.78(+2.03) 33.51(+2.65) This application 20% GT + 80% noise DNF+MIS 24.02(+2.27) 33.95(+3.09)

[0162] Table 1

[0163] Secondly, adjusting the model structure based on BEVDet, as shown in Table 2, can also improve the NDS index.

[0164] plan Training data mAP NDS Upper-Bound 100% GT 33.1 40.16 Base Model (BEVDepth) 20% GT + 80% noise 25.91 34.29 This application 20% GT + 80% noise 29.93(+4.02) 38.14(+3.85)

[0165] Table 2

[0166] Obviously, the method provided by this application can achieve a significant improvement compared to existing solutions, with significant improvements in both accuracy and NDS indicators.

[0167] The above describes the method flow provided by this application. The following describes the device for executing the above method flow.

[0168] See Figure 10 , a schematic structural diagram of a detection device provided in this application, the detection device comprising:

[0169] An acquisition module 1001 is configured to acquire a first data set, where the first data set includes unlabeled data, and a first label corresponding to the unlabeled data includes noise.

[0170] The detection module 1002 is used to use the first data set as the input of the detection model to obtain a second data set, where the second data set includes unlabeled data and a second label corresponding to the unlabeled data, the noise included in the second label corresponding to the unlabeled data is lower than the noise included in the first label, and the detection model is used to output the label corresponding to the unlabeled data, wherein the detection model includes at least one of the first module or the second module, the first module is used to search from the feature map extracted based on the input data to obtain an output feature value, and the second module is used to obtain multiple output results based on the output feature value, and determine the second label corresponding to the input data according to the multiple output results, and the multiple output results are different.

[0171] In one possible embodiment, the second module includes multiple sub-modules, and the multiple sub-modules are used to output multiple output results. The multiple sub-modules are pre-training sub-modules. The data proportions used in the pre-training process of each of the multiple sub-modules are different. The data proportion is the proportion of labeled data or unlabeled data in the training set, and the labeled data is the labeled data.

[0172] In a possible implementation, the difference between the outputs of the multiple submodules is used to obtain a second label corresponding to the input data input to the detection model based on the first label corresponding to the input data.

[0173] In a possible implementation, the aforementioned multiple submodules include a first submodule and a second submodule; during the pre-training process, the proportion of labeled data corresponding to the first submodule is greater than the proportion of labeled data corresponding to the second submodule;

[0174] The second label corresponding to the input data is obtained by weighted fusion of the first label corresponding to the input data and the first divergence value. The first divergence value includes the difference between the output value of the first submodule and the output value of the second submodule. The value of the second label corresponding to the input data is positively correlated with the first divergence value.

[0175] In one possible implementation, the aforementioned multiple submodules further include a third submodule; during the pre-training process, the proportion of unlabeled data corresponding to the third submodule is smaller than the proportion of unlabeled data corresponding to the first submodule, and the proportion of unlabeled data corresponding to the third submodule is smaller than the proportion of unlabeled data corresponding to the second submodule;

[0176] The second divergence value is used to update the learning weight, and the second divergence value is used to represent the difference between the output result of the third submodule and the output result of the first submodule. The learning weight includes the weight used to fuse the first label corresponding to the input data and the first divergence value.

[0177] In a possible implementation, the aforementioned first module is specifically used to: extract features from input data to obtain a feature map, and sample from the feature map based on a first label corresponding to the input data to obtain an output feature value.

[0178] In a possible embodiment, the aforementioned first module is specifically used to: obtain a first area corresponding to the first label in the feature map; search the second area in the feature map based on the confidence corresponding to each feature value in the second area to obtain an output feature value, and the second area is a different area from the first area.

[0179] In a possible implementation, the aforementioned detection model is specifically used for performing 3D object detection.

[0180] In a possible implementation, the data in the aforementioned first data set includes images or point clouds.

[0181] like Figure 11 , which is a schematic diagram of the hardware structure of a detection device 110 provided in an embodiment of the present application, and the detection device can be used to perform the steps of the aforementioned training phase.

[0182] Figure 11 The detection device 110 shown may include: a processor 1101 , a memory 1102 , a communication interface 1103 , and a bus 1104 . The processor 1101 , the memory 1102 , and the communication interface 1103 may be connected via the bus 1104 .

[0183] The processor 1101 is the control center of the detection device 110 and can be a general-purpose central processing unit (CPU) or other general-purpose processors, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0184] As an example, the processor 1101 may include one or more CPUs, such as Figure 11 CPU 0 and CPU 1 are shown in Figure 1.

[0185] The memory 1102 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0186] In one possible implementation, memory 1102 may exist independently of processor 1101. Memory 1102 may be connected to processor 1101 via bus 1104 and used to store data, instructions, or program codes. When processor 1101 calls and executes the instructions or program codes stored in memory 1102, the method provided in the embodiments of the present application can be implemented.

[0187] In another possible implementation, the memory 1102 may also be integrated with the processor 1101 .

[0188] The communication interface 1103 is used to connect the detection device 110 to other devices via a communication network, which can be Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc. The communication interface 1103 can include a receiving unit for receiving data and a sending unit for sending data.

[0189] The bus 1104 may be an ISA bus, a PCI bus, or an EISA bus. The bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of presentation, Figure 11 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0190] It should be pointed out that Figure 11 The structure shown in the figure does not constitute a limitation on the detection device 110, except Figure 11 In addition to the components shown, the detection device 110 may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0191] Optionally, the aforementioned Figure 11 The detection device shown in is a chip.

[0192] The present application also provides a digital processing chip. The digital processing chip integrates circuitry and one or more interfaces for implementing the aforementioned processor 1101 or the functions of processor 1101. When the digital processing chip integrates memory, it can perform the method steps of any one or more of the aforementioned embodiments. When the digital processing chip does not integrate memory, it can be connected to an external memory via a communication interface. The digital processing chip implements the actions performed by the detection device in the aforementioned embodiments based on program code stored in the external memory.

[0193] The present application also provides a computer program product which, when executed on a computer, enables the computer to execute the aforementioned Figure 5-Figure 9 The illustrated embodiments describe steps in a method.

[0194] The detection device provided in the embodiment of the present application can be a chip, which includes: a processing unit and a communication unit. The processing unit can be, for example, a processor, and the communication unit can be, for example, an input / output interface, a pin or a circuit. The processing unit can execute the computer execution instructions stored in the storage unit, so that the chip can execute the above Figure 5-Figure 9 The method described in the embodiment shown. Optionally, the storage unit is a storage unit within the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip within the wireless access device, such as a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM), etc.

[0195] Specifically, the aforementioned processing unit or processor may be a central processing unit (CPU), a neural-network processing unit (NPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general processor may be a microprocessor or any conventional processor, etc. The processing unit or processor may be used to execute the aforementioned Figure 5-Figure 9 The corresponding method steps.

[0196] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0197] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general-purpose hardware, and of course can also be implemented by dedicated hardware including application-specific integrated circuits, dedicated CPUs, dedicated memories, dedicated components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits, or dedicated circuits. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc., including a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0198] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0199] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website, a computer, a server, or a data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0200] The terms "first," "second," "third," "fourth," and the like in the specification and claims of this application and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions, e.g., a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or apparatus.

[0201] Finally, it should be noted that the above is only a specific implementation method of the present application, but the protection scope of the present application is not limited to this. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in this application, which should be covered by the protection scope of the present application.

Claims

1. A detection method, characterized in that: include: Acquire a first data set, where the first data set includes the unlabeled data, and a first label corresponding to the unlabeled data includes noise; The first data set is used as the input of the detection model to obtain a second data set, wherein the second data set includes the unlabeled data and a second label corresponding to the unlabeled data, the noise included in the second label corresponding to the unlabeled data is lower than the noise included in the first label, and the detection model is used to output the label corresponding to the unlabeled data, and the training set used when pre-training the detection model includes labeled data and unlabeled data, wherein the detection model includes at least one of the first module or the second module, the first module is used to search from a feature map extracted based on the input data to obtain an output feature value, and the second module is used to obtain multiple output results based on the output feature value, and the first label of the input data is adjusted according to the multiple output results to obtain a second label, and the multiple output results are different.

2. The method according to claim 1, characterized in that The second module includes multiple sub-modules, which are used to output the multiple output results. The multiple sub-modules are sub-modules that are pre-trained using a training set. The data proportions used in the pre-training process of each of the multiple sub-modules are different. The data proportion is the proportion of labeled data or unlabeled data in the training set, and the labeled data is the labeled data.

3. The method according to claim 2, characterized in that The difference between the outputs of the multiple sub-modules is used to obtain a second label corresponding to the input data input to the detection model based on the first label corresponding to the input data.

4. The method according to claim 3, characterized in that The plurality of submodules include a first submodule and a second submodule; During the pre-training process, the proportion of labeled data corresponding to the first submodule is greater than the proportion of labeled data corresponding to the second submodule; The second label corresponding to the input data is obtained by weighted fusion of the first label corresponding to the input data and the first divergence value, the first divergence value includes the difference between the output value of the first submodule and the output value of the second submodule, and the value of the second label corresponding to the input data is positively correlated with the first divergence value.

5. The method according to claim 4, characterized in that The plurality of submodules further includes a third submodule; During the pre-training process, the proportion of unlabeled data corresponding to the third submodule is smaller than the proportion of unlabeled data corresponding to the first submodule, and the proportion of unlabeled data corresponding to the third submodule is smaller than the proportion of unlabeled data corresponding to the second submodule; The second divergence value is used to update the learning weight, and the second divergence value is used to represent the difference between the output result of the third submodule and the output result of the first submodule. The learning weight includes the weight used to fuse the first label corresponding to the input data and the first divergence value.

6. The method according to any one of claims 1 to 5, characterized in that The first module is specifically used to: Extract features from input data to obtain feature maps; Based on the first label corresponding to the input data, sampling is performed from the feature map to obtain the output feature value.

7. The method according to claim 6, characterized in that The sampling from the feature map based on the first label corresponding to the input data to obtain the output feature value includes: Obtain a first area corresponding to the first label in the feature map; The second area in the feature map is searched based on the confidence corresponding to each eigenvalue in the second area to obtain the output eigenvalue, and the second area is a different area from the first area.

8. The method according to any one of claims 1 to 7, characterized in that The detection model is specifically used for performing 3D object detection.

9. The method according to any one of claims 1 to 8, characterized in that The data in the first data set includes images or point clouds.

10. A detection device, characterized in that: include: an acquisition module, configured to acquire a first data set, wherein the first data set includes the unlabeled data, and a first label corresponding to the unlabeled data includes noise; A detection module is used to use the first data set as the input of a detection model to obtain a second data set, wherein the second data set includes the unlabeled data and a second label corresponding to the unlabeled data, the noise included in the second label corresponding to the unlabeled data is lower than the noise included in the first label, and the detection model is used to output the label corresponding to the unlabeled data. The training set used when pre-training the detection model includes labeled data and unlabeled data, wherein the detection model includes at least one of the first module or the second module, the first module is used to search from a feature map extracted based on the input data to obtain an output feature value, and the second module is used to obtain multiple output results based on the output feature value, and the first label of the input data is adjusted according to the multiple output results to obtain a second label, and the multiple output results are different.

11. The device according to claim 10, characterized in that The second module includes multiple sub-modules, which are used to output the multiple output results. The multiple sub-modules are sub-modules that are pre-trained using training gold. The data ratios used in the pre-training process of each of the multiple sub-modules are different. The data ratio is the ratio of labeled data or unlabeled data in the training set, and the labeled data is the labeled data.

12. The device according to claim 11, characterized in that The difference between the outputs of the multiple sub-modules is used to obtain a second label corresponding to the input data input to the detection model based on the first label corresponding to the input data.

13. The device according to claim 12, characterized in that The plurality of submodules include a first submodule and a second submodule; During the pre-training process, the proportion of labeled data corresponding to the first submodule is greater than the proportion of labeled data corresponding to the second submodule; The second label corresponding to the input data is obtained by weighted fusion of the first label corresponding to the input data and the first divergence value, the first divergence value includes the difference between the output value of the first submodule and the output value of the second submodule, and the value of the second label corresponding to the input data is positively correlated with the first divergence value.

14. The device according to claim 13, characterized in that The plurality of submodules further includes a third submodule; During the pre-training process, the proportion of unlabeled data corresponding to the third submodule is smaller than the proportion of unlabeled data corresponding to the first submodule, and the proportion of unlabeled data corresponding to the third submodule is smaller than the proportion of unlabeled data corresponding to the second submodule; The second divergence value is used to update the learning weight, and the second divergence value is used to represent the difference between the output result of the third submodule and the output result of the first submodule. The learning weight includes the weight used to fuse the first label corresponding to the input data and the first divergence value.

15. The device according to any one of claims 10 to 14, characterized in that The first module is specifically used to: Extract features from input data and obtain feature maps. Based on the first label corresponding to the input data, sampling is performed from the feature map to obtain the output feature value.

16. The device according to claim 15, characterized in that The first module is specifically used to: Obtain a first area corresponding to the first label in the feature map; The second area in the feature map is searched based on the confidence corresponding to each eigenvalue in the second area to obtain the output eigenvalue, and the second area is a different area from the first area.

17. The device according to any one of claims 10 to 16, characterized in that The detection model is specifically used for performing 3D object detection.

18. The device according to any one of claims 10 to 17, characterized in that The data in the first data set includes images or point clouds.

19. A detection device, characterized in that: The method comprises a processor coupled to a memory, wherein the memory stores a program, and when the program instructions stored in the memory are executed by the processor, the steps of the method according to any one of claims 1 to 9 are implemented.

20. A computer-readable storage medium comprising a program, which, when executed by a processing unit, performs the steps of the method according to any one of claims 1 to 9.

21. A computer program product, characterized in that The computer program product comprises software code for executing the steps of the method according to any one of claims 1 to 9.

Images