Labeling method and apparatus

Abstract

This application provides an annotation method and apparatus. The method includes: annotating objects in an image to be annotated according to each of N target annotation models to obtain N sets of annotation results; clustering candidate annotations in the N sets of annotation results to obtain at least one target cluster; and determining at least one target annotation based on the at least one target cluster, wherein the target annotation is an annotation identifier for annotating objects in the image to be annotated. This application improves the annotation coverage of objects in the image to be annotated, and improves the annotation accuracy and recall.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a labeling method and apparatus. Background Technology

[0002] Object detection technology, also known as object extraction, is a comprehensive technique for identifying, locating, and outlining the coordinates of objects (such as objects or people) in an image. In other words, it is not only necessary to identify objects in an image, but also to label the specific location of the objects in the image.

[0003] With the rapid development of computer technology and neural networks, key breakthroughs have been achieved in neural network-based object detection technology. This technology involves training a neural network with a large number of labeled samples (images labeled with objects) to obtain an object detection model. Then, when detecting objects in an image, the image is input into the object detection model, which outputs the detection result. As can be seen, neural network-based object detection technology requires a large number of labeled samples.

[0004] Currently, to improve the efficiency and reduce the cost of acquiring labeled samples, automatic annotation is commonly used to label objects in the samples. The automatic annotation method involves training an annotation model that outputs bounding boxes and a confidence score for each object on the image to be labeled. Then, a confidence threshold is set, and bounding boxes with confidence scores greater than the threshold are identified as labeled results for that image. Clearly, in this method, while a low confidence threshold can retain most correct bounding boxes, it also results in a large number of false positives, leading to a decrease in annotation accuracy. Conversely, while a high confidence threshold can filter out many false positives, it can also result in the rejection of correct bounding boxes, reducing the annotation recall rate. Summary of the Invention

[0005] This application provides a labeling method and apparatus to address the problems of low accuracy and recall in labeling.

[0006] In a first aspect, a labeling method is provided, comprising: labeling objects in an image to be labeled according to each of N target labeling models to obtain N sets of labeling results; clustering candidate labeling identifiers in the N sets of labeling results to obtain at least one target cluster; and determining at least one target labeling identifier according to the at least one target cluster, wherein the target labeling identifier is a labeling identifier for labeling objects in the image to be labeled.

[0007] By using N target annotation models to annotate objects in the image to be annotated, N sets of annotation results are obtained. Then, the N sets of annotation results are clustered to obtain at least one target cluster. At least one target label is determined based on the at least one target cluster. Compared with a single target annotation model, this method improves the annotation coverage of objects in the image to be annotated. Furthermore, by determining the target label by clustering the N sets of annotation results, mislabeled labels can be accurately filtered out and correctly labeled labels can be accurately retained. Compared with the method of determining the target label based on the confidence of a single label, this method improves the accuracy and recall of annotation.

[0008] In one possible implementation, the method further includes: determining N target annotation models from T candidate annotation models, where T ≥ N.

[0009] In one possible implementation, determining N target annotation models from T candidate annotation models includes: dividing the labeled area into S area intervals based on the labeled area of ​​objects in each test sample in the test set; annotating the objects in each test sample based on each candidate annotation model to obtain the annotation identifier of each candidate annotation model on each test sample; calculating the recall rate of each candidate annotation model in each area interval based on the area of ​​the annotation identifier of each candidate annotation model in each test sample and the S area intervals; and determining N target annotation models from the T candidate annotation models based on the recall rate of each candidate annotation model in each area interval.

[0010] In one possible implementation, determining N target labeled models from the T candidate labeled models based on the recall rate of each candidate labeled model in each area interval includes: sorting the candidate labeled models in descending order of recall rate for each area interval based on the recall rate of each candidate labeled model in each area interval; selecting the top K candidate labeled models for each area interval; and determining N target labeled models based on the top K candidate labeled models corresponding to each area interval.

[0011] N target annotation models are formed by selecting candidate annotation models with the highest recall rates within each area interval. This improves the annotation accuracy, coverage, and recall of the N target annotation models for objects within multiple area intervals, thereby increasing the annotation coverage, accuracy, and recall of the N target annotation models for objects in the image to be annotated.

[0012] In one possible implementation, clustering the candidate label identifiers in the N sets of annotation results to obtain at least one target cluster includes: clustering the candidate label identifiers in the N sets of annotation results to obtain at least one candidate cluster; wherein each of the at least one candidate cluster includes at least one candidate label identifier, and the candidate label identifiers included in a candidate cluster correspond to different target annotation models; and determining the at least one target cluster in the at least one candidate cluster based on the number of candidate label identifiers in each candidate cluster and in combination with a preset number.

[0013] In one possible implementation, clustering the candidate label identifiers in the N groups of annotation results includes: clustering the candidate label identifiers in the N groups of annotation results based on a density clustering algorithm.

[0014] In one possible implementation, the clustering of candidate labels in the N sets of annotation results based on the density clustering algorithm includes: calculating the IOU between any two candidate labels in the N sets of annotation results; and clustering the candidate labels in the N sets of annotation results based on the IOU between the two candidate labels, the IOU threshold, and the neighborhood density threshold, combined with the density clustering algorithm.

[0015] In one possible implementation, clustering the candidate label identifiers in the N sets of annotation results to obtain at least one candidate cluster includes: calculating the IOU between any two candidate label identifiers in the N sets of annotation results; determining a first candidate label identifier corresponding to each candidate label identifier in the N sets of annotation results based on the IOU between the two candidate label identifiers; wherein, the second candidate label identifier is any one of the candidate label identifiers in the N sets of annotation results, the first candidate label identifier corresponding to the second candidate label identifier is a candidate label identifier in the N sets of annotation results whose IOU with the second candidate label identifier is less than an IOU threshold, and the target annotation model corresponding to the first candidate label identifier is different from the target annotation model corresponding to the second candidate label identifier, and the target annotation models corresponding to the first candidate label identifiers are different; determining the score of each candidate label identifier based on the number of first candidate label identifiers corresponding to each candidate label identifier; in the A third candidate label is determined from the N sets of annotation results, wherein the third candidate label is the candidate label with the highest score in the N sets of annotation results; a fourth candidate label is determined from the N sets of annotation results, wherein the fourth candidate label is a candidate label in the N sets of annotation results whose IOU with the third candidate label is less than the IOU threshold and whose score is greater than a preset score, and the target annotation model corresponding to the fourth candidate label is different from that of the third candidate label, and the target annotation models corresponding to the fourth candidate labels are different from each other; the third candidate label and the fourth candidate label are combined to obtain a candidate cluster; the N sets of annotation results are updated by removing the third candidate label and the fourth candidate label from the N sets of annotation results; the third candidate label, the fourth candidate label, and the candidate cluster are determined again in the updated N sets of annotation results, until the updated N sets of annotation results are empty.

[0016] In one possible implementation, determining at least one target label based on the at least one target cluster comprises: determining at least one target label based on the at least one target cluster and in conjunction with a calibration model, wherein the calibration model is trained by a second neural network.

[0017] In one possible implementation, for each target cluster, determining at least one target label based on the at least one target cluster and a correction model includes: supplementing the candidate labels in the first target cluster to N to obtain a second target cluster, wherein the first target cluster is any one of the at least one target cluster; performing feature extraction on the image to be labeled to obtain a feature image of the image to be labeled; obtaining N ROI feature maps based on the feature image, the second target cluster, and a ROI algorithm, wherein the N ROI feature maps correspond one-to-one with the N candidate labels in the second target cluster; and performing correction and fusion on the N ROI feature maps to obtain the target label corresponding to the first target cluster.

[0018] In one possible implementation, the correction and fusion of the N ROI feature maps includes: calculating the error of each ROI feature map based on each of the N ROI feature maps and combining it with an error calculation sub-model, wherein the error calculation sub-model is part of the correction model, and the error of each ROI feature map is used to indicate the error of the candidate label corresponding to each ROI feature map; correcting the corresponding candidate label based on the error of each ROI feature map to obtain N corrected candidate label labels; and averaging the N corrected candidate label labels to obtain the target label label corresponding to the first target cluster.

[0019] By calculating the error of each ROI feature map, correcting the corresponding candidate labels based on the error of each ROI feature map, averaging each corrected candidate label, and determining the target label based on the average result, the accuracy of the target label is improved, enabling the target label to more accurately mark the position of the object in the image to be labeled, thus making the target label more consistent with the actual label of the object.

[0020] In one possible implementation, the correction and fusion of the N ROI feature maps includes: determining a multi-channel heatmap corresponding to the first target cluster based on each ROI feature map in the N ROI feature maps and in conjunction with a probability calculation sub-model; wherein the probability calculation sub-model is part of the correction model, and the multi-channel heatmap is used to indicate the probability that the boundary point on each candidate label in the first target cluster is the corresponding boundary point on the target label corresponding to the first target cluster; determining the target boundary point based on the probability that the boundary point on each candidate label in the multi-channel heatmap is the corresponding boundary point on the target label corresponding to the first target cluster; and determining the target label corresponding to the first target cluster based on the target boundary point.

[0021] The multi-channel heatmap corresponding to the first target cluster is determined by a probability calculation sub-model. Then, the target boundary point is determined based on the probability that the boundary point on each candidate label in the multi-channel heatmap is the corresponding boundary point on the target label of the first target cluster. That is, the boundary point most likely to be the target label is selected from the boundary points of the multi-channel heatmap, and the target label is determined based on the selected boundary point. In other words, the target label is determined by the boundary point most likely to be the target label, which improves the accuracy of the target label and enables the target label to more accurately mark the position of the object in the image to be labeled, so that the target label is more consistent with the actual label of the object.

[0022] In a second aspect, an annotation device is provided, comprising: an annotation module, configured to annotate objects in an image to be annotated according to each of N target annotation models, thereby obtaining N sets of annotation results; a clustering module, configured to cluster candidate annotation identifiers in the N sets of annotation results, thereby obtaining at least one target cluster; and a first determining module, configured to determine at least one target annotation identifier based on the at least one target cluster, wherein the target annotation identifier is an annotation identifier for annotating objects in the image to be annotated.

[0023] In one possible implementation, the apparatus further includes: a second determining module, configured to determine N target labeled models from T candidate labeled models, wherein T ≥ N.

[0024] In one possible implementation, the second determining module includes: a partitioning unit, configured to divide the labeled area into S area intervals based on the labeled area of ​​the object in each test sample in the test set; a labeling unit, configured to label the object in each test sample based on each candidate labeling model, to obtain the label identifier of each candidate labeling model on each test sample; a calculation unit, configured to calculate the recall rate of each candidate labeling model in each area interval based on the area of ​​the label identifier of each candidate labeling model in each test sample and the S area intervals; and a first determining unit, configured to determine N target labeling models from the T candidate labeling models based on the recall rate of each candidate labeling model in each area interval.

[0025] In one possible implementation, the first determining unit is specifically configured to: sort the candidate labeled models in descending order of recall rate for each area interval based on the recall rate of each candidate labeled model in each area interval; select the top K candidate labeled models for each area interval; and determine N target labeled models based on the top K candidate labeled models corresponding to each area interval.

[0026] In one possible implementation, the clustering module includes: a clustering unit, configured to cluster candidate label identifiers in the N sets of annotation results to obtain at least one candidate cluster; wherein each candidate cluster includes at least one candidate label identifier, and the candidate label identifiers included in a candidate cluster correspond to different target annotation models; and a second determining unit, configured to determine the at least one target cluster in the at least one candidate cluster based on the number of candidate label identifiers in each candidate cluster and in combination with a preset number.

[0027] In one possible implementation, the clustering unit is specifically used to cluster candidate label identifiers in the N groups of labeling results based on a density clustering algorithm.

[0028] In one possible implementation, the clustering unit is specifically used to calculate the IOU between any two candidate labels in the N sets of annotation results; and to cluster the candidate labels in the N sets of annotation results based on the IOU between the two candidate labels, the IOU threshold, and the neighborhood density threshold, combined with the density clustering algorithm.

[0029] In one possible implementation, the clustering unit includes: a first calculation subunit, configured to calculate the Interchange of Union (IOU) between any two candidate label identifiers in the N sets of annotation results; a first determination subunit, configured to determine a first candidate label identifier corresponding to each candidate label identifier in the N sets of annotation results based on the IOU between the two candidate label identifiers; wherein, the second candidate label identifier is any one of the candidate label identifiers in the N sets of annotation results, the first candidate label identifier corresponding to the second candidate label identifier is a candidate label identifier in the N sets of annotation results whose IOU with the second candidate label identifier is less than an IOU threshold, and the target labeling model corresponding to the first candidate label identifier is different from the target labeling model corresponding to the second candidate label identifier, and the target labeling models corresponding to the first candidate label identifiers are different; a second determination subunit, configured to determine the score of each candidate label identifier based on the number of first candidate label identifiers corresponding to each candidate label identifier; and a third determination subunit, configured to determine a third [missing information] in the N sets of annotation results. The system comprises: a candidate label identifier, wherein the third candidate label identifier is the candidate label identifier with the highest score among the N groups of labeling results; a fourth determining subunit, configured to determine a fourth candidate label identifier among the N groups of labeling results, wherein the fourth candidate label identifier is a candidate label identifier among the N groups of labeling results whose IOU with the third candidate label identifier is less than an IOU threshold and whose score is greater than a preset score, and the target labeling model corresponding to the fourth candidate label identifier is different from that of the third candidate label identifier, and the target labeling models corresponding to the fourth candidate label identifiers are different; a set subunit, configured to set the third candidate label identifier and the fourth candidate label identifier to obtain a candidate cluster; a removal subunit, configured to update the N groups of labeling results by removing the third candidate label identifier and the fourth candidate label identifier from the N groups of labeling results; and a fifth determining subunit, configured to re-determine the third candidate label identifier, the fourth candidate label identifier, and the candidate cluster in the updated N groups of labeling results, until the updated N groups of labeling results are empty.

[0030] In one possible implementation, the first determining module is specifically configured to determine at least one target label based on the at least one target cluster and in conjunction with a correction model, wherein the correction model is trained by a second neural network.

[0031] In one possible implementation, for each target cluster, the first determining module includes: a supplementation unit, configured to supplement the candidate labels in the first target cluster to N, to obtain a second target cluster, wherein the first target cluster is any one of the at least one target cluster; an extraction unit, configured to perform feature extraction on the image to be labeled, to obtain a feature image of the image to be labeled; an acquisition unit, configured to acquire N ROI feature maps based on the feature image, the second target cluster, and in combination with an ROI algorithm, wherein the N ROI feature maps correspond one-to-one with the N candidate labels in the second target cluster; and a correction unit, configured to correct and fuse the N ROI feature maps to obtain the target label corresponding to the first target cluster.

[0032] In one possible implementation, the correction unit includes: a second calculation subunit, configured to calculate the error of each of the N ROI feature maps based on each ROI feature map and in conjunction with an error calculation submodel, wherein the error calculation submodel is part of the correction model, and the error of each ROI feature map is used to indicate the error of the candidate label corresponding to each ROI feature map; a correction subunit, configured to correct the corresponding candidate label based on the error of each ROI feature map to obtain N corrected candidate label labels; and an averaging subunit, configured to average the N corrected candidate label labels to obtain the target label label corresponding to the first target cluster.

[0033] In one possible implementation, the correction unit includes: a sixth determining subunit, configured to determine a multi-channel heatmap corresponding to the first target cluster based on each ROI feature map in the N ROI feature maps and in conjunction with a probability calculation submodel; wherein the probability calculation submodel is part of the correction model, and the multi-channel heatmap is used to indicate the probability that the boundary point on each candidate label in the first target cluster is the corresponding boundary point on the target label corresponding to the first target cluster; a seventh determining subunit, configured to determine a target boundary point based on the probability that the boundary point on each candidate label in the multi-channel heatmap is the corresponding boundary point on the target label corresponding to the first target cluster; and an eighth determining subunit, configured to determine the target label corresponding to the first target cluster based on the target boundary point.

[0034] Thirdly, a computer-readable storage medium is provided, including a computer program that, when executed on a computer, causes the computer to perform the method described in any one of the first aspects.

[0035] Fourthly, a computer program is provided, which, when executed by a computer, performs the method described in any one of the first aspects.

[0036] Fifthly, a chip is provided, including a processor and a memory, the memory for storing a computer program, and the processor for calling and running the computer program stored in the memory to perform the method described in any one of the first aspects. Attached Figure Description

[0037] Figure 1 A flowchart illustrating an annotation method provided in an embodiment of this application;

[0038] Figure 2 This application provides a flowchart illustrating the clustering process for candidate annotations from N sets of annotation results. Figure 1 :

[0039] Figure 3 This application provides a flowchart illustrating the clustering process for candidate annotations from N sets of annotation results. Figure 2 ;

[0040] Figure 4 A schematic diagram illustrating the process of determining target labels based on at least one target cluster and a correction model, as provided in an embodiment of this application;

[0041] Figure 5 The flowchart of correcting and fusing N ROI feature maps provided in this application embodiment is as follows. Figure 1 ;

[0042] Figure 6 The flowchart of correcting and fusing N ROI feature maps provided in this application embodiment is as follows. Figure 2 ;

[0043] Figure 7 The structural schematic diagram of the calibration model provided in this application embodiment Figure 1 ;

[0044] Figure 8 The structural schematic diagram of the calibration model provided in this application embodiment Figure 2 ;

[0045] Figure 9 This application provides a flowchart illustrating the process of determining N target annotation models in its embodiments;

[0046] Figure 10 This is a schematic diagram of a labeling device provided in an embodiment of this application. Detailed Implementation

[0047] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0049] The terms "first," "second," etc., used in the specification, embodiments, claims, and drawings of this application are for distinguishing purposes only and should not be construed as indicating or implying relative importance or order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as including a series of steps or units. A method, system, product, or apparatus is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses.

[0050] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0051] Figure 1 This is a flowchart illustrating an annotation method provided in an embodiment of this application. This annotation method can be applied to scenarios involving the annotation of objects in images. The executing entity of this annotation method can be, for example, a single server or a server cluster consisting of multiple servers; this application does not impose any special limitations on this. Figure 1 As shown, the method includes the following steps:

[0052] Step 101: Label the objects in the image to be labeled according to each of the N target labeling models, and obtain N sets of labeling results. Among them, the N sets of labeling results correspond one-to-one with the N target labeling models. Each set of labeling results includes the candidate label identifiers of the corresponding target labeling model for the objects in the image to be labeled. Each target labeling model is trained by a first neural network, and N≥2.

[0053] In this embodiment, the objects in the image to be labeled can be, for example, pedestrians, vehicles, or other objects; this application does not impose any special limitations on this. The first neural network can be, for example, a convolutional neural network, a fast deep convolutional neural network, or other neural networks applicable to image processing; this application does not impose any special limitations on this.

[0054] Each target annotation model is trained by a first neural network, meaning each target annotation model corresponds to one first neural network. The type of the first neural network corresponding to each target annotation model can be the same, completely different, or not entirely the same; this application does not impose any special limitations on this. The structure of the first neural network corresponding to each target annotation model is different. The structure of the first neural network refers to the number of neural network layers included in the first neural network, the number of neurons in each neural network layer, the connection method between adjacent neural network layers, and the loss function in the first neural network, etc.

[0055] The process of training the first neural network corresponding to the target annotation model to obtain the target annotation model includes: firstly, acquiring training samples, which consist of a large number of images; annotating the objects in each image in the training samples to obtain annotated training samples; then, inputting the annotated training samples into the first neural network corresponding to the target annotation model to train the first neural network; and finally, determining the trained first neural network as the target annotation model. It should be noted that the purpose of training the first neural network is to obtain the weight values ​​of each parameter in the first neural network, so that the first neural network (i.e., the target annotation model) based on the weight values ​​of each parameter can achieve the desired annotation effect.

[0056] It should be noted that since the training principle of the first neural network corresponding to each target annotation model is the same, and the training principle of the first neural network corresponding to one target annotation model has already been explained above, the training principle of the first neural network corresponding to other target annotation models will not be repeated here.

[0057] The principle of labeling objects in an image to be labeled based on a target labeling model is as follows:

[0058] The image to be labeled is input into the target labeling model, which then outputs the labels for the objects detected in the image. For ease of distinction, these labels are named candidate labels. After collecting the candidate labels for the objects detected in the image, a set of labeling results corresponding to the target labeling model is obtained. These candidate labels can be, for example, bounding boxes; this application does not impose any special limitations on this.

[0059] It should be noted that since each target annotation model uses the same principle to annotate objects in the image to be annotated, and the principle of one target annotation model annotating objects in the image to be annotated has already been explained above, the principle of other target annotation models annotating objects in the image to be annotated will not be explained here.

[0060] As shown above, by annotating the objects in the image to be annotated by each target annotation model, a set of annotation results corresponding to each target annotation model can be obtained, that is, N annotation results are finally obtained.

[0061] Step 102: Cluster the candidate labels in the N groups of annotation results to obtain at least one target cluster. Each target cluster includes multiple candidate labels. The number of candidate labels included in each target cluster is greater than M, and M≤N. The candidate labels included in a target cluster correspond to different target annotation models. That is, a target annotation model provides at most one candidate label for each target cluster.

[0062] In this embodiment, candidate annotations from N groups of annotation results are first clustered to obtain at least one candidate cluster. Each candidate cluster includes at least one candidate annotation, and the candidate annotations within a candidate cluster correspond to different target annotation models. That is, a target annotation model provides at most one candidate annotation for each candidate cluster. Then, based on the number of candidate annotations in each candidate cluster and a preset number, at least one target cluster is determined from the at least one candidate cluster. In other words, candidate clusters with a number of candidate annotations greater than the preset number are determined as target clusters.

[0063] Specifically, the clustering of candidate labels in N sets of annotation results can be performed by: clustering candidate labels in N sets of annotation results based on density clustering algorithm, or clustering candidate labels in N sets of annotation results based on distance clustering algorithm, or clustering candidate labels in N sets of annotation results based on custom clustering algorithm, etc. This application does not impose any special limitations on this.

[0064] The following will illustrate the clustering process using a density-based clustering algorithm, with the Intersection over Union (IOU) between candidate labels as the distance metric. Based on this, the process of clustering candidate labels from N sets of labeling results using the density-based clustering algorithm to obtain at least one candidate cluster includes:

[0065] First, calculate the IOU (intersection to union ratio) between any two candidate labels in the N sets of annotation results. The calculation process for the IOU between any two candidate labels is as follows: calculate the intersection between any two candidate labels (e.g., the intersection between any two candidate bounding boxes), calculate the union between any two candidate labels (e.g., the union between any two candidate bounding boxes), and determine the ratio of the intersection to the union between any two candidate labels as the IOU between any two candidate labels.

[0066] Then, based on the IOU between any two candidate labels, the IOU threshold, and the neighborhood density threshold, and combined with a density clustering algorithm, the candidate labels in the N sets of annotation results are clustered. The IOU threshold can be set according to the clustering effect; this application does not impose any special limitations on it. For example, the IOU threshold can be set to 0.8. The specific value of the neighborhood density threshold can be set according to the clustering effect and the number of target annotation models, etc.; this application does not impose any special limitations on it.

[0067] like Figure 2 As shown, the specific process of clustering candidate labels in N sets of annotation results based on the IOU between any two candidate labels, the IOU threshold, and the neighborhood density threshold, combined with the density clustering algorithm, is as follows:

[0068] Step 201: Determine the neighborhood of each candidate label in the N sets of annotation results based on the IOU and IOU threshold between any two candidate labels. The neighborhood of each candidate label consists of candidate labels whose IOU with the corresponding candidate label is greater than the IOU threshold. The target label model corresponding to a candidate label is different from the target label model corresponding to the candidate label in its neighborhood. The target label models corresponding to the candidate labels in the neighborhood of a candidate label are different. That is, a target label model provides at most one candidate label for the neighborhood of each candidate label.

[0069] Step 202: Count the number of candidate labels in the neighborhood of each candidate label.

[0070] Step 203: Select a candidate label from the N groups of labeling results. If the selected candidate label has not been labeled, mark the selected candidate label as the first candidate label.

[0071] Step 204: Determine whether the number of candidate labels in the neighborhood of the first candidate label is greater than the neighborhood density threshold.

[0072] Step 205: If yes, then create a candidate cluster corresponding to the first candidate label and add the first candidate label to the corresponding candidate cluster.

[0073] Step 206: Select a candidate label in the neighborhood of the first candidate label. If the selected candidate label has not been labeled, mark the selected candidate label as the second candidate label.

[0074] Step 207: Determine whether the number of candidate labels in the neighborhood of the second candidate label is greater than the neighborhood density threshold.

[0075] Step 208: If the number of candidate labels in the neighborhood of the second candidate label is greater than the neighborhood density threshold, and the second candidate label does not belong to any candidate cluster, and the target label model corresponding to the second candidate label is different from the target label model corresponding to each candidate label in the candidate cluster corresponding to the first candidate label, then add the candidate label in the neighborhood of the second candidate label to the neighborhood of the first candidate label to update the neighborhood of the first candidate label, and add the second candidate label to the candidate cluster corresponding to the first candidate label to update the candidate cluster corresponding to the first candidate label.

[0076] Step 209: If the number of candidate labels in the neighborhood of the second candidate label is not greater than the neighborhood density threshold, and the second candidate label does not belong to any candidate cluster, and the target label model corresponding to the second candidate label is different from the target label model corresponding to each candidate label in the candidate cluster corresponding to the first candidate label, then add the second candidate label to the candidate cluster corresponding to the first candidate label to update the candidate cluster corresponding to the first candidate label.

[0077] Step 210: Select a candidate label from the neighborhood of the first candidate label (if updated, the updated neighborhood; if not updated, the unupdated neighborhood). If the selected candidate label has not been labeled, mark the selected candidate label as the second candidate label. Repeat steps 207 to 210 based on the second candidate label marked in this step until all candidate labels in the neighborhood corresponding to the first candidate label (i.e., the finally updated neighborhood) are labeled.

[0078] Step 211: After all candidate labels in the neighborhood (i.e. the finally updated neighborhood) corresponding to the first candidate label are marked, the last updated candidate cluster corresponding to the first candidate label is determined as the finally determined candidate cluster corresponding to the first candidate label.

[0079] Step 212: After all candidate labels in the neighborhood (i.e. the finally updated neighborhood) corresponding to the first candidate label are labeled, repeat steps 203 to 212 to obtain other candidate clusters corresponding to the first candidate label, until all candidate labels in the N sets of labeling results are labeled.

[0080] Based on the above steps, the clustering of N sets of annotation results can be completed to obtain at least one candidate cluster.

[0081] As shown above, the clustering process described is actually a process of clustering candidate labels for the same object using different target annotation models. That is, one candidate cluster corresponds to one labeled object, and different candidate clusters correspond to different labeled objects. Based on this, since the target cluster is determined from the candidate clusters, and the target cluster is a candidate cluster where the number of candidate labels is greater than a preset number, one target cluster corresponds to one correctly labeled object.

[0082] Since the above clustering process only clusters candidate labels corresponding to different target labeling models, in order to reduce the amount of computation and improve the clustering efficiency, when calculating the IOU between any two candidate labels in the N groups of labeling results, we can only calculate the IOU between any two candidate labels with different target labeling models in the N groups of labeling results.

[0083] It should be noted that the above clustering process, which uses the IOU between candidate labels as the distance metric of the density clustering algorithm, is merely exemplary and is not intended to limit the present invention. For example, the positional distance between candidate labels can also be used as the distance metric of the density clustering algorithm to cluster the candidate labels in N sets of labeling results.

[0084] Next, we will use a custom clustering algorithm as an example to illustrate the clustering process. The specific clustering process is as follows: Figure 3 As shown:

[0085] Step 301: Calculate the IOU between any two candidate labels in the N sets of annotation results. Since this process has already been explained above, it will not be repeated here.

[0086] Step 302: Determine the first candidate label corresponding to each candidate label in the N sets of annotation results based on the IOU between any two candidate label labels; wherein, the second candidate label is any candidate label in the N sets of annotation results, and the first candidate label corresponding to the second candidate label is a candidate label in the N sets of annotation results whose IOU with the second candidate label is less than the IOU threshold, and the target annotation model corresponding to the first candidate label is different from the target annotation model corresponding to the second candidate label, and the target annotation models corresponding to the first candidate labels corresponding to the second candidate label are different, that is, one target annotation model can provide at most one first candidate label.

[0087] Step 303: Determine the score of each candidate label based on the number of first candidate labels corresponding to each candidate label. Specifically, for a candidate label, the number of first candidate labels corresponding to that candidate label is determined as the score of that candidate label. The principle for determining the score of each candidate label is the same, so it will not be repeated here.

[0088] Step 304: Determine the third candidate label in the N groups of labeling results, where the third candidate label is the candidate label with the highest score in the N groups of labeling results.

[0089] Step 305: Determine the fourth candidate annotation from the N sets of annotation results. The fourth candidate annotation is a candidate annotation in the N sets of annotation results whose IOU with the third candidate annotation is less than the IOU threshold and whose score is greater than a preset score. The target annotation models corresponding to the fourth candidate annotation are different from those corresponding to the third candidate annotation, and the target annotation models corresponding to the fourth candidate annotations are different from each other. The specific values ​​of the IOU threshold and the preset score can be set according to the specific scenario, and this application does not impose any special limitations on them.

[0090] Step 306: Combine the third candidate label and the fourth candidate label to obtain a candidate cluster.

[0091] Step 307: Update the N groups of annotation results by removing the third and fourth candidate annotations from the N groups of annotation results.

[0092] Step 308: In the updated N sets of annotation results, determine the third candidate annotation, the fourth candidate annotation, and the candidate cluster again, until the updated N sets of annotation results are empty. That is, determine the third candidate annotation in the updated N sets of annotation results and repeat steps 305 to 308 to obtain other candidate clusters, and end the clustering after the last updated N sets of annotation results are empty, that is, after the N sets of annotation results do not contain candidate annotations.

[0093] Based on the above steps, the N sets of annotation results can be clustered to obtain at least one candidate cluster.

[0094] As shown above, the clustering process described is actually a process of clustering candidate labels for the same object using different target annotation models. That is, one candidate cluster corresponds to one labeled object, and different candidate clusters correspond to different labeled objects. Based on this, since the target cluster is determined from the candidate clusters, and the target cluster is a candidate cluster where the number of candidate labels is greater than a preset number, one target cluster corresponds to one correctly labeled object.

[0095] Since the above clustering process only clusters candidate labels corresponding to different target labeling models, in order to reduce the amount of computation and improve the clustering efficiency, when calculating the IOU between any two candidate labels in the N groups of labeling results, we can only calculate the IOU between any two candidate labels with different target labeling models in the N groups of labeling results.

[0096] Step 103: Determine at least one target label based on at least one target cluster, wherein the target label is the label of the object in the image to be labeled.

[0097] In this embodiment of the application, since each target cluster includes multiple candidate labels, the method of determining at least one target label based on at least one target cluster may include the following two methods:

[0098] The first method involves determining any candidate label from the first target cluster as the target label corresponding to the first target cluster. The target label corresponding to the first target cluster is the label that labels the object corresponding to the first target cluster. Here, the first target cluster is any one of at least one target cluster. Clearly, the number of target clusters is the same as the number of target labels.

[0099] The second method involves averaging all candidate labels within the first target cluster. The resulting average label is then used as the target label for the first target cluster, which is the label used to label the object within that cluster. Here, the first target cluster can be any one of at least one target cluster. Clearly, the number of target clusters is the same as the number of target labels.

[0100] In summary, by using N target annotation models to annotate objects in the image to be annotated, resulting in N sets of annotation results, and then clustering these N sets of annotation results to obtain at least one target cluster, and determining at least one target label based on this target cluster, the annotation coverage of objects in the image to be annotated is improved compared to using a single target annotation model. Furthermore, determining the target label by clustering the N sets of annotation results can accurately filter out mislabeled labels and accurately retain correctly labeled labels. Compared to determining the target label based solely on the confidence level of a single label, this method improves both the accuracy and recall of the annotation process.

[0101] To improve the accuracy of target annotation, that is, to enable target annotation to more accurately identify the location of objects on the image to be annotated, and to avoid the problem that although the objects in the image are accurately annotated by the target annotation, there is an IOU difference between the target annotation (e.g., target bounding box) and the actual annotation (e.g., actual bounding box) that matches the object, a correction model can be provided when determining at least one target annotation based on at least one target cluster, and at least one target annotation can be determined based on at least one target cluster and combined with the correction model, wherein the correction model is trained by a second neural network.

[0102] In the embodiments of this application, such as Figure 4 As shown, for each target cluster, the steps for determining at least one target label based on at least one target cluster and in conjunction with the calibration model are as follows:

[0103] Step 401: Supplement the candidate labels in the first target cluster to N to obtain the second target cluster, wherein the first target cluster is any one of at least one target cluster.

[0104] Specifically, first, determine if the number of candidate labels in the first target cluster is equal to N. If yes, no further additions are needed to the candidate labels in the first target cluster; otherwise, additions are required. The addition method is as follows:

[0105] Averaging all candidate labels in the first target cluster yields an averaged label. This averaged label is then used to supplement the number of candidate labels in the first target cluster to N. In other words, the number of averaged labels added corresponds to the number of candidate labels missing from the first target cluster. For example, if the candidate labels are candidate bounding boxes, the coordinates of each of the four corners of all candidate bounding boxes in the first target cluster can be averaged to obtain the average coordinates of each of the four corners. Then, the averaged bounding box is constructed based on the average coordinates of each of the four corners, and this averaged bounding box is used to supplement the candidate bounding boxes in the first target cluster.

[0106] It should be noted that after step 401, the first target cluster, which includes N candidate labels, is determined as the second target cluster.

[0107] Step 402: Extract features from the image to be labeled to obtain the feature image of the image to be labeled.

[0108] Specifically, a feature extraction sub-model can be set up to obtain the feature image of the image to be labeled. This feature extraction sub-model is part of the calibration model. This feature extraction sub-model is trained by a third neural network, which is a part of the second neural network. The third neural network is a neural network capable of processing images.

[0109] Step 403: Based on the feature image, the second target cluster, and the ROI algorithm, obtain N ROI feature maps. The N ROI feature maps correspond one-to-one with the N candidate labels in the second target cluster, and the size of each ROI feature map is equal to the size of the corresponding candidate label.

[0110] Specifically, the process of obtaining the ROI feature map corresponding to a candidate label in the second target cluster using the ROI algorithm includes: extracting a feature map of the same size and corresponding position to the candidate label from the feature map based on the coordinates of the candidate label, and determining the extracted feature map as the ROI feature map corresponding to the candidate label.

[0111] It should be noted that the principle of obtaining the ROI feature map corresponding to each candidate label in the second target cluster is the same, and the principle of obtaining the ROI feature map corresponding to one candidate label has already been explained above. Therefore, the principle of obtaining the ROI feature map corresponding to other candidate labels will not be explained here.

[0112] Step 404: Correct and fuse the N ROI feature maps to obtain the target label corresponding to the first target cluster. By correcting and fusing the N ROI feature maps, the target label corresponding to the first target cluster becomes more accurate.

[0113] Step 404 will be explained in the following two ways:

[0114] The first type, such as Figure 5 As shown, it includes the following steps:

[0115] Step 501: Based on each of the N ROI feature maps and combined with an error calculation sub-model, calculate the error of each ROI feature map. The error calculation sub-model is part of the calibration model. The error sub-model is trained by a fourth neural network, which is part of the second neural network. The fourth neural network (i.e., the error calculation sub-model) can be a neural network with N inputs, N outputs, and multiple convolutional layers / multiple fully connected layers; alternatively, it can be a neural network with 1 input, 1 output, and multiple convolutional layers / multiple fully connected layers. The specific structure of the fourth neural network is not limited here. The error of each ROI feature map is used to indicate the error of the candidate label corresponding to each ROI feature map.

[0116] For example, if the error calculation sub-model includes N inputs and N outputs, then N ROI feature maps can be input into the N inputs of the error calculation sub-model. The error calculation sub-model processes the N ROI feature maps and outputs the error of each ROI feature map through the N outputs.

[0117] For example, if the error calculation sub-model has one input and one output, then one of the N ROI feature maps can be input into the error calculation sub-model, and the error calculation sub-model can output the error of that ROI feature map. This process is repeated to obtain the errors of the other ROI feature maps.

[0118] It should be noted that the error of each ROI feature map can be the coordinate error of the candidate label identifier corresponding to each ROI feature map. For example, if the candidate label identifier is a candidate bounding box, then the error of the ROI feature map can be the coordinate error of the candidate bounding box corresponding to that ROI feature map.

[0119] It should be noted that the above method of calculating the error of each ROI feature map by combining an error calculation sub-model is merely exemplary and is not intended to limit this application.

[0120] Step 502: Correct the corresponding candidate labels based on the error of each ROI feature map to obtain N corrected candidate labels.

[0121] For example, if the error of the ROI feature map is the coordinate error of the corresponding candidate label, then the process of correcting the candidate label corresponding to an ROI feature map includes: calculating the sum of the coordinates of the candidate label corresponding to the ROI feature map and the coordinate error of the ROI feature map to obtain the corrected coordinates; determining the coordinates of the candidate label corresponding to the ROI feature map as the corrected coordinates to obtain the corrected candidate label. Repeating the above process yields the corrected candidate label for each ROI feature map.

[0122] Step 503: Average the N corrected candidate labels to obtain the target label corresponding to the first target cluster. For example, calculate the average coordinates of the N corrected candidate labels, and determine the target label corresponding to the first target cluster based on the obtained average coordinates.

[0123] As can be seen from steps 501 to 503, by calculating the error of each ROI feature map, correcting the corresponding candidate labels based on the error of each ROI feature map, averaging each corrected candidate label, and determining the target label based on the average result, the accuracy of the target label is improved, enabling the target label to more accurately mark the position of the object in the image to be labeled, thus making the target label more consistent with the actual label of the object.

[0124] The second type, such as Figure 6 As shown, it includes the following steps:

[0125] Step 601: Based on each ROI feature map in the N ROI feature maps and combined with a probability calculation sub-model, obtain the multi-channel heat map corresponding to the first target cluster. The probability calculation sub-model is part of the calibration model, and the multi-channel heat map is used to indicate the probability that the boundary point on each candidate label in the first target cluster is the corresponding boundary point on the target label of the first target cluster.

[0126] Boundary points on candidate labels include points that can determine the boundary of the candidate label, and there are multiple boundary points on candidate labels. Boundary points on target labels include points that can determine the boundary of the target label, and there are also multiple boundary points on target labels. The specific number of boundary points is determined based on the number of points that can uniquely represent the boundary. It should be noted that the number of boundary points on each candidate label is the same as the number of boundary points on the target label, and there is a one-to-one correspondence between multiple boundary points on any candidate label and multiple boundary points on the target label; that is, the corresponding boundary points on the candidate label and the target label are all located at the same position on the boundary of the label.

[0127] For example, if both candidate and target labels are rectangular detection boxes, and since a rectangular detection box can be uniquely represented by the two endpoints of any diagonal (either the diagonal in the upper-left-lower-right direction or the diagonal in the lower-left-upper-right direction), then both candidate and target labels have two boundary points. These two boundary points are the first and second endpoints along a diagonal. For example, if the first endpoint is the upper-left endpoint, then the second endpoint is the lower-right endpoint; if the first endpoint is the lower-left endpoint, then the second endpoint is the upper-right endpoint. Thus, the multi-channel heatmap is a two-channel heatmap, where the two channels indicate the probability that the first endpoint of each candidate label is the first endpoint of the target label, and the probability that the second endpoint of each candidate label is the second endpoint of the target label.

[0128] The probability calculation sub-model can be trained by a fifth neural network, which is part of the second neural network. The fifth neural network (probability calculation sub-model) may include, for example, multiple convolutional layers and fully connected layers, etc., and this application does not make any special limitations on it.

[0129] Step 602: Determine the target boundary point based on the probability that the boundary point on each candidate label in the multi-channel heatmap is the corresponding boundary point on the target label of the first target cluster. Specifically, in the multi-channel heatmap, the boundary point with the highest probability of being the corresponding boundary point on the target label is determined as the target boundary point.

[0130] For example, taking the example in step 601, in the two-channel heatmap, the first endpoint with the highest probability of being the first endpoint on the target label is determined as the target boundary point, and the second endpoint with the highest probability of being the second endpoint on the target label is determined as the target boundary point, thus obtaining two target boundary points.

[0131] Step 603: Determine the target label corresponding to the first target cluster based on the target boundary points. Specifically, the boundary of the target label corresponding to the first target cluster can be determined based on the target boundary points, and the label formed by the boundary can be determined as the target label corresponding to the first target cluster. For example, taking the example in step 602, the boundary of the target label is determined based on two target boundary points, and the label formed by the boundary can be determined as the target label corresponding to the first target cluster.

[0132] As shown in steps 601-603, the multi-channel heatmap corresponding to the first target cluster is determined by the probability calculation sub-model. Then, the boundary point is determined as the target boundary point based on the probability that the boundary point on each candidate label in the multi-channel heatmap is the corresponding boundary point on the target label corresponding to the first target cluster. That is, the boundary point most likely to be the target label is selected from the boundary points of the multi-channel heatmap, and the target label is determined based on the selected boundary point. In other words, the target label is determined by the boundary point most likely to be the target label, which improves the accuracy of the target label and enables the target label to more accurately mark the position of the object in the image to be labeled, so that the target label is more consistent with the actual label of the object.

[0133] It should be noted that, due to the passage Figure 4 and Figure 5 The method of composition or Figure 4 and Figure 6 The method described above explains the principle of obtaining the target label identifier corresponding to the first target cluster. The first target cluster is any one of at least one target cluster, and the principle for obtaining the target label identifier corresponding to each target cluster is the same. Therefore, the above is repeated. Figure 4 and Figure 5 The method of composition or Figure 4 and Figure 6 The method of construction can obtain the target label identifier corresponding to each target cluster in at least one target cluster.

[0134] The structure and training method of the calibration model (second neural network) will be explained below.

[0135] For example, if according to Figure 4 and Figure 5 The way the target is constructed determines the labeling identifier, such as... Figure 7 As shown, the correction model includes: a first component, a feature extraction sub-model (third neural network), an ROI algorithm component, and an error calculation sub-model (fourth neural network). The first component is used to complete the N candidate labels. The training methods for this correction model include the following two approaches:

[0136] The first approach involves training the third and fourth neural networks separately using the corresponding training samples to obtain a feature extraction sub-model and an error calculation sub-model. Then, the first component, the feature extraction sub-model, the ROI algorithm component, and the error calculation sub-model are combined into a whole, and this whole is trained using the corresponding training samples to obtain the calibration model.

[0137] The second approach involves combining the first component, the third neural network, the ROI algorithm component, and the fourth neural network into a single unit. Then, the unit is trained based on the corresponding training samples to obtain a corrected model.

[0138] For example, if based on Figure 4 and Figure 6 The way the target is constructed determines the labeling identifier, such as... Figure 8 As shown, the calibration model includes: a first component, a feature extraction sub-model (third neural network), an ROI algorithm component, and a probability calculation sub-model (fifth neural network). The first component is used to complete the N candidate labels. The calibration model can be trained in two ways:

[0139] The first approach involves training the third and fifth neural networks separately using the corresponding training samples to obtain a feature extraction sub-model and a probability calculation sub-model. Then, the first component, the feature extraction sub-model, the ROI algorithm component, and the probability calculation sub-model are combined into a whole, and this whole is trained using the corresponding training samples to obtain a corrected model.

[0140] The second approach involves combining the first component, the third neural network, the ROI algorithm component, and the fifth neural network into a single unit. Then, the unit is trained based on the corresponding training samples to obtain a corrected model.

[0141] It should be noted that the structure and training method of the above-mentioned correction model are exemplary and are not intended to limit this application.

[0142] It should be noted that in other embodiments of this application, when based on Figure 4 and Figure 5 The way of composition or Figure 4 and Figure 6 In determining the target label identifiers corresponding to the first target cluster using the construction method, it is not necessary to supplement the candidate label identifiers in the first target cluster to N; that is, based on the original candidate label identifiers in the first target cluster and using the method of... Figure 4 and Figure 5 The method of composition or Figure 4 and Figure 6 The same principle applies to determining the target label identifier corresponding to the first target cluster.

[0143] To further improve the annotation coverage of objects in the image to be annotated by N target annotation models, thereby further improving the accuracy and recall of object annotation, this application also provides a method for determining N target annotation models. Specifically, the method involves determining N target annotation models from T candidate annotation models, where T ≥ N. Each candidate annotation model is trained by a first neural network. The first neural network can be, for example, a convolutional neural network, a fast deep convolutional neural network, or other neural networks applicable to image processing; no special limitation is made here.

[0144] Each candidate annotation model is trained by a first neural network, meaning each candidate annotation model corresponds to a first neural network. The type of the first neural network corresponding to each candidate annotation model can be the same, completely different, or not entirely the same; this application does not impose any special limitations on this. The structure of the first neural network corresponding to each candidate annotation model is different. The structure of the first neural network refers to the number of neural network layers, the number of neurons in each neural network layer, the connection method between adjacent neural network layers, and the loss function in the first neural network. Since the training principle of the candidate annotation model is the same as that of the target annotation model described above, the training principle of the candidate annotation model will not be elaborated here.

[0145] The following two methods will be used as examples to illustrate the process of determining N target annotation models from T candidate annotation models:

[0146] Method 1: First, each test sample in the test set is input into each candidate annotation model, allowing each model to annotate the objects in each test sample. This results in a label for each candidate model on each test sample, indicating that the label identifies the object detected by the model. Then, the recall rate of each candidate model is calculated based on its label on each test sample. The principle for calculating the recall rate of a candidate model is as follows: Based on the label on each test sample in the test set, determine the number of objects correctly labeled by the candidate model in the test set; count the total number of objects in the test set; and determine the recall rate of the candidate model as the ratio of the number of correctly labeled objects to the total number of objects in the test set. Repeating this process calculates the recall rate for each candidate model. Since for a single candidate annotation model, a higher recall rate indicates that the candidate annotation model can correctly detect more objects in the image, and the higher the accuracy, the candidate annotation model with a higher recall rate has a higher coverage of objects in the image. Based on this, the T candidate annotation models are sorted in descending order of recall rate, and the top N candidate annotation models are determined as N target annotation models.

[0147] Obviously, in Method 1, since the object annotation coverage, accuracy and recall of a single target annotation model are high, the combination of the annotation results of N target annotation models to annotate the same image further improves the annotation coverage of the objects in the image to be annotated by the N target annotation models, thereby further improving the annotation accuracy and recall.

[0148] Method 2, such as Figure 9 As shown, it includes the following steps:

[0149] Step 901: Divide the labeled area into S area intervals based on the labeled area of ​​the object in each test sample in the test set.

[0150] Specifically, the labeled area of ​​each object in each test sample of the statistical test set is counted. The largest and smallest labeled areas are determined from the labeled areas of the objects. Then, using the smallest labeled area as the lower limit and the largest labeled area as the upper limit, the labeled area is divided into S area intervals. For example, if the smallest labeled area is 0.5 and the largest labeled area is 3, then the labeled area is divided in units of 0.5, resulting in 5 area intervals: [0.5, 1), [1, 1.5), [1.5, 2), [2, 2.5), [2.5, 3].

[0151] Step 902: Based on each candidate annotation model, annotate the objects in each test sample to obtain the annotation identifier of each candidate annotation model on each test sample. Since the principle of the candidate annotation model annotating the objects in the test samples here is the same as the annotation principle of Method 1, it will not be described again here.

[0152] Step 903: Based on the area of ​​the annotation mark of each candidate annotation model on each test sample and S area intervals, calculate the recall rate of each candidate annotation model in each area interval.

[0153] Specifically, the principle for calculating the recall of a candidate annotation model within a given area interval is as follows: Based on the annotation identifiers of the candidate annotation model on each test sample in the test set, determine the number of objects in the test set that are correctly annotated by the candidate annotation model and belong to that area interval; count the total number of objects in the test set that belong to that area interval; and determine the recall of the candidate annotation model within that area interval as the ratio of the number of objects correctly annotated by the candidate annotation model in the test set to the total number of objects in the test set that belong to that area interval. Repeating the above process allows for the calculation of the recall of each candidate annotation model within each area interval.

[0154] Step 904: Based on the recall rate of each candidate annotation model in each area interval, determine N target annotation models from the T candidate annotation models.

[0155] Specifically, firstly, based on the recall rate of each candidate labeled model in each area interval, the candidate labeled models are sorted in descending order of recall rate for each area interval. That is, for an area interval, T candidate labeled models are sorted in descending order of recall rate for each candidate labeled model in that area interval to obtain the sorting result of T candidate labeled models for that area interval. Repeating the above process, the sorting result of T candidate labeled models for other area intervals can be obtained.

[0156] Then, for each area interval, the top K candidate labeled models are selected. That is, from the ranking results of an area interval, the top K candidate labeled models are selected to obtain the K candidate labeled models corresponding to that area interval. This process is repeated to obtain the K candidate labeled models for each other area interval. The value of K for each area interval can be the same or different. It should be noted that, to improve the coverage, accuracy, and recall of the annotations, the value of K is the same for each area interval.

[0157] Finally, N target annotation models are determined based on the first K candidate annotation models corresponding to each area interval. That is, if the first K candidate annotation models corresponding to each area interval do not overlap, then the first K candidate annotation models corresponding to each area interval are determined as target annotation models to obtain N target annotation models, where the value of N is determined based on the values ​​of K and S.

[0158] If the top K candidate annotation models corresponding to two area intervals overlap, and the number of overlapping candidate annotation models is only one, then the overlapping candidate annotation models can be removed from the top K candidate annotation models corresponding to an area interval. Additionally, the K candidate annotation models ranked at position K+1 in the sorting results corresponding to that area interval can be used to complete the K candidate annotation models for that area interval. Finally, all K candidate annotation models corresponding to each area interval are determined as target annotation models, resulting in N target annotation models.

[0159] If the top K candidate labeling models of two area intervals overlap, and there are multiple overlapping candidate labeling models, we can determine the overlapping candidate labeling models to be removed in each area interval. Then, we remove the corresponding overlapping candidate labeling models from the top K candidate labeling models of each area interval. Next, we select the candidate labeling models that are ranked higher and have not been selected in the ranking results of each area interval to complete the K candidate labeling models of the corresponding area interval. Finally, we determine the K candidate labeling models of each area interval as target labeling models to obtain N target labeling models.

[0160] In other embodiments of this application, the value of N can be determined first, and then the number of candidate labeled models corresponding to each area interval can be determined based on the value of N. That is, the number of candidate labeled models corresponding to each area interval is N / S, where N is an integer multiple of S. Based on this, for each area interval, the top N / S candidate labeled models need to be selected, and then N target labeled models are determined based on the top N / S candidate labeled models. Since the principle of determining N target labeled models based on the top N / S candidate labeled models is the same as above, it will not be repeated here.

[0161] As can be seen from the above, since a higher recall rate of a candidate annotation model within an area interval indicates a higher accuracy and coverage of the annotation of objects within that area interval, N target annotation models are formed by combining the candidate annotation models with the highest recall rates within each area interval. This improves the annotation accuracy, coverage, and recall of the N target annotation models for objects within multiple area intervals, thereby improving the annotation coverage, accuracy, and recall of the N target annotation models for objects in the image to be annotated.

[0162] Figure 10 This is a schematic diagram of the structure of a labeling device provided in an embodiment of this application, as shown below. Figure 10 As shown, the device 1000 includes: a labeling module 1001, a clustering module 1002, and a first determination module 1003, wherein:

[0163] The annotation module 1001 is used to annotate the objects in the image to be annotated according to each of the N target annotation models, and obtain N sets of annotation results; the clustering module 1002 is used to cluster the candidate annotation identifiers in the N sets of annotation results to obtain at least one target cluster; the first determination module 1003 is used to determine at least one target annotation identifier according to the at least one target cluster, wherein the target annotation identifier is an annotation identifier for the objects in the image to be annotated.

[0164] In one possible implementation, the apparatus 1000 further includes: a second determining module, configured to determine N target labeled models from T candidate labeled models, wherein T ≥ N.

[0165] In one possible implementation, the second determining module includes: a partitioning unit, configured to divide the labeled area into S area intervals based on the labeled area of ​​the object in each test sample in the test set; a labeling unit, configured to label the object in each test sample based on each candidate labeling model, to obtain the label identifier of each candidate labeling model on each test sample; a calculation unit, configured to calculate the recall rate of each candidate labeling model in each area interval based on the area of ​​the label identifier of each candidate labeling model on each test sample and the S area intervals; and a first determining unit, configured to determine N target labeling models from the T candidate labeling models based on the recall rate of each candidate labeling model in each area interval.

[0166] In one possible implementation, the first determining unit is specifically configured to: sort the candidate labeled models in descending order of recall rate for each area interval based on the recall rate of each candidate labeled model in each area interval; select the top K candidate labeled models for each area interval; and determine N target labeled models based on the top K candidate labeled models corresponding to each area interval.

[0167] In one possible implementation, the clustering module 1002 includes: a clustering unit, configured to cluster candidate label identifiers in the N sets of annotation results to obtain at least one candidate cluster; wherein each candidate cluster includes at least one candidate label identifier, and the candidate label identifiers included in a candidate cluster correspond to different target annotation models; and a second determining unit, configured to determine the at least one target cluster in the at least one candidate cluster based on the number of candidate label identifiers in each candidate cluster and in combination with a preset number.

[0168] In one possible implementation, the clustering unit is specifically used to cluster candidate label identifiers in the N groups of labeling results based on a density clustering algorithm.

[0169] In one possible implementation, the clustering unit is specifically used to calculate the IOU between any two candidate labels in the N sets of annotation results; and to cluster the candidate labels in the N sets of annotation results based on the IOU between the two candidate labels, the IOU threshold, and the neighborhood density threshold, and in conjunction with the density clustering algorithm.

[0170] In one possible implementation, the clustering unit includes: a first calculation subunit, configured to calculate the Interchange of Units (IOU) between any two candidate label identifiers in the N sets of annotation results; a first determination subunit, configured to determine a first candidate label identifier corresponding to each candidate label identifier in the N sets of annotation results based on the IOU between the two candidate label identifiers; wherein, the second candidate label identifier is any one of the candidate label identifiers in the N sets of annotation results, the first candidate label identifier corresponding to the second candidate label identifier is a candidate label identifier in the N sets of annotation results whose IOU with the second candidate label identifier is less than an IOU threshold, and the target labeling model corresponding to the first candidate label identifier is different from the target labeling model corresponding to the second candidate label identifier, and the target labeling models corresponding to the first candidate label identifiers are different; a second determination subunit, configured to determine the score of each candidate label identifier based on the number of first candidate label identifiers corresponding to each candidate label identifier; and a third determination subunit, configured to determine a third [missing information] in the N sets of annotation results. The system comprises: a candidate label identifier, wherein the third candidate label identifier is the candidate label identifier with the highest score among the N groups of labeling results; a fourth determining subunit, configured to determine a fourth candidate label identifier among the N groups of labeling results, wherein the fourth candidate label identifier is a candidate label identifier among the N groups of labeling results whose IOU with the third candidate label identifier is less than an IOU threshold and whose score is greater than a preset score, and the target labeling model corresponding to the fourth candidate label identifier is different from that of the third candidate label identifier, and the target labeling models corresponding to the fourth candidate label identifiers are different; a set subunit, configured to set the third candidate label identifier and the fourth candidate label identifier to obtain a candidate cluster; a removal subunit, configured to update the N groups of labeling results by removing the third candidate label identifier and the fourth candidate label identifier from the N groups of labeling results; and a fifth determining subunit, configured to re-determine the third candidate label identifier, the fourth candidate label identifier, and the candidate cluster in the updated N groups of labeling results, until the updated N groups of labeling results are empty.

[0171] In one possible implementation, the first determining module 1003 is specifically configured to determine the at least one target label based on the at least one target cluster and in conjunction with a correction model, wherein the correction model is trained by a second neural network.

[0172] In one possible implementation, for each target cluster, the first determining module 1003 includes: a supplementing unit, configured to supplement the candidate labels in the first target cluster to N, to obtain a second target cluster, wherein the first target cluster is any one of the at least one target cluster; an extraction unit, configured to extract features from the image to be labeled, to obtain a feature image of the image to be labeled; an acquisition unit, configured to acquire N ROI feature maps based on the feature image, the second target cluster, and in combination with an ROI algorithm, wherein the N ROI feature maps correspond one-to-one with the N candidate labels in the second target cluster; and a correction unit, configured to correct and fuse the N ROI feature maps to obtain the target label corresponding to the first target cluster.

[0173] In one possible implementation, the correction unit includes: a second calculation subunit, configured to calculate the error of each of the N ROI feature maps based on each ROI feature map and in conjunction with an error calculation submodel, wherein the error calculation submodel is part of the correction model, and the error of each ROI feature map is used to indicate the error of the candidate label corresponding to each ROI feature map; a correction subunit, configured to correct the corresponding candidate label based on the error of each ROI feature map to obtain N corrected candidate label labels; and an averaging subunit, configured to average the N corrected candidate label labels to obtain the target label label corresponding to the first target cluster.

[0174] In one possible implementation, the correction unit includes: a sixth determining subunit, configured to determine a multi-channel heatmap corresponding to the first target cluster based on each ROI feature map in the N ROI feature maps and in conjunction with a probability calculation submodel; wherein the probability calculation submodel is part of the correction model, and the multi-channel heatmap is used to indicate the probability that the boundary point on each candidate label in the first target cluster is the corresponding boundary point on the target label corresponding to the first target cluster; a seventh determining subunit, configured to determine a target boundary point based on the probability that the boundary point on each candidate label in the multi-channel heatmap is the corresponding boundary point on the target label corresponding to the first target cluster; and an eighth determining subunit, configured to determine the target label corresponding to the first target cluster based on the target boundary point.

[0175] The apparatus described above in this application can be used to perform... Figures 1-9 The technical solutions of any of the method embodiments are similar in principle and technical effect, and will not be described again here.

[0176] This application also provides an annotation device, which may be, for example, a computer, a server, etc., and this application does not specifically limit it. The annotation device may include a communication module, one or more memory units, and one or more processors, wherein: the communication module is used to communicate with other devices, the one or more memory units are used to store one or more computer programs, and the one or more processors are used to execute the one or more computer programs, causing the annotation device to perform actions such as… Figures 1-9 The technical solution of any of the method embodiments.

[0177] This application also provides a computer-readable storage medium including a computer program, which, when executed on a computer, causes the computer to perform... Figures 1-9 The technical solution of any of the method embodiments.

[0178] This application also provides a computer program, which, when executed by a computer, is used to execute the technical solution of any of the method embodiments 1 to 9.

[0179] This application also provides a chip, including a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the technical solution of any of the method embodiments 1 to 9.

[0180] Furthermore, the chip may also include a memory and a communication interface. The communication interface may be an input / output interface, pins, or input / output circuits, etc.

[0181] In implementation, each step of the above method embodiments can be completed by integrated logic circuits in the processor hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this application can be directly implemented by a hardware encoding processor, or implemented by a combination of hardware and software modules in the encoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0182] The memory mentioned in the above embodiments can be volatile memory or non-volatile memory, or may include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0183] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0184] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0185] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0186] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0187] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0188] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0189] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A labeling method, characterized in that, include: Annotate the objects in the image to be annotated according to each of the N target annotation models, and obtain N sets of annotation results; Cluster the candidate labels in the N sets of labeling results to obtain at least one target cluster; At least one target annotation identifier is determined based on the at least one target cluster, wherein the target annotation identifier is an annotation identifier for an object in the image to be annotated; The method further includes: determining N target annotation models from T candidate annotation models, where T≥N; The step of determining N target annotation models from T candidate annotation models includes: Based on the labeled area of ​​the object in each test sample in the test set, the labeled area is divided into S area intervals; Based on each candidate annotation model, the objects in each test sample are annotated to obtain the annotation identifier of each candidate annotation model on each test sample; Based on the area of ​​the labeled identifier of each candidate labeling model on each test sample and the S area intervals, the recall rate of each candidate labeling model in each area interval is calculated respectively. Based on the recall rate of each candidate annotation model in each area interval, N target annotation models are determined from the T candidate annotation models.

2. The method according to claim 1, characterized in that, The step of determining N target annotation models from the T candidate annotation models based on the recall rate of each candidate annotation model in each area interval includes: Based on the recall rate of each candidate annotation model in each area interval, the candidate annotation models are sorted in descending order of recall rate for each area interval; For each area interval, select the top K candidate annotation models; N target annotation models are determined based on the top K candidate annotation models corresponding to each area interval.

3. The method according to claim 1 or 2, characterized in that, The step of clustering the candidate annotations in the N sets of annotation results to obtain at least one target cluster includes: Cluster the candidate annotations in the N sets of annotation results to obtain at least one candidate cluster; Wherein, each of the at least one candidate cluster includes at least one candidate label identifier, and the candidate label identifiers included in a candidate cluster correspond to different target labeling models; Based on the number of candidate labels in each candidate cluster and in combination with a preset number, at least one target cluster is determined from the at least one candidate cluster.

4. The method according to claim 3, characterized in that, The clustering of candidate annotations in the N groups of annotation results includes: The candidate labels in the N groups of labeling results are clustered based on the density clustering algorithm.

5. The method according to claim 4, characterized in that, The clustering of candidate annotations in the N groups of annotation results based on the density clustering algorithm includes: Calculate the Interchange of Union (IOU) between any two candidate annotations in the N sets of annotation results; Based on the IOU, IOU threshold, and neighborhood density threshold between any two candidate labels, and in conjunction with the density clustering algorithm, the candidate labels in the N sets of labeling results are clustered.

6. The method according to claim 3, characterized in that, The step of clustering the candidate annotations in the N sets of annotation results to obtain at least one candidate cluster includes: Calculate the Interchange of Union (IOU) between any two candidate annotations in the N sets of annotation results; Based on the IOU between any two candidate label identifiers, determine the first candidate label identifier corresponding to each candidate label identifier in the N sets of labeling results; Wherein, the second candidate label identifier is any one of the candidate label identifiers in the N groups of labeling results, the first candidate label identifier corresponding to the second candidate label identifier is a candidate label identifier in the N groups of labeling results whose IOU with the second candidate label identifier is less than the IOU threshold, and the target labeling model corresponding to the first candidate label identifier is different from the target labeling model corresponding to the second candidate label identifier, and the target labeling models corresponding to the first candidate label identifiers are different; The score of each candidate label is determined based on the number of first candidate labels corresponding to each candidate label; A third candidate label is determined from the N sets of labeling results, wherein the third candidate label is the candidate label with the highest score in the N sets of labeling results; A fourth candidate label is determined from the N sets of annotation results. The fourth candidate label is a candidate label in the N sets of annotation results whose IOU with the third candidate label is less than the IOU threshold and whose score is greater than a preset score. The target annotation model corresponding to the fourth candidate label is different from that of the third candidate label. The target annotation models corresponding to the fourth candidate labels are different from each other. The third candidate label and the fourth candidate label are combined to obtain a candidate cluster; The N sets of annotation results are updated by removing the third candidate annotation and the fourth candidate annotation from the N sets of annotation results; In the updated N sets of annotation results, the third candidate annotation identifier, the fourth candidate annotation identifier, and the candidate cluster are determined again until the updated N sets of annotation results are empty.

7. The method according to any one of claims 1 to 6, characterized in that, The step of determining at least one target label based on the at least one target cluster includes: The at least one target label is determined based on the at least one target cluster and in conjunction with a calibration model.

8. The method according to claim 7, characterized in that, For each target cluster, determining the at least one target label based on the at least one target cluster and in conjunction with a calibration model includes: The number of candidate labels in the first target cluster is increased to N to obtain the second target cluster, wherein the first target cluster is any one of the at least one target cluster; Feature extraction is performed on the image to be labeled to obtain the feature image of the image to be labeled; Based on the feature image, the second target cluster, and the ROI algorithm, N ROI feature maps are obtained, wherein the N ROI feature maps correspond one-to-one with the N candidate labels in the second target cluster; The N ROI feature maps are corrected and fused to obtain the target label corresponding to the first target cluster.

9. The method according to claim 8, characterized in that, The correction and fusion of the N ROI feature maps includes: Based on each of the N ROI feature maps and combined with an error calculation sub-model, the error of each ROI feature map is calculated, wherein the error calculation sub-model is part of the correction model, and the error of each ROI feature map is used to indicate the error of the candidate label identifier corresponding to each ROI feature map; The corresponding candidate label is corrected based on the error of each ROI feature map to obtain N corrected candidate label. The target label corresponding to the first target cluster is obtained by averaging the N corrected candidate labels.

10. The method according to claim 8, characterized in that, The correction and fusion of the N ROI feature maps includes: Based on each ROI feature map in the N ROI feature maps and combined with a probability calculation sub-model, determine the multi-channel heat map corresponding to the first target cluster. The probability calculation sub-model is part of the correction model, and the multi-channel heatmap is used to indicate the probability that the boundary point on each candidate label in the first target cluster is the corresponding boundary point on the target label corresponding to the first target cluster. The target boundary point is determined based on the probability that the boundary point on each candidate label in the multi-channel heat map is the corresponding boundary point on the target label corresponding to the first target cluster. The target label identifier corresponding to the first target cluster is determined based on the target boundary points.

11. A labeling device, characterized in that, include: The annotation module is used to annotate the objects in the image to be annotated according to each of the N target annotation models, and obtain N sets of annotation results; The clustering module is used to cluster the candidate annotations in the N groups of annotation results to obtain at least one target cluster. The first determining module is configured to determine at least one target annotation identifier based on the at least one target cluster, wherein the target annotation identifier is an annotation identifier for an object in the image to be annotated; The device further includes: The second determining module is used to determine N target annotation models from T candidate annotation models, where T≥N; The second determining module includes: A partitioning unit is used to divide the labeled area into S area intervals based on the labeled area of ​​the object in each test sample in the test set; An annotation unit is used to annotate the objects in each test sample based on each candidate annotation model, so as to obtain the annotation identifier of each candidate annotation model on each test sample; The calculation unit is used to calculate the recall rate of each candidate annotation model in each area interval based on the area of ​​the annotation mark of each candidate annotation model on each test sample and the S area intervals. The first determining unit is configured to determine N target annotation models from the T candidate annotation models based on the recall rate of each candidate annotation model in each area interval.

12. The apparatus according to claim 11, characterized in that, The first determining unit is specifically configured to: sort the candidate labeling models in descending order of recall rate for each area interval based on the recall rate of each candidate labeling model in each area interval; select the top K candidate labeling models for each area interval; and determine N target labeling models based on the top K candidate labeling models corresponding to each area interval.

13. The apparatus according to claim 11 or 12, characterized in that, The clustering module includes: A clustering unit is used to cluster the candidate annotations in the N groups of annotation results to obtain at least one candidate cluster. Wherein, each of the at least one candidate cluster includes at least one candidate label identifier, and the candidate label identifiers included in a candidate cluster correspond to different target labeling models; The second determining unit is used to determine the at least one target cluster in the at least one candidate cluster based on the number of candidate labels in each candidate cluster and in combination with a preset number.

14. The apparatus according to claim 13, characterized in that, The clustering unit is specifically used to cluster candidate label identifiers in the N groups of labeling results based on the density clustering algorithm.

15. The apparatus according to claim 14, characterized in that, The clustering unit is specifically used to calculate the IOU between any two candidate labels in the N sets of annotation results; and to cluster the candidate labels in the N sets of annotation results based on the IOU between any two candidate labels, the IOU threshold, and the neighborhood density threshold, combined with the density clustering algorithm.

16. The apparatus according to claim 13, characterized in that, The clustering unit includes: The first calculation subunit is used to calculate the IOU between any two candidate annotations in the N sets of annotation results; The first determining subunit is used to determine the first candidate labeling identifier corresponding to each candidate labeling identifier in the N sets of labeling results based on the IOU between any two candidate labeling identifiers; Wherein, the second candidate label identifier is any one of the candidate label identifiers in the N groups of labeling results, the first candidate label identifier corresponding to the second candidate label identifier is a candidate label identifier in the N groups of labeling results whose IOU with the second candidate label identifier is less than the IOU threshold, and the target labeling model corresponding to the first candidate label identifier is different from the target labeling model corresponding to the second candidate label identifier, and the target labeling models corresponding to the first candidate label identifiers are different; The second determining subunit is used to determine the score of each candidate label based on the number of first candidate label corresponding to each candidate label; The third determining subunit is used to determine a third candidate label identifier in the N groups of labeling results, wherein the third candidate label identifier is the candidate label identifier with the largest score in the N groups of labeling results; The fourth determining subunit is used to determine the fourth candidate label in the N sets of labeling results, wherein the fourth candidate label is a candidate label in the N sets of labeling results whose IOU with the third candidate label is less than the IOU threshold and whose score is greater than a preset score, the target labeling model corresponding to the fourth candidate label is different from that of the third candidate label, and the target labeling models corresponding to the fourth candidate labels are different. A set subunit is used to combine the third candidate label and the fourth candidate label to obtain a candidate cluster; The removal subunit is used to update the N sets of annotation results by removing the third candidate annotation identifier and the fourth candidate annotation identifier from the N sets of annotation results; The fifth determining subunit is used to re-determine the third candidate label, the fourth candidate label, and the candidate cluster in the updated N sets of labeling results, until the updated N sets of labeling results are empty.

17. The apparatus according to any one of claims 11 to 16, characterized in that, The first determining module is specifically used to determine the at least one target label based on the at least one target cluster and in conjunction with a correction model.

18. The apparatus according to claim 17, characterized in that, For each of the target clusters, the first determining module includes: The supplementary unit is used to supplement the candidate labels in the first target cluster to N, so as to obtain a second target cluster, wherein the first target cluster is any one of the at least one target cluster; An extraction unit is used to extract features from the image to be labeled in order to obtain a feature image of the image to be labeled. The acquisition unit is used to acquire N ROI feature maps based on the feature image, the second target cluster, and the ROI algorithm, wherein the N ROI feature maps correspond one-to-one with the N candidate label identifiers in the second target cluster; The correction unit is used to correct and fuse the N ROI feature maps to obtain the target label corresponding to the first target cluster.

19. The apparatus according to claim 18, characterized in that, The correction unit includes: The second calculation subunit is used to calculate the error of each of the N ROI feature maps based on each ROI feature map and in combination with an error calculation submodel, wherein the error calculation submodel is part of the correction model, and the error of each ROI feature map is used to indicate the error of the candidate label corresponding to each ROI feature map. The correction subunit is used to correct the corresponding candidate label based on the error of each ROI feature map to obtain N corrected candidate label. The averaging subunit is used to average the N corrected candidate labels to obtain the target label corresponding to the first target cluster.

20. The apparatus according to claim 18, characterized in that, The correction unit includes: The sixth determining subunit is used to determine the multi-channel heatmap corresponding to the first target cluster based on each ROI feature map in the N ROI feature maps and in combination with a probability calculation sub-model. The probability calculation sub-model is part of the correction model, and the multi-channel heatmap is used to indicate the probability that the boundary point on each candidate label in the first target cluster is the corresponding boundary point on the target label corresponding to the first target cluster. The seventh determining subunit is used to determine the target boundary point based on the probability that the boundary point on each candidate label in the multi-channel heat map is the corresponding boundary point on the target label corresponding to the first target cluster. The eighth determining subunit is used to determine the target label identifier corresponding to the first target cluster based on the target boundary points.

21. A computer-readable storage medium comprising a computer program that, when executed on a computer, causes the computer to perform the method of any one of claims 1 to 10.

22. A computer program product comprising computer program code, which, when run on a computer, causes the computer to perform the method of any one of claims 1 to 10.

23. A chip comprising a processor and a memory, the memory for storing a computer program, and the processor for calling and running the computer program stored in the memory to perform the method of any one of claims 1 to 10.

Images