Image labeling model training method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide an image labeling model training method and device, electronic equipment and storage medium. The image labeling model training method comprises: obtaining a labeling data set comprising at least two images to be labeled; determining the similarity between the images to be labeled in the labeling data set; labeling each image to be labeled in the labeling data set by a first labeling model to be trained to obtain a first labeling result of each image to be labeled; determining a sample subset according to the similarity between the images to be labeled and the first labeling result of each image to be labeled; and training the first labeling model by using the images to be labeled included in the sample subset and the corresponding artificial labeling results. The present application can solve the problem of poor algorithm labeling effect when the algorithm assists artificial labeling, fully utilize the labeled data in the data set, and improve the algorithm labeling effect.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of image processing, and in particular to an image labeling model training method and device, an electronic device, and a storage medium. BACKGROUND

[0002] With the development of technology, autonomous driving is becoming more and more popular. In order to enable autonomous driving algorithms to handle more and more complex scenarios, a large amount of real image data is needed to support, and therefore the demand for image labeling is increasing. Mass and high-quality, fine-grained image labeling can greatly improve the safety and practicality of autonomous driving.

[0003] Currently, images are labeled using an algorithm-assisted manual labeling method. An image is labeled by a labeling model, and then the labeling result is manually corrected.

[0004] However, there are various complex scenarios in the autonomous driving scenario, which puts higher requirements on the number and quality of samples in the labeling model. Moreover, there is a deviation between the model training dataset and the dataset to be labeled, which leads to poor algorithm labeling effect. In addition, the labeled data in the dataset is not fully utilized in the labeling process. SUMMARY

[0005] Therefore, embodiments of the present application provide an image labeling model training method and device, an electronic device, and a storage medium to at least partially solve the above problems.

[0006] According to a first aspect of embodiments of the present application, an image labeling model training method is provided. The method comprises: obtaining a to-be-labeled dataset comprising at least two to-be-labeled images; determining the similarity between the to-be-labeled images in the to-be-labeled dataset; labeling each to-be-labeled image in the to-be-labeled dataset by a first labeling model to be trained to obtain a first labeling result of each to-be-labeled image, wherein the first labeling model is used to label a target object in an image; determining a sample subset according to the similarity between the to-be-labeled images and the first labeling result of each to-be-labeled image, wherein the sample subset comprises at least two to-be-labeled images, the similarity between each to-be-labeled image in the sample subset is less than a similarity threshold, and the accuracy of the first labeling result of each to-be-labeled image in the sample subset is less than an accuracy threshold; obtaining a manual labeling result corresponding to each to-be-labeled image in the sample subset; and training the first labeling model by using each to-be-labeled image included in the sample subset and the corresponding manual labeling result.

[0007] According to a second aspect of the embodiments of the present application, an image labeling device is provided, the device comprising: an acquisition module configured to acquire a labeling data set comprising at least two images to be labeled and to acquire a manual labeling result corresponding to each image to be labeled in a sample subset; a labeling module configured to label each image to be labeled in the labeling data set by a first labeling model to be trained to obtain a first labeling result of each image to be labeled; an image processing module configured to determine a similarity between the images to be labeled in the labeling data set and to determine a sample subset according to the similarity between the images to be labeled and the first labeling result of each image to be labeled; and a training module configured to train the labeling model by using each image to be labeled included in the sample subset and the corresponding manual labeling result.

[0008] According to a third aspect of the embodiments of the present application, an electronic device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, the processor, the memory, and the communication interface being in communication with each other through the communication bus; the memory is configured to store at least one executable instruction, and the executable instruction causes the processor to perform operations corresponding to the method of the first aspect.

[0009] According to a fourth aspect of the embodiments of the present application, a computer storage medium is provided, and the computer storage medium stores a computer program, and the computer program is executed by a processor to implement the method of the first aspect.

[0010] According to a fifth aspect of the embodiments of the present application, a computer program product is provided, and when the computer program is executed by a processor, the method of the first aspect is implemented.

[0011] According to the labeling method provided in the above scheme, the similarity between the images to be labeled is determined, the images to be labeled are labeled by the first labeling model to obtain the first labeling result, the sample subset is selected according to the similarity between the images to be labeled and the first labeling result, the manual labeling result of the sample subset is acquired, and the first labeling model is trained by using each image to be labeled in the sample subset and the corresponding manual labeling result. It can be seen that by determining the similarity between the images to be labeled and acquiring the first labeling result of the images to be labeled, a sample subset with low similarity and low labeling accuracy can be selected, and then the manual labeling result of the sample subset is acquired, so that a representative and well-labeled sample subset can be obtained, and then the labeling model is trained by using the representative and well-labeled sample subset, so that the labeling model is more targeted, and the labeling result of the labeling model is more accurate, thereby improving the effect of algorithm labeling in the labeling process. BRIEF DESCRIPTION OF DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only represent some of the embodiments of the present application, and other drawings can also be obtained by those skilled in the art based on these drawings.

[0013] Figure 1 The flow chart of the image annotation model training method provided by an embodiment of the present application;

[0014] Figure 2 The flow chart of the target detection task annotation model training method provided by an embodiment of the present application;

[0015] Figure 3 The schematic diagram of the image annotation model training device provided by an embodiment of the present application;

[0016] Figure 4 The structural schematic diagram of the electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to make the personnel in the art better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments in the present application should belong to the scope of protection of the present application.

[0018] The terms used in the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "an" and "the" used in the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein means and includes any or all possible combinations of one or more associated listed items.

[0019] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information, without departing from the scope of the present application. Depending on the context, the word "if" as used herein can be interpreted as "when" or "upon determining" or "in response to determining".

[0020] As Figure 1 shown, Figure 1is a flowchart of an image labeling model training method provided by an embodiment of the present application, and the method comprises the following steps 101 to 106:

[0021] Step 101, obtaining a labeling data set comprising at least two to-be-labeled images.

[0022] The whole to-be-labeled data set is determined, and the data set comprises at least two to-be-labeled images. The to-be-labeled images can be of any type, such as landscape pictures, portrait pictures, road video frames, etc.

[0023] Step 102, determining the similarity between the to-be-labeled images in the to-be-labeled data set.

[0024] The similarity between the to-be-labeled images in the to-be-labeled data set is determined. The more similar two images are, the higher the similarity measure is, and vice versa.

[0025] Step 103, labeling each to-be-labeled image in the to-be-labeled data set by a first labeling model to be trained to obtain a first labeling result of each to-be-labeled image, wherein the first labeling model is used to label target objects in images.

[0026] The to-be-labeled images are labeled by the first labeling model to be trained to obtain the first labeling result corresponding to each to-be-labeled image. The first labeling result is different according to different labeling tasks, for example, if the labeling task is to identify the type of objects in the image, the first labeling result is used to indicate the type of objects in the to-be-labeled image; if the labeling task is to label all vehicles in the image, the first labeling result is used to indicate the positions of all vehicles in the to-be-labeled image, etc.

[0027] It should be understood that the corresponding first labeling model to be trained is different according to different labeling tasks. The first labeling model to be trained is determined according to a specific visual perception labeling task.

[0028] It should also be understood that the first labeling model to be trained is a labeling model pre-trained on a general data set or a self-owned data set, which has the ability to label, rather than a completely untrained blank model.

[0029] Step 104, determining a sample subset according to the similarity between the to-be-labeled images and the first labeling result of each to-be-labeled image, wherein the sample subset comprises at least two to-be-labeled images, the similarity between the to-be-labeled images in the sample subset is less than a similarity threshold, and the accuracy of the first labeling result of each to-be-labeled image in the sample subset is less than an accuracy threshold.

[0030] The similarity threshold between images and the accuracy threshold of the labeling result are set in advance, and the to-be-labeled images with a similarity less than the similarity threshold and an accuracy less than the accuracy threshold are determined as the sample subset, so that the sample subset with representativeness and lower labeling accuracy is selected, and the sample subset includes at least two images.

[0031] It should be understood that, since the first labeling model to be trained is a labeling model pre-trained by using a general data set or a self-owned data set, the labeling result output by the first labeling model for a more complex to-be-labeled image cannot be completely accurate.

[0032] It should also be understood that, since the similarity between the images in the sample subset is less than the preset similarity threshold, the similarity between the images in the sample subset is low, and therefore the sample subset is representative.

[0033] In step 105, the artificial labeling result corresponding to each to-be-labeled image in the sample subset is obtained.

[0034] The sample subset with representativeness and lower labeling accuracy is output to the artificial end, the artificial corrects the labeling result according to the labeling task, and the correct labeling result is input into the labeling system.

[0035] In step 106, the first labeling model is trained by using each to-be-labeled image in the sample subset and the corresponding artificial labeling result.

[0036] The first labeling model to be trained is trained by using the sample subset corrected by the artificial, and since the labeling result is corrected by the artificial, the sample subset is a sample subset with representativeness and high labeling accuracy.

[0037] In the embodiment of the present application, the similarity between to-be-labeled images is determined, the first labeling result is obtained by labeling the to-be-labeled images by using the first labeling model, the sample subset is selected according to the similarity between the to-be-labeled images and the first labeling result, the artificial labeling result of the sample subset is obtained, and the first labeling model is trained by using each to-be-labeled image in the sample subset and the corresponding artificial labeling result. It can be seen that, by determining the similarity between the to-be-labeled images and obtaining the first labeling result of the to-be-labeled images, the sample subset with low similarity and low labeling accuracy can be selected, and then the artificial labeling result of the sample subset is obtained, so that the sample subset with representativeness and high labeling accuracy can be obtained, and then the labeling model is trained by using the sample subset with representativeness and high labeling accuracy, so that the labeling model is more targeted, and the labeling result of the labeling model is more accurate, thereby improving the effect of algorithm labeling in the labeling process.

[0038] In a possible implementation, when determining the similarity between the to-be-labeled images in the to-be-labeled data set, each to-be-labeled image included in the to-be-labeled data set can be encoded by using a pre-trained encoding model to obtain a feature vector of each to-be-labeled image, and then the distance between the feature vectors of different to-be-labeled images in the to-be-labeled data set is calculated to serve as the similarity between the corresponding two to-be-labeled images.

[0039] The to-be-labeled images are feature-encoded, the to-be-labeled images are converted into 256-dimensional vectors by using the encoding model, the distance between the feature vectors in the to-be-labeled data set is calculated, the closer the distance, the higher the similarity between the two images, and the farther the distance, the lower the similarity between the two images.

[0040] It should be understood that the encoding model is a model trained by self-supervised learning, and the self-supervised learning can have various forms, for example, SimSiam, SimCLR, MoCo, and the like, which are not limited by the embodiments of the present application.

[0041] It should also be understood that when feature encoding is performed, various sampling methods such as farthest point sampling and inverse density sampling can be used for sampling, which are not limited by the embodiments of the present application.

[0042] In the embodiments of the present application, the to-be-labeled images are feature-encoded by using the encoding model to obtain the feature vectors of the to-be-labeled images, and then the distance between the feature vectors corresponding to different to-be-labeled images is calculated to obtain the similarity between different to-be-labeled images.

[0043] In a possible implementation, when determining the sample subset according to the similarity between the to-be-labeled images and the first labeling result of each to-be-labeled image, the to-be-labeled images corresponding to the similarity greater than the similarity threshold in the to-be-labeled data set can be subjected to a deduplication process to obtain a candidate data set, the accuracy of the first labeling result of each to-be-labeled image in the candidate data set is then calculated, and then at least two to-be-labeled images corresponding to the accuracy less than the accuracy threshold are extracted from the candidate data set, and the set of the extracted to-be-labeled images is determined as the sample subset.

[0044] The to-be-labeled images are deduplicated according to the similarity between different to-be-labeled images, one image in the image pair with a similarity higher than a similarity threshold is deleted, the similarity threshold can be an arbitrary value, which is not limited in the present application, for example, the similarity threshold is 8, if the distance between the feature vectors corresponding to two to-be-labeled images is less than 8, then any one of the images is deleted, until the distance between the feature vectors corresponding to all images is greater than 8, and the set of images is taken as a candidate data set, the accuracy of the first labeling result of the to-be-labeled images in the candidate data set is calculated, and the image with an accuracy lower than an accuracy threshold is selected as a sample subset.

[0045] It should be understood that, since the first labeling model to be trained needs to be trained to have good labeling effect, representative and inaccurately labeled images need to be selected as the sample subset, and the representative images can be filtered out by the deduplication operation on the to-be-labeled images in the to-be-labeled data set.

[0046] In the embodiment of the present application, the to-be-labeled images are deduplicated to obtain a candidate data set, and then the accuracy of the to-be-labeled images in the candidate data set is calculated, and the to-be-labeled images with accuracy lower than the accuracy threshold are taken as the sample subset, so that a representative and poorly labeled sample subset can be selected.

[0047] In a possible implementation, when the accuracy of the first labeling result of each to-be-labeled image in the candidate data set is calculated respectively, the probability distribution map of each to-be-labeled image in the candidate data set can be determined respectively, wherein the probability distribution map is used to indicate the probability that each pixel point in the corresponding to-be-labeled image is contained in the image of the target object, and then the candidate box superimposition map of each to-be-labeled image in the candidate data set is determined respectively, wherein the candidate box superimposition map is formed by at least two candidate boxes superimposed, and the candidate box is used to indicate the region where the candidate object in the to-be-labeled image is located, and then for each to-be-labeled image in the candidate data set, the intersection over union of the probability distribution map and the candidate box superimposition map corresponding to the to-be-labeled image is calculated, and the accuracy of the first labeling result of the to-be-labeled image is determined according to the intersection over union.

[0048] The first labeling model outputs the first labeling result, the probability distribution map and the candidate box superimposition map in the labeling process, the probability distribution map and the candidate box superimposition map of each to-be-labeled image are determined, and the intersection over union is calculated as the accuracy of the first labeling result of each to-be-labeled image.

[0049] In the embodiment of the present application, the probability distribution map and the candidate box superimposition map of each to-be-labeled image are determined, and the intersection over union of the two maps is calculated as the accuracy of the first labeling result of each to-be-labeled image, so that the sample subset with low labeling accuracy can be selected from the candidate data set according to the accuracy of the first labeling result.

[0050] In a possible implementation, when at least two to-be-labeled images corresponding to the accuracy less than the accuracy threshold are extracted from the candidate data set, and the set of the extracted to-be-labeled images is determined as the sample subset, 2N to-be-labeled images corresponding to the accuracy less than the accuracy threshold can be randomly extracted from the candidate data set, wherein N is a positive integer, then the 2N to-be-labeled images are sorted in the order of corresponding accuracy from small to large, and then the set of the first N to-be-labeled images in the sorted 2N to-be-labeled images is determined as the sample subset.

[0051] The sampling algorithm is used to extract 2N images with accuracy less than the accuracy threshold from the sample subset, sort the accuracy corresponding to the extracted 2N images from small to large, and take the first N images with low accuracy as the sample subset, where N is a positive integer.

[0052] It should be understood that the accuracy corresponding to the image to be labeled can also be sorted from large to small. If this sorting method is adopted, the last N images with low accuracy are taken as the sample subset.

[0053] In the embodiment of the present application, 2N images to be labeled are extracted from the candidate data set, and the accuracy corresponding to the images to be labeled is sorted, and N images with low accuracy are selected as the sample subset. Since the candidate data set is a de-duplicated labeled data set, it has a certain representativeness, so the images to be labeled in the sample subset are representative and have low accuracy.

[0054] In a possible implementation, when the candidate box overlay image of each image to be labeled in the candidate data set is determined respectively, for each image to be labeled in the candidate data set, the probability that each candidate box in the image to be labeled includes a candidate object image is determined, and then at least two candidate boxes corresponding to a probability greater than a preset probability threshold are overlaid to obtain the candidate box overlay image.

[0055] When the candidate box overlay image output by the first labeling model is determined, there are a large number of candidate boxes in each image to be labeled, but the candidate boxes may not all enclose the target object, and may enclose some other objects, for example: the labeling task is to identify vehicles on the road, and in the labeling process, the labeling model may enclose pedestrians and other objects on the road while enclosing vehicles on the road, therefore, the probability that the candidate object image is included in the candidate box is calculated, and then the candidate boxes with a probability greater than the probability threshold are overlaid to form the candidate box overlay image.

[0056] It should be understood that the candidate boxes in the candidate box overlay image are not necessarily selected according to the probability threshold. All candidate boxes can be sorted from large to small according to the probability that the object image is included, and the first part of the candidate boxes in the sorting are overlaid to form the candidate box overlay image, for example: there are ten thousand candidate boxes, the probability that the target object is included in each of the ten thousand candidate boxes is calculated, then the probabilities are sorted from large to small, and the first one thousand candidate boxes are overlaid to form the candidate box overlay image.

[0057] In the embodiment of the present application, the probability that the object image is included in the candidate box is calculated, and the candidate boxes with a probability higher than a preset probability threshold are overlaid to form the candidate box overlay image, so that the accuracy of the first labeling result of the image to be labeled is calculated by the probability distribution graph and the candidate box overlay image, and the result is more accurate.

[0058] In one possible implementation, each image to be annotated in the sample subset is annotated separately using a second annotation model to obtain a second annotation result for each image to be annotated, wherein the second annotation model is used to annotate the target object in the image, and then the second annotation result of each image to be annotated in the sample subset is displayed to assist users in manually annotating the images to be annotated in the sample subset.

[0059] The second annotation model is used to perform a second annotation on the selected sample subset to obtain a second annotation result, thereby reducing the workload of manual correction and improving the efficiency of annotation. The second annotation model can be the first annotation model or other annotation models.

[0060] In an embodiment of the present application, before obtaining the manual annotation results, the sample subset is annotated using the second annotation model, thereby assisting humans in manually correcting the annotation results of the images to be annotated in the sample subset, reducing manual workload and improving the overall annotation efficiency.

[0061] In order to better understand the image annotation model training method disclosed in this application, the image annotation model training method provided in the embodiment of this application is described in detail below using the target detection task as an example.

[0062] In related technologies, image data is mainly annotated by algorithm-assisted manual labeling. Images are annotated through labeling models, and then the labeling results are manually corrected. However, there is often a deviation between the model training data set and the data set to be annotated, resulting in poor algorithm labeling effect.

[0063] To address the above issues, this application provides an image annotation model training method. The solution provided in this application can be used for the above-mentioned object detection task. Of course, it is understood that the solution provided in this application can also be used for various tasks such as 3D object detection, lane line detection, semantic segmentation, etc., which will not be detailed here.

[0064] like Figure 2 As shown, Figure 2 20 is a flowchart of a method for training a target detection task labeling model provided by an embodiment of the present application, the method comprising steps 201 to 208:

[0065] Step 201: Obtain a dataset of street scenes to be annotated that are perceived by the vehicle.

[0066] Select the dataset to be labeled that has been preprocessed by cleaning and screening.

[0067] Step 202: annotate each image to be annotated in the dataset using a first annotation model to obtain a first annotation result for each image to be annotated.

[0068] The image to be labeled is labeled using a first labeling model to be trained.

[0069] Step 203: Feature encode the images to be annotated in the annotated dataset using the encoding model learned through self-supervision, and calculate the similarity between the images.

[0070] The images to be labeled are compressed into 256-dimensional feature vectors through a self-supervised learning encoding model, and the similarity between the images to be labeled is calculated pairwise.

[0071] Step 204: Determine a sample subset based on the similarity between the images to be labeled and the first labeling result of each image to be labeled.

[0072] The substitute-annotated images with low similarity and inaccurate first annotation results are selected as the sample subset to make the sample subset representative.

[0073] Step 205: Label the sample subset using a second labeling model to obtain a second labeling result.

[0074] The sample subset is annotated using the second annotation model to reduce the workload of manual correction.

[0075] Step 206: Output the to-be-annotated image contained in the sample subset and the first annotation result and the second annotation result corresponding to the to-be-annotated image to the manual end, and obtain the manual annotation result.

[0076] The annotation results of the images to be annotated in the sample subset are manually corrected to obtain a well-annotated sample subset.

[0077] Step 207: Train the first annotation model using the images to be annotated included in the sample subset and the corresponding manual annotation results.

[0078] The first annotation model is trained on a subset of well-annotated data.

[0079] Step 208: Determine whether the dataset to be labeled is completely labeled. If yes, end the current process. Otherwise, execute step 209.

[0080] Step 209 : Use the trained first annotation model to annotate the unannotated images in the dataset to be annotated, obtain a first annotation result, and execute step 203 .

[0081] Re-label the remaining unlabeled data samples based on the iterative first labeling model.

[0082] like Figure 3 As shown, Figure 3 : is a schematic diagram of an image annotation model training device provided in an embodiment of the present application, the device comprising:

[0083] The acquisition module 301 is configured to acquire a labeling data set including at least two to-be-labeled images and acquire a manual labeling result corresponding to each to-be-labeled image in a sample subset;

[0084] The labeling module 302 is configured to label each to-be-labeled image in the labeling data set by using the first labeling model to be trained, to obtain a first labeling result of each to-be-labeled image.

[0085] The image processing module 303 is configured to determine a similarity between to-be-labeled images in the labeling data set and determine the sample subset according to the similarity between the to-be-labeled images and the first labeling result of each to-be-labeled image.

[0086] The training module 304 is configured to train the labeling model by using each to-be-labeled image included in the sample subset and the corresponding manual labeling result.

[0087] In a possible implementation, each to-be-labeled image included in the labeling data set is encoded by using a pre-trained encoding model to obtain a feature vector of each to-be-labeled image; and a distance between feature vectors of different to-be-labeled images in the labeling data set is calculated as a similarity between the corresponding two to-be-labeled images.

[0088] In a possible implementation, to-be-labeled images corresponding to a similarity greater than a similarity threshold in the labeling data set are processed to remove duplicates to obtain a candidate data set; an accuracy of the first labeling result of each to-be-labeled image in the candidate data set is calculated; and at least two to-be-labeled images corresponding to an accuracy less than an accuracy threshold are extracted from the candidate data set, and a set of the extracted to-be-labeled images is determined as the sample subset.

[0089] In a possible implementation, a probability distribution map of each to-be-labeled image in the candidate data set is determined, where the probability distribution map is used to indicate a probability that each pixel point in the corresponding to-be-labeled image is included in an image of a target object; a candidate box superimposition map of each to-be-labeled image in the candidate data set is determined, where the candidate box superimposition map is formed by superimposition of at least two candidate boxes, and the candidate box is used to indicate a region where a candidate object is located in the to-be-labeled image; for each to-be-labeled image in the candidate data set, an intersection over union of the probability distribution map and the candidate box superimposition map corresponding to the to-be-labeled image is calculated, and the accuracy of the first labeling result of the to-be-labeled image is determined according to the intersection over union.

[0090] In a possible implementation, 2N to-be-labeled images corresponding to the accuracy less than the accuracy threshold are randomly sampled from the candidate dataset, where N is a positive integer; the 2N to-be-labeled images are sorted in descending order of the corresponding accuracy; and a set of the first N to-be-labeled images in the sorted 2N to-be-labeled images is determined as the sample subset.

[0091] In a possible implementation, for each to-be-labeled image in the candidate dataset, a probability that each candidate box in the to-be-labeled image includes a candidate object image is determined; and at least two candidate boxes corresponding to the probability greater than a preset probability threshold are superimposed to obtain a candidate box superimposition image.

[0092] In a possible implementation, each to-be-labeled image in the sample subset is labeled by the second labeling model respectively to obtain a second labeling result of each to-be-labeled image, where the second labeling model is used to label a target object in an image; and the second labeling result of each to-be-labeled image in the sample subset is displayed to assist a user in manually labeling the to-be-labeled images in the sample subset.

[0093] It should be noted that the information interaction and execution process between the modules in the image labeling apparatus described above are based on the same concept as the simulation model calibration method embodiments described above, and the specific content can be referred to the description in the image labeling model training method embodiments described above, which will not be repeated here.

[0094] Referring to Figure 4 , a structural schematic diagram of an electronic device according to an embodiment of the present application is shown, and the specific implementation of the electronic device is not limited in the embodiments of the present application.

[0095] As Figure 4 shown, the electronic device can include a processor 402, a communications interface 404, a memory 406, and a communications bus 408.

[0096] Among them:

[0097] The processor 402, the communications interface 404, and the memory 406 complete the communication among each other through the communications bus 408.

[0098] The communications interface 404 is configured to communicate with other electronic devices or servers.

[0099] The processor 402 is configured to execute the program 410, and specifically can execute the related steps in the image labeling model training method embodiments described above.

[0100] In particular, the program 410 can include program code comprising computer operation instructions.

[0101] The processor 402 can be a central processing unit CPU, or a graphics processing unit GPU, or an application specific integrated circuit ASIC, or one or more integrated circuits configured to implement one or more embodiments of the present application. The one or more processors included in the smart device can be the same type of processor, such as one or more CPUs; one or more GPUs; or different types of processors, such as one or more CPUs and one or more GPUs and one or more ASICs.

[0102] The memory 406 is configured to store the program 410. The memory 406 can include a high-speed RAM memory, and can also include a non-volatile memory, such as at least one disk memory.

[0103] The program 410 can be specifically configured to cause the processor 402 to perform the image labeling model training method in any of the preceding embodiments.

[0104] The specific implementation of each step in the program 410 can refer to the corresponding description in the corresponding step and unit of any of the preceding image labeling model training method embodiments, which will not be described here. Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the preceding method embodiments, which will not be described here.

[0105] In the embodiments of the present application, the similarity between the to-be-labeled images is determined, the first labeling result is obtained by labeling the to-be-labeled images through the first labeling model, the sample subset is selected according to the similarity between the to-be-labeled images and the first labeling result, the manual labeling result of the sample subset is obtained, and the first labeling model is trained through each to-be-labeled image in the sample subset and the corresponding manual labeling result. It can be seen that by determining the similarity between the to-be-labeled images and obtaining the first labeling result of the to-be-labeled images, a sample subset with low similarity and low labeling accuracy can be selected, and then the manual labeling result of the sample subset is obtained, so that a representative and well-labeled sample subset can be obtained, and then the labeling model is trained through the representative and well-labeled sample subset, so that the labeling model is more targeted, and the labeling result of the labeling model is more accurate, thereby improving the effect of algorithm labeling in the labeling process.

[0106] The embodiment of the present application further provides a computer program product comprising computer instructions for instructing a computing device to perform operations corresponding to any of the above-mentioned method embodiments.

[0107] It should be noted that, according to the needs of implementation, each component / step described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps, to achieve the purpose of the embodiments of the present application.

[0108] The above-mentioned method according to the embodiments of the present application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium such as a CD ROM, a RAM, a floppy disk, a hard disk or an optical disk, or be downloaded from a network and stored in a local recording medium, so that the method described herein can be processed by such software on a recording medium using a general computer, a special processor or programmable or special hardware such as an ASIC or an FPGA. It can be understood that the computer, the processor, the microprocessor controller or the programmable hardware include storage components (for example, RAM, ROM, flash memory, etc.) that can store or receive software or computer code, when the software or computer code is accessed and executed by the computer, the processor or the hardware, the image labeling model training method described herein is implemented. In addition, when a general computer accesses the code for implementing the image labeling model training method shown herein, the execution of the code will convert the general computer into a special computer for executing the image labeling model training method shown herein.

[0109] Those of ordinary skill in the art can realize that the units and method steps of the examples described in combination with the embodiments disclosed herein can be realized in electronic hardware, or in a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of the present application.

[0110] The above embodiments are only used to illustrate the embodiments of the present application, and not to limit the embodiments of the present application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application, therefore all equivalent technical solutions also belong to the scope of the embodiments of the present application, the patent protection scope of the embodiments of the present application should be defined by the claims.

Claims

1. An image labeling model training method, characterized in that, The method comprises: obtaining a to-be-labeled data set comprising at least two to-be-labeled images; determining the similarity between the to-be-labeled images in the to-be-labeled data set; labeling each to-be-labeled image in the to-be-labeled data set by a first to-be-trained labeling model to obtain a first labeling result of each to-be-labeled image, wherein the first labeling model is used to label target objects in images; determining a sample subset according to the similarity between the to-be-labeled images and the first labeling result of each to-be-labeled image, wherein the sample subset comprises at least two to-be-labeled images, the similarity between each to-be-labeled image in the sample subset is less than a similarity threshold, and the accuracy of the first labeling result of each to-be-labeled image in the sample subset is less than an accuracy threshold; obtaining a manual labeling result corresponding to each to-be-labeled image in the sample subset; training the first labeling model by using each to-be-labeled image included in the sample subset and the corresponding manual labeling result; calculating the accuracy of the first labeling result of each to-be-labeled image, comprising: determining a probability distribution map of each to-be-labeled image, wherein the probability distribution map is used to indicate the probability that each pixel point in the corresponding to-be-labeled image is included in the image of a target object; and determining a candidate box superimposition map of each to-be-labeled image, wherein the candidate box superimposition map is formed by at least two candidate boxes, the candidate box is used to indicate the region where a candidate object is located in the to-be-labeled image, and the first labeling model outputs the first labeling result, the probability distribution map and the candidate box superimposition map during the labeling process; for each to-be-labeled image, calculating the intersection over union of the probability distribution map and the candidate box superimposition map corresponding to the to-be-labeled image, and determining the accuracy of the first labeling result of the to-be-labeled image according to the intersection over union.

2. The method of claim 1, wherein, The determination of the similarity between the to-be-labeled images in the to-be-labeled data set comprises: encoding each to-be-labeled image included in the to-be-labeled data set by a pre-trained encoding model to obtain a feature vector of each to-be-labeled image; calculating the distance between the feature vectors of different to-be-labeled images in the to-be-labeled data set as the similarity between the corresponding two to-be-labeled images.

3. The method of claim 1, wherein, The determination of the sample subset according to the similarity between the to-be-labeled images and the first labeling result of each to-be-labeled image comprises: performing a deduplication process on the to-be-labeled images in the to-be-labeled data set corresponding to a similarity greater than a similarity threshold to obtain a candidate data set; calculating the accuracy of the first labeling result of each to-be-labeled image in the candidate data set; extracting at least two to-be-labeled images corresponding to an accuracy less than an accuracy threshold from the candidate data set, and determining the set of the extracted to-be-labeled images as the sample subset.

4. The method of claim 3, wherein, The extracting at least two of the to-be-labeled images corresponding to the accuracy less than the accuracy threshold from the candidate dataset, and determining a set of the extracted to-be-labeled images as the sample subset, comprises: randomly extracting 2N to-be-labeled images corresponding to the accuracy less than the accuracy threshold from the candidate dataset, wherein N is a positive integer; sorting the 2N to-be-labeled images in the order from small to large according to the corresponding accuracy; determining a set of the first N to-be-labeled images in the sorted 2N to-be-labeled images as the sample subset.

5. The method of claim 1, wherein, The separately determining a candidate box superposition map of each to-be-labeled image comprises: determining, for each to-be-labeled image, a probability that each candidate box in the to-be-labeled image includes a candidate object image; superimposing at least two candidate boxes corresponding to the probability greater than a preset probability threshold to obtain the candidate box superposition map.

6. The method of claim 1, wherein, The method further comprises: labeling each to-be-labeled image in the sample subset by a second labeling model respectively to obtain a second labeling result of each to-be-labeled image, wherein the second labeling model is used to label the target object in the image; displaying the second labeling result of each to-be-labeled image in the sample subset to assist a user in manually labeling the to-be-labeled images in the sample subset.

7. An image annotation model training apparatus, characterized by comprising: The apparatus comprises: an acquisition module configured to acquire a to-be-labeled dataset comprising at least two to-be-labeled images; a labeling module configured to label each to-be-labeled image in the to-be-labeled dataset by a first labeling model to be trained to obtain a first labeling result of each to-be-labeled image; an image processing module configured to determine a similarity between the to-be-labeled images in the to-be-labeled dataset and determine a sample subset according to the similarity between the to-be-labeled images and the first labeling result of each to-be-labeled image; the acquisition module is further configured to acquire a manual labeling result corresponding to each to-be-labeled image in the sample subset; a training module configured to train the labeling model by using each to-be-labeled image included in the sample subset and the corresponding manual labeling result; respectively calculating the accuracy of the first labeling result of each to-be-labeled image comprises: respectively determining a probability distribution map of each to-be-labeled image, wherein the probability distribution map is used to indicate a probability that each pixel point in the corresponding to-be-labeled image is included in an image of a target object; and respectively determining a candidate box superposition map of each to-be-labeled image, wherein the candidate box superposition map is formed by superimposing at least two candidate boxes, the candidate box is used to indicate a region where a candidate object is located in the to-be-labeled image, and the first labeling model outputs the first labeling result, the probability distribution map, and the candidate box superposition map in a labeling process; for each to-be-labeled image, calculating an intersection over union of the probability distribution map and the candidate box superposition map corresponding to the to-be-labeled image, and determining the accuracy of the first labeling result of the to-be-labeled image according to the intersection over union.

8. An electronic device, comprising: comprises: a processor, a memory, a communication interface, and a communication bus, which complete communication with each other through the communication bus; The memory is used to store at least one executable instruction, and the executable instruction makes the processor execute the corresponding operation of the method in any one of claims 1-6.

9. A computer storage medium, characterized in that A computer program is stored thereon, and the program is executed by a processor to implement the method in any one of claims 1-6.

10. A computer program product, characterised in that, When the computer program is executed by the processor, the image labeling model training method in any one of claims 1-6 is implemented.

Images