Image data labeling method and device and storage medium

Abstract

The invention relates to the technical field of image processing, in particular to an image data labeling method and device and a storage medium. Comprising the steps of obtaining a to-be-labeled image, a basic classification result and an advanced classification result; the basic classification result is generated by the basic classification model according to the to-be-labeled image, and the advanced classification result is generated by the advanced classification model according to the to-be-labeled image; when both the basic classification result and the advanced classification result meet a preset condition, determining any one of the basic category and the advanced category as a target category; the preset conditions include that the basic score is greater than or equal to a score threshold, the basic category is the same as the advanced category, and the difference between the basic score and the advanced score is smaller than a difference threshold; and marking the to-be-marked image according to the target category to obtain a marked image. The embodiment of the invention is used for solving the problem of relatively low efficiency of a mode of manually annotating an image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image data annotation method, device and storage medium. Background Art

[0002] Image data annotation refers to marking each image in the data set so that it can be better recognized and understood by machine learning algorithms or other artificial intelligence algorithms. For example, in the scene of gesture image recognition, the deep neural network model can be trained with gesture images annotated with gesture data, and then the trained deep neural network model can be used to perform gesture recognition on the collected gesture images to obtain recognition results. In this way, in the process of training the deep neural network model, a large number of images with annotated image data are required as training samples.

[0003] Currently, the way to obtain training samples is usually to manually annotate images. However, the efficiency of manually annotating images is low and the data processing cost is high. Summary of the invention

[0004] In order to solve the above technical problems, the present application provides an image data annotation method, device and storage medium, which can improve efficiency and reduce data processing costs.

[0005] In a first aspect, the present application provides a method for labeling image data, including: obtaining an image to be labeled, a basic classification result, and an advanced classification result; the basic classification result includes a basic category and a basic score corresponding to the image to be labeled, and the advanced classification result includes an advanced category and an advanced score corresponding to the image to be labeled; the basic classification result is generated by a basic classification model according to the image to be labeled, and the advanced classification result is generated by an advanced classification model according to the image to be labeled, and the depth of the advanced classification model is greater than or equal to the depth of the basic classification model; when both the basic classification result and the advanced classification result meet preset conditions, any one of the basic category and the advanced category is determined as a target category; the preset conditions include: the basic score is greater than or equal to a score threshold, the basic category and the advanced category are the same, and the difference between the basic score and the advanced score is less than a difference threshold; label the image to be labeled according to the target category to obtain a labeled image.

[0006] In some embodiments, after obtaining the image to be annotated, the basic classification results, and the advanced classification results, the method also includes: sending an abnormal prompt when the basic classification results and the advanced classification results meet at least one of the following abnormal conditions; wherein the abnormal prompt is used to prompt the annotation of the image to be annotated; the abnormal conditions include: the basic score is less than the score threshold; the basic category and the advanced category are different; the difference between the basic score and the advanced score is greater than or equal to the difference threshold.

[0007] In some embodiments, after sending the abnormal prompt, the method also includes: obtaining final annotation data corresponding to at least one image to be annotated that prompts an abnormality; the final annotation data is the annotation data obtained after sending the abnormal prompt; using at least one image to be annotated that prompts an abnormality and the final annotation data, correcting the basic classification model and the advanced classification model.

[0008] In some embodiments, before obtaining the basic classification results, the method also includes: obtaining a training data set; the training data set includes at least one image and at least one category corresponding to the image; based on the training data set, training a first deep neural network and a second deep neural network respectively to obtain a basic classification model and an advanced classification model; wherein the depth of the second deep neural network is greater than or equal to the depth of the first deep neural network.

[0009] In some embodiments, when the depth of the second deep neural network is equal to the depth of the first deep neural network, the first deep neural network and the second deep neural network are trained respectively based on the training data set to obtain a basic classification model and an advanced classification model, including: classifying the training data set based on the size of the image to obtain a first training set and a second training set; wherein the size of the images in the first training set is smaller than the size of the images in the second training set; using the images in the first training set as the input of the first deep neural network, and the categories corresponding to the images in the first training set as the output of the first deep neural network, to train and generate a basic classification model; using the images in the second training set as the input of the second deep neural network, and the categories corresponding to the images in the second training set as the output of the second deep neural network, to train and generate an advanced classification model.

[0010] In some embodiments, when the depth of the second deep neural network is greater than the depth of the first deep neural network, the first deep neural network and the second deep neural network are trained separately based on the training data set to obtain a basic classification model and an advanced classification model, including: taking at least one image as the input of the first deep neural network, taking the category corresponding to the at least one image as the output of the first deep neural network, and training to generate the basic classification model; taking at least one image as the input of the second deep neural network, taking the category corresponding to the at least one image as the output of the second deep neural network, and training to generate the advanced classification model.

[0011] In a second aspect, the present application provides an image data annotation device, including: an acquisition module, used to acquire an image to be annotated, a basic classification result, and an advanced classification result; the basic classification result includes a basic category and a basic score corresponding to the image to be annotated, and the advanced classification result includes an advanced category and an advanced score corresponding to the image to be annotated; the basic classification result is generated by a basic classification model based on the image to be annotated, and the advanced classification result is generated by an advanced classification model based on the image to be annotated, and the depth of the advanced classification model is greater than or equal to the depth of the basic classification model; a determination module, used to determine any one of the basic category and the advanced category as a target category when both the basic classification result and the advanced classification result meet preset conditions; the preset conditions include: the basic score is greater than or equal to a score threshold, the basic category and the advanced category are the same, and the difference between the basic score and the advanced score is less than a difference threshold; a processing module, used to annotate the image to be annotated according to the target category to obtain an annotated image.

[0012] In some embodiments, the device also includes: a sending module, used to send an abnormal prompt when the basic classification result and the advanced classification result meet at least one of the following abnormal conditions; wherein the abnormal prompt is used to prompt the annotation of the image to be annotated; the abnormal conditions include: the basic score is less than the score threshold; the basic category and the advanced category are different; the difference between the basic score and the advanced score is greater than or equal to the difference threshold.

[0013] In some embodiments, the acquisition module is also used to obtain final annotation data corresponding to at least one image to be annotated indicating an abnormality; the final annotation data is the annotation data obtained after sending the abnormality prompt; the processing module is also used to use at least one image to be annotated indicating an abnormality and the final annotation data to correct the basic classification model and the advanced classification model.

[0014] In some embodiments, the device also includes a training module; an acquisition module, which is also used to acquire a training data set; the training data set includes at least one image and at least one category corresponding to the image; the training module is used to train the first deep neural network and the second deep neural network respectively based on the training data set to obtain a basic classification model and an advanced classification model; wherein the depth of the second deep neural network is greater than or equal to the depth of the first deep neural network.

[0015] In some embodiments, the training module is specifically used to classify the training data set based on the size of the image to obtain a first training set and a second training set when the depth of the second deep neural network is equal to the depth of the first deep neural network; wherein the size of the images in the first training set are smaller than the size of the images in the second training set; using the images in the first training set as the input of the first deep neural network, and the categories corresponding to the images in the first training set as the output of the first deep neural network, to train and generate a basic classification model; using the images in the second training set as the input of the second deep neural network, and the categories corresponding to the images in the second training set as the output of the second deep neural network, to train and generate an advanced classification model.

[0016] In some embodiments, the training module is specifically used to, when the depth of the second deep neural network is greater than the depth of the first deep neural network, use at least one image as the input of the first deep neural network, use the category corresponding to the at least one image as the output of the first deep neural network, and train to generate a basic classification model; use at least one image as the input of the second deep neural network, use the category corresponding to the at least one image as the output of the second deep neural network, and train to generate an advanced classification model.

[0017] In a third aspect, the present application provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the image data labeling method of the first aspect is implemented.

[0018] In a fourth aspect, the present application provides a computer-readable storage medium, comprising: a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the image data labeling method of the first aspect is implemented.

[0019] In a fifth aspect, the present application provides a computer program product, comprising: when the computer program product runs on a computer, the computer implements the image data labeling method as in the first aspect.

[0020] The technical solution provided by the present application has the following advantages compared with the prior art: first, obtain the image to be annotated, the basic classification result, and the advanced classification result, and when the basic classification result and the advanced classification result both meet the preset conditions, determine any one of the basic category and the advanced category as the target category. Afterwards, annotate the image to be annotated according to the target category to obtain the annotated image. Among them, the basic classification result includes the basic category and the basic score corresponding to the image to be annotated, and the advanced classification result includes the advanced category and the advanced score corresponding to the image to be annotated; the basic classification result is generated by the basic classification model according to the image to be annotated, and the advanced classification result is generated by the advanced classification model according to the image to be annotated; the preset conditions include: the basic score is greater than or equal to the score threshold, the basic category and the advanced category are the same, and the difference between the basic score and the advanced score is less than the difference threshold. In this way, first, when annotating the image to be annotated, the basic classification model and the advanced classification model can be used to generate the classification result corresponding to the image to be annotated, avoiding the problem of low efficiency and high data processing cost when manually annotating the image, improving the annotation efficiency and reducing the processing cost. Secondly, it is also possible to determine whether the classification results meet the preset conditions based on the basic classification results of the image to be labeled by the basic classification model and the advanced classification results of the image to be labeled by the advanced classification model. Only when the preset conditions are met will the image to be labeled according to the target category to obtain the labeled image, which further improves the accuracy of image labeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0023] Figure 1 One of the flowcharts of the image data annotation method provided in the embodiment of the present application;

[0024] Figure 2 One of the network architectures applied to the image data annotation method provided in the embodiment of the present application;

[0025] Figure 3 The second network architecture applied to the image data annotation method provided in the embodiment of the present application;

[0026] Figure 4 The second flowchart of the method for labeling image data provided in the embodiment of the present application;

[0027] Figure 5 The third flowchart of the method for labeling image data provided in the embodiment of the present application;

[0028] Figure 6 A fourth flowchart of the method for labeling image data provided in an embodiment of the present application;

[0029] Figure 7 A fifth flowchart of the method for labeling image data provided in an embodiment of the present application;

[0030] Figure 8 Schematic diagram of the sixth process of the method for labeling image data provided in the embodiment of the present application;

[0031] Fig. 9 A schematic diagram of the structure of an image data annotation device provided in an embodiment of the present application;

[0032] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0034] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only part of the embodiments of the present application, rather than all of the embodiments.

[0035] With the development of artificial intelligence, the interactive mode of controlling the operation of devices through gestures and realizing the dialogue between people and devices has entered people's lives. As an intuitive and dynamic interactive mode, gesture interaction between people and devices provides users with a more natural and convenient control method, and also provides solutions for many specific scenarios, greatly enriching the form and experience of human-computer interaction. The most important link in the gesture interaction process is the gesture recognition link. If the gesture recognition result is not accurate enough, it will greatly reduce the user experience. Among them, in the gesture recognition process, the deep neural network model can be first trained by the gesture image annotated with gesture data, and then the gesture recognition of the collected gesture image can be performed by the trained deep neural network model to obtain the recognition result. In this way, in the process of training the deep neural network model, a large number of gesture images annotated with gesture data are required as training samples.

[0036] Currently, the way to obtain training samples is usually to manually annotate images. However, the efficiency of manually annotating images is low and the data processing cost is high.

[0037] The embodiment of the present application provides a method for labeling image data, which can obtain an image to be labeled, a basic classification result, and an advanced classification result, and when both the basic classification result and the advanced classification result meet the preset conditions, any one of the basic category and the advanced category is determined as the target category. Afterwards, the image to be labeled is labeled according to the target category to obtain the labeled image. This avoids the problem of low efficiency and high data processing cost when manually labeling images, improves labeling efficiency, and reduces processing costs.

[0038] The image data annotation method provided in the embodiment of the present application can be implemented by an annotation device for image data, and the image data annotation device provided in the embodiment of the present application can be hardware or software. When the image data annotation device is hardware, it can be various electronic devices with the annotation function of running image data, including but not limited to vehicle-mounted equipment, smart vehicles, mobile phones, computers, smart TVs, wearable devices, etc. When the image data annotation device is software, it can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules, or it can be implemented as a single software or software module, which is not specifically limited here.

[0039] Figure 1 A schematic diagram of a flow chart of a method for labeling image data provided in an embodiment of the present application, such as Figure 1 As shown, the image data annotation method may include the following steps.

[0040] S11. Obtain the image to be labeled, basic classification results, and advanced classification results.

[0041] First, obtain the image to be annotated.

[0042] In some embodiments, the method of acquiring the image to be annotated may be to acquire the image to be annotated through an image acquisition device.

[0043] Afterwards, basic classification results and advanced classification results are obtained.

[0044] Among them, the basic classification results include the basic categories and basic scores corresponding to the image to be annotated, and the advanced classification results include the advanced categories and advanced scores corresponding to the image to be annotated. The basic classification results are generated by the basic classification model based on the image to be annotated, and the advanced classification results are generated by the advanced classification model based on the image to be annotated, and the depth of the advanced classification model is greater than or equal to the depth of the basic classification model.

[0045] Combine the following Figure 2 and Figure 3The network architecture used in the image data annotation method provided in the text describes how to obtain basic and advanced classification results.

[0046] In some embodiments, reference Figure 2 As shown, the basic classification model and the advanced classification model can be set separately from the image data annotation device. In this way, the basic classification result can be obtained by inputting the image to be annotated into the basic classification model to obtain the basic classification result; similarly, the advanced classification result can be obtained by inputting the image to be annotated into the advanced classification model to obtain the advanced classification result.

[0047] In some embodiments, reference Figure 3 As shown, the basic classification model and the advanced classification model can also be placed inside the image data annotation device. In this way, the basic classification result can be obtained by inputting the image to be annotated into the image data annotation device, and the image data annotation device inputs the image to be annotated into the basic classification model and the advanced classification model to obtain the basic classification result and the advanced classification result.

[0048] In some embodiments, Figure 4 As shown, before obtaining the basic classification results and the advanced classification results, the method further includes training to generate the basic classification model and the advanced classification model, and the training process is as follows:

[0049] S09. Obtain a training data set.

[0050] The training data set includes at least one image and at least one category corresponding to the image.

[0051] In some embodiments, the at least one image may be acquired by using an image acquisition device, wherein the image acquisition device is a device capable of acquiring images, such as a camera, an infrared camera, and the like.

[0052] In some embodiments, a method for obtaining the category corresponding to at least one image may be to manually label the image with a category to obtain the category corresponding to the at least one image.

[0053] S10. Based on the training data set, train the first deep neural network and the second deep neural network respectively to obtain a basic classification model and an advanced classification model.

[0054] The depth of the second deep neural network is greater than or equal to the depth of the first deep neural network. Both the first deep neural network and the second deep neural network include a softmax function, which is used to perform a softmax operation on the outputs of the first deep neural network and the second deep neural network to obtain the scores of each category corresponding to the output, and take the category with the largest score as the final output category.

[0055] In some embodiments, Figure 5 As shown, when the depth of the second deep neural network is equal to the depth of the first deep neural network, for example, the first deep neural network and the second deep neural network are both resnet18. At this time, based on the training data set, the first deep neural network and the second deep neural network are trained respectively to obtain the basic classification model and the advanced classification model, which may include the following steps:

[0056] S101. Classify a training data set based on the size of the image to obtain a first training set and a second training set.

[0057] The sizes of the images in the first training set are all smaller than the sizes of the images in the second training set.

[0058] In some embodiments, the training data set is classified based on the size of the image to obtain the first training set and the second training set by determining images that are smaller than or equal to a size threshold as training data in the first training set, and determining images that are larger than the size threshold as training data in the second training set. The size threshold is preset, for example, a default value, or a value set by relevant personnel according to actual conditions.

[0059] S102: Using the images in the first training set as inputs of the first deep neural network, using the categories corresponding to the images in the first training set as outputs of the first deep neural network, and training to generate a basic classification model.

[0060] Specifically, the process of training and generating a basic classification model can be: using the images in the first training set as the input of the first deep neural network, and the categories corresponding to the images in the first training set as the output of the first deep neural network, training the first deep neural network n times until the loss function of the first deep neural network converges, and obtaining the basic classification model.

[0061] S103: Using the images in the second training set as input of the second deep neural network, using the categories corresponding to the images in the second training set as output of the second deep neural network, and training to generate an advanced classification model.

[0062] Specifically, the process of training and generating an advanced classification model can be: using the images in the second training set as the input of the second deep neural network, the categories corresponding to the images in the second training set as the output of the second deep neural network, training the second deep neural network m times until the loss function of the second deep neural network converges, and obtaining the advanced classification model.

[0063] In some embodiments, Figure 6As shown, when the depth of the second deep neural network is greater than the depth of the first deep neural network, for example, when the first deep neural network is resnet18, the first deep neural network can be VIT (vision transformer) or resnet152. At this time, based on the training data set, the first deep neural network and the second deep neural network are trained respectively to obtain the basic classification model and the advanced classification model, which may include the following steps:

[0064] S104: Using at least one image as input of a first deep neural network, using a category corresponding to the at least one image as output of the first deep neural network, and training to generate a basic classification model.

[0065] Specifically, the process of training the basic classification model can be: taking at least one image as the input of the first deep neural network, taking the category corresponding to the at least one image as the output of the first deep neural network, training the first deep neural network i times until the loss function of the first deep neural network converges, and obtaining the basic classification model.

[0066] S105: Using at least one image as input of a second deep neural network, using a category corresponding to the at least one image as output of the second deep neural network, and training to generate an advanced classification model.

[0067] Specifically, the process of training the advanced classification model can be: taking at least one image as the input of the second deep neural network, taking the category corresponding to the at least one image as the output of the second deep neural network, training the second deep neural network j times until the loss function of the second deep neural network converges, and obtaining the advanced classification model.

[0068] S12. When both the basic classification result and the advanced classification result meet the preset conditions, any one of the basic category and the advanced category is determined as the target category.

[0069] The preset conditions include: the basic score is greater than or equal to the score threshold, the basic category and the advanced category are the same, and the difference between the basic score and the advanced score is less than the difference threshold. The score threshold and the difference threshold are both preset, for example, a default value, or a value set by relevant personnel according to actual conditions, for example, the score threshold can be 0.6.

[0070] Specifically, when the basic score is greater than or equal to the score threshold, the basic category and the advanced category are the same, and the difference between the basic score and the advanced score is less than the difference threshold, it is determined that both the basic classification result and the advanced classification result meet the preset conditions, then the output results of the basic classification model and the advanced classification model are considered to be consistent. At this time, the basic category and the advanced category are the same, and any one of the basic category and the advanced category is determined as the target category.

[0071] S13. Label the image to be labeled according to the target category to obtain a labeled image.

[0072] In the above scheme, firstly, the image to be annotated, the basic classification result, and the advanced classification result are obtained, and when both the basic classification result and the advanced classification result meet the preset conditions, any one of the basic category and the advanced category is determined as the target category. Afterwards, the image to be annotated is annotated according to the target category to obtain the annotated image. Among them, the basic classification result includes the basic category and the basic score corresponding to the image to be annotated, and the advanced classification result includes the advanced category and the advanced score corresponding to the image to be annotated; the basic classification result is generated by the basic classification model according to the image to be annotated, and the advanced classification result is generated by the advanced classification model according to the image to be annotated; the preset conditions include: the basic score is greater than or equal to the score threshold, the basic category and the advanced category are the same, and the difference between the basic score and the advanced score is less than the difference threshold. In this way, when annotating the image to be annotated, the basic classification model and the advanced classification model can be used to generate the classification result corresponding to the image to be annotated, avoiding the problem of low efficiency and high data processing cost when manually annotating the image, improving the annotation efficiency and reducing the processing cost. Secondly, it is also possible to determine whether the classification results meet the preset conditions based on the basic classification results of the image to be labeled by the basic classification model and the advanced classification results of the image to be labeled by the advanced classification model. Only when the preset conditions are met will the image to be labeled according to the target category to obtain the labeled image, which further improves the accuracy of image labeling.

[0073] In some embodiments, Figure 7 As shown, after obtaining the image to be labeled, the basic classification result, and the advanced classification result, the method further includes:

[0074] S14. When the basic classification result and the advanced classification result meet at least one of the following abnormal conditions, an abnormal prompt is sent.

[0075] The abnormal prompt is used to prompt the image to be annotated. The abnormal conditions include: the basic score is less than the score threshold; the basic category and the advanced category are different; the difference between the basic score and the advanced score is greater than or equal to the difference threshold.

[0076] Specifically, when the basic score is less than the score threshold, or the basic category and the advanced category are different, or the difference between the basic score and the advanced score is greater than or equal to the difference threshold, it is determined that the basic classification result and the advanced classification result meet at least one of the abnormal conditions, and the output results of the basic classification model and the advanced classification model are considered inconsistent. At this time, an abnormal prompt is sent. In this way, when relevant personnel see the abnormal prompt, they can manually annotate the image to be annotated with the abnormal prompt to improve the accuracy of image annotation.

[0077] In the above scheme, when it is determined that the output results of the basic classification model and the advanced classification model are inconsistent, an abnormal prompt is sent to prompt manual labeling of the image to be labeled, thereby further improving the accuracy of image labeling.

[0078] In some embodiments, Figure 8 As shown, after sending the abnormal prompt, the method also includes:

[0079] S15. Obtain final annotation data corresponding to at least one image to be annotated that indicates an abnormality.

[0080] The final annotation data is the annotation data obtained after sending the abnormal prompt, that is, the annotation data corresponding to the manually annotated image is obtained.

[0081] S16. Correct the basic classification model and the advanced classification model using at least one image to be annotated indicating an abnormality and the final annotated data.

[0082] In some embodiments, when the depth of the second deep neural network is greater than the depth of the first deep neural network, the method of using at least one image to be annotated indicating an abnormality and the final annotated data to correct the basic classification model and the advanced classification model may be: inputting the image to be annotated indicating an abnormality into the basic classification model, comparing the output of the basic classification model with the final annotated data, and adjusting the parameters of the basic classification model according to the comparison results, so that the loss function of the basic classification model converges to obtain a corrected basic classification model. Similarly, inputting the image to be annotated indicating an abnormality into the advanced classification model, comparing the output of the advanced classification model with the final annotated data, and adjusting the parameters of the advanced classification model according to the comparison results, so that the loss function of the advanced classification model converges to obtain a corrected advanced classification model.

[0083] In some embodiments, when the depth of the second deep neural network is equal to the depth of the first deep neural network, the method of using at least one abnormal image to be annotated and the final annotation data to correct the basic classification model and the advanced classification model can be: first, based on the size of the image, the abnormal image to be annotated and the final annotation data are classified, and the abnormal image to be annotated with a size greater than the size threshold and the corresponding final annotation data are used to correct the advanced classification model to obtain the corrected advanced classification model; the abnormal image to be annotated with a size less than or equal to the size threshold and the corresponding final annotation data are used to correct the basic classification model to obtain the corrected basic classification model. Among them, the specific process of correcting the basic classification model and the advanced classification model is the same as the aforementioned process of correcting the basic classification model and the advanced classification model when the depth of the second deep neural network is greater than the depth of the first deep neural network, and will not be repeated here.

[0084] In the above scheme, the basic classification model and the advanced classification model can be corrected based on the final annotation data corresponding to the image to be annotated that indicates an abnormality, thereby further improving the accuracy of the output results of the basic classification model and the advanced classification model.

[0085] The embodiment of the present application can divide the functional modules of the image data annotation device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0086] like Fig. 9 , which is a schematic diagram of the structure of an image data annotation device 200 provided in an embodiment of the present application. The image data annotation device 200 includes an acquisition module 201 , a determination module 202 , and a processing module 203 .

[0087] The acquisition module 201 is used to obtain the image to be labeled, the basic classification result, and the advanced classification result; the basic classification result includes the basic category and the basic score corresponding to the image to be labeled, and the advanced classification result includes the advanced category and the advanced score corresponding to the image to be labeled; the basic classification result is generated by the basic classification model according to the image to be labeled, and the advanced classification result is generated by the advanced classification model according to the image to be labeled, and the depth of the advanced classification model is greater than or equal to the depth of the basic classification model; the determination module 202 is used to determine any one of the basic category and the advanced category as the target category when both the basic classification result and the advanced classification result meet the preset conditions; the preset conditions include: the basic score is greater than or equal to the score threshold, the basic category and the advanced category are the same, and the difference between the basic score and the advanced score is less than the difference threshold; the processing module 203 is used to label the image to be labeled according to the target category to obtain the labeled image.

[0088] In some embodiments, the device also includes: a sending module 204, which is used to send an abnormal prompt when the basic classification result and the advanced classification result meet at least one of the following abnormal conditions; wherein the abnormal prompt is used to prompt the labeling of the image to be labeled; the abnormal conditions include: the basic score is less than the score threshold; the basic category and the advanced category are different; the difference between the basic score and the advanced score is greater than or equal to the difference threshold.

[0089] In some embodiments, the acquisition module 201 is also used to obtain the final annotation data corresponding to at least one image to be annotated indicating an abnormality; the final annotation data is the annotation data obtained after the abnormality prompt is sent; the processing module 203 is also used to use at least one image to be annotated indicating an abnormality and the final annotation data to correct the basic classification model and the advanced classification model.

[0090] In some embodiments, the device also includes a training module 205; the acquisition module 201 is also used to acquire a training data set; the training data set includes at least one image and at least one category corresponding to the image; the training module 205 is used to train the first deep neural network and the second deep neural network respectively based on the training data set to obtain a basic classification model and an advanced classification model; wherein the depth of the second deep neural network is greater than or equal to the depth of the first deep neural network.

[0091] In some embodiments, the training module 205 is specifically used to classify the training data set based on the size of the image to obtain a first training set and a second training set when the depth of the second deep neural network is equal to the depth of the first deep neural network; wherein the size of the images in the first training set are smaller than the size of the images in the second training set; using the images in the first training set as the input of the first deep neural network, and the categories corresponding to the images in the first training set as the output of the first deep neural network, to train and generate a basic classification model; using the images in the second training set as the input of the second deep neural network, and the categories corresponding to the images in the second training set as the output of the second deep neural network, to train and generate an advanced classification model.

[0092] In some embodiments, the training module 205 is specifically used to, when the depth of the second deep neural network is greater than the depth of the first deep neural network, use at least one image as the input of the first deep neural network, use the category corresponding to the at least one image as the output of the first deep neural network, and train to generate a basic classification model; use at least one image as the input of the second deep neural network, use the category corresponding to the at least one image as the output of the second deep neural network, and train to generate an advanced classification model.

[0093] The image data annotation device provided in this embodiment can execute the image data annotation method provided in the above method embodiment. Its implementation principle and technical effect are similar to those of the above method and will not be described in detail here.

[0094] Fig.10 An electronic device provided in an embodiment of the present application may include a processor 302, and the processor 302 is used to execute application code to implement the image data annotation method in the present application.

[0095] The processor 302 may be a central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0096] like Fig.10 As shown, the electronic device may further include a memory 303. The memory 303 is used to store application program codes for executing the solution of the present application, and the execution is controlled by the processor 302.

[0097] The memory 303 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 303 may exist independently and be connected to the processor 302 via the bus 304. The memory 303 may also be integrated with the processor 302.

[0098] like Fig.10 As shown, the electronic device may further include a communication interface 301, wherein the communication interface 301, the processor 302, and the memory 303 may be coupled to each other, for example, via a bus 304. The communication interface 301 is used to exchange information with other devices, for example, to support information exchange between the electronic device and other devices.

[0099] It should be pointed out that Fig.10 The device structure shown in the figure does not constitute a limitation on the electronic device, except Fig.10 In addition to the components shown, the electronic device may include more or fewer components than shown, or combine certain components, or arrange the components differently. The electronic device provided in this embodiment can execute the image data annotation method provided in the above method embodiment, and its implementation principle and technical effect are similar to the above method, which will not be described in detail here.

[0100] An embodiment of the present application provides a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, each process of the image data labeling method in the above method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0101] The computer readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0102] An embodiment of the present application provides a computer program product, which stores a computer program. When the computer program is executed by a processor, each process of the image data labeling method in the above method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0103] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media that include computer-usable program code.

[0104] In the present application, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0105] In this application, memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0106] In this application, computer-readable media includes permanent and non-permanent, removable and non-removable storage media. Storage media can be implemented by any method or technology to store information, and the information can be computer-readable instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. According to the definition in this article, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data and carrier waves.

[0107] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0108] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for labeling image data, characterized in that: include: Obtain images to be annotated, basic classification results, and advanced classification results; The basic classification result includes a basic category and a basic score corresponding to the image to be labeled, and the advanced classification result includes an advanced category and an advanced score corresponding to the image to be labeled; The basic classification result is generated by a basic classification model according to the image to be annotated, and the advanced classification result is generated by an advanced classification model according to the image to be annotated, and the depth of the advanced classification model is greater than or equal to the depth of the basic classification model; When both the basic classification result and the advanced classification result meet a preset condition, determining any one of the basic category and the advanced category as a target category; The preset conditions include: the basic score is greater than or equal to a score threshold, the basic category is the same as the advanced category, and the difference between the basic score and the advanced score is less than a difference threshold; The image to be annotated is annotated according to the target category to obtain an annotated image.

2. The image data annotation method according to claim 1, characterized in that: After obtaining the image to be annotated, the basic classification result, and the advanced classification result, the method further includes: When the basic classification result and the advanced classification result meet at least one of the following abnormal conditions, sending an abnormal prompt; Among them, the abnormal prompt is used to prompt the annotation of the image to be annotated; the abnormal conditions include: the basic score is less than the score threshold; the basic category and the advanced category are different; the difference between the basic score and the advanced score is greater than or equal to the difference threshold.

3. The image data annotation method according to claim 2, characterized in that: After sending the abnormal prompt, the method further includes: Acquire final annotation data corresponding to at least one image to be annotated indicating an abnormality; the final annotation data is the annotation data acquired after sending the abnormality prompt; The basic classification model and the advanced classification model are corrected using the at least one image to be annotated indicating an abnormality and the final annotated data.

4. The image data annotation method according to claim 1, characterized in that: Before obtaining the basic classification result, the method further includes: Acquire a training data set; the training data set includes at least one image and a category corresponding to the at least one image; Based on the training data set, a first deep neural network and a second deep neural network are trained respectively to obtain a basic classification model and an advanced classification model; wherein the depth of the second deep neural network is greater than or equal to the depth of the first deep neural network.

5. The image data annotation method according to claim 4, characterized in that: When the depth of the second deep neural network is equal to the depth of the first deep neural network, the first deep neural network and the second deep neural network are trained based on the training data set to obtain a basic classification model and an advanced classification model, including: Based on the size of the image, the training data set is classified to obtain a first training set and a second training set; wherein the size of the images in the first training set is smaller than the size of the images in the second training set; Using the images in the first training set as inputs of the first deep neural network, using the categories corresponding to the images in the first training set as outputs of the first deep neural network, and training to generate the basic classification model; The images in the second training set are used as inputs of the second deep neural network, and the categories corresponding to the images in the second training set are used as outputs of the second deep neural network, so as to train and generate the advanced classification model.

6. The image data annotation method according to claim 4, characterized in that: When the depth of the second deep neural network is greater than the depth of the first deep neural network, the first deep neural network and the second deep neural network are trained respectively based on the training data set to obtain a basic classification model and an advanced classification model, including: Using the at least one image as input of the first deep neural network, using the category corresponding to the at least one image as output of the first deep neural network, and training to generate the basic classification model; The at least one image is used as the input of the second deep neural network, and the category corresponding to the at least one image is used as the output of the second deep neural network to train and generate the high-level classification model.

7. A device for labeling image data, characterized in that: include: The acquisition module is used to obtain the image to be labeled, the basic classification results, and the advanced classification results; The basic classification result includes a basic category and a basic score corresponding to the image to be labeled, and the advanced classification result includes an advanced category and an advanced score corresponding to the image to be labeled; The basic classification result is generated by a basic classification model according to the image to be annotated, and the advanced classification result is generated by an advanced classification model according to the image to be annotated, and the depth of the advanced classification model is greater than or equal to the depth of the basic classification model; A determination module, configured to determine any one of the basic category and the advanced category as a target category when both the basic classification result and the advanced classification result meet a preset condition; The preset conditions include: the basic score is greater than or equal to a score threshold, the basic category is the same as the advanced category, and the difference between the basic score and the advanced score is less than a difference threshold; The processing module is used to label the image to be labeled according to the target category to obtain a labeled image.

8. The image data labeling device according to claim 7, characterized in that: The device also includes: A sending module, configured to send an abnormal prompt when the basic classification result and the advanced classification result meet at least one of the following abnormal conditions; Among them, the abnormal prompt is used to prompt the annotation of the image to be annotated; the abnormal conditions include: the basic score is less than the score threshold; the basic category and the advanced category are different; the difference between the basic score and the advanced score is greater than or equal to the difference threshold.

9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the image data labeling method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: include: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image data labeling method according to any one of claims 1 to 6 is implemented.

Images