Image labeling method and device, equipment and storage medium

Through the annotation model, the candidate data sets of traffic video data are labeled and screened, and the problems of low image labeling efficiency and poor accuracy in the prior art are solved, and more efficient and accurate image labeling results are achieved.

CN119048856APending Publication Date: 2024-11-29BEIJING JIAOTONG UNIV +3
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410887796.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-03
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

The prior art is inefficient and poorly accurate in the image labeling process of traffic video data, and relies on manual screening and labeling, resulting in a long time and inconsistent standards.

Method used

Through the labeling model, the candidate data sets are labeled, filtered and sorted, and the initial and candidate data sets are generated, and the filtering standards are unified to improve the labeling efficiency and accuracy.

Benefits of technology

It improves the efficiency and accuracy of image annotation, reduces manual intervention, unifies screening standards, and reduces correction costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119048856A_ABST
    Figure CN119048856A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides an image annotation method and device, equipment and a storage medium. The image annotation method comprises the following steps: acquiring video data, and determining an initial data set and a candidate data set according to the video data; determining a first annotation data set according to the initial data set, and performing annotation processing on the candidate data set through an annotation model to determine a second annotation data set; wherein the annotation model is obtained by training based on a first annotation data set; screening the second annotation data set to obtain a third annotation data set; and determining an image annotation result corresponding to the video data according to the first annotation data set and the third annotation data set. According to the embodiment of the invention, the image annotated by the annotation model is screened, so that the efficiency and accuracy of image annotation are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to an image annotation method, device, equipment and storage medium. Background Art

[0002] At present, by annotating the image datasets corresponding to traffic video data, we can more accurately and efficiently implement traffic statistics and study traffic laws based on the annotated image datasets, thereby improving urban planning, traffic flow management and traffic safety.

[0003] In the process of implementing image dataset annotation, the screening and initial annotation of unlabeled images, and the correction of labeled images all rely on manual labor. For long video data such as traffic videos, which include a large number of images, screening and annotating a large number of images is time-consuming and inefficient. In addition, problems such as inconsistent screening standards may occur during manual screening, further reducing the accuracy of the annotation results. It can be seen that the common methods for implementing image annotation have the problems of low efficiency and poor accuracy. Summary of the invention

[0004] The embodiments of the present application provide an image annotation method, apparatus, device and storage medium, which improve the efficiency and accuracy of image annotation.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides an image annotation method, comprising:

[0007] Acquire video data, and determine an initial data set and a candidate data set based on the video data;

[0008] Determine a first annotated data set according to the initial data set, and annotate the candidate data set through an annotation model to determine a second annotated data set; wherein the annotation model is obtained by training based on the first annotated data set;

[0009] Screening the second annotated data set to obtain a third annotated data set;

[0010] An image annotation result corresponding to the video data is determined according to the first annotation data set and the third annotation data set.

[0011] In some embodiments of the present application, the screening of the second annotated data set to obtain a third annotated data set includes:

[0012] Traversing each image in the second annotated data set, and determining a first screening value for each image in the second annotated data set;

[0013] The images in the second annotated data set are filtered according to the first filtering value to obtain the third annotated data set.

[0014] In some embodiments of the present application, the filtering of images in the second annotated data set according to the first filtering value to obtain the third annotated data set includes:

[0015] Sort the images in the second labeled data set according to the first screening value to obtain a first sorting result;

[0016] The third labeled data set is determined according to the first sorting result and a first preset number of images.

[0017] In some embodiments of the present application, determining the initial data set and the candidate data set according to the video data includes:

[0018] Determine a basic data set according to the video data;

[0019] Selecting a representative image from each image of the basic data set, and adding the representative image to the initial data set;

[0020] The candidate images in the basic data set are added to the candidate data set; wherein the candidate images are other images in the basic data set except the representative image.

[0021] In some embodiments of the present application, determining a basic data set according to the video data includes:

[0022] Performing frame extraction processing on the image corresponding to the video data according to a preset frequency to obtain a frame extracted image;

[0023] The extracted frame image is added to the basic data set.

[0024] In some embodiments of the present application, the step of selecting a representative image from each image in the basic data set includes:

[0025] Traversing each image in the basic data set, and determining a second screening value for each image in the basic data set;

[0026] The images in the basic data set are screened according to the second screening value to obtain the representative image.

[0027] In some embodiments of the present application, the step of filtering the images in the basic data set according to the second filtering value to obtain the representative image includes:

[0028] Sorting the images in the basic data set according to the second screening value to obtain a second sorting result;

[0029] The representative image is determined according to the second sorting result and a second preset number of images.

[0030] In a second aspect, the embodiment of the present application further provides an image annotation device, including:

[0031] A data determination module is used to obtain video data, determine an initial data set and a candidate data set according to the video data; determine a first annotated data set according to the initial data set, and annotate the candidate data set through an annotation model to determine a second annotated data set; wherein the annotation model is trained based on the first annotated data set;

[0032] A data screening module, used for screening the second annotated data set to obtain a third annotated data set;

[0033] A result determination module is used to determine the image annotation result corresponding to the video data according to the first annotation data set and the third annotation data set.

[0034] In a third aspect, an embodiment of the present application further provides an image annotation device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements any of the above-mentioned image annotation methods when executing the computer program.

[0035] In a fourth aspect, an embodiment of the present application further provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any of the above-described image annotation methods.

[0036] In summary, in the image annotation method provided in the embodiment of the present application, video data is acquired, and an initial data set and a candidate data set are determined based on the video data; a first annotation data set is determined based on the initial data set, and the candidate data set is annotated by an annotation model to determine a second annotation data set; wherein the annotation model is obtained based on the training of the first annotation data set; the second annotation data set is screened to obtain a third annotation data set; and the image annotation results corresponding to the video data are determined based on the first annotation data set and the third annotation data set. On the one hand, the embodiment of the present application annotates the images in the candidate data set through the annotation model to obtain a second data set including annotated images, thereby improving the efficiency of image annotation; on the other hand, the embodiment of the present application further screens the images in the second annotation data set, unifies the image screening standards, and improves the accuracy of image annotation.

[0037] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and do not limit the technical solutions of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings are embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0039] Figure 1 A schematic diagram of a method for annotating an image provided in an embodiment of the present application Figure 1 ;

[0040] Figure 2 A schematic diagram of a method for annotating an image provided in an embodiment of the present application Figure 2 ;

[0041] Figure 3 A schematic diagram of a method for annotating an image provided in an embodiment of the present application Figure 3 ;

[0042] Figure 4 A schematic diagram of a method for annotating an image provided in an embodiment of the present application Figure 4 ;

[0043] Figure 5 A schematic diagram of a method for annotating an image provided in an embodiment of the present application Figure 5 ;

[0044] Figure 6 A schematic diagram of a method for annotating an image provided in an embodiment of the present application Figure 6 ;

[0045] Figure 7 A schematic diagram of a method for annotating an image provided in an embodiment of the present application Figure 7 ;

[0046] Figure 8 A schematic diagram of a method for annotating an image provided in an embodiment of the present application Figure 8 ;

[0047] Fig. 9 A schematic diagram of an image annotation device provided in an embodiment of the present application;

[0048] Fig.10 A schematic diagram of the structure of an image annotation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0050] It should be understood that the "embodiment of the present application" or "the aforementioned embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in the embodiment of the present application" or "in the aforementioned embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. In the various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The above-mentioned sequence numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments.

[0051] In addition, the accompanying drawings are only schematic illustrations of embodiments of the present application and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and thus their repeated descriptions will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0052] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit the present disclosure. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0053] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing this application and are not intended to limit this application.

[0055] At present, by annotating the image datasets corresponding to traffic video data, we can more accurately and efficiently implement traffic statistics and study traffic laws based on the annotated image datasets, thereby improving urban planning, traffic flow management and traffic safety.

[0056] In the process of implementing image dataset annotation in related technologies, the screening and initial annotation of unlabeled images, and the correction of labeled images all rely on manual labor. For long video data such as traffic videos, a large number of images will be obtained after frame extraction. Screening and annotating a large number of images will consume a lot of manpower, and problems such as inconsistent screening standards may occur during the manual screening process, which may lead to low accuracy of the annotation results. Therefore, the efficiency and accuracy of image annotation achieved manually are low.

[0057] Based on the above problems, an embodiment of the present application proposes an image annotation method. On the one hand, the embodiment of the present application annotates the images in the candidate data set through a annotation model to obtain a second data set including annotated images, thereby improving the efficiency of image annotation; on the other hand, the embodiment of the present application further screens the images in the second annotated data set, unifies the image screening standards, and improves the accuracy of image annotation.

[0058] The image annotation method in the embodiment of the present application can be applied to an image annotation device and an image annotation apparatus, or to a hardware device or system equipped with an image annotation device or an image annotation apparatus. In the implementation process of the image annotation method in the embodiment of the application, the image annotation device and the image annotation apparatus can be a processor of a computer device, and the present application does not specifically limit this. The computer device can refer to a server, a laptop computer, a tablet computer, a desktop computer, a mobile device (such as a mobile phone, a personal digital assistant, a portable gaming device), and other devices with data processing capabilities.

[0059] The following will take the image annotation device as an example to illustrate the image annotation method in the embodiment of the present application. It can be understood that the execution subject here is only exemplary, and the embodiment of the present application does not impose any special restrictions on the execution subject.

[0060] Figure 1 A schematic diagram of a method for annotating an image provided in an embodiment of the present application Figure 1 .like Figure 1 As shown, the image annotation method at least includes steps S101 to S104.

[0061] Step S101: Acquire video data, and determine an initial data set and a candidate data set according to the video data.

[0062] In some embodiments, each image in the video data needs to be annotated so as to perform statistics on required information through a data set composed of each annotated image corresponding to the video data.

[0063] Here, the video data is a video file obtained by a video shooting tool, which may be a traffic video for recording road and traffic conditions, or other types of videos for recording other data.

[0064] The initial data set is a data set consisting of images selected from each image corresponding to the video data. The candidate data set is a data set consisting of images in some images in the video data except the images in the initial data set. For example, the video data can be subjected to frame extraction processing to obtain partial images, and the images selected from the partial images according to a preset ratio or other filtering conditions can be added to the initial image set, and the part of the partial images obtained after frame extraction processing that is not added to the initial image set can be added to the candidate image set.

[0065] In some embodiments, the images corresponding to the video data acquired by the video shooting tool can be subjected to frame extraction processing according to a preset frequency to obtain frame-extracted images, and the frame-extracted images can be added to a basic data set. The basic data set may include unlabeled images obtained after frame extraction of the video data. After the basic data set is determined, representative images can be screened out from the images of the basic data set according to preset screening rules, and the representative images can be added to the initial data set. The preset frequency can be 3 frames per second, or other frequencies. It can be understood that the preset frequency can be determined according to the frame rate of the video data.

[0066] Taking traffic video as an example, if the frame rate of the traffic video is 30 frames per second, the difference between the corresponding 30 frames of images within 1 second may not be large. Therefore, there is no need to annotate all 30 frames of images within 1 second. The beginning, middle and end images of the 30 frames of images within 1 second can be obtained for annotation, so as to achieve similar results as annotating all 30 frames of images within 1 second while improving efficiency.

[0067] In some embodiments, the image corresponding to the video data may be subjected to frame extraction processing at a higher preset frequency, that is, by extracting more frames per second, so as to improve the accuracy of the image annotation result.

[0068] In some embodiments, the preset screening rule may be to extract representative images from the basic data set according to a certain interval. For example, the preset screening rule may be to extract one frame as a representative image from each image of the basic data set every 10 frames, or to extract one frame as a representative image from each image of the basic data set every 5 frames, or to extract one frame as a representative image from each image of the basic data set every other number of frames. The specific number of intervals may be determined according to actual needs. Since adjacent images in the basic data set obtained by extracting frames of video data may have a high degree of similarity, extracting representative images from the basic data set according to a certain interval can not only reduce the similarity between images, but also reduce the number of images that need to be annotated subsequently, thereby improving the efficiency of image annotation.

[0069] In some embodiments, the preset screening rule may be to evenly distribute each image in the basic data set in each group according to a certain number, and select the most representative image from each group as a representative image through an active learning strategy. Among them, the most representative image may be an image with the highest uncertainty, and the uncertainty of the image may be determined by the image signal-to-noise ratio, image contrast, image clarity, image category probability entropy value, etc. Taking a total of 100 frames of images in the basic data set as an example, the 100 frames of images in the basic data set may be divided into a group of 10 frames each, and 10 groups may be obtained. One frame of the most representative image may be selected from each group as a representative image through an active learning strategy, and a total of 10 representative images may be obtained from the 10 groups.

[0070] In some embodiments, the preset screening rule may be to screen out a preset number of the most representative images from the basic data set as representative images through an active learning strategy. Taking a total of 100 frames of images in the basic data set as an example, the preset number may be 10 or other numbers, and the embodiments of the present application do not specifically limit this.

[0071] In some embodiments, the candidate data set may be a data set consisting of other images in the basic data set except the representative image, and the candidate data set may include other unlabeled images in the basic data set except the representative image.

[0072] In the above embodiment, the initial data set and the candidate data set are determined by screening the images corresponding to the video data, so that when the images corresponding to the video data are subsequently labeled, the model labeling method and other labeling methods can be used to label the images in the candidate data set and the initial data set respectively, thereby improving the efficiency of image labeling.

[0073] Step S102: determine a first annotated data set according to the initial data set, and annotate the candidate data set through an annotation model to determine a second annotated data set; wherein the annotation model is trained based on the first annotated data set.

[0074] In some embodiments, after obtaining the initial data set and the candidate data set, each image in the initial data set can be annotated first to determine the first annotated data set corresponding to the annotated images, and then the annotation model can be trained using the first annotated data set to annotate the candidate data sets corresponding to the unlabeled images using the trained annotation model.

[0075] Here, the annotation model may be a target detection model, for example, a target detection model composed of YOLOv8X or other target detection networks. The second annotation data set is a data set obtained by annotating each image in the candidate data set through the annotation model.

[0076] In some embodiments, the first labeled data set can be obtained by labeling each image in the initial data set according to the preset labeling category. Taking each image corresponding to the traffic video as an example, the preset labeling category can include but is not limited to: a person riding a bicycle, a person riding an electric vehicle, a person riding a motorcycle, a person riding an electric three-wheeled or four-wheeled vehicle, a bicycle without a rider (or parked in a parking area), a motorcycle without a rider (or parked in a parking area), an electric scooter without a rider (or parked in a parking area), a tricycle without a rider (or parked in a parking area), a pedestrian, a car, a bus, etc.

[0077] In some embodiments, after obtaining the first annotated data set, the annotation model can be trained using the first annotated data set. Taking the annotation model as an example, the first annotated data set can be divided into a training set, a validation set, and a test set. The training set is used to train the annotation model so that the annotation model learns how to identify the annotation category corresponding to the target type in the image and completes the image annotation; the validation set is used to evaluate the performance of the annotation model, and the image in the validation set is input into the annotation model for forward propagation to obtain the image annotation result, and the accuracy, recall rate, confusion matrix and other parameters of the annotation result are determined to evaluate the performance of the annotation model based on the above parameters. If the performance of the annotation model does not meet the preset indicators, the training set can be adjusted, and the annotation model can be trained using a new training set. After the performance of the annotation model reaches the preset indicators, the test set can also be used to further verify the generalization ability of the annotation model.

[0078] In some embodiments, after the annotation model is trained using the first annotation data set, the images in the candidate data set may be annotated using the trained annotation model, and the annotated images may be added to the second annotation data set to determine the second annotation data set.

[0079] In the above embodiment, the annotation model is trained by the first annotation data set, and the images in the candidate data set are annotated using the annotation model, thereby improving the efficiency of image annotation.

[0080] Step S103: Screen the second annotated data set to obtain a third annotated data set.

[0081] In some embodiments, after the second annotated data set is obtained, the image with the most annotation value can be screened out from the second annotated data set, and the image with the most annotation value can be added to the third annotated data set.

[0082] In some embodiments, the annotation value of the image may be measured by the uncertainty of the image category. The higher the uncertainty of the image category, the higher the annotation value of the image.

[0083] In some embodiments, a first screening value for measuring the uncertainty of the image category and a screening criterion for images in the second annotated data set can be preset, and the first screening value of each image in the second annotated data set can be determined respectively, and images in the second annotated data set whose first screening value meets the screening criterion are added to the third annotated data set.

[0084] Taking the example that the first screening value is the category probability entropy value of the image, a number can be preset as the screening criterion for the image. When the category probability entropy value of the image reaches the screening criterion, the image passes the screening and the image can be added to the third annotation data set.

[0085] In the above embodiment, the third annotated data set is obtained after screening the second annotated data set, and only the third annotated data set needs to be corrected subsequently, which reduces the cost of image correction and improves the efficiency of image annotation.

[0086] Step S104: determining an image annotation result corresponding to the video data according to the first annotation data set and the third annotation data set.

[0087] In some embodiments, the annotated images in the first annotated data set and the third annotated data set may be added to the image annotation results corresponding to the video data, and the image annotation results corresponding to the video data may also be added to the target annotation results.

[0088] In some embodiments, the annotation results of each image in the third annotation data set may be corrected to improve the accuracy of image annotation.

[0089] In some embodiments, after determining the image annotation results corresponding to the video data, the images corresponding to other video data may be annotated, and the obtained image annotation results of the other video data may be added to the target annotation results. Taking the case where the video data is a traffic video, the traffic videos corresponding to different time periods are different. After determining the image annotation results corresponding to the currently processed traffic video, the different traffic videos of each time period may be annotated to obtain a plurality of different image annotation results, and the different image annotation results may be added to the target annotation results to obtain a target annotation result including the image annotation results of each traffic video.

[0090] The embodiment of the present application annotates images in a candidate data set through an annotation model, thereby improving the efficiency of image annotation. By screening the second annotated data set to reduce the number of images in the third annotated data set that need to be corrected, not only the cost is reduced, but also the image screening standard is unified, thereby improving the accuracy of image annotation.

[0091] Figure 2 A schematic diagram of a method for annotating an image provided in an embodiment of the present application Figure 2 .like Figure 2 As shown, the second annotated data set can be screened through step S201 and step S202 to obtain a third annotated data set.

[0092] Step S201: traverse each image in the second annotated data set to determine a first screening value for each image in the second annotated data set.

[0093] Here, the second annotated data set is obtained by annotating the candidate data set through the annotation model. Among them, the images in the candidate data set are all unannotated images. The first screening value is the screening condition of the images in the second annotated data set, and the first screening value can be the category probability entropy value of the image, the confidence of the image annotation result output by the annotation model, the category probability distribution variance of the image, etc.

[0094] Taking the class probability entropy value of an image as an example, in some embodiments, the probability value of the labeled class corresponding to each image in the second labeled data set may be determined first; and then the class probability entropy value of each image in the second labeled data set may be determined based on the probability value of the labeled class. Specifically, the class probability entropy value of each image in the second labeled data set may be calculated using formula (1):

[0095]

[0096] Among them, x irepresents the labeled category of the image, p(x i ) represents the label category x i The probability value of , n represents the total number of labeled categories, and H(x) represents the category probability entropy value of the image.

[0097] Step S202: Filter the images in the second annotated data set according to the first filtering value to obtain a third annotated data set.

[0098] Here, the third annotated data set includes annotated images obtained by screening images in the second annotated data set.

[0099] In some embodiments, image screening criteria can be preset. Taking the first screening value being the category probability entropy value of the image as an example, a number can be preset as the image screening criteria. When the category probability entropy value of the image reaches the screening criteria, the image passes the screening and the image can be added to the third annotation data set.

[0100] like Figure 3 As shown, in some embodiments, when determining the third labeled data set, it can also be implemented through step S301 and step S302.

[0101] Step S301: sort the images in the second annotated data set according to the first screening value to obtain a first sorting result.

[0102] Here, the first sorting result is a sorting result of the images in the second annotated data set based on the first screening value.

[0103] Taking the first screening value as the class probability entropy value of the image as an example, the images in the second annotated data set can be sorted according to the class probability entropy value of each image to obtain a first sorting result. When the images in the second annotated data set are sorted in order from large to small according to the class probability entropy value of the image, the class probability entropy value corresponding to the image at the front is larger; when the images in the second annotated data set are sorted in order from small to large according to the class probability entropy value of the image, the class probability entropy value corresponding to the image at the back is larger.

[0104] Step S302: Determine a third annotated data set according to the first sorting result and the first preset number of images.

[0105] In some embodiments, the image with the most annotation value can be selected from the second annotation data set, and the image with the most annotation value can be added to the third annotation data set. The annotation value of the image can be measured by the uncertainty of the image category, and the higher the uncertainty of the image category, the higher the annotation value of the image.

[0106] Here, the first preset number of images refers to the number of images screened from the second annotated data set, which can be determined according to the total number of images in the second annotated data set and the actual demand for annotation efficiency. For example, if the total number of images in the second annotated data set is 900 frames, the first preset number of images can be 100 frames or other frames not greater than 900.

[0107] Taking the first screening value as the image's category probability entropy value as an example, the larger the image's category probability entropy is, the higher the uncertainty is, and the greater the corresponding annotation value is. When the images in the second annotation data set are sorted in descending order of the image's category probability entropy value, the first preset number of images at the front can be added to the third annotation data set; when the images in the second annotation data set are sorted in descending order of the image's category probability entropy value, the first preset number of images at the back can be added to the third annotation data set.

[0108] Taking the case where the first screening value is the confidence of the image annotation result output by the annotation model as an example, the lower the confidence of the image annotation result, the higher the uncertainty of the image annotation result, and the greater the corresponding annotation value. When the images in the second annotation data set are sorted in order from small to large according to the confidence of the image annotation result, the first preset number of images at the front can be added to the third annotation data set; when the images in the second annotation data set are sorted in order from large to small according to the confidence of the image annotation result, the first preset number of images at the back can be added to the third annotation data set.

[0109] In the above embodiment, the images in the second annotated data set are screened to determine the third annotated data set, thereby unifying the screening criteria and improving the accuracy of image annotation.

[0110] Figure 4 A schematic diagram of a method for annotating an image provided in an embodiment of the present application Figure 3 .like Figure 4 As shown, the initial data set and the candidate data set can be determined through steps S401 to S403.

[0111] Step S401: Determine a basic data set according to video data.

[0112] In some embodiments, when counting the number, flow rate and other information of target objects in the video data through video data, the video data can be converted into image sequence frames, and then the images therein can be added to the data set. The images in the data set can be labeled to facilitate the counting of the number, flow rate and other information of target objects in the video data.

[0113] Here, the basic dataset includes unlabeled images corresponding to the video data.

[0114] like Figure 5 As shown, in some embodiments, when determining the basic data set according to the video data, it can be implemented through step S501 and step S502.

[0115] Step S501: Perform frame extraction processing on an image corresponding to video data according to a preset frequency to obtain a frame extracted image.

[0116] Here, the preset frequency is a preset frequency for performing frame extraction processing on the images corresponding to the video data. The preset frequency may be once every 5 frames or other frequencies. It is understandable that the preset frequency may be determined according to the frame rate of the video data. For example, if the video data is a traffic video, if the frame rate of the traffic video is 15 frames per second, the difference between the corresponding 15 frames of images within 1 second may not be large, so there is no need to annotate all 15 frames of images within 1 second. The images at the beginning, middle and end of the 15 frames of images within 1 second may be obtained for annotation, so as to improve efficiency while achieving similar results as annotating all 15 frames of images within 1 second. In some embodiments, the images corresponding to the video data may be subjected to frame extraction processing at a higher preset frequency to improve the accuracy of the image annotation results.

[0117] In some embodiments, all images corresponding to the video data may be subjected to frame extraction processing according to a preset frequency, and the partial images obtained after frame extraction are used as the frame extracted images. For example, if all images corresponding to the video data have 100 frames and the preset frequency is 3 frames extracted per second, 10 frames of frame extracted images may be obtained after frame extraction processing of all images corresponding to the video data.

[0118] Step S502: adding the extracted frame image to the basic data set.

[0119] In some embodiments, after the decimated image is obtained, the decimated image may be added to the basic data set.

[0120] Step S402: Select representative images from the images in the basic data set, and add the representative images to the initial data set.

[0121] Here, the representative image is an image selected from each image of the basic data set.

[0122] In some embodiments, representative images can be screened out from the images in the basic data set according to preset screening rules, and the representative images can be added to the initial data set. For example, the preset screening rules can be to extract representative images from the basic data set according to a certain interval, or to evenly distribute the images in the basic data set in each group according to a certain number, and to screen out the most representative images from each group as representative images through an active learning strategy. Among them, the most representative image can be the image with the highest uncertainty, and the uncertainty of the image can be determined by the image signal-to-noise ratio, image contrast, image clarity, image category probability entropy value, etc.

[0123] like Figure 6 As shown, the representative image can be determined through step S601 and step S602.

[0124] Step S601: traverse each image in the basic data set to determine a second screening value for each image in the basic data set.

[0125] Here, the second screening value is a screening condition for images in the basic data set, and the second screening value may be image signal-to-noise ratio, image contrast, image clarity, and the like.

[0126] Taking image contrast as an example, the image contrast can be calculated by formula (2):

[0127] C=(G max -G min ) / (G max +G min ) (2)

[0128] Among them, C represents the image contrast, G max Indicates the maximum pixel gray value, G min Indicates the minimum pixel gray value.

[0129] Step S602: Filter the images in the basic data set according to the second filtering value to obtain a representative image.

[0130] In some embodiments, taking the second screening value being the image contrast as an example, a number can be preset as the screening standard for the image. When the image contrast value of the image reaches the screening standard, the image passes the screening and the image can be used as a representative image.

[0131] In some embodiments, when determining the representative image, it can also be implemented through step S701 and step S702.

[0132] Step S701: sort the images in the basic data set according to the second screening value to obtain a second sorting result.

[0133] Here, the second sorting result is a sorting result of images in the basic data set.

[0134] Taking the case where the second screening value is the image contrast of the image as an example, the images in the basic data set can be sorted according to the image contrast of each image to obtain a second sorting result. When the images in the basic data set are sorted in descending order of the image contrast of the images, the images that are ranked closer to the front have a greater image contrast; when the images in the basic data set are sorted in descending order of the image contrast of the images, the images that are ranked closer to the back have a greater image contrast.

[0135] Step S702: Determine a representative image according to the second sorting result and the second preset number of images.

[0136] In some embodiments, the image with the most annotation value can be screened out from the basic data set and used as the representative image. The annotation value of the image can be measured by the uncertainty of the image category. The higher the uncertainty of the image category, the higher the annotation value of the image.

[0137] Here, the second preset number of images refers to the number of images screened from the basic data set, which can be determined based on the total number of images in the basic data set and the actual demand for annotation efficiency. For example, if the total number of images in the basic data set is 100 frames, the second preset number of images can be 10 frames or other frames not greater than 100.

[0138] Taking the case where the second screening value is the image contrast of the image as an example, the smaller the image contrast of the image, the higher the uncertainty, and the greater the corresponding annotation value. When the images in the basic data set are sorted in order from large to small according to the image contrast of the image, the second preset number of images at the back can be used as representative images; when the images in the basic data set are sorted in order from small to large according to the image contrast of the image, the second preset number of images at the front can be used as representative images.

[0139] Step S403: adding the candidate images in the basic data set to the candidate data set; wherein the candidate images are other images in the basic data set other than the representative images.

[0140] In some embodiments, representative images selected from the basic data set can be added to the initial data set, and other images in the basic data set can be added to the candidate data set. This facilitates the subsequent annotation of images in the initial data set and the candidate data set by different methods. For example, the initial data set with a small number of images can be annotated, and the annotation model can be trained on the annotated image data set to annotate the candidate data set with a large number of images through the annotation model, thereby improving the efficiency of image annotation.

[0141] In the above embodiment, the images in the basic data set are screened to determine the initial data set, thereby reducing the number of annotated images and improving the efficiency of image annotation.

[0142] On the one hand, the embodiment of the present application annotates the images in the candidate data set through the annotation model to obtain a second data set including the annotated images, thereby improving the efficiency of image annotation; on the other hand, the embodiment of the present application further screens the images in the second annotated data set, unifies the image screening standards, and improves the accuracy of image annotation.

[0143] Based on the above embodiments, another embodiment of the present application exemplarily illustrates the application of the image annotation method in an actual scenario.

[0144] At present, the image datasets corresponding to traffic video data are annotated to more accurately and efficiently realize traffic statistics and study traffic laws based on the annotated image datasets, thereby improving urban planning, traffic flow management and traffic safety. Traffic video data usually consists of long-term surveillance videos. These videos cover the traffic conditions at different locations in the city, including the number, speed, flow pattern, traffic congestion, etc. of pedestrians and bicycles. This provides rich data for urban traffic research and planning, and is a powerful resource for traffic statistics and travel law research. Existing traffic detection and statistics are often based on deep learning target detection methods, which require video frame extraction and conversion to images for annotation. However, in general, for a day's surveillance video of an intersection, there are tens of thousands to hundreds of thousands of frames of images after frame extraction. How to annotate bicycle traffic video datasets for surveillance videos containing a large amount of information is a challenging problem.

[0145] The traditional labeling method is to manually label long videos by extracting frames. This method consumes a lot of manpower, has low labeling efficiency, and takes a long labeling cycle. In addition to traditional methods, in recent years, with the development of various classic target detection networks, some semi-automatic labeling methods based on target detection networks have emerged. However, the existing semi-automatic labeling methods are often manually non-strategically screening the results of model labeling and making corrections. For large-scale traffic video datasets, there is still the problem of low labeling efficiency.

[0146] Based on the above problems, the embodiment of the present application proposes a semi-automatic image annotation method based on active learning. The images extracted from traffic video frames are semi-automatically annotated through a target detection network, and the annotated images are screened through an active learning method, thereby improving the efficiency and accuracy of image annotation.

[0147] The image annotation method in the embodiment of the present application can be applied to an image annotation device and an image annotation apparatus, or to a hardware device or system equipped with an image annotation device or an image annotation apparatus. In the implementation process of the image annotation method in the embodiment of the application, the image annotation device and the image annotation apparatus can be a processor of a computer device, and the present application does not specifically limit this. The computer device can refer to a server, a laptop computer, a tablet computer, a desktop computer, a mobile device (such as a mobile phone, a personal digital assistant, a portable gaming device), and other devices with data processing capabilities. Figure 8 A schematic diagram of a method for annotating an image provided in an embodiment of the present application Figure 8 ,like Figure 8 As shown, the embodiment of the present application includes steps S801 to S806:

[0148] Step S801: Acquire traffic video data.

[0149] A traffic video is acquired and preprocessed to obtain an image data set A (corresponding to the basic data set in the above embodiment).

[0150] Step S801 specifically includes steps S8011 to S8012.

[0151] Step S8011, using the high-position video and low-pile video equipment of the Beijing Parking Management Affairs Center, combined with manual shooting, to shoot the collected data sections at different time periods, different angles and different scenes, and quickly obtain a large number of non-motorized vehicle lane videos of traffic roads. The scenes include the driving conditions of pedestrians, bicycles and electric vehicles on the roads during peak hours, off-peak hours and night time.

[0152] Step S8012: Use Open CV or other video processing software to perform frame extraction on the acquired video of the non-motorized vehicle lane of the traffic road, extract image data every 10 frames, and obtain a high-quality image data set A corresponding to the target video.

[0153] Step S802: label the basic data set.

[0154] A data set label is formulated, and the image data set A is screened according to the data set label to obtain a data set to be labeled (corresponding to the initial data set in the above embodiment), and the images in the data set to be labeled are labeled to obtain a labeled image set B (corresponding to the first labeled data set in the above embodiment).

[0155] Step S802 specifically includes step S8021 and step S8022.

[0156] Step S8021: Label each person and non-motor vehicle in the non-motor vehicle lane video of the traffic road. In some embodiments, the labels may include but are not limited to: a person riding a bicycle, a person riding an electric vehicle, a person riding a motorcycle, a person riding an electric three-wheeled or four-wheeled vehicle, a bicycle without a rider (or parked in a parking area), a motorcycle without a rider (or parked in a parking area), an electric scooter without a rider (or parked in a parking area), a tricycle without a rider (or parked in a parking area), a pedestrian, a car, a bus, etc.

[0157] Step S8022: annotate each image in the image data set A using the defined label, and add each annotated image to the annotated image set B.

[0158] Step S803: training the algorithm model.

[0159] Use the labeled image set B to train YOLOv8x or other target detection models to obtain the target detection model weight P.

[0160] Step S804: use the model annotation.

[0161] The target detection model weight P is used to infer and annotate images that are not included in the dataset to be annotated (corresponding to the images in the candidate dataset in the above embodiment), and the obtained images with annotated information are added to the image dataset C (corresponding to the second annotated dataset in the above embodiment).

[0162] Step S805: Active learning strategy screening.

[0163] Active learning screening is performed on the image data set C, and the images with the most annotation value in the image data set C are added to the image set D to be corrected (corresponding to the third annotated data set in the above embodiment).

[0164] In some embodiments, uncertainty can be measured by the category probability entropy value of the image. The category probability entropy value of each image in the image data set C can be determined by formula (1), and each image is sorted in order from large to small according to the category probability entropy value. The larger the category probability entropy value of the image, the less likely the image is to be predicted, and the more valuable the image is to be labeled. Exemplarily, the first K images can be selected to be added to the image set D to be corrected (corresponding to the third labeled data set in the above embodiment). Among them, K is the first preset number of images, and the first preset number of images can be set according to the specific situation. For example, there are 100 frames of images in the image data set C, and the first preset number of images can be 10 or other numbers less than 100.

[0165] Step S806: Correct the image set.

[0166] Correct each image and the corresponding annotation in the image set D to be corrected, and add the corrected image to the annotated image result set.

[0167] Step S807, proceed to the next round of iteration: repeat steps S1001 to S806 to complete the creation of image data sets corresponding to multiple traffic road non-motorized vehicle lane videos.

[0168] Based on the above embodiment, by performing frame extraction processing on the non-motorized vehicle lane video of the traffic road, the similarity between images is reduced, the number of images that need to be labeled is reduced, and the uncertainty is measured through active learning to select images with higher labeling value for labeling, thereby improving the image labeling efficiency.

[0169] In yet another embodiment of the present application, based on the same inventive concept as the above-mentioned embodiment, Fig. 9 Schematic diagram of an image annotation device provided in an embodiment of the present application. Fig. 9 As shown, the image annotation device 900 includes a data determination module 901, a data screening module 902 and a result determination module 903, wherein:

[0170] The data determination module 901 is used to obtain video data, determine an initial data set and a candidate data set according to the video data; determine a first annotated data set according to the initial data set, and annotate the candidate data set through an annotation model to determine a second annotated data set; wherein the annotation model is obtained by training based on the first annotated data set;

[0171] A data screening module 902, configured to screen the second annotated data set to obtain a third annotated data set;

[0172] The result determination module 903 is used to determine the image annotation result corresponding to the video data according to the first annotation data set and the third annotation data set.

[0173] In some embodiments of the present application, the data filtering module 902 is specifically used to traverse each image in the second annotated data set, determine a first filtering value for each image in the second annotated data set; and filter the images in the second annotated data set according to the first filtering value to obtain the third annotated data set.

[0174] In some embodiments of the present application, the data screening module 902 is further specifically used to sort the images in the second annotated data set according to the first screening value to obtain a first sorting result; and determine the third annotated data set according to the first sorting result and a first preset number of images.

[0175] In some embodiments of the present application, the first screening value includes an entropy value, and the data screening module 902 is further specifically used to determine the probability value of the annotation category corresponding to each image in the second annotation data set; and determine the category probability entropy value of each image in the second annotation data set based on the probability value of the annotation category.

[0176] In some embodiments of the present application, the data determination module 901 is specifically used to determine a basic data set based on the video data; screen representative images from each image in the basic data set, and add the representative images to the initial data set; add candidate images in the basic data set to the candidate data set; wherein the candidate images are other images in the basic data set other than the representative images.

[0177] In some embodiments of the present application, the data determination module 901 is further specifically used to perform frame extraction processing on the image corresponding to the video data according to a preset frequency to obtain a frame-extracted image; and add the frame-extracted image to the basic data set.

[0178] In some embodiments of the present application, the data determination module 901 is further specifically used to traverse each image in the basic data set, determine a second filtering value for each image in the basic data set; and filter the images in the basic data set according to the second filtering value to obtain the representative image.

[0179] In some embodiments of the present application, the data determination module 901 is further specifically used to sort the images in the basic data set according to the second screening value to obtain a second sorting result; and determine the representative image according to the second sorting result and a second preset number of images.

[0180] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. In some embodiments, the functions or modules included in the device provided in the embodiment of the present application can be used to execute the method described in the above method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0181] An embodiment of the present application provides an image annotation device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, some or all of the steps in the above method are implemented.

[0182] The embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, some or all of the steps in the above method are implemented. The computer-readable storage medium can be transient or non-transient.

[0183] An embodiment of the present application provides a computer program, including a computer-readable code. When the computer-readable code is run in a computer device, a processor in the computer device executes some or all of the steps for implementing the above method.

[0184] The embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, some or all of the steps in the above method are implemented. The computer program product can be implemented specifically by hardware, software or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium, and in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK) and the like.

[0185] It should be noted here that the description of the various embodiments above tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. The description of the above device, storage medium, computer program and computer program product embodiments is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the embodiments of the device, storage medium, computer program and computer program product of this application, please refer to the description of the method embodiment of this application for understanding.

[0186] Fig.10 A hardware entity diagram of an image annotation device provided in an embodiment of the present application is shown in FIG. Fig.10As shown, the hardware entity of the image annotation device 1000 includes: a processor 1001 and a memory 1002, wherein the memory 1002 stores a computer program that can be run on the processor 1001, and the processor 1001 implements the steps in the method of any of the above embodiments when executing the program.

[0187] The memory 1002 stores a computer program that can be run on the processor. The memory 1002 is configured to store instructions and applications executable by the processor 1001. It can also cache data to be processed or processed by the processor 1001 and each module in the image annotation device 1000 (for example, image data, audio data, voice communication data, and video communication data). This can be achieved through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0188] When the processor 1001 executes the program, the steps of any of the above-mentioned image annotation methods can be implemented. The processor 1001 generally controls the overall operation of the image annotation device 1000 .

[0189] An embodiment of the present application provides a computer storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps of the image annotation method in any of the above embodiments.

[0190] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0191] The processor may be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a central processing unit, a controller, a microcontroller, and a microprocessor. It is understandable that the electronic device that implements the functions of the processor may also be other, and the embodiments of the present application are not specifically limited.

[0192] The above-mentioned computer storage medium / memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory (Flash Memory), a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM) and the like; it can also be various terminals including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0193] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the serial numbers of the above-mentioned steps / processes does not mean the order of execution, and the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments.

[0194] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0195] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0196] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0197] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0198] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0199] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for enabling an image annotation device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0200] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. An image annotation method, It is characterized in that include: Acquire video data, and determine an initial data set and a candidate data set based on the video data; Determine a first annotated data set according to the initial data set, and annotate the candidate data set through an annotation model to determine a second annotated data set; wherein the annotation model is obtained by training based on the first annotated data set; Screening the second annotated data set to obtain a third annotated data set; An image annotation result corresponding to the video data is determined according to the first annotation data set and the third annotation data set.

2. The image annotation method according to claim 1, It is characterized in that The step of screening the second annotated data set to obtain a third annotated data set includes: Traversing each image in the second annotated data set, and determining a first screening value for each image in the second annotated data set; The images in the second annotated data set are filtered according to the first filtering value to obtain the third annotated data set.

3. The image annotation method according to claim 2, It is characterized in that The step of filtering the images in the second annotated data set according to the first filtering value to obtain the third annotated data set includes: Sort the images in the second labeled data set according to the first screening value to obtain a first sorting result; The third labeled data set is determined according to the first sorting result and a first preset number of images.

4. The image annotation method according to claim 1, It is characterized in that The determining the initial data set and the candidate data set according to the video data includes: Determine a basic data set according to the video data; Selecting a representative image from each image of the basic data set, and adding the representative image to the initial data set; The candidate images in the basic data set are added to the candidate data set; wherein the candidate images are other images in the basic data set except the representative image.

5. The image annotation method according to claim 4, It is characterized in that The determining of a basic data set according to the video data comprises: Performing frame extraction processing on the image corresponding to the video data according to a preset frequency to obtain a frame extracted image; The extracted frame image is added to the basic data set.

6. The image annotation method according to claim 4, It is characterized in that The step of selecting a representative image from each image of the basic data set comprises: Traversing each image in the basic data set, and determining a second screening value for each image in the basic data set; The images in the basic data set are screened according to the second screening value to obtain the representative image.

7. The image annotation method according to claim 6, It is characterized in that The step of screening the images in the basic data set according to the second screening value to obtain the representative image includes: Sorting the images in the basic data set according to the second screening value to obtain a second sorting result; The representative image is determined according to the second sorting result and a second preset number of images.

8. An image annotation device, It is characterized in that include: A data determination module, used to obtain video data and determine an initial data set and a candidate data set according to the video data; Determine a first annotated data set according to the initial data set, and annotate the candidate data set through an annotation model to determine a second annotated data set; wherein the annotation model is obtained by training based on the first annotated data set; A data screening module, used for screening the second annotated data set to obtain a third annotated data set; A result determination module is used to determine the image annotation result corresponding to the video data according to the first annotation data set and the third annotation data set.

9. An image annotation device, comprising a memory and a processor, wherein the memory stores a computer program. It is characterized in that When the processor executes the computer program, the image annotation method according to any one of claims 1 to 7 is implemented.

10. A storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the image annotation method according to any one of claims 1 to 7 is implemented.