Picture labeling method and device, electronic equipment and storage medium

Abstract

The invention provides a picture labeling method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a manual labeling picture and a to-be-labeled picture, determining a task state as a pre-labeling state in response to a task association table, carrying out the model training through employing the manual labeling picture, and obtaining a picture pre-labeling model, the picture pre-labeling model is used for pre-labeling a to-be-labeled picture, outputting an algorithm pre-labeling picture, sending the algorithm pre-labeling picture output by the picture pre-labeling model to a user, obtaining the algorithm pre-labeling picture uploaded after the user performs secondary labeling and verification on the algorithm pre-labeling picture according to labeling information, and outputting the algorithm pre-labeling picture to the user. And carrying out model training on the manually annotated picture and the algorithm pre-annotated picture after secondary annotation and verification to obtain a picture annotation model, and carrying out picture annotation on the picture set uploaded by the user by adopting the picture annotation model. According to the method, the picture annotation model is obtained through high-quality annotation picture training, user annotation requirements are matched, and flexible and efficient picture annotation is achieved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, electronic device and storage medium for image annotation. Background Art

[0002] Image annotation refers to the process of adding descriptive labels or bounding boxes to objects or regions in an image. Labels and bounding boxes are usually used for object detection, image classification, semantic segmentation, etc. to understand and identify the content in the image.

[0003] At present, annotators usually use an annotation system to annotate images. The annotation system mainly includes Web-based online annotation. Specifically, annotators need to manually draw boxes near multiple targets. The Web end sends the box information and picture information to the server end. The server end returns the recognition results to the Web end. The annotators see that multiple targets are boxed. The annotators can only receive the box selection results determined after the server fixes the annotation, resulting in the inability to flexibly match the user's annotation needs, further affecting the effect of image annotation. Summary of the invention

[0004] In view of this, the present application aims to propose a method, device, electronic device and storage medium for image annotation, so as to solve the problem that the annotators can only receive the box selection results determined after the fixed annotation by the server, resulting in the inability to flexibly match the annotation requirements of the users, further affecting the image annotation effect, and to achieve flexible and efficient image annotation.

[0005] According to a first aspect of the present application, a method for labeling an image is provided, the method comprising:

[0006] Obtaining manually annotated pictures and pictures to be annotated; wherein the manually annotated pictures and pictures to be annotated are associated with a task association table, and the task association table is used to determine the task status;

[0007] In response to the task association table determining that the task state is a pre-annotation state, the manually annotated picture is used to perform model training to obtain a picture pre-annotation model; the picture pre-annotation model is used to pre-annotate the picture to be annotated, and output the algorithm pre-annotated picture;

[0008] Sending the algorithm-pre-annotated picture output by the picture pre-annotation model to the user; the algorithm-pre-annotated picture includes annotation information;

[0009] Obtaining the algorithm-pre-annotated picture uploaded by the user after secondary annotation and verification of the algorithm-pre-annotated picture according to the annotation information;

[0010] The manually annotated pictures and the algorithm-pre-annotated pictures after secondary annotation and verification are used for model training to obtain a picture annotation model, and the picture annotation model is used to annotate the picture set uploaded by the user.

[0011] Optionally, the step of obtaining manually annotated pictures and pictures to be annotated includes:

[0012] Obtain manually annotated images and images to be annotated;

[0013] The manually annotated pictures and the pictures to be annotated are stored in a pre-created sharing group table to obtain a picture set; the picture set is used for users to add, modify or delete pictures and the annotated information of the pictures;

[0014] The manually annotated pictures and annotation information in the picture set are stored in an object storage tool to generate a picture data source table.

[0015] Optionally, before obtaining the manually annotated pictures and the pictures to be annotated, the method further includes:

[0016] Get the batch of images to be annotated uploaded by users;

[0017] Associating a preset number of to-be-annotated pictures with a pre-created task table to obtain a task association table; the task association table includes a picture list and a task status;

[0018] Sending the task association table to the user; the task association table is used to synchronize task status, and allows the user to preview the picture list and manually annotate the pictures to be annotated;

[0019] In response to the task association table determining that the task status is manually labeled, the manually labeled picture in the task association table is obtained.

[0020] Optionally, in response to the task association table determining that the task state is a pre-annotation state, using the manually annotated picture to perform model training to obtain a picture pre-annotation model includes:

[0021] In response to the task association table determining that the task state is a pre-annotated state, calling a preset image to start a pre-annotated algorithm container;

[0022] Downloading the manually annotated image to a pre-annotation algorithm container through an object storage tool for model training to obtain an image pre-annotation model;

[0023] The image pre-labeling model is used to pre-label the image to be labeled, and an algorithm pre-labeled image is output.

[0024] Optionally, after pre-annotating the to-be-annotated picture by using the picture pre-annotation model and outputting the algorithm pre-annotated picture, the method further includes:

[0025] Pre-label the picture according to the algorithm output by the picture pre-labeling model, and update the picture data source table in the object storage tool.

[0026] Optionally, the manually annotated pictures and the algorithm-annotated pictures after secondary annotation and verification are used for model training to obtain a picture annotation model, including:

[0027] In response to the task association table determining that the task state is a pre-marking end state, calling a preset image to start a marking algorithm container;

[0028] Download the manually annotated images and the algorithm-annotated images after secondary annotation and verification to the annotation algorithm container for model training to obtain an image annotation model;

[0029] The trained image annotation model is stored in an object storage tool.

[0030] Optionally, the adopting the picture annotation model to perform picture annotation on a picture set uploaded by a user includes:

[0031] In response to a user uploading or updating a to-be-annotated picture in the picture set, calling a picture annotation model of the object storage tool;

[0032] The image annotation model is used to annotate the image to be annotated, so as to obtain annotated images and annotation information.

[0033] According to a second aspect of the present application, a picture annotation device is provided, the method comprising:

[0034] An image acquisition module is used to acquire manually annotated images and images to be annotated; wherein the manually annotated images and images to be annotated are associated with a task association table, and the task association table is used to determine the task status;

[0035] A pre-labeling module, configured to, in response to the task association table determining that the task state is a pre-labeling state, use the manually labeled picture to perform model training to obtain a picture pre-labeling model; the picture pre-labeling model is used to pre-label the picture to be labeled, and output the algorithm pre-labeling picture;

[0036] A sending module, used to send the algorithm pre-annotated picture output by the picture pre-annotation model to a user; the algorithm pre-annotated picture includes annotation information;

[0037] The image verification module is used to obtain the algorithm-annotated image uploaded by the user after secondary annotation and verification of the algorithm-annotated image according to the annotation information;

[0038] The labeling module is used to use the manually labeled pictures and the algorithm-pre-labeled pictures after secondary labeling and verification to perform model training, obtain a picture labeling model, and use the picture labeling model to label the picture set uploaded by the user.

[0039] According to another aspect of the present application, there is also provided an electronic device, including:

[0040] processor;

[0041] a memory for storing instructions executable by the processor;

[0042] The processor is configured to execute the instructions to implement the image tagging method as described above.

[0043] According to another aspect of the present application, a readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the image tagging method described above are implemented.

[0044] The image annotation method provided in the embodiment of the present application obtains manually annotated images and images to be annotated, determines that the task status is a pre-annotation status in response to a task association table, uses manually annotated images to perform model training, obtains an image pre-annotation model, the image pre-annotation model is used to pre-annotate the images to be annotated, and outputs algorithm pre-annotated images, sends the algorithm pre-annotated images output by the image pre-annotation model to the user, obtains the algorithm pre-annotated images uploaded by the user after secondary annotation and verification of the algorithm pre-annotated images according to the annotation information, uses manually annotated images and the algorithm pre-annotated images after secondary annotation and verification to perform model training, obtains an image annotation model, and uses the image annotation model to annotate the image set uploaded by the user. The embodiment of the present application uses manually annotated pictures as training data to train a model that can pre-annotate pictures to be annotated, obtains algorithm pre-annotated pictures and sends them to users, ensures the accuracy and reliability of the pictures annotated by the pre-annotated model, and the user re-annotates and verifies the algorithm pre-annotated pictures according to the annotation information, trains the model on the annotated pictures that meet the user's requirements, and obtains a picture annotation model that matches the user's annotation needs. High-quality annotated pictures are used as training data to train an accurate picture annotation model, ensuring that the picture annotation model can automatically and efficiently annotate a large number of pictures, thereby improving the annotation efficiency and annotation quality and achieving flexible and efficient image annotation.

[0045] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0047] Figure 1 is a flowchart of a method for annotating an image provided by an embodiment of the present application;

[0048] Figure 2 yes Figure 1 A flowchart of step 101 in the image annotation method provided in an embodiment of the present application;

[0049] Figure 3 yes Figure 1 A flowchart of step 102 in the image annotation method provided in an embodiment of the present application;

[0050] Figure 4 yes Figure 1 A flowchart of step 105 in the picture annotation method provided in an embodiment of the present application;

[0051] Figure 5 is a flowchart of another method for marking pictures provided in an embodiment of the present application;

[0052] Figure 6 is a structural schematic diagram of a picture annotation device provided in an embodiment of the present application;

[0053] Figure 7 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, each embodiment of the present application will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in each embodiment of the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is for the convenience of description, and the specific implementation of the present application should not constitute any limitation, and the various embodiments can be combined with each other and referenced to each other without contradiction.

[0055] Reference Figure 1 , shows a flowchart of the steps of the image annotation method provided by an embodiment of the present application, and the method may include:

[0056] Step 101, obtaining manually annotated pictures and pictures to be annotated; wherein the manually annotated pictures and pictures to be annotated are associated with a task association table, and the task association table is used to determine the task status.

[0057] In an embodiment of the present application, in order to solve the problem that the labeling personnel can only receive the box selection results determined after the server fixes the labeling, resulting in the labeling results being unable to flexibly match the user's labeling needs, further affecting the image labeling effect, this embodiment first uses a small number of manually labeled pictures to train a model that can pre-label the pictures to be labeled, and automatically pre-labels to obtain labeled pictures. The user performs secondary labeling and verification on the pictures labeled by the algorithm, and uses high-quality labeled pictures that meet user needs to train to obtain a more accurate picture labeling model, which flexibly matches the user's labeling needs, improves labeling efficiency and data quality, and realizes flexible and efficient labeling of pictures.

[0058] Specifically, the system obtains manually annotated pictures and pictures to be annotated, specifically, pictures uploaded by users, including manually annotated pictures and pictures to be annotated. Manually annotated pictures can be a small number of pictures pre-annotated by technical personnel according to annotation requirements. The pictures uploaded by users are stored in a pre-created sharing group table to obtain a picture set. The picture set is used for users to add, modify or delete pictures and picture annotation information. Users can manage picture sets through the front-end interface. The system stores the manually annotated pictures and annotation information in the picture set in the object storage tool. The object storage tool MinIO provides an efficient storage and access mechanism to facilitate subsequent annotation and model training.

[0059] It should be noted that users can upload pictures to be labeled to the system through the batch upload function. The system obtains the pictures to be labeled uploaded by users in batches, associates a preset number of manually labeled pictures with the pre-created task table, obtains a task association table, and associates the pictures to be labeled with the task association table. The task association table includes a picture list and task status. The task status may include manual labeling, pre-labeling, manual verification and picture labeling, which will not be elaborated here.

[0060] Step 102, in response to the task association table determining that the task state is a pre-labeling state, the manually labeled picture is used for model training to obtain a picture pre-labeling model; the picture pre-labeling model is used to pre-label the picture to be labeled, and output the algorithm pre-labeling picture.

[0061] It should be noted that this embodiment uses a task association table to synchronize the annotation progress. Specifically, the task association table includes a list of pictures to be annotated and a task status. The task status includes manual annotation, pre-annotation, manual verification and picture annotation.

[0062] Specifically, when the manually annotated picture is determined and the upload progress of the picture to be annotated meets the pre-annotation conditions, the task association table determines that the task status is the pre-annotation status. In response to the task status of the task association table being pre-annotation, that is, when the task status of the task association table is updated to pre-annotation, the system responds and prepares to perform the pre-annotation operation, and uses the manually annotated pictures to train the model to obtain the picture pre-annotation model. During the training process, the annotation information of the manually annotated pictures is used as training data to train a model that can pre-annotate the pictures to be annotated. The picture pre-annotation model is used to pre-annotate the pictures to be annotated, and the algorithm pre-annotated pictures are output. The pre-annotation model analyzes the pictures to be annotated through an algorithm to generate annotation information. The algorithm pre-annotated pictures output by the pre-annotation model include the annotation information. The system sends these algorithm pre-annotated pictures to the user, and the user can view and download these algorithm pre-annotated pictures through the front-end interface.

[0063] This embodiment uses a pre-labeling algorithm and manually labeled pictures to perform model training, and can flexibly call different pre-labeling algorithms to adapt to different labeling requirements, ensure the accuracy and reliability of the pre-labeling model, and thus use the picture pre-labeling model to automatically pre-label the pictures to be labeled, thereby reducing the workload of manual labeling and improving labeling efficiency.

[0064] Step 103: Send the algorithm-pre-annotated picture output by the picture pre-annotation model to the user; the algorithm-pre-annotated picture includes annotation information.

[0065] Step 104 , obtaining the algorithm-pre-annotated image uploaded by the user after secondary annotation and verification based on the annotation information.

[0066] Specifically, the algorithm pre-annotated pictures uploaded by the user after secondary annotation and verification according to the annotation information are obtained, so that the model training can be carried out using the algorithm pre-annotated pictures after secondary annotation and verification by the user. In this embodiment, the algorithm pre-annotated pictures output by the picture pre-annotation model are sent to the user, and the front end reverses the picture annotation through json data. The user can perform secondary annotation and other operations and verify the data. Among them, the sharing function can share the annotated pictures with other users for joint verification, and the verification progress threshold can be set. When the threshold is reached, the next step can be put into picture annotation model training.

[0067] Step 105, manually annotated pictures and algorithm pre-annotated pictures after secondary annotation and verification are used for model training to obtain a picture annotation model, and the picture annotation model is used to annotate the picture set uploaded by the user.

[0068] Specifically, when the task status of the task association table is updated to pre-labeling completed, the system responds and prepares to perform image labeling model training operations, uses manually labeled images and secondary labeled and verified algorithm pre-labeled images for model training, obtains the image labeling model, and uses the image labeling model to label the image set uploaded by the user.

[0069] Specifically, the image annotation model training process is similar to the image pre-annotation model training process. The manually annotated images and the algorithm pre-annotated images after secondary annotation and verification are downloaded to the annotation algorithm container for model training. In the annotation algorithm container, all annotated images are used for model training to obtain the image annotation model. During the training process, high-quality annotated images that have been manually annotated and the algorithm pre-annotated images after secondary annotation and verification are used as training data to train a more accurate image annotation model. The trained image annotation model is stored in the object storage tool to ensure the safe storage and fast access of the model, so that the image annotation model is used to annotate the image set uploaded by the user.

[0070] The image annotation method provided in the embodiment of the present application obtains manually annotated images and images to be annotated, determines that the task status is a pre-annotation status in response to a task association table, uses manually annotated images to perform model training, obtains an image pre-annotation model, the image pre-annotation model is used to pre-annotate the images to be annotated, and outputs algorithm pre-annotated images, sends the algorithm pre-annotated images output by the image pre-annotation model to the user, obtains the algorithm pre-annotated images uploaded by the user after secondary annotation and verification of the algorithm pre-annotated images according to the annotation information, uses manually annotated images and the algorithm pre-annotated images after secondary annotation and verification to perform model training, obtains an image annotation model, and uses the image annotation model to annotate the image set uploaded by the user. The embodiment of the present application uses manually annotated pictures as training data to train a model that can pre-annotate pictures to be annotated, obtains algorithm pre-annotated pictures and sends them to users, ensures the accuracy and reliability of the pictures annotated by the pre-annotated model, and the user re-annotates and verifies the algorithm pre-annotated pictures according to the annotation information, trains the model on the annotated pictures that meet the user's requirements, and obtains a picture annotation model that matches the user's annotation needs. High-quality annotated pictures are used as training data to train an accurate picture annotation model, ensuring that the picture annotation model can automatically and efficiently annotate a large number of pictures, thereby improving the annotation efficiency and annotation quality and achieving flexible and efficient image annotation.

[0071] Further, see Figure 2 , showing Figure 1 A flowchart of step 101 in a picture annotation method is provided. The method is basically the same as the picture annotation method provided in the first embodiment of the present application. Step 101 may include:

[0072] Step 1011, obtaining manually annotated pictures and pictures to be annotated;

[0073] Step 1012, the manually annotated pictures and the pictures to be annotated are stored in a pre-created sharing group table to obtain a picture set; the picture set is used for users to add, modify or delete pictures and the annotated information of the pictures.

[0074] Step 1013: store the manually annotated pictures and annotation information in the picture set into an object storage tool to generate a picture data source table.

[0075] In an embodiment of the present application, the system obtains pictures uploaded by users, including manually annotated pictures and pictures to be annotated. The system stores the pictures uploaded by users in a pre-created sharing group table amap_share_team_users to obtain a picture set. The sharing group table records information such as the creator ID and sharing group ID of the picture set. The picture set is used by users to add, modify or delete pictures and the annotation information of the pictures. Users can manage the picture set through the front-end interface, and the system stores the manually annotated pictures and annotation information in the picture set to the object storage tool.

[0076] It should be noted that in each picture set, a unique picture is determined by name, and the picture format is jpg. The annotation file corresponding to the picture must be consistent with the picture name, and the file format is xml. Since the picture name in the picture set is unique, deleting or modifying the picture only requires deleting or modifying the picture data with the corresponding name. Similarly, to annotate the picture, you only need to modify the annotation_json field in the data row with the same name.

[0077] Specifically, the system stores the manually annotated images and annotation information in the image collection to the object storage tool MinIO, and generates the image data source table amap_pic_data_source. The image data source table records the image name, width, height, annotation box array, annotation information, etc. The images and annotation information are stored in MinIO to ensure data security and reliability. MinIO provides an efficient storage and access mechanism to facilitate subsequent annotation and model training.

[0078] The embodiment of the present application realizes centralized management of picture sets by storing picture sets in a pre-created sharing group table, making it convenient for users to add, modify or delete pictures and picture annotation information. MinIO is used to store pictures and annotation information, which improves the efficiency and reliability of data management and ensures the security and consistency of data.

[0079] Reference Figure 3, shows a flowchart of the steps of another image annotation method provided in an embodiment of the present application, which method is basically the same as the image annotation method provided in the first embodiment of the present application, except that the method may further include:

[0080] Step 106, obtaining the pictures to be labeled uploaded in batches by the user.

[0081] Step 107 , associating a preset number of images to be annotated with a pre-created task table to obtain a task association table; the task association table includes an image list and a task status.

[0082] In an embodiment of the present application, a user can upload pictures to be annotated to the system through a batch upload function. The system obtains pictures to be annotated uploaded in batches by the user, and associates a preset number of pictures to be annotated with a pre-created task table. The task table can be amap_pic_annotation_task, and a task association table amap_mark_task_relation is obtained. The task association table includes a list of pictures to be annotated and a task status. The task status includes manual annotation, pre-annotation, manual verification and picture annotation. Specifically, the task association table records information such as the task name, associated data source ID, and task status.

[0083] Step 108, sending the task association table to the user; the task association table is used to synchronize the task status, and to allow the user to preview the picture list and manually annotate the pictures to be annotated.

[0084] Specifically, the system sends the task association table to the user for synchronizing the task status, and for the user to preview the image list and manually annotate a small number or a preset number of images to be annotated. The user can view the task status through the front-end interface and perform manual annotation operations. The annotation results are saved in JSON format to the image data source table amap_pic_data_source.

[0085] Step 109 , in response to the task association table determining that the task status is manually labeled, the manually labeled image in the task association table is obtained.

[0086] In an embodiment of the present application, when the task status of the task association table is manually labeled, the system obtains a preset number of manually labeled images in the task association table, and the manually labeled images are used for subsequent model training and pre-labeling.

[0087] Step 101, obtaining manually annotated pictures and pictures to be annotated; wherein the manually annotated pictures and pictures to be annotated are associated with a task association table, and the task association table is used to determine the task status.

[0088] Step 102, in response to the task association table determining that the task state is a pre-labeling state, manually labeled pictures are used for model training to obtain a picture pre-labeling model; the picture pre-labeling model is used to pre-label the picture to be labeled, and output the algorithm pre-labeled picture.

[0089] Step 103: Send the algorithm-pre-annotated picture output by the picture pre-annotation model to the user; the algorithm-pre-annotated picture includes annotation information.

[0090] Step 104 , obtaining the algorithm-pre-annotated image uploaded by the user after secondary annotation and verification based on the annotation information.

[0091] Step 105, manually annotated pictures and algorithm pre-annotated pictures after secondary annotation and verification are used for model training to obtain a picture annotation model, and the picture annotation model is used to annotate the picture set uploaded by the user.

[0092] The above steps 101 to 105 are described in the foregoing description and will not be repeated here.

[0093] Compared with the prior art, the implementation method of the present application, on the basis of achieving the beneficial effects brought by the first implementation method, realizes refined management of tasks through the task table and the task association table. Users can easily view and manage the task status, and through manual labeling, ensure the accuracy and high quality of the labeled data, providing a reliable data basis for subsequent model training and pre-labeling. The task association table is used to synchronize the task status, and users can understand the progress of the task in real time to ensure the timeliness and consistency of the labeled data.

[0094] Further, see Figure 4 , showing Figure 1 A flowchart of step 102 in a picture annotation method is provided. The method is basically the same as the picture annotation method provided in the first embodiment of the present application. Step 102 may include:

[0095] Step 1021 , in response to the task association table determining that the task state is a pre-annotation state, calling a preset image to start a pre-annotation algorithm container.

[0096] Step 1022: Download the manually annotated image to the pre-annotation algorithm container through the object storage tool for model training to obtain the image pre-annotation model.

[0097] Step 1023: Use the image pre-labeling model to pre-label the images to be labeled in the image set, and output the algorithm pre-labeled images.

[0098] In an embodiment of the present application, in response to the task status of the task association table being pre-labeling, that is, when the task status of the task association table is updated to pre-labeling, the system responds and prepares to perform the pre-labeling operation, and calls the preset image to start the pre-labeling algorithm container, wherein the preset image contains the operating environment and dependent libraries required for the pre-labeling algorithm to ensure that the algorithm can run correctly. Specifically, the K8s API can be used to call the preset image to start the pre-labeling algorithm container, and the algorithm corresponding image and the labeled MinIO address, task ID and callback address are passed in. The manually labeled images in the image set are downloaded to the pre-labeling algorithm container through the object storage tool MinIO for model training. The manually labeled images in the image set are downloaded to the pre-labeling algorithm container through the object storage tool MinIO, wherein the object storage tool MinIO is a high-performance object storage that supports high-concurrency read and write operations and is suitable for large-scale data storage and access. MinIO provides an efficient storage and access mechanism to ensure fast download of data and facilitate subsequent labeling and model training.

[0099] Specifically, in the pre-labeling algorithm container, manually labeled pictures are used for model training to obtain a picture pre-labeling model. During the training process, the labeling information of the manually labeled pictures is used as training data to train a model that can pre-label pictures to be labeled. The picture pre-labeling model is used to pre-label the pictures to be labeled, and the algorithm pre-labeling pictures are output. Specifically, the trained picture pre-labeling model is used to pre-label the pictures to be labeled in the picture set. The pre-labeling model analyzes the pictures to be labeled through algorithms to generate labeling information. The algorithm pre-labeling pictures output by the pre-labeling model include the labeling information. The system sends these algorithm pre-labeling pictures to users, and users can view and download these algorithm pre-labeling pictures through the front-end interface.

[0100] It should be noted that the system front end displays image annotations through JSON data, and users can intuitively view and modify annotation results, which improves data quality. MinIO is used to store images and annotation files, which improves storage efficiency and data security. Through asynchronous threads and K8s API calls, resource utilization is optimized, and the system's concurrent processing capabilities are improved. A friendly annotation interface is provided, and users can easily annotate and modify images, which improves user experience. The task progress is updated by percentage, and users can understand the task progress in real time, which enhances user participation and satisfaction. Through the pre-annotation algorithm container, the system can flexibly call different pre-annotation algorithms to adapt to different annotation needs.

[0101] This embodiment uses a small amount of manually annotated pictures for model training, ensures the accuracy and reliability of the pre-annotation model, improves the efficiency and quality of pre-annotation, adopts the picture pre-annotation model to automatically pre-annotate pictures to be annotated, flexibly pre-annotates pictures to be annotated, adapts to different annotation requirements, reduces the workload of manual annotation, and improves annotation efficiency.

[0102] Specifically, after step 1023 uses the picture pre-labeling model to pre-label the pictures to be labeled in the picture set and outputs the algorithm pre-labeled pictures, it can also include: updating the picture data source table in the object storage tool according to the algorithm pre-labeled pictures output by the picture pre-labeling model.

[0103] It should be noted that, in the embodiment of the present application, a picture pre-labeling model is used to pre-label the pictures to be labeled, and the algorithm pre-labeling pictures are output. The trained picture pre-labeling model is used to pre-label the pictures to be labeled in the picture set. The pre-labeling model analyzes the pictures to be labeled through an algorithm to generate labeling information. The algorithm pre-labeling pictures output by the pre-labeling model include the labeling information. The system sends these algorithm pre-labeling pictures to the user, and the user can view and download these algorithm pre-labeling pictures through the front-end interface. According to the algorithm pre-labeling pictures output by the picture pre-labeling model, the picture data source table in the object storage tool is updated.

[0104] Specifically, the system pre-labels images according to the algorithm output by the image pre-labeling model, and updates the image data source table amap_pic_data_source in the object storage tool MinIO. The updated content includes the image name, width, height, label box array, label information, etc. The label information generated by the pre-labeling model is stored in MinIO to ensure the security and reliability of the label information.

[0105] This embodiment implements centralized management of images and annotation information by updating the algorithm pre-annotated images output by the pre-annotation model to the image data source table in the object storage tool, making it convenient for users to add, modify or delete images and image annotation information. Images and annotation information are efficiently managed and accessed through the image data source table, reducing delays in data query and processing.

[0106] Further, see Figure 5 , showing Figure 1 A flowchart of step 105 in a picture annotation method is provided. The method is basically the same as the picture annotation method provided in the first embodiment of the present application. Step 105 may include:

[0107] Step 1051 , in response to the task association table determining that the task status is pre-labeling completed, calling a preset image to start a labeling algorithm container.

[0108] Step 1052 , download the manually annotated pictures and the algorithm pre-annotated pictures after secondary annotation and verification to the annotation algorithm container for model training to obtain the picture annotation model.

[0109] Step 1053: Store the trained image annotation model in an object storage tool.

[0110] It should be noted that in the embodiment of the present application, when the task status of the task association table is updated to pre-labeling completion, the system responds and prepares to perform the image labeling model training operation. The image labeling model training process is similar to the image pre-labeling model training process. Specifically, the system calls the preset image to start the labeling algorithm container. The preset image contains the operating environment and dependent libraries required by the labeling algorithm to ensure that the algorithm can run correctly. The preset image is called by the K8s API to start the labeling algorithm container, and the algorithm corresponding image and the MinIO address, task ID and callback address are passed in. The manually labeled images and the algorithm pre-labeled images after secondary labeling and verification are downloaded to the labeling algorithm container for model training. Through the object storage tool MinIO, the manually labeled images and the algorithm pre-labeled images after secondary labeling and verification are downloaded to the labeling algorithm container. In the labeling algorithm container, the manually labeled images and the algorithm pre-labeled images after secondary labeling and verification are used for model training to obtain the image labeling model. During the training process, high-quality labeled images are used as training data to train a more accurate image labeling model. The trained image labeling model is stored in the object storage tool to ensure the safe storage and fast access of the model.

[0111] In the embodiment of the present application, high-quality annotated images that have been pre-annotated and verified by the user for secondary annotation are used for model training to obtain a high-quality image annotation model, thereby ensuring the accuracy and reliability of the image annotation model and improving the efficiency and quality of image annotation.

[0112] Specifically, the step of using the image annotation model to annotate the image set uploaded by the user may include the following steps:

[0113] First, in response to a user uploading or updating a to-be-annotated picture in the picture set, calling a picture annotation model of the object storage tool;

[0114] Secondly, the picture annotation model is used to annotate the picture to be annotated to obtain annotated pictures and annotation information.

[0115] It should be noted that in the embodiment of the present application, the system responds to the user uploading or updating the pictures to be labeled in the picture set, that is, the user can upload the pictures to be labeled to the system through the batch upload function, and the user can also update the existing pictures to be labeled through the front-end interface. The system will respond and process these update operations, call the picture annotation model in the object storage tool MinIO, and specifically through the interface call, the system pulls the picture annotation model from MinIO and loads it into the memory, preparing to perform the picture annotation operation, and uses the loaded picture annotation model to automatically annotate the pictures to be labeled uploaded or updated by the user. The annotated pictures output by the picture annotation model include annotation information. The system sends these annotated pictures to the user, and the user can view and download these annotated pictures through the front-end interface.

[0116] The embodiment of the present application efficiently calls the image annotation model through the object storage tool to ensure the fast and accurate annotation operation, automatically annotates the images to be annotated uploaded or updated by the user through the image annotation model, and flexibly annotates the images to be annotated through the image annotation model to adapt to different annotation requirements, thereby reducing the workload of manual annotation and improving annotation efficiency.

[0117] Reference Figure 6 , shows a schematic diagram of the structure of a picture annotation device provided in an embodiment of the present application, the device comprising:

[0118] The image acquisition module 201 is used to acquire manually annotated images and images to be annotated; wherein the manually annotated images and images to be annotated are associated with a task association table, and the task association table is used to determine the task status;

[0119] A pre-labeling module 202 is used for, in response to the task association table determining that the task state is a pre-labeling state, using the manually labeled picture to perform model training to obtain a picture pre-labeling model; the picture pre-labeling model is used to pre-label the picture to be labeled, and output the algorithm pre-labeling picture;

[0120] A sending module 203 is used to send the algorithm pre-annotated picture output by the picture pre-annotation model to the user; the algorithm pre-annotated picture includes annotation information;

[0121] The image verification module 204 is used to obtain the algorithm-annotated image uploaded by the user after secondary annotation and verification of the algorithm-annotated image according to the annotation information.

[0122] The labeling module 205 is used to use the manually labeled pictures and the algorithm-pre-labeled pictures after secondary labeling and verification to perform model training to obtain a picture labeling model, and use the picture labeling model to label the picture set uploaded by the user.

[0123] Furthermore, the picture acquisition module 201 includes:

[0124] Get submodules for manually annotating pictures and pictures to be annotated;

[0125] The first storage submodule is used to store manually annotated pictures and pictures to be annotated in a pre-created sharing group table to obtain a picture set; the picture set is used for users to add, modify or delete pictures and their annotated information;

[0126] The generation submodule is used to store the manually annotated pictures and annotation information in the picture set into an object storage tool to generate a picture data source table.

[0127] Furthermore, the device also includes:

[0128] The picture upload module is used to obtain the pictures to be annotated uploaded by users in batches;

[0129] A task association module is used to associate a preset number of to-be-annotated pictures with a pre-created task table to obtain a task association table; the task association table includes a picture list and a task status;

[0130] A user annotation module is used to send the task association table to the user; the task association table is used to synchronize the task status, and to allow the user to preview the picture list and manually annotate the pictures to be annotated;

[0131] The second acquisition module is used to determine that the task status is manually labeled in response to the task association table, and obtain the manually labeled picture in the task association table.

[0132] Furthermore, the pre-marking module 202 includes:

[0133] A first calling submodule is used for calling a preset image to start a pre-annotation algorithm container in response to the task association table determining that the task state is a pre-annotation state;

[0134] A first training submodule is used to download the manually annotated image to a pre-annotation algorithm container through an object storage tool for model training to obtain an image pre-annotation model;

[0135] The pre-labeling submodule is used to pre-label the to-be-labeled pictures in the picture set by using the picture pre-labeling model, and output the algorithm pre-labeled pictures.

[0136] Furthermore, the pre-marking module 202 further includes:

[0137] The data updating submodule is used to pre-annotate pictures according to the algorithm output by the picture pre-annotation model, and to update the picture data source table in the object storage tool.

[0138] Furthermore, the marking module 205 includes:

[0139] A second calling submodule is used for calling a preset image to start a labeling algorithm container in response to the task association table determining that the task state is a pre-labeling end state;

[0140] The second training submodule is used to download the manually annotated pictures and the algorithm-annotated pictures after secondary annotation and verification to the annotation algorithm container for model training to obtain the picture annotation model;

[0141] The second storage submodule is used to store the trained image annotation model in an object storage tool.

[0142] Furthermore, the marking module 205 includes:

[0143] A third calling submodule is used to call the picture annotation model of the object storage tool in response to a user uploading or updating a picture to be annotated in the picture set;

[0144] The annotation submodule is used to annotate the to-be-annotated picture using the picture annotation model to obtain annotated pictures and annotation information.

[0145] The image annotation device provided in the embodiment of the present application obtains manually annotated images and images to be annotated, determines that the task status is a pre-annotation status in response to a task association table, uses manually annotated images to perform model training, obtains an image pre-annotation model, the image pre-annotation model is used to pre-annotate the images to be annotated, and outputs algorithm pre-annotated images, sends the algorithm pre-annotated images output by the image pre-annotation model to the user, obtains the algorithm pre-annotated images uploaded by the user after secondary annotation and verification of the algorithm pre-annotated images according to the annotation information, uses manually annotated images and the algorithm pre-annotated images after secondary annotation and verification to perform model training, obtains the image annotation model, and uses the image annotation model to perform image annotation on the image set uploaded by the user. The embodiment of the present application uses manually annotated pictures as training data to train a model that can pre-annotate pictures to be annotated, obtains algorithm pre-annotated pictures and sends them to users, ensures the accuracy and reliability of the pictures annotated by the pre-annotated model, and the user re-annotates and verifies the algorithm pre-annotated pictures according to the annotation information, trains the model on the annotated pictures that meet the user's requirements, and obtains a picture annotation model that matches the user's annotation needs. High-quality annotated pictures are used as training data to train an accurate picture annotation model, ensuring that the picture annotation model can automatically and efficiently annotate a large number of pictures, thereby improving the annotation efficiency and annotation quality and achieving flexible and efficient image annotation.

[0146] Reference Figure 7 , the present application embodiment also provides an electronic device, such as Figure 7As shown, it includes a processor 301, a communication interface 302, a memory 303 and a communication bus 304, wherein the processor 301, the communication interface 302, and the memory 303 communicate with each other through the communication bus 304.

[0147] A processor 301, a memory 303 for storing processor executable instructions;

[0148] The processor 301 is configured to execute the instructions to implement the following image annotation method:

[0149] Obtaining manually annotated pictures and pictures to be annotated; wherein the manually annotated pictures and pictures to be annotated are associated with a task association table, and the task association table is used to determine the task status;

[0150] In response to the task association table determining that the task state is a pre-annotation state, the manually annotated picture is used to perform model training to obtain a picture pre-annotation model; the picture pre-annotation model is used to pre-annotate the picture to be annotated, and output the algorithm pre-annotated picture;

[0151] Sending the algorithm-pre-annotated picture output by the picture pre-annotation model to the user; the algorithm-pre-annotated picture includes annotation information;

[0152] Obtaining the algorithm-pre-annotated picture uploaded by the user after secondary annotation and verification of the algorithm-pre-annotated picture according to the annotation information;

[0153] The manually annotated pictures and the algorithm-pre-annotated pictures after secondary annotation and verification are used for model training to obtain a picture annotation model, and the picture annotation model is used to annotate the picture set uploaded by the user.

[0154] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0155] The communication interface is used for communication between the above terminal and other devices.

[0156] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0157] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0158] In another embodiment provided in the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the image labeling method described in any of the above embodiments is implemented.

[0159] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0160] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0161] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0162] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A method for labeling an image, characterized in that: The method comprises: Obtaining manually annotated pictures and pictures to be annotated; wherein the manually annotated pictures and pictures to be annotated are associated with a task association table, and the task association table is used to determine the task status; In response to the task association table determining that the task state is a pre-annotation state, the manually annotated picture is used to perform model training to obtain a picture pre-annotation model; the picture pre-annotation model is used to pre-annotate the picture to be annotated, and output the algorithm pre-annotated picture; Sending the algorithm-pre-annotated picture output by the picture pre-annotation model to the user; the algorithm-pre-annotated picture includes annotation information; Obtaining the algorithm-pre-annotated picture uploaded by the user after secondary annotation and verification of the algorithm-pre-annotated picture according to the annotation information; The manually annotated pictures and the algorithm-pre-annotated pictures after secondary annotation and verification are used for model training to obtain a picture annotation model, and the picture annotation model is used to annotate the picture set uploaded by the user.

2. The method according to claim 1, characterized in that The step of obtaining manually annotated images and images to be annotated includes: Obtain manually annotated images and images to be annotated; The manually annotated pictures and the pictures to be annotated are stored in a pre-created sharing group table to obtain a picture set; the picture set is used for users to add, modify or delete pictures and the annotated information of the pictures; The manually annotated pictures and annotation information in the picture set are stored in an object storage tool to generate a picture data source table.

3. The method according to claim 1, characterized in that Before obtaining the manually annotated pictures and the pictures to be annotated, the method further includes: Get the batch of images to be annotated uploaded by users; Associating a preset number of to-be-annotated pictures with a pre-created task table to obtain a task association table; the task association table includes a picture list and a task status; Sending the task association table to the user; the task association table is used to synchronize task status, and allows the user to preview the picture list and manually annotate the pictures to be annotated; In response to the task association table determining that the task status is manually labeled, the manually labeled picture in the task association table is obtained.

4. The method according to claim 1, characterized in that The step of determining that the task state is a pre-annotation state in response to the task association table, using the manually annotated picture to perform model training to obtain a picture pre-annotation model includes: In response to the task association table determining that the task state is a pre-annotated state, calling a preset image to start a pre-annotated algorithm container; Downloading the manually annotated image to a pre-annotation algorithm container through an object storage tool for model training to obtain an image pre-annotation model; The image pre-labeling model is used to pre-label the image to be labeled, and an algorithm pre-labeled image is output.

5. The method according to claim 4, characterized in that After the image pre-labeling model is used to pre-label the image to be labeled and the algorithm pre-labeled image is output, the method further includes: Pre-label the picture according to the algorithm output by the picture pre-labeling model, and update the picture data source table in the object storage tool.

6. The method according to claim 1, characterized in that The method of using the manually annotated pictures and the algorithm-annotated pictures after secondary annotation and verification to perform model training to obtain a picture annotation model includes: In response to the task association table determining that the task state is a pre-marking end state, calling a preset image to start a marking algorithm container; Download the manually annotated images and the algorithm-annotated images after secondary annotation and verification to the annotation algorithm container for model training to obtain an image annotation model; The trained image annotation model is stored in an object storage tool.

7. The method according to claim 6, characterized in that The step of using the picture annotation model to annotate the picture set uploaded by the user includes: In response to a user uploading or updating a to-be-annotated picture in the picture set, calling a picture annotation model of the object storage tool; The image annotation model is used to annotate the image to be annotated, so as to obtain annotated images and annotation information.

8. A picture annotation device, characterized in that: The device comprises: An image acquisition module is used to acquire manually annotated images and images to be annotated; wherein the manually annotated images and images to be annotated are associated with a task association table, and the task association table is used to determine the task status; A pre-labeling module, configured to, in response to the task association table determining that the task state is a pre-labeling state, use the manually labeled picture to perform model training to obtain a picture pre-labeling model; the picture pre-labeling model is used to pre-label the picture to be labeled, and output the algorithm pre-labeling picture; A sending module, used to send the algorithm pre-annotated picture output by the picture pre-annotation model to a user; the algorithm pre-annotated picture includes annotation information; The image verification module is used to obtain the algorithm-annotated image uploaded by the user after secondary annotation and verification of the algorithm-annotated image according to the annotation information; The labeling module is used to use the manually labeled pictures and the algorithm-pre-labeled pictures after secondary labeling and verification to perform model training, obtain a picture labeling model, and use the picture labeling model to label the picture set uploaded by the user.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the instructions to implement the image labeling method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the image labeling method according to any one of claims 1 to 7 is implemented.

Images