Image labeling method, system and device, storage medium and program product

Abstract

The embodiment of the invention provides an image annotation method, system and device, a storage medium and a program product. The method comprises the steps that an image labeling request is received, the image labeling request is used for labeling a preset image, and the image labeling request comprises an execution sequence corresponding to a plurality of labeling sub-requests and configuration information corresponding to the labeling sub-requests; according to the configuration information corresponding to each labeling sub-request, labeling sub-tasks corresponding to each labeling sub-request are generated, and each labeling sub-task is stored in a target task queue according to an execution sequence corresponding to each labeling sub-request; and executing each annotation subtask according to the storage sequence of each annotation subtask in the target task queue, so as to carry out annotation processing on the preset image to obtain an annotated image. The method is used for achieving the effect of improving the processing efficiency of image annotation.

Description

Technical Field

[0001] The present application relates to the field of annotation technology, and in particular to an image annotation method, system, device, storage medium, and program product. Background Art

[0002] Image annotation technology has become a key link in building high-quality training data sets. In the field of autonomous driving, it is necessary to annotate road features, traffic lights, vehicles, pedestrians and other elements.

[0003] In related technologies, a labeling request may include multiple sub-requests. After uploading the labeling request to the image labeling system, the image labeling system will determine the sub-task corresponding to each sub-request. Different sub-tasks are independent of each other. After the sub-task corresponding to each sub-request is completed, it is necessary to manually determine the next sub-task to be executed until all sub-tasks are completed. Then, the intermediate labeling images corresponding to each sub-task are manually selected, and the selected intermediate labeling images are integrated to obtain the target labeling image.

[0004] However, the subtasks after splitting cannot be automatically connected in series, requiring a lot of manual operations to select the next subtask to be executed. Manual participation in the connection between each subtask is required, which makes the annotation processing efficiency low. Summary of the Invention

[0005] The embodiments of the present application provide an image annotation method, system, device, storage medium, and program product to improve the processing efficiency of annotation.

[0006] In a first aspect, an embodiment of the present application provides an image annotation method, comprising:

[0007] receiving an image annotation request for annotating a preset image, the image annotation request including an execution order corresponding to a plurality of annotation sub-requests and configuration information corresponding to each annotation sub-request;

[0008] Generate a labeling subtask corresponding to each labeling subrequest according to the configuration information corresponding to each labeling subrequest, and store each labeling subtask into a target task queue according to the execution order corresponding to each labeling subrequest;

[0009] Each annotation subtask is executed according to the storage order of each annotation subtask in the target task queue to perform annotation processing on the preset image to obtain the annotated image.

[0010] In a possible implementation, before storing each labeling subtask into the target task queue according to the execution order corresponding to each labeling subrequest, the method further includes:

[0011] Obtaining at least one preset subtask and the execution order of the at least one preset subtask in the image annotation task, where the preset subtask is a subtask in the image annotation task excluding each annotation subtask;

[0012] Determining the subqueue corresponding to each of the marking subtasks according to the execution order corresponding to each of the marking subrequests;

[0013] Determining a storage position of the subqueue in the target task queue according to an execution order of at least one of the preset subtasks in the image annotation task;

[0014] The marking subtasks are stored in the target task queue according to the storage location.

[0015] In a possible implementation, generating, according to the configuration information corresponding to each of the annotation sub-requests, a labeling sub-task corresponding to each of the annotation sub-requests includes:

[0016] For any marking sub-request, determine the marking object corresponding to the marking sub-request and the marking policy corresponding to the marking object according to the configuration information;

[0017] For any labeling sub-request, a labeling sub-task corresponding to the labeling sub-request is generated according to the labeling object and the labeling strategy corresponding to the labeling object.

[0018] In a possible implementation, for any one of the annotation subtasks, executing each annotation subtask to annotate the preset image to obtain the annotated image includes:

[0019] Obtaining a labeling model corresponding to the labeling subtask from the labeling strategy corresponding to the labeling subtask;

[0020] The annotation subtask is executed by using the annotation model corresponding to the annotation subtask to perform annotation processing on the preset image to obtain the annotated image.

[0021] In one possible implementation, the target task queue is stored in an initial annotation queue; and each annotation subtask is executed in the order in which it is stored in the target task queue to annotate the preset image. Before obtaining the annotated image, the method further includes:

[0022] Obtain the target priority corresponding to the target task queue;

[0023] If it is determined that the storage of the target task queue in the initial marking queue does not meet the target priority requirement, then in the initial marking queue, the storage position of the target task queue is updated to obtain the target marking queue, wherein the target marking queue includes multiple task queues to be executed, and the multiple task queues to be executed include the target task queue.

[0024] In one possible implementation, the method further includes:

[0025] Detecting the verification identifier of each target subtask in the target task queue;

[0026] If the verification flag of the target subtask is a verification exception flag, the target subtask is determined to be an abnormal subtask;

[0027] Exception handling is performed on the abnormal subtask, where the exception handling includes rolling back execution or skipping execution.

[0028] In one possible implementation, the method further includes:

[0029] Receive a task change request corresponding to the target task queue;

[0030] If the task change request is a request to change the priority of the target task queue, the task change request includes the change priority corresponding to the target task queue; and according to the change priority, updating the storage location of the target task queue in the target marking queue;

[0031] If the task change request is a request to change the marking subtask of the target task queue, the task change request includes change information corresponding to the marking subtask; and the marking subtask of the target task queue is updated according to the change information.

[0032] In a second aspect, an embodiment of the present application provides an image annotation system, comprising: a request receiving module, a subtask generating module, a task queue generating module, and an annotation module, wherein:

[0033] The request receiving module is used to receive an image annotation request, wherein the image annotation request is used to annotate a preset image, and the image annotation request includes an execution order corresponding to multiple annotation sub-requests and configuration information corresponding to each annotation sub-request;

[0034] The subtask generating module is used to generate a labeling subtask corresponding to each labeling subrequest according to the configuration information corresponding to each labeling subrequest;

[0035] The task queue generation module is used to store each annotation subtask into the target task queue according to the execution order corresponding to each annotation subrequest;

[0036] The labeling module is used to execute each labeling subtask according to the storage order of each labeling subtask in the target task queue, so as to label the preset image and obtain the labeled image.

[0037] In one possible implementation, the task queue generation module is further configured to:

[0038] Obtaining at least one preset subtask and the execution order of the at least one preset subtask in the image annotation task, where the preset subtask is a subtask in the image annotation task excluding each annotation subtask;

[0039] Determining the subqueue corresponding to each of the marking subtasks according to the execution order corresponding to each of the marking subrequests;

[0040] Determining a storage position of the subqueue in the target task queue according to an execution order of at least one of the preset subtasks in the image annotation task;

[0041] The marking subtasks are stored in the target task queue according to the storage location.

[0042] In a possible implementation, the subtask generation module is specifically configured to:

[0043] For any marking sub-request, determine the marking object corresponding to the marking sub-request and the marking policy corresponding to the marking object according to the configuration information;

[0044] For any labeling sub-request, a labeling sub-task corresponding to the labeling sub-request is generated according to the labeling object and the labeling strategy corresponding to the labeling object.

[0045] In a possible implementation, the marking module 704 is further configured to:

[0046] Obtaining a labeling model corresponding to the labeling subtask from the labeling strategy corresponding to the labeling subtask;

[0047] The annotation subtask is executed by using the annotation model corresponding to the annotation subtask to perform annotation processing on the preset image to obtain the annotated image.

[0048] In a possible implementation, the storage module is used to store the target task queue into the initial marking queue, and the storage module has the function of:

[0049] Obtain the target priority corresponding to the target task queue;

[0050] If it is determined that the storage of the target task queue in the initial marking queue does not meet the target priority requirement, then in the initial marking queue, the storage position of the target task queue is updated to obtain a target marking queue, wherein the target marking queue includes multiple task queues to be executed, and the multiple task queues to be executed include the target task queue.

[0051] In one possible implementation, the detection module is used to:

[0052] Detecting the verification identifier of each target subtask in the target task queue;

[0053] If the verification flag of the target subtask is a verification exception flag, the target subtask is determined to be an abnormal subtask;

[0054] Exception handling is performed on the abnormal subtask, where the exception handling includes rolling back execution or skipping execution.

[0055] In one possible implementation, the change module is used to:

[0056] Receive a task change request corresponding to the target task queue;

[0057] If the task change request is a request to change the priority of the target task queue, the task change request includes the change priority corresponding to the target task queue; and according to the change priority, updating the storage location of the target task queue in the target marking queue;

[0058] If the task change request is a request to change the marking subtask of the target task queue, the task change request includes change information corresponding to the marking subtask; and the marking subtask of the target task queue is updated according to the change information.

[0059] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0060] The memory stores computer-executable instructions;

[0061] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0062] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementation methods of the first aspect.

[0063] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0064] The image annotation method, system, device, storage medium and program product provided by the embodiments of the present application, the image annotation request may include the execution order corresponding to multiple annotation sub-requests, and the configuration information corresponding to each annotation sub-request. According to the configuration information corresponding to each annotation sub-request, the annotation subtask corresponding to each annotation sub-request can be generated, and according to the execution order corresponding to each annotation sub-request, each annotation subtask is stored in the target task queue. Each annotation subtask is executed according to the storage order of each annotation subtask in the target task queue to perform annotation processing on the preset image and obtain a labeled image. The annotation task can be automatically executed through the target task queue to obtain a labeled image, without the need for manual connection between the subtasks, which can improve processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0066] Figure 1 A schematic diagram of an application scenario provided in an embodiment of the present application;

[0067] Figure 2 A flowchart of an image annotation method provided in an embodiment of the present application;

[0068] Figure 3 A schematic diagram of a process for executing a labeling subtask provided in an embodiment of the present application;

[0069] Figure 4 A schematic diagram of an interface for configuring a labeling model provided in an embodiment of the present application;

[0070] Figure 5 A schematic diagram of a process for storing a target task queue provided in an embodiment of the present application;

[0071] Figure 6 A schematic diagram of a process flow for a detection and labeling subtask provided in an embodiment of the present application;

[0072] Figure 7 A schematic diagram of the architecture of an image annotation method provided in an embodiment of the present application;

[0073] Figure 8 A schematic diagram of the architecture of an image annotation system provided in an embodiment of the present application;

[0074] Figure 9A schematic diagram of the architecture of another image annotation system provided in an embodiment of the present application;

[0075] Figure 10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0076] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0077] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0078] As data annotation is applied in more and more scenarios, the scenarios it faces are becoming increasingly complex. For example, in autonomous driving, precise image annotation can help vehicles identify road features, traffic lights, vehicles, pedestrians, and traffic signs, enabling them to make informed driving decisions. In security monitoring, it can be used to annotate objects for inspection and behavior classification. In medical imaging, it can be used to annotate features such as inspection areas, lesions, or tumors within an image.

[0079] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of this application. Figure 1 , user device 102 and server 102. An image annotation system may be installed in user device 101, a user may generate an image annotation request through user device 101, and server 102 may receive the image annotation request and annotate a preset image according to the image annotation request.

[0080] In related technologies, an image annotation request may include multiple sub-requests. After uploading the annotation request to the image annotation system, the image annotation system will determine the sub-task corresponding to each sub-request. Different sub-tasks are independent of each other. After the sub-task corresponding to each sub-request is completed, it is necessary to manually determine the next sub-task to be executed until all sub-tasks are completed. Then, the intermediate annotation images corresponding to each sub-task are manually selected, and the selected intermediate annotation images are integrated to obtain the target annotation image.

[0081] The image annotation method provided by the embodiment of the present application, the image annotation request may include the execution order corresponding to multiple annotation sub-requests, and the configuration information corresponding to each annotation sub-request. According to the configuration information corresponding to each annotation sub-request, the annotation subtask corresponding to each annotation sub-request can be generated, and according to the execution order corresponding to each annotation sub-request, each annotation subtask can be stored in the target task queue. Each annotation subtask is executed according to the storage order of each annotation subtask in the target task queue to perform annotation processing on the preset image and obtain a labeled image. The above-mentioned execution process can automatically execute the annotation task through the target task queue to obtain the labeled image, without the need for manual concatenation between the subtasks, which can improve the processing efficiency of image annotation.

[0082] Figure 2 This is a flow chart of an image annotation method provided in an embodiment of the present application. Figure 2 , the method may include:

[0083] S201: Receive an image annotation request.

[0084] Image annotation requests can be used to annotate preset images.

[0085] The preset image may be an image to be annotated specified by the user. The user may upload the preset image together with the image annotation request, or may specify the preset image from a preset storage device.

[0086] The preset storage device can store images for sharing. When image annotation is required, the preset image can be directly obtained from the preset storage device, thereby realizing the flow and sharing of data.

[0087] In some possible embodiments, after a preset image has been annotated once and an annotated image is obtained, the annotated image can be stored in a preset storage device. When the annotated image needs to be annotated again in the future, the annotated image can be determined as the next preset image and annotated again.

[0088] In some possible embodiments, if a previous image annotation request and a corresponding preset image exist for an image annotation request, the annotated image corresponding to the image annotation request is retrieved from a preset storage device. The preset storage device can be directly read and processed for subsequent repeated and similar requests, thereby reducing duplicate annotations and avoiding waste of annotation resources.

[0089] The image annotation request may include the execution order corresponding to multiple annotation sub-requests and the configuration information corresponding to each annotation sub-request.

[0090] The image annotation system includes a graphical user interface, and a user can configure configuration information corresponding to each annotation sub-request on the graphical user interface through a user device, and determine the execution order corresponding to multiple annotation sub-requests.

[0091] After configuring each annotation sub-request, the user can adjust the execution order between the annotation sub-requests by dragging and dropping components in the graphical user interface to determine the execution order corresponding to multiple annotation sub-requests.

[0092] In some possible embodiments, the execution order may include parallel execution and / or serial execution. For example, parallel execution may be used to indicate that at least two annotation sub-requests may annotate a preset image simultaneously, and serial execution may be used to indicate that after one annotation sub-request completes annotation, the next annotation sub-request is executed.

[0093] S202: Generate a labeling subtask corresponding to each labeling subrequest according to configuration information corresponding to each labeling subrequest.

[0094] Each annotation sub-request can annotate at least one annotation object. The configuration information corresponding to the annotation sub-request can include at least one annotation object and the annotation strategy corresponding to each annotation object. The annotation objects can include polylines, traffic lights, passable areas, and turn arrows.

[0095] For any annotation object corresponding to the annotation strategy, the annotation strategy can include a basic setting strategy and an attribute setting strategy. The basic setting strategy is used to indicate the storage field of the category to which the annotation object belongs, and the attribute setting strategy is used to indicate the policy information corresponding to each attribute in the annotation object.

[0096] The basic setting strategy may include at least one of the following: major category name, major category field, and minor category type. The attribute setting strategy may include at least one of the following: attribute name, attribute field, attribute type, initial value, attribute configuration, and attribute custom configuration.

[0097] In some possible embodiments, for any marking sub-request, the marking object corresponding to the marking sub-request and the marking strategy corresponding to the marking object can be determined based on the configuration information; and the marking sub-task corresponding to the marking sub-request can be generated based on the marking object and the marking strategy corresponding to the marking object.

[0098] The labeling subtask may be used to label at least one labeling object corresponding to the labeling subrequest.

[0099] For example, assuming there are 4 labeling sub-requests, namely labeling sub-requests 1-4, labeling sub-task 1 corresponding to labeling sub-request 1, labeling sub-task 2 corresponding to labeling sub-request 2, labeling sub-task 3 corresponding to labeling sub-request 3, and labeling sub-task 4 corresponding to labeling sub-request 4 can be generated.

[0100] S203 : storing each labeling subtask into a target task queue according to the execution order corresponding to each labeling subrequest.

[0101] In some possible embodiments, at least one preset subtask and the execution order of at least one preset subtask in the image annotation task are obtained; the subqueue corresponding to each annotation subtask is determined according to the execution order corresponding to each annotation subrequest; the storage position of the subqueue in the target task queue is determined according to the execution order of at least one preset subtask in the image annotation task; and each annotation subtask is stored in the target task queue according to the storage position.

[0102] When annotating an image, in addition to the preset annotation subtasks, there are also fixed subtasks. The preset subtasks can be subtasks in the image annotation task except for the various annotation subtasks.

[0103] For example, pre-set subtasks can include point cloud pre-check subtasks, single-pass point cloud subtasks, dynamic keyframe subtasks, and multi-pass point cloud subtasks. The point cloud pre-check subtask can be used for data preprocessing and quality control to ensure the compliance of annotation input; the single-pass point cloud subtask can be used to process independent point cloud frames collected in a single pass; the dynamic keyframe subtask can be used to optimize the temporal annotation of dynamic objects to reduce annotation redundancy through keyframes; and the multi-pass point cloud subtask is used for the fusion and global annotation of multiple data collections.

[0104] For example, assume there are four annotation subrequests, namely, annotation subrequests 1-4. Annotation subrequest 1 corresponds to annotation subtask 1, annotation subrequest 2 corresponds to annotation subtask 2, annotation subrequest 3 corresponds to annotation subtask 3, and annotation subrequest 4 corresponds to annotation subtask 4. Assuming the execution order is serial execution of annotation subrequest 1, annotation subrequest 2, annotation subrequest 3, and annotation subrequest 4, then the subqueue is {annotation subtask 1, annotation subtask 2, annotation subtask 3, and annotation subtask 4}. Assuming there are three preset subtasks, namely, preset subtasks 1-3, and assuming that the execution order of at least one preset subtask in the image annotation task is preset subtask 1, preset subtask 2, and preset subtask 3. Assuming that, based on the execution order of at least one preset subtask in the image annotation task, the subqueue's storage position in the target task queue is determined to be between preset subtask 2 and preset subtask 3, then the target task queue can be determined to be {preset subtask 1, preset subtask 2, annotation subtask 1, annotation subtask 2, annotation subtask 3, annotation subtask 4, preset subtask 3}.

[0105] The execution order of at least one preset subtask in the image annotation task may include the execution sequence numbers corresponding to the preset subtasks and the execution sequence numbers corresponding to the subqueues.

[0106] The storage position of the subqueue in the target task queue can be determined according to the execution sequence number corresponding to the subqueue.

[0107] In the present application, the necessary annotation subtasks in each image annotation task can be preset in advance by presetting subtasks. When the user's image annotation request is received, the subqueue of the annotation subtask corresponding to the image annotation request can be determined in a suitable storage location in the image annotation task, and the target task queue can be directly generated. There is no need to generate preset subtasks every time, which can improve the generation efficiency of the target task queue and thereby improve the annotation efficiency of image annotation.

[0108] In some possible embodiments, the target task queue may be stored as a queue template. Next time the same annotation is performed, the user may directly select the corresponding target task queue in the queue template, thereby improving the efficiency of image annotation.

[0109] S204 , executing each labeling subtask according to the storage order of each labeling subtask in the target task queue to label the preset image and obtain a labeled image.

[0110] The target task queue may include multiple sub-queues, and the multiple sub-queues include at least one preset sub-task and various marked sub-tasks.

[0111] The target task queue can be a serial task queue or a parallel task queue, wherein the serial task queue can execute the subtask at the head of the queue to the subtask at the end of the queue in sequence; the parallel task queue can execute the subtask at the head of the queue to the subtask at the end of the queue in sequence, and when executing at least two parallel marking subtasks, two threads can be opened to execute at least two marking subtasks.

[0112] The annotated image may be sent to the user device of the user and stored in a preset storage device.

[0113] The image annotation method provided by the embodiment of the present application can generate the annotation subtask corresponding to each annotation subrequest according to the configuration information corresponding to each annotation subrequest, and store each annotation subtask in the target task queue according to the execution order corresponding to each annotation subrequest. Each annotation subtask is executed according to the storage order of each annotation subtask in the target task queue to perform annotation processing on the preset image to obtain an annotated image. The annotation task can be automatically executed by the target task queue to obtain an annotated image, without the need for manual connection between the subtasks, which can improve processing efficiency. In addition, the preset subtask and the annotation subtask can be automatically connected in series to form a complete image annotation process. When annotating the image, no manual participation is required, which can improve the accuracy of the annotation.

[0114] Each annotation subtask can be executed to annotate the preset image and obtain an annotated image. For any annotation sub-request, the annotation subtask can be executed through the annotation model corresponding to the annotation subtask to achieve the annotation of the image. Figure 3 , further explains the execution process of the generation annotation subtask provided in the embodiment of the present application.

[0115] Figure 3 This is a flow chart of executing a labeling subtask provided in an embodiment of the present application. Figure 3 , the method may include:

[0116] S301: Obtain a labeling model corresponding to the labeling subtask from the labeling strategy corresponding to the labeling subtask.

[0117] An image annotation system can include multiple preset annotation models. For example, a preset model could be a Segment Anything Model (SAM). SAM can achieve fully automatic or semi-automatic image segmentation and annotation without domain-specific training through zero-shot learning and hint engineering.

[0118] When configuring each annotation sub-request, the user can configure the corresponding annotation model for each annotation sub-request. The annotation model can be selected from multiple preset annotation models or customized. For example, the customized annotation model can be the Open Neural Network Exchange (ONNX) model.

[0119] Figure 4 This is a schematic diagram of an interface for configuring a labeling model provided in an embodiment of the present application. Figure 4 The left side of the page contains a navigation bar, which includes My Tasks, Annotation Platform Analysis, and Annotation Management. Annotation Management includes process nodes and process templates. After selecting a process node, you can configure the annotation sub-request. The configuration includes four options: basic information, annotation process, advanced functions, and quality inspection process. The annotation process allows you to configure the configuration information corresponding to the annotation sub-request. The interface for advanced functions of the process node includes automation tool configuration and input and output format verification. The automation tool configuration includes multiple preset annotation models. Users can select the annotation model corresponding to the annotation sub-request from these preset annotation models (preset annotation model 1, preset annotation model 2, and preset annotation model 3 in the figure). The input and output format verification section includes the input format section and the output format section. After entering text in the input format section and the output format section, you can customize the annotation model in the input format section and the output format section instead of selecting a annotation model in the automation tool configuration section.

[0120] In some possible embodiments, the user may configure multiple annotation models, and the multiple annotation models may assist in performing annotation, thereby improving the annotation efficiency and the accuracy of the annotation.

[0121] S302: Execute the annotation subtask by using the annotation model corresponding to the annotation subtask to annotate the preset image and obtain an annotated image.

[0122] The image annotation method provided in the embodiment of the present application can call the annotation model corresponding to the annotation subtask to annotate the image according to the configured annotation model, thereby improving the efficiency of image annotation.

[0123] The task execution module in the image annotation system includes an annotation queue, and the task execution module can be used to execute multiple to-be-executed task queues in the annotation queue.

[0124] After generating the target task queue, the target task queue can be stored in the initial marking queue. Figure 5 The execution process of the storage target task queue of this application is described.

[0125] Figure 5 This is a flowchart of a target task queue storage method provided by the embodiment of the present application. Figure 5 , the method may include:

[0126] S501: Obtain the target priority corresponding to the target task queue.

[0127] An image processing request can include a target priority for the request. This is the user-configured priority for the image processing request. For example, the priority can be selected from low, medium, or high. The target priority is related to the storage position of the target task queue within the annotation queue. If two pending task queues have different priorities, the queue with the higher priority is stored at the front of the queue.

[0128] The user may be prompted to select one between low priority and medium priority, and the high priority may be reserved for subsequent target processing tasks for emergency adjustments.

[0129] S502: If it is determined that the storage of the target task queue in the initial marking queue does not meet the target priority requirement, then in the initial marking queue, the storage position of the target task queue is updated to obtain the target marking queue.

[0130] The target marking queue includes multiple task queues to be executed, and the multiple task queues to be executed include a target task queue.

[0131] For example, the initial marking queue includes task queue 1 and task queue 2, where task queue 1 is medium priority and task queue 2 is low priority. Task queue 1 is at the head of the queue and task queue 2 is at the tail of the queue. Assuming the target task queue is task queue 3, after storing task queue 3 in the initial marking queue, task queue 1 is at the head of the queue, task queue 2 is in the middle of the queue, and task queue 3 is at the tail of the queue. Assuming task queue 3 is low priority, the target priority requirement is met and there is no need to update the marking queue. Assuming task queue 3 is medium priority, the target priority requirement is not met and task queue 3 is adjusted to the middle of the queue and task queue 2 is adjusted to the tail of the queue to obtain the target marking queue.

[0132] When executing each of the to-be-executed task queues of the marked queue in sequence, there may be a task queue that needs to be changed. After the user reconfigures the task queue, a task change request corresponding to the target task queue may be received, and the target task queue may be changed.

[0133] A task change request may be submitted by a user corresponding to the target task queue or by an administrator. The task change request may include at least one of the following: a priority change and a marked subtask change.

[0134] In some possible embodiments, if a target task queue occupies a large amount of memory resources, and the resources provided to the entire image annotation system are limited, the efficiency of the target task queue in performing task annotation is low. In this case, an alarm can be issued to the target task queue. The alarm can indicate that the current resources are limited and the target task queue will occupy a large amount of memory. Based on the alarm, the administrator can determine a task change request and change the priority of the target task queue.

[0135] If the task change request is to request a priority change of the target task queue, the task change request includes the change priority corresponding to the target task queue; according to the change priority, the storage position of the target task queue is updated in the target marking queue.

[0136] If the task change request is to request a change of a marked subtask of the target task queue, the task change request may include change information corresponding to the marked subtask; and the marked subtask of the target task queue is updated according to the change information.

[0137] It is worth noting that after the target task queue is changed, the current version of the target task queue can be modified on the visual interface. For example, the previous version is V1 and the current version is V2. Multiple version information corresponding to each target queue can be recorded.

[0138] The image annotation method provided in the embodiments of this application can automatically allocate and optimize annotation tasks and dynamically adjust the task order based on task priorities when the number of pending annotation tasks is large, thereby improving task processing efficiency. Furthermore, the priority of annotation tasks and annotation subtasks can be changed based on task change requests, thus achieving flexibility in image annotation.

[0139] During the execution of the target task queue, the execution status of each marked subtask can be detected. Figure 6 Provide specific instructions.

[0140] Figure 6 This is a flow chart of a detection and annotation subtask provided in an embodiment of the present application. Figure 6 , the method may include:

[0141] S601: Detect the verification identifier of each target subtask in the target task queue.

[0142] The verification flag of the target subtask can be a verification exception flag or a verification pass flag.

[0143] In some possible embodiments, after the target subtask is executed, the intermediate image labeled by the target subtask can be detected through the detection model to detect whether the intermediate image meets the labeling requirements. If it meets the labeling requirements, the verification mark is updated to the verification pass mark; if it does not meet the labeling requirements, the verification mark is updated to the verification exception mark.

[0144] The detection model can be one of multiple preset models or a custom model.

[0145] In some possible embodiments, after discovering that the target subtask is abnormal, the administrator may change the verification identifier corresponding to the target subtask.

[0146] S602: If the verification flag of the target subtask is a verification exception flag, determine the target subtask as an abnormal subtask.

[0147] S603: Perform exception handling on the abnormal subtask, where the exception handling includes rolling back the execution or skipping the execution.

[0148] If the abnormal subtask is in the unexecuted state, the abnormal subtask can be skipped and the next marked subtask can be executed;

[0149] If the abnormal subtask is in the executed state, the abnormal subtask can be rolled back to execute the abnormal subtask again.

[0150] The image annotation method provided in the embodiment of the present application can detect the verification mark of each subtask. If there is an abnormal subtask, the abnormal subtask can be skipped or rolled back in time, which can improve the stability of the annotation process.

[0151] Figure 7 This is a schematic diagram of the architecture of an image annotation method provided in an embodiment of the present application. Figure 7 , there is a need for image annotation on the user side ( Figure 7 When the user has requirements A, B, and N, the user can configure each requirement on the visual interface of the user terminal and generate an image annotation request corresponding to each requirement (for example, Figure 7 In the image annotation system, the image annotation request A corresponds to the requirement A, the image annotation request B corresponds to the requirement B, and the image annotation request N corresponds to the requirement N. The image annotation system can receive the image annotation request, annotate the image according to the image annotation request, and obtain the annotated image (for example, Figure 7 In the example, the labeled image A corresponds to requirement A, the labeled image B corresponds to requirement B, and the labeled image N corresponds to requirement N), and finally the labeled images are stored in a preset storage device.

[0152] When users configure each requirement on the visual interface, they can manage the process of annotation sub-requests corresponding to the requirement, calculate and arrange the execution order of each annotation sub-request, and set the annotation model corresponding to each annotation sub-request. When users customize the input configuration information, the input configuration is pre-annotated to ensure that the user-entered configuration information matches the image annotation system. In addition, users can dynamically update the task queue corresponding to the image annotation request through task change requests.

[0153] Users simply configure image annotation requests according to their needs through a visual interface to automatically annotate images and obtain annotated images. Within the underlying management of the image annotation system, image annotation methods enable the automatic concatenation of subtasks, the rollback / skipping of subtasks, templated annotation tasks, and the automatic execution of subtasks. Furthermore, through image annotation methods, the image annotation system can provide preset annotation models, allowing each subtask to be independently packaged and run, and automatically merging annotation results through preset subtasks, achieving fully automated annotation.

[0154] Figure 8 This is a schematic diagram of the architecture of an image annotation system provided in an embodiment of the present application. Figure 8 The image annotation system 800 may include a request receiving module 801, a subtask generating module 802, a task queue generating module 803 and an annotation module 804, wherein:

[0155] The request receiving module 801 is used to receive an image annotation request;

[0156] The subtask generating module 802 is used to generate a labeling subtask corresponding to each labeling subrequest according to the configuration information corresponding to each labeling subrequest;

[0157] The task queue generation module 803 is used to store each marking subtask into the target task queue according to the execution order corresponding to each marking subrequest;

[0158] The labeling module 804 is configured to execute each labeling subtask according to the order in which the labeling subtasks are stored in the target task queue, so as to label the preset image and obtain a labeled image.

[0159] In some possible embodiments, the task queue generation module 803 is further configured to:

[0160] Obtaining at least one preset subtask and the execution order of the at least one preset subtask in the image annotation task, where the preset subtask is a subtask in the image annotation task excluding each annotation subtask;

[0161] Determine the subqueue corresponding to each annotation subtask based on the execution order of each annotation subrequest;

[0162] Determining a storage position of the subqueue in the target task queue according to an execution order of at least one preset subtask in the image annotation task;

[0163] Store each labeled subtask into the target task queue according to the storage location.

[0164] In some possible embodiments, the subtask generation module 802 is specifically configured to:

[0165] For any annotation sub-request, determine the annotation object corresponding to the annotation sub-request and the annotation strategy corresponding to the annotation object based on the configuration information;

[0166] For any labeling sub-request, a labeling sub-task corresponding to the labeling sub-request is generated according to the labeling object and the labeling strategy corresponding to the labeling object.

[0167] In some possible embodiments, the annotation module 804 is further configured to:

[0168] Obtain the labeling model corresponding to the labeling subtask from the labeling strategy corresponding to the labeling subtask;

[0169] The annotation subtask is executed by using the annotation model corresponding to the annotation subtask to annotate the preset image and obtain an annotated image.

[0170] Figure 9 This is a schematic diagram of the architecture of another image annotation system provided in an embodiment of the present application. Figure 9 The image annotation system 800 further includes a storage module 805, a detection module 806 and a change module 807.

[0171] In some possible embodiments, the storage module 805 is used to store the target task queue into the initial marking queue, and the storage module 804 is used to:

[0172] Get the target priority corresponding to the target task queue;

[0173] If it is determined that the storage of the target task queue in the initial marking queue does not meet the target priority requirement, the storage position of the target task queue is updated in the initial marking queue to obtain the target marking queue, which includes multiple task queues to be executed, and the multiple task queues to be executed include the target task queue.

[0174] In some possible embodiments, the detection module 806 is configured to:

[0175] Detect the verification identifier of each target subtask in the target task queue;

[0176] If the verification flag of the target subtask is a verification exception flag, the target subtask is determined to be an abnormal subtask;

[0177] Perform exception handling on abnormal subtasks, including rolling back execution or skipping execution.

[0178] In some possible embodiments, the change module 807 is used to:

[0179] Receive the task change request corresponding to the target task queue;

[0180] If the task change request is to request a priority change of the target task queue, the task change request includes the change priority corresponding to the target task queue; according to the change priority, the storage location of the target task queue is updated in the target marking queue;

[0181] If the task change request is to request a change of a marked subtask of the target task queue, the task change request includes change information corresponding to the marked subtask; based on the change information, the marked subtask of the target task queue is updated.

[0182] The image annotation system provided in the embodiment of the present application can implement the technical solution shown in the above method embodiment. Its implementation principles and beneficial effects are similar and will not be repeated here.

[0183] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 10 The electronic device 1000 may include a processor 1001 and a memory 1002. Exemplarily, the processor 1001 and the memory 1002 are interconnected via a bus 1003.

[0184] Memory 1002 stores computer-executable instructions;

[0185] The processor 1001 executes the computer-executable instructions stored in the memory 1002 , so that the processor 1001 performs the image annotation method shown in the above method embodiment.

[0186] Accordingly, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the image annotation method of the above method embodiment.

[0187] Accordingly, an embodiment of the present application may further provide a computer program product, including a computer program, which, when executed by a processor, may implement the image annotation method shown in the above method embodiment.

[0188] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0189] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0190] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0191] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0192] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0193] Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0194] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can be implemented using any method or technology for information storage. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change RAM (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0195] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0196] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. An image annotation method, characterized in that: include: receiving an image annotation request for annotating a preset image, the image annotation request including an execution order corresponding to a plurality of annotation sub-requests and configuration information corresponding to each annotation sub-request; Generate a labeling subtask corresponding to each labeling subrequest according to the configuration information corresponding to each labeling subrequest, and store each labeling subtask into a target task queue according to the execution order corresponding to each labeling subrequest; Each annotation subtask is executed according to the storage order of each annotation subtask in the target task queue to perform annotation processing on the preset image to obtain the annotated image.

2. The method according to claim 1, characterized in that Before storing each labeling subtask into the target task queue according to the execution order corresponding to each labeling subrequest, the method further includes: Obtaining at least one preset subtask and the execution order of the at least one preset subtask in the image annotation task, where the preset subtask is a subtask in the image annotation task excluding each annotation subtask; Determining the subqueue corresponding to each of the marking subtasks according to the execution order corresponding to each of the marking subrequests; Determining a storage position of the subqueue in the target task queue according to an execution order of at least one of the preset subtasks in the image annotation task; The marking subtasks are stored in the target task queue according to the storage location.

3. The method according to claim 1, characterized in that Generating a labeling subtask corresponding to each labeling subrequest according to the configuration information corresponding to each labeling subrequest includes: For any marking sub-request, determine the marking object corresponding to the marking sub-request and the marking policy corresponding to the marking object according to the configuration information; For any labeling sub-request, a labeling sub-task corresponding to the labeling sub-request is generated according to the labeling object and the labeling strategy corresponding to the labeling object.

4. The method according to claim 3, characterized in that For any one of the annotation subtasks, executing each annotation subtask to annotate the preset image to obtain the annotated image includes: Obtaining a labeling model corresponding to the labeling subtask from the labeling strategy corresponding to the labeling subtask; The annotation subtask is executed by using the annotation model corresponding to the annotation subtask to perform annotation processing on the preset image to obtain the annotated image.

5. The method according to any one of claims 1 to 4, characterized in that The target task queue is stored in the initial marking queue; Before performing the labeling subtasks according to the order in which the labeling subtasks are stored in the target task queue to label the preset image and obtain the labeled image, the method further includes: Obtain the target priority corresponding to the target task queue; If it is determined that the storage position of the target task queue in the initial marking queue does not meet the target priority requirement, then in the initial marking queue, the storage position of the target task queue is updated to obtain the target marking queue, wherein the target marking queue includes multiple task queues to be executed, and the multiple task queues to be executed include the target task queue.

6. The method according to claim 1, wherein The method further comprises: Detecting the verification identifier of each target subtask in the target task queue; If the verification flag of the target subtask is a verification exception flag, the target subtask is determined to be an abnormal subtask; Exception handling is performed on the abnormal subtask, where the exception handling includes rolling back execution or skipping execution.

7. The method according to claim 5, characterized in that The method further comprises: Receive a task change request corresponding to the target task queue; If the task change request is a request to change the priority of the target task queue, the task change request includes the change priority corresponding to the target task queue; and according to the change priority, updating the storage location of the target task queue in the target marking queue; If the task change request is a request to change the marking subtask of the target task queue, the task change request includes change information corresponding to the marking subtask; and the marking subtask of the target task queue is updated according to the change information.

8. An image annotation system, characterized in that: The image annotation system includes a request receiving module, a subtask generating module, a task queue generating module and an annotation module, wherein: The request receiving module is used to receive an image annotation request, wherein the image annotation request is used to annotate a preset image, and the image annotation request includes an execution order corresponding to multiple annotation sub-requests and configuration information corresponding to each annotation sub-request; The subtask generating module is used to generate a labeling subtask corresponding to each labeling subrequest according to the configuration information corresponding to each labeling subrequest; The task queue generation module is used to store each annotation subtask into the target task queue according to the execution order corresponding to each annotation subrequest; The labeling module is used to execute each labeling subtask according to the storage order of each labeling subtask in the target task queue, so as to label the preset image and obtain the labeled image.

9. An electronic device, characterized in that: include: memory and processor, The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the image annotation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the image annotation method according to any one of claims 1 to 7.

11. A computer program product, characterized in that The invention comprises a computer program, which implements the image annotation method according to any one of claims 1 to 7 when executed by a processor.

Images