Data labeling method and device, and storage medium

Abstract

The application relates to the computer technical field, and provides a data labeling method, equipment and storage medium, the method comprises the following steps: in response to an operation instruction of a user, obtaining labeling requirements and to-be-labeled data; according to the labeling requirements, the to-be-labeled data is labeled to obtain labeled data. Through accurate acquisition of the labeling requirements, the operation process of the user when labeling the data is simplified, the effectiveness of the labeled data is improved, and then, large-scale and fine-grained model training is realized, and the user experience is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data annotation method, device and storage medium. Background Technology

[0002] Currently, with the development of computer technology, artificial intelligence has been widely applied in various scenarios. Training AI requires a large amount of labeled data to teach machines to analyze differences and establish relationships between data points. Data labeling is a crucial step in building flexible and high-performing machine learning algorithms, and it has become especially important in the era of big data.

[0003] Because data annotation is a complex, labor-intensive, and time-consuming process, many users rely on various annotation platforms to obtain labeled data. The accuracy of the labeled data produced by these platforms depends on the validity of the material data provided by the user. This requires users to possess specialized knowledge related to data annotation, creating significant obstacles for them and making it difficult to obtain accurate labeled data. Consequently, this results in an excessively high barrier to model training, hindering large-scale and refined model training. Summary of the Invention

[0004] The main purpose of this application is to provide a data annotation method, device, and storage medium, which aims to simplify the user's data annotation process by accurately obtaining annotation requirements, improve the effectiveness of annotated data, and thus achieve large-scale and refined model training, thereby improving the user experience.

[0005] Firstly, this application provides a data annotation method, which includes the following steps:

[0006] Responding to user commands, it retrieves annotation requirements and data to be annotated;

[0007] The data to be labeled is labeled according to the labeling requirements to obtain labeled data.

[0008] Secondly, this application also provides a computer device, which includes a memory and a processor;

[0009] The memory is used to store computer programs;

[0010] The processor is configured to execute the computer program and, when executing the computer program, implement any of the data annotation methods provided in the embodiments of this application.

[0011] Thirdly, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement any of the data annotation methods provided in the embodiments of this application.

[0012] This application provides a data annotation method, device, and storage medium. Responding to user operation commands, this application acquires annotation requirements and data to be annotated; it then annotates the data to be annotated according to the annotation requirements to obtain annotated data. By accurately acquiring the annotation requirements, the user's data annotation process is simplified, the effectiveness of the annotated data is improved, and thus large-scale, refined model training is achieved, enhancing the user experience. Attached Figure Description

[0013] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart illustrating a data annotation method provided in an embodiment of this application;

[0015] Figure 2 This is a schematic diagram of a data annotation method provided in an embodiment of this application;

[0016] Figure 3 This is a flowchart illustrating a data annotation method provided in an embodiment of this application;

[0017] Figure 4 This is a schematic diagram of a data annotation method provided in an embodiment of this application;

[0018] Figure 5 This is a schematic block diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0021] The term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items, as well as all possible combinations, and includes such combinations.

[0022] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0023] Please see Figure 1 , Figure 1 This is a schematic flowchart of a data annotation method provided in an embodiment of this application.

[0024] Currently, artificial intelligence (AI) is widely applied in many scenarios of daily life and industrial production, effectively penetrating fields such as finance, smart cities, manufacturing, transportation, autonomous driving, healthcare, robotics, education, and the internet. However, traditional algorithm production capabilities are at odds with the fragmented and diversified customization needs brought about by these needs. Customization requires specifically trained models, which in turn require training on specific data. This specific data requires labeling the original data, a complex, labor-intensive, and time-consuming process. This has led to a bottleneck in the large-scale and refined commercial application of AI.

[0025] The data annotation method provided in this application includes responding to a user's operation command by obtaining annotation requirements and data to be annotated; and annotating the data to be annotated according to the annotation requirements to obtain annotated data. By accurately obtaining the annotation requirements, the user's operation process for data annotation is simplified, the effectiveness of the annotated data is improved, and thus large-scale, refined model training is achieved, improving the user experience.

[0026] like Figure 1 As shown, the data annotation method includes steps S101 to S102.

[0027] S101. Responding to the user's operation command, obtain the annotation requirements and the data to be annotated.

[0028] The data to be labeled can be raw or unprocessed data, such as images, text, audio, and video. Labeling requirements refer to the user's specifications for the labeled data, such as the object to be labeled and the labeling method; these are not specified here.

[0029] Specifically, to achieve rapid, efficient, and high-quality personalized data annotation, users can issue operation commands through the page. Based on these commands, the annotation requirements are obtained, and the appropriate annotation method can be determined. The data to be annotated is then retrieved based on the user's commands. For example, if a user's annotation requirements include annotating faces in an image using a 2D bounding box, they can upload these requirements and the image to be annotated to the page.

[0030] In some embodiments, the step of obtaining annotation requirements and data to be annotated in response to a user's operation instruction includes: displaying a data acquisition page, the data acquisition page being used by a first user to upload data to be annotated and a first annotation requirement; and obtaining the first annotation requirement and the data to be annotated in response to a creation operation instruction from the first user on the data acquisition page.

[0031] The first user refers to an ordinary user who needs to obtain labeled data, and may be a user without professional knowledge related to data labeling. The first labeling requirement is a description of the first user's need for labeled data, which may be a requirement based on the intended use of the labeled data.

[0032] Specifically, the data acquisition page is displayed, where the first user can upload the data to be labeled and the first labeling requirement based on the data acquisition page. In response to the first user's creation operation command on the data acquisition page, the first labeling requirement and the data to be labeled uploaded by the first user are obtained.

[0033] In some embodiments, before obtaining the first annotation requirement and the data to be annotated, the method further includes: in response to the first user's creation operation instruction on the data acquisition page, determining that the data to be annotated needs to meet a preset quantity standard; if not, prompting the first user on the data acquisition page to re-upload or continue uploading.

[0034] The preset quantity standard can be flexibly set according to actual needs. For example, when the data to be labeled is images, the preset quantity standard can be a limit on the number of images; when the data to be labeled is audio, the preset quantity standard can be a limit on the audio duration. For example, if the number of images in the data to be labeled uploaded by the first user is less than the preset quantity standard, the first user will be prompted to re-upload. By checking the quantity of data to be labeled, submission restrictions are implemented, improving the effectiveness of the data uploaded by users.

[0035] In some embodiments, before displaying the data acquisition page, the method further includes: displaying a labeling task creation page, wherein the labeling task creation page is used for the first user to upload data names, and the data acquisition page is generated based on the first user's operation instruction to create a new data labeling task.

[0036] The annotation task creation page can also be used by the first user to fill in remarks and upload a data cover. The remarks can include other requirements of the first user for the annotated data.

[0037] In some embodiments, the first annotation requirement includes at least a description of the annotation requirement, a usage scenario, and annotation examples.

[0038] Specifically, in order to simplify the user's data annotation process and lower the threshold for data annotation, users can express their annotation needs in a simpler way, that is, by using a simple and concise text description as the annotation requirement description, using labeled samples as annotation examples, and providing the usage scenarios of the model or algorithm.

[0039] For example, if the labeled data is used to train a model or algorithm to identify whether people entering a target area are wearing safety helmets, the first user's labeling requirement description could be "identify whether people entering a target area are wearing safety helmets," the use case could be "construction site," and the labeling example could be an image with a safety helmet highlighted in red. It is clear that the first user does not need to have professional knowledge; they only need to provide basic requirements based on the actual use, which lowers the threshold for data labeling and improves the user experience.

[0040] In some embodiments, the step of obtaining annotation requirements and data to be annotated in response to a user's operation instruction further includes: generating a requirement matching page based on the first annotation requirements, displaying the requirement matching page, wherein the requirement matching page is used to provide the first annotation requirements and the data to be annotated to a second user, and is used for the second user to upload supplementary data; and obtaining the supplementary data in response to a supplementary operation instruction from the second user on the requirement matching page.

[0041] The second user is a user with professional knowledge related to data annotation.

[0042] Supplementary data refers to the actions taken by the second user based on their professional knowledge related to data annotation. This can be supplementary to the annotation requirements, supplementary to the data to be annotated, or other content related to data annotation. It should be understood that the second user's supplementary actions are refinements and optimizations based on the initial annotation requirements and data to be annotated provided by the first user, with the aim of improving the effectiveness of the annotated data.

[0043] Specifically, to improve the effectiveness of labeled data, the system analyzes the first user's uploaded labeling requirements and generates a requirement matching page. This page allows second users to access the first user's uploaded labeling requirements and the data to be labeled. Based on this, second users can combine the labeling tool's algorithms and their professional knowledge of data labeling to formulate supplementary content and then execute corresponding instructions on the requirement matching page.

[0044] In some embodiments, the supplementary data includes a second annotation requirement; the step of obtaining the supplementary data in response to the supplementary operation instruction of the second user on the requirement docking page includes: obtaining the second annotation requirement in response to the second user's editing operation instruction on the annotation requirement document and the annotation requirement type, wherein the annotation requirement document includes at least annotation requirements, annotation targets and labels, special cases and handling solutions.

[0045] The second annotation requirement is a description of the second user's needs for annotated data, which can be formed by combining the first annotation requirement with the algorithm of the annotation tool and the professional knowledge related to data annotation.

[0046] Among them, the annotation requirement document is a document that specifies the specific details of the annotation. The annotation requirement type can be object detection, audio transcription, natural language processing, object tracking, etc.

[0047] Specifically, in response to the second user's editing instructions on the requirement docking page regarding the annotation requirement document and annotation requirement type, the system retrieves the second annotation requirements supplemented by the second user.

[0048] Please see Figure 2 , Figure 2 This is a schematic diagram of a data annotation method provided in an embodiment of this application. The annotation requirements document includes at least annotation requirements, annotation targets and labels, and special cases and handling solutions. The annotation requirements specify the annotation method; for example, in target detection, the annotation box must fit the target, targets with more than 90% occlusion do not need annotation, and targets smaller than a preset area do not need annotation, etc. The annotation targets and labels specify the objects to be annotated; for example, in data annotation for fruit recognition, the annotation target could be a banana. The special cases and handling solutions specify the handling methods for situations that may occur during the annotation process; for example, in target detection, if there is no target object to be annotated in an image file, the handling method could be to skip this image file.

[0049] In some embodiments, the second user may also upload sample images of annotations based on annotation requirements, annotation targets and labels, special cases and specific handling methods, so as to improve the effectiveness of annotation data.

[0050] In some embodiments, the annotation requirement document can also be used by a second user to fill in remarks. The remarks may include other requirements from the second user regarding the annotation data.

[0051] It should be understood that by refining the annotation requirement document and annotation requirement type, the second user can significantly improve the effectiveness of the annotation data. The generation of a second requirement type based on the annotation requirement document and annotation requirement type is used for subsequent annotation data, which is conducive to achieving large-scale and refined model training.

[0052] In some embodiments, the supplementary data further includes newly added data to be labeled; the step of obtaining the supplementary data in response to the supplementary operation instruction from the second user on the demand docking page includes: obtaining the newly added data to be labeled in response to the second user's modification operation instruction on the labeled data status.

[0053] Specifically, to achieve large-scale and refined model training, the second user can obtain the unlabeled data uploaded by the first user, and then add additional unlabeled data based on the first user's labeling requirements, the algorithm of the labeling tool, and their professional knowledge related to data labeling. It should be understood that the newly added unlabeled data will be added to the unlabeled data uploaded by the first user and will participate in subsequent data labeling tasks.

[0054] It should be noted that the second user can download and view the data uploaded by the first user, but cannot modify it. They can only add new data to improve the annotation effect. The second user has permission to delete the newly added data, but not the data uploaded by the first user, thus preventing accidental loss of annotated data.

[0055] In some embodiments, before the second user views the data uploaded by the first user, the request interface page will display a message indicating that the data is pending confirmation; after the second user has viewed the data uploaded by the first user, a message indicating that the data has been confirmed will be displayed. The specific prompting method can be flexibly set according to actual needs and is not limited here.

[0056] S102. Label the data to be labeled according to the labeling requirements to obtain labeled data.

[0057] Specifically, the annotation operation on the data to be annotated according to the annotation requirements can highly satisfy the user's personalized data annotation needs, and the resulting annotated data can meet the user's needs for the training model.

[0058] In some embodiments, please refer to Figure 3 , Figure 3This is a schematic flowchart illustrating a data annotation method provided in an embodiment of this application. The step of annotating the data to be annotated according to the annotation requirements to obtain annotated data includes the following steps:

[0059] S1021. Generate a labeling requirement work order based on the first labeling requirement and the second labeling requirement.

[0060] Among them, the annotation requirement work order is a work order that fully records the annotation requirements.

[0061] Specifically, the first annotation request uploaded by the first user and the second annotation request uploaded by the second user are combined to generate a comprehensive annotation request work order. It should be understood that combining the first and second annotation requests can effectively improve the compatibility between annotation requests and annotation tools, thereby improving annotation results and enabling large-scale, refined model training.

[0062] S1022. Label the data to be labeled according to the labeling requirement work order to obtain the labeled data.

[0063] Specifically, the data to be annotated is annotated according to the comprehensive annotation requirements on the annotation requirement work order, and the annotated data is obtained.

[0064] It should be understood that the first annotation request uploaded by the first user stems from their need for the final annotated data, and may be based on the intended use of the model to be trained. The second annotation request uploaded by the second user, however, arises from their professional considerations, and may be based on the first annotation request and their expertise in the algorithms and data annotation related to the annotation tools. Therefore, an annotation request form combining the first and second annotation requests can provide comprehensive annotation requirements, satisfying both the user's subsequent needs and the annotation tool's need for accurate annotation, thus contributing to the quality of the final annotated data.

[0065] In some embodiments, the step of annotating the data to be annotated according to the annotation requirement work order to obtain the annotated data includes: inputting the annotation requirement work order and the data to be annotated into an annotation tool; parsing the annotation requirement work order through the annotation tool, and performing annotation operations on the target data to be annotated based on the parsing results to obtain the annotated data.

[0066] The annotation tools can be various annotation tools that use different algorithms. First, users can choose according to their annotation needs. Second, users can also choose based on their understanding of each annotation tool. For example, if the current annotation requirement is object detection, and annotation tool A has a better data annotation algorithm for object detection, then users can select annotation tool A on the requirement matching page.

[0067] Specifically, when sending annotation requests to the annotation platform, the annotation request work order information can be automatically processed into a PDF file and input into the annotation tool. All data in the data to be annotated can also be automatically packaged and input into the annotation tool. The annotation tool parses the annotation request work order to obtain comprehensive annotation request results, and performs annotation operations on the target data to be annotated based on these comprehensive results, thus obtaining the annotated data.

[0068] It should be noted that if a second user uploads new data to be labeled through the requirements matching page, the new data uploaded by the second user and the data uploaded by the first user should be packaged together and entered into the labeling tool.

[0069] In some embodiments, the status of the data to be labeled is obtained; and a labeling progress is generated based on the status of the data to be labeled.

[0070] Specifically, please refer to Figure 4 , Figure 4 This is a schematic diagram of a data annotation method provided in an embodiment of this application. After the first user uploads the data to be annotated, the data is uploaded to the requirements matching page, awaiting supplementary operations from the second user. At this time, the annotation progress corresponding to the status of the data to be annotated is in the "requirements confirmed" stage. When the second user completes the supplementary operation and begins annotating the data, the annotation progress corresponding to the status of the data to be annotated is in the "annotation in progress" stage. When the annotation operation is completed, the annotated data is obtained, and the annotation progress corresponding to the status of the data to be annotated is in the "review and acceptance" stage. At this point, the annotated data is reviewed. If the review fails, it returns to the "annotation in progress" stage; if the review passes, the annotation progress corresponding to the status of the data to be annotated is in the "completed" stage.

[0071] In another embodiment, the status of the annotation requirement is obtained; and an annotation progress is generated based on the status of the annotation requirement.

[0072] Specifically, please refer to Figure 4 , Figure 4This is a schematic diagram of a data annotation method provided in an embodiment of this application. After the first user uploads the data to be annotated, the first annotation requirement is uploaded to the requirement matching page, awaiting supplementary operations from the second user. At this time, the annotation progress corresponding to the status of the annotation requirement is in the "requirement confirmation" stage. When the second user completes the supplementary operation and begins the annotation operation on the data to be annotated, the annotation progress corresponding to the status of the annotation requirement is in the "annotation in progress" stage. When the annotation operation is completed, the annotated data is obtained, and the annotation progress corresponding to the status of the annotation requirement is in the "review and acceptance" stage. At this point, the annotated data is reviewed. If the review fails, it returns to the "annotation in progress" stage; if the review passes, the annotation progress corresponding to the status of the annotation requirement is in the "completion" stage.

[0073] It should be understood that generating annotation progress based on the status of the uploaded data or annotation requirements helps the first user clearly understand the annotation progress and improves the user experience.

[0074] In some embodiments, after generating the annotation progress based on the state of the data to be annotated, the method further includes: responding to a submission operation instruction from a first user on the data acquisition page, displaying the annotation progress on an annotation progress page. Specifically, after the first user uploads the data to be annotated, the annotation progress page is displayed to the first user, such as... Figure 4 As shown, the annotation progress is displayed on the annotation progress page, making it easy for the first user to understand the data annotation progress.

[0075] In some embodiments, after the annotation operation is completed, a data details display interface will be shown to the first user. The first user can view and download the annotated data based on the data details display interface, and the downloaded annotated data can be used for model training.

[0076] In some embodiments, the data annotation method provided in any embodiment of this application is applied to an AI training platform. The AI ​​training platform can use different deep learning frameworks for large-scale training, manage and iterate datasets and models, and thus obtain various models. It should be understood that, based on the data annotation method provided in the embodiments of this application, high-quality annotated data that highly meets the user's annotation requirements can be obtained. Using high-quality annotated data in an AI training platform can achieve large-scale and refined model training, meeting fragmented and diversified customization needs.

[0077] For example, the above method can be implemented as a computer program, which can be used in, for example... Figure 5 It runs on the computer device shown.

[0078] Please see Figure 5 , Figure 5This is a schematic block diagram of a computer device provided in an embodiment of this application. The computer device can be a terminal device, such as a mobile phone, tablet computer, or wearable device.

[0079] like Figure 5 As shown, the computer device includes a processor, memory, and network interface connected via a system bus, wherein the memory may include volatile storage media, non-volatile storage media, and internal memory.

[0080] Non-volatile storage media can store operating systems and computer programs. These computer programs include program instructions that, when executed, cause the processor to perform any data labeling method.

[0081] The processor provides computing and control capabilities, supporting the operation of the entire computer device.

[0082] Internal memory provides an environment for the execution of computer programs stored in non-volatile storage media. When these computer programs are executed by a processor, the processor can perform any data annotation method.

[0083] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that the structure of this computer device is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0084] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0085] In some embodiments, the processor is used to run a computer program stored in memory to perform the following steps:

[0086] Responding to user commands, it retrieves annotation requirements and data to be annotated;

[0087] The data to be labeled is labeled according to the labeling requirements to obtain labeled data.

[0088] In some embodiments, the processor is further configured to: acquire the status of the data to be labeled; and generate a labeling progress based on the status of the data to be labeled.

[0089] In some embodiments, the first annotation requirement includes at least a description of the annotation requirement, a usage scenario, and annotation examples.

[0090] In some embodiments, the processor is further configured to: generate a requirement matching page based on the first annotation requirement, display the requirement matching page, the requirement matching page being used to provide the first annotation requirement and the data to be annotated to the second user, and for the second user to upload supplementary data; and in response to a supplementary operation instruction from the second user on the requirement matching page, obtain the supplementary data.

[0091] In some embodiments, the supplementary data includes a second annotation requirement; the processor is further configured to: in response to the second user's editing operation instruction on the annotation requirement document and the annotation requirement type, obtain the second annotation requirement, wherein the annotation requirement document includes at least annotation requirements, annotation targets and labels, special cases and handling solutions.

[0092] In some embodiments, the supplementary data further includes newly added data to be labeled; the processor is further configured to: in response to a modification operation instruction from the second user regarding the labeled data, acquire the newly added data to be labeled.

[0093] In some embodiments, the processor is further configured to: generate a labeling requirement work order based on the first labeling requirement and the second labeling requirement; and label the data to be labeled according to the labeling requirement work order to obtain the labeled data.

[0094] In some embodiments, the processor is further configured to: input the annotation request work order and the data to be annotated into an annotation tool; parse the annotation request work order through the annotation tool, and perform annotation operations on the target data to be annotated based on the parsing results to obtain the annotated data.

[0095] In some embodiments, the processor is further configured to: display the annotation progress on the annotation progress page in response to a submission operation instruction from a first user on the data acquisition page.

[0096] This application also provides a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed, implement any of the data annotation methods provided in this application.

[0097] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiments, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device.

[0098] Furthermore, the computer-readable storage medium may primarily include a program storage area and a data storage area, wherein the program storage area may store the operating system, at least one application required for a function, etc.; and the data storage area may store data created based on the use of blockchain nodes, etc.

[0099] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data annotation method, characterized in that, The method, applied to an AI training platform, includes: In response to a user's operation command, the method retrieves annotation requirements and data to be annotated, including: displaying a data retrieval page, wherein the data retrieval page is used by a first user to upload data to be annotated and a first annotation requirement; in response to a creation operation command from the first user on the data retrieval page, retrieving the first annotation requirement and the data to be annotated; the first annotation requirement includes at least a description of the annotation requirement, a usage scenario, and an annotation example; the method of retrieving annotation requirements and data to be annotated in response to a user's operation command further includes: A requirement matching page is generated and displayed based on the first annotation requirement. This page provides the first annotation requirement and the data to be annotated to the second user, and allows the second user to upload supplementary data. In response to a supplementary operation instruction from the second user on the requirement matching page, the supplementary data is retrieved. The supplementary data includes a second annotation requirement. Retrieving the supplementary data in response to the second user's supplementary operation instruction on the requirement matching page includes: retrieving the second annotation requirement in response to the second user's editing operation instruction on the annotation requirement document and annotation requirement type. The annotation requirement document includes at least annotation requirements, annotation targets and tags, special cases, and handling solutions. The supplementary data also includes newly added data to be annotated. Retrieving the supplementary data in response to the second user's supplementary operation instruction on the requirement matching page includes: retrieving the newly added data to be annotated in response to the second user's modification operation instruction on the annotation data. The process of annotating the data to be annotated according to the annotation requirements to obtain annotated data includes: generating an annotation requirement work order based on the first annotation requirement and the second annotation requirement; annotating the data to be annotated according to the annotation requirement work order to obtain the annotated data; inputting the annotation requirement work order and the data to be annotated into an annotation tool; parsing the annotation requirement work order through the annotation tool, and performing annotation operations on the data to be annotated based on the parsing results to obtain the annotated data.

2. The method according to claim 1, characterized in that, The method further includes: Obtain the status of the data to be labeled; Annotation progress is generated based on the status of the data to be annotated.

3. The method according to claim 2, characterized in that, After generating the annotation progress based on the status of the data to be annotated, the process also includes: In response to the first user's submission instruction on the data acquisition page, the annotation progress is displayed on the annotation progress page.

4. A computer device, characterized in that, The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program and, in executing the computer program, implement the data annotation method as described in any one of claims 1 to 3.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to implement the data annotation method as described in any one of claims 1 to 3.

Images