A data labeling method and system
By configuring an independent pre-trained annotation model for each attribute and performing online learning and manual correction, the problems of low efficiency and insufficient accuracy in existing data annotation systems are solved, achieving efficient and accurate data annotation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI YUNJI YUEDONG INTELLIGENT TECH DEV CO LTD
- Filing Date
- 2023-04-06
- Publication Date
- 2026-07-21
AI Technical Summary
Existing data annotation systems are inefficient, require a lot of manpower, and AI models cannot guarantee the accuracy of each attribute in multi-attribute annotation tasks. Offline training methods cannot learn new data characteristics in a timely manner, increasing process complexity.
Configure an independent pre-trained annotation model for each attribute, and perform manual correction and online learning when performance is not up to standard. Support online model self-learning and set model priority to improve annotation efficiency.
It improved the labeling accuracy of each attribute, simplified the labeling system process, enhanced labeling efficiency and accuracy, and reduced labor costs.
Smart Images
Figure CN116595314B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of data processing and autonomous driving, specifically to the field of data annotation, and more specifically to a data annotation method and system. Background Technology
[0002] With the rapid development of technology, massive amounts of data have been generated across various fields. Effective utilization of this data can significantly promote technological progress. Data annotation is a common operation in data processing. Data annotation can be understood as processing unprocessed data such as speech, images, text, and video, converting it into machine-readable information. Raw data is generally obtained through data collection; subsequent data annotation is equivalent to processing the data before it is fed into artificial intelligence algorithms and models for use. Currently, the mainstream machine learning approach is supervised deep learning, which has a strong dependence on labeled data. Unlabeled raw data is mostly unstructured and cannot be recognized or learned by machines. Only labeled data, becoming structured data, can be used for algorithm training.
[0003] Common data annotation tasks are completed through annotation systems, which are generally software-developed tools. Annotators manually annotate on user terminals within these systems. Some systems integrate simple AI functions to improve efficiency. While annotation tools exist, they all have limitations. Annotation systems that only allow manual annotation are inefficient and require significant manpower. While systems offering basic AI functions are more efficient than manual systems, they still have two main drawbacks: First, existing platforms require a single AI model per annotation task. Typically, a task involves annotating multiple attributes, and training multiple attributes on the same AI model results in suboptimal accuracy for each attribute. Second, current platforms use offline training and inference for AI models, feeding them back into the annotation system. This increases the complexity of the data annotation process, and offline training cannot learn new data characteristics in a timely manner, hindering both timely improvement in model accuracy and efficiency in practical applications. Summary of the Invention
[0004] Therefore, the purpose of this invention is to overcome the shortcomings of the prior art and provide a new data annotation method and system.
[0005] According to a first aspect of the present invention, a data annotation method is provided, the method comprising: S1, receiving data to be annotated and an annotation task, the annotation task including multiple attributes; S2, selecting a pre-trained annotation model corresponding to each attribute to annotate the data to obtain the attribute annotation result; wherein, each pre-trained annotation model is trained with samples bearing its corresponding attribute annotation labels as input and its corresponding attribute annotation result as output; S3, fusing all attribute annotation results in step S2 to obtain the annotation result of the data for the annotation task.
[0006] Preferably, in step S2, the pre-trained annotation models corresponding to multiple attributes are used to annotate the data according to a preset priority.
[0007] Preferably, the method further includes: S4, when the pre-trained annotation model does not meet the preset performance requirements, manually correcting the attribute annotation results of the pre-trained annotation model on the data to obtain correct attribute annotation results, and using the corrected attribute annotation results for online self-learning until the performance of the pre-trained annotation model reaches the preset performance requirements. In some embodiments of the present invention, manual correction is stopped when the performance of the pre-trained annotation model reaches the preset performance requirements. The preset performance requirements are accuracy thresholds for the annotation model set according to the annotation task requirements corresponding to the data.
[0008] Preferably, step S4 includes: S41, displaying the attribute annotation results of each pre-trained annotation model on the data to the user on the user terminal, the user judging each attribute annotation result in the annotation results, and manually correcting the attributes with annotation errors to obtain the correct attribute annotation results; S42, receiving the manually corrected attribute annotation results; S43, training the corresponding pre-trained annotation model with the manually corrected attribute annotation results received in step S42 until convergence.
[0009] Preferably, each pre-trained annotation model is obtained in the following manner: T1, obtaining the historical annotation dataset corresponding to the annotation task, wherein the historical annotation dataset includes multiple samples, and each sample contains one or more attribute annotation labels corresponding to the annotation task; T2, performing multiple preprocessing operations on the historical annotation dataset to obtain the attribute annotation dataset corresponding to each attribute, wherein each preprocessing operation is performed on one attribute as the object, and other attribute annotation labels in the original dataset are removed to obtain the attribute annotation dataset corresponding to that attribute; T3, training the annotation model corresponding to the annotation task type with the attribute annotation dataset of each attribute in step T2 until convergence to obtain the pre-trained annotation model corresponding to that attribute.
[0010] Preferably, the annotation task type is a text annotation task, an image annotation task, a speech annotation task, a video annotation task, or a point cloud annotation task.
[0011] According to a second aspect of the present invention, a data annotation system is provided, the system comprising: a data receiving module for receiving data to be annotated and an annotation task, the annotation task including multiple attributes; an annotation module including multiple pre-trained annotation models, each pre-trained annotation model being configured to annotate the data for one attribute to obtain an attribute annotation result for the data; wherein each pre-trained annotation model is trained with samples bearing its corresponding attribute annotation labels as input and its corresponding attribute annotation result as output; and an annotation result processing module for merging the different attribute annotation results of the data by each pre-trained annotation model to obtain an annotation result of the data for the annotation task.
[0012] Preferably, the system further includes: a user terminal, used to display the attribute annotation results of each pre-trained annotation model on the data to the user, allowing the user to judge and correct each attribute annotation result in the annotation results; and an annotation correction module, used to manually correct the attribute annotation results of the pre-trained annotation model to obtain correct attribute annotation results when the pre-trained annotation model does not meet the preset performance requirements. In some embodiments of the present invention, each pre-trained annotation module is further configured to: respond to the correction of the annotation correction module and perform online learning using the attribute annotation results corrected by the annotation correction module until the performance of its corresponding pre-trained annotation model reaches the preset performance requirements.
[0013] Compared with existing technologies, the advantages of this invention are as follows: 1. This invention configures an independent and targeted model for each attribute in the annotation task. Compared with all attributes being in the same AI model, there is no interference between the AI models for each attribute, which can maximize the performance and accuracy of the model corresponding to each attribute; and the priority of the model is set according to the attribute features, which can greatly improve the efficiency of attribute annotation. 2. The model configuration for each attribute is in the annotation task, which supports independent self-learning of online models. Compared with exporting the annotation data offline for model training, it greatly simplifies the workflow of the annotation system and improves work efficiency. Attached Figure Description
[0014] The embodiments of the present invention will be further described below with reference to the accompanying drawings, wherein:
[0015] Figure 1 This is a schematic diagram of the data annotation method according to an embodiment of the present invention;
[0016] Figure 2 This is a schematic diagram of the data annotation system framework according to an embodiment of the present invention;
[0017] Figure 3 This is a schematic diagram illustrating the working principle of data annotation in a data annotation system according to an embodiment of the present invention, comprising a data receiving module, a data annotation module, and a user terminal.
[0018] Figure 4 This is a schematic diagram illustrating the working principle of manual correction and online self-learning of the annotation correction module, user terminal, and annotation model in the data annotation module in the data annotation system according to an embodiment of the present invention. Detailed Implementation
[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention is further described in detail below through specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0020] As mentioned in the background section, existing AI-based data annotation solutions have shortcomings and cannot achieve efficient data annotation. Data annotation types include computer vision, speech engineering, natural language understanding, and autonomous driving, among others. The complexity of the annotation tasks varies depending on their attributes. For example, computer vision involves bounding box annotation, keypoint annotation, line segment annotation, semantic segmentation, instance segmentation annotation, OCR annotation, image classification, and video annotation; speech engineering involves ASR speech transcription, speech segmentation, speech cleaning, emotion recognition, speaker recognition, phoneme annotation, and prosody annotation. Specifically, annotation tasks become more detailed and have more diverse attributes. For instance, in autonomous driving object detection applications, it's necessary to annotate 2D bounding boxes, 3D bounding boxes, object types, and object driving directions. Using a single AI model for annotation becomes problematic when there are multiple attributes in the task, as it cannot guarantee the accuracy of each attribute. In other words, using only a simple AI model to annotate all attributes in a task is far from sufficient to guarantee the accuracy of each attribute's annotation.
[0021] To address the aforementioned problems, this invention proposes a novel data annotation scheme. This scheme improves the annotation accuracy of each attribute by configuring a pre-trained AI model for each attribute in the annotation task. Furthermore, when the pre-trained annotation model fails to meet accuracy requirements, manual error correction is performed on the model's attribute annotation results, and the model is trained online to further improve its attribute annotation accuracy. According to one embodiment of the invention, as... Figure 1As shown, a data annotation method of the present invention includes: A1. Receiving data to be annotated and an annotation task, wherein the annotation task includes multiple attributes; A2. Selecting a pre-trained annotation model corresponding to each attribute to annotate the data to obtain the attribute annotation result; wherein each pre-trained annotation model is trained with samples bearing its corresponding attribute annotation labels as input and its corresponding attribute annotation result as output; A3. Determining whether the pre-trained annotation model meets the preset performance requirements; if the pre-trained annotation model does not meet the preset performance requirements, manually correcting the attribute annotation result of the pre-trained annotation model on the data to obtain the correct attribute annotation result, and using the corrected attribute annotation result for online learning until the performance of the pre-trained annotation model reaches the preset performance requirements; A4. Merging all attribute annotation results to obtain the annotation result of the data for the annotation task.
[0022] To better understand the present invention, each step of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.
[0023] In step A1, the data to be labeled and the labeling task are received. The labeling task contains multiple attributes. For those skilled in the art, to execute a data labeling task, it is necessary to obtain the data to be labeled and the labeling task to confirm the specific labeling type and the attributes corresponding to the labeling task. This is knowledge known to those skilled in the art and will not be elaborated here.
[0024] In step A2, for each attribute, a pre-trained annotation model corresponding to that attribute is selected to annotate the data to obtain the attribute annotation result. Each pre-trained annotation model is trained by taking samples with corresponding attribute labels as input and the corresponding attribute annotation result as output. As described in the background section, existing AI training models used in annotation systems are trained using supervised deep learning, and one model corresponds to one annotation task, meaning one model is used to annotate all attributes in the annotation task. It is well known that when one model corresponds to multiple tasks, it can only obtain a local optimum, not a global optimum, meaning it cannot guarantee the best accuracy for each annotation task. Therefore, this invention proposes pre-training an annotation model for each attribute to obtain high-accuracy annotation results for each attribute. According to an embodiment of the present invention, each pre-trained annotation model is obtained as follows: T1, obtaining a historical annotation dataset corresponding to the annotation task, wherein the historical annotation dataset includes multiple samples, each sample containing one or more attribute annotation labels corresponding to the annotation task; T2, performing multiple preprocessing operations on the historical annotation dataset to obtain an attribute annotation dataset corresponding to each attribute, wherein each preprocessing operation focuses on one attribute, removing other attribute annotation labels from the original dataset to obtain the attribute annotation dataset corresponding to that attribute; T3, training the annotation model corresponding to the annotation task type with the attribute annotation dataset of each attribute in step T2 until convergence to obtain the pre-trained annotation model corresponding to that attribute. By pre-training an annotation model in a supervised manner for each attribute, an optimal annotation model corresponding to each attribute can be obtained, with one model corresponding to one attribute, ensuring the accuracy of each attribute.
[0025] In a preferred embodiment of the present invention, the present invention uses pre-trained annotation models corresponding to multiple attributes to annotate the data according to a preset execution priority. Under this mechanism, some data does not need to be annotated with all attributes; only the attributes it contains need to be annotated, which can greatly save computing power and improve annotation efficiency. For example, taking image data in intelligent transportation, including traffic light image data and traffic cone image data, if it involves the annotation of category and direction attributes, without priority, both types of image data need to be annotated with both attributes. However, traffic cones do not have direction attributes, and annotating the direction attribute is a waste of computing power. With priority, assuming that category attributes have higher priority than direction attributes, traffic cones do not involve direction attributes, only category attributes. Therefore, after annotating the category attributes of the traffic cone image data, it is not necessary to annotate the direction attributes, which can save computing power and improve the efficiency of data annotation. Data annotation in other scenarios only requires prioritizing the pre-trained annotation models according to the characteristics of the attributes, which will not be elaborated on here.
[0026] In step A3, it is determined whether the pre-trained annotation model meets the preset performance requirements. If the pre-trained annotation model does not meet the preset performance requirements, the attribute annotation results of the pre-trained annotation model on the data are manually corrected to obtain correct attribute annotation results. The corrected attribute annotation results are then used for online learning until the performance of the pre-trained annotation model reaches the preset performance requirements. As can be seen from step A2, this invention uses offline training to pre-train an annotation model for each attribute and directly uses the pre-trained model for data annotation. It should be noted that the offline data is labeled historical data, and offline training and model application are two independent processes. The accuracy of the annotation model pre-trained on offline data meets the requirements. However, in practical applications, pre-trained annotation models face constantly changing new data, making it impossible to guarantee that they will still fully meet accuracy requirements when faced with new data. This necessitates further optimization training of the model. If offline training is still used during this optimization process, it adds an extra offline training step to the data annotation system, further increasing platform complexity. Therefore, this invention proposes an online training method. During the annotation task, attribute annotation results for models whose attribute annotation accuracy does not meet preset performance requirements (the preset performance requirements are accuracy thresholds set according to the annotation task requirements of the data; different annotation tasks have different requirements and need to be set according to the actual situation, for example, accuracy thresholds can be set to 99%, 99.9%, 99.99%, etc.) are manually corrected online. The manually corrected attribute annotation results are then directly used to perform online self-learning on the pre-trained annotation model until its performance meets the requirements. By using online self-learning, no additional offline training step is needed, making the overall process simpler and ensuring that the pre-trained annotation models for each attribute achieve high accuracy. If all the pre-trained annotation models corresponding to the attributes have met the performance requirements, then there is no need for manual correction. The pre-trained annotation models can be used to automatically annotate the new data, which greatly improves the annotation efficiency.
[0027] In step A4, all attribute annotation results are fused to obtain the annotation results of the data for the annotation task. When all pre-trained annotation models corresponding to all attributes have met the performance requirements, the attribute annotation results of all models are directly fused. When the performance of pre-trained annotation models corresponding to some attributes has not yet met the performance requirements, the attribute annotation results of the pre-trained annotation models that have met the performance requirements and the manually corrected attribute annotation results of the pre-trained annotation models that have not met the performance requirements are fused to obtain the annotation results of the data for the annotation task.
[0028] According to another embodiment of the present invention, the present invention also provides a data annotation system corresponding to the annotation method of the present invention, such as... Figure 2 As shown, the data annotation system of the present invention includes: a data receiving module for receiving data to be annotated and an annotation task, wherein the annotation task includes multiple attributes; an annotation module including multiple pre-trained annotation models (also referred to as AI models in this embodiment), each pre-trained annotation model being configured to annotate the data for one attribute to obtain the attribute annotation result of the data, wherein each pre-trained annotation model is trained with samples bearing its corresponding attribute annotation labels as input and its corresponding attribute annotation result as output; an annotation result processing module for merging the different attribute annotation results of the data by each pre-trained annotation model to obtain the annotation result of the data for the annotation task; a user terminal for displaying the attribute annotation results of the data by each pre-trained annotation model to the user, allowing the user to judge and correct each attribute annotation result in the annotation result; and an annotation correction module for manually correcting the attribute annotation results of the pre-trained annotation models to obtain the correct attribute annotation results when the pre-trained annotation models do not meet the preset performance requirements.
[0029] In the data annotation system of this invention, the data to be annotated received by the data receiving module is a specific object to be annotated, such as an image, video, laser point cloud, speech, text, etc. The annotation task received by the data receiving module is a specific annotation task configuration file, such as... Figure 3 As shown, the annotation task configuration file contains specific configurations for each attribute, indicating what each attribute is and how the corresponding pre-trained model is invoked. For example, attribute 1 has: attribute 1 key (indicating what attribute 1 is), attribute 1 options (indicating the possible values for attribute 1), and the pre-trained model path for attribute 1 (indicating where the data annotation system invokes the pre-trained annotation model corresponding to attribute 1). Similarly, other attributes have similar configuration information to attribute 1, which will not be elaborated here. In an example of image object detection in the field of autonomous driving, when the key is the target type, the corresponding options are the possible values for the target type attribute, such as pedestrians, autonomous vehicles, cars, buses, traffic lights, etc. By obtaining the annotation task configuration file, targeted, pre-trained annotation models can be configured for each attribute in the annotation task for automatic attribute annotation. The UI interface of the user terminal in the annotation system directly displays the attribute annotation results corresponding to the annotated object, displayed as annotated attributes: attribute 1, attribute 2, ... attribute n.
[0030] like Figure 4As shown, after configuring the corresponding AI model for each attribute of the annotation task, each AI model displays the prediction results (attribute annotation results) of the current data on the user terminal. If the AI model for each attribute does not reach the performance threshold (i.e., the preset performance requirement), the annotator needs to submit manual annotation results on the annotation system. That is, the annotator corrects any errors in the prediction results of each AI model and submits the corrected annotation results. After receiving the annotation structure from the annotator, the annotation system starts self-learning for the AI model corresponding to each attribute. That is, the AI model corresponding to attribute 1 learns from the annotation results corresponding to attribute 1 in the annotation data provided by the annotator, the AI model corresponding to attribute 2 learns from the annotation results corresponding to attribute 2 in the annotation data provided by the annotator, and so on, until the AI model corresponding to attribute n learns from the annotation results corresponding to attribute n in the annotation data provided by the annotator. When the accuracy of the models corresponding to all attributes of a annotation task reaches the specified threshold, training stops. These models can be directly used for automatic annotation of all new data without manual intervention, which can greatly reduce labor costs.
[0031] To more intuitively understand the solution of this invention, the following example uses image data (including images, video frames, etc.) from autonomous driving as the data to be labeled to illustrate the implementation process of the solution of this invention. Common attributes corresponding to the labeling tasks of image data include semantic segmentation, entity segmentation, object detection, image classification, key points, and line segments. In this embodiment, the labeling task includes object detection and key points as examples. However, this does not limit the labeling task attributes of image data to only these two. These two are used as examples for illustrative purposes only. The implementation process of other attributes is similar and will not be repeated. For ease of description, object detection is configured as attribute x1 (the key of attribute x1 is the target type, and the option of attribute x1 is vehicle or pedestrian), and key points are configured as attribute x2 (the key of attribute x2 is key point, and the option of attribute x2 is pedestrian).
[0032] When processing the above-mentioned image data using the method of the present invention, the following steps are included:
[0033] First, obtain the image data to be labeled and the corresponding labeling task. The labeling task includes attribute x1 and attribute x2.
[0034] Then, the pre-trained annotation models corresponding to attributes x1 and x2 are selected to perform attribute annotation on the acquired image data to be annotated to obtain attribute annotation results. Among them, the pre-trained annotation model corresponding to attribute x1 is trained with image samples with vehicle and pedestrian labels as input and the attribute annotation results of vehicles and pedestrians as output. The annotation result obtained by the pre-trained annotation model corresponding to attribute x1 after performing attribute annotation on the image data to be annotated is that the image data contains annotations of target types of vehicles and pedestrians, and no annotation is made for other target types. Similarly, the pre-trained annotation model corresponding to attribute x2 is trained with image samples with pedestrian keypoint labels as input and the attribute annotation results of pedestrian keypoints as output. The annotation result obtained by the pre-trained annotation model corresponding to attribute x2 after performing attribute annotation on the image data to be annotated is that the image data contains pedestrian keypoint annotations.
[0035] Secondly, the vehicle and pedestrian annotation results and pedestrian key point annotation results corresponding to the image data are fused to obtain the annotation results of the image data for the annotation task.
[0036] Preferably, when annotating image data, the present invention annotates attributes first, then attributes x1, so that when there is no image of the target type pedestrian in the image data, attribute x2 does not need to be annotated.
[0037] Furthermore, as described in the preceding embodiments, when the pre-trained annotation model corresponding to attribute x1 and / or attribute x2 fails to meet the preset performance requirements, the attribute annotation results of the pre-trained annotation model on the data are manually corrected to obtain correct attribute annotation results. The corrected attribute annotation results are then used for online self-learning until the performance of the pre-trained annotation model reaches the preset performance requirements. During manual correction, the attribute annotation results of the pre-trained annotation model corresponding to attribute x1 and attribute x2 on the image data are displayed to the user on the user terminal. The user judges each attribute annotation result and manually corrects any incorrectly labeled attributes to obtain correct attribute annotation results. Based on the manually corrected attribute annotation results, the corresponding pre-trained annotation model is trained until convergence.
[0038] As described in the above embodiments, compared with the prior art: 1. This invention configures an independent and targeted model for each attribute in the annotation task. Compared with all attributes being in the same AI model, there is no interference between the AI models of each attribute, which can maximize the performance and accuracy of the model corresponding to each attribute; and the priority of the model is set according to the attribute features, which can greatly improve the efficiency of attribute annotation. 2. The model configuration for each attribute is in the annotation task, which supports independent self-learning of the online model. Compared with exporting the annotation data offline for model training, it greatly simplifies the workflow of the annotation system and improves work efficiency.
[0039] It should be noted that although the steps are described in a specific order above, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently, or even in a different order, as long as the required function can be achieved.
[0040] This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.
[0041] Computer-readable storage media can be tangible devices that hold and store instructions for use by an instruction execution device. Computer-readable storage media can be, for example, including but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof.
[0042] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.
Claims
1. A data annotation method for annotating image data in autonomous driving, characterized in that, The method includes: S1. Receive the image data to be labeled and the labeling task, wherein the labeling task contains multiple attributes; S2. Select the pre-trained annotation models corresponding to the plurality of attributes respectively to perform attribute annotation on the image data to obtain attribute annotation results; wherein, each pre-trained annotation model is trained with image samples with their corresponding attribute annotation labels as input and their corresponding attribute annotation results as output; wherein, the pre-trained annotation models corresponding to the plurality of attributes respectively perform attribute annotation on the data according to a preset execution priority; S3. Merge all attribute annotation results from step S2 to obtain the annotation results of the image data for the annotation task; S4. When the pre-trained annotation model does not meet the preset performance requirements, the attribute annotation results of the pre-trained annotation model on the image data are manually corrected to obtain the correct attribute annotation results, and the corrected attribute annotation results are used for online self-learning until the performance of the pre-trained annotation model reaches the preset performance requirements.
2. The method according to claim 1, characterized in that, Step S4 includes: S41. On the user terminal, the attribute annotation results of each pre-trained annotation model on the image data are displayed to the user. The user judges each attribute annotation result in the annotation results and manually corrects the attributes with annotation errors to obtain the correct attribute annotation results. S42. Receive the attribute annotation results after manual correction; S43. Use the manually corrected attribute annotation results received in step S42 to train the corresponding pre-trained annotation model until convergence.
3. The method according to any one of claims 1-2, characterized in that, Each pre-trained labeled model is obtained in the following way: T1. Obtain the historical labeled image dataset corresponding to the labeling task. The historical labeled image dataset includes multiple samples, and each sample contains one or more attribute labeling labels corresponding to the labeling task. T2. Perform multiple preprocessing steps on the historical labeled image dataset to obtain the attribute labeled dataset corresponding to each attribute. Each preprocessing step takes one attribute as the object and removes other attribute labels from the original dataset to obtain the attribute labeled dataset corresponding to that attribute. T3. Train the annotation model corresponding to the annotation task type using the attribute annotation dataset of each attribute in step T2 until convergence to obtain the pre-trained annotation model corresponding to that attribute.
4. A data annotation system based on the method of any one of claims 1-3, used for annotating image data in autonomous driving, characterized in that, The system includes: The data receiving module is used to receive image data to be labeled and labeling tasks, wherein the labeling tasks contain multiple attributes; The annotation module includes multiple pre-trained annotation models, each configured to annotate an attribute of the image data to obtain the attribute annotation result of the image data; wherein each pre-trained annotation model is trained by taking an image sample with its corresponding attribute annotation label as input and its corresponding attribute annotation result as output. The annotation result processing module is used to merge the different attribute annotation results of each pre-trained annotation model for the image data to obtain the annotation result of the image data for the annotation task.
5. The system according to claim 4, characterized in that, The system also includes: The user terminal is used to display the attribute annotation results of each pre-trained annotation model for the image data to the user, so that the user can judge and correct each attribute annotation result in the annotation results; The annotation correction module is used to manually correct the attribute annotation results of the pre-trained annotation model to obtain the correct attribute annotation results when the pre-trained annotation model does not meet the preset performance requirements.
6. The system according to claim 4, characterized in that, Each pre-trained annotation model in the annotation module is further configured to: respond to the correction of the annotation correction module and use the attribute annotation results corrected by the annotation correction module for online learning until the performance of its corresponding pre-trained annotation model reaches the preset performance requirements.
7. A computer-readable storage medium, characterized in that, It stores a computer program that can be executed by a processor to implement the steps of the method according to any one of claims 1-3.
8. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs that, when executed by one or more processors, cause the electronic device to perform the steps of the method as described in any one of claims 1-3.