Data labeling method and device, equipment and storage medium
By performing format alignment and label propagation correction on the objects to be labeled when a labeling instruction is detected, the high cost of manual labeling in existing technologies is solved, enabling fast and automated data labeling and improving labeling efficiency and consistency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA MOBILE GROUP ZHEJIANG
- Filing Date
- 2021-05-26
- Publication Date
- 2026-05-12
AI Technical Summary
In existing technologies, sample data requires manual labeling, resulting in high labor costs.
When a labeling instruction is detected, the object to be labeled is determined and its format is aligned. The target labeling model is determined and the label propagation correction is performed using a preset label propagation correction method to obtain the target label.
It enables rapid and automated data annotation, reduces the labor costs of annotation personnel, and improves annotation efficiency and consistency.
Smart Images
Figure CN115409076B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to a data annotation method, apparatus, device and storage medium. Background Technology
[0002] The widespread application of artificial intelligence products requires a large amount of labeled sample data, such as images, text, and audio, to train machine learning models. Then, based on the trained machine learning models, predictions such as the type of data to be processed can be made. However, existing sample data usually requires manual labeling, which is labor-intensive. Summary of the Invention
[0003] The main purpose of this application is to provide a data annotation method, apparatus, device and storage medium, which aims to solve the technical problem that sample data needs to be manually annotated and is costly in the prior art.
[0004] To achieve the above objectives, this application provides a data annotation method, which includes:
[0005] When a labeling instruction is detected, the object to be labeled is determined, and the object to be labeled is formatted and aligned to obtain an aligned object.
[0006] Determine the target annotation model of the alignment object;
[0007] Based on the target annotation model and the preset label propagation correction method, the alignment object is subjected to label propagation correction processing to obtain the target label of the alignment object.
[0008] Optionally, the step of performing label propagation correction processing on the alignment object according to the target annotation model and a preset label propagation correction method to obtain the target label of the alignment object includes:
[0009] Determine the weakly enhanced version of the alignment object and the strongly enhanced version of the alignment object;
[0010] The weakly enhanced version object is labeled based on the target annotation model to obtain the pseudo-label of the aligned object;
[0011] The pseudo-labels are compared with a preset threshold to determine the hot pseudo-labels of the alignment object;
[0012] Based on the preset prediction model, object prediction is performed on the enhanced version object to obtain the object prediction value;
[0013] The target label of the aligned object is calculated by using a preset standard cross-entropy loss matching calculation method, the hot pseudo-label, and the predicted value of the object.
[0014] Optionally, the step of determining the target annotation model of the alignment object includes:
[0015] Determine the target type of the alignment object;
[0016] From the preset object set, determine whether there exists a target object whose type matches the target type;
[0017] If a target object of the same type as the target type exists, then the annotation model of the target object is determined, and the annotation model of the target object is used as the target annotation model.
[0018] Optionally, after the step of determining whether there exists a target object of the same type as the target type in the preset object set, the method includes:
[0019] If no target object of the same type as the target type exists, a set of tags in the form of preset tags will be displayed;
[0020] Receive the initial annotation results after annotating the aligned object portion based on the tag set;
[0021] Based on the initial annotation results and the preset initial annotation model, the target annotation model of the alignment object is determined.
[0022] Optionally, the step of performing label propagation correction processing on the alignment object according to the target annotation model and a preset label propagation correction method to obtain the target label of the alignment object includes:
[0023] Based on the target annotation model and the preset label propagation correction method, the alignment object is subjected to label propagation correction processing to obtain the corrected label;
[0024] Determine whether the approval instruction for the corrected label has been received;
[0025] If the approval instruction for the correction tag is received, the correction tag will be used as the target tag for the alignment object.
[0026] Optionally, after the step of performing label propagation correction processing on the alignment object according to the target annotation model and the preset label propagation correction method to obtain the target label of the alignment object, the method includes:
[0027] Based on the alignment object after the label, the preset training base model is iteratively trained to obtain the training result model.
[0028] Based on the training results model, the predicted label of the data to be predicted is obtained;
[0029] The target labeling model is iteratively updated based on the predicted labels of the data to be predicted.
[0030] Optionally, the object to be labeled includes the image to be processed, the sound data to be processed, and the text data to be processed.
[0031] This application also provides a data annotation apparatus, the data annotation apparatus comprising:
[0032] The first determining module is used to determine the object to be labeled when a labeling instruction is detected, and to perform format alignment processing on the object to be labeled to obtain an aligned object;
[0033] The second determining module is used to determine the target annotation model of the alignment object;
[0034] The correction module is used to perform label propagation correction processing on the alignment object according to the target annotation model and the preset label propagation correction method to obtain the target label of the alignment object.
[0035] Optionally, the correction module includes:
[0036] The first determining unit is used to determine the weakly enhanced version object of the alignment object and the strongly enhanced version object of the alignment object;
[0037] The first acquisition unit is used to annotate the weakly enhanced version object based on the target annotation model to obtain the pseudo-label of the aligned object;
[0038] The second determining unit is used to compare the pseudo-label with a preset threshold to determine the hot pseudo-label of the alignment object;
[0039] An object prediction unit is used to perform object prediction on the enhanced version object according to a preset prediction model to obtain the object prediction value.
[0040] The calculation unit is used to calculate the target label of the aligned object by using a preset standard cross-entropy loss matching calculation method, the hot pseudo-label and the object prediction value.
[0041] Optionally, the second determining module includes:
[0042] The third determining unit is used to determine the target type of the alignment object;
[0043] The fourth determining unit is used to determine from the preset object set whether there is a target object whose type is consistent with the target type;
[0044] The fifth determining unit is used to determine the annotation model of the target object if there is a target object of the same type as the target type, and to use the annotation model of the target object as the target annotation model.
[0045] Optionally, the second determining module further includes:
[0046] The display unit is used to display a set of tags in the form of preset tags if no target object of the same type exists.
[0047] The receiving unit is configured to receive the initial annotation result after the alignment object portion is annotated based on the tag set;
[0048] The sixth determining unit is used to determine the target annotation model of the aligned object based on the initial annotation results and the preset initial annotation model.
[0049] Optionally, the correction module includes:
[0050] The correction unit is used to perform label propagation correction processing on the alignment object according to the target annotation model and the preset label propagation correction method to obtain the corrected label;
[0051] The seventh determining unit is used to determine whether the approval instruction for the correction label has been received;
[0052] The eighth determining unit is used to use the correction label as the target label of the alignment object when it receives the approval instruction of the correction label.
[0053] Optionally, the data annotation device includes:
[0054] The training module is used to iteratively train a preset base model to be trained based on the alignment object after the label, so as to obtain the trained model.
[0055] The prediction module is used to predict the data to be predicted based on the training result model, and obtain the predicted label of the data to be predicted;
[0056] The update module is used to iteratively update the target labeling model based on the predicted labels of the data to be predicted.
[0057] Optionally, the object to be labeled includes the image to be processed, the sound data to be processed, and the text data to be processed.
[0058] This application also provides a data annotation device, which is an entity node device. The data annotation device includes: a memory, a processor, and a program of the data annotation method stored in the memory and executable on the processor. When the program of the data annotation method is executed by the processor, it can implement the steps of the data annotation method as described above.
[0059] This application also provides a storage medium storing a program that implements the above-described data annotation method. When the program is executed by a processor, it implements the steps of the data annotation method as described above.
[0060] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described data annotation method.
[0061] This application provides a data annotation method, apparatus, device, and storage medium. Compared with the prior art, which requires manual annotation of sample data, resulting in high annotation costs, this application, upon detecting an annotation instruction, determines the object to be annotated, performs format alignment processing on the object to be annotated to obtain an aligned object; determines the target annotation model of the aligned object; and performs label propagation correction processing on the aligned object according to the target annotation model and a preset label propagation correction method to obtain the target label of the aligned object. That is, in this application, instead of manually annotating sample data, upon detecting an annotation instruction and determining the object to be annotated (i.e., sample data), it first performs format alignment processing on the object to be annotated to obtain an aligned object, then determines the target annotation model of the aligned object; and performs label propagation correction processing on the aligned object according to the target annotation model and a preset label propagation correction method to obtain the target label of the aligned object. In other words, in this application, the object to be annotated is quickly annotated during the data preparation stage, and label propagation correction processing is also performed on the aligned object, thus ensuring the consistency of rapid annotation, reducing the labor costs of annotators, and improving annotation efficiency. Attached Figure Description
[0062] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0063] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0064] Figure 1 This is a flowchart illustrating the first embodiment of the data annotation method of this application;
[0065] Figure 2 This is a detailed flowchart illustrating step S30 in the data annotation method of this application.
[0066] Figure 3 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of this application;
[0067] Figure 4 This is a schematic diagram illustrating the scenarios involved in the data annotation method of this application.
[0068] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0069] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.
[0070] This application provides a data annotation method. In the first embodiment of the data annotation method of this application, referring to... Figure 1 The data annotation method includes:
[0071] Step S10: When a labeling instruction is detected, the object to be labeled is determined, and the object to be labeled is formatted and aligned to obtain an aligned object.
[0072] Step S20: Determine the target annotation model of the alignment object;
[0073] Step S30: According to the target annotation model and the preset label propagation correction method, the alignment object is subjected to label propagation correction processing to obtain the target label of the alignment object.
[0074] The specific steps are as follows:
[0075] Step S10: When a labeling instruction is detected, the object to be labeled is determined, and the object to be labeled is formatted and aligned to obtain an aligned object.
[0076] In this embodiment, it should be noted that the data annotation method is applied to the data annotation system, and the blood pressure monitoring system is a data annotation device.
[0077] In this embodiment, generally speaking, as Figure 4 As shown, a data annotation system may include the following components:
[0078] (1) Tag Set Management Section: The tag set management section manages the tagged tags and the original data associated with the tagged tags. That is, the tag set management section stores the tagged tags and the original data associated with the tagged tags. In this embodiment, the tag set management section also provides tag import, tag query, and tag maintenance functions. Specifically, after the user-tagged tags are imported, they are stored in the form of tag sets and can be queried by the corresponding type or keywords. In this embodiment, the user-tagged tags can also be displayed in the form of lists or charts, that is, they are visualized in the form of lists or charts, which facilitates subsequent editing of the user-tagged tags, such as deletion, addition, modification, and query.
[0079] (2) Data annotation task management: In this embodiment, the data annotation system provides full-process management of data annotation. Specifically, the full-process management of data annotation includes task creation, task maintenance, role configuration, tool association, data association, tag association process management, result review, etc.
[0080] (3) Annotation Module: In this embodiment, based on the annotation module, a pre-set annotation model can be provided (wherein, the pre-set annotation model can automatically iterate and update itself based on newly added annotation labels), so that users can select to annotate based on the corresponding annotation model. Furthermore, in this embodiment, an automatic annotation process can be triggered to obtain the annotation results of the corresponding sample data or the object to be annotated. In this embodiment, a manual annotation function can also be provided to users. That is, it should be noted that the annotation function of this embodiment is complete and dynamic, and can make full use of the latest data in real time to improve the intelligence of use in various scenarios.
[0081] (4) Labeling result management section: In the embodiment, the labeling result data and labeling file (the object to be labeled) can be published to the dataset management module for management. It should be noted that the labeling result data can also be converted in format in the labeling result management section, and the labeling result data can be returned to the tag set management section or tag set management module for data accumulation or data update. That is, in this embodiment, there is a feedback mechanism.
[0082] In addition, in this embodiment, the annotation results can also be managed by version. That is, as the label information is optimized and accumulated, the annotation results will also be updated, so different versions of the annotation results will be distinguished, which is beneficial to data governance.
[0083] Specifically, in this embodiment, in the data annotation system, if it is detected that a part of the image in the sample (assuming it is an image sample) has been manually annotated, the new labels of the manually annotated image and the historical labels of the image can be stored. After the storage operation, since the new labels of the manually annotated image have been introduced, the training of the corresponding annotation model can be triggered by the preset triggering device of the data annotation system.
[0084] In other words, this embodiment has a preset model update label management process. Based on the preset model update label management process, new labels are introduced to automatically train and label the model, thereby realizing the full utilization of the latest data in real time and improving the intelligence of use in various scenarios.
[0085] In this embodiment, the data annotation system may further include:
[0086] (5) Review section: In this embodiment, through the review section, data labelers can return data that does not meet the requirements for re-labeling, thereby improving the avoidance of labeling errors.
[0087] In this embodiment, the application scenarios for data annotation can be:
[0088] The process begins with the first person creating a labeling task. The data labeling system then uses this task to determine the appropriate labeling tools, dataset (the set of objects to be labeled), and label set for the labeler. After allocation, the data is automatically or manually labeled using the labeling tools, and the labeling results are saved. After saving, a second person can resubmit unsuitable data for relabeling, and publish suitable data to the dataset management system, which then inputs the suitable data into the model training module.
[0089] In other words, in this embodiment, multiple people can collaborate to annotate the objects to be annotated. That is, the data annotation system application can be implemented in a team collaboration manner. In this embodiment, the annotation of the objects to be annotated can also be fully automated without human intervention.
[0090] In this embodiment, multiple objects to be labeled are supported, including but not limited to images, sounds, and text.
[0091] In this embodiment, if the object to be labeled is an image, the data labeling system supports image processing of multiple data formats, that is, the image or video to be labeled can be a mainstream format such as BMP, JPEG, PNG, MP4, etc.
[0092] In this embodiment, if the object to be annotated is text, after the object to be annotated is completed, the annotated object supports output in formats such as XML, VOC, and YOLO. Furthermore, if the object to be annotated is text, topic labeling and classification can be performed, that is, the entire text can be labeled with multiple topic category tags.
[0093] In this embodiment, if the object to be annotated is speech, multiple languages such as Chinese, English, and Japanese are supported when annotating the speech object.
[0094] In this embodiment, the data annotation system is equipped with annotation shortcut keys to improve annotation efficiency. In addition, this embodiment supports the definition of the shortcut key label.
[0095] In this embodiment, the data annotation system supports different colors to distinguish the data during the annotation process, thereby improving accuracy.
[0096] In this embodiment, the data annotation system can encrypt the objects to be annotated with watermarks. That is, after secondary development, the watermarks on the pre-annotated images are encrypted to ensure security.
[0097] In this embodiment, the data annotation system can be configured with annotation modes that can cover images, text, and speech. The basic annotation methods in the annotation modes can be box segmentation, point segmentation, polygon segmentation, semantic segmentation, etc. In addition, in this embodiment, the bounding boxes of targets in the image can also be annotated.
[0098] In this embodiment, the annotation mode can also be an interpolation mode: that is, in this embodiment, by integrating the semi-automatic annotation model based on TensorFlow, a portion of the data in the object to be annotated can be preprocessed, that is, keyframe annotation is performed on the image, and the bounding boxes between keyframes are obtained directly by interpolation. For example, if the annotation results of two keyframes are 1 and 3, the annotation result of the image between the two keyframes can be obtained by interpolation, such as interpolation of 2 or 1.5.
[0099] In this embodiment, the annotation mode can also be the attribute mode: the attribute mode is used to annotate multiple attributes of a single image, such as annotating age, gender, expression, etc. on a face image.
[0100] In this embodiment, the annotation of the object to be annotated is completed based on the data annotation system with the above-mentioned functions. Specifically, the object to be annotated is determined when an annotation instruction is detected.
[0101] The annotation instructions can be triggered manually by clicking or touching on the interface of the data annotation system. In this embodiment, the annotation instructions can also be triggered automatically by a program segment.
[0102] When a labeling instruction is detected, the object to be labeled is extracted from the labeling instruction, and the object to be labeled is formatted and aligned to obtain an aligned object. The object to be labeled includes the image to be processed, the audio data to be processed, and the text data to be processed.
[0103] The specific process of performing format alignment processing on the object to be annotated to obtain an aligned object can be as follows:
[0104] Traverse the entire image, determine the image width, then determine the number of pixels in the image, determine the number of bytes per pixel, and effectively expand the number of bytes so that the image is a multiple of 32 at the bit level.
[0105] For example, for a bitmap image (the object to be labeled), the width is 13, meaning each row has 13 effective pixels. If the image needs to be stored in RGB format (the corresponding RGB storage space is 13 * 3 * 8 = 312, multiplied by 3 because it's converted to red, green, and blue pixels), then based on the raw data portion of the bitmap image, the entire image is traversed row by row starting from the bottom left corner to determine the number of bytes or storage space per pixel. Note that the storage space occupied by each row in a bitmap image is not necessarily the number of effective pixels per row multiplied by 3. For example, in this case, with 13 effective pixels per row, 13 * 3 * 8 = 312, which is not a multiple of 32 (4 bits * 8 = 32 bits). To achieve a multiple of 32 at the bit level (i.e., 4-byte alignment), it needs to be expanded to at least 320 bits, or 40 bytes.
[0106] In this embodiment, after obtaining the alignment object, the alignment object can be preprocessed. This preprocessing can be noise reduction, enhancement, or other preprocessing of the acquired image data. In this embodiment, it should be noted that image preprocessing can be performed before or after generating the alignment object.
[0107] Step S20: Determine the target annotation model of the alignment object;
[0108] In this embodiment, the target annotation model of the alignment object is determined. The target annotation model may refer to a temporarily generated or pre-stored annotation model corresponding to the alignment object. The annotation model corresponding to the alignment object may refer to the online annotation object of the annotation model being of the same type as the alignment object.
[0109] In this embodiment, refer to Figure 2 The step of determining the target annotation model of the alignment object includes:
[0110] Step S21: Determine the target type of the alignment object;
[0111] In this embodiment, since the data annotation system includes multiple annotation models, it is necessary to select the target annotation model from the multiple annotation models. In this embodiment, the target type of the alignment object is first determined, such as the target type of the alignment image.
[0112] Step S22: Determine from the preset object set whether there exists a target object whose type is consistent with the target type;
[0113] Step S23: If there is a target object whose type is consistent with the target type, then determine the annotation model of the target object and use the annotation model of the target object as the target annotation model.
[0114] From a preset object set, it is determined whether there exists a target object whose type matches the target type. If a target object with the same type exists, the annotation model of the target object is determined and used as the target annotation model. For example, user A uploads a medical image and corresponding annotation tags to a data annotation system. The medical image information and the annotation tags are stored and accumulated in the data annotation system as data, and a corresponding annotation model A is generated. Subsequently, another user B uploads a set of similar medical images that need annotation. The data annotation system, through image analysis, can, for example, select annotation model A as the target annotation model.
[0115] Wherein, after the step of determining whether there is a target object of the same type as the target type from the preset object set, the method includes:
[0116] Step S23: If no target object of the same type as the target type exists, then display a set of tags in the form of preset tags;
[0117] Step S24: Receive the initial annotation results after annotating the aligned object portion based on the tag set;
[0118] Step S25: Based on the initial annotation results and the preset initial annotation model, determine the target annotation model of the alignment object.
[0119] In this embodiment, if there is no target object of the same type as the target type, a set of tags in the form of preset tags is displayed, that is, the set of tags in the form of preset tags is displayed visually. Specifically, the preset tag form can be in the form of a chart.
[0120] In this embodiment, the initial annotation results of the aligned object portion after the user annotates it based on the tag set are received; that is, in this embodiment, the initial annotation results manually annotated by the user are received. In the other embodiment, based on the initial annotation results and a preset initial annotation model, the target annotation model of the aligned object is determined; that is, in this embodiment, an automatically added annotation model is used as the target annotation model. In this embodiment, for newly added industry or field data to be annotated, only the first person needs to manually annotate some tag information. Subsequently, the annotation information can be accumulated and learned, and the annotation model can be automatically added. Moreover, the added annotation model can be automatically configured, continuously improving the intelligence of the smart device, promoting the automated evolution of the smart device's annotation capabilities, and facilitating the automated connection of annotation work and subsequent model training, inference, and other stages.
[0121] Step S30: According to the target annotation model and the preset label propagation correction method, the alignment object is subjected to label propagation correction processing to obtain the target label of the alignment object.
[0122] In this embodiment, after obtaining the target annotation model, the alignment object is subjected to label propagation correction processing based on the target annotation model and the preset label propagation correction method to obtain the target label of the alignment object. In this embodiment, since the alignment object is subjected to label propagation correction processing by combining the target annotation model and the preset label propagation correction method, the consistency of prediction can be guaranteed even for minor interferences (there is a correction process), reducing the manpower cost of annotation personnel and improving annotation efficiency.
[0123] The step of performing label propagation correction processing on the alignment object according to the target annotation model and the preset label propagation correction method to obtain the target label of the alignment object includes:
[0124] Step S31: Determine the weakly enhanced version of the alignment object and the strongly enhanced version of the alignment object;
[0125] Step S32: Annotate the weakly enhanced version object based on the target annotation model to obtain the pseudo-label of the aligned object;
[0126] Step S33: Compare the pseudo-label with a preset threshold to determine the hot pseudo-label of the alignment object;
[0127] Overall, in this embodiment, a semi-supervised learning algorithm is adopted. First, by using a preset enhancement method, a weakly enhanced version of the alignment object and a strongly enhanced version of the alignment object are determined. Further, a weakly enhanced version of the unlabeled object, such as an unlabeled image (top), is input into the target labeling model to obtain a label prediction value. When the label prediction value exceeds a threshold, the predicted label corresponding to the label prediction value will be converted into a hot pseudo-label.
[0128] Step S34: Based on the preset prediction model, perform object prediction on the enhanced version object to obtain the object prediction value;
[0129] Step S35: Using a preset standard cross-entropy loss matching calculation method, the target label of the aligned object is calculated from the hot pseudo-label and the predicted value of the object.
[0130] In this embodiment, during the generation of pseudo-labels, a preset prediction model is used to simultaneously predict the strongly enhanced version of the object, obtaining the predicted object value. The preset prediction model, using a preset standard cross-entropy loss matching calculation method, calculates the target label of the aligned object based on the hot pseudo-labels and the predicted object value. That is, the predicted object value and the hot pseudo-labels are input into the cross-entropy calculation module, and the final pseudo-label is calculated using standard cross-entropy loss matching. This annotation method improves the accuracy of the model's image annotation (because a definite final pseudo-label is obtained, rather than using the hot pseudo-label as the target label of the aligned object).
[0131] This application provides a data annotation method, apparatus, device, and storage medium. Compared with the prior art, which requires manual annotation of sample data, resulting in high annotation costs, this application, upon detecting an annotation instruction, determines the object to be annotated, performs format alignment processing on the object to be annotated to obtain an aligned object; determines the target annotation model of the aligned object; and performs label propagation correction processing on the aligned object according to the target annotation model and a preset label propagation correction method to obtain the target label of the aligned object. That is, in this application, instead of manually annotating sample data, upon detecting an annotation instruction and determining the object to be annotated (i.e., sample data), it first performs format alignment processing on the object to be annotated to obtain an aligned object, then determines the target annotation model of the aligned object; and performs label propagation correction processing on the aligned object according to the target annotation model and a preset label propagation correction method to obtain the target label of the aligned object. In other words, in this application, the object to be annotated is quickly annotated during the data preparation stage, and label propagation correction processing is also performed on the aligned object, thus ensuring the consistency of rapid annotation, reducing the labor costs of annotators, and improving annotation efficiency.
[0132] Furthermore, based on the first embodiment of this application, another embodiment of this application is provided. In this embodiment, after the step of performing label propagation correction processing on the alignment object according to the target annotation model and the preset label propagation correction method to obtain the target label of the alignment object, the method includes:
[0133] Step A1: Based on the alignment object after the label, iteratively train the preset base model to be trained to obtain the training result model;
[0134] Step A2: Based on the training result model, predict the data to be predicted to obtain the predicted label of the data to be predicted;
[0135] Step A3: Iteratively update the target labeling model based on the predicted labels of the data to be predicted.
[0136] In this embodiment, the data annotation work mainly serves the prediction model training and inference service in the corresponding intelligent application of the data annotation system. The prediction model training and inference service directly calls the annotation results in the dataset management module as input, and then completes the intelligent application.
[0137] In other words, in this embodiment, since the alignment object after the label can be directly input into the processing module corresponding to the preset training base model to perform iterative training on the preset training base model to obtain the training result model, the automation performance of the process from data labeling to prediction model training and inference is greatly improved.
[0138] In this embodiment, the model based on the training results is used to predict the data to be predicted, thereby obtaining the predicted label for the data to be predicted; the target labeling model is then iteratively updated based on the predicted label for the data to be predicted. That is, this embodiment provides a feedback mechanism, thus improving the accuracy of label prediction.
[0139] In this embodiment, the preset training model is iteratively trained based on the alignment object after the label to obtain a training result model; prediction is performed on the data to be predicted based on the training result model to obtain the predicted label of the data to be predicted; and the target labeling model is iteratively updated according to the predicted label of the data to be predicted. In this embodiment, due to rapid labeling, the training speed of the corresponding prediction model is improved, and due to the label feedback mechanism, the accuracy of label prediction is improved.
[0140] Furthermore, based on the first and second embodiments of this application, another embodiment of this application is provided. In this embodiment, the step of performing label propagation correction processing on the alignment object according to the target annotation model and a preset label propagation correction method to obtain the target label of the alignment object includes:
[0141] Step B1: Based on the target annotation model and the preset label propagation correction method, perform label propagation correction processing on the alignment object to obtain the corrected label;
[0142] Step B2: Determine whether the approval instruction for the corrected label has been received;
[0143] Step B3: If the approval instruction for the correction tag is received, the correction tag is used as the target tag for the alignment object.
[0144] In this embodiment, after performing label propagation correction processing on the alignment object according to the target annotation model and the preset label propagation correction method to obtain the corrected label, it is necessary to determine whether the approval instruction for the corrected label has been received. That is, in this embodiment, there is an approval process. Only when the approval is passed, that is, if the approval instruction for the corrected label is received, is the corrected label used as the target label of the alignment object. If the approval is not passed, that is, if the approval instruction for the corrected label is not received, it is necessary to return to the step of re-performing the label propagation correction processing on the alignment object according to the target annotation model and the preset label propagation correction method until the approval is passed.
[0145] In this embodiment, the alignment object is subjected to label propagation correction processing according to the target annotation model and a preset label propagation correction method to obtain a corrected label; it is then determined whether an approval instruction for the corrected label has been received; if the approval instruction for the corrected label has been received, the corrected label is used as the target label for the alignment object. In this embodiment, human intervention is used to review and control the target label of the alignment object to prevent obviously erroneous target labels from being included.
[0146] Reference Figure 3 , Figure 3 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of this application.
[0147] like Figure 3As shown, the data labeling device may include: a processor 1001, such as a CPU, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to establish communication between the processor 1001 and the memory 1005. The memory 1005 may be a high-speed RAM or a stable, non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.
[0148] Optionally, the data labeling device may also include a rectangular user interface, a network interface, a camera, RF (Radio Frequency) circuitry, sensors, audio circuitry, a WiFi module, etc. The rectangular user interface may include a display screen and an input submodule such as a keyboard. Optionally, the rectangular user interface may also include a standard wired interface or a wireless interface. The network interface may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).
[0149] Those skilled in the art will understand that Figure 3 The data annotation device structure shown does not constitute a limitation on the data annotation device. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0150] like Figure 3 As shown, the memory 1005, serving as a storage medium, may include an operating system, a network communication module, and a data annotation program. The operating system is a program that manages and controls the hardware and software resources of the data annotation device, supporting the operation of the data annotation program and other software and / or programs. The network communication module is used to enable communication between the various components within the memory 1005, as well as communication with other hardware and software in the data annotation system.
[0151] exist Figure 3 In the data annotation device shown, the processor 1001 is used to execute the data annotation program stored in the memory 1005 to implement the steps of the data annotation method described above.
[0152] The specific implementation of the data annotation device in this application is basically the same as the embodiments of the data annotation method described above, and will not be repeated here.
[0153] This application also provides a data annotation apparatus, the data annotation apparatus comprising:
[0154] The first determining module is used to determine the object to be labeled when a labeling instruction is detected, and to perform format alignment processing on the object to be labeled to obtain an aligned object;
[0155] The second determining module is used to determine the target annotation model of the alignment object;
[0156] The correction module is used to perform label propagation correction processing on the alignment object according to the target annotation model and the preset label propagation correction method to obtain the target label of the alignment object.
[0157] Optionally, the correction module includes:
[0158] The first determining unit is used to determine the weakly enhanced version object of the alignment object and the strongly enhanced version object of the alignment object;
[0159] The first acquisition unit is used to annotate the weakly enhanced version object based on the target annotation model to obtain the pseudo-label of the aligned object;
[0160] The second determining unit is used to compare the pseudo-label with a preset threshold to determine the hot pseudo-label of the alignment object;
[0161] An object prediction unit is used to perform object prediction on the enhanced version object according to a preset prediction model to obtain the object prediction value.
[0162] The calculation unit is used to calculate the target label of the aligned object by using a preset standard cross-entropy loss matching calculation method, the hot pseudo-label and the object prediction value.
[0163] Optionally, the second determining module includes:
[0164] The third determining unit is used to determine the target type of the alignment object;
[0165] The fourth determining unit is used to determine from the preset object set whether there is a target object whose type is consistent with the target type;
[0166] The fifth determining unit is used to determine the annotation model of the target object if there is a target object of the same type as the target type, and to use the annotation model of the target object as the target annotation model.
[0167] Optionally, the second determining module further includes:
[0168] The display unit is used to display a set of tags in the form of preset tags if no target object of the same type exists.
[0169] The receiving unit is configured to receive the initial annotation result after the alignment object portion is annotated based on the tag set;
[0170] The sixth determining unit is used to determine the target annotation model of the aligned object based on the initial annotation results and the preset initial annotation model.
[0171] Optionally, the correction module includes:
[0172] The correction unit is used to perform label propagation correction processing on the alignment object according to the target annotation model and the preset label propagation correction method to obtain the corrected label;
[0173] The seventh determining unit is used to determine whether the approval instruction for the correction label has been received;
[0174] The eighth determining unit is used to use the correction label as the target label of the alignment object when it receives the approval instruction of the correction label.
[0175] Optionally, the data annotation device includes:
[0176] The training module is used to iteratively train a preset base model to be trained based on the alignment object after the label, so as to obtain the trained model.
[0177] The prediction module is used to predict the data to be predicted based on the training result model, and obtain the predicted label of the data to be predicted;
[0178] The update module is used to iteratively update the target labeling model based on the predicted labels of the data to be predicted.
[0179] Optionally, the object to be labeled includes the image to be processed, the sound data to be processed, and the text data to be processed.
[0180] The specific implementation of the data annotation device in this application is basically the same as the embodiments of the data annotation method described above, and will not be repeated here.
[0181] This application provides a storage medium that stores one or more programs, which can be executed by one or more processors to implement the steps of the data annotation method described in any of the above claims.
[0182] The specific implementation of the storage medium in this application is basically the same as the various embodiments of the data annotation method described above, and will not be repeated here.
[0183] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described data annotation method.
[0184] The specific implementation of the computer program product of this application is basically the same as the various embodiments of the data annotation method described above, and will not be repeated here.
[0185] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0186] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0187] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.
[0188] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.
Claims
1. A data annotation method, characterized in that, The data annotation method includes: When a labeling instruction is detected, the object to be labeled is determined, and the object to be labeled is formatted and aligned to obtain an aligned object. The object to be labeled includes an image to be processed, audio data to be processed, and text data to be processed. Determine the target annotation model of the alignment object; Based on the target annotation model and the preset label propagation correction method, the alignment object is subjected to label propagation correction processing to obtain the target label of the alignment object; The step of performing label propagation correction processing on the aligned object according to the target annotation model and the preset label propagation correction method to obtain the target label of the aligned object includes: Determine the weakly enhanced version of the alignment object and the strongly enhanced version of the alignment object; The weakly enhanced version object is labeled based on the target annotation model to obtain the pseudo-label of the aligned object; The pseudo-labels are compared with a preset threshold to determine the hot pseudo-labels of the alignment object; Based on the preset prediction model, object prediction is performed on the enhanced version object to obtain the object prediction value; The target label of the aligned object is calculated using a preset standard cross-entropy loss matching calculation method, based on the hot pseudo-label and the predicted value of the object.
2. The data annotation method as described in claim 1, characterized in that, The step of determining the target annotation model of the alignment object includes: Determine the target type of the alignment object; From the preset object set, determine whether there exists a target object whose type matches the target type; If a target object of the same type as the target type exists, then the annotation model of the target object is determined, and the annotation model of the target object is used as the target annotation model.
3. The data annotation method as described in claim 2, characterized in that, After the step of determining whether there exists a target object of the same type as the target type from the preset object set, the method includes: If no target object of the same type as the target type exists, a set of tags in the form of preset tags will be displayed; Receive the initial annotation results after annotating the aligned object portion based on the tag set; Based on the initial annotation results and the preset initial annotation model, the target annotation model of the alignment object is determined.
4. The data annotation method as described in claim 1, characterized in that, The step of performing label propagation correction processing on the aligned object according to the target annotation model and the preset label propagation correction method to obtain the target label of the aligned object includes: Based on the target annotation model and the preset label propagation correction method, the alignment object is subjected to label propagation correction processing to obtain the corrected label; Determine whether the approval instruction for the corrected label has been received; If the approval instruction for the correction tag is received, the correction tag will be used as the target tag for the alignment object.
5. The data annotation method as described in claim 1, characterized in that, After the step of performing label propagation correction processing on the alignment object according to the target annotation model and the preset label propagation correction method to obtain the target label of the alignment object, the method includes: Based on the alignment object after the label, the preset training base model is iteratively trained to obtain the training result model. Based on the training results model, the predicted label of the data to be predicted is obtained; The target labeling model is iteratively updated based on the predicted labels of the data to be predicted.
6. A data annotation device, characterized in that, The data annotation device includes: The first determining module is used to determine the object to be labeled when a labeling instruction is detected, and to perform format alignment processing on the object to be labeled to obtain an aligned object. The object to be labeled includes an image to be processed, sound data to be processed, and text data to be processed. The second determining module is used to determine the target annotation model of the alignment object; The correction module is used to perform label propagation correction processing on the alignment object according to the target annotation model and the preset label propagation correction method to obtain the target label of the alignment object; The correction module includes: The first determining unit is used to determine the weakly enhanced version object of the alignment object and the strongly enhanced version object of the alignment object; The first acquisition unit is used to annotate the weakly enhanced version object based on the target annotation model to obtain the pseudo-label of the aligned object; The second determining unit is used to compare the pseudo-label with a preset threshold to determine the hot pseudo-label of the alignment object; An object prediction unit is used to perform object prediction on the enhanced version object according to a preset prediction model to obtain the object prediction value. The calculation unit is used to calculate the target label of the aligned object by using a preset standard cross-entropy loss matching calculation method, the hot pseudo-label and the object prediction value.
7. A data annotation device, characterized in that, The data annotation device includes: a memory, a processor, and a program stored in the memory for implementing the data annotation method. The memory is used to store programs that implement the data annotation method; The processor is configured to execute a program that implements the data annotation method to implement the steps of the data annotation method as described in any one of claims 1 to 5.
8. A storage medium, characterized in that, The storage medium stores a program that implements the data annotation method, which is executed by a processor to implement the steps of the data annotation method as described in any one of claims 1 to 5.