Data annotation methods, devices and electronic equipment

Abstract

This application discloses a data annotation method, apparatus, and electronic device. The method includes: acquiring an image to be annotated; if it is determined that the image to be annotated contains a subject, acquiring a matting result corresponding to the image to be annotated; acquiring quality evaluation information; and acquiring sample category and annotation information of the image to be annotated based on the quality evaluation information, wherein the quality evaluation information is obtained through manual evaluation of the matting result. This method allows for the acquisition of the matting result after determining that the image to be annotated contains a subject, and the acquisition of the sample category and annotation information of the image to be annotated through quality evaluation information obtained from manual evaluation of the matting result. This achieves semi-automated data annotation with minimal human intervention, thereby improving data annotation efficiency.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a data annotation method, apparatus, and electronic device. Background Technology

[0002] With the continuous development of data annotation, image matting algorithms have become a research hotspot. The improvement of matting algorithm capabilities largely depends on the matting data. In some methods, matting data can be generated through manual annotation. However, this data annotation method still suffers from low efficiency. Summary of the Invention

[0003] In view of the above problems, this application proposes a data annotation method, apparatus, and electronic device to improve the above problems.

[0004] In a first aspect, this application provides a data annotation method applied to an electronic device. The method includes: acquiring an image to be annotated; if it is determined that the image to be annotated contains a subject, acquiring the matting result corresponding to the image to be annotated; acquiring quality evaluation information, and acquiring the sample category and annotation information of the image to be annotated based on the quality evaluation information, wherein the quality evaluation information is obtained by manually evaluating the matting result.

[0005] Obtain the image to be labeled; if it is determined that the image to be labeled contains a subject, obtain the image matting result corresponding to the image to be labeled; obtain the quality evaluation information obtained by manually evaluating the image matting result, and obtain the sample category and labeling information of the image to be labeled based on the quality evaluation information.

[0006] Secondly, this application provides a data annotation device that operates in an electronic device. The device includes: an image acquisition unit for acquiring an image to be annotated; a matting result acquisition unit for acquiring a matting result corresponding to the image to be annotated if it is determined that the image to be annotated contains a subject; and an annotation information generation unit for acquiring quality evaluation information and acquiring the sample category and annotation information of the image to be annotated based on the quality evaluation information, wherein the quality evaluation information is obtained by manually evaluating the matting result.

[0007] Thirdly, this application provides an electronic device including one or more processors and a memory; one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs being configured to perform the methods described above.

[0008] Fourthly, this application provides a computer-readable storage medium storing program code, wherein the above-described method is executed when the program code is run.

[0009] This application provides a data annotation method, apparatus, electronic device, and storage medium. After acquiring an image to be annotated, if it is determined that the image contains a subject, the method obtains the matting result corresponding to the image, acquires quality evaluation information obtained through manual evaluation of the matting result, and obtains the sample category and annotation information of the image based on the quality evaluation information. This method allows for semi-automated data annotation with minimal human intervention, thereby improving data annotation efficiency. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A flowchart of a data annotation method proposed in an embodiment of this application is shown;

[0012] Figure 2 A flowchart of a data annotation method according to another embodiment of this application is shown;

[0013] Figure 3 A flowchart of a data annotation method proposed in a beneficial embodiment of this application is shown;

[0014] Figure 4 A schematic diagram of the business process of a data annotation method proposed in an embodiment of this application is shown;

[0015] Figure 5 A structural block diagram of a data annotation device according to an embodiment of this application is shown;

[0016] Figure 6 A structural block diagram of an electronic device proposed in this application is shown;

[0017] Figure 7 This is a storage unit in this application embodiment for storing or carrying program code that implements the data annotation method according to this application embodiment. Detailed Implementation

[0018] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the appendices. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0019] With the continuous development of artificial intelligence technology, image matting algorithms have become a research hotspot. Image matting algorithms can be used to segment an image into foreground and background parts, and extract the content of the foreground part, which can be called the subject, i.e., the content to be extracted, such as people, animals, etc. To achieve better matting results, a large amount of matting data is needed to train the model in the algorithm. One approach is to generate matting data through manual annotation.

[0020] However, the inventors discovered in their research that data annotation methods still suffer from low efficiency among related approaches.

[0021] Therefore, the inventors have proposed a data annotation method, apparatus, and electronic device as described in this application. After acquiring an image to be annotated, if it is determined that the image contains a subject, the method obtains the matting result corresponding to the image, acquires quality evaluation information obtained through manual evaluation of the matting result, and obtains the sample category and annotation information of the image to be annotated based on the quality evaluation information. This method allows for the acquisition of the matting result after determining that the image contains a subject, and the acquisition of the sample category and annotation information of the image to be annotated through the quality evaluation information obtained through manual evaluation of the matting result. This achieves semi-automated data annotation with minimal human intervention, thereby improving data annotation efficiency.

[0022] The embodiments of this application will now be described in conjunction with the accompanying drawings.

[0023] Please see Figure 1 This application provides a data annotation method applied to electronic devices, the method comprising:

[0024] S110: Obtain the image to be labeled.

[0025] The image to be labeled can refer to the image that needs to be labeled with cutout information.

[0026] One method is to acquire images to be labeled from image acquisition devices (such as cameras, webcams, mobile phones, etc.).

[0027] As another approach, images to be labeled can be obtained from the internet.

[0028] S120: If it is determined that the image to be labeled contains a subject, obtain the image cutout result corresponding to the image to be labeled.

[0029] In one approach, if an electronic device determines that the image to be labeled contains a subject, it can input the image to be labeled into a matting model to obtain the matting result.

[0030] The matting model can refer to a model pre-trained using a saliency matting algorithm based on matting data, such as UNET2. The matting result can be a predicted mask image of the image to be labeled. This predicted mask image can be a grayscale image, meaning an image with pixel values ​​ranging from 0 to 255. This grayscale image can represent the pixel value of each pixel in the corresponding second subject image, and distinguish the subject from the background. For example, a pixel value of 0 can represent the background, and other non-zero pixel values ​​can represent parts of the subject in the image to be labeled. The subject in the image to be labeled can refer to the target area that is most visually appealing to the human eye, which typically has a large area within the entire image.

[0031] Optionally, the image to be labeled can be input into an image classification model to obtain a classification result, which can characterize whether the image to be labeled contains a subject.

[0032] The image classification model can refer to a model pre-trained on a corresponding dataset using a subject classification algorithm, such as ResNet101 or VGG16. This dataset can include images labeled as containing a subject and images labeled as not containing a subject.

[0033] As another approach, if it is determined that the image to be labeled contains a subject and the quality evaluation information characterization is manually confirmed to not contain a subject, the image to be labeled is determined to be a negative sample, and the absence of a subject is used as the first labeling information of the image to be labeled; if it is determined that the image to be labeled contains a subject and the quality evaluation information characterization is manually confirmed to contain a subject, the matting result corresponding to the image to be labeled is obtained.

[0034] After obtaining the classification results, the quality assessment information can refer to information on whether the image to be labeled contains a subject, as confirmed manually. Negative samples can refer to images to be labeled that do not contain a subject.

[0035] In the embodiments of this application, there are multiple ways to enable electronic devices to obtain quality evaluation information.

[0036] In one approach, after acquiring the classification results of the image to be labeled, the electronic device can display the image to be labeled, the classification results, and an interactive window or control on its screen. This allows the annotator to intuitively see whether the image to be labeled contains a subject. If it does, the annotator can enter information indicating that the image to be labeled contains a subject, or click a control indicating that the image to be labeled contains a subject, in the interactive window. If it does not contain a subject, the annotator can enter information indicating that the image to be labeled does not contain a subject, or click a control indicating that the image to be labeled does not contain a subject, in the interactive window. This allows the electronic device to obtain quality evaluation information based on the content of the interactive window or the click status of the control.

[0037] As another approach, after acquiring the classification result of the image to be labeled, the electronic device can send the image and the classification result to another electronic device. The screen of the other electronic device can display the image to be labeled, the classification result, and a window or control for interaction with the labeler. This allows the labeler to visually determine whether the image contains a subject. If it does, the labeler can enter information indicating that the image contains a subject (as confirmed by human intervention) in the interactive window, or click a control indicating that the image contains a subject (as confirmed by human intervention). If it does not contain a subject, the labeler can enter information indicating that the image does not contain a subject (as confirmed by human intervention) in the interactive window, or click a control indicating that the image does not contain a subject (as confirmed by human intervention). After acquiring quality evaluation information based on the content of the interactive window or the click status of the control, the other electronic device can send the acquired quality evaluation information to the electronic device that generated the classification result.

[0038] Optionally, the quality evaluation information of the labelers regarding the correctness of the classification results is a subjective evaluation. It mainly depends on whether there is a target area in the labeled image that is most eye-catching, that is, whether there is content that can be seen at first glance. This area usually has a large area in the entire image to be labeled. If it exists, it means that the image to be labeled has been manually confirmed to contain the subject; if it does not exist, it means that the image to be labeled has been manually confirmed not to contain the subject. After the image to be labeled is manually confirmed to contain the subject, the quality evaluation information is then confirmed based on the classification results.

[0039] In this embodiment, directly obtaining the matting results based on the classification results of the electronic device can save labor costs and speed up the annotation efficiency. After obtaining the classification results of the electronic device, obtaining the matting results based on the quality evaluation information of manually confirming whether the classification results are correct can further filter out the images to be annotated that were incorrectly classified by the electronic device as containing a subject (but did not actually contain a subject), thereby reducing unnecessary matting result acquisition operations and saving electronic device resources.

[0040] S130: Obtain quality evaluation information, and obtain the sample category and annotation information of the image to be annotated based on the quality evaluation information. The quality evaluation information is obtained by manually evaluating the image matting results.

[0041] Among them, after obtaining the image cutout result, the quality evaluation information can refer to the information on whether the image cutout quality, as confirmed by human verification, meets the target conditions.

[0042] In the embodiments of this application, there are multiple ways to enable electronic devices to obtain quality evaluation information.

[0043] In one approach, after acquiring the matting result of the image to be annotated, the electronic device can display the image to be annotated, the matting result, a pseudo-color image, and an interactive window or control on its screen. The pseudo-color image can be an image obtained by overlaying the image to be annotated with the matting result, allowing the annotator to visually assess whether the matting quality completely matches the main outline of the image to be annotated. If completely matched, the annotator can input information indicating that the manually confirmed matting quality meets the target conditions in the interactive window, or click the control indicating that the manually confirmed matting quality meets the target conditions. If not completely matched, the annotator can input information indicating that the manually confirmed matting quality does not meet the target conditions in the interactive window, or click the control indicating that the manually confirmed matting quality does not meet the target conditions. This allows the electronic device to obtain quality evaluation information based on the content of the interactive window or the click status of the control.

[0044] As another approach, after acquiring the matting result of the image to be annotated, the electronic device can send the image to be annotated, the matting result, and the pseudo-color image to another electronic device. The screen of the other electronic device can display the image to be annotated, the matting result, the pseudo-color image, and a window or control for interaction with the annotator. This allows the annotator to visually assess whether the matting quality completely overlaps with the main outline of the image to be annotated based on the pseudo-color image. If they completely overlap, the annotator can input information indicating that the matting quality meets the target conditions in the interactive window, or click the control indicating that the matting quality meets the target conditions. If they do not completely overlap, the annotator can input information indicating that the matting quality does not meet the target conditions in the interactive window, or click the control indicating that the matting quality does not meet the target conditions. The other electronic device, after acquiring quality evaluation information based on the content of the interactive window or the click status of the control, can then send the acquired quality evaluation information to the electronic device that generated the matting result.

[0045] After obtaining the quality evaluation information, the method for obtaining the sample category and annotation information of the image to be labeled based on the quality evaluation information can be as follows: If the quality evaluation information indicates that the matting quality of the matting result confirmed by manual verification meets the target conditions, the image to be labeled is determined as the first positive sample, the image containing the subject is used as the first annotation information of the image to be labeled, and the matting result is used as the second annotation information of the image to be labeled; if the quality evaluation information indicates that the matting quality of the matting result confirmed by manual verification does not meet the target conditions, the image to be labeled is determined as the second positive sample, the manual annotation information of the image to be labeled is obtained, the image containing the subject is used as the first annotation information of the image to be labeled, and the manual annotation information of the image to be labeled is used as the second annotation information.

[0046] The first positive sample can refer to a sample that is easy for the matting model to generate a matting result that meets the target conditions, while the second positive sample can refer to a sample that is difficult for the matting model to generate a matting result that meets the target conditions.

[0047] In the embodiments of this application, there are multiple ways to enable electronic devices to obtain manually labeled information of images to be labeled.

[0048] In one approach, after acquiring quality evaluation information indicating that the matting quality of the manually confirmed matting result does not meet the target conditions, the electronic device can display the operation interface of an interactive annotation tool on its screen, or prompt the annotator to access the interactive annotation tool installed on the electronic device. This interactive annotation tool can be tools like removeebg or Elsog, allowing the annotator to annotate the image using this tool, or to access the corresponding interactive tool's operation interface via voice control, screen swiping, or clicking based on the prompt information, and then annotate the image within the interface. After the annotator completes the annotation, the electronic device can use the annotation information obtained from the interactive annotation tool as the manually annotated information.

[0049] As another approach, after an electronic device receives quality evaluation information indicating that the matting quality of the manually confirmed matting result does not meet the target conditions, it can send a control command to another electronic device. This control command can control the screen of the other electronic device to display the operation interface of an interactive annotation tool or prompt the annotator to enter the interactive annotation tool installed on the other electronic device. After the annotator completes the annotation using the interactive annotation tool on the other electronic device, the other electronic device can send the annotation information obtained from the interactive annotation tool as the manually annotated information to the electronic device.

[0050] This embodiment provides a data annotation method. After acquiring an image to be annotated, if it is determined that the image contains a subject, the method obtains the matting result corresponding to the image, acquires quality evaluation information obtained through manual evaluation of the matting result, and obtains the sample category and annotation information of the image to be annotated based on the quality evaluation information. This method allows for semi-automated data annotation with minimal human intervention after determining that the image contains a subject, and the sample category and annotation information of the image can be obtained through quality evaluation information obtained through manual evaluation of the matting result. This improves data annotation efficiency.

[0051] Please see Figure 2 This application provides a data annotation method applied to electronic devices, the method comprising:

[0052] S210: Obtain the image to be labeled.

[0053] S220: If it is determined that the image to be labeled contains a subject, obtain the image cutout result corresponding to the image to be labeled.

[0054] S230: Obtain quality evaluation information, and obtain the sample category and annotation information of the image to be annotated based on the quality evaluation information. The quality evaluation information is obtained by manually evaluating the image matting results.

[0055] S240: If it is determined that the image to be labeled does not contain a subject, and the quality evaluation information characterization is manually confirmed that the image to be labeled does not contain a subject, the image to be labeled is determined to be a negative sample, and the absence of a subject is used as the first labeling information of the image to be labeled.

[0056] As one approach, if an electronic device determines that the image to be labeled does not contain a subject based on the classification results, and the obtained quality evaluation information characterizes that the image to be labeled does not contain a subject as confirmed by a human, the electronic device can determine that the image to be labeled is a negative sample and use the absence of a subject as the first labeling information of the image to be labeled.

[0057] Optionally, multiple images can be annotated during each annotation process. When the electronic device determines that an image to be annotated does not contain a subject, it can store the image in the first reflow data pool and stop the image matting operation. When the quality evaluation information obtained by the electronic device is manually confirmed to indicate that the image to be annotated does not contain a subject, the image can be retrieved from the first data reflow pool, re-stored in the negative sample data pool, and identified as a negative sample. The absence of a subject is then used as the first annotation information for the image to be annotated.

[0058] Optionally, after the electronic device obtains the classification results, the annotator can input the quality assessment information, which indicates whether the image to be annotated, as confirmed by humans, does not contain the subject, into the electronic device. Alternatively, after the electronic device obtains the matting results, the annotator can input both the quality assessment information, which indicates whether the image to be annotated, as confirmed by humans, does not contain the subject, and the matting quality, which indicates that the matting results meet the target conditions, into the electronic device.

[0059] S250: If it is determined that the image to be labeled does not contain a subject, and the quality evaluation information indicates that the image to be labeled contains a subject as confirmed by manual verification, the image matting result corresponding to the image to be labeled is obtained.

[0060] As one approach, if an electronic device determines, based on the classification results, that the image to be labeled does not contain a subject, and the obtained quality evaluation information indicates that the image to be labeled contains a subject as confirmed by a human, the electronic device can obtain the matting result corresponding to the image to be labeled based on the matting model.

[0061] Optionally, multiple images can be annotated during each annotation process. Once the electronic device determines that an image does not contain a subject, it can store the image in the first reflow data pool and stop the image matting operation. When the quality evaluation information obtained by the electronic device is manually confirmed to contain a subject, the image can be retrieved from the first data reflow pool and the image matting operation can be performed.

[0062] S260: If the quality evaluation information further indicates that the matting quality of the matting result confirmed by human verification meets the target conditions, the image to be labeled is determined as the first positive sample, the subject is used as the first labeling information of the image to be labeled, and the matting result is used as the second labeling information of the image to be labeled.

[0063] In one approach, if the quality evaluation information obtained by the electronic device indicates that the image to be labeled contains a subject as confirmed by human intervention and that the matting quality of the matting result confirmed by human intervention meets the target conditions, the electronic device can determine the image to be labeled as the first positive sample, use the subject as the first annotation information of the image to be labeled, and use the matting result as the second annotation information of the image to be labeled.

[0064] Optionally, multiple images can be annotated during each annotation process. Once the electronic device determines that an image contains a subject, it can perform a cutout operation on that image. When the quality evaluation information obtained by the electronic device indicates that the image contains a subject as confirmed by human intervention, and the cutout quality of the cutout result meets the target conditions, the image can be stored in the first positive sample pool. The presence of the subject is used as the first annotation information of the image, and the cutout result is used as the second annotation information of the image.

[0065] Optionally, multiple images can be annotated during each annotation process. Once the electronic device determines that an image contains a subject, it can perform a cutout operation on that image. If the quality evaluation information obtained by the electronic device is manually confirmed to indicate that the image does not contain a subject, the image can be stored in the negative sample data pool, and the first annotation information for the image can be determined as "does not contain a subject."

[0066] S270: If the quality evaluation information also indicates that the matting quality of the matting result confirmed by human intervention does not meet the target conditions, the image to be labeled is determined as the second positive sample, the human annotation information of the image to be labeled is obtained, the subject is used as the first annotation information of the image to be labeled, and the human annotation information of the image to be labeled is used as the second annotation information.

[0067] In one approach, if the quality evaluation information obtained by the electronic device indicates that the image to be labeled contains a subject as confirmed by human intervention, and the matting quality of the matting result confirmed by human intervention does not meet the target conditions, the electronic device can determine the image to be labeled as the second positive sample, use the subject as the first annotation information of the image to be labeled, and use the human annotation information as the second annotation information of the image to be labeled.

[0068] Optionally, multiple images can be annotated during each annotation process. Once the electronic device determines that an image contains a subject, it can perform a cutout operation on that image. When the quality evaluation information obtained by the electronic device indicates that the image contains a subject as confirmed by human intervention, and the cutout quality does not meet the target conditions as confirmed by human intervention, the image can be stored in the second reflow data pool. The first annotation information of the image is determined to be the presence of a subject, and the human annotation information of the image is used as the second annotation information. After obtaining the human annotation information, the image is stored in the second positive sample data pool.

[0069] This embodiment provides a data annotation method that, through the aforementioned approach, allows for the acquisition of image matting results after determining that the image to be annotated contains a subject. The sample category and annotation information of the image to be annotated are then obtained through quality evaluation information obtained by manual assessment of the matting results. This achieves semi-automatic data annotation with minimal human intervention, thereby improving data annotation efficiency. Furthermore, in this embodiment, by receiving quality evaluation information indicating whether the matting quality of the manually confirmed matting results meets the target conditions, it is determined whether to directly use the matting results obtained from the controlled electronic device as the second annotation information of the image to be annotated, or to use the manually annotated information. This allows for the generation of accurate second annotation information with minimal human intervention, thereby improving the accuracy of tasks requiring images with second annotation information. Moreover, a data recirculation mechanism is formed through the first data recirculation pool, the second data recirculation pool, and human intervention, which can improve the accuracy of the annotation information of the image to be annotated and the data annotation efficiency.

[0070] Please see Figure 3 This application provides a data annotation method applied to electronic devices, the method comprising:

[0071] S310: Obtain the image to be labeled.

[0072] In a single data annotation process, there can be multiple images to be annotated.

[0073] S320: Input the image to be labeled into an image classification model to obtain a classification result, wherein the classification result indicates whether the image to be labeled contains a subject.

[0074] The image classification model can refer to a model pre-trained on a corresponding dataset, such as ResNet101 or VGG16. This dataset can include images labeled as containing a subject and images labeled as not containing a subject.

[0075] S330: If it is determined that the image to be labeled contains a subject, obtain the image cutout result corresponding to the image to be labeled.

[0076] S340: Obtain quality evaluation information, and obtain the sample category and annotation information of the image to be annotated based on the quality evaluation information. The quality evaluation information is obtained by manually evaluating the image matting results.

[0077] S350: After the current annotation process is completed, the image classification model is retrained based on the plurality of images to be annotated and the first annotation information corresponding to each of the plurality of images to be annotated, and the retrained image classification model is used as the image classification model for the next annotation process.

[0078] As one approach, after the current annotation process is completed, the image classification model can be retrained based on multiple images to be annotated and the first annotation information corresponding to each of the multiple images to be annotated, and the retrained image classification model can be used as the image classification model for the next annotation process.

[0079] As another approach, after each annotation process, negative sample images to be annotated can be stored in a negative sample data pool, and first and second positive sample images to be annotated can be stored in a positive sample data pool. Then, the image classification model can be retrained periodically using multiple images to be annotated in the negative and positive sample data pools, as well as the first annotation information corresponding to each of the multiple images to be annotated. The retrained image classification model can then be used as the image classification model for the next annotation process.

[0080] Optionally, the positive sample data pool may include a first positive sample data pool and a second positive sample data pool. The first positive sample image to be labeled can be stored in the first positive sample data pool, and the second positive sample image to be labeled can be stored in the second positive sample data pool.

[0081] Optionally, the iteration period can be determined based on the speed at which the data annotation method of this application generates annotation information. The faster the annotation information is generated, the shorter the training period. For example, when the annotation information is generated at a rate of 2,000 images to be annotated per day, the iteration period can be one month; when the annotation information is generated at a rate of 6,000 images to be annotated per day, the iteration period can be one week.

[0082] S360: After the current annotation process is completed, the matting model is retrained based on the multiple target images to be annotated and the second annotation information corresponding to each of the multiple target images to be annotated, and the retrained matting model is used as the matting model for the next annotation process.

[0083] There can be multiple target images to be labeled, and the target images to be labeled can be the images containing the subject as the first labeling information.

[0084] As one approach, after the current annotation process is completed, the matting model can be retrained based on multiple target images to be annotated and the second annotation information corresponding to each of the multiple target images to be annotated, and the retrained matting model can be used as the matting model for the next annotation process.

[0085] As another approach, after each annotation process, the first positive sample image to be annotated can be stored in the first positive sample data pool, and the second positive sample image to be annotated can be stored in the second positive sample data pool. Then, the image classification model can be retrained periodically using multiple images to be annotated in the first and second positive sample data pools, as well as the second annotation information corresponding to each of the multiple images to be annotated. The retrained image classification model can then be used as the image classification model for the next annotation process.

[0086] This embodiment provides a data annotation method that, after determining that an image to be annotated contains a subject, obtains the image matting result. The quality evaluation information obtained through manual assessment of the matting result reveals the sample category and annotation information of the image to be annotated. This achieves semi-automatic data annotation with minimal human intervention, thereby improving data annotation efficiency. Furthermore, in this embodiment, iterative training of the image classification model and the matting model using pre-annotated images gradually improves their accuracy, thus reducing manual costs and increasing data annotation efficiency. Moreover, the semi-automated data annotation method enhances efficiency, enabling the generation of usable matted data (images to be matted with second annotation information) in a short time. This shortens the iteration cycle of image classification models, matting models, and other models requiring matting data, accelerating the implementation of related applications.

[0087] To better understand the solutions of all embodiments of this application, a business process of the data annotation method of this application is described below.

[0088] Please see Figure 4After obtaining the image to be labeled based on step S1, it can be determined whether the image to be labeled contains a subject based on steps S2 and S3. If it does not contain a subject, step S4 can be executed: store the image to be labeled in the first data return pool, and obtain the quality evaluation information representing whether the image to be labeled contains a subject based on manual confirmation based on step S5. If it is confirmed based on step S6 that the quality evaluation information represents whether the image to be labeled does not contain a subject based on manual confirmation, then step S7 can be executed: confirm that the image to be labeled is a negative sample and that the first annotation information of the image to be labeled does not contain a subject, and store the image to be labeled in the negative sample data pool. If, based on step S6, it is confirmed that the quality evaluation information characterizes the image to be labeled as containing a subject as manually confirmed, or if, based on steps S2 and S3, it is confirmed that the image to be labeled contains a subject, then, based on steps S8 and S9, the matting result and quality evaluation information characterizing whether the matting quality of the manually confirmed matting result meets the target conditions can be obtained. If, based on step S10, it is confirmed that the quality evaluation information characterizes the matting result as manually confirmed, meets the target conditions, then, based on steps S11 and S12, the image to be labeled is confirmed as the first positive sample, the first annotation information of the image to be labeled is "containing a subject," and the second annotation information is the matting result, and the image to be labeled is stored in the first positive sample data pool. If, based on step S10, it is confirmed that the quality evaluation information characterizes the matting result as manually confirmed, does not meet the target conditions, then, based on step S13, the image to be labeled is stored in the second data return pool, and based on steps S14 and S15, the manually labeled information of the image to be labeled is obtained, confirming that the image to be labeled is the second positive sample, the first annotation information of the image to be labeled is "containing a subject," and the second annotation information is the matting result, and the image to be labeled is stored in the second positive sample data pool. Optionally, based on steps S8 and S9, in addition to obtaining the matting result and the quality evaluation information indicating whether the matting quality of the manually confirmed matting result meets the target conditions, it is also possible to obtain the quality evaluation information indicating whether the manually confirmed image to be labeled contains the subject. If step S10 confirms that the quality evaluation information indicates that the manually confirmed image to be labeled does not contain the subject, the image to be labeled can be stored in the first data reflux pool or the negative sample data pool.

[0089] Please see Figure 5 This application provides a data annotation device 600, which operates in an electronic device. The device 600 includes:

[0090] The image acquisition unit 610 is used to acquire the image to be labeled.

[0091] The image matting result acquisition unit 620 is used to acquire the image matting result corresponding to the image to be labeled if it is determined that the image to be labeled contains a subject.

[0092] The annotation information generation unit 630 is used to obtain quality evaluation information, and to obtain the sample category and annotation information of the image to be annotated based on the quality evaluation information. The quality evaluation information is obtained by manually evaluating the image matting results.

[0093] As one approach, the matting result acquisition unit 620 is specifically used to determine the image to be labeled as a negative sample if it is determined that the image to be labeled contains a subject and the quality evaluation information characterization is manually confirmed to be that the image to be labeled does not contain a subject, and to use the absence of a subject as the first annotation information of the image to be labeled; and to acquire the matting result corresponding to the image to be labeled if it is determined that the image to be labeled contains a subject and the quality evaluation information characterization is manually confirmed to be that the image to be labeled contains a subject.

[0094] In one approach, the image matting result acquisition unit 620 is specifically used to input the image to be labeled into an image classification model to obtain a classification result, wherein the classification result indicates whether the image to be labeled contains a subject.

[0095] In one approach, the matting result acquisition unit 620 is specifically used to input the image to be labeled into the matting model if it is determined that the image to be labeled contains a subject, so as to obtain the matting result.

[0096] In one approach, the image matting result acquisition unit 620 is specifically used to input the image to be labeled into an image classification model to obtain a classification result, wherein the classification result indicates whether the image to be labeled contains a subject.

[0097] In one manner, the annotation information generation unit 630 is specifically used to determine the image to be annotated as a first positive sample if the quality evaluation information characterization indicates that the image matting quality of the matting result has met the target conditions after manual confirmation, and to use the image containing the subject as the first annotation information of the image to be annotated, and to use the matting result as the second annotation information of the image to be annotated; if the quality evaluation information characterization indicates that the image matting quality of the matting result has not met the target conditions after manual confirmation, the image to be annotated is determined as a second positive sample, and to obtain the manual annotation information of the image to be annotated, using the image containing the subject as the first annotation information of the image to be annotated, and to use the manual annotation information of the image to be annotated as the second annotation information.

[0098] In one approach, the annotation information generation unit 630 is specifically used to determine that the image to be annotated does not contain a subject, and the quality evaluation information characterization is manually confirmed to indicate that the image to be annotated does not contain a subject, and to determine that the image to be annotated is a negative sample, and to use the absence of a subject as the first annotation information of the image to be annotated.

[0099] Alternatively, the annotation information generation unit 630 is specifically configured to: if it is determined that the image to be annotated does not contain a subject, and the quality evaluation information indicates that the image to be annotated contains a subject as confirmed by human intervention, obtain the matting result corresponding to the image to be annotated; if the quality evaluation information also indicates that the matting quality of the matting result confirmed by human intervention meets the target conditions, determine the image to be annotated as a first positive sample, use the inclusion of a subject as the first annotation information of the image to be annotated, and use the matting result as the second annotation information of the image to be annotated; if the quality evaluation information also indicates that the matting quality of the matting result confirmed by human intervention does not meet the target conditions, determine the image to be annotated as a second positive sample, obtain the human annotation information of the image to be annotated, use the inclusion of a subject as the first annotation information of the image to be annotated, and use the human annotation information of the image to be annotated as the second annotation information.

[0100] There are multiple images to be labeled, and the device 600 also includes:

[0101] The model update unit 640 is used to retrain the image classification model based on the plurality of unlabeled images and the first annotation information corresponding to each of the plurality of unlabeled images after the current annotation process is completed, and use the retrained image classification model as the image classification model for the next annotation process.

[0102] In one approach, there are multiple target images to be labeled, and the first labeling information is an image containing a subject. The model update unit 640 is specifically used to retrain the matting model based on the multiple target images to be labeled and the second labeling information corresponding to each of the multiple target images to be labeled after the current labeling process is completed, and use the retrained matting model as the matting model for the next labeling process.

[0103] The following will combine Figure 6 This application describes an electronic device.

[0104] Please see Figure 6 Based on the aforementioned data annotation method and apparatus, this application embodiment also provides another electronic device 100 capable of executing the aforementioned data annotation method. The electronic device 100 includes one or more (only one shown) processors 102 and a memory 104 coupled together. The memory 104 stores a program capable of executing the contents of the aforementioned embodiments, and the processors 102 can execute the program stored in the memory 104.

[0105] The processor 102 may include one or more processing cores. The processor 102 connects to various parts within the electronic device 100 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 104, and by calling data stored in the memory 104. Optionally, the processor 102 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 102 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 102 and may be implemented separately using a communication chip.

[0106] The memory 104 may include random access memory (RAM) or read-only memory (ROM). The memory 104 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 104 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described below. The data storage area may also store data created by the terminal 100 during use (such as phonebook data, audio and video data, chat log data, etc.).

[0107] Please refer to Figure 7 This diagram illustrates a structural block of a computer-readable storage medium provided in an embodiment of this application. The computer-readable storage medium 800 stores program code that can be called by a processor to execute the methods described in the above method embodiments.

[0108] The computer-readable storage medium 800 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 800 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 800 has storage space for program code 810 that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code 810 may, for example, be compressed in a suitable form.

[0109] In summary, the data annotation method, apparatus, and electronic device provided in this application, after acquiring an image to be annotated, if it is determined that the image to be annotated contains a subject, obtains the matting result corresponding to the image to be annotated, obtains quality evaluation information obtained through manual evaluation of the matting result, and obtains the sample category and annotation information of the image to be annotated based on the quality evaluation information. This method enables semi-automated data annotation with minimal human intervention after determining that the image to be annotated contains a subject, and the sample category and annotation information of the image to be annotated can be obtained through quality evaluation information obtained through manual evaluation of the matting result. This improves data annotation efficiency by achieving semi-automated data annotation with minimal human intervention.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A data annotation method, characterized in that, Applied to electronic devices, the method includes: Obtain multiple images to be labeled. The image to be labeled is input into an image classification model to obtain a classification result, which indicates whether the image to be labeled contains a subject. If it is determined that the image to be labeled contains a subject, the matting result corresponding to the image to be labeled is obtained, and the matting result is obtained through a matting model; Obtain quality evaluation information, and obtain the sample category and annotation information of the image to be annotated based on the quality evaluation information. The quality evaluation information is obtained by manually evaluating the image matting results. After the current annotation process is completed, the image classification model is retrained based on multiple images to be annotated and the first annotation information corresponding to each of the multiple images to be annotated. The retrained image classification model is used as the image classification model for the next annotation process. The first annotation information indicates whether the image to be annotated contains a subject. After the current annotation process is completed, the matting model is retrained based on multiple images to be annotated and the second annotation information corresponding to each of the multiple images to be annotated. The retrained matting model is used as the matting model for the next annotation process. If the quality evaluation information of the matting result corresponding to the image to be annotated indicates that the matting quality meets the target condition, the second annotation information is the matting result. If the quality evaluation information of the matting result corresponding to the image to be annotated indicates that the matting quality does not meet the target condition, the second annotation information is the manual annotation information of the image to be annotated.

2. The method according to claim 1, characterized in that, Based on the quality assessment information, the sample category and annotation information of the image to be labeled are obtained, including: If the quality evaluation information characterizes the matting quality of the matting result as confirmed by manual verification, the image to be labeled is determined as the first positive sample, the subject is used as the first labeling information of the image to be labeled, and the matting result is used as the second labeling information of the image to be labeled. If the quality evaluation information indicates that the matting quality of the matting result confirmed by human verification does not meet the target conditions, the image to be labeled is determined as the second positive sample, the human annotation information of the image to be labeled is obtained, the subject is used as the first annotation information of the image to be labeled, and the human annotation information of the image to be labeled is used as the second annotation information.

3. The method according to claim 1, characterized in that, The method further includes: If it is determined that the image to be labeled does not contain a subject, and the quality evaluation information characterization is manually confirmed that the image to be labeled does not contain a subject, the image to be labeled is determined to be a negative sample, and the absence of a subject is used as the first labeling information of the image to be labeled.

4. The method according to claim 1, characterized in that, The method further includes: If it is determined that the image to be labeled does not contain a subject, and the quality evaluation information characterizes that the image to be labeled contains a subject as confirmed by manual verification, the image matting result corresponding to the image to be labeled is obtained; If the quality evaluation information further indicates that the matting quality of the matting result confirmed by human verification meets the target conditions, the image to be labeled is determined as the first positive sample, the subject is used as the first labeling information of the image to be labeled, and the matting result is used as the second labeling information of the image to be labeled; If the quality evaluation information also indicates that the matting quality of the matting result confirmed by human intervention does not meet the target conditions, the image to be labeled is determined as the second positive sample, the human annotation information of the image to be labeled is obtained, the subject is used as the first annotation information of the image to be labeled, and the human annotation information of the image to be labeled is used as the second annotation information.

5. The method according to claim 1, characterized in that, If it is determined that the image to be labeled contains a subject, obtaining the image cutout result corresponding to the image to be labeled includes: If it is determined that the image to be labeled contains a subject and the quality evaluation information characterizes that the image to be labeled does not contain a subject after manual confirmation, the image to be labeled is determined to be a negative sample, and the absence of a subject is used as the first labeling information of the image to be labeled. If it is determined that the image to be labeled contains a subject and the quality evaluation information characterizes that the image to be labeled contains a subject as confirmed by manual verification, the image matting result corresponding to the image to be labeled is obtained.

6. A data annotation device, characterized in that, Operating in an electronic device, the device includes: An image acquisition unit is used to acquire multiple images to be labeled. The image matting result acquisition unit is used to input the image to be labeled into an image classification model to obtain a classification result, wherein the classification result indicates whether the image to be labeled contains a subject. The matting result acquisition unit is used to acquire the matting result corresponding to the image to be labeled if it is determined that the image to be labeled contains a subject. The matting result is obtained through a matting model. The annotation information generation unit is used to obtain quality evaluation information, and to obtain the sample category and annotation information of the image to be annotated based on the quality evaluation information. The quality evaluation information is obtained by manually evaluating the image matting results. The model update unit is used to retrain the image classification model based on multiple images to be labeled and their respective first annotation information after the current annotation process ends, and use the retrained image classification model as the image classification model for the next annotation process. The first annotation information indicates whether the image to be labeled contains a subject. After the current annotation process ends, the matting model is retrained based on multiple images to be labeled and their respective second annotation information, and used as the matting model for the next annotation process. If the quality evaluation information of the matting result corresponding to the image to be labeled indicates that the matting quality meets the target condition, the second annotation information is the matting result. If the quality evaluation information of the matting result corresponding to the image to be labeled indicates that the matting quality does not meet the target condition, the second annotation information is the manual annotation information of the image to be labeled.

7. An electronic device, characterized in that, Includes one or more processors and memory; One or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs being configured to perform the method of any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code, wherein the method described in any one of claims 1-5 is executed when the program code is run.

Images