Data processing method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application disclose a data processing method and device, electronic equipment and storage medium. The method comprises: obtaining a to-be-labeled image; obtaining a text label corresponding to the to-be-labeled image; inputting the to-be-labeled image and the text label into a semantic segmentation model, and obtaining a labeling result image output by the semantic segmentation model; based on the to-be-labeled image and the determined labeling result image, optimizing the semantic segmentation model to reduce the proportion of images labeled by human intervention in the labeling result image output by the semantic segmentation model, and obtaining a target semantic segmentation model; and based on the target semantic segmentation model, performing image data labeling. The method can perform image data labeling based on a semantic segmentation model with a smaller proportion of images labeled by human intervention in the labeling result image, which can reduce the cost of artificial labeling and improve the efficiency of data processing.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a data processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] Semantic segmentation is a deep learning algorithm that associates labels or categories with each pixel of an image. It is used to identify sets of pixels that constitute distinguishable categories. For example, autonomous vehicles need to identify vehicles, pedestrians, traffic signals, crosswalks, and other road features. Semantic segmentation can be used in a variety of applications, such as autonomous driving, medical imaging, and industrial inspection. However, current improvements in semantic segmentation algorithms largely rely on manually labeled data, which is costly, inefficient, and hinders user experience improvement. Summary of the Invention

[0003] In view of the above problems, this application proposes a data processing method, apparatus, electronic device, and storage medium to improve the above problems.

[0004] In a first aspect, embodiments of this application provide a data processing method, the method comprising: acquiring an image to be labeled; acquiring text labels corresponding to the image to be labeled; inputting the image to be labeled and the text labels into a semantic segmentation model, and acquiring a labeled result image output by the semantic segmentation model; optimizing the semantic segmentation model based on the image to be labeled and the determined labeled result image to reduce the proportion of manually labeled images in the labeled result image output by the semantic segmentation model, thereby obtaining a target semantic segmentation model; and performing image data labeling based on the target semantic segmentation model.

[0005] Secondly, embodiments of this application provide a data processing apparatus, the apparatus comprising: a data acquisition module for acquiring an image to be labeled; a text label acquisition module for acquiring text labels corresponding to the image to be labeled; an annotation result image acquisition module for inputting the image to be labeled and the text labels into a semantic segmentation model and acquiring an annotation result image output by the semantic segmentation model; a model optimization module for optimizing the semantic segmentation model based on the image to be labeled and the determined annotation result image, so as to reduce the proportion of manually annotated images in the annotation result image output by the semantic segmentation model, thereby obtaining a target semantic segmentation model; and a data processing module for performing image data annotation based on the target semantic segmentation model.

[0006] Thirdly, this application provides an electronic device including one or more processors and a memory; one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs being configured to perform the method described in the first aspect above.

[0007] Fourthly, this application provides a computer-readable storage medium storing program code, wherein the method described in the first aspect is executed when the program code is run.

[0008] This application provides a data processing method, apparatus, electronic device, and storage medium. The method involves: acquiring an image to be labeled; acquiring text labels corresponding to the image to be labeled; inputting the image to be labeled and the text labels into a semantic segmentation model and acquiring the labeled result image output by the semantic segmentation model; optimizing the semantic segmentation model based on the image to be labeled and the determined labeled result image to reduce the proportion of manually labeled images in the labeled result image output by the semantic segmentation model, thereby obtaining a target semantic segmentation model; and performing image data labeling based on the target semantic segmentation model. This method enables image data labeling based on a semantic segmentation model with a lower proportion of manually labeled images in the labeled result image, thus reducing manual labeling costs and improving data processing efficiency. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A flowchart of a data processing method according to an embodiment of this application is shown.

[0011] Figure 2 A flowchart of a data processing method according to another embodiment of this application is shown.

[0012] Figure 3 An example flowchart of the data processing method provided in an embodiment of this application is shown.

[0013] Figure 4 A flowchart of a data processing method according to another embodiment of this application is shown.

[0014] Figure 5 A structural block diagram of a data processing apparatus according to an embodiment of this application is shown.

[0015] Figure 6 A structural block diagram of an electronic device for performing a data processing method according to an embodiment of the present application is shown.

[0016] Figure 7 It is a storage unit in this application embodiment for storing or carrying program code that implements a data processing method according to this application embodiment. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0018] In the field of computer vision, one area is semantic segmentation, which generally involves pixel-level classification of images. Specifically, semantic image segmentation involves labeling each pixel with its corresponding category. Because all pixels must be considered, semantic image segmentation is also considered a type of dense prediction. However, current improvements in semantic segmentation algorithms largely rely on manually labeled data, which is costly, inefficient, and results in a poor user experience.

[0019] To address the aforementioned issues, the inventors, through extensive research, have proposed the data processing method, apparatus, electronic device, and storage medium provided in the embodiments of this application. This method is applied to electronic devices and can enable image data annotation based on a semantic segmentation model with a lower proportion of manually annotated images in the annotation result map. This reduces the cost of manual annotation and improves the efficiency of data processing.

[0020] In this application, the specific type of electronic device is not limited. For example, it can be an electronic device such as a mobile phone, tablet, PC, or smart wearable watch.

[0021] To facilitate a better understanding of the solutions described in this application, the relevant terms used in the embodiments of this application are briefly explained below:

[0022] Fine-grained image tagging: This algorithm represents the input image and outputs the object category in the image in text form (e.g., if there is a person walking a dog in the image, the fine-grained image tagging algorithm will output: human, dog).

[0023] Multimodal semantic segmentation algorithm: Represents the input image and text, and the algorithm outputs the segmentation results of the objects corresponding to the text in the image, with different objects marked with different colors.

[0024] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0025] Please see Figure 1 One embodiment of this application provides a data processing method that can be applied to electronic devices, the method comprising:

[0026] Step S110: Obtain the image to be labeled.

[0027] In this embodiment, the image to be labeled can be understood as an image that has not been classified and labeled at the pixel level, and the type of image to be labeled is not limited. The source of the image to be labeled can be various. For example, the image to be labeled can be an image downloaded from the network, an image read from a specific unlabeled data pool (library), an image taken locally by an electronic device, or an image sent by other devices.

[0028] Therefore, as one implementation method, the image to be labeled can be downloaded from the network, read from a specific unlabeled data pool (library), acquired locally by an electronic device, or used as the image to be labeled from an image sent by another device. When semantic segmentation of the image is required, the image to be semantically segmented can be used as the image to be labeled.

[0029] In this embodiment, the image to be labeled is a color image. In at least one embodiment, the image to be labeled may also be a grayscale image. To improve the distinguishability when performing pixel-level classification and labeling of images, if the image to be labeled is a single-channel grayscale image, it can be converted into a three-channel color image through channel conversion.

[0030] Step S120: Obtain the text label corresponding to the image to be labeled.

[0031] An image to be labeled may include multiple objects, each representing a type of object. To improve the accuracy of image labeling, text labels corresponding to the images to be labeled can be obtained first, so that the labeled result image can be obtained subsequently based on the text labels corresponding to the images to be labeled.

[0032] The number of text labels corresponding to each image to be labeled is the same as the number of types of objects to be labeled included in that image. For example, assuming that the image to be labeled includes three types of objects: a little boy, a tree, and a flower, then the number of text labels corresponding to that image is 3.

[0033] Step S130: Input the image to be labeled and the text label into the semantic segmentation model, and obtain the labeled result image output by the semantic segmentation model.

[0034] In this application, the semantic segmentation model can be a multimodal semantic segmentation model, for example, the semantic segmentation model can be LSeg (Language-driven semantic segmentation).

[0035] One approach is to input the image to be labeled and the text label into the semantic segmentation model, and train the semantic segmentation model in a semi-supervised manner to enable it to label images, thereby obtaining the labeled result image output by the semantic segmentation model.

[0036] The annotation result image can be understood as a mask image of the object to be annotated in the image to be annotated. Different images to be annotated will result in different annotation result images.

[0037] Step S140: Optimize the semantic segmentation model based on the image to be labeled and the determined labeling result image to reduce the proportion of manually labeled images in the labeling result image output by the semantic segmentation model, and obtain the target semantic segmentation model.

[0038] In this context, a definitive annotation result image can be understood as an annotation result image that does not miss any objects to be labeled. For example, if the image to be labeled contains five objects to be labeled, and the annotation result image output by the semantic segmentation model also contains masks of five objects to be labeled, then this annotation result image can be used as a definitive annotation result image; however, if the image to be labeled contains five objects to be labeled, but the annotation result image output by the semantic segmentation model only contains masks of three objects to be labeled, then this annotation result image can be discarded or a new annotation result image can be obtained.

[0039] One approach is to optimize the semantic segmentation model based on the image to be labeled and the determined labeled result image. Specifically, the semantic segmentation model can be iteratively optimized using multiple rounds of the image to be labeled and the determined labeled result image until the proportion of manually annotated images in the output labeled result image of the semantic segmentation model is lower than a specified threshold. The semantic segmentation model obtained under this condition is then used as the target semantic segmentation model. Alternatively, the semantic segmentation model obtained after iterative optimization using multiple rounds of the image to be labeled and the determined labeled result image can be directly used as the target semantic segmentation model. The specific value of the specified threshold is not limited.

[0040] In this model, the content of the images to be labeled differs in each round. The proportion of manually labeled images in the semantic segmentation model's output image is lower in each round than in the previous round. By iteratively optimizing the semantic segmentation model through multiple rounds, the model's automatic semantic segmentation labeling capability and accuracy can be improved, thereby achieving a higher proportion of manually labeled images in the semantic segmentation model's output image.

[0041] Step S150: Perform image data annotation based on the target semantic segmentation model.

[0042] Once the target semantic segmentation model is obtained, image data can be labeled based on the target semantic segmentation model to achieve fast and efficient semi-automatic labeling of semantic segmentation data, reduce the labeling cost in the semantic segmentation process, and enhance the accuracy of labeling, thereby improving the labeling effect.

[0043] This application provides a data processing method that involves: acquiring an image to be labeled; acquiring text labels corresponding to the image to be labeled; inputting the image to be labeled and the text labels into a semantic segmentation model and acquiring the labeled result image output by the semantic segmentation model; optimizing the semantic segmentation model based on the image to be labeled and the determined labeled result image to reduce the proportion of manually labeled images in the labeled result image output by the semantic segmentation model, thereby obtaining a target semantic segmentation model; and performing image data labeling based on the target semantic segmentation model. This method enables image data labeling based on a semantic segmentation model with a lower proportion of manually labeled images in the labeled result image, thus reducing manual labeling costs and improving data processing efficiency.

[0044] Please see Figure 2 Another embodiment of this application provides a data processing method that can be applied to an electronic device, the method comprising:

[0045] Step S210: Obtain the image to be labeled.

[0046] The specific implementation of step S210 can be found in the relevant description in step S110, and will not be repeated here.

[0047] Step S220: Input the image to be labeled into the fine-grained image labeling algorithm.

[0048] One method for obtaining text labels corresponding to an image to be labeled is to input the image into a fine-grained image labeling algorithm, which then automatically labels the subject categories (or object categories) in the image. The specific type of fine-grained image labeling algorithm is not limited.

[0049] Step S230: Obtain the subject category information in the image to be labeled from the output of the fine-grained image labeling algorithm.

[0050] Each subject category represents an object category. In at least one embodiment, subject category information in the labeled image can be obtained during the automated labeling process of the fine-grained image labeling algorithm, or it can be obtained when the automated labeling of the fine-grained image labeling algorithm is completed.

[0051] Step S240: Use the subject category information as a text label corresponding to the image to be labeled.

[0052] In this embodiment of the application, when the subject category information in the image to be labeled is obtained from the fine-grained image labeling algorithm, the subject category information in the image to be labeled can be used as the text label corresponding to the image to be labeled.

[0053] Step S250: Input the image to be labeled and the text label into the semantic segmentation model, and obtain the labeled result image output by the semantic segmentation model.

[0054] The specific implementation of step S250 can be found in the relevant description in step S130, and will not be repeated here.

[0055] Step S260: Optimize the semantic segmentation model based on the image to be labeled and the labeled result image that meets the preset conditions, so as to reduce the proportion of manually labeled images in the labeled result image output by the semantic segmentation model, and obtain the target semantic segmentation model.

[0056] It should be noted that the annotation result image in this embodiment can include annotation result images that meet preset conditions. Specifically, if the number of subject categories in the annotation result image is equal to a preset threshold (the specific value of the preset threshold is not limited), the annotation result image can be determined to meet the preset conditions. In this way, when optimizing the semantic segmentation model based on the image to be annotated and the determined annotation result image, the semantic segmentation model can be optimized based on the image to be annotated and the annotation result image that meets the preset conditions, so as to reduce the proportion of manually annotated images in the annotation result image output by the semantic segmentation model and obtain the target semantic segmentation model.

[0057] When optimizing a semantic segmentation model based on an image to be labeled and labeled result images that meet preset conditions, one approach is to send the image to be labeled and labeled result images that meet preset conditions into a positive sample data pool (i.e., the image to be labeled and labeled result images that meet preset conditions can be stored in the positive sample data pool). Then, the data in the positive sample data pool is used to continuously optimize the semantic segmentation model, thereby continuously improving its image segmentation (image matting) capabilities and reducing the proportion of manually annotated images in the labeled result images output by the semantic segmentation model.

[0058] It should be noted that when optimizing the semantic segmentation model based on the image to be labeled and the labeled result image that meets the preset conditions, multiple rounds of the image to be labeled and the labeled result image that meets the preset conditions can be used to iteratively optimize the semantic segmentation model until the proportion of manually labeled images in the labeled result image output by the semantic segmentation model is lower than a specified threshold.

[0059] In this embodiment, the annotation result image output by the semantic segmentation model is a mask image. When optimizing the semantic segmentation model based on the image to be annotated and the annotation result image that meets the preset conditions, the weight parameters of the subject category in the annotation result image can be adjusted based on the mask image (for example, increasing the image proportion of a certain object to be annotated in the annotation result image) to optimize the semantic segmentation model. The specific adjustment strategy can be set according to actual needs and is not limited here.

[0060] Step S270: Perform image data annotation based on the target semantic segmentation model.

[0061] The specific implementation of step S270 can be found in the relevant description in step S150, and will not be repeated here.

[0062] In a specific application scenario, please refer to Figure 3 This illustrates an example flowchart of a data processing method provided in an embodiment of this application. Figure 3 As shown, the image to be labeled includes two types of objects: dogs and trees. The image to be labeled can be input into a fine-grained image tagging model, yielding the subject category information of "Dog" and "Tree" in the output image. Then, the image to be labeled and the subject category information "Dog" and "Tree" from the fine-grained image tagging model can be simultaneously input into a pre-trained multimodal semantic segmentation model to train its image segmentation capabilities. Finally, the labeled result image output by the pre-trained multimodal semantic segmentation model can be obtained. Figure 3 As can be seen, the labeled result image includes mask images corresponding to two categories of objects to be segmented: dogs and trees.

[0063] This application provides a data processing method that involves: acquiring an image to be labeled; inputting the image to be labeled into a fine-grained image labeling algorithm; obtaining subject category information from the image to be labeled as output by the fine-grained image labeling algorithm; using the subject category information as a text label corresponding to the image to be labeled; inputting the image to be labeled and the text label into a semantic segmentation model, and obtaining the labeled result image output by the semantic segmentation model; optimizing the semantic segmentation model based on the image to be labeled and the labeled result image that meets preset conditions to reduce the proportion of manually labeled images in the labeled result image output by the semantic segmentation model, thereby obtaining a target semantic segmentation model; and performing image data labeling based on the target semantic segmentation model. This method enables image data labeling based on a semantic segmentation model with a lower proportion of manually labeled images in the labeled result image, reducing manual labeling costs and improving data processing efficiency.

[0064] Please see Figure 4 Another embodiment of this application provides a data processing method that can be applied to electronic devices, the method comprising:

[0065] Step S310: Obtain the image to be labeled.

[0066] The specific implementation of step S310 can be found in the relevant description in step S110, and will not be repeated here.

[0067] Step S320: Input the image to be labeled into the fine-grained image labeling algorithm.

[0068] The specific implementation of step S320 can be found in the relevant description of step S220, and will not be repeated here.

[0069] Step S330: Obtain the subject category information in the image to be labeled from the output of the fine-grained image labeling algorithm.

[0070] The specific implementation of step S330 can be found in the relevant description in step S230, and will not be repeated here.

[0071] Step S340: Use the subject category information as a text label corresponding to the image to be labeled.

[0072] The specific implementation of step S340 can be found in the relevant description in step S240, and will not be repeated here.

[0073] Step S350: Input the image to be labeled and the text label into the semantic segmentation model, and obtain the labeled result image output by the semantic segmentation model.

[0074] The specific implementation of step S350 can be found in the relevant description in step S130, and will not be repeated here.

[0075] Step S360: Manually annotate the annotation result map that does not meet the preset conditions to obtain the first annotation result map.

[0076] In this embodiment, the annotation result image may include annotation result images that meet preset conditions and annotation result images that do not meet preset conditions. As described in the foregoing embodiments, an annotation result image that meets preset conditions can be understood as an annotation result image in which the number of subject categories in the annotation result image is equal to a preset threshold. It is easy to understand that an annotation result image that does not meet preset conditions can be understood as an annotation result image in which the number of subject categories in the annotation result image is not equal to the preset threshold, for example, an annotation result image in which objects to be annotated are omitted or under-annotated.

[0077] For labeled result images that do not meet the preset conditions, as one implementation method, they can be manually labeled to obtain a first labeled result image. Specifically, manual labeling can be performed using interactive labeling tools (such as Eiseg or Labelme).

[0078] Step S370: Place the image to be labeled and the labeled result image that meets the preset conditions into the first sample pool.

[0079] As one approach, the image to be labeled and the labeled result image that meets the preset conditions can be placed into the first sample pool for storage with a mapping relationship. The first sample pool can also be called the simple positive sample data pool.

[0080] Step S380: Place the image to be labeled and the first labeled result image into the second sample pool.

[0081] As one approach, the image to be labeled and the first labeled result image mentioned above can be placed into a second sample pool for storage with a mapping relationship. The second sample pool can also be called a hard positive sample data pool.

[0082] Step S390: Put the images in the first sample pool and the images in the second sample pool into the third sample pool.

[0083] As a method to better train and optimize the semantic segmentation model in a semi-supervised manner, images from the first sample pool and the second sample pool can be placed into a third sample pool, which combines manually annotated and unannotated images to guide the semantic segmentation model in better learning its image segmentation capabilities. The third sample pool can also be called the positive sample data pool.

[0084] Step S391: Optimize the semantic segmentation model based on the images in the third sample pool to reduce the proportion of manually annotated images in the annotation result image output by the semantic segmentation model, thereby obtaining the target semantic segmentation model.

[0085] In this embodiment, the semantic segmentation model can be optimized based on the images in the third sample pool to reduce the proportion of manually annotated images in the annotation result image output by the semantic segmentation model, thereby obtaining the target semantic segmentation model. The specific optimization method and process can be referred to the relevant description in the foregoing embodiments, and will not be repeated here.

[0086] Step S392: Perform image data annotation based on the target semantic segmentation model.

[0087] The specific implementation of step S392 can be found in the relevant description in step S150, and will not be repeated here.

[0088] This application provides a data processing method that involves: acquiring an image to be labeled; inputting the image to be labeled into a fine-grained image labeling algorithm; acquiring the subject category information in the image to be labeled output by the fine-grained image labeling algorithm; using the subject category information as a text label corresponding to the image to be labeled; inputting the image to be labeled and the text label into a semantic segmentation model, and acquiring the labeled result image output by the semantic segmentation model; manually labeling the labeled result images that do not meet preset conditions to obtain a first labeled result image; placing the image to be labeled and the labeled result images that meet preset conditions into a first sample pool; placing the image to be labeled and the first labeled result image into a second sample pool; placing the images in the first sample pool and the images in the second sample pool into a third sample pool; optimizing the semantic segmentation model based on the images in the third sample pool to reduce the proportion of manually labeled images in the labeled result image output by the semantic segmentation model, thereby obtaining a target semantic segmentation model; and performing image data labeling based on the target semantic segmentation model. This method enables image data labeling based on a semantic segmentation model with a lower proportion of manually labeled images in the labeled result image, reducing manual labeling costs and improving data processing efficiency.

[0089] Please see Figure 5 This application provides a data processing device 400, which operates in an electronic device. The device 400 includes:

[0090] The data acquisition module 410 is used to acquire the image to be labeled.

[0091] In this embodiment of the application, the image to be labeled is a color image.

[0092] The text label acquisition module 420 is used to acquire the text labels corresponding to the image to be labeled.

[0093] In one approach, the text label acquisition module 420 can be used to input the image to be labeled into a fine-grained image labeling algorithm; acquire the subject category information in the image to be labeled output by the fine-grained image labeling algorithm; and use the subject category information as a text label corresponding to the image to be labeled.

[0094] The annotation result image acquisition module 430 is used to input the image to be annotated and the text label into the semantic segmentation model, and acquire the annotation result image output by the semantic segmentation model.

[0095] The model optimization module 440 is used to optimize the semantic segmentation model based on the image to be labeled and the determined labeling result image, so as to reduce the proportion of manually labeled images in the labeling result image output by the semantic segmentation model, and obtain the target semantic segmentation model.

[0096] In this embodiment of the application, the annotation result map may include an annotation result map that meets preset conditions. In this way, the model optimization module 440 can be used to optimize the semantic segmentation model based on the image to be annotated and the annotation result map that meets the preset conditions, so as to reduce the proportion of manually annotated images in the annotation result map output by the semantic segmentation model and obtain the target semantic segmentation model.

[0097] In this embodiment of the application, the annotation result image may include annotation result images that meet preset conditions and annotation result images that do not meet preset conditions. In this way, the model optimization module 440 can be used to manually annotate the annotation result images that do not meet preset conditions to obtain a first annotation result image; put the image to be annotated and the annotation result images that meet preset conditions into a first sample pool; put the image to be annotated and the first annotation result image into a second sample pool; put the images in the first sample pool and the images in the second sample pool into a third sample pool; optimize the semantic segmentation model based on the images in the third sample pool to reduce the proportion of manually annotated images in the annotation result image output by the semantic segmentation model, and obtain a target semantic segmentation model.

[0098] In this embodiment, the annotation result image output by the semantic segmentation model is a mask image. In at least one embodiment, the model optimization module 440 can be specifically used to adjust the weight parameters of the subject category in the annotation result image based on the mask image, so as to optimize the semantic segmentation model, reduce the proportion of manually annotated images in the annotation result image output by the semantic segmentation model, and obtain the target semantic segmentation model.

[0099] In this embodiment of the application, the device 400 may further include a judgment module, which is used to determine that the annotation result image is an annotation result image that meets the preset conditions if the number of subject categories in the annotation result image is equal to a preset threshold.

[0100] The data processing module 450 is used to annotate image data based on the target semantic segmentation model.

[0101] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device and module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0102] In the several embodiments provided in this application, the coupling or direct coupling or communication connection between the modules shown or discussed may be an indirect coupling or communication connection through some interface, device or module, and may be electrical, mechanical or other forms.

[0103] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0104] Please see Figure 6 Based on the data processing method and apparatus described above, this application also provides an electronic device 100 capable of executing the aforementioned data processing method. The electronic device 100 includes a memory 102 and one or more (only one shown in the figure) processors 104 coupled to each other, with a communication line connecting the memory 102 and the processors 104. The memory 102 stores a program capable of executing the contents of the aforementioned embodiments, and the processors 104 can execute the program stored in the memory 102.

[0105] The processor 104 may include one or more processing cores. The processor 104 connects to various parts within the electronic device 100 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 102, and by calling data stored in the memory 102. Optionally, the processor 104 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 104 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 104 and may be implemented separately using a communication chip.

[0106] The memory 102 may include random access memory (RAM) or read-only memory (ROM). The memory 102 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 102 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the aforementioned embodiments. The data storage area may also store data created by the electronic device 100 during use (such as phonebook data, audio and video data, chat log data, etc.).

[0107] Please refer to Figure 7 This diagram illustrates a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. The computer-readable storage medium 500 stores program code that can be called by a processor to execute the methods described in the above method embodiments.

[0108] The computer-readable storage medium 500 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 500 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 500 has storage space for program code 510 that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code 510 may be compressed, for example, in a suitable form.

[0109] In summary, the data processing method, apparatus, electronic device, and storage medium provided in this application involve: acquiring an image to be labeled; acquiring text labels corresponding to the image to be labeled; inputting the image to be labeled and the text labels into a semantic segmentation model and acquiring the labeled result image output by the semantic segmentation model; optimizing the semantic segmentation model based on the image to be labeled and the determined labeled result image to reduce the proportion of manually labeled images in the labeled result image output by the semantic segmentation model, thereby obtaining a target semantic segmentation model; and performing image data labeling based on the target semantic segmentation model. Thus, this method enables image data labeling based on a semantic segmentation model with a lower proportion of manually labeled images in the labeled result image, reducing manual labeling costs and improving data processing efficiency.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A data processing method, characterized in that, The method includes: Obtain the image to be labeled; The image to be labeled is input into a fine-grained image labeling algorithm, which outputs the object categories in the image to be labeled in text form. Obtain the subject category information in the image to be labeled from the output of the fine-grained image labeling algorithm, and use the subject category information as the text label corresponding to the image to be labeled; The image to be labeled and the text label are input into the semantic segmentation model, and the labeled result image output by the semantic segmentation model is obtained. The labeled result image includes labeled result images that meet preset conditions and labeled result images that do not meet preset conditions. If the number of subject categories in the labeled result image is equal to a preset threshold, the labeled result image is determined to be a labeled result image that meets the preset conditions. The annotation result images that do not meet the preset conditions are manually annotated to obtain the first annotation result image; The image to be labeled and the labeled result image that meets the preset conditions are placed into the first sample pool; The image to be labeled and the first labeled result image are placed into the second sample pool; The images from the first sample pool and the images from the second sample pool are placed into the third sample pool; The semantic segmentation model is optimized based on the images in the third sample pool to reduce the proportion of manually annotated images in the output annotation results of the semantic segmentation model, thereby obtaining the target semantic segmentation model. Image data annotation is performed based on the target semantic segmentation model.

2. The method according to claim 1, characterized in that, The semantic segmentation model outputs a labeled result image as a mask image, and the method further includes: The semantic segmentation model is optimized by adjusting the weight parameters of the subject category in the labeled result image based on the mask image.

3. The method according to any one of claims 1-2, characterized in that, The image to be labeled is a color image.

4. A data processing apparatus, characterized in that, The device includes: The module for acquiring data to be labeled is used to acquire images to be labeled. The text label acquisition module is used to input the image to be labeled into a fine-grained image labeling algorithm, which outputs the object categories in the image to be labeled in text form; to acquire the subject category information in the image to be labeled output by the fine-grained image labeling algorithm, and to use the subject category information as the text label corresponding to the image to be labeled; The annotation result image acquisition module is used to input the image to be annotated and the text label into the semantic segmentation model, and obtain the annotation result image output by the semantic segmentation model. The annotation result image includes annotation result images that meet preset conditions and annotation result images that do not meet preset conditions. If the number of subject categories in the annotation result image is equal to a preset threshold, the annotation result image is determined to be an annotation result image that meets the preset conditions. The model optimization module is used to manually annotate the labeled result images that do not meet the preset conditions to obtain a first labeled result image; to put the image to be labeled and the labeled result images that meet the preset conditions into a first sample pool; to put the image to be labeled and the first labeled result image into a second sample pool; to put the images in the first sample pool and the images in the second sample pool into a third sample pool; and to optimize the semantic segmentation model based on the images in the third sample pool to reduce the proportion of manually annotated images in the labeled result images output by the semantic segmentation model, thereby obtaining a target semantic segmentation model. The data processing module is used to annotate image data based on the target semantic segmentation model.

5. An electronic device, characterized in that, Includes one or more processors and memory; One or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs being configured to perform the method of any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code, wherein the program code, when executed by a processor, performs the method according to any one of claims 1-3.

Images