Method, device, electronic device and storage medium for determining image tags

Abstract

The disclosed embodiments disclose a method, apparatus, electronic device, and storage medium for determining image labels, wherein the method comprises: obtaining first image data; predicting the first image data based on a first semi-supervised semantic segmentation model obtained through training to obtain a corresponding first prediction result; the first prediction result includes first probability data corresponding to the first image data, the first probability data including the probability that each first pixel in the first image data belongs to each type; determining first pseudo-label data corresponding to the first image data based on the first probability data, the first pseudo-label data including the type to which each first pixel belongs; determining first confidence levels corresponding to each first pixel based on the first probability data; and determining first label data corresponding to the first image data based on the first confidence levels corresponding to each first pixel and the first pseudo-label data. Automatic image labeling is achieved, thereby effectively reducing the workload of manual labeling and improving work efficiency.

Description

Technical Field

[0001] The present disclosure relates to computer vision technology, and in particular to a method, device, electronic device, and storage medium for determining image tags. Background Art

[0002] In the field of computer vision, it is necessary to realize semantic segmentation of images based on various models. The training of the models requires a large amount of labeled image data. In order to reduce the manual participation of data labeling, the training method of the semantic segmentation model based on semi-supervised learning has gradually become one of the important technologies. The training of the semantic segmentation model based on semi-supervised learning can combine labeled data and unlabeled data to realize the training of the semantic segmentation model. In order to further optimize the performance of the semantic segmentation model, some difficult sample image data that are more valuable for model training are usually determined and manually labeled. Then, the difficult sample image data and the corresponding manually labeled labels are used for the optimized training of the semantic segmentation model. However, manually labeling the difficult samples increases the workload of manual labeling and has low work efficiency. Summary of the Invention

[0003] In order to solve the above-mentioned technical problems such as low efficiency of manual labeling of difficult samples, the present disclosure is proposed. The embodiments of the present disclosure provide a method, apparatus, electronic device and storage medium for determining image labels.

[0004] According to one aspect of an embodiment of the present disclosure, a method for determining an image label is provided, comprising: acquiring first image data; predicting the first image data based on a first semi-supervised semantic segmentation model obtained through training to obtain a corresponding first prediction result; the first prediction result including first probability data corresponding to the first image data, the first probability data including the probability that each first pixel in the first image data belongs to each type; determining first pseudo-label data corresponding to the first image data based on the first probability data, the first pseudo-label data including the type to which each first pixel belongs; determining first confidence levels corresponding to each first pixel based on the first probability data; and determining first label data corresponding to the first image data based on the first confidence levels corresponding to each first pixel and the first pseudo-label data.

[0005] According to another aspect of an embodiment of the present disclosure, a device for determining an image label is provided, comprising: a first acquisition module for acquiring first image data; a first processing module for predicting the first image data based on a first semi-supervised semantic segmentation model obtained through training to obtain a corresponding first prediction result; the first prediction result includes first probability data corresponding to the first image data, the first probability data including the probability that each first pixel in the first image data belongs to each type; a second processing module for determining first pseudo-label data corresponding to the first image data based on the first probability data, the first pseudo-label data including the type to which each first pixel belongs; a third processing module for determining first confidence levels corresponding to each first pixel based on the first probability data; and a fourth processing module for determining first label data corresponding to the first image data based on the first confidence levels corresponding to each first pixel and the first pseudo-label data.

[0006] According to another aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the method for determining an image tag according to any of the above embodiments of the present disclosure.

[0007] According to another aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing instructions executable by the processor; and the processor for reading the executable instructions from the memory and executing the instructions to implement the method for determining an image label described in any of the above embodiments of the present disclosure.

[0008] Based on the image label determination method, device, electronic device and storage medium provided by the above-mentioned embodiments of the present disclosure, the first image data to be labeled can be predicted based on the semi-supervised semantic segmentation model, and then the pseudo-label corresponding to the first image data and the confidence corresponding to each pixel in the first image data can be determined based on the prediction result, and then the label data corresponding to the first image data can be determined based on the pseudo-label and the confidence corresponding to each pixel, thereby realizing automatic image labeling, thereby effectively reducing the workload of manual labeling and improving work efficiency.

[0009] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other purposes, features, and advantages of the present disclosure will become more apparent through a more detailed description of the embodiments of the present disclosure in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and are not intended to limit the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps.

[0011] Figure 1 This is an exemplary application scenario of the method for determining image labels provided by the present disclosure;

[0012] Figure 2 is a flowchart of a method for determining an image label provided by an exemplary embodiment of the present disclosure;

[0013] Figure 3 is a flowchart of step 205 provided by an exemplary embodiment of the present disclosure;

[0014] Figure 4 is a flowchart of step 201 provided by an exemplary embodiment of the present disclosure;

[0015] Figure 5 is a flowchart of step 2012 provided by an exemplary embodiment of the present disclosure;

[0016] Figure 6 is a flowchart of step 2012 provided by another exemplary embodiment of the present disclosure;

[0017] Figure 7 is a flowchart of a method for determining an image label provided by another exemplary embodiment of the present disclosure;

[0018] Figure 8 is a flowchart of a method for determining an image label provided by yet another exemplary embodiment of the present disclosure;

[0019] Figure 9 is a structural diagram of an apparatus for determining an image label provided by an exemplary embodiment of the present disclosure;

[0020] Figure 10 is a structural diagram of a fourth processing module 505 provided by an exemplary embodiment of the present disclosure;

[0021] Figure 11 is a structural diagram of a first acquisition module 501 provided by an exemplary embodiment of the present disclosure;

[0022] Figure 12 is a structural diagram of a third processing unit 5012 provided by an exemplary embodiment of the present disclosure;

[0023] Figure 13is a structural diagram of a third processing unit 5012 provided by another exemplary embodiment of the present disclosure;

[0024] Figure 14 is a structural diagram of an apparatus for determining an image label provided by another exemplary embodiment of the present disclosure;

[0025] Figure 15 It is a structural diagram of an application embodiment of the electronic device disclosed in the present invention. DETAILED DESCRIPTION

[0026] Below, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0027] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0028] Those skilled in the art will understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, and do not represent any specific technical meanings, nor do they indicate a necessary logical order between them.

[0029] It should also be understood that in the embodiments of the present disclosure, “a plurality of” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.

[0030] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0031] In addition, the term "and / or" in this disclosure is merely a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this disclosure generally indicates that the related objects are in an "or" relationship.

[0032] It should also be understood that the description of the various embodiments in this disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced with each other. For the sake of brevity, they will not be described one by one.

[0033] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0034] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0035] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0036] It should be noted that like reference numerals and letters refer to like items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0037] The embodiments of the present disclosure can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate in conjunction with numerous other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, minicomputer systems, mainframe computer systems, and distributed cloud computing technology environments including any of the above systems, among others.

[0038] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system-executable instructions (such as program modules) executed by a computer system. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked via a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media, including storage devices.

[0039] Overview of the Disclosure

[0040] In the process of implementing the present disclosure, the inventors discovered that in the field of computer vision, the training of semantic segmentation models requires a large amount of labeled image data. In order to reduce the manual participation of data labeling, the semi-supervised semantic segmentation model training method has gradually become one of the important technologies. The training of the semi-supervised semantic segmentation model can combine labeled data and unlabeled data to realize the training of the semantic segmentation model. In order to further optimize the performance of the semantic segmentation model, some difficult sample image data that are more valuable for model training are usually determined, and they are manually labeled. Then, the difficult sample image data and the corresponding manually labeled labels are used for the optimized training of the semantic segmentation model. However, manually labeling the difficult samples increases the workload of manual labeling, resulting in low work efficiency.

[0041] Exemplary Overview

[0042] Figure 1 This is an exemplary application scenario of the method for determining image labels provided by the present invention. For the mined difficult sample image data, the image label determination device of the present invention is used to execute the method for determining image labels of the present invention, and the label data of the difficult sample image data can be automatically determined. Specifically, the difficult sample image data to be labeled can be predicted based on the semi-supervised semantic segmentation model, and then the pseudo-label corresponding to the difficult sample image data and the confidence corresponding to each pixel in the difficult sample image data can be determined based on the prediction result, and then the label data corresponding to the difficult sample image data can be determined based on the pseudo-label and the confidence corresponding to each pixel, so as to realize automatic labeling of images, thereby effectively reducing the workload of manual labeling and improving work efficiency.

[0043] The image label determination method provided by the present disclosure can be applied to any application field involving image semantic segmentation, including but not limited to autonomous driving, geographic information systems, medical image analysis, robotics and other fields.

[0044] Exemplary Methods

[0045] Figure 2 This is a flow chart of a method for determining an image tag provided by an exemplary embodiment of the present disclosure. This embodiment can be applied to electronic devices, such as servers or terminals, such as Figure 2 As shown, the following steps are included:

[0046] Step 201: Acquire first image data.

[0047] The first image data may be any image data requiring annotation, such as high-value, difficult-to-use image data mined through any feasible implementation method, without specific limitation. The first image data may be pre-acquired and stored in a specific location, from which it is retrieved when annotation is required. The first image data may include one or more frames.

[0048] Step 202: predict the first image data based on the first semi-supervised semantic segmentation model obtained through training to obtain a corresponding first prediction result; the first prediction result includes first probability data corresponding to the first image data, and the first probability data includes the probability that each first pixel in the first image data belongs to each type.

[0049] Among them, the first semi-supervised semantic segmentation model can be any implementable semi-supervised semantic segmentation model, which is not limited in this disclosure, such as a semi-supervised semantic segmentation model based on deeplabv3+ and its related series. The type can be set to one or more according to actual needs. For example, in the field of autonomous driving, the types include obstacles such as people, animals, vehicles, and curbs. The first probability data includes the probability that each first pixel in the first image data belongs to each type. For example, the probability that a first pixel belongs to a person is 0.1, the probability that it belongs to an animal is 0.2, the probability that it belongs to a vehicle is 0.6, and the probability that it belongs to a curb is 0.1. The first image data is predicted based on the first semi-supervised semantic segmentation model, that is, the first image data is used as the input of the first semi-supervised semantic segmentation model, and the output is obtained through reasoning of the first semi-supervised semantic segmentation model. The specific principles will not be repeated here.

[0050] Step 203 : Determine first pseudo label data corresponding to the first image data based on the first probability data, where the first pseudo label data includes the type to which each first pixel belongs.

[0051] The first pseudo label data corresponding to the first image data is determined based on the first probability data, that is, the first probability data is encoded, such as one-hot encoding, and the encoding result is used as the first pseudo label data corresponding to the first image data.

[0052] For example, the probability that a first pixel belongs to a person is 0.1, the probability that it belongs to an animal is 0.2, the probability that it belongs to a vehicle is 0.6, and the probability that it belongs to a curb is 0.1. The label of the first pixel obtained after encoding is 0010.

[0053] Step 204 : Determine a first confidence level corresponding to each first pixel based on the first probability data.

[0054] The first confidence level of the first pixel is information indicating the reliability of the prediction result of the first pixel, and the first confidence level can be determined based on any feasible method, such as by calculating the difference between the two probability values ​​of the top 2.

[0055] For example, the probability that a first pixel belongs to a person is 0.1, the probability that it belongs to an animal is 0.2, the probability that it belongs to a vehicle is 0.6, and the probability that it belongs to a curb is 0.1, then the first confidence corresponding to the first pixel is 0.6-0.2=0.4.

[0056] Step 205 : Determine first label data corresponding to the first image data based on the first confidence level and the first pseudo label data corresponding to each first pixel.

[0057] Specifically, based on the first confidence level corresponding to each first pixel, it is determined whether the type corresponding to each first pixel is valid for supervision. For example, for a first pixel whose first confidence level is less than the second threshold, the credibility of the classification result is poor, and the type to which the first pixel belongs can be set to be invalid for supervision. That is, when used for supervised learning, the type of the first pixel is not learned as a supervisory label to avoid learning incorrect classification experience. For a first pixel whose first confidence level is greater than the second threshold, the credibility of the classification result is good, and its type can be used as a label for supervised learning. The specific second threshold can be set according to actual needs and is not limited in this embodiment.

[0058] In an optional example, the second confidence level corresponding to the first image data can be determined by averaging the first confidence levels corresponding to the first pixels, and the first image data can be screened based on the second confidence level. Images with a second confidence level less than the first threshold value are selected as difficult sample image data with higher value for subsequent label determination. The specific settings can be made according to actual needs.

[0059] The method for determining image labels provided in this embodiment can predict the first image data to be labeled based on a semi-supervised semantic segmentation model, and then determine the pseudo-label corresponding to the first image data and the confidence corresponding to each pixel in the first image data based on the prediction result, and then determine the label data corresponding to the first image data based on the pseudo-label and the confidence corresponding to each pixel, thereby realizing automatic labeling of images, thereby effectively reducing the workload of manual labeling and improving work efficiency.

[0060] In an alternative example, Figure 3 FIG2 is a flow chart of step 205 provided by an exemplary embodiment of the present disclosure. In this example, step 205 may specifically include the following steps:

[0061] Step 2051: Determine a second confidence level corresponding to the first image data based on the first confidence levels corresponding to the first pixels.

[0062] Optionally, certain rules may be set according to actual needs to determine the second confidence level, such as summing and averaging the first confidence levels corresponding to the first pixels, and using the average as the second confidence level corresponding to the first image data.

[0063] Step 2052: If the second confidence is less than the first threshold, the type of the first pixel in the first pseudo-label data whose first confidence is less than the second threshold is updated to the first type, and the second pseudo-label data corresponding to the first pseudo-label data is obtained. The first type indicates that the first pixel is invalid for supervision.

[0064] Specifically, the first image data is first screened based on the second confidence level and the first threshold value, and the target image whose second confidence level is less than the first threshold value is screened out as a difficult sample image with higher value. The type of the first pixel corresponding to the target image in the first pseudo-label data is updated according to the first confidence level and the second threshold value, and the type of the first pixel whose first confidence level is less than the second threshold value is updated to the first type. After the update, the second pseudo-label data is obtained, and the second pseudo-label data includes the type of the first pixel that is valid for supervision and the first type of the first pixel that is invalid for supervision.

[0065] For images whose second confidence is greater than the first threshold, the type of the first pixel may not be updated, and the first pseudo-label data may be directly used as the corresponding label data. Alternatively, the labels of these images may not be determined, which can be set according to actual needs.

[0066] Step 2053: Use the second pseudo label data as the first label data corresponding to the first image data.

[0067] Specifically, for a first pixel whose first confidence score is less than the second threshold, the classification result has a poor credibility, and the type of the first pixel is set to invalid for supervision. That is, when used for supervised learning, the type of the first pixel is not learned as a supervisory label to avoid learning incorrect classification experience. For a first pixel whose first confidence score is greater than the second threshold, the classification result has a high credibility, and its type can be used as a label for supervised learning. Therefore, the updated second pseudo-label data can be used as the first label data corresponding to the first image data.

[0068] Among them, the second pseudo-label data can be updated pseudo-label data that only includes the target image in the first image data whose second confidence is less than the first threshold, or it can be updated pseudo-label data that includes both the aforementioned target image and the first pseudo-label data corresponding to other images in the first image data. The specific setting can be based on actual needs.

[0069] The present disclosure filters the first image data based on the second confidence level and the first threshold corresponding to the first image data, filters out the target image whose second confidence level is less than the first threshold, and uses it as a difficult sample image with higher value. The type of the first pixel corresponding to the target image in the first pseudo-label data is updated according to the first confidence level and the second threshold, and the type of the first pixel whose first confidence level is less than the second threshold is updated to the first type. After the update, the second pseudo-label data is obtained, so that the type of the first pixel with a smaller confidence level is not used for supervised learning, thereby effectively avoiding learning incorrect classification experience and improving the accuracy and effectiveness of the determined label.

[0070] In an alternative example, Figure 4 2 is a flow chart of step 201 provided by an exemplary embodiment of the present disclosure. In this example, the first image data is difficult sample image data; step 201 may specifically include the following steps:

[0071] Step 2011: Acquire unlabeled image data.

[0072] The unlabeled image data may be image data without labels obtained in any manner, and the unlabeled image data may include one or more images. For example, a large number of unlabeled images may be obtained as the unlabeled image data.

[0073] Step 2012: Determine first image data based on the unlabeled image data.

[0074] Any feasible method may be used to mine the first image data belonging to the difficult sample from the unlabeled image data.

[0075] For example, based on predictions from multiple semantic segmentation models, images with widely varying prediction results can be used as difficult samples to form the first image data. Another example is based on predictions from a single semantic segmentation model, images with low confidence levels can be used as difficult samples to form the first image data. Alternatively, a combination of multiple methods can be used to form the first image data. The specific settings can be tailored to actual needs.

[0076] The present disclosure mines out difficult sample image data with high value from a large amount of unlabeled image data as the first image data for subsequent automatic labeling, thereby further improving work efficiency.

[0077] In an alternative example, Figure 5 FIG2 is a flow chart of step 2012 provided by an exemplary embodiment of the present disclosure. In this example, step 2012 determines first image data based on unlabeled image data, including:

[0078] Step 20121a: Based on the first semantic segmentation model obtained through training, the unlabeled image data is predicted to obtain a second prediction result corresponding to the unlabeled image data; the second prediction result includes a first probability that each second pixel in the unlabeled image data belongs to each type.

[0079] The first semantic segmentation model can be any implementable semantic segmentation model, with no limitation on the specific network architecture, such as semantic segmentation models based on FCN (Fully Convolutional Networks) and their series, semantic segmentation models based on UNet and their series, semantic segmentation models based on DeepLab and their series, etc., and can be set according to actual needs. The specific prediction principle will not be elaborated here.

[0080] Step 20122a: Based on the second semantic segmentation model obtained through training, the unlabeled image data is predicted to obtain a third prediction result corresponding to the unlabeled image data; the third prediction result includes a second probability that each second pixel in the unlabeled image data belongs to each type.

[0081] The second semantic segmentation model is a semantic segmentation model different from the first semantic segmentation model. The difference can be a different network structure or the same network structure but different network parameters. The specific setting can be based on actual needs and is not limited in this disclosure. The specific prediction principle is not repeated here.

[0082] Among them, steps 20121a and 20122a are performed in no particular order.

[0083] Step 20123a, determine the corresponding third pseudo-label data based on the second prediction result, and determine the corresponding fourth pseudo-label data based on the third prediction result; the third pseudo-label data includes the type of each second pixel corresponding to the second prediction result; the fourth pseudo-label data includes the type of each second pixel corresponding to the third prediction result.

[0084] The determination principle of the third pseudo label data and the fourth pseudo label data is similar to that of the first pseudo label data, and will not be repeated here.

[0085] Step 20124a: Determine a difference value between the third pseudo-label data and the fourth pseudo-label data; the difference value is an IOU value and / or a pixel difference number.

[0086] The IOU value is obtained by IOU (Intersection Over Union) calculation, and the number of pixel differences refers to the number of pixels at the same position that do not belong to the same type.

[0087] In an optional example, determining an IOU value between the third pseudo-label data and the fourth pseudo-label data according to the third pseudo-label data includes:

[0088] For each type, determine the first number of second pixels at the same position in the third pseudo-label data and the fourth pseudo-label data that belong to the type, and the second number of pixels included in the union of the second pixels in the third pseudo-label data and the fourth pseudo-label data that belong to the type; take the ratio of the first number to the second number as the IOU value corresponding to the type, calculate the average of the IOU values ​​of each type, and obtain the IOU value of the third pseudo-label data and the fourth pseudo-label data.

[0089] Exemplarily, the types include people, animals, vehicles, and curbs. Taking people as an example, the number of second pixels belonging to people in the third pseudo-label data is 40, and the number of second pixels belonging to people in the fourth pseudo-label data is 50. Among them, there are 35 second pixels belonging to people in the third pseudo-label data and the fourth pseudo-label data that are located at the same position, that is, the first number is 35, and the second number is (40-35)+35+(50-35)=55. The IOU value corresponding to the type "people" is 35 / 55=0.636. Similarly, the IOU values ​​corresponding to other types can be obtained.

[0090] In an optional example, determining the number of pixel differences between the third pseudo-label data and the fourth pseudo-label data includes:

[0091] For each type, the number of pixel differences of different second pixel types at the same position under the type is determined, and then the number of pixel differences between the third pseudo label data and the fourth pseudo label data is obtained by averaging the number of pixel differences of each type.

[0092] Exemplarily, the types include people, animals, vehicles, and curbs. Taking people as an example, the number of second pixels belonging to people in the third pseudo-label data is 40, and the number of second pixels belonging to people in the fourth pseudo-label data is 50. Among them, 35 of the second pixels belonging to people in the third pseudo-label data and the fourth pseudo-label data are located at the same position. Therefore, the number of pixel differences between different types at the same position is (40-35)+(50-35)=20, that is, the number of pixel differences corresponding to the type "people" is 20. Similarly, the number of pixel differences corresponding to other types can be obtained, and the number of pixel differences between the third pseudo-label data and the fourth pseudo-label data can be obtained by taking the average.

[0093] Step 20125a: Determine the first image data based on the difference value.

[0094] Specifically, after determining the difference between the third pseudo-label data and the fourth pseudo-label data, images can be filtered based on the difference based on a preset filtering rule, thereby determining the first image data from the unlabeled image data. The preset filtering rule can be set based on actual needs and is not limited in this embodiment.

[0095] The present disclosure determines the first image data of difficult samples from the unlabeled image data by using the difference values ​​of the prediction results of different semantic segmentation models for the unlabeled image data, thereby realizing automatic mining of difficult samples and further improving work efficiency.

[0096] In an optional example, the unlabeled image data includes at least one unlabeled image; the difference value includes the difference value corresponding to each unlabeled image; step 20125a determines the first image data based on the difference value, including: for any unlabeled image in the unlabeled image data as the first unlabeled image, if the IOU value corresponding to the first unlabeled image is less than the IOU threshold and / or the number of pixel differences corresponding to the first unlabeled image is greater than the difference number threshold, then the first unlabeled image is used as the first image data.

[0097] Specifically, if the IOU value corresponding to the first unlabeled image is less than the IOU threshold, it means that the prediction results of different semantic segmentation models are quite different, that is, the first unlabeled image is not easy to be semantically segmented and has a higher learning value. Therefore, it can be added to the first image data as a difficult sample image. Similarly, if the number of pixel differences corresponding to the first unlabeled image is greater than the difference number threshold, it also means that the first unlabeled image is not easy to be semantically segmented and has a higher learning value. Therefore, it can be added to the first image data as a difficult sample image.

[0098] Based on the differences in prediction results of different semantic segmentation models, this paper mines difficult sample images with large differences and high learning value to form the first image data, which enters the subsequent automatic labeling process to further improve the effectiveness of the automatic labeling work, thereby providing more valuable labeled data for further optimization training of the semantic segmentation model to improve model performance.

[0099] In an alternative example, Figure 6 FIG2 is a flow chart of step 2012 provided in another exemplary embodiment of the present disclosure. In this example, the unlabeled image data includes at least one unlabeled image. Determining first image data based on the unlabeled image data in step 2012 includes:

[0100] Step 20121b: predict the unlabeled image data based on the third semantic segmentation model to obtain a fourth prediction result; the fourth prediction result includes the probability that each third pixel of each unlabeled image in the unlabeled image data belongs to each type.

[0101] The third semantic segmentation model can be the first semantic segmentation model or the second semantic segmentation model, or any other applicable semantic segmentation model. The third semantic segmentation model is also a pre-trained semantic segmentation model. The specific prediction principle will not be described in detail.

[0102] Step 20122b: Based on the fourth prediction result, determine the confidence level corresponding to each third pixel of each unlabeled image.

[0103] The principle of determining the confidence level in this step is described above and will not be repeated here.

[0104] Step 20123b: Determine the confidence level corresponding to each unlabeled image based on the confidence level corresponding to each third pixel of each unlabeled image.

[0105] In this step, the confidences corresponding to the unlabeled images can be obtained by averaging the confidences corresponding to the third pixels, which will not be described in detail.

[0106] Step 20124b: Determine the first image data based on the confidence corresponding to each unlabeled image and a preset confidence threshold.

[0107] Specifically, when the confidence level corresponding to the unlabeled image is lower than a preset confidence threshold, it indicates that the semantic segmentation credibility of the unlabeled image is poor, and therefore the image can be added to the first image data as a difficult sample.

[0108] The present disclosure mines the first image data of difficult samples from a large number of unlabeled images through the confidence of the semantic segmentation model prediction results, realizes the automatic mining of difficult samples, and improves the work efficiency of related mining work.

[0109] In an optional example, step 20122b determines the confidence corresponding to each third pixel of each unlabeled image based on the fourth prediction result, including: for a third pixel, taking the difference between the maximum probability corresponding to the third pixel and the second largest probability as the confidence corresponding to the third pixel.

[0110] Specifically, when the difference between the maximum probability of the third pixel and the second maximum probability is large, it means that the type of the third pixel can be determined more clearly, that is, the third pixel is more recognizable. When the difference between the maximum probability of the third pixel and the second maximum probability is small, it means that the type of the third pixel is not very clear and the recognizable is weak. Therefore, if the confidence of the unlabeled image determined based on the confidence of each third pixel is less than the preset confidence threshold, it means that the segmentability of the unlabeled image is poor, which has a high learning value for the optimization of the semantic segmentation model and can be used as a difficult sample.

[0111] In an optional example, the first semi-supervised semantic segmentation model is obtained as follows:

[0112] Based on the first image data, the pre-obtained second image data and the second label data corresponding to the second image data, the initial semi-supervised semantic segmentation model is trained to obtain a first semi-supervised semantic segmentation model.

[0113] Among them, the first image data is unlabeled image data, and the second image data is labeled image data with corresponding second label data obtained in advance through any feasible implementation method. The acquisition of the second image data and the corresponding second label data can be set according to actual needs and is not limited in this embodiment. The initial semi-supervised segmentation model is semi-supervisedly trained with unlabeled data and labeled data to obtain a first semi-supervised semantic segmentation model. The initial semi-supervised semantic segmentation model can be a semantic segmentation model obtained by preliminary supervised training based on labeled data.

[0114] In an alternative example, Figure 7 is a flow chart of a method for determining an image label provided by another exemplary embodiment of the present disclosure. In this example, after determining the first label data corresponding to the first image data based on the first confidence level and the first pseudo-label data corresponding to each first pixel in step 205, the method of the present disclosure further includes:

[0115] Step 206: Optimize the fourth semantic segmentation model based on the first image data and the first label data to obtain an optimized model corresponding to the fourth semantic segmentation model.

[0116] Among them, the fourth semantic segmentation model can be any implementable semantic segmentation model, and the fourth semantic segmentation model can be an untrained or trained semantic segmentation model. It can be set specifically according to actual needs. After obtaining the first label data corresponding to the first image data, the present disclosure can use it as training data for supervised training of the semantic segmentation model.

[0117] After obtaining the first label data corresponding to the first image data of the difficult sample, the present disclosure can use it to further optimize the semantic segmentation model, thereby effectively improving the model performance.

[0118] In an alternative example, Figure 8 FIG. 1 is a flow chart of a method for determining an image tag provided by another exemplary embodiment of the present disclosure. In this example, the method of the present disclosure includes:

[0119] 1. Obtain unlabeled image data.

[0120] 2. Determine first image data based on the unlabeled image data.

[0121] 3. Based on the first image data, the pre-obtained second image data and the second label data corresponding to the second image data, the initial semi-supervised semantic segmentation model is trained to obtain a first semi-supervised semantic segmentation model.

[0122] 4. Predict the first image data based on the first semi-supervised semantic segmentation model obtained through training to obtain a corresponding first prediction result; the first prediction result includes first probability data corresponding to the first image data, and the first probability data includes the probability that each first pixel in the first image data belongs to each type.

[0123] 5. Based on the first probability data, determine first pseudo label data corresponding to the first image data and first confidence levels corresponding to each first pixel, where the first pseudo label data includes the type to which each first pixel belongs.

[0124] 6. Determine a second confidence level corresponding to the first image data based on the first confidence levels corresponding to the first pixels.

[0125] 7. If the second confidence is less than the first threshold, the type of the first pixel in the first pseudo-label data whose first confidence is less than the second threshold is updated to the first type, and the second pseudo-label data corresponding to the first pseudo-label data is obtained. The first type indicates that the first pixel is invalid for supervision.

[0126] 8. Use the second pseudo-label data as the first label data corresponding to the first image data.

[0127] 9. Optimize the fourth semantic segmentation model based on the first image data and the first label data to obtain an optimized model corresponding to the fourth semantic segmentation model.

[0128] The specific operations of each step in this example can be found in the above embodiments or examples, and will not be described in detail again.

[0129] The present disclosure realizes automatic mining of valuable data from massive data through automatic mining of difficult samples based on semi-supervised semantic segmentation, automatic labeling of difficult samples, and further optimization training of semantic segmentation models based on labeled difficult samples, thereby obtaining difficult sample data for model training, performing semi-supervised semantic segmentation training on the mined difficult sample data and existing labeled data, automatically iterating the semi-supervised semantic segmentation model, obtaining an updated semantic segmentation model, obtaining pseudo-labels through confidence analysis, combining the pseudo-labels with the existing labeled data to perform optimization training of the supervised semantic segmentation model, realizing automatic optimization and updating of the model, and obtaining a better model without increasing the labeling cost. The second updated model can then be used to perform the above operations on massive unlabeled data, realizing a closed loop of data mining and model training, thereby continuously improving model performance.

[0130] Any of the methods for determining image tags provided in the embodiments of the present disclosure can be executed by any appropriate device with data processing capabilities, including but not limited to terminal devices and servers. Alternatively, any of the methods for determining image tags provided in the embodiments of the present disclosure can be executed by a processor, such as by invoking corresponding instructions stored in a memory to execute any of the methods for determining image tags mentioned in the embodiments of the present disclosure. This will not be further described below.

[0131] Exemplary devices

[0132] Figure 9 FIG is a schematic diagram of a structure of an apparatus for determining an image tag provided by an exemplary embodiment of the present disclosure. The apparatus of this embodiment can be used to implement the corresponding method embodiments of the present disclosure, such as Figure 9 The device shown includes: a first acquisition module 501 , a first processing module 502 , a second processing module 503 , a third processing module 504 and a fourth processing module 505 .

[0133] The first acquisition module 501 is configured to acquire first image data.

[0134] The first processing module 502 is used to predict the first image data obtained by the first acquisition module 501 based on the first semi-supervised semantic segmentation model obtained through training to obtain a corresponding first prediction result; the first prediction result includes first probability data corresponding to the first image data, and the first probability data includes the probability that each first pixel in the first image data belongs to each type.

[0135] The second processing module 503 is configured to determine first pseudo label data corresponding to the first image data based on the first probability data obtained by the first processing module 502 , where the first pseudo label data includes the type to which each first pixel belongs.

[0136] The third processing module 504 is configured to determine a first confidence level corresponding to each first pixel based on the first probability data obtained by the first processing module 502 .

[0137] The fourth processing module 505 is configured to determine first label data corresponding to the first image data based on the first confidence values ​​corresponding to the first pixels obtained by the third processing module 504 and the first pseudo label data obtained by the second processing module 503 .

[0138] In an alternative example, Figure 10 FIG. 5 is a schematic diagram of the structure of a fourth processing module 505 provided by an exemplary embodiment of the present disclosure. In this example, the fourth processing module 505 includes: a first determining unit 5051 , a first processing unit 5052 , and a second processing unit 5053 .

[0139] The first determination unit 5051 is used to determine the second confidence corresponding to the first image data based on the first confidence corresponding to each first pixel obtained by the third processing module 504; the first processing unit 5052 is used to update the type of the first pixel in the first pseudo-label data whose first confidence is less than the second threshold to the first type if the second confidence obtained by the first determination unit 5051 is less than the first threshold, and obtain second pseudo-label data corresponding to the first pseudo-label data, the first type indicating that the first pixel is invalid for supervision; the second processing unit 5053 is used to use the second pseudo-label data obtained by the first processing unit 5052 as the first label data corresponding to the first image data.

[0140] In an alternative example, Figure 11 Figure 5 is a schematic diagram of the structure of a first acquisition module 501 provided in an exemplary embodiment of the present disclosure. In this example, the first image data is difficult sample image data; the first acquisition module 501 includes a first acquisition unit 5011 and a third processing unit 5012. The first acquisition unit 5011 is configured to acquire unlabeled image data; the third processing unit 5012 is configured to determine the first image data based on the unlabeled image data acquired by the first acquisition unit 5011.

[0141] In an alternative example, Figure 12 FIG2 is a schematic diagram of the structure of a third processing unit 5012 provided by an exemplary embodiment of the present disclosure. In this example, the third processing unit 5012 includes: a first prediction subunit 50121, a second prediction subunit 50122, a first processing subunit 50123, a second processing subunit 50124, and a first determination subunit 50125.

[0142] The first prediction subunit 50121 is used to predict the unlabeled image data obtained by the first acquisition unit 5011 based on the first semantic segmentation model obtained through training, and obtain a second prediction result corresponding to the unlabeled image data; the second prediction result includes a first probability that each second pixel in the unlabeled image data belongs to each type; the second prediction subunit 50122 is used to predict the unlabeled image data obtained by the first acquisition unit 5011 based on the second semantic segmentation model obtained through training, and obtain a third prediction result corresponding to the unlabeled image data; the third prediction result includes a second probability that each second pixel in the unlabeled image data belongs to each type; the first processing subunit 50123 is used to predict the unlabeled image data obtained by the first prediction subunit 50124 based on the first semantic segmentation model obtained through training, and obtain a third prediction result corresponding to the unlabeled image data; the third prediction result includes a second probability that each second pixel in the unlabeled image data belongs to each type; The second prediction result obtained by 50121 determines the corresponding third pseudo-label data, and determines the corresponding fourth pseudo-label data according to the third prediction result obtained by the second prediction sub-unit 50122; the third pseudo-label data includes the type to which each second pixel corresponding to the second prediction result belongs; the fourth pseudo-label data includes the type to which each second pixel corresponding to the third prediction result belongs; the second processing sub-unit 50124 is used to determine the difference value between the third pseudo-label data and the fourth pseudo-label data obtained by the first processing sub-unit 50123; the difference value is the IOU value and / or the number of pixel differences; the first determination sub-unit 50125 is used to determine the first image data according to the difference value obtained by the second processing sub-unit 50124.

[0143] In an optional example, the unlabeled image data includes at least one unlabeled image; the difference value includes the difference value corresponding to each unlabeled image; the first determination subunit 50125 is specifically used to: for any unlabeled image in the unlabeled image data as the first unlabeled image, if the IOU value corresponding to the first unlabeled image is less than the IOU threshold and / or the number of pixel differences corresponding to the first unlabeled image is greater than the difference number threshold, then the first unlabeled image is used as the first image data.

[0144] In an alternative example, Figure 13 FIG2 is a schematic diagram of the structure of a third processing unit 5012 provided in another exemplary embodiment of the present disclosure. In this example, the unlabeled image data includes at least one unlabeled image; the third processing unit 5012 includes a third prediction subunit 50126, a second determination subunit 50127, a third determination subunit 50128, and a fourth determination subunit 50129.

[0145] The third prediction subunit 50126 is used to predict the unlabeled image data obtained by the first acquisition unit 5011 based on the third semantic segmentation model to obtain a fourth prediction result; the fourth prediction result includes the probability that each third pixel of each unlabeled image in the unlabeled image data belongs to each type; the second determination subunit 50127 is used to determine the confidence corresponding to each third pixel of each unlabeled image based on the fourth prediction result obtained by the third prediction subunit 50126; the third determination subunit 50128 is used to determine the confidence corresponding to each unlabeled image based on the confidence corresponding to each third pixel of each unlabeled image obtained by the second determination subunit 50127; the fourth determination subunit 50129 is used to determine the first image data based on the confidence corresponding to each unlabeled image obtained by the third determination subunit 50128 and a preset confidence threshold.

[0146] In an optional example, the second determining subunit 50127 is specifically configured to:

[0147] For a third pixel, the difference between the maximum probability corresponding to the third pixel and the second maximum probability is used as the confidence corresponding to the third pixel.

[0148] In an optional example, the first semi-supervised semantic segmentation model is obtained by training the initial semi-supervised semantic segmentation model based on the first image data, the pre-obtained second image data and the second label data corresponding to the second image data to obtain the first semi-supervised semantic segmentation model.

[0149] In an alternative example, Figure 14 It is a structural diagram of an image label determination device provided by another exemplary embodiment of the present disclosure. In this example, the device of the present disclosure also includes: a model optimization module 506, which is used to optimize the fourth semantic segmentation model based on the first image data and the first label data to obtain an optimized model corresponding to the fourth semantic segmentation model.

[0150] Exemplary electronic devices

[0151] An embodiment of the present disclosure further provides an electronic device, comprising: a memory for storing a computer program;

[0152] The processor is configured to execute the computer program stored in the memory, and when the computer program is executed, implements the method for determining the image tag described in any one of the above embodiments of the present disclosure.

[0153] Figure 15 FIG. 1 is a schematic diagram of a structure of an application embodiment of an electronic device disclosed in the present invention. In this embodiment, the electronic device 10 includes one or more processors 11 and a memory 12.

[0154] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0155] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute the program instructions to implement the image tag determination method of the various embodiments of the present disclosure described above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.

[0156] In one example, the electronic device 10 may further include an input device 13 and an output device 14 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0157] For example, the input device 13 may be the aforementioned microphone or microphone array, used to capture input signals from a sound source.

[0158] In addition, the input device 13 may also include, for example, a keyboard, a mouse, and the like.

[0159] The output device 14 can output various information to the outside, including determined distance information, direction information, etc. The output device 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0160] Of course, to simplify, Figure 15 Only some of the components related to the present disclosure in the electronic device 10 are shown, and components such as buses, input / output interfaces, etc. are omitted. In addition, the electronic device 10 may further include any other appropriate components according to specific application scenarios.

[0161] Exemplary computer program products and computer-readable storage media

[0162] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to perform the steps of the method according to various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of this specification.

[0163] The computer program product may be written in any combination of one or more programming languages ​​to implement the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0164] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.

[0165] The computer-readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, include but is not limited to a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0166] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this disclosure are merely illustrative and not restrictive, and should not be construed as necessarily possessed by each embodiment of the present disclosure. Furthermore, the specific details disclosed above are provided for illustrative purposes and to facilitate understanding, rather than as limitations. These details do not limit the present disclosure to necessarily being implemented using these specific details.

[0167] Each embodiment in this specification is described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. References to the same or similar parts between the various embodiments are sufficient. For system embodiments, since they largely correspond to method embodiments, their description is relatively simple. For relevant parts, references to the description of the method embodiments are sufficient.

[0168] The block diagrams of the devices, devices, equipment, and systems involved in this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "include," "comprise," "have," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0169] The methods and apparatus of the present disclosure may be implemented in many ways. For example, the methods and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless otherwise specified. In addition, in some embodiments, the present disclosure may also be implemented as programs recorded in a recording medium, which include machine-readable instructions for implementing the methods according to the present disclosure. Thus, the present disclosure also covers recording media that store programs for executing the methods according to the present disclosure.

[0170] It should also be noted that in the apparatus, device, and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0171] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0172] The above description has been provided for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for determining an image label, comprising: acquiring first image data; Predicting the first image data based on the first semi-supervised semantic segmentation model obtained through training to obtain a corresponding first prediction result; The first prediction result includes first probability data corresponding to the first image data, and the first probability data includes a probability that each first pixel in the first image data belongs to each type; determining, based on the first probability data, first pseudo-label data corresponding to the first image data, where the first pseudo-label data includes a type to which each of the first pixels belongs; Determining, based on the first probability data, first confidence scores corresponding to the first pixels, wherein the first confidence scores corresponding to the first pixels are used to determine whether the corresponding types of the first pixels are valid for supervision; First label data corresponding to the first image data is determined based on the first confidence levels and the first pseudo-label data corresponding to the first pixels.

2. The method according to claim 1, wherein The determining, based on the first confidence levels and the first pseudo-label data corresponding to the first pixels, first label data corresponding to the first image data includes: Determining a second confidence level corresponding to the first image data based on the first confidence levels corresponding to the first pixels; If the second confidence level is less than a first threshold, updating the type of the first pixel in the first pseudo-label data whose first confidence level is less than the second threshold level to the first type, and obtaining second pseudo-label data corresponding to the first pseudo-label data, wherein the first type indicates that the first pixel is invalid for supervision; The second pseudo label data is used as the first label data corresponding to the first image data.

3. The method according to claim 1, wherein The first image data is difficult sample image data; and obtaining the first image data includes: Obtain unlabeled image data; The first image data is determined based on the unlabeled image data.

4. The method according to claim 3, wherein: The determining the first image data based on the unlabeled image data includes: Based on the trained first semantic segmentation model, predicting the unlabeled image data to obtain a second prediction result corresponding to the unlabeled image data; the second prediction result includes a first probability that each second pixel in the unlabeled image data belongs to each type; performing prediction on the unlabeled image data based on the trained second semantic segmentation model to obtain a third prediction result corresponding to the unlabeled image data; the third prediction result including a second probability that each second pixel in the unlabeled image data belongs to each type; the second semantic segmentation model is different from the first semantic segmentation model; Determining third pseudo label data corresponding to the second prediction result, and determining fourth pseudo label data corresponding to the third prediction result; the third pseudo label data including the type of each second pixel corresponding to the second prediction result; and the fourth pseudo label data including the type of each second pixel corresponding to the third prediction result; Determine a difference value between the third pseudo-label data and the fourth pseudo-label data according to the third pseudo-label data; the difference value is an IOU value and / or a pixel difference number; The first image data is determined according to the difference value.

5. The method according to claim 4, wherein The unlabeled image data includes at least one unlabeled image; the difference value includes the difference value corresponding to each of the unlabeled images; The determining the first image data according to the difference value includes: For any unlabeled image in the unlabeled image data as the first unlabeled image, if the IOU value corresponding to the first unlabeled image is less than the IOU threshold and / or the number of pixel differences corresponding to the first unlabeled image is greater than the difference number threshold, the first unlabeled image is used as the first image data.

6. The method according to claim 3, wherein: The unlabeled image data includes at least one unlabeled image; The determining the first image data based on the unlabeled image data includes: Predicting the unlabeled image data based on the third semantic segmentation model to obtain a fourth prediction result; the fourth prediction result includes a probability that each third pixel of each unlabeled image in the unlabeled image data belongs to each type; Determining, based on the fourth prediction result, a confidence level corresponding to each of the third pixels of each of the unlabeled images; Determining the confidence level corresponding to each of the unlabeled images based on the confidence level corresponding to each of the third pixels of each of the unlabeled images; The first image data is determined based on the confidence levels corresponding to the unlabeled images and a preset confidence threshold.

7. The method according to claim 6, wherein: Determining, based on the fourth prediction result, the confidence corresponding to each of the third pixels of each of the unlabeled images, includes: For a third pixel, the difference between the maximum probability corresponding to the third pixel and the second maximum probability is used as the confidence corresponding to the third pixel.

8. The method according to claim 1, wherein The first semi-supervised semantic segmentation model is obtained by: Based on the first image data, the pre-obtained second image data and the second label data corresponding to the second image data, an initial semi-supervised semantic segmentation model is trained to obtain the first semi-supervised semantic segmentation model.

9. The method according to any one of claims 1 to 8, further comprising, after determining the first label data corresponding to the first image data based on the first confidence level corresponding to each first pixel and the first pseudo-label data: The fourth semantic segmentation model is optimized based on the first image data and the first label data to obtain an optimized model corresponding to the fourth semantic segmentation model.

10. A device for determining an image label, comprising: A first acquisition module, configured to acquire first image data; a first processing module, configured to predict the first image data based on the trained first semi-supervised semantic segmentation model to obtain a corresponding first prediction result; the first prediction result includes first probability data corresponding to the first image data, the first probability data including a probability that each first pixel in the first image data belongs to each type; a second processing module, configured to determine first pseudo label data corresponding to the first image data based on the first probability data, where the first pseudo label data includes a type to which each of the first pixels belongs; a third processing module, configured to determine, based on the first probability data, a first confidence level corresponding to each of the first pixels, wherein the first confidence level corresponding to each of the first pixels is used to determine whether the type corresponding to each of the first pixels is valid for supervision; The fourth processing module is configured to determine first label data corresponding to the first image data based on the first confidence levels corresponding to the first pixels and the first pseudo-label data.

11. A computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the method for determining an image tag according to any one of claims 1 to 9.

12. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method for determining an image tag according to any one of claims 1 to 9.

Images