Method, device and computer readable storage medium for image processing

Abstract

Embodiments of the present disclosure relate to a method, device and computer readable storage medium for image processing. A method of image processing includes obtaining a first image and at least one second image, wherein each of the at least one second image is a transformed image of the first image; and generating a first semantic segmentation label of the first image based on at least one semantic prediction result of the at least one second image. By training an image semantic segmentation model with the first image and the first semantic segmentation label as training data, the accuracy of the trained image semantic segmentation model is improved.

Description

Technical Field

[0001] Embodiments of this disclosure relate to the field of image processing, and more specifically, to methods, apparatus, and computer-readable storage media for image processing. Background Technology

[0002] Image semantic segmentation is a crucial technique in image processing and computer vision, forming the foundation of image understanding. It involves classifying each pixel in an image, determining its semantic category (e.g., belonging to background, person, or vehicle), and then dividing the image into regions according to these categories.

[0003] Typically, image semantic segmentation models (e.g., deep neural networks) are trained based on training datasets to achieve image semantic segmentation. A training sample in the training dataset can include a training image and a semantic segmentation label labeled for that training image. In some cases, the semantic segmentation labels labeled for the training image may be inaccurate, also known as "noise labels." If an image semantic segmentation model is trained using training samples with noisy labels, the trained image semantic segmentation model will struggle to produce accurate semantic segmentation results. Summary of the Invention

[0004] The embodiments of this disclosure provide a scheme for image processing.

[0005] In a first aspect of this disclosure, an image processing method is provided. The method includes: acquiring a first image and at least one second image, wherein each of the at least one second image is a transformed image of the first image; and generating a first semantic segmentation label for the first image based on at least one semantic prediction result of the at least one second image.

[0006] In a second aspect of this disclosure, a computing device is provided. The computing device includes at least one processor configured to: acquire a first image and at least one second image, wherein each of the at least one second image is a transformed image of the first image; and generate a first semantic segmentation label for the first image based on at least one semantic prediction result of the at least one second image.

[0007] In a third aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium has machine-executable instructions stored thereon, which, when executed by a device, cause the device to perform any step of the method described in the first aspect of this disclosure.

[0008] In a fourth aspect of this disclosure, a computer program product is provided. The computer program product is tangibly stored in a computer-readable storage medium and includes machine-executable instructions. When executed by a device, the machine-executable instructions cause the device to perform any step of the method described in the first aspect of this disclosure.

[0009] The summary section is provided to introduce a series of concepts in a simplified form, which will be further described in the detailed description below. The summary section is not intended to identify key or essential features of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The objects, advantages, and other features of the invention will become more apparent from the following disclosure and claims. A non-limiting description of preferred embodiments is given herein with reference to the accompanying drawings, in which:

[0011] Figure 1 A schematic block diagram of an image processing system according to an embodiment of the present disclosure is shown;

[0012] Figure 2 A schematic diagram of image processing according to an embodiment of the present disclosure is shown;

[0013] Figure 3 A schematic diagram of image processing according to an embodiment of the present disclosure is shown;

[0014] Figure 4 A flowchart illustrating an example method for image processing according to an embodiment of the present disclosure is shown;

[0015] Figure 5 A flowchart illustrating an example method for image processing according to an embodiment of the present disclosure is shown;

[0016] Figure 6 A flowchart illustrating an example model training method according to an embodiment of the present disclosure is shown;

[0017] Figure 7 A flowchart illustrating an example method for image processing according to embodiments of the present disclosure is shown; and

[0018] Figure 8 A schematic block diagram of an example computing device that can be used to implement embodiments of the present disclosure is shown.

[0019] In the various figures, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0020] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0021] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0022] As used herein, the term "circuit" can refer to hardware circuitry and / or a combination of hardware circuitry and software. For example, the circuitry can be a combination of analog and / or digital hardware circuitry with software / firmware. As another example, a circuit can be any part of a hardware processor (including a digital signal processor) with software, software, and memory, which work together to enable a device to perform various functions. In yet another example, a circuit can be hardware circuitry and / or a processor, such as a microprocessor or a portion thereof, which requires software / firmware to operate but may be absent when the software is not required to operate. As used herein, the term "circuit" also encompasses hardware circuitry or a processor alone, or a portion thereof and its accompanying software and / or firmware implementation.

[0023] As mentioned above, image semantic segmentation models (e.g., deep neural networks) are typically trained based on training datasets to achieve image semantic segmentation. A training sample in the training dataset can include a training image and a semantic segmentation label labeled for that training image. The semantic segmentation labels for the training images can be manually annotated. If the annotator lacks experience or the image itself is difficult to annotate, the semantic segmentation labels may be inaccurate. In some cases, the semantic segmentation labels for training images can be batch-annotated using certain devices, thereby reducing the cost of manual annotation. In these cases, the semantic segmentation labels for training images may also be inaccurate. Such inaccurate semantic segmentation labels are also called "noise labels." If an image semantic segmentation model is trained using training samples with noisy labels, the trained image semantic segmentation model will be unlikely to produce correct semantic segmentation results.

[0024] Embodiments of this disclosure propose a scheme for image processing to address the above-mentioned problems and / or other potential problems.

[0025] In some embodiments, a first image and at least one second image can be acquired, wherein each of the at least one second image is a transformed image of the first image. Then, based on at least one semantic prediction result from the at least one second image, a first semantic segmentation label for the first image is generated. Training the image semantic segmentation model using the first image and the first semantic segmentation label as training data helps improve the accuracy of the trained image semantic segmentation model.

[0026] Additionally or alternatively, in other embodiments, multiple images and a first set of semantic segmentation labels for the multiple images can be obtained. Furthermore, multiple image masks corresponding to the multiple images can be obtained, where each image mask is used to select a target region (e.g., a region with low noise) in the corresponding image among the multiple images. Then, based on the multiple image masks and the first set of semantic segmentation labels, a second set of semantic segmentation labels for the multiple images is generated; and based on the multiple images and the second set of semantic segmentation labels, a mixed image and its semantic segmentation label are generated. Training the image semantic segmentation model using the generated mixed image and its semantic segmentation labels as training data helps to further reduce the impact of noise labels on model training, thereby improving the accuracy of the trained image semantic segmentation model.

[0027] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings. In the embodiments of this disclosure, the term "model" is capable of processing inputs and providing corresponding outputs. Taking a neural network model as an example, it typically includes an input layer, an output layer, and one or more hidden layers between the input and output layers. Models used in deep learning applications (also called "deep learning models") typically include many hidden layers, thereby extending the depth of the network. The layers of a neural network model are sequentially connected such that the output of the previous layer is used as the input of the next layer, where the input layer receives the inputs of the neural network model, and the output of the output layer serves as the final output of the neural network model. Each layer of a neural network model includes one or more nodes (also called processing nodes or neurons), each node processing the input from the previous layer. In this document, the terms "neural network," "model," "network," and "neural network model" are used interchangeably.

[0028] Figure 1 A schematic block diagram of an image processing system 100 according to an embodiment of the present disclosure is shown. Figure 1 As shown, system 100 may include an image collection device 110, an image annotation device 120, an image processing device 130, a model training device 140, and a model application device 150. In some embodiments, the above-mentioned devices may be implemented in different physical devices. Alternatively, at least some of the above-mentioned devices may be implemented in the same physical device. It should be understood that... Figure 1 The system 100 shown is merely one example of an embodiment that can be implemented according to this disclosure and is not intended to limit the scope of this disclosure. The embodiments of this disclosure are equally applicable to other systems or architectures.

[0029] The image collection device 110 can collect an image set 101 for training the model. It should be understood that the images in the image set 101 can be of any format. Furthermore, the image collection device 110 can utilize any known or future developed method to collect the image set 101. The scope of this disclosure is not limited in this respect.

[0030] The image annotation device 120 can generate semantic segmentation labels 102 (also referred to herein as "initial semantic segmentation labels") for each image 101 to indicate the semantic category of each pixel in the image. The semantic category of a pixel refers to the type of object corresponding to that pixel in the image; objects in the image can include, but are not limited to, background, people, animals, vehicles, etc. For example, the generated semantic segmentation labels can be a vector or array indicating the true semantic category of each pixel, or they can be... Figure 1 The visualized image shows pixels of different semantic categories that can be identified using different colors. It should be understood that image annotation can be done manually. In this case, the image annotation device 120 can be a device or apparatus that is manually controlled to annotate each pixel in each image 101.

[0031] Image set 101 and its corresponding semantic segmentation label 102 can be provided to image processing device 130. Image processing device 130 can process image set 101 and its corresponding semantic segmentation label 102 to generate training dataset 103 including multiple training samples.

[0032] In some embodiments, the image processing apparatus 130 may perform image augmentation on image 101 (hereinafter also referred to as "first image") in the image set 101 to obtain at least one transformed image of the first image (hereinafter also referred to as "at least one second image"). The image processing apparatus 130 may determine at least one semantic prediction result for the at least one second image. For example, for each of the at least one transformed image, the image processing apparatus 130 may generate a semantic segmentation label or a semantic confidence label for that transformed image as a semantic prediction result for that transformed image, wherein the semantic confidence label may, for example, indicate the probability that each pixel in the transformed image belongs to a plurality of predetermined semantic categories. The image processing apparatus 130 may generate a semantic segmentation label for the first image 101 based on at least one semantic prediction result. For example, the first image 101 and the generated semantic segmentation label may be used as a training sample in the training dataset 103.

[0033] Additionally or alternatively, in some embodiments, the image processing apparatus 130 may acquire a plurality of images 101 from the image set 101. The image processing apparatus 130 may acquire a first set of semantic segmentation labels for the plurality of images 101. For example, one semantic segmentation label in the first set of semantic segmentation labels may be an initial semantic segmentation label 102 for the corresponding image 101 among the plurality of images 101. As another example, one semantic segmentation label in the first set of semantic segmentation labels may be a semantic segmentation label generated based on at least one semantic prediction result of at least one transformed image of the corresponding image 101, as described above. The image processing apparatus 130 may acquire a plurality of image masks corresponding to the plurality of images 101, each image mask used to select a target region in the corresponding image 101. The target region may include multiple pixels in the image 101 whose semantic category is relatively easy to predict, i.e., regions with low noise levels, also referred to herein as "non-noise regions". The image processing apparatus 130 may generate a second set of semantic segmentation labels for the plurality of images 101 based on the plurality of image masks and the first set of semantic segmentation labels. For example, for each of the plurality of images 101, the image processing device 130 can perform a masking operation (i.e., dot product) between the image mask corresponding to that image and the semantic segmentation label corresponding to that image in the first set of semantic segmentation labels to select the pixels in the semantic segmentation label that correspond to the target region, while ignoring the pixels that do not belong to the target region, thereby obtaining the semantic segmentation label corresponding to that image in the second set of semantic segmentation labels. Then, the image processing device 130 can generate a hybrid image and its semantic segmentation label based on the plurality of images 101 and the second set of semantic segmentation labels. For example, the generated hybrid image and its semantic segmentation label can be used as a training sample in the training dataset 103.

[0034] Additionally or alternatively, in some embodiments, the image processing apparatus 130 may also use each original image 101 and its corresponding initial semantic segmentation label 102 as a training sample in the training dataset 103.

[0035] In this way, the image processing device 130 can generate a training dataset 103 that includes multiple training samples. The training dataset 103 can be provided to the model training device 140 for training the target semantic segmentation model 104.

[0036] The target semantic segmentation model 104 can be trained to predict the semantic category of each pixel in an input image. For example, the model training device 140 can train the model based on the training dataset 103 to determine the model parameters of the target semantic segmentation model 104. The trained target semantic segmentation model 104 (e.g., the determined model parameters) can be provided to the model application device 150. The model application device 150 can acquire an input image 105 and use the trained target semantic segmentation model 104 to predict the semantic category of each pixel in the input image 105. The model application device 150 can generate a semantic segmentation label 106 indicating the semantic category of the input image 105 based on the prediction results. For example, the generated semantic segmentation label 106 can be a vector or array indicating the semantic category of each pixel in the input image 105, or it can be such as Figure 1 The visualization shown illustrates how pixels of different semantic categories can be filled with different colors.

[0037] The target semantic segmentation model 104 can be applied to various fields. In the medical field, the image semantic segmentation model can perform semantic segmentation on various input medical images to identify and understand lesions, thereby helping doctors make disease diagnoses. For example, a medical image processing method may include: acquiring an input medical image; using a trained image semantic segmentation model to generate a semantic segmentation label for the medical image, wherein the semantic segmentation label indicates the semantic category of pixels in the medical image; and identifying lesions in the medical image based on the semantic segmentation label. In the field of autonomous driving, the image semantic segmentation model can be used to perform semantic segmentation on images captured by a vehicle during driving, thereby identifying and understanding street scenes to help autonomous driving systems make driving decisions, and so on. For example, an autonomous driving method may include: acquiring an image captured by a vehicle during driving; using a trained image semantic segmentation model to generate a semantic segmentation label for the image, wherein the semantic segmentation label indicates the semantic category of pixels in the image; identifying street scenes in the image based on the semantic segmentation label; and generating autonomous driving decisions based on the identification results.

[0038] The following will be further combined Figure 2 and Figure 3 To describe in detail, such as Figure 1 The image processing apparatus 130 shown performs the operations. It should be understood that the image processing apparatus 130 may only perform operations such as... Figure 2 The image processing operations shown, or simply performing such... Figure 3 The image processing operations shown, or first perform as follows Figure 2 The image processing operation shown is then performed as follows: Figure 3 The image processing operations shown.

[0039] Figure 2 A schematic diagram of image processing according to an embodiment of the present disclosure is shown. Figure 2 The illustration shows multiple transformed images 201-1, 201-2...201-4 (collectively or individually referred to as "transformed images 201") obtained by performing image augmentation on images 101 acquired from image set 101. For example, image augmentation may include, but is not limited to, mirror transformation, rotation, scaling, cropping, translation, brightness modification, noise addition, and color transformation.

[0040] like Figure 2 As shown, the image processing device 130 can utilize the image semantic segmentation model 210 to predict the semantic category of pixels in each transformed image 201, thereby generating a semantic prediction result for each transformed image 201. The image semantic segmentation model 210 can be used with, for example... Figure 1 The target semantic segmentation model 104 shown may be the same model or a different model. For example, the image semantic segmentation model 210 may be a pre-trained image semantic segmentation model, a pre-trained image semantic segmentation model, or an image semantic segmentation model trained using training samples with noisy labels. That is, the image semantic segmentation model 210 may be implemented using a semantic segmentation model with low prediction accuracy, or other algorithms or devices capable of semantic segmentation. In some embodiments, the semantic prediction result of each transformed image 201 may be represented as a semantic segmentation label of the transformed image 201, indicating the semantic category to which each pixel in the transformed image 201 belongs. Alternatively, in other embodiments, the semantic prediction result of each transformed image 201 may be represented as a semantic confidence label of the transformed image 201, indicating the corresponding probability of each pixel in the transformed image 201 for multiple predetermined semantic categories. Figure 2 As shown, for example, the prediction result for transformed image 201-1 is represented as semantic confidence label 202-1, the prediction result for transformed image 201-2 is represented as semantic confidence label 202-2, the prediction result for transformed image 201-3 is represented as semantic confidence label 202-3, and the prediction result for transformed image 201-4 is represented as semantic confidence label 202-4. In the following text, semantic confidence labels 202-1 to 202-4 may be collectively referred to or individually as "semantic confidence label 202".

[0041] Image processing apparatus 130 can generate semantic segmentation labels 203 for the original image 101 based on semantic prediction results of multiple transformed images 201. For example, image processing apparatus 130 can first perform inverse transformations corresponding to image augmentation on multiple semantic confidence labels 202 of multiple transformed images 201, and then fuse the multiple inversely transformed semantic confidence labels to generate fused semantic confidence labels. The fused semantic confidence labels can indicate the confidence level of each pixel in the original image 101 belonging to multiple predetermined semantic categories. Then, image processing apparatus 130 can determine the semantic category with the highest confidence level for each pixel in the original image 101 as the semantic category to which the pixel belongs, thereby generating semantic segmentation labels 203. Optionally, in some embodiments, image processing apparatus 130 can also smooth the generated semantic segmentation labels 203 to obtain optimized semantic segmentation labels 204. It should be understood that the smoothing operation can be based on any known or future suitable algorithm, such as the Conditional Random Field (CRF) algorithm.

[0042] Figure 3 Multiple images 301-1 and 301-2 (e.g., obtained from image set 101) and their corresponding semantic segmentation labels 304-1 and 304-2 are shown. For example, semantic segmentation label 304-1 can be the initial semantic segmentation label of image 301-1, or according to... Figure 2 The semantic segmentation labels are generated in the manner shown. Semantic segmentation label 304-2 can be the initial semantic segmentation label of image 301-2, or generated as shown... Figure 2 The semantic segmentation labels are generated in the manner shown. Although... Figure 3 The processing shown is for only one pair of images and should be understood as being for illustrative purposes only and not implying any limitation on the scope of this disclosure. In some embodiments, the image processing apparatus 130 may acquire more than two images from the image set 101 for processing.

[0043] In some embodiments, such as Figure 3 As shown, for each of the multiple images 301-1 and 301-2, the image processing device 130 can use the image semantic segmentation model 310 to predict the corresponding semantic category of multiple pixels in that image. The image semantic segmentation model 310 can be the same model as the target semantic segmentation model 104 or a different model. For example, the image semantic segmentation model 310 can be a pre-trained image semantic segmentation model, a pre-trained image semantic segmentation model, or an untrained image semantic segmentation model. That is, the image semantic segmentation model 310 can be implemented using a semantic segmentation model with low prediction accuracy, or other algorithms or devices capable of semantic segmentation.

[0044] The image processing apparatus 130 can further determine the corresponding scores of multiple pixels in the image based on the semantic prediction results for each image, wherein the score of each pixel can indicate the degree of association between the pixel and a noisy region in the image. In other words, the score of each pixel can indicate the difficulty of predicting the semantic category of the pixel. Figure 3 As shown, the corresponding scores of multiple pixels in image 301-1 can be represented by heatmap 302-1, and the corresponding scores of multiple pixels in image 301-2 can be represented by heatmap 302-2.

[0045] In some embodiments, for each pixel in each image, the image processing apparatus 130 may determine a score for that pixel based on the difference between the pixel's predicted semantic category and its true semantic category, wherein the true semantic category of the pixel is indicated by the semantic segmentation label of the image. For example, the image semantic segmentation model 310 may predict the probability that each pixel in each image belongs to a different semantic category, while the semantic segmentation label of the image may indicate the true semantic category to which each pixel belongs. The image processing apparatus 130 may calculate the loss between the probability predicted by the image semantic segmentation model 310 that each pixel belongs to its true semantic category and a predetermined probability value (e.g., 100%) as the score for each pixel. That is, the higher the score of a pixel, the greater the difference between the pixel's predicted semantic category and its true semantic category.

[0046] Alternatively, in some embodiments, for each pixel in each image, the image processing apparatus 130 may determine a score for that pixel based on the degree of uncertainty or confusion (e.g., entropy) of the pixel's predicted semantic category. For example, the image semantic segmentation model 310 may predict the probability that each pixel in each image belongs to different semantic categories. The image processing apparatus 130 may utilize any known or suitable algorithm to determine the degree of uncertainty or confusion of the pixel's predicted semantic category based on the predicted probability of each pixel belonging to different semantic categories, as a score for that pixel. That is, the higher the score of a pixel, the higher the degree of uncertainty or confusion of the pixel's predicted semantic category.

[0047] In some embodiments, such as Figure 2 As shown, the image processing apparatus 130 can determine an image mask 303-1 corresponding to image 301-1 based on a heatmap 302-1 indicating the corresponding scores of multiple pixels in image 301-1. Similarly, the image processing apparatus 130 can determine an image mask 303-2 corresponding to image 301-2 based on a heatmap 302-2 indicating the corresponding scores of multiple pixels in image 301-2.

[0048] In some embodiments, the image processing device 130 may sort the scores of multiple pixels in each image from largest to smallest, and then select the bottom N% of pixels or pixels with scores below a threshold score n. For example, the value of N% or n can be determined based on a verification experiment. The image processing device 130 can generate an image mask for the image by setting the mask value of the selected pixel's location to 1 and the mask value of other locations to 0. Alternatively, in some embodiments, the image processing device 130 may sort the scores of all pixels in a batch of images from largest to smallest, and then select the bottom N% of pixels or pixels with scores below a threshold score n. For example, the value of N% or n can be determined based on a verification experiment. The image processing device 130 can generate an image mask for each image in the batch of images by setting the mask value of the selected pixel's location to 1 and the mask value of other locations to 0. Alternatively, in other embodiments, the image processing device 130 may also directly use a heatmap indicating the corresponding scores of multiple pixels in each image as the image mask for that image.

[0049] In some embodiments, such as Figure 2 As shown, the image processing apparatus 130 can perform a masking operation (i.e., dot product) between the image mask 303-1 and the semantic segmentation label 304-1 of the image 301-1 to generate a semantic segmentation label 305-1 for the image 301-1. Similarly, the image processing apparatus 130 can perform a masking operation (i.e., dot product) between the image mask 303-2 and the initial semantic segmentation label 304-2 of the image 301-2 to generate a semantic segmentation label 305-2 for the image 301-2. From Figure 2 As can be seen, the regenerated semantic segmentation labels 305-1 and 305-2 retain the semantic segmentation labels of those pixels in the target region (i.e., the region with low noise) in semantic segmentation labels 304-1 and 304-2, respectively, while ignoring the semantic segmentation labels of those pixels in the non-target region.

[0050] In some embodiments, such as Figure 3As shown, the image processing apparatus 130 can further generate a hybrid image 306 and its semantic segmentation label 307 based on multiple images 301-1 and 301-2 and the regenerated semantic segmentation labels 305-1 and 305-2. In some embodiments, the image processing apparatus 130 can perform a weighted summation of multiple images 301-1 and 301-2 based on their respective weights to obtain the hybrid image 306. The image processing apparatus 130 can also perform a weighted summation of semantic segmentation labels 305-1 and 305-2 based on their respective weights to obtain the semantic segmentation label 307 of the hybrid image 306. It should be understood that the image processing apparatus 130 can also utilize other methods to generate the hybrid image 306 and its semantic segmentation label 307. Figure 3 As shown, the hybrid image 306 and its semantic segmentation label 307 can be used as... Figure 1 A training sample from the training dataset 103 shown is used to train the target semantic segmentation model 104.

[0051] Figure 4 A flowchart of an example method 400 for image processing according to an embodiment of the present disclosure is shown. For example, method 400 may be performed by, for example... Figure 1 The image processing device 130 shown is used to perform this operation. The following is in conjunction with... Figure 1 and Figure 2 Method 400 is described herein. It should be understood that method 400 may also include additional boxes not shown and / or certain boxes shown may be omitted. The scope of this disclosure is not limited in this respect.

[0052] At box 410, acquire the first image and at least one second image (e.g., such as...). Figure 2 The transformed images shown are 201-1 to 201-4, wherein each of at least one of the second images is a transformed image of the first image.

[0053] At box 420, based on at least one semantic prediction result of at least one second image (e.g., such as...) Figure 2 Using the semantic confidence labels shown (202-1 to 202-4), generate the first semantic segmentation label for the first image (e.g., as shown in the image). Figure 2 The semantic segmentation labels 203 and / or 204 are shown.

[0054] In some embodiments, acquiring at least one second image includes: acquiring at least one second image by performing image augmentation on a first image. Image augmentation includes at least one of the following: mirror transformation, rotation, scaling, cropping, translation, brightness modification, adding noise, and color transformation.

[0055] In some embodiments, method 300 further includes: using an image semantic segmentation model to generate at least one semantic segmentation label or at least one semantic confidence label for at least one second image as at least one semantic prediction result, wherein the at least one semantic confidence label indicates the confidence level of each pixel in at least one second image belonging to a predetermined semantic category.

[0056] In some embodiments, generating a first semantic segmentation label includes: performing an inverse transform corresponding to image augmentation on at least one semantic prediction result to obtain at least one semantic prediction result after inverse transformation; and generating a first semantic segmentation label based on at least one semantic prediction result after inverse transformation.

[0057] Figure 5 A flowchart of an example method 500 for image processing according to an embodiment of the present disclosure is shown. For example, method 500 may be performed by, for example... Figure 1 The image processing device 130 shown is used to perform this operation. The following is in conjunction with... Figure 1 and Figure 2 Method 500 is described herein. It should be understood that method 500 may also include additional boxes not shown and / or certain boxes shown may be omitted. The scope of this disclosure is not limited in this respect.

[0058] At box 510, acquire multiple images (e.g., such as...). Figure 3 Images 301-1 and 301-2 shown) and the first set of semantic segmentation labels for multiple images (e.g., such as...) Figure 3 (Semantic segmentation labels 304-1 and 304-2 shown). In some embodiments, the plurality of images includes a first image and the first set of semantic segmentation labels includes the first semantic segmentation label of the first image.

[0059] At box 520, obtain multiple image masks corresponding to multiple images (e.g., such as...). Figure 3 Image masks 303-1 and 303-2 are shown, where each image mask is used to select a target region in a corresponding image among multiple images.

[0060] In some embodiments, obtaining multiple image masks corresponding to multiple images includes: determining corresponding predicted semantic categories of multiple pixels in a first image; determining corresponding scores of multiple pixels based on the corresponding predicted semantic categories of the multiple pixels, wherein the score of each pixel indicates the degree of association between the pixel and a noise region in the first image; and determining a first image mask of the first image based on the corresponding scores of the multiple pixels, wherein the first image mask is used to select at least a portion of the pixels with lower scores among the multiple pixels.

[0061] In some embodiments, determining the corresponding predicted semantic category of a plurality of pixels in a first image includes: using an image semantic segmentation model to determine the corresponding predicted semantic category of a plurality of pixels in a first image.

[0062] In some embodiments, determining the corresponding score of a plurality of pixels in a first image includes, for each of the plurality of pixels, determining the score of that pixel based on any of the following: the difference between the predicted semantic category of the pixel and the true semantic category of the pixel, wherein the true semantic category of the pixel is indicated by the semantic segmentation label of the first image; the uncertainty of the predicted semantic category of the pixel; and the degree of confusion of the predicted semantic category of the pixel.

[0063] At box 530, based on multiple image masks and the first set of semantic segmentation labels, a second set of semantic segmentation labels for multiple images is generated (e.g., such as...). Figure 3 The semantic segmentation labels shown are 305-1 and 305-2.

[0064] In some embodiments, the plurality of image masks includes a first image mask corresponding to the first image, and generating a second set of semantic segmentation labels includes: generating a second set of semantic segmentation labels for the first image by performing a masking operation between the first semantic segmentation labels of the first image and the first image mask, wherein the second set of semantic segmentation labels includes a second set of semantic segmentation labels.

[0065] At box 540, a hybrid image is generated based on multiple images and a second set of semantic segmentation labels (e.g., ...). Figure 3 The hybrid image 306 shown) and the semantic segmentation label of the hybrid image (e.g., such as Figure 3 The semantic segmentation label shown is 307.

[0066] In some embodiments, generating a hybrid image and a semantic segmentation label for the hybrid image includes: performing a weighted summation of multiple images based on their respective weights to generate a hybrid image; and performing a weighted summation of a second set of semantic segmentation labels based on the respective weights of the multiple images to generate a semantic segmentation label for the hybrid image.

[0067] Figure 6 A flowchart of an example model training method 600 according to an embodiment of the present disclosure is shown. For example, method 600 can be performed by, for example... Figure 1 The model training device 140 shown is used to perform this. The following is in conjunction with... Figure 1 and Figure 2 Method 400 is described herein. It should be understood that method 600 may also include additional boxes not shown and / or certain boxes shown may be omitted. The scope of this disclosure is not limited in this respect.

[0068] At box 610, training data is acquired. For example, model training device 140 can obtain training data from, for example, Figure 1 The training dataset 103 shown is used to acquire training data. In some embodiments, the acquired training data may include a first training sample, for example, the first training sample includes a first image obtained according to method 400 and a first semantic segmentation label of the first image. Additionally or alternatively, in some embodiments, the acquired training data may include a second training sample, for example, the second training sample includes a mixed image obtained according to method 500 and a semantic segmentation label of the mixed image.

[0069] At box 620, the target semantic segmentation model is trained based on the training data (e.g., ...). Figure 1 The target semantic segmentation model 104 shown is used to generate semantic segmentation labels for the input image. In some embodiments, the model training device 140 can train the model based on the acquired training data to determine the model parameters of the target semantic segmentation model 104.

[0070] Figure 7 A flowchart of an example method 700 for image processing according to an embodiment of the present disclosure is shown. For example, method 700 may be performed by, for example... Figure 1 The model shown is executed using device 150. The following is in conjunction with... Figure 1 and Figure 2 Method 700 is described herein. It should be understood that method 700 may also include additional boxes not shown and / or certain boxes shown may be omitted. The scope of this disclosure is not limited in this respect.

[0071] At box 710, obtain the input image (e.g., as shown in box 710). Figure 1 The input image 105 is shown. At box 720, the semantic segmentation label of the input image is generated using the trained target semantic segmentation model (e.g., as shown in box 720). Figure 1 The semantic segmentation label 106 shown indicates the semantic category of a pixel in the input image. For example, the generated semantic segmentation label 106 may be a vector or array indicating the semantic category of each pixel in the input image 105, or it may be as follows: Figure 1 The visualization shown illustrates how pixels of different semantic categories can be filled with different colors.

[0072] As can be seen from the above description, the embodiments of this disclosure can effectively reduce the impact of noisy labeled training data on model training, thereby improving the accuracy of the trained image semantic segmentation model. The embodiments of this disclosure are applicable to weakly supervised learning scenarios, where the initial training dataset may include limited, noisy, or inaccurately labeled training samples. The embodiments of this disclosure can optimize such training data and then use the optimized training data for model training.

[0073] In addition to being applicable to semantic segmentation tasks, embodiments of this disclosure are also applicable to instance segmentation tasks. As described above, the purpose of a semantic segmentation task is to identify the semantic category to which each pixel in an image belongs. Unlike semantic segmentation tasks, the purpose of instance segmentation tasks is to identify the instance category to which each pixel in an image belongs. For example, a semantic segmentation task can group pixels belonging to people in an image into the same category, while an instance segmentation task can group pixels belonging to different people in an image into different categories.

[0074] To apply the embodiments of this disclosure to an instance segmentation task, the initial semantic label 102 annotated for each image 101 needs to indicate the instance category to which each pixel in the image belongs. For example, if two pixels are semantically categorized as "person" but correspond to different people, then these two pixels will be labeled as belonging to different instance categories. Subsequent processing is similar to the processing described above for the semantic segmentation task and will not be repeated here.

[0075] Figure 8 A schematic block diagram of an example computing device 800 that can be used to implement embodiments of the present disclosure is shown. For example, as Figure 1 One or more devices in the system 100 shown may be implemented by device 800. Examples of device 800 may include any general-purpose computer, special-purpose computer, server, virtual machine, and any other computing-capable electronic device (such as user equipment, mobile phone, cellular phone, smartphone, personal digital assistant, image capture device (such as digital camera), gaming device, music storage and playback device, or Internet device that enables wireless or wired Internet access and browsing, etc.).

[0076] As shown in the figure, device 800 includes a central processing unit (CPU) 801 (also referred to as "processor 801"), which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 802 or loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 can also store various programs and data required for the operation of device 800. CPU 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0077] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0078] Processor 801 may be configured to perform the various processes and procedures described above, such as methods 400, 500, 600, and / or 700. For example, in some embodiments, methods 400, 500, 600, and / or 700 may be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by CPU 801, one or more steps of methods 400, 500, 600, and / or 700 described above may be performed.

[0079] In some embodiments, a computing device includes at least one circuit configured to: acquire a first image and at least one second image, wherein each of the at least one second image is a transformed image of the first image; and generate a first semantic segmentation label for the first image based on at least one semantic prediction result of the at least one second image.

[0080] In some embodiments, the at least one circuit is configured to acquire the at least one second image by performing image augmentation on the first image.

[0081] In some embodiments, the image augmentation includes at least one of the following: mirror transformation, rotation, scaling, cropping, translation, brightness modification, adding noise, and color transformation.

[0082] In some embodiments, the at least one circuit is further configured to: generate at least one semantic segmentation label or at least one semantic confidence label for the at least one second image using an image semantic segmentation model, as the at least one semantic prediction result, wherein the at least one semantic confidence label indicates the confidence level of each pixel in the at least one second image belonging to a predetermined semantic category.

[0083] In some embodiments, the at least one circuit is configured to: perform an inverse transform corresponding to the image augmentation on the at least one semantic prediction result to obtain at least one semantic prediction result after inverse transformation; and generate the first semantic segmentation label based on the at least one semantic prediction result after inverse transformation.

[0084] In some embodiments, the at least one circuit is further configured to: acquire a plurality of images and a first set of semantic segmentation labels for the plurality of images, wherein the plurality of images include the first image and the first set of semantic segmentation labels includes the first semantic segmentation label; acquire a plurality of image masks corresponding to the plurality of images, wherein each image mask is used to select a target region in a corresponding image among the plurality of images; generate a second set of semantic segmentation labels for the plurality of images based on the plurality of image masks and the first set of semantic segmentation labels; and generate a hybrid image and semantic segmentation labels for the hybrid image based on the plurality of images and the second set of semantic segmentation labels.

[0085] In some embodiments, the at least one circuit is configured to: determine a corresponding predicted semantic category for a plurality of pixels in the first image; determine a corresponding score for the plurality of pixels based on the corresponding predicted semantic category, wherein the score of each pixel indicates the degree of association between the pixel and a noise region in the first image; and determine a first image mask for the first image based on the corresponding scores of the plurality of pixels, wherein the first image mask is used to select at least a subset of pixels with lower scores among the plurality of pixels.

[0086] In some embodiments, the at least one circuit is configured to: determine the corresponding predicted semantic category of the plurality of pixels in the first image using an image semantic segmentation model.

[0087] In some embodiments, the at least one circuit is configured to determine a score for each of the plurality of pixels based on any of the following: the difference between the predicted semantic category of the pixel and the true semantic category of the pixel, wherein the true semantic category of the pixel is indicated by the semantic segmentation label of the first image; the uncertainty of the predicted semantic category of the pixel; and the degree of confusion of the predicted semantic category of the pixel.

[0088] In some embodiments, the plurality of image masks includes a first image mask corresponding to the first image, and the at least one circuit is configured to generate a second semantic segmentation label for the first image by performing a masking operation between the first semantic segmentation label and the first image mask, wherein the second set of semantic segmentation labels includes the second semantic segmentation label.

[0089] In some embodiments, the at least one circuit is configured to: perform a weighted summation on the plurality of images based on their respective weights to generate the hybrid image; and perform a weighted summation on the second set of semantic segmentation labels based on their respective weights to generate semantic segmentation labels for the hybrid image.

[0090] In some embodiments, the at least one circuit is further configured to: acquire training data, wherein the training data includes at least one of the following: a first training sample including the first image and the first semantic segmentation label, and a second training sample including the mixed image and the semantic segmentation label of the mixed image; and train a target semantic segmentation model based on the training data to generate semantic segmentation labels for the input image.

[0091] In some embodiments, the at least one circuit is further configured to: acquire the input image; and generate semantic segmentation labels for the input image using a trained target semantic segmentation model.

[0092] This disclosure can be implemented as a system, method, and / or computer program product. When this disclosure is implemented as a system, in addition to being able to be implemented on a single device, the components described herein can also be implemented in the form of a cloud computing architecture. In a cloud computing environment, these components can be remotely deployed and can work together to achieve the functions described herein. Cloud computing can provide computing, software, data access, and storage services without requiring end users to know the physical location or configuration of the systems or hardware providing these services. Cloud computing can provide services over a wide area network (such as the Internet) using appropriate protocols. For example, cloud computing providers offer applications over a wide area network, and they can be accessed through a browser or any other computing component. The components of cloud computing, along with the corresponding data, can be stored on remote servers. Computing resources in a cloud computing environment can be consolidated at a remote data center location, or these computing resources can be distributed. Cloud computing infrastructure can provide services through a shared data center, even if they appear as a single access point for users. Therefore, the various functions described herein can be provided from remote service providers using a cloud computing architecture. Alternatively, they can be provided from conventional servers, or they can be installed directly or otherwise on client devices. Furthermore, this disclosure can also be implemented as a computer program product, which may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of this disclosure.

[0093] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0094] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0095] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0096] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0097] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0098] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0099] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0100] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. An image processing method, comprising: Acquire a first image and at least one second image, wherein each of the at least one second image is a transformed image of the first image; Based on at least one semantic prediction result of the at least one second image, a first semantic segmentation label is generated for the first image; Obtain multiple images and a first set of semantic segmentation labels for the multiple images, wherein the multiple images include the first image and the first set of semantic segmentation labels includes the first semantic segmentation label; Obtaining multiple image masks corresponding to the plurality of images, wherein each image mask is used to select a target region in a corresponding image among the plurality of images, wherein the target region includes a region with low noise level in the corresponding image, and obtaining multiple image masks corresponding to the plurality of images includes: Determine the corresponding predicted semantic category of multiple pixels in the first image; Based on the corresponding predicted semantic categories of the plurality of pixels, a corresponding score is determined for each pixel, wherein the score of each pixel indicates the degree of association between that pixel and a noise region in the first image; and Based on the corresponding scores of the plurality of pixels, a first image mask is determined for the first image, wherein the first image mask is used to select at least a portion of the pixels with lower scores among the plurality of pixels; Based on the multiple image masks and the first set of semantic segmentation labels, a second set of semantic segmentation labels for the multiple images is generated; Based on the multiple images and the second set of semantic segmentation labels, a hybrid image and semantic segmentation labels for the hybrid image are generated; Acquire training data, wherein the training data includes training samples, and the training samples include the blended image and semantic segmentation labels of the blended image; and The target semantic segmentation model is trained based on the training data to generate semantic segmentation labels for the input image.

2. The method according to claim 1, wherein acquiring the at least one second image comprises: The at least one second image is obtained by performing image augmentation on the first image.

3. The method according to claim 2, wherein the image augmentation includes at least one of the following: mirror transformation, rotation, scaling, cropping, translation, brightness modification, noise addition, and color transformation.

4. The method according to claim 1, further comprising: Using an image semantic segmentation model, at least one semantic segmentation label or at least one semantic confidence label is generated for the at least one second image, serving as the at least one semantic prediction result. The at least one semantic confidence label indicates the confidence level of each pixel in the at least one second image belonging to a predetermined semantic category.

5. The method according to claim 2, wherein generating the first semantic segmentation tag comprises: Perform an inverse transformation corresponding to the image augmentation on each of the at least one semantic prediction results to obtain at least one semantic prediction result after inverse transformation; as well as The first semantic segmentation label is generated based on at least one semantic prediction result after inverse transformation.

6. The method of claim 1, wherein determining the corresponding predicted semantic category of a plurality of pixels in the first image comprises: Using an image semantic segmentation model, the corresponding predicted semantic category of the plurality of pixels in the first image is determined.

7. The method of claim 1, wherein determining the corresponding scores of a plurality of pixels in the first image comprises: For each of the plurality of pixels, a score is determined based on any of the following: The difference between the predicted semantic category of the pixel and the true semantic category of the pixel, wherein the true semantic category of the pixel is indicated by the semantic segmentation label of the first image; The uncertainty of the predicted semantic category of the pixel; or The degree of confusion in the predicted semantic category of the pixel.

8. The method according to claim 1, wherein the plurality of image masks includes a first image mask corresponding to the first image, and generating the second set of semantic segmentation labels includes: By performing a masking operation between the first semantic segmentation label and the first image mask, a second semantic segmentation label for the first image is generated, wherein the second set of semantic segmentation labels includes the second semantic segmentation label.

9. The method of claim 1, wherein generating the blended image and the semantic segmentation labels of the blended image comprises: Based on the corresponding weights of the multiple images, the multiple images are weighted and summed to generate the hybrid image; as well as Based on the corresponding weights of the multiple images, the second set of semantic segmentation labels are weighted and summed to generate the semantic segmentation labels of the hybrid image.

10. The method of claim 1, wherein the training data further comprises: The other training samples include the first image and the first semantic segmentation label.

11. The method of claim 10, further comprising: Obtain the input image; as well as Using a trained target semantic segmentation model, semantic segmentation labels for the input image are generated.

12. A computing device, comprising: At least one processor is configured as follows: Acquire a first image and at least one second image, wherein each of the at least one second image is a transformed image of the first image; Based on at least one semantic prediction result of the at least one second image, a first semantic segmentation label is generated for the first image; The processor is configured to acquire multiple images and a first set of semantic segmentation labels for the multiple images, wherein the multiple images include the first image and the first set of semantic segmentation labels includes the first semantic segmentation label, and to acquire multiple image masks corresponding to the multiple images by: Determine the corresponding predicted semantic category of multiple pixels in the first image; Based on the corresponding predicted semantic categories of the plurality of pixels, the corresponding scores of the plurality of pixels are determined, wherein the score of each pixel indicates the degree of association between the pixel and the noise region in the first image; as well as Based on the corresponding scores of the plurality of pixels, a first image mask is determined for the first image, wherein the first image mask is used to select at least a portion of the pixels with lower scores among the plurality of pixels; Obtain multiple image masks corresponding to the plurality of images, wherein each image mask is used to select a target region in a corresponding image among the plurality of images, wherein the target region includes a region with low noise level in the corresponding image; Based on the multiple image masks and the first set of semantic segmentation labels, a second set of semantic segmentation labels for the multiple images is generated; Based on the multiple images and the second set of semantic segmentation labels, a hybrid image and semantic segmentation labels for the hybrid image are generated; Acquire training data, wherein the training data includes training samples, and the training samples include the mixed image and the semantic segmentation labels of the mixed image; as well as The target semantic segmentation model is trained based on the training data to generate semantic segmentation labels for the input image.

13. The computing device of claim 12, wherein the at least one processor is configured to: The at least one second image is obtained by performing image augmentation on the first image.

14. The computing device of claim 13, wherein the image augmentation includes at least one of the following: mirror transformation, rotation, scaling, cropping, translation, brightness modification, noise addition, and color transformation.

15. The computing device of claim 12, wherein the at least one processor is further configured to: Using an image semantic segmentation model, at least one semantic segmentation label or at least one semantic confidence label is generated for the at least one second image, serving as the at least one semantic prediction result. The at least one semantic confidence label indicates the confidence level of each pixel in the at least one second image belonging to a predetermined semantic category.

16. The computing device of claim 13, wherein the at least one processor is configured to: Perform an inverse transform corresponding to the image augmentation on each of the at least one semantic prediction results to obtain at least one semantic prediction result after the inverse transform; and The first semantic segmentation label is generated based on at least one semantic prediction result after inverse transformation.

17. The computing device of claim 12, wherein the at least one processor is configured to: Using an image semantic segmentation model, the corresponding predicted semantic category of the plurality of pixels in the first image is determined.

18. The computing device of claim 12, wherein the at least one processor is configured to: For each of the plurality of pixels, a score is determined based on any of the following: The difference between the predicted semantic category of the pixel and the true semantic category of the pixel, wherein the true semantic category of the pixel is indicated by the semantic segmentation label of the first image; The uncertainty of the predicted semantic category of the pixel; or The degree of confusion in the predicted semantic category of the pixel.

19. The computing device of claim 12, wherein the plurality of image masks includes a first image mask corresponding to the first image, and the at least one processor is configured to: By performing a masking operation between the first semantic segmentation label and the first image mask, a second semantic segmentation label for the first image is generated, wherein the second set of semantic segmentation labels includes the second semantic segmentation label.

20. The computing device of claim 12, wherein the at least one processor is configured to: Based on the corresponding weights of the multiple images, a weighted sum is performed on the multiple images to generate the hybrid image; and Based on the corresponding weights of the multiple images, the second set of semantic segmentation labels are weighted and summed to generate the semantic segmentation labels of the hybrid image.

21. The computing device of claim 12, wherein the training data further comprises: The other training samples include the first image and the first semantic segmentation label.

22. The computing device of claim 21, wherein the at least one processor is further configured to: Acquire the input image; and Using a trained target semantic segmentation model, semantic segmentation labels for the input image are generated.

Images