Image processing method, device, apparatus and storage medium

Abstract

The application discloses an image processing method, device and equipment and a storage medium, and related embodiments can be applied to artificial intelligence and other scenarios. The method comprises: performing semantic segmentation processing on a target image through a plurality of reference semantic segmentation models respectively to obtain a set of semantic prediction probabilities, any semantic prediction probability set comprising each pixel in the target image and a semantic prediction probability of being predicted as each semantic category in a plurality of semantic categories; for any pixel of the target image, based on the differences between the semantic prediction probabilities of the pixel predicted in the plurality of semantic prediction probability sets, determining the uncertainty of the pseudo-segmentation label of the pixel; performing label correction processing on the pseudo-segmentation labels in the target image whose uncertainty is greater than an uncertainty threshold respectively, and constructing a target segmentation label set of the target image based on the obtained each corrected segmentation label and other pseudo-segmentation labels; the correction of the pseudo-segmentation label can be realized, and the annotation accuracy is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and more particularly to an image processing method, apparatus, device, and storage medium. Background Technology

[0002] Deep learning-based fully supervised semantic segmentation techniques require a large number of pixel-level segmentation labels, which are computationally expensive and time-consuming. To reduce these costs, weakly supervised or semi-supervised semantic segmentation techniques are often used to train the target semantic segmentation model. For example, pixel-level pseudo-segmentation labels can be generated using various techniques and then used to train the target semantic segmentation model. However, these generated pixel-level pseudo-segmentation labels typically contain numerous errors, significantly limiting the performance of the trained model. Therefore, improving the accuracy of these corrected pseudo-segmentation labels is a pressing issue that needs to be addressed. Summary of the Invention

[0003] This application provides an image processing method, apparatus, device, storage medium, and computer program product that can correct false segmentation labels and improve annotation accuracy.

[0004] On one hand, embodiments of this application provide an image processing method, including:

[0005] Obtain a target image and a set of pseudo-segmentation labels for the target image; the set of pseudo-segmentation labels includes pseudo-segmentation labels that indicate the semantic category to which each pixel in the target image belongs;

[0006] The target image is semantically segmented using multiple reference semantic segmentation models to obtain semantic prediction probability sets corresponding to each reference semantic segmentation model. Each semantic prediction probability set includes: each pixel in the target image is predicted as a semantic prediction probability of each semantic category among multiple semantic categories. Each reference semantic segmentation model is obtained by training an initial semantic segmentation model.

[0007] For any pixel in the target image, based on multiple semantic prediction probability sets, the uncertainty of the pseudo segmentation label of any pixel is determined by the difference between the semantic prediction probabilities predicted for any pixel.

[0008] For each pseudo segmentation label in the set of pseudo segmentation labels, the pseudo segmentation labels with uncertainty greater than the uncertainty threshold are subjected to label correction processing to obtain corrected segmentation labels;

[0009] Based on each corrected segmentation label and other pseudo-segmentation labels in the pseudo-segmentation label set, a target segmentation label set for the target image is constructed; the target segmentation label set is used to train the initial semantic segmentation model.

[0010] On one hand, embodiments of this application provide an image processing apparatus, including:

[0011] An acquisition unit is configured to acquire a target image and a set of pseudo-segmentation labels for the target image; the set of pseudo-segmentation labels includes pseudo-segmentation labels that indicate the semantic category to which each pixel in the target image belongs;

[0012] The processing unit is configured to perform semantic segmentation processing on the target image using multiple reference semantic segmentation models to obtain a semantic prediction probability set corresponding to each reference semantic segmentation model; any semantic prediction probability set includes: each pixel in the target image is predicted as the semantic prediction probability of each semantic category among multiple semantic categories, and each reference semantic segmentation model is obtained by training an initial semantic segmentation model.

[0013] The processing unit is further configured to determine the uncertainty of the pseudo segmentation label of any pixel for any pixel in the target image, based on the difference between the semantic prediction probabilities predicted for any pixel in multiple semantic prediction probability sets.

[0014] The processing unit is also used to perform label correction processing on the pseudo segmentation labels in the pseudo segmentation label set whose uncertainty is greater than the uncertainty threshold, so as to obtain corrected segmentation labels.

[0015] The processing unit is further configured to construct a target segmentation label set for the target image based on each corrected segmentation label and other pseudo-segmentation labels in the pseudo-segmentation label set; the target segmentation label set is used to train the initial semantic segmentation model.

[0016] On one hand, embodiments of this application provide an image processing device, characterized in that the image processing device includes an input interface and an output interface, and further includes:

[0017] A processor, adapted to implement one or more instructions; and,

[0018] A computer storage medium storing one or more instructions adapted to be loaded by the processor and executed by the image processing method described above.

[0019] On one hand, embodiments of this application provide a computer storage medium, characterized in that the computer storage medium stores computer program instructions, which, when executed by a processor, are used to perform the above-described image processing method.

[0020] On one hand, embodiments of this application provide a computer program product, which includes a computer program stored in a computer storage medium; the processor of an image processing device reads the computer program from the computer storage medium and executes the computer program, causing the image processing device to perform the above-described image processing method.

[0021] In this embodiment, the target image can be semantically segmented using multiple reference semantic segmentation models trained on the initial semantic segmentation model, resulting in a semantic prediction probability set corresponding to each reference semantic segmentation model. Each semantic prediction probability set includes the semantic prediction probabilities of each pixel in the target image as a semantic category among multiple semantic categories. Then, for any pixel in the target image, based on the differences between the predicted semantic prediction probabilities in the multiple semantic prediction probability sets, the uncertainty of the pseudo-segmentation label for that pixel can be determined. For pseudo-segmentation labels in the pseudo-segmentation label set whose uncertainty exceeds an uncertainty threshold, label correction processing is performed to obtain corrected segmentation labels. Furthermore, based on each corrected segmentation label and other pseudo-segmentation labels in the target image's pseudo-segmentation label set, a target segmentation label set for training the initial semantic segmentation model can be constructed. Errors in the target image's pseudo-segmentation label set can be determined based on the uncertainty of the pseudo-segmentation labels of each pixel in the target image, thereby correcting errors and improving label accuracy to ensure the performance of the target semantic segmentation model trained on the initial semantic segmentation model based on the target segmentation label set. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram illustrating a method for label correction of a target image provided in an embodiment of this application;

[0024] Figure 2 This is a schematic flowchart of an image processing method provided in an embodiment of this application;

[0025] Figure 3 This is a schematic diagram illustrating the generation of a target segmentation label set based on image category labels of a target image, as provided in an embodiment of this application.

[0026] Figure 4 This is a flowchart illustrating another image processing method provided in an embodiment of this application;

[0027] Figure 5 This is a flowchart illustrating another image processing method provided in an embodiment of this application;

[0028] Figure 6a This is a schematic diagram illustrating the determination of the uncertainty of pseudo-segmentation labels for each pixel in a target image, provided by an embodiment of this application.

[0029] Figure 6b This is a schematic diagram illustrating another method for determining the uncertainty of pseudo-segmentation labels for each pixel in a target image, provided by an embodiment of this application.

[0030] Figure 6c This is a schematic diagram illustrating another method for determining the uncertainty of pseudo-segmentation labels for each pixel in a target image, provided by an embodiment of this application.

[0031] Figure 6d This is a schematic diagram illustrating another method for determining the uncertainty of pseudo-segmentation labels for each pixel in a target image, provided by an embodiment of this application.

[0032] Figure 7a This is a schematic diagram illustrating how to obtain target segmentation labels for each pixel in a target image, as provided in an embodiment of this application.

[0033] Figure 7b This is a schematic diagram illustrating another method for obtaining target segmentation labels for each pixel in a target image, provided by an embodiment of this application.

[0034] Figure 8 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application;

[0035] Figure 9 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application. Detailed Implementation

[0036] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0037] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0038] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision (CV), speech processing, natural language processing, and machine learning (ML) / deep learning (DL).

[0039] Computer vision is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in recognizing, identifying, and measuring targets, and then performs image processing to create images more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, 3D technology, 3D object reconstruction, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and many others. Image semantic understanding technology can include image classification, image segmentation, and other techniques.

[0040] Based on the image segmentation technology in computer vision mentioned above, this application provides an image processing scheme that can obtain a target image and a pseudo-segmentation label set of the target image. The pseudo-segmentation label set includes: pseudo-segmentation labels indicating the semantic category to which each pixel in the target image belongs; semantic segmentation processing of the target image is performed using multiple reference semantic segmentation models to obtain semantic prediction probability sets corresponding to each reference semantic segmentation model, wherein each semantic prediction probability set includes: the semantic prediction probability of each pixel in the target image being predicted to each of the multiple semantic categories, and each reference semantic segmentation model is obtained by training an initial semantic segmentation model; for any pixel in the target image, based on the difference between the semantic prediction probabilities predicted for that pixel in the multiple semantic prediction probability sets, the uncertainty of the pseudo-segmentation label for that pixel is determined; label correction processing is performed on pseudo-segmentation labels in the pseudo-segmentation label set whose uncertainty is greater than an uncertainty threshold to obtain corrected segmentation labels; based on each corrected segmentation label and other pseudo-segmentation labels in the pseudo-segmentation label set, a target segmentation label set of the target image is constructed, wherein the target segmentation label set is used to train the initial semantic segmentation model.

[0041] In one embodiment, the target image can be any sample image from the sample image set used to train the initial semantic segmentation model. This sample image set can be constructed or selected according to specific training requirements. For example, the commonly used semantic segmentation dataset "PASCAL VOC2012" in the field of image segmentation can be selected. Multiple semantic categories can be set according to specific needs, so that the target semantic segmentation model trained on the initial semantic segmentation model can semantically segment any image into these multiple semantic categories. If the number of semantic categories is H, where H is an integer greater than or equal to 2, then the H semantic categories typically include H-1 preset categories and a background category. Any category not belonging to the H-1 preset categories can be identified as the background category. For example, when the sample image set is selected from the commonly used semantic segmentation dataset "PASCAL VOC2012" in the field of image segmentation... When using "VOC2012", its corresponding H semantic categories include 20 preset categories such as: people, birds, cats, cows, dogs, horses, sheep, airplanes, bicycles, ships, buses, cars, motorcycles, trains, bottles, chairs, dining tables, potted plants, sofas, and televisions / monitors, as well as a background category. For example, if an image contains a cat, a dog, and a tree, then the semantic category of the pixels that make up the cat is "cat", the semantic category of the pixels that make up the dog is "dog", and the semantic category of the pixels that make up the content other than the cat and dog is "background category". For example, the semantic category of the pixels that make up the tree is "background category".

[0042] In one embodiment, each reference semantic segmentation model is obtained by training an initial semantic segmentation model; wherein, the initial semantic segmentation model can be any semantic segmentation model capable of performing semantic segmentation processing on images, and this embodiment of the application does not impose any restrictions. Furthermore, each reference semantic segmentation model can be obtained by iteratively training the initial semantic segmentation model based on each sample image in the sample image set to which the target image belongs, and the pseudo segmentation label set of each sample image. The number of iterations for different reference semantic segmentation models is different. For example, if the initial semantic segmentation model is trained N times based on each sample image in the sample image set and the pseudo segmentation label set of each sample image, then the initial semantic segmentation model after each round of training can be used as a reference semantic segmentation model to obtain N reference semantic segmentation models. Alternatively, the initial semantic segmentation model after every two rounds of training can be used as a reference semantic segmentation model to obtain multiple reference semantic segmentation models, and so on. For ease of explanation, the embodiments of this application will be described below using the example of obtaining N reference semantic segmentation models by training the initial semantic segmentation model N times. In the process of iteratively training the initial semantic segmentation model, each sample image in the sample image set can be used as the input of the initial semantic segmentation model, and the pseudo segmentation label set of the corresponding sample image can be used as the expected output of the initial semantic segmentation model to iteratively train the initial semantic segmentation model.

[0043] In specific implementations, the image processing scheme proposed in this application embodiment can be executed by an image processing device, which can be a terminal device or a server. The terminal device here may include, but is not limited to, computers, smartphones, tablets, laptops, smart home appliances, in-vehicle terminals, and smart wearable devices. The server here can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Further optionally, the image processing scheme proposed in this application embodiment can also be executed individually or collaboratively by other computing-capable electronic devices; this application embodiment does not impose any limitations.

[0044] See Figure 1This is a schematic diagram of label correction for a target image provided in an embodiment of this application. If the number of multiple semantic categories is H, H=5, and the first to fifth semantic categories are represented as 1, 2, 3, 4 and 5 respectively, and if the pseudo segmentation labels of each pixel in the target image are as marked 101, the image processing device can determine the uncertainty of the pseudo segmentation labels of each pixel, and perform label correction processing on the pseudo segmentation labels with uncertainties greater than the uncertainty threshold to obtain corrected segmentation labels. Then, based on each corrected segmentation label and other pseudo segmentation labels in the pseudo segmentation label set of the target image, a target segmentation label set of the target image is constructed. The target segmentation label set of the target image constructed can be as marked 102.

[0045] It should be noted that in the specific embodiments of this application, data related to the object, such as target images, are involved. When the embodiments of this application are applied to specific products or technologies, permission or consent from the object is required, and the collection, use and processing of related data must comply with local laws, regulations and standards.

[0046] Based on the above image processing scheme, this application provides an image processing method. See also... Figure 2 This is a flowchart illustrating an image processing method provided in an embodiment of this application. Figure 2 The image processing method shown can be executed by an image processing device, or by other electronic devices with computing power, either alone or in collaboration. In this application embodiment, an image processing device is used as an example. Figure 2 The image processing method shown may include the following steps:

[0047] S201, Obtain the target image and the pseudo-segmentation label set of the target image.

[0048] The pseudo-segmentation label set includes pseudo-segmentation labels used to indicate the semantic category to which each pixel in the target image belongs.

[0049] In one embodiment, the pseudo-segmentation label set of the target image usually contains erroneous labels. When the semantic category indicated by the pseudo-segmentation label of a pixel is different from the semantic category to which the pixel actually belongs, the pseudo-segmentation label of that pixel is considered to be an erroneous label. When the semantic category indicated by the pseudo-segmentation label of a pixel is the same as the semantic category to which the pixel actually belongs, the pseudo-segmentation label of that pixel is considered to be a correct label. For example, if there are H semantic categories, H=5, if the semantic category indicated by the pseudo-segmentation label of a pixel in the target image is the first semantic category, but the semantic category to which the pixel actually belongs is the second semantic category, then the pseudo-segmentation label of that pixel is considered to be an erroneous label.

[0050] In one embodiment, the pseudo-segmentation label set of the target image can be generated using certain techniques in scenarios related to weakly supervised semantic segmentation or semi-supervised semantic segmentation. Weakly supervised semantic segmentation (WSSS) refers to training a semantic segmentation model solely based on image category labels; semi-supervised semantic segmentation (SSSS) refers to training a semantic segmentation model based on a large amount of unlabeled data and simultaneously on some data with pixel-level annotations (i.e., data with pixel-level segmentation labels). For example, a pseudo-segmentation label set of the target image can be generated based on image category labels. Specifically, the image processing device can train a multi-label classification model based on each sample image in the sample image set to which the target image belongs, and the image category labels of each sample image, to obtain a trained multi-label classification model. The trained multi-label classification model is then used to process the target image to obtain the class activation maps (CAMs) of the target image under each semantic category. Furthermore, by optimizing the class activation maps of the target image under each semantic category, the pseudo-segmentation label set of the target image can be obtained. In this embodiment, the image category label of any sample image is used to indicate the semantic category to which the image content in the sample image belongs. For example, if the image content in a sample image includes a person and a bottle, then the image category label of the sample image may include "person" and "bottle". The multi-label classification model can be any model that can perform multi-label classification of images, and this embodiment does not impose any restrictions. The category activation map is obtained during the multi-classification process of the trained multi-label classification model and is a spatial attention map that can indicate the target location. When optimizing the category activation maps of the target image under each semantic category to obtain the pseudo-segmentation label set of the target image, post-processing algorithms such as dense conditional random field (dCRF) and pixel affinity-based random walk can be used for optimization.

[0051] S202, the target image is semantically segmented using multiple reference semantic segmentation models to obtain the semantic prediction probability set corresponding to each reference semantic segmentation model.

[0052] Each reference semantic segmentation model is obtained by training an initial semantic segmentation model. Each reference semantic segmentation model can be obtained by iteratively training the initial semantic segmentation model based on sample images from the sample image set to which the target image belongs, and the pseudo-segmentation label set of each sample image. The number of iterations varies among different reference semantic segmentation models. This embodiment will subsequently illustrate this by taking the example of obtaining N reference semantic segmentation models through N iterations of training the initial semantic segmentation model. Any semantic prediction probability set includes: the semantic prediction probability of each pixel in the target image for each of multiple semantic categories. The image processing device performs semantic segmentation processing on the target image using any reference semantic segmentation model to obtain the semantic prediction probability set corresponding to that reference semantic segmentation model, a process adapted to the model structure of that reference semantic segmentation model.

[0053] S203, for any pixel in the target image, based on the difference between the semantic prediction probabilities predicted for any pixel in multiple semantic prediction probability sets, determine the uncertainty of the pseudo segmentation label of any pixel.

[0054] In one embodiment, the uncertainty of the pseudo-segmentation label of any pixel is used to determine whether the pseudo-segmentation label of that pixel is incorrectly labeled. Specifically, when the uncertainty of the pseudo-segmentation label of any pixel is greater than the uncertainty threshold, the pseudo-segmentation label of that pixel is determined to be incorrectly labeled; when the uncertainty of the pseudo-segmentation label of any pixel is less than or equal to the uncertainty threshold, the pseudo-segmentation label of that pixel is determined to be correctly labeled. Furthermore, label correction processing can be performed on the pseudo-segmentation labels in the pseudo-segmentation label set that are determined to be incorrectly labeled, that is, label correction processing can be performed on the pseudo-segmentation labels in the pseudo-segmentation label set whose uncertainty is greater than the uncertainty threshold. The uncertainty threshold can be set according to specific needs, and this embodiment does not impose any restrictions.

[0055] In one embodiment, if the number of multiple semantic categories is H, where H is an integer greater than or equal to 2; the image processing device determines the uncertainty of the pseudo segmentation label of any pixel based on the difference between the semantic prediction probabilities predicted for any pixel in the multiple semantic prediction probability sets, which may include: traversing the H semantic categories, and based on the difference between the semantic prediction probabilities of any pixel when it is predicted to be the h-th semantic category in the multiple semantic prediction probability sets, determining the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category, as a prediction uncertainty of any pixel, h∈[1,H]; selecting one prediction uncertainty from the H prediction uncertainties of any pixel as the uncertainty of the pseudo segmentation label of any pixel. Since each reference semantic segmentation model is obtained by iteratively training the initial semantic segmentation model based on each sample image in the sample image set to which the target image belongs, and the pseudo-segmentation label set of each sample image, the iteration rounds of different reference semantic segmentation models are different. Therefore, when the pseudo-segmentation label of any pixel is incorrectly labeled, the difference between the semantic prediction probabilities of any pixel being predicted to be of the same semantic category by each reference semantic segmentation model should be large, that is, the fluctuation of the semantic prediction probability of any pixel being predicted to be of the same semantic category by each reference semantic segmentation model should be large. When the pseudo-segmentation label of any pixel is correctly labeled, the difference between the semantic prediction probabilities of any pixel being predicted to be of the same semantic category by each reference semantic segmentation model should be small, that is, the fluctuation of the semantic prediction probability of any pixel being predicted to be of the same semantic category by each reference semantic segmentation model should be small. Therefore, the uncertainty of the pseudo-segmentation label of any pixel determined from the H prediction uncertainties of any pixel can be used to determine whether the pseudo-segmentation label of any pixel should be corrected.

[0056] S204. For each pseudo segmentation label in the pseudo segmentation label set whose uncertainty is greater than the uncertainty threshold, label correction processing is performed to obtain corrected segmentation labels.

[0057] In one embodiment, the image processing device performs label correction processing on pseudo-segmentation labels in the pseudo-segmentation label set whose uncertainty is greater than the uncertainty threshold, to obtain corrected segmentation labels. This means that the device can perform label correction processing on pseudo-segmentation labels determined to be incorrectly labeled in the pseudo-segmentation label set to obtain corrected segmentation labels. Furthermore, the corrected segmentation label obtained by performing label correction processing on any pseudo-segmentation label with uncertainty greater than the uncertainty threshold should meet the following conditions: in multiple semantic prediction probability sets, the semantic prediction probability of the target pixel corresponding to any pseudo-segmentation label when predicted to be the semantic category indicated by the corrected segmentation label is high; and in multiple semantic prediction probability sets, the fluctuation of the semantic prediction probability of the target pixel corresponding to any pseudo-segmentation label when predicted to be the semantic category indicated by the corrected segmentation label is small.

[0058] S205. Based on each corrected segmentation label and other pseudo-segmentation labels in the pseudo-segmentation label set, construct the target segmentation label set for the target image.

[0059] The target segmentation label set of the target image can be used to train the initial semantic segmentation model to ensure the performance of the trained target semantic segmentation model.

[0060] In one embodiment, the image processing method proposed in this application can generate a target segmentation label set for training an initial semantic segmentation model, provided that the target image is labeled with image category tags. This saves the manpower and time costs associated with pixel-level segmentation labeling of the target image and improves the training efficiency of the initial semantic segmentation model. See also Figure 3 This is a schematic diagram of generating a target segmentation label set based on the image category labels of a target image, provided by an embodiment of this application. If the target image is an image as shown by label 301, and the image category labels of the target image are "person" and "bottle", then the image processing device can generate a pseudo segmentation label set of the target image based on the trained multi-label classification model; then it can determine the uncertainty of the pseudo segmentation label of each pixel of the target image, and perform label correction processing on the pseudo segmentation labels with uncertainty greater than the uncertainty threshold to obtain corrected segmentation labels. Based on each corrected segmentation label and other pseudo segmentation labels in the pseudo segmentation label set of the target image, a target segmentation label set of the target image is constructed. The target segmentation label set of the target image constructed can be as shown by label 302.

[0061] In an optional implementation, the image processing method provided in this application embodiment can be implemented on the PyTorch (1.7.1) neural network framework. When training the initial semantic segmentation model, the Adam algorithm can be used as the optimizer during training, with an initial learning rate of 3e-4. Furthermore, a cosine annealing learning rate scheduling mechanism can be employed during training. Optionally, the correction of the pseudo-segmentation label set of each sample image in the sample image set to which the target image belongs can be done online. That is, while correcting the pseudo-segmentation label set of the sample images in the sample image set, the initial semantic segmentation model is trained using the corrected target segmentation label set of the sample images. This can further improve the training rate of the initial semantic segmentation model, making the training of the initial semantic segmentation model more efficient.

[0062] In this embodiment, the target image can be semantically segmented using multiple reference semantic segmentation models trained on the initial semantic segmentation model, resulting in a semantic prediction probability set corresponding to each reference semantic segmentation model. Each semantic prediction probability set includes the semantic prediction probabilities of each pixel in the target image as a semantic category among multiple semantic categories. Then, for any pixel in the target image, based on the differences between the predicted semantic prediction probabilities in the multiple semantic prediction probability sets, the uncertainty of the pseudo-segmentation label for that pixel can be determined. For pseudo-segmentation labels in the pseudo-segmentation label set whose uncertainty exceeds an uncertainty threshold, label correction processing is performed to obtain corrected segmentation labels. Furthermore, based on each corrected segmentation label and other pseudo-segmentation labels in the target image's pseudo-segmentation label set, a target segmentation label set for training the initial semantic segmentation model can be constructed. Errors in the target image's pseudo-segmentation label set can be determined based on the uncertainty of the pseudo-segmentation labels of each pixel in the target image, thereby correcting errors and improving label accuracy to ensure the performance of the target semantic segmentation model trained on the initial semantic segmentation model based on the target segmentation label set.

[0063] Based on the above image processing scheme, this application provides another image processing method, which is described using H as the number of semantic categories, where H is an integer greater than or equal to 2. See also Figure 4 This is a schematic flowchart of another image processing method provided in an embodiment of this application. Figure 4 The image processing method shown can be executed by an image processing device, or by other electronic devices with computing power, either alone or in collaboration. In this application embodiment, an image processing device is used as an example. Figure 4 The image processing method shown may include the following steps:

[0064] S401, Obtain the target image and the pseudo-segmentation label set of the target image.

[0065] The pseudo segmentation label set includes pseudo segmentation labels used to indicate the semantic category to which each pixel in the target image belongs; the relevant process of step S401 is similar to the relevant process of step S201 above, and will not be repeated here.

[0066] S402, the target image is semantically segmented using multiple reference semantic segmentation models to obtain the semantic prediction probability set corresponding to each reference semantic segmentation model.

[0067] Each reference semantic segmentation model is obtained by training the initial semantic segmentation model. Any semantic prediction probability set includes: each pixel in the target image is predicted as the semantic prediction probability of each semantic category in H semantic categories. The relevant process of step S402 is similar to the relevant process of step S202 above, and will not be repeated here.

[0068] S403, for any pixel in the target image, traverse H semantic categories, and based on the difference between the semantic prediction probabilities when any pixel is predicted to be the h-th semantic category in multiple semantic prediction probability sets, determine the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category, and use it as a prediction uncertainty of any pixel.

[0069] In one embodiment, if the number of reference semantic segmentation models is N, where N is an integer greater than or equal to 2, the N reference semantic segmentation models are obtained by training the initial semantic segmentation model through N rounds of iterations; if the target image is represented by X, H * and W * are positive integers, representing the height and width of the target image, respectively; the sequence consisting of the semantic prediction probability sets corresponding to each reference semantic segmentation model can be called the original prediction probability sequence, which can be specifically represented as: Where n is the independent variable, n∈[1,N], P n Let n be the nth semantic prediction probability set among N semantic prediction probability sets. H represents the number of semantic categories.

[0070] Furthermore, the sequence of semantic prediction probabilities formed by any pixel in the N semantic prediction probability sets when it is predicted to be the h-th semantic category can be called the first subsequence of that pixel in the h-th semantic category, where h is the independent variable and h∈[1,H]; if that pixel is the pixel in the i-th row and j-th column of the target image, i and j are independent variables and i∈H * ,j∈W * Then the first subsequence of any pixel under the h-th semantic category can be represented as: Sh,i,j ={P1(h,i,j),…,P n (h,i,j),…,P N (h,i,j)}, where P n (h,i,j) represents the semantic prediction probability of the pixel in the i-th row and j-th column of the target image in the n-th semantic prediction probability set when it is predicted to be the h-th semantic category. That is, the semantic prediction probability of the pixel in the i-th row and j-th column of the target image when it is predicted to be the h-th semantic category, obtained by processing the n-th reference semantic segmentation model.

[0071] For example, if the number of reference semantic segmentation models is N, N=4, and the number of semantic categories is H=5, and any pixel is the pixel in the i-th row and j-th column of the target image; if in the first semantic prediction probability set obtained through the first reference semantic segmentation model, the semantic prediction probabilities of any pixel being predicted as belonging to each of the five semantic categories are {0.05, 0.8, 0.05, 0.05, 0.05}, and in the second semantic prediction probability set obtained through the second reference semantic segmentation model, the semantic prediction probabilities of any pixel being predicted as belonging to each of the five semantic categories are {0.04, 0.8, 0.05}, respectively. Given the semantic prediction probability set obtained through the third reference semantic segmentation model, the semantic prediction probabilities of any pixel being predicted as belonging to each of the five semantic categories are {0.04, 0.85, 0.06, 0.02, 0.03}. Similarly, in the semantic prediction probability set obtained through the fourth reference semantic segmentation model, the semantic prediction probabilities of any pixel being predicted as belonging to each of the five semantic categories are {0.02, 0.9, 0.03, 0.02, 0.03}. Therefore, the first subsequence of any pixel in the first semantic category can be represented as: S 1,i,j ={0.05,0.04,0.04,0.02}, the first subsequence of any pixel in the second semantic category can be represented as: S 2,i,j ={0.8,0.8,0.85,0.9}, and so on.

[0072] In one embodiment, an image processing device determines the uncertainty of the semantic prediction probability of any pixel in the h-th semantic category based on the difference between the semantic prediction probabilities of any pixel being predicted as the h-th semantic category in multiple semantic prediction probability sets. This can include: performing variance calculation on the semantic prediction probability of any pixel being predicted as the h-th semantic category in multiple semantic prediction probability sets to obtain a first variance; using the first variance as the uncertainty of the semantic prediction probability of any pixel in the h-th semantic category; wherein, the magnitude of the first variance corresponding to any pixel can be used to measure the difference between the semantic prediction probabilities of any pixel being predicted as the h-th semantic category in multiple semantic prediction probability sets, and can also be used to measure the volatility of the semantic prediction probability of any pixel being predicted as the h-th semantic category in multiple semantic prediction probability sets. In a specific implementation, if any pixel is a pixel in the i-th row and j-th column of the target image, then the first variance corresponding to any pixel can be shown by the following formula 1.1:

[0073]

[0074] Where N represents the number of semantic prediction probability sets, i.e., the number of reference semantic segmentation models, and also represents the length of the first subsequence of the pixel in the i-th row and j-th column of the target image under the h-th semantic category, and n is the independent variable, n∈[1,N]; S h,i,j (n) represents the semantic prediction probability when the pixel in the i-th row and j-th column of the target image is predicted to be the h-th semantic category in the n-th semantic prediction probability set, that is, the n-th sequence value of the pixel in the i-th row and j-th column of the target image in the first subsequence under the h-th semantic category; This represents the mean of the h-th predicted probability of the pixel in the i-th row and j-th column of the target image. The mean of the h-th predicted probability of the pixel in the i-th row and j-th column of the target image is obtained by averaging the semantic prediction probabilities of the pixel in the i-th row and j-th column of the target image when it is predicted to be the h-th semantic category from multiple semantic prediction probability sets. In other words, it is the expectation of the first subsequence of the pixel in the i-th row and j-th column of the target image under the h-th semantic category.

[0075] Furthermore, if using Let H represent the uncertainty of the semantic prediction probability of the pixel in the i-th row and j-th column of the target image under the h-th semantic category, that is, the h-th prediction uncertainty of the pixel in the i-th row and j-th column of the target image. Then, the first variance corresponding to the pixel in the i-th row and j-th column of the target image can be used as the uncertainty of the semantic prediction probability of that pixel under the h-th semantic category, which can be expressed by the following formula 1.2:

[0076]

[0077] In one embodiment, when determining the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category, in addition to the differences between the semantic prediction probabilities of any pixel being predicted to be the h-th semantic category in multiple semantic prediction probability sets, the differences between the semantic prediction probabilities of any pixel being predicted to be the h-th semantic category in multiple semantic prediction probability sets and the pseudo-segmentation label of any pixel can also be introduced. Based on this, the image processing device determines the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category based on the differences between the semantic prediction probabilities of any pixel being predicted to be the h-th semantic category in multiple semantic prediction probability sets, which may include: determining a reference indication value based on the pseudo-segmentation label of any pixel to indicate whether the semantic category to which the pixel belongs is the h-th semantic category; performing variance calculation processing on the semantic prediction probability of any pixel being predicted to be the h-th semantic category in multiple semantic prediction probability sets and the reference indication value corresponding to the pixel to obtain a second variance; and using the second variance as the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category. In specific implementation, if the pseudo-segmentation label of any pixel indicates that the semantic category to which the pixel belongs is the h-th semantic category, then the reference indication value corresponding to the pixel can be determined as a first value, for example, 1. If the pseudo-segmentation label of any pixel indicates that the semantic category to which the pixel belongs is not the h-th semantic category, then the reference indication value corresponding to the pixel can be determined as a second value, for example, 0. Furthermore, when the pseudo-segmentation label of any pixel is one-hot encoded, the pseudo-segmentation label of any pixel includes H pseudo-label indication values, then the h-th pseudo-label indication value of any pixel can be determined as the h-th semantic category. h pseudo-label indication values ​​are determined as the reference indication value corresponding to any pixel. For example, if H = 5, the one-hot encoding of the pseudo segmentation label of any pixel is {0,1,0,0,0,}. Then, in the first semantic category, the reference indication value corresponding to any pixel is 0; in the second semantic category, the reference indication value corresponding to any pixel is 1; in the third semantic category, the reference indication value corresponding to any pixel is 0; in the fourth semantic category, the reference indication value corresponding to any pixel is 0; and in the fifth semantic category, the reference indication value corresponding to any pixel is 0.

[0078] Furthermore, the sequence consisting of the semantic prediction probability of any pixel being predicted to be the h-th semantic category from multiple semantic prediction probability sets, and the reference indicator value corresponding to that pixel, can be called the second subsequence of that pixel under the h-th semantic category, where h is the independent variable and h∈[1,H]; if that pixel is the pixel in the i-th row and j-th column of the target image, i and j are the independent variables and i∈H * ,j∈W * Under the h-th semantic category, the reference indicator value corresponding to the pixel in the i-th row and j-th column of the target image is represented by M.p (h,i,j) indicates that the second subsequence of the pixels in the i-th row and j-th column of the target image under the h-th semantic category can be represented as: S′ h,i,j ={P1(h,i,j),M p (h,i,j),…,P n (h,i,j),M p (h,i,j),…,P N (h,i,j),M p (h,i,j)};Since the pseudo-segmentation label of the pixel in the i-th row and j-th column of the target image is one-hot encoded, the h-th pseudo-label indicator value of that pixel can be determined as the reference indicator value corresponding to that pixel under the h-th semantic category, then M p (h,i,j) can also represent the h-th pseudo-label indicator value of the pixel in the i-th row and j-th column of the target image. In this case, the pseudo-segmentation label set of the target image using one-hot encoding can be represented as M. p ,

[0079] Furthermore, if any pixel is the pixel in the i-th row and j-th column of the target image, then the second variance corresponding to that pixel can be expressed by the following formula 2.1:

[0080]

[0081] Where N represents the number of semantic prediction probability sets, i.e., the number of reference semantic segmentation models, 2N represents the length of the second subsequence of the pixel in the i-th row and j-th column of the target image under the h-th semantic category, and n is the independent variable, n∈[1,2N]; S′ h,i,j (n) represents the nth sequence value of the pixel in the i-th row and j-th column of the target image in the second subsequence under the h-th semantic category; This represents the expectation of the second subsequence of the pixel in the i-th row and j-th column of the target image under the h-th semantic category.

[0082] Furthermore, if using Let H represent the uncertainty of the semantic prediction probability of the pixel in the i-th row and j-th column of the target image under the h-th semantic category, which is the h-th prediction uncertainty of the pixel in the i-th row and j-th column of the target image. Then, the second variance corresponding to the pixel in the i-th row and j-th column of the target image can be used as the uncertainty of the semantic prediction probability of that pixel under the h-th semantic category, as shown in the following formula 2.2:

[0083]

[0084] In one embodiment, an image processing device determines the uncertainty of the semantic prediction probability of any pixel in the h-th semantic category based on the difference between the semantic prediction probabilities of any pixel being predicted as the h-th semantic category in multiple semantic prediction probability sets. This can include: using a post-processing algorithm to post-process the semantic prediction probabilities of any pixel being predicted as the h-th semantic category in multiple semantic prediction probability sets to obtain a post-processed semantic prediction probability; and determining the uncertainty of the semantic prediction probability of any pixel in the h-th semantic category based on the difference between the multiple post-processed semantic prediction probabilities of any pixel being predicted as the h-th semantic category. The post-processing algorithm can be a dense conditional random field or similar algorithm. The post-processing algorithm can be used to improve the accuracy of each semantic prediction probability in multiple semantic prediction probability sets. The sequence composed of the post-processed semantic prediction probabilities corresponding to each semantic prediction probability in multiple semantic prediction probability sets can be called a post-processed prediction probability sequence, which can be specifically represented as: Furthermore, the sequence consisting of multiple post-processed semantic prediction probabilities when any pixel is predicted to be in the h-th semantic category can be called the third subsequence of that pixel in the h-th semantic category, if that pixel is the pixel in the i-th row and j-th column of the target image, i∈H * ,j∈W * Then the third subsequence of any pixel under the h-th semantic category can be represented as: in, This represents the semantic prediction probability of the nth post-processing when the pixel in the i-th row and j-th column of the target image is predicted to be the h-th semantic category. In other words, it represents the post-processing semantic prediction probability corresponding to the semantic prediction probability of the pixel in the i-th row and j-th column of the target image being predicted to be the h-th semantic category in the n-th semantic prediction probability set.

[0085] In one feasible implementation, the image processing device determines the uncertainty of the semantic prediction probability of any pixel in the h-th semantic category based on the difference between the semantic prediction probabilities of multiple post-processing operations when any pixel is predicted to be in the h-th semantic category. This can include: performing variance calculation on the semantic prediction probabilities of multiple post-processing operations when any pixel is predicted to be in the h-th semantic category to obtain a third difference; using the third difference as the uncertainty of the semantic prediction probability of any pixel in the h-th semantic category; wherein, the magnitude of the third difference corresponding to any pixel can be used to measure the difference between the semantic prediction probabilities of multiple post-processing operations when any pixel is predicted to be in the h-th semantic category, and can also be used to measure the volatility of the semantic prediction probabilities of multiple post-processing operations when any pixel is predicted to be in the h-th semantic category. In a specific implementation, if any pixel is a pixel in the i-th row and j-th column of the target image, then the third difference corresponding to any pixel can be shown by the following formula 3.1:

[0086]

[0087] Where N represents the number of semantic prediction probability sets, i.e. the number of reference semantic segmentation models, and also represents the length of the third subsequence of the pixel in the i-th row and j-th column of the target image under the h-th semantic category, and n is the independent variable, n∈[1,N]; This represents the nth sequence value of the pixel in the i-th row and j-th column of the target image within the third subsequence under the h-th semantic category; This represents the expectation of the third subsequence of the pixel in the i-th row and j-th column of the target image under the h-th semantic category.

[0088] Furthermore, if using Let H represent the uncertainty of the semantic prediction probability of the pixel in the i-th row and j-th column of the target image under the h-th semantic category, which is the h-th prediction uncertainty of the pixel in the i-th row and j-th column of the target image. Then, the third difference corresponding to the pixel in the i-th row and j-th column of the target image can be used as the uncertainty of the semantic prediction probability of that pixel under the h-th semantic category, as shown by the following formula 3.2:

[0089]

[0090] In another feasible implementation, when determining the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category, in addition to the difference between the semantic prediction probabilities of multiple post-processing when any pixel is predicted to be under the h-th semantic category, the difference between the semantic prediction probabilities of multiple post-processing when any pixel is predicted to be under the h-th semantic category and the pseudo-segmentation label of any pixel can also be introduced. Based on this, the image processing device determines the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category based on the difference between the semantic prediction probabilities of multiple post-processing when any pixel is predicted to be under the h-th semantic category, which may include: determining a reference indication value for indicating whether the semantic category to which any pixel belongs is the h-th semantic category based on the pseudo-segmentation label of any pixel; performing variance calculation processing on the semantic prediction probabilities of multiple post-processing when any pixel is predicted to be under the h-th semantic category and the reference indication value corresponding to any pixel to obtain a fourth variance; and using the fourth variance as the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category. The process of determining the reference indicator value for indicating whether the semantic category to which any pixel belongs is the h-th semantic category based on the pseudo segmentation label of any pixel has been described in detail above and will not be repeated here.

[0091] Furthermore, the sequence consisting of multiple post-processed semantic prediction probabilities when any pixel is predicted to be in the h-th semantic category, and the reference indicator value corresponding to that pixel, can be called the fourth subsequence of that pixel in the h-th semantic category, where h is the independent variable and h∈[1,H]; if that pixel is the pixel in the i-th row and j-th column of the target image, where i and j are independent variables and i∈H * ,j∈W * Under the h-th semantic category, the reference indicator value corresponding to the pixel in the i-th row and j-th column of the target image is represented by M. p (h,i,j) indicates that the fourth subsequence of the pixels in the i-th row and j-th column of the target image under the h-th semantic category can be represented as:

[0092]

[0093] Furthermore, if any pixel is the pixel in the i-th row and j-th column of the target image, then the fourth variance corresponding to that pixel can be expressed by the following formula 4.1:

[0094]

[0095] Where N represents the number of semantic prediction probability sets, i.e. the number of reference semantic segmentation models, 2N represents the length of the fourth subsequence of the pixel in the i-th row and j-th column of the target image under the h-th semantic category, and n is the independent variable, n∈[1,2N]; This represents the nth sequence value of the pixel in the i-th row and j-th column of the target image within the fourth subsequence under the h-th semantic category; This represents the expectation of the fourth subsequence of the pixel in the i-th row and j-th column of the target image under the h-th semantic category.

[0096] Furthermore, if using Let H represent the uncertainty of the semantic prediction probability of the pixel in the i-th row and j-th column of the target image under the h-th semantic category, which is the h-th prediction uncertainty of the pixel in the i-th row and j-th column of the target image. Then, the fourth variance corresponding to the pixel in the i-th row and j-th column of the target image can be used as the uncertainty of the semantic prediction probability of that pixel under the h-th semantic category, as shown in the following formula 4.2:

[0097]

[0098] S404: Select one prediction uncertainty from H prediction uncertainties of any pixel as the uncertainty of the pseudo segmentation label of any pixel.

[0099] In one embodiment, the image processing device selects one prediction uncertainty from H prediction uncertainties of any pixel as the uncertainty of the pseudo-segmentation label of any pixel, which may include: taking the maximum prediction uncertainty from the H prediction uncertainties of any pixel as the uncertainty of the pseudo-segmentation label of that pixel; if the pixel is a pixel in the i-th row and j-th column of the target image, then the uncertainty of the pseudo-segmentation label of that pixel can be specifically expressed by the following formula 5.1:

[0100]

[0101] In one embodiment, the image processing device selects one prediction uncertainty from H prediction uncertainties of any pixel as the uncertainty of the pseudo-segmentation label of any pixel, which may include: determining H-1 prediction uncertainties corresponding to H-1 preset categories from the H prediction uncertainties of any pixel; taking the largest prediction uncertainty among the H-1 prediction uncertainties as the uncertainty of the pseudo-segmentation label of any pixel; if the pixel is the pixel in the i-th row and j-th column of the target image, and the first semantic category among the H semantic categories is the background category, and the second to H semantic categories are preset categories, then the uncertainty of the pseudo-segmentation label of the pixel can be specifically shown by the following formula 5.2:

[0102]

[0103] Furthermore, after selecting one prediction uncertainty from the H prediction uncertainties of any given pixel as the uncertainty of the pseudo-segmentation label for that pixel, the image processing device can also normalize the uncertainty of the pseudo-segmentation label for that pixel to update it. Subsequent processing related to the uncertainty of the pseudo-segmentation label for that pixel can all be performed on the updated uncertainty. For example, based on the uncertainties of the pseudo-segmentation labels of each pixel in the target image, a minimum-maximum normalization calculation can be performed on the uncertainty of the pseudo-segmentation label for that pixel to update it. The specific updated uncertainty of the pseudo-segmentation label for that pixel can be calculated using the following formula...

[0104] Equation 5.3 shows that:

[0105]

[0106] Where Min(U) represents the minimum uncertainty of the pseudo-segmentation label for each pixel in the target image, Max(U) represents the maximum uncertainty of the pseudo-segmentation label for each pixel in the target image, and U represents the uncertainty of the pseudo-segmentation label for each pixel in the target image.

[0107] S405, perform label correction processing on the pseudo segmentation labels in the pseudo segmentation label set whose uncertainty is greater than the uncertainty threshold to obtain the corrected segmentation labels.

[0108] In one embodiment, the corrected segmentation label obtained by label correction processing of any pseudo-segmentation label with uncertainty greater than the uncertainty threshold should meet the following conditions: in a set of multiple semantic prediction probabilities, the semantic prediction probability of the target pixel corresponding to the pseudo-segmentation label being predicted as the semantic category indicated by the corrected segmentation label is high, and the fluctuation of the semantic prediction probability of the target pixel corresponding to the pseudo-segmentation label being predicted as the semantic category indicated by the corrected segmentation label is small. Based on this, the image processing device performs label correction processing on the pseudo-segmentation labels with uncertainty greater than the uncertainty threshold in the pseudo-segmentation label set to obtain the corrected segmentation label, which may include: for any pseudo-segmentation label with uncertainty greater than the uncertainty threshold, obtaining the target pixel corresponding to the pseudo-segmentation label. The h-th prediction uncertainty of the target pixel and the mean h-th prediction probability of the target pixel are defined as follows: the h-th prediction uncertainty of the target pixel refers to the uncertainty of the semantic prediction probability of the target pixel under the h-th semantic category, and the mean h-th prediction probability of the target pixel is obtained by averaging the semantic prediction probabilities of the target pixel when it is predicted to be the h-th semantic category in multiple semantic prediction probability sets; based on the h-th prediction uncertainty of the target pixel and the mean h-th prediction probability of the target pixel, the correction reference parameter of the target pixel under the h-th semantic category is determined, h∈[1,H]; the target correction reference parameter is determined from the correction reference parameters of the target pixel under each semantic category, and the semantic category indicated by the target correction reference parameter is used as the correction segmentation label corresponding to any pseudo segmentation label.

[0109] In one feasible implementation, the image processing device determines a correction reference parameter for the target pixel in the h-th semantic category based on the h-th prediction uncertainty and the h-th prediction probability mean of the target pixel. This may include: determining the ratio between the h-th prediction uncertainty and the h-th prediction probability mean of the target pixel as the correction reference parameter for the target pixel in the h-th semantic category; further, the image processing device determines a target correction reference parameter from the correction reference parameters of the target pixel in each semantic category. This may include: taking the minimum correction reference parameter among the correction reference parameters of the target pixel in each semantic category as the target correction reference parameter.

[0110] Optionally, when the image processing device determines the correction reference parameter of the target pixel under the h-th semantic category based on the h-th prediction uncertainty of the target pixel and the mean h-th prediction probability of the target pixel, it can determine the ratio between the square root of the h-th prediction uncertainty of the target pixel and the mean h-th prediction probability of the target pixel as the correction reference parameter of the target pixel under the h-th semantic category. Optionally, since the h-th prediction uncertainty of the target pixel can be the determined first variance, second variance, third variance, or fourth variance, only the first variance is obtained by directly calculating the variance of the semantic prediction probability when the target pixel is predicted to be the h-th semantic category in multiple semantic prediction probability sets. This more intuitively reflects the volatility of the semantic prediction probability when the target pixel is predicted to be the h-th semantic category in multiple semantic prediction probability sets. Therefore, when determining the correction reference parameter for the target pixel in the h-th semantic category, the ratio between the first variance obtained by calculating the variance of the semantic prediction probability when the target pixel is predicted to be the h-th semantic category in multiple semantic prediction probability sets and the mean of the h-th prediction probability of the target pixel can be determined as the correction reference parameter for the target pixel in the h-th semantic category. Alternatively, the ratio between the square root of the first variance corresponding to the target pixel and the mean of the h-th prediction probability of the target pixel can also be determined as the correction reference parameter for the target pixel in the h-th semantic category. In this case, the correction reference parameter for the target pixel in the h-th semantic category is the normalized variance of category probability. Probability (NVCP); This application uses the corrected reference parameter of the target pixel in the h-th semantic category as the normalized category probability variance as an example for illustration.

[0111] In specific implementation, the image processing device can traverse each pixel in the target image. If the uncertainty of the pseudo-segmentation label of any pixel is greater than the uncertainty threshold, the ratio between the square root of the first variance corresponding to that pixel and the mean of the h-th predicted probability of that pixel can be determined as the correction reference parameter of that pixel in the h-th semantic category. Then, the minimum correction reference parameter among the correction reference parameters of that pixel in each semantic category can be used as the target correction reference parameter, and the semantic category indicated by the target correction reference parameter can be used as the correction segmentation label corresponding to the pseudo-segmentation label of that pixel. Then, based on each correction segmentation label and other pseudo-segmentation labels in the pseudo-segmentation label set of the target image, a target segmentation label set of the target image can be constructed. That is, if the segmentation label in the target segmentation label set of the target image is called the target segmentation label, then for any pixel in the target image, if the uncertainty of the pseudo-segmentation label of that pixel is greater than the uncertainty threshold, the target segmentation label can be determined based on the correction reference parameter of that pixel in each semantic category. If the uncertainty of the pseudo-segmentation label of that pixel is less than or equal to the uncertainty threshold, the pseudo-segmentation label of that pixel can be determined as the target segmentation label of that pixel. If any pixel is the pixel in the i-th row and j-th column of the target image, then the target segmentation label of any pixel can be expressed by the following formula 6.1:

[0112]

[0113] Among them, M r (i,j) represents the target segmentation label of the pixel in the i-th row and j-th column of the target image, U(i,j) represents the uncertainty of the pseudo segmentation label of the pixel in the i-th row and j-th column of the target image, t1 represents the uncertainty threshold, which can be set according to specific needs, M p (h,i,j) represents the h-th pseudo-label indicator value of the pixel in the i-th row and j-th column of the target image during one-hot encoding, C h,i,j The correction reference parameter for the pixel in the i-th row and j-th column of the target image under the h-th semantic category can be specifically shown by the following formula 6.2:

[0114]

[0115] Among them, V h,i,j Let represent the first variance corresponding to the pixel in the i-th row and j-th column of the target image under the h-th semantic type. It represents the mean of the h-th predicted probability of the pixel in the i-th row and j-th column of the target image, which is the expectation of the first subsequence of the pixel in the i-th row and j-th column of the target image under the h-th semantic category.

[0116] In another feasible implementation, when the image processing device determines the correction reference parameter of the target pixel in the h-th semantic category based on the h-th prediction uncertainty and the mean h-th prediction probability of the target pixel, it can also use other methods to determine the correction reference parameter of the target pixel in the h-th semantic category. This allows for the joint measurement of the h-th prediction uncertainty and the mean h-th prediction probability of the target pixel in the h-th semantic category based on the correction reference parameter of the target pixel in the h-th semantic category. That is, the semantic prediction probability of the target pixel being predicted as the h-th semantic category in multiple semantic prediction probability sets and the volatility of the semantic prediction probability of the target pixel being predicted as the h-th semantic category in multiple semantic prediction probability sets can be measured together. For example, the ratio between the mean h-th prediction probability of the target pixel and the h-th prediction uncertainty of the target pixel can be determined as the correction reference parameter of the target pixel in the h-th semantic category. Further, the image processing device can determine the target correction reference parameter from the correction reference parameters of the target pixel in each semantic category, which may include: taking the maximum correction reference parameter of the target pixel in each semantic category as the target correction reference parameter. For example, the ratio between the mean of the h-th predicted probability of the target pixel and the square root of the h-th predicted uncertainty of the target pixel can be determined as the correction reference parameter of the target pixel in the h-th semantic category; or, the ratio between the mean of the h-th predicted probability of the target pixel and the first variance corresponding to the target pixel can be determined as the correction reference parameter of the target pixel in the h-th semantic category; or, the ratio between the mean of the h-th predicted probability of the target pixel and the square root of the first variance corresponding to the target pixel can be determined as the correction reference parameter of the target pixel in the h-th semantic category. This application does not limit the method of determining the correction reference parameter of the target pixel in the h-th semantic category, as long as it can achieve the joint measurement of the h-th predicted uncertainty and the mean of the h-th predicted probability of the target pixel based on the correction reference parameter of the target pixel in the h-th semantic category.

[0117] S406. Based on each corrected segmentation label and other pseudo-segmentation labels in the pseudo-segmentation label set, construct the target segmentation label set for the target image.

[0118] The target segmentation label set is used to train the initial semantic segmentation model.

[0119] In this embodiment, when determining the uncertainty of the pseudo-segmentation label of any pixel in the target image, H semantic categories can be traversed. Based on the difference between the semantic prediction probabilities of any pixel being predicted as the h-th semantic category in multiple semantic prediction probability sets, the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category is determined as a prediction uncertainty of any pixel. Then, one prediction uncertainty can be selected from the H prediction uncertainties of any pixel as the uncertainty of the pseudo-segmentation label of any pixel. Since when the pseudo-segmentation label of any pixel is incorrectly labeled, the difference between the semantic prediction probabilities of any pixel being predicted as the same semantic category obtained through processing by various reference semantic segmentation models should be large. When the pseudo-segmentation label of any pixel is correctly labeled, the difference between the semantic prediction probabilities of any pixel being predicted as the same semantic category obtained through processing by various reference semantic segmentation models should be large. The difference between semantic prediction probabilities for a given semantic category should be small. Therefore, based on the difference between semantic prediction probabilities when any pixel is predicted to be in the same semantic category from multiple semantic prediction probability sets, the uncertainty of the pseudo segmentation label for any pixel is determined to be highly accurate in determining whether the pseudo segmentation label for any pixel is incorrectly labeled. Furthermore, the corrected segmentation label obtained by label correction processing on any pseudo segmentation label with uncertainty greater than the uncertainty threshold satisfies the following: in multiple semantic prediction probability sets, the semantic prediction probability of the target pixel corresponding to any pseudo segmentation label when predicted to be in the semantic category indicated by the corrected segmentation label is high, and the fluctuation of the semantic prediction probability of the target pixel corresponding to any pseudo segmentation label when predicted to be in the semantic category indicated by the corrected segmentation label is small. This can correct incorrect labels and improve labeling accuracy.

[0120] Based on the above image processing scheme, this application provides another image processing method, which is described using H as the number of semantic categories, where H is an integer greater than or equal to 2. See also Figure 5 This is a schematic flowchart of another image processing method provided in an embodiment of this application. Figure 5 The image processing method shown can be executed by an image processing device, or by other electronic devices with computing power, either alone or in collaboration. In this application embodiment, an image processing device is used as an example. Figure 5 The image processing method shown may include the following steps:

[0121] S501, Obtain the target image and the pseudo-segmentation label set of the target image.

[0122] The pseudo-segmentation label set includes pseudo-segmentation labels used to indicate the semantic category to which each pixel in the target image belongs.

[0123] S502, the target image is semantically segmented using multiple reference semantic segmentation models to obtain the semantic prediction probability set corresponding to each reference semantic segmentation model.

[0124] Each semantic prediction probability set includes: each pixel in the target image, which is predicted as the semantic prediction probability of each of the H semantic categories, and each reference semantic segmentation model is obtained by training the initial semantic segmentation model.

[0125] S503, for any pixel in the target image, traverse H semantic categories, and based on the difference between the semantic prediction probabilities when any pixel is predicted to be the h-th semantic category in multiple semantic prediction probability sets, determine the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category, and use it as a prediction uncertainty of any pixel.

[0126] The relevant processes of steps S501 to S503 are similar to those of steps S401 to S403 above, and will not be repeated here, h∈[1,H].

[0127] S504, Obtain the semantic prediction result set; the semantic prediction result set includes: the semantic prediction result of each pixel in the target image.

[0128] The semantic prediction result of any pixel is used to indicate the semantic category to which the predicted pixel belongs. The semantic prediction result of any pixel is determined based on the semantic prediction probabilities of each semantic category in any semantic prediction probability set. The semantic category indicated by the semantic prediction result of any pixel is the semantic category indicated by the maximum semantic prediction probability corresponding to the pixel in any semantic prediction probability set used to determine the semantic prediction result. Since each reference semantic segmentation model is trained from the initial semantic segmentation model, if the number of reference semantic segmentation models is N, where N is an integer greater than or equal to 2, the N reference semantic segmentation models are obtained by iteratively training the initial semantic segmentation model for N rounds. In the following embodiments of this application, the semantic prediction probability set used to determine the semantic prediction result is the Nth semantic prediction probability set determined by the Nth reference semantic segmentation model. Furthermore, when the semantic prediction result of any pixel is one-hot encoded, the semantic prediction result of any pixel may include H prediction label indication values. Among the H prediction label indication values ​​of any pixel, the prediction label indication value of the semantic category indicated by the maximum semantic prediction probability corresponding to any pixel is 1, and the other prediction label indication values ​​of any pixel are 0. For example, if in the Nth semantic prediction probability set, any pixel is predicted to be in H semantic categories with semantic prediction probabilities of {0.01,0.9,0.02,0.03,0.04}, then the one-hot encoding of the semantic prediction result of any pixel is {0,1,0,0,0}.

[0129] S505: Based on the difference between the semantic prediction results of each pixel in the semantic prediction result set and the pseudo segmentation labels of the corresponding pixels in the pseudo segmentation label set, Z uncertainties are obtained from the H prediction uncertainties of any pixel.

[0130] Where Z∈[0,H]; the image processing device selects Z uncertainties from H prediction uncertainties of any pixel based on the difference between the semantic prediction results of each pixel in the semantic prediction result set and the pseudo segmentation labels of the corresponding pixels in the pseudo segmentation label set. This is to eliminate semantic categories that are difficult to identify, because for semantic categories that are difficult to identify, such as "sofa" and "chair", even if the initial semantic segmentation model is trained with correctly labeled pseudo segmentation labels, the reference semantic segmentation model is still difficult to identify semantic categories that are difficult to identify, and the semantic prediction probability of the corresponding pixel is also more uncertain under semantic categories that are difficult to identify.

[0131] In specific implementation, the image processing device, based on the difference between the semantic prediction results of each pixel in the semantic prediction result set and the pseudo segmentation labels of the corresponding pixels in the pseudo segmentation label set, selects Z filtered uncertainties from H prediction uncertainties of any pixel. This can include: traversing the H prediction uncertainties of any pixel; for the h-th prediction uncertainty of any pixel, based on the semantic prediction result of any pixel, determining a prediction indication value to indicate whether the predicted semantic category of any pixel belongs to the h-th semantic category, and based on the pseudo segmentation label of any pixel, determining a reference indication value to indicate whether the semantic category of any pixel belongs to the h-th semantic category; based on the difference between the prediction indication value corresponding to each pixel in the target image and the reference indication value corresponding to the corresponding pixel, determining whether to include the h-th prediction uncertainty of any pixel as a filtered uncertainty; until Z filtered uncertainties are obtained. The process of determining a reference indicator value for indicating whether the semantic category to which the pixel belongs is the h-th semantic category based on the pseudo segmentation label of any pixel has been described in step S403 above and will not be repeated here. The process of determining a predicted indicator value for indicating whether the semantic category to which the pixel belongs is the h-th semantic category based on the semantic prediction result of any pixel is similar to the process of determining a reference indicator value for indicating whether the semantic category to which the pixel belongs is the h-th semantic category based on the pseudo segmentation label of any pixel, and will not be repeated here.

[0132] In one feasible implementation, the image processing device determines whether to include the h-th prediction uncertainty of any pixel as a filtered uncertainty based on the difference between the predicted indicator value corresponding to each pixel in the target image and the reference indicator value corresponding to the corresponding pixel. This can include: calculating the intersection-union ratio (IUR) between the predicted indicator value corresponding to each pixel in the target image and the reference indicator value corresponding to each pixel in the target image; if the IUR is greater than a filtering threshold, then the h-th prediction uncertainty of any pixel is included as a filtered uncertainty. Further optionally, if the IUR is less than or equal to the filtering threshold, a preset value can be used as a filtered uncertainty. This preset value can be set according to specific needs, and should be less than or equal to the minimum IUR determined under each semantic category. For example, the preset value can be 0 or a negative number. This embodiment uses the preset value as 0, in which case Z = H. See Formula 7 below, which is a formula for determining the filtered uncertainty provided by this embodiment:

[0133]

[0134] in, This represents the matrix consisting of the uncertainty of the semantic prediction probability of each pixel in the target image under the h-th semantic category; that is, the matrix consisting of the h-th prediction uncertainty of each pixel in the target image. Let represent the matrix composed of the predicted indicator values ​​corresponding to each pixel in the target image under the h-th semantic category, that is, the matrix composed of the h-th predicted label indicator values ​​of each pixel in the target image. This represents a matrix composed of the reference indicator values ​​corresponding to each pixel in the target image, specifically a matrix composed of the h-th pseudo-label indicator values ​​of each pixel in the target image; t2 is the filtering threshold, which can be set according to specific requirements; O is an all-zero matrix. This represents the matrix consisting of the uncertainties of each pixel in the target image after filtering, under the h-th semantic category.

[0135] S506: Select one prediction uncertainty from the Z filtered uncertainties of any pixel as the uncertainty of the pseudo segmentation label of any pixel.

[0136] S507, perform label correction processing on pseudo segmentation labels in the pseudo segmentation label set whose uncertainty is greater than the uncertainty threshold to obtain corrected segmentation labels.

[0137] S508. Based on each corrected segmentation label and other pseudo-segmentation labels in the pseudo-segmentation label set, construct the target segmentation label set for the target image.

[0138] The target segmentation label set is used to train the initial semantic segmentation model. The related processes of steps S506 to S508 are similar to those of steps S404 to S406 above, and will not be described again here.

[0139] In one embodiment, if the fourth variance corresponding to a pixel is selected as the uncertainty of the semantic prediction probability of that pixel under the h-th semantic category, Z uncertainties are obtained from the H prediction uncertainties of the pixel using the method indicated by Formula 7, and then the uncertainty of the pseudo-segmentation label of the pixel is determined from the Z uncertainties of the pixel; see also Figure 6a This is a schematic diagram illustrating the determination of the uncertainty of pseudo-segmentation labels for each pixel in a target image, provided by an embodiment of this application. The target image is shown as labeled 601; the true segmentation labels for each pixel in the target image are shown as labeled 602, indicating the actual semantic category to which each pixel in the target image belongs; the pseudo-segmentation labels for each pixel in the target image are shown as labeled 603; and the region shown as labeled 604 represents a region with a relatively high uncertainty in the pseudo-segmentation labels of pixels. (See also...) Figure 6bThis is a schematic diagram illustrating another method for determining the uncertainty of pseudo-segmentation labels for each pixel in a target image, provided by an embodiment of this application. The target image is shown as labeled 611, the true segmentation labels for each pixel in the target image are shown as labeled 612, the pseudo-segmentation labels for each pixel in the target image are shown as labeled 613, and the area shown as labeled 614 represents a region with a relatively high uncertainty in the pseudo-segmentation labels of pixels. See also... Figure 6c This is a schematic diagram illustrating another method for determining the uncertainty of pseudo-segmentation labels for each pixel in a target image, provided by an embodiment of this application. The target image is shown as labeled 621, the true segmentation labels for each pixel in the target image are shown as labeled 622, the pseudo-segmentation labels for each pixel in the target image are shown as labeled 623, and the area shown as labeled 624 represents a region with a relatively high uncertainty in the pseudo-segmentation labels of pixels. See also... Figure 6d This is a schematic diagram of another method for determining the uncertainty of pseudo segmentation labels of each pixel in a target image, provided by an embodiment of this application. The target image is shown as marked 631, the true segmentation labels of each pixel in the target image are shown as marked 632, the pseudo segmentation labels of each pixel in the target image are shown as marked 633, and the area shown as marked 634 is an area with a large uncertainty of the pseudo segmentation labels of pixels.

[0140] In one embodiment, when the image processing method provided in this application is applied to a corresponding method in weakly supervised semantic segmentation or semi-supervised semantic segmentation, for example: a self-supervised equivariant attention mechanism for weakly supervised semantic segmentation (referred to as Method 1), anti-adversarially manipulated attributions for weakly and semi-supervised semantic segmentation (referred to as Method 2), cross-language image matching for weakly supervised semantic segmentation (referred to as Method 3), and an embedded discriminative attention mechanism for weakly supervised semantic segmentation. Semantic segmentation, referred to as method 4), compares the pseudo segmentation labels of pixels generated without the image processing method provided in this application with the target segmentation labels of pixels corrected by the image processing method provided in this application. It is found that the corrected target segmentation labels of pixels have higher accuracy and are closer to the true segmentation labels of pixels. See also... Figure 7aThis is a schematic diagram illustrating how to obtain target segmentation labels for each pixel in a target image according to an embodiment of this application. The target image is shown as labeled 700. Label 701 shows the actual segmentation labels for each pixel in the target image, indicating the semantic category to which each pixel in the target image actually belongs. Label 702 shows the pseudo-segmentation labels for each pixel in the target image obtained based on method 1. Label 703 shows the corrected target segmentation labels for each pixel in the target image. Label 704 shows the pseudo-segmentation labels for each pixel in the target image obtained based on method 2. Label 705 shows the corrected target segmentation labels for each pixel in the target image. Label 706 shows the pseudo-segmentation labels for each pixel in the target image obtained based on method 3. Label 707 shows the corrected target segmentation labels for each pixel in the target image. Label 708 shows the pseudo-segmentation labels for each pixel in the target image obtained based on method 4. Label 709 shows the corrected target segmentation labels for each pixel in the target image.

[0141] See Figure 7b This is a schematic diagram of another method for obtaining target segmentation labels for each pixel in a target image, provided by an embodiment of this application. The target image is shown as labeled 710. Label 711 shows the actual segmentation labels for each pixel in the target image, indicating the actual semantic category to which each pixel in the target image belongs. Label 712 shows the pseudo segmentation labels for each pixel in the target image obtained based on method 1. Label 713 shows the corrected target segmentation labels for each pixel in the target image. Label 714 shows the pseudo segmentation labels for each pixel in the target image obtained based on method 2. Label 715 shows the corrected target segmentation labels for each pixel in the target image. Label 716 shows the pseudo segmentation labels for each pixel in the target image obtained based on method 3. Label 717 shows the corrected target segmentation labels for each pixel in the target image. Label 718 shows the pseudo segmentation labels for each pixel in the target image obtained based on method 4. Label 719 shows the corrected target segmentation labels for each pixel in the target image.

[0142] In one embodiment, commonly used evaluation metrics in the field of semantic segmentation can be used to evaluate the label correction effect of the image processing method provided in this application embodiment. For example, the mean Intersection over Union (mIoU) can be used for evaluation, where the mean Intersection over Union (mIoU) is the average value of the intersection over union ratio under each semantic category. Referring to Table 1, it shows the evaluation values ​​of Method 1, Method 2, Method 3, and Method 4 in weakly supervised semantic segmentation technology before and after introducing the image processing method provided in this application embodiment (referred to as this method). Here, "uncorrected" corresponds to the evaluation of the pseudo segmentation labels of each pixel in the image generated without introducing this method, and "corrected" corresponds to the evaluation of the target segmentation labels of each pixel in the image obtained after correcting the pseudo segmentation labels of each pixel in the image after introducing this method. As can be seen from Table 1, the evaluation values ​​after introducing this method are significantly increased compared with those without introducing this method, thus indicating that the label correction effect of the image processing method provided in this application embodiment is good.

[0143] Table 1

[0144]

[0145] In one embodiment, commonly used evaluation metrics in the field of semantic segmentation can also be used to evaluate the performance of the semantic segmentation model. Table 2 shows the evaluation values ​​of the comparative semantic segmentation model and the target semantic segmentation model trained on the same initial semantic segmentation model using the same training set and the same validation set and the same test set. The comparative semantic segmentation model is trained using pseudo-segmentation labels generated by methods 1, 2, 3, or 4 in weakly supervised semantic segmentation techniques. The target semantic segmentation model is trained using target segmentation labels obtained by correcting the pseudo-segmentation labels. As shown in Table 2, the target semantic segmentation model trained using target segmentation labels obtained by correcting the pseudo-segmentation labels performs better.

[0146] Table 2

[0147]

[0148] In this embodiment, when determining the uncertainty of the pseudo-segmentation label of any pixel in the target image, the difference between the semantic prediction results of each pixel in the semantic prediction result set and the pseudo-segmentation labels of the corresponding pixels in the pseudo-segmentation label set can be used to filter Z uncertainties from H uncertainties of any pixel. Furthermore, one uncertainty can be selected from the Z uncertainties of any pixel as the uncertainty of the pseudo-segmentation label of that pixel. Semantic categories that are difficult to identify can be eliminated because even if the initial semantic segmentation model is trained with correctly labeled pseudo-segmentation labels, the reference semantic segmentation model still has difficulty identifying these categories. The uncertainty of the semantic prediction probability of the corresponding pixel under these categories is also relatively large. Therefore, after eliminating the uncertainty of the semantic prediction probability of pixels under these categories, the uncertainty of the determined pseudo-segmentation label of the pixel is more accurate, thus making the identification of incorrectly labeled pseudo-segmentation labels in each pixel of the target image more accurate.

[0149] Based on the above-described image processing method embodiments, this application provides an image processing apparatus. See also... Figure 8 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application. The image processing device may include an acquisition unit 801 and a processing unit 802. Figure 8 The image processing apparatus shown can operate the following units:

[0150] The acquisition unit 801 is used to acquire a target image and a pseudo segmentation label set of the target image; the pseudo segmentation label set includes: pseudo segmentation labels used to indicate the semantic category to which each pixel in the target image belongs;

[0151] The processing unit 802 is used to perform semantic segmentation processing on the target image through multiple reference semantic segmentation models to obtain a semantic prediction probability set corresponding to each reference semantic segmentation model; any semantic prediction probability set includes: each pixel in the target image is predicted as the semantic prediction probability of each semantic category in multiple semantic categories, and each reference semantic segmentation model is obtained by training an initial semantic segmentation model.

[0152] The processing unit 802 is further configured to determine the uncertainty of the pseudo segmentation label of any pixel for any pixel in the target image, based on the difference between the semantic prediction probabilities predicted for any pixel in multiple semantic prediction probability sets.

[0153] The processing unit 802 is further configured to perform label correction processing on the pseudo segmentation labels in the pseudo segmentation label set whose uncertainty is greater than the uncertainty threshold, to obtain corrected segmentation labels.

[0154] The processing unit 802 is further configured to construct a target segmentation label set for the target image based on each corrected segmentation label and other pseudo-segmentation labels in the pseudo-segmentation label set; the target segmentation label set is used to train the initial semantic segmentation model.

[0155] In one embodiment, the number of the plurality of semantic categories is H, where H is an integer greater than or equal to 2;

[0156] When determining the uncertainty of the pseudo-segmentation label of any pixel based on the difference between the semantic prediction probabilities predicted for any pixel in multiple semantic prediction probability sets, the processing unit 802 specifically performs the following operations:

[0157] Traverse H semantic categories, and based on the multiple semantic prediction probability sets, determine the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category based on the difference between the semantic prediction probabilities when any pixel is predicted to be the h-th semantic category, and use it as a prediction uncertainty of any pixel, h∈[1,H];

[0158] From the H prediction uncertainties of any given pixel, select one prediction uncertainty as the uncertainty of the pseudo segmentation label of that given pixel.

[0159] In one embodiment, when the processing unit 802 determines the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category based on the difference between the semantic prediction probabilities of any pixel being predicted as the h-th semantic category in the plurality of semantic prediction probability sets, it specifically performs the following operations:

[0160] For the semantic prediction probability of any pixel being predicted as the h-th semantic category in the multiple semantic prediction probability sets, the variance is calculated to obtain the first variance.

[0161] The first variance is used as the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category.

[0162] In one embodiment, when the processing unit 802 determines the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category based on the difference between the semantic prediction probabilities of any pixel being predicted as the h-th semantic category in the plurality of semantic prediction probability sets, it specifically performs the following operations:

[0163] Based on the pseudo segmentation label of any pixel, a reference indication value is determined to indicate whether the semantic category to which any pixel belongs is the h-th semantic category;

[0164] For the multiple semantic prediction probability sets, the semantic prediction probability when any pixel is predicted to be the h-th semantic category, and the reference indication value corresponding to any pixel are subjected to variance calculation to obtain the second variance;

[0165] The second variance is used as the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category.

[0166] In one embodiment, when the processing unit 802 determines the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category based on the difference between the semantic prediction probabilities of any pixel being predicted as the h-th semantic category in the plurality of semantic prediction probability sets, it specifically performs the following operations:

[0167] A post-processing algorithm is used to post-process the semantic prediction probability of any pixel being predicted as the h-th semantic category in the multiple semantic prediction probability sets to obtain the post-processed semantic prediction probability.

[0168] Based on the difference between multiple post-processed semantic prediction probabilities when any pixel is predicted to be the h-th semantic category, the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category is determined.

[0169] In one embodiment, when the processing unit 802 selects one prediction uncertainty from H prediction uncertainties of any pixel as the uncertainty of the pseudo segmentation label of any pixel, it specifically performs the following operation:

[0170] The maximum prediction uncertainty among the H prediction uncertainties of any given pixel is taken as the uncertainty of the pseudo segmentation label of that given pixel.

[0171] In one embodiment, the H semantic categories include H-1 preset categories and a background category;

[0172] When the processing unit 802 selects one prediction uncertainty from the H prediction uncertainties of any pixel as the uncertainty of the pseudo segmentation label of any pixel, it specifically performs the following operation:

[0173] From the H prediction uncertainties of any pixel, determine H-1 prediction uncertainties corresponding to the H-1 preset categories;

[0174] The largest prediction uncertainty among the H-1 prediction uncertainties is taken as the uncertainty of the pseudo segmentation label of any pixel.

[0175] In one embodiment, the processing unit 802 is further configured to:

[0176] Obtain a semantic prediction result set; the semantic prediction result set includes: the semantic prediction result of each pixel in the target image, the semantic prediction result of any pixel is used to indicate: the semantic category to which the predicted pixel belongs, the semantic prediction result of any pixel is based on any semantic prediction probability set, and the pixel is predicted to be determined by the semantic prediction probability of each semantic category;

[0177] Based on the semantic prediction results of each pixel in the semantic prediction result set and the difference between the pseudo segmentation labels of the corresponding pixels in the pseudo segmentation label set, Z uncertainties are obtained from the H prediction uncertainties of any pixel, Z∈[0,H].

[0178] When the processing unit 802 selects one prediction uncertainty from the H prediction uncertainties of any pixel as the uncertainty of the pseudo segmentation label of any pixel, it specifically performs the following operation:

[0179] From the Z filtered uncertainties of any given pixel, select one prediction uncertainty as the uncertainty of the pseudo segmentation label of that given pixel.

[0180] In one embodiment, when the processing unit 802 selects Z uncertainties from H prediction uncertainties of any pixel based on the difference between the semantic prediction results of each pixel in the semantic prediction result set and the pseudo segmentation labels of the corresponding pixels in the pseudo segmentation label set, it specifically performs the following operation:

[0181] Traverse the H prediction uncertainties of any pixel. For the h-th prediction uncertainty of any pixel, based on the semantic prediction result of any pixel, determine a prediction indication value to indicate whether the predicted semantic category of any pixel belongs to the h-th semantic category. Also, based on the pseudo segmentation label of any pixel, determine a reference indication value to indicate whether the semantic category of any pixel belongs to the h-th semantic category.

[0182] Based on the difference between the predicted indicator value corresponding to each pixel in the target image and the reference indicator value corresponding to the corresponding pixel, it is determined whether to use the h-th prediction uncertainty of any pixel as a filtered uncertainty; until Z filtered uncertainties are obtained.

[0183] In one embodiment, when the processing unit 802 determines whether to include the h-th prediction uncertainty of any pixel as a filtered uncertainty based on the difference between the predicted indicator value corresponding to each pixel in the target image and the reference indicator value corresponding to the corresponding pixel, it specifically performs the following operations:

[0184] Calculate the cross-union ratio between the predicted indicator value corresponding to each pixel in the target image and the reference indicator value corresponding to each pixel in the target image;

[0185] If the intersection-union ratio is greater than the filtering threshold, then the h-th prediction uncertainty of any pixel is taken as a filtered uncertainty.

[0186] In one embodiment, the number of the plurality of semantic categories is H, where H is an integer greater than or equal to 2;

[0187] The processing unit 802 performs label correction processing on the pseudo segmentation labels in the pseudo segmentation label set whose uncertainty is greater than the uncertainty threshold. When obtaining the corrected segmentation labels, the following operations are specifically performed:

[0188] For any pseudo-segmentation label with an uncertainty greater than the uncertainty threshold, obtain the h-th prediction uncertainty of the target pixel corresponding to the pseudo-segmentation label and the h-th prediction probability mean of the target pixel; the h-th prediction uncertainty of the target pixel refers to the uncertainty of the semantic prediction probability of the target pixel under the h-th semantic category, and the h-th prediction probability mean of the target pixel is obtained by averaging the semantic prediction probabilities when the target pixel is predicted to be the h-th semantic category in the multiple semantic prediction probability sets;

[0189] Based on the h-th prediction uncertainty of the target pixel and the mean h-th prediction probability of the target pixel, the correction reference parameter of the target pixel under the h-th semantic category is determined, h∈[1,H];

[0190] The target correction reference parameter is determined from the correction reference parameters of the target pixel under each semantic category, and the semantic category indicated by the target correction reference parameter is used as the correction segmentation label corresponding to any pseudo segmentation label.

[0191] In one embodiment, when the processing unit 802 determines the corrected reference parameter of the target pixel under the h-th semantic category based on the h-th prediction uncertainty of the target pixel and the h-th prediction probability mean of the target pixel, it specifically performs the following operations:

[0192] The ratio between the h-th prediction uncertainty of the target pixel and the mean of the h-th prediction probability of the target pixel is determined as the correction reference parameter of the target pixel under the h-th semantic category;

[0193] When the processing unit 802 determines the target correction reference parameters from the correction reference parameters of the target pixel under each semantic category, it specifically performs the following operations:

[0194] The minimum correction reference parameter among the correction reference parameters of the target pixel under each semantic category is taken as the target correction reference parameter.

[0195] According to one embodiment of this application, Figure 2 , Figure 4 as well as Figure 5 The image processing method shown can involve various steps that can be derived from... Figure 8 This is performed by the individual units in the image processing apparatus shown. For example, Figure 2 The step S201 shown can be performed by Figure 8 The acquisition unit 801 in the image processing apparatus shown performs this operation. Figure 2 Steps S202 to S205 shown can be derived from Figure 8 The image processing unit 802 in the illustrated image processing apparatus performs the operation. For example, Figure 4 The step S401 shown can be performed by Figure 8 The acquisition unit 801 in the image processing apparatus shown performs this operation. Figure 4 Steps S402 to S406 shown can be derived from Figure 8 The image processing unit 802 in the illustrated image processing apparatus performs the operation. For example, Figure 5 The step S501 shown can be performed by Figure 8 The acquisition unit 801 in the image processing apparatus shown performs this operation. Figure 5 Steps S502 to S508 shown can be derived from... Figure 8 The processing unit 802 in the image processing apparatus shown performs the operation.

[0196] According to another embodiment of this application, Figure 8 The various units in the illustrated image processing apparatus can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. The above-mentioned units are based on logical function division. In practical applications, the function of one unit can also be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the image processing apparatus based on logical function division may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0197] According to another embodiment of this application, the following can be achieved by running on a general-purpose computing device, such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), a device capable of performing operations such as... Figure 2 , Figure 4 as well as Figure 5 The computer program (including program code) for each step involved in the corresponding method shown, to construct such... Figure 8 The image processing apparatus shown herein, and the image processing method for implementing the embodiments of this application, are described. The computer program may be recorded on, for example, a computer-readable storage medium, loaded onto the aforementioned computing device via the computer-readable storage medium, and run therein.

[0198] In this embodiment, the target image can be semantically segmented using multiple reference semantic segmentation models trained on the initial semantic segmentation model, resulting in a semantic prediction probability set corresponding to each reference semantic segmentation model. Each semantic prediction probability set includes the semantic prediction probabilities of each pixel in the target image as a semantic category among multiple semantic categories. Then, for any pixel in the target image, based on the differences between the predicted semantic prediction probabilities in the multiple semantic prediction probability sets, the uncertainty of the pseudo-segmentation label for that pixel can be determined. For pseudo-segmentation labels in the pseudo-segmentation label set whose uncertainty exceeds an uncertainty threshold, label correction processing is performed to obtain corrected segmentation labels. Furthermore, based on each corrected segmentation label and other pseudo-segmentation labels in the target image's pseudo-segmentation label set, a target segmentation label set for training the initial semantic segmentation model can be constructed. Errors in the target image's pseudo-segmentation label set can be determined based on the uncertainty of the pseudo-segmentation labels of each pixel in the target image, thereby correcting errors and improving label accuracy to ensure the performance of the target semantic segmentation model trained on the initial semantic segmentation model based on the target segmentation label set.

[0199] Based on the above-described image processing method and apparatus embodiments, this application also provides an image processing device. See also... Figure 9 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application. Figure 9 The image processing device shown may include at least a processor 901, an input interface 902, an output interface 903, and a computer storage medium 904. The processor 901, input interface 902, output interface 903, and computer storage medium 904 may be connected via a bus or other means.

[0200] The computer storage medium 904 can be stored in the memory of the image processing device. The computer storage medium 904 is used to store computer programs, which include program instructions. The processor 901 is used to execute the program instructions stored in the computer storage medium 904. The processor 901 (or CPU (Central Processing Unit)) is the computing and control core of the image processing device. It is suitable for implementing one or more instructions, specifically for loading and executing one or more instructions to realize the above-mentioned image processing method flow or corresponding functions.

[0201] This application embodiment also provides a computer storage medium (Memory), which is a memory device in an image processing device used to store programs and data. It is understood that the computer storage medium here can include the built-in storage medium in the terminal, or it can include an extended storage medium supported by the terminal. The computer storage medium provides storage space, which stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by the processor 901. These instructions can be one or more computer programs (including program code). It should be noted that the computer storage medium here can be a high-speed random access memory (RAM), or a non-volatile memory, such as at least one disk storage device; optionally, it can also be at least one computer storage medium located remotely from the aforementioned processor.

[0202] In one embodiment, the processor 901 may load and execute one or more instructions stored in the computer storage medium to implement the aforementioned related... Figure 2 , Figure 4 as well as Figure 5 In the corresponding steps of the image processing method embodiment, specifically, the processor 901 can be used for:

[0203] Obtain a target image and a set of pseudo-segmentation labels for the target image; the set of pseudo-segmentation labels includes pseudo-segmentation labels that indicate the semantic category to which each pixel in the target image belongs;

[0204] The target image is semantically segmented using multiple reference semantic segmentation models to obtain semantic prediction probability sets corresponding to each reference semantic segmentation model. Each semantic prediction probability set includes: each pixel in the target image is predicted as a semantic prediction probability of each semantic category among multiple semantic categories. Each reference semantic segmentation model is obtained by training an initial semantic segmentation model.

[0205] For any pixel in the target image, based on multiple semantic prediction probability sets, the uncertainty of the pseudo segmentation label of any pixel is determined by the difference between the semantic prediction probabilities predicted for any pixel.

[0206] For each pseudo segmentation label in the set of pseudo segmentation labels, the pseudo segmentation labels with uncertainty greater than the uncertainty threshold are subjected to label correction processing to obtain corrected segmentation labels;

[0207] Based on each corrected segmentation label and other pseudo-segmentation labels in the pseudo-segmentation label set, a target segmentation label set for the target image is constructed; the target segmentation label set is used to train the initial semantic segmentation model.

[0208] In one embodiment, the number of the plurality of semantic categories is H, where H is an integer greater than or equal to 2;

[0209] When determining the uncertainty of the pseudo-segmentation label of any pixel based on the difference between the semantic prediction probabilities predicted for any pixel in multiple semantic prediction probability sets, the processor 901 specifically performs the following operations:

[0210] Traverse H semantic categories, and based on the multiple semantic prediction probability sets, determine the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category based on the difference between the semantic prediction probabilities when any pixel is predicted to be the h-th semantic category, and use it as a prediction uncertainty of any pixel, h∈[1,H];

[0211] From the H prediction uncertainties of any given pixel, select one prediction uncertainty as the uncertainty of the pseudo segmentation label of that given pixel.

[0212] In one embodiment, when determining the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category based on the difference between the semantic prediction probabilities of any pixel being predicted as the h-th semantic category in the plurality of semantic prediction probability sets, the processor 901 specifically performs the following operations:

[0213] For the semantic prediction probability of any pixel being predicted as the h-th semantic category in the multiple semantic prediction probability sets, the variance is calculated to obtain the first variance.

[0214] The first variance is used as the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category.

[0215] In one embodiment, when determining the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category based on the difference between the semantic prediction probabilities of any pixel being predicted as the h-th semantic category in the plurality of semantic prediction probability sets, the processor 901 specifically performs the following operations:

[0216] Based on the pseudo segmentation label of any pixel, a reference indication value is determined to indicate whether the semantic category to which any pixel belongs is the h-th semantic category;

[0217] For the multiple semantic prediction probability sets, the semantic prediction probability when any pixel is predicted to be the h-th semantic category, and the reference indication value corresponding to any pixel are subjected to variance calculation to obtain the second variance;

[0218] The second variance is used as the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category.

[0219] In one embodiment, when determining the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category based on the difference between the semantic prediction probabilities of any pixel being predicted as the h-th semantic category in the plurality of semantic prediction probability sets, the processor 901 specifically performs the following operations:

[0220] A post-processing algorithm is used to post-process the semantic prediction probability of any pixel being predicted as the h-th semantic category in the multiple semantic prediction probability sets to obtain the post-processed semantic prediction probability.

[0221] Based on the difference between multiple post-processed semantic prediction probabilities when any pixel is predicted to be the h-th semantic category, the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category is determined.

[0222] In one embodiment, when the processor 901 selects one prediction uncertainty from H prediction uncertainties of any pixel as the uncertainty of the pseudo segmentation label of any pixel, it specifically performs the following operation:

[0223] The maximum prediction uncertainty among the H prediction uncertainties of any given pixel is taken as the uncertainty of the pseudo segmentation label of that given pixel.

[0224] In one embodiment, the H semantic categories include H-1 preset categories and a background category;

[0225] When the processor 901 selects one prediction uncertainty from the H prediction uncertainties of any pixel as the uncertainty of the pseudo segmentation label of any pixel, it specifically performs the following operations:

[0226] From the H prediction uncertainties of any pixel, determine H-1 prediction uncertainties corresponding to the H-1 preset categories;

[0227] The largest prediction uncertainty among the H-1 prediction uncertainties is taken as the uncertainty of the pseudo segmentation label of any pixel.

[0228] In one embodiment, the processor 901 is further configured to:

[0229] Obtain a semantic prediction result set; the semantic prediction result set includes: the semantic prediction result of each pixel in the target image, the semantic prediction result of any pixel is used to indicate: the semantic category to which the predicted pixel belongs, the semantic prediction result of any pixel is based on any semantic prediction probability set, and the pixel is predicted to be determined by the semantic prediction probability of each semantic category;

[0230] Based on the semantic prediction results of each pixel in the semantic prediction result set and the difference between the pseudo segmentation labels of the corresponding pixels in the pseudo segmentation label set, Z uncertainties are obtained from the H prediction uncertainties of any pixel, Z∈[0,H].

[0231] When the processor 901 selects one prediction uncertainty from the H prediction uncertainties of any pixel as the uncertainty of the pseudo segmentation label of any pixel, it specifically performs the following operations:

[0232] From the Z filtered uncertainties of any given pixel, select one prediction uncertainty as the uncertainty of the pseudo segmentation label of that given pixel.

[0233] In one embodiment, when the processor 901 selects Z uncertainties from H prediction uncertainties of any pixel based on the difference between the semantic prediction result of each pixel in the semantic prediction result set and the pseudo segmentation label of the corresponding pixel in the pseudo segmentation label set, it specifically performs the following operation:

[0234] Traverse the H prediction uncertainties of any pixel. For the h-th prediction uncertainty of any pixel, based on the semantic prediction result of any pixel, determine a prediction indication value to indicate whether the predicted semantic category of any pixel belongs to the h-th semantic category. Also, based on the pseudo segmentation label of any pixel, determine a reference indication value to indicate whether the semantic category of any pixel belongs to the h-th semantic category.

[0235] Based on the difference between the predicted indicator value corresponding to each pixel in the target image and the reference indicator value corresponding to the corresponding pixel, it is determined whether to use the h-th prediction uncertainty of any pixel as a filtered uncertainty; until Z filtered uncertainties are obtained.

[0236] In one embodiment, when the processor 901 determines whether to include the h-th prediction uncertainty of any pixel as a filtered uncertainty based on the difference between the predicted indicator value corresponding to each pixel in the target image and the reference indicator value corresponding to the corresponding pixel, the following operations are specifically performed:

[0237] Calculate the cross-union ratio between the predicted indicator value corresponding to each pixel in the target image and the reference indicator value corresponding to each pixel in the target image;

[0238] If the intersection-union ratio is greater than the filtering threshold, then the h-th prediction uncertainty of any pixel is taken as a filtered uncertainty.

[0239] In one embodiment, the number of the plurality of semantic categories is H, where H is an integer greater than or equal to 2;

[0240] The processor 901 performs label correction processing on the pseudo-segmentation labels in the pseudo-segmentation label set whose uncertainty is greater than the uncertainty threshold. When obtaining the corrected segmentation labels, the processor performs the following operations:

[0241] For any pseudo-segmentation label with an uncertainty greater than the uncertainty threshold, obtain the h-th prediction uncertainty of the target pixel corresponding to the pseudo-segmentation label and the h-th prediction probability mean of the target pixel; the h-th prediction uncertainty of the target pixel refers to the uncertainty of the semantic prediction probability of the target pixel under the h-th semantic category, and the h-th prediction probability mean of the target pixel is obtained by averaging the semantic prediction probabilities when the target pixel is predicted to be the h-th semantic category in the multiple semantic prediction probability sets;

[0242] Based on the h-th prediction uncertainty of the target pixel and the mean h-th prediction probability of the target pixel, the correction reference parameter of the target pixel under the h-th semantic category is determined, h∈[1,H];

[0243] The target correction reference parameter is determined from the correction reference parameters of the target pixel under each semantic category, and the semantic category indicated by the target correction reference parameter is used as the correction segmentation label corresponding to any pseudo segmentation label.

[0244] In one embodiment, when the processor 901 determines the correction reference parameter of the target pixel under the h-th semantic category based on the h-th prediction uncertainty of the target pixel and the mean h-th prediction probability of the target pixel, it specifically performs the following operations:

[0245] The ratio between the h-th prediction uncertainty of the target pixel and the mean of the h-th prediction probability of the target pixel is determined as the correction reference parameter of the target pixel under the h-th semantic category;

[0246] When the processor 901 determines the target correction reference parameters from the correction reference parameters of the target pixel under each semantic category, it specifically performs the following operations:

[0247] The minimum correction reference parameter among the correction reference parameters of the target pixel under each semantic category is taken as the target correction reference parameter.

[0248] This application provides a computer program product, which includes a computer program stored in a computer storage medium. A processor of an image processing device reads the computer program from the computer storage medium and executes the computer program, causing the image processing device to perform the aforementioned actions. Figure 2 , Figure 4 as well as Figure 5 The method embodiment shown. The computer-readable storage medium may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0249] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, include: Obtain a target image and a set of pseudo-segmentation labels for the target image; the set of pseudo-segmentation labels includes pseudo-segmentation labels that indicate the semantic category to which each pixel in the target image belongs; The target image is semantically segmented using multiple reference semantic segmentation models to obtain semantic prediction probability sets corresponding to each reference semantic segmentation model. Each semantic prediction probability set includes: each pixel in the target image is predicted as the semantic prediction probability of each semantic category among multiple semantic categories. Each reference semantic segmentation model is obtained by training an initial semantic segmentation model. The number of multiple semantic categories is H, where H is an integer greater than or equal to 2. For any pixel in the target image, H semantic categories are traversed. Based on the differences between the semantic prediction probabilities when any pixel is predicted to be the h-th semantic category in multiple semantic prediction probability sets, the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category is determined as a prediction uncertainty of any pixel, h∈[1,H]. Obtain a semantic prediction result set; the semantic prediction result set includes: the semantic prediction result of each pixel in the target image, the semantic prediction result of any pixel is used to indicate: the semantic category to which the predicted pixel belongs, the semantic prediction result of any pixel is based on any semantic prediction probability set, and the pixel is predicted to be determined by the semantic prediction probability of each semantic category; Based on the semantic prediction results of each pixel in the semantic prediction result set and the difference between the pseudo segmentation labels of the corresponding pixels in the pseudo segmentation label set, Z uncertainties are obtained from the H prediction uncertainties of any pixel, Z∈[0,H]. From the Z filtered uncertainties of any pixel, select one prediction uncertainty as the uncertainty of the pseudo segmentation label of any pixel; For each pseudo segmentation label in the set of pseudo segmentation labels, the pseudo segmentation labels with uncertainty greater than the uncertainty threshold are subjected to label correction processing to obtain corrected segmentation labels; Based on each corrected segmentation label and other pseudo-segmentation labels in the pseudo-segmentation label set, a target segmentation label set for the target image is constructed; the target segmentation label set is used to train the initial semantic segmentation model.

2. The method as described in claim 1, characterized in that, The determination of the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category, based on the difference between the semantic prediction probabilities of any pixel being predicted as the h-th semantic category in the plurality of semantic prediction probability sets, includes: For the semantic prediction probability of any pixel being predicted as the h-th semantic category in the multiple semantic prediction probability sets, the variance is calculated to obtain the first variance. The first variance is used as the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category.

3. The method as described in claim 1, characterized in that, The determination of the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category, based on the difference between the semantic prediction probabilities of any pixel being predicted as the h-th semantic category in the plurality of semantic prediction probability sets, includes: Based on the pseudo segmentation label of any pixel, a reference indication value is determined to indicate whether the semantic category to which any pixel belongs is the h-th semantic category; For the multiple semantic prediction probability sets, the semantic prediction probability when any pixel is predicted to be the h-th semantic category, and the reference indication value corresponding to any pixel are subjected to variance calculation to obtain the second variance; The second variance is used as the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category.

4. The method according to any one of claims 1 to 3, characterized in that, The determination of the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category, based on the difference between the semantic prediction probabilities of any pixel being predicted as the h-th semantic category in the plurality of semantic prediction probability sets, includes: A post-processing algorithm is used to post-process the semantic prediction probability of any pixel being predicted as the h-th semantic category in the multiple semantic prediction probability sets to obtain the post-processed semantic prediction probability. Based on the difference between multiple post-processed semantic prediction probabilities when any pixel is predicted to be the h-th semantic category, the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category is determined.

5. The method according to any one of claims 1 to 3, characterized in that, The step of selecting a prediction uncertainty from the Z filtered uncertainties of any pixel as the uncertainty of the pseudo segmentation label of any pixel includes: The maximum prediction uncertainty among the Z filtered uncertainties of any given pixel is taken as the uncertainty of the pseudo segmentation label of that given pixel.

6. The method according to any one of claims 1 to 3, characterized in that, The H semantic categories include H-1 preset categories and a background category; The step of selecting a prediction uncertainty from the Z filtered uncertainties of any pixel as the uncertainty of the pseudo segmentation label of any pixel includes: From the H prediction uncertainties of any pixel, determine H-1 prediction uncertainties corresponding to the H-1 preset categories; The maximum prediction uncertainty among the H-1 prediction uncertainties that belong to the Z filtered uncertainties is taken as the uncertainty of the pseudo segmentation label of any pixel.

7. The method as described in claim 1, characterized in that, The method involves selecting Z uncertainties from H prediction uncertainties for any pixel based on the difference between the semantic prediction result of each pixel in the semantic prediction result set and the pseudo segmentation label of the corresponding pixel in the pseudo segmentation label set, including: Traverse the H prediction uncertainties of any pixel. For the h-th prediction uncertainty of any pixel, based on the semantic prediction result of any pixel, determine a prediction indication value to indicate whether the predicted semantic category of any pixel belongs to the h-th semantic category. Also, based on the pseudo segmentation label of any pixel, determine a reference indication value to indicate whether the semantic category of any pixel belongs to the h-th semantic category. Based on the difference between the predicted indicator value corresponding to each pixel in the target image and the reference indicator value corresponding to the corresponding pixel, it is determined whether to use the h-th prediction uncertainty of any pixel as a filtered uncertainty; until Z filtered uncertainties are obtained.

8. The method as described in claim 7, characterized in that, The step of determining whether to include the h-th prediction uncertainty of any pixel as a filtered uncertainty based on the difference between the predicted indicator value corresponding to each pixel in the target image and the reference indicator value corresponding to the corresponding pixel includes: Calculate the cross-union ratio between the predicted indicator value corresponding to each pixel in the target image and the reference indicator value corresponding to each pixel in the target image; If the intersection-union ratio is greater than the filtering threshold, then the h-th prediction uncertainty of any pixel is taken as a filtered uncertainty.

9. The method as described in claim 1, characterized in that, The number of the multiple semantic categories is H, where H is an integer greater than or equal to 2; The step of performing label correction processing on pseudo-segmentation labels in the pseudo-segmentation label set whose uncertainty is greater than the uncertainty threshold to obtain corrected segmentation labels includes: For any pseudo-segmentation label with an uncertainty greater than the uncertainty threshold, obtain the h-th prediction uncertainty of the target pixel corresponding to the pseudo-segmentation label and the h-th prediction probability mean of the target pixel; the h-th prediction uncertainty of the target pixel refers to the uncertainty of the semantic prediction probability of the target pixel under the h-th semantic category, and the h-th prediction probability mean of the target pixel is obtained by averaging the semantic prediction probabilities when the target pixel is predicted to be the h-th semantic category in the multiple semantic prediction probability sets; Based on the h-th prediction uncertainty of the target pixel and the mean h-th prediction probability of the target pixel, the correction reference parameter of the target pixel under the h-th semantic category is determined, h∈[1,H]; The target correction reference parameter is determined from the correction reference parameters of the target pixel under each semantic category, and the semantic category indicated by the target correction reference parameter is used as the correction segmentation label corresponding to any pseudo segmentation label.

10. The method as described in claim 9, characterized in that, The step of determining the correction reference parameter of the target pixel in the h-th semantic category based on the h-th prediction uncertainty of the target pixel and the mean h-th prediction probability of the target pixel includes: The ratio between the h-th prediction uncertainty of the target pixel and the mean of the h-th prediction probability of the target pixel is determined as the correction reference parameter of the target pixel under the h-th semantic category; Determining the target correction reference parameter from the correction reference parameters of the target pixel under each semantic category includes: The minimum correction reference parameter among the correction reference parameters of the target pixel under each semantic category is taken as the target correction reference parameter.

11. An image processing apparatus, characterized in that, include: The acquisition unit is used to acquire the target image and the pseudo-segmentation label set of the target image; The pseudo-segmentation label set includes: pseudo-segmentation labels used to indicate the semantic category to which each pixel in the target image belongs; The processing unit is configured to perform semantic segmentation processing on the target image using multiple reference semantic segmentation models to obtain a semantic prediction probability set corresponding to each reference semantic segmentation model; any semantic prediction probability set includes: each pixel in the target image is predicted as the semantic prediction probability of each semantic category among multiple semantic categories, and each reference semantic segmentation model is obtained by training an initial semantic segmentation model; the number of multiple semantic categories is H, where H is an integer greater than or equal to 2. The processing unit is further configured to, for any pixel in the target image, traverse H semantic categories, and based on the difference between the semantic prediction probabilities when any pixel is predicted to be the h-th semantic category in multiple semantic prediction probability sets, determine the uncertainty of the semantic prediction probability of any pixel under the h-th semantic category, and use it as a prediction uncertainty of any pixel, h∈[1,H]; The processing unit is further configured to acquire a semantic prediction result set; the semantic prediction result set includes: the semantic prediction result of each pixel in the target image, wherein the semantic prediction result of any pixel is used to indicate: the semantic category to which the predicted pixel belongs, wherein the semantic prediction result of any pixel is based on any semantic prediction probability set, and the pixel is predicted to be determined by the semantic prediction probability of each semantic category. The processing unit is further configured to, based on the difference between the semantic prediction results of each pixel in the semantic prediction result set and the pseudo segmentation labels of the corresponding pixels in the pseudo segmentation label set, select Z uncertainties from the H prediction uncertainties of any pixel, where Z∈[0,H]. The processing unit is further configured to select a prediction uncertainty from the Z filtered uncertainties of any pixel as the uncertainty of the pseudo segmentation label of any pixel; The processing unit is also used to perform label correction processing on the pseudo segmentation labels in the pseudo segmentation label set whose uncertainty is greater than the uncertainty threshold, so as to obtain corrected segmentation labels. The processing unit is further configured to construct a target segmentation label set for the target image based on each corrected segmentation label and other pseudo-segmentation labels in the pseudo-segmentation label set; the target segmentation label set is used to train the initial semantic segmentation model.

12. An image processing device, characterized in that, The image processing device includes an input interface and an output interface, and further includes: A processor, adapted to implement one or more instructions; and, A computer storage medium storing one or more instructions adapted to be loaded by the processor and executed as described in any one of claims 1-10.

13. A computer storage medium, characterized in that, The computer storage medium stores computer program instructions, which, when executed by a processor, are used to perform the image processing method as described in any one of claims 1-10.

14. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, performs the image processing method as described in any one of claims 1-10.

Images