Weakly supervised semantic segmentation method based on cascade decision making and interactive annotation self-improvement

Abstract

The present invention discloses a weakly supervised semantic segmentation method based on cascade decision and interactive annotation self-boosting. Specifically, the method comprises the following steps: obtaining a target domain data set and a network domain data set; using the target domain data set to train on an existing weakly supervised semantic segmentation network training framework to obtain a target domain segmentation network; the target domain segmentation network infers the network domain data set to obtain a pixel point set of each network image segmentation result, and performs data cleaning to obtain a single-class label network data set and a multi-class label network data set; using the obtained data set to train on a weakly supervised semantic segmentation network training framework to obtain a network domain segmentation network; the target domain segmentation network, the network domain segmentation network and the existing significant target detection network infer the target domain data set to obtain a target domain pixel-level mask, a network domain pixel-level mask and a significant map to cascade decision to obtain a final pixel-level mask; using the final pixel-level mask to supervise the training to obtain a final segmentation network.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a weakly supervised semantic segmentation method based on cascade decision making and interactive annotation self-enhancement. Background Art

[0002] Semantic segmentation is one of the key issues in today's computer vision field. As a high-level task, it paves the way for a complete understanding of the scene. Semantic segmentation technology is widely used in fields such as unmanned vehicle driving, medical image analysis, and geographic information systems, and plays an important role in promoting social development. In recent years, semantic segmentation methods based on supervised learning have achieved remarkable results, but it costs a lot to construct pixel-level labels required for training. In order to obtain high-performance semantic segmentation while reducing the cost of sample construction, researchers explored weakly supervised semantic segmentation methods based on network images under class label supervision.

[0003] In related methods, existing images with accurate class labels are used as target domain data, and network images retrieved from search engines using keywords are used as network domain data. Network images are massive and easy to obtain. They can be retrieved from search engines using keywords, and the search keywords are converted into their class labels. However, this easily introduces two types of noisy images: images with incorrect labels whose class labels are completely inconsistent with the image content, and images with inaccurate labels whose class labels are consistent with only part of the content. If the noise interference of network images cannot be reduced and the reliability of network domain knowledge cannot be improved, the performance of weakly supervised semantic segmentation will be affected. Existing methods use external technologies such as saliency detection, collaborative segmentation, and Grabcut to reduce the impact of network domain noise and assist in acquiring network domain knowledge. However, existing methods do not significantly utilize target domain knowledge, so that the reliability of their knowledge cannot greatly help the weakly supervised semantic segmentation training system to improve its semantic segmentation performance.

[0004] In addition, the target domain segmentation network is trained on target domain images with accurate class labels, and it has good segmentation effect on the pixel-level mask foreground features inferred from the target domain dataset. However, in the training process of the weakly supervised semantic segmentation network, the class label as supervisory information only indicates the category in the image, and does not provide any category-related spatial information. In addition, it is restricted by the inherent contextual prior of the dataset, which makes the target domain segmentation network unable to perform complete segmentation. The network domain semantic information is rich but noisy. The inference results of the trained network domain segmentation network on the target domain dataset can be used as important supplementary information for foreground feature segmentation. The background inferred from the target domain dataset for salient object detection can be used as important background feature information. How to effectively perform decision fusion based on the pixel-level results inferred from different networks and realize robust knowledge transfer of reliability association to improve the final semantic segmentation performance has become a challenge. Summary of the invention

[0005] In view of the above problems, the present invention provides a weakly supervised semantic segmentation method based on cascade decision making and interactive annotation self-boosting. The specific technical solutions adopted by the present invention are as follows:

[0006] A weakly supervised semantic segmentation method based on cascade decision making and interactive annotation self-boosting, comprising the following steps:

[0007] S1, obtain the target domain dataset and the network domain dataset;

[0008] S2. Use the target domain dataset to train the target domain segmentation network on the weakly supervised semantic segmentation network training framework;

[0009] S3. Use the target domain segmentation network to infer the network domain dataset, obtain the pixel point set of each network image segmentation result, and perform data cleaning to obtain a single-class label network dataset and a multi-class label network dataset;

[0010] S4, using the single-class label network dataset and the multi-class label network dataset to train the network domain segmentation network on the weakly supervised semantic segmentation network training framework;

[0011] S5. Use the target domain segmentation network, the network domain segmentation network and the salient object detection network to infer the target domain dataset respectively, and obtain the target domain pixel-level mask, the network domain pixel-level mask and the saliency map respectively, and obtain the final pixel-level mask through cascade decision of the target domain pixel-level mask, the network domain pixel-level mask and the saliency map;

[0012] S6. Use the final pixel-level mask supervision to train the semantic segmentation network to obtain the final segmentation network, and use the final segmentation network to segment the target image.

[0013] Preferably, in S1, the target domain dataset I t Contains several training images and their corresponding manually annotated category labels, network domain dataset I w Contains several images searched on search engines and their corresponding category labels converted from search keywords.

[0014] Preferably, in S2, the network structure in the weakly supervised semantic segmentation network training framework includes two parts: SSENet and AffinityNet, wherein SSENet first takes the training image as input and outputs a category activation map, and then AffinityNet takes the category activation map as input and outputs a pixel-level mask; using the target domain dataset I t After training on the weakly supervised semantic segmentation network training framework, the target domain segmentation network F is obtained. t .

[0015] Preferably, S3 comprises the following sub-steps:

[0016] S31. Use the target domain segmentation network F t For the network domain dataset I w Infer and obtain the pixel point set Φ of each image segmentation result;

[0017] S32, each network domain image X i definition and where Φ i is the pixel set of the segmentation result of the i-th image, |Φ i | indicates Φ i The total number of pixels in, p k is Φ i The semantic label of pixel k in ; χ(A) represents the judgment result of conditional expression A. When conditional expression A holds, χ(A) = 1, otherwise χ(A) = 0; bg represents background pixels; λ i Represents a single category l i Accounting for image X i Proportion, μ i Indicates that other foreground categories account for image X i Proportion;

[0018] S33, network domain data set I w Perform preliminary cleaning and traverse each network image X i , if the network image X i Satisfy δ1≤λ i ≤δ2, then retain, where δ1 and δ2 are threshold control single category l i Accounting for image X i ratio, otherwise it will be directly judged as a noise image from the network domain dataset I w Eliminate;

[0019] S34, performing single-category cleaning on the network images retained in step S33, traversing each network image X retained therein i , if the network image X i Satisfy μ i =0, then add it to the single-class label network dataset, otherwise it will not be added to the single-class label network dataset;

[0020] S35, performing multi-category cleaning on the network images retained in step S33, traversing each network image X retained therein i , if the network image X i Satisfy μ i ≥δ3, then Φ i All the category labels that appear in the network image X iclass label, and the network image X with class label i Add a multi-class label network dataset, where δ3 is used as a threshold to control the proportion of other foreground categories in image X i Otherwise, the multi-class labeled network dataset will not be added.

[0021] Preferably, in S4, the network structure in the weakly supervised semantic segmentation network training framework includes two parts, SSENet and AffinityNet, wherein SSENet first takes the training image as input and outputs a category activation map, and then AffinityNet takes the category activation map as input and outputs a pixel-level mask; a single-class label network dataset and a multi-class label network dataset are used together as training datasets, and after training on the weakly supervised semantic segmentation network training framework, a network domain segmentation network F is obtained. w .

[0022] Preferably, S5 comprises the following sub-steps:

[0023] S51. Use the target domain segmentation network F t , Network Domain Segmentation Network F w and salient object detection network respectively on the target domain dataset I t Infer and obtain the pixel-level mask M t , pixel-level mask M w and saliency map M b ;

[0024] S52, initialize the pixel-level mask M without semantic label f All pixel values ​​of are 255, which is used to store the cascade decision results, and the pixel-level mask M f The resolution of is the same as the input image resolution;

[0025] S53, traverse pixel-level mask M t For all pixels p1 in , if the semantic label of pixel p1 is foreground category l1, then the foreground category l1 is assigned to the pixel-level mask M f The pixel corresponding to pixel p1 in is taken as its semantic label;

[0026] S54, traverse pixel-level mask M w For all pixels p2 in , if the semantic label of pixel p2 is foreground category l2 and the pixel-level mask M f The pixel corresponding to pixel p2 has not been assigned a semantic label, so the foreground category l2 is assigned a pixel-level mask M f The pixel corresponding to pixel p2 in is taken as its semantic label;

[0027] S55. Traverse the saliency map M bFor all pixels p3 in , if pixel p3 is background and pixel-level mask M f The pixel corresponding to pixel p3 has been assigned a semantic label, so the pixel-level mask M f The pixel corresponding to pixel p3 is reassigned to 255 and is regarded as the first type of pixel; if pixel p3 is the foreground and the pixel-level mask M f The pixel value corresponding to pixel p3 is 255, so the pixel-level mask M is maintained. f The pixel value corresponding to the pixel p3 is 255 and is regarded as the second type of pixel; for the pixel-level mask M f Except for the first and second type pixels, the remaining pixel values ​​of 255 are uniformly regarded as background and the pixel value is assigned to 0, thus obtaining the final pixel-level mask M f .

[0028] Preferably, S6 comprises the following sub-steps:

[0029] S61. Based on target domain dataset I t And the final pixel-level mask of each image, the semantic segmentation network is trained using the cross entropy loss function and the multi-label classification loss function, and the pixels with a pixel value of 255 in the final pixel-level mask are not involved in the training. After the training is completed, the final segmentation network F is obtained. f ;

[0030] S62: Input the target image into the final segmentation network to segment it and obtain a semantic segmentation result.

[0031] Preferably, the semantic segmentation network trained using the final pixel-level mask supervision in S6 is a semantic segmentation network based on ResNet38.

[0032] The present invention implements a weakly supervised semantic segmentation method based on dual-domain cascade decision and interactive annotation self-boosting. The invention implements robust knowledge transfer using network images and saliency detection results under class label supervision, and improves the final segmentation performance, achieving high-performance semantic segmentation while reducing sample construction costs. Among them, dual-domain interactive annotation self-boosting reduces the noise of network domain data sets, improves the quality of image annotation, enriches the semantic information of the network domain, and improves the reliability of network domain knowledge from a data perspective. The knowledge transfer of cascade decision realizes pixel-level knowledge transfer from the perspective of knowledge transfer. Its decision-making process is related to knowledge reliability, and effectively utilizes the advantages of knowledge in different domains to achieve robust knowledge transfer. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Schematic diagram of the basic steps of the weakly supervised semantic segmentation method based on cascade decision making and interactive annotation self-enhancement in the present invention.

[0034] Figure 2 It is a flowchart of the weakly supervised semantic segmentation method based on cascade decision and interactive annotation self-enhancement in the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0036] On the contrary, the present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention as defined by the claims. Further, in order to make the public have a better understanding of the present invention, some specific details are described in detail in the detailed description of the present invention below. Those skilled in the art can fully understand the present invention without the description of these details.

[0037] refer to Figure 1 In a preferred embodiment of the present invention, a weakly supervised semantic segmentation method based on cascade decision making and interactive annotation self-boosting is provided, which is used to train a high-performance weakly supervised semantic segmentation network. The process is as follows: Figure 2 As shown, the specific steps include:

[0038] S1. Obtain target domain dataset and network domain dataset.

[0039] In this embodiment, in the above step S1, the target domain dataset I t Contains several training images and their corresponding manually annotated category labels, network domain dataset I w It contains several images searched on the search engine and the corresponding category labels formed by the search keywords. The specific search engine is not limited, as long as it can realize the search of the corresponding image by keyword. Each keyword can be used as the category label of each image searched under the keyword. The construction of the network domain dataset can be achieved through crawler technology.

[0040] S2. Use the target domain dataset to train the target domain segmentation network on the weakly supervised semantic segmentation network training framework.

[0041] It should be noted that the weakly supervised semantic segmentation network training framework in the present invention can be any existing public method as long as it can be used to construct a weakly supervised semantic segmentation network.

[0042] In this embodiment, in the above step S2, the network structure in the weakly supervised semantic segmentation network training framework is SSENet+AffinityNet, which includes two parts, SSENet and AffinityNet, wherein SSENet first takes the training image as input and outputs a category activation map, and then AffinityNet takes the category activation map as input and outputs a pixel-level mask; using the target domain dataset I t After training on the weakly supervised semantic segmentation network training framework, the target domain segmentation network F is obtained. t .

[0043] S3. Use the target domain segmentation network to infer the network domain dataset, obtain the pixel point set of each network image segmentation result, and perform data cleaning to obtain single-class label network datasets and multi-class label network datasets.

[0044] In this embodiment, the above step S3 includes the following sub-steps:

[0045] S31. Use the target domain segmentation network F t For the network domain dataset I w Infer and obtain the pixel point set Φ of each image segmentation result, where the pixel point set of the image segmentation result can be recorded as Φ i ;

[0046] S32, each network domain image X i definition and where Φ i is the pixel set of the segmentation result of the i-th image, |Φ i | indicates Φ i The total number of pixels in, p k is Φ i The semantic label of pixel k in ; χ(A) represents the judgment result of conditional expression A (A is a general formula used for reference, which can be replaced by the conditional expression in each formula). When conditional expression A is established, χ(A) = 1, otherwise χ(A) = 0; bg represents background pixels; λ i Represents a single category l i Accounting for image X i Proportion, μ i Indicates that other foreground categories account for image X i Proportion;

[0047] S33, network domain data set I w Perform preliminary cleaning and traverse each network image X i , if the network image X i Satisfy δ1≤λ i ≤x2, then retain, where δ1 and δ2 are threshold control single category li Accounting for image X i ratio, otherwise it will be directly judged as a noise image from the network domain dataset I w Eliminate;

[0048] S34, for the network domain data set I retained in step S33 w Perform single-class cleaning on the network images in the image, and traverse each network image X retained in it i , if the network image X i Satisfy μ i =0, then add it to the single-class label network dataset, otherwise it will not be added to the single-class label network dataset;

[0049] S35, for the network domain data set I retained in step S33 w Perform multi-class cleaning on the network images in the image, and traverse each network image X retained in it i , if the network image X i Satisfy μ i ≥δ3, then Φ i All the category labels that appear in the network image X i class label, and the network image X with class label i Add a multi-class label network dataset, where δ3 is used as a threshold to control the proportion of other foreground categories in image X i Otherwise, the multi-class labeled network dataset will not be added.

[0050] S4. Use single-class labeled network datasets and multi-class labeled network datasets to train a network domain segmentation network on a weakly supervised semantic segmentation network training framework.

[0051] It should be noted that the weakly supervised semantic segmentation network training framework in the present invention can be any existing public method as long as it can be used to construct a weakly supervised semantic segmentation network.

[0052] In this embodiment, in the above step S4, the network structure in the weakly supervised semantic segmentation network training framework is SSENet+AffinityNet, which includes two parts, SSENet and AffinityNet, wherein SSENet first takes the training image as input and outputs a category activation map, and then AffinityNet takes the category activation map as input and outputs a pixel-level mask; a single-class label network dataset and a multi-class label network dataset are used together as training datasets, and after training on the weakly supervised semantic segmentation network training framework, a network domain segmentation network F is obtained. w .

[0053] It should be noted that, although the weakly supervised semantic segmentation network training framework used in the above S2 and S4 steps is the same in the present invention, in other embodiments, different frameworks can be used to implement them respectively, as long as the semantic segmentation function can be achieved. The loss function used in the weakly supervised semantic segmentation network training framework can be adjusted according to actual needs to meet the model training accuracy.

[0054] S5. Use the target domain segmentation network, the network domain segmentation network and the salient target detection network to infer the target domain dataset respectively, and obtain the target domain pixel-level mask, the network domain pixel-level mask and the saliency map respectively, and then obtain the final pixel-level mask through cascade decision of the target domain pixel-level mask, the network domain pixel-level mask and the saliency map.

[0055] In this embodiment, the above step S5 includes the following sub-steps:

[0056] S51. Use the target domain segmentation network F t , Network Domain Segmentation Network F w and salient object detection network respectively on the target domain dataset I t Infer and obtain the pixel-level mask M t , pixel-level mask M w and saliency map M b .

[0057] S52, initialize a pixel-level mask M without semantic label f , used to store the cascade decision results, the pixel-level mask M f All pixel values ​​in are initialized to 255, and the pixel-level mask M f The resolution of is the same as the input image resolution.

[0058] It should be noted that the pixel-level mask M f , pixel-level mask M t , pixel-level mask M w and saliency map M b The image resolution, i.e., the image size, is consistent with the input image. Each pixel of the input image is t 、M w 、M f 、M b There is a corresponding pixel point at the same position in each mask and saliency map. The pixel values ​​in each mask and saliency map reflect the semantic label of the corresponding pixel point, that is, the classification category. The subsequent steps can be based on M t 、M w 、M b Through cascade decision making, the final semantic label is recorded in M f In, that is M f Assign a value to each pixel in .

[0059] S53, traverse pixel-level mask M t For all pixels p1 in , if the semantic label of pixel p1 is foreground category l1, then the foreground category l1 is assigned to the pixel-level mask M f The pixel corresponding to pixel p1 in is taken as its semantic label.

[0060] S54, traverse pixel-level mask M w For all pixels p2 in , if the semantic label of pixel p2 is foreground category l2 and the pixel-level mask M f The pixel corresponding to pixel p2 has not been assigned a semantic label, so the foreground category l2 is assigned a pixel-level mask M f The pixel corresponding to pixel p2 in is taken as its semantic label.

[0061] S55. Traverse the saliency map M b For all pixels p3 in , if pixel p3 is background and pixel-level mask M f The pixel corresponding to pixel p3 has been assigned a semantic label, so the pixel-level mask M f The pixel corresponding to pixel p3 is reassigned to 255 and is regarded as the first type of pixel; if pixel p3 is the foreground and the pixel-level mask M f The pixel value corresponding to pixel p3 is 255, so the pixel-level mask M is maintained. f The pixel value corresponding to the pixel p3 is 255 and is regarded as the second type of pixel; for the pixel-level mask M f Except for the first and second type pixels, the remaining pixel values ​​of 255 are uniformly regarded as background and the pixel value is assigned to 0, thus obtaining the final pixel-level mask M f .

[0062] Assuming that the number of foreground categories is N, and their category labels are represented by 1 to N, the final pixel-level mask M is obtained. f There are three types of labels, the first type is the background with a pixel value of 0, the second type is the foreground with a pixel value of any value between 1 and N, and the third type is the pixel value of 255. Among them, the third type of pixels are not involved in the calculation of the loss function when used for subsequent S6 training, because these pixels are likely to be contradictory in different inference results, and not participating in the supervised training of the semantic segmentation network is more conducive to improving the accuracy of the network.

[0063] S6. Use the final pixel-level mask supervision to train the semantic segmentation network to obtain the final segmentation network, and use the final segmentation network to segment the target image.

[0064] In this embodiment, the above step S6 includes the following sub-steps:

[0065] S61. Based on target domain dataset I t And the final pixel-level mask of each image, the semantic segmentation network is trained using the cross entropy loss function and the multi-label classification loss function, and the pixels with a pixel value of 255 in the final pixel-level mask are not involved in the training. After the training is completed, the final segmentation network F is obtained. f ;

[0066] S62: Input the target image into the final segmentation network to segment it and obtain a semantic segmentation result.

[0067] It should be noted that the semantic segmentation network trained using the final pixel-level mask supervision in the present invention can be any existing public method as long as it can be used to construct a semantic segmentation network.

[0068] In this embodiment, the semantic segmentation network trained using the final pixel-level mask supervision may be a semantic segmentation network based on ResNet38.

[0069] The weakly supervised semantic segmentation method based on cascade decision and interactive annotation self-improvement shown in the above steps S1 to S6 is applied to a specific example to demonstrate its specific technical effect.

[0070] Example

[0071] The implementation method of this embodiment is as described in the above steps S1 to S6, and the specific steps will not be elaborated in detail. The following only demonstrates its effect based on case data.

[0072] The present invention is implemented on target domain datasets and network domain datasets:

[0073] Target domain dataset: This example uses the PASCAL VOC 2012 training set as the target domain dataset, which contains 10,582 images in 21 categories. The final segmentation network is tested on a validation set containing 1,449 images and a test set containing 1,456 images.

[0074] Web domain dataset: This example uses public network images as the web domain dataset. The images are retrieved through keywords on the search engine, and the search keywords are converted into their class labels. The categories are consistent with the target domain. The image width and height do not exceed 500, and the class labels only contain a single category. In this example, δ1=0.2 and δ2=0.8 are set to perform preliminary cleaning on the web domain dataset. The number of images of each specific category in the single-class cleaning is set to no more than 500, thus obtaining a single-class labeled network dataset containing 9742 images. δ3=0.05 is set to obtain a multi-class labeled network dataset containing 5415 images.

[0075] This embodiment uses an existing weakly supervised semantic segmentation network training framework (SSENet+AffinityNet) as a basis. The segmentation network trained on the PASCAL VOC 2012 training set has a performance of 63.5% mIoU and 64.2% mIoU on the PASCAL VOC2012 validation set and test set, respectively. mIoU is the average intersection-over-union ratio, which calculates the ratio of correctly predicted positive samples (the intersection of the two sets of true values ​​and predicted values) to the sum (union) of correctly predicted positive samples, incorrectly predicted positive samples, and incorrectly predicted negative samples. The semantic segmentation network based on ResNet38 is trained using a weakly supervised semantic segmentation method based on cascade decision making and interactive annotation self-boosting. The final segmentation network obtained has a performance of 68.2% mIoU and 69.6% mIoU on the PASCAL VOC 2012 validation set and test set, respectively, with a significant improvement in performance.

[0076] It can be seen that through the above technical solutions, the embodiment of the present invention has developed a weakly supervised semantic segmentation method based on cascade decision-making and interactive annotation self-boosting based on deep learning technology. The invention realizes the robust knowledge transfer using network images and saliency detection results under class label supervision, and improves the final segmentation performance, achieving high-performance semantic segmentation while reducing sample construction costs. Among them, the dual-domain interactive annotation self-boosting reduces the noise of the network domain data set, improves the quality of image annotation, enriches the semantic information of the network domain, and improves the reliability of network domain knowledge from a data perspective. The knowledge transfer of cascade decision-making realizes pixel-level knowledge transfer from the perspective of knowledge transfer. Its decision-making process is related to knowledge reliability, and effectively utilizes the advantages of knowledge in different domains to achieve robust knowledge transfer.

[0077] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A weakly supervised semantic segmentation method based on cascade decision making and interactive annotation self-boosting, characterized in that: The following steps are involved: S1, obtain the target domain dataset and the network domain dataset; S2. Use the target domain dataset to train the target domain segmentation network on the weakly supervised semantic segmentation network training framework; S3. Use the target domain segmentation network to infer the network domain dataset, obtain the pixel point set of each network image segmentation result, and perform data cleaning to obtain a single-class label network dataset and a multi-class label network dataset; S4, using the single-class label network dataset and the multi-class label network dataset to train the network domain segmentation network on the weakly supervised semantic segmentation network training framework; S5. Use the target domain segmentation network, the network domain segmentation network and the salient object detection network to infer the target domain dataset respectively, and obtain the target domain pixel-level mask, the network domain pixel-level mask and the saliency map respectively, and obtain the final pixel-level mask through cascade decision of the target domain pixel-level mask, the network domain pixel-level mask and the saliency map; S6. Use the final pixel-level mask supervision to train the semantic segmentation network to obtain the final segmentation network, and use the final segmentation network to segment the target image.

2. According to claim 1, a weakly supervised semantic segmentation method based on cascade decision making and interactive annotation self-boosting is characterized in that: In S1, the target domain dataset I t Contains several training images and their corresponding manually annotated category labels, network domain dataset I w Contains several images searched on search engines and their corresponding category labels converted from search keywords.

3. According to claim 1, a weakly supervised semantic segmentation method based on cascade decision making and interactive annotation self-boosting is characterized in that: In S2, the network structure in the weakly supervised semantic segmentation network training framework includes two parts: SSENet and AffinityNet. SSENet first takes the training image as input and outputs the category activation map, and then AffinityNet takes the category activation map as input and outputs the pixel-level mask. Using the target domain dataset I t After training on the weakly supervised semantic segmentation network training framework, the target domain segmentation network F is obtained. t .

4. According to claim 1, a weakly supervised semantic segmentation method based on cascade decision making and interactive annotation self-boosting is characterized in that: The S3 comprises the following sub-steps: S31. Use the target domain segmentation network F t For the network domain dataset I w Infer and obtain the pixel point set Φ of each image segmentation result; S32, each network domain image X i definition and where Φ i is the pixel set of the segmentation result of the i-th image, |Φ i | indicates Φ i The total number of pixels in, p k is Φ i The semantic label of pixel k in ; χ(A) represents the judgment result of conditional expression A. When conditional expression A is established, χ(A)=1, otherwise χ(A)=0; bg represents background pixel; λ i Represents a single category l i Accounting for image X i Proportion, μ i Indicates that other foreground categories account for image X i Proportion; S33, network domain data set I w Perform preliminary cleaning and traverse each network image X i , if the network image X i Satisfy δ1≤λ i ≤δ2, then retain, where δ1 and δ2 are threshold control single category l i Accounting for image X i ratio, otherwise it will be directly judged as a noise image from the network domain dataset I w Eliminate; S34, performing single-category cleaning on the network images retained in step S33, traversing each network image X retained therein i , if the network image X i Satisfy μ i =0, then add it to the single-class label network dataset, otherwise it will not be added to the single-class label network dataset; S35, perform multi-category cleaning on the network images retained in step S33, and traverse each network image X retained therein. i , if the network image X i Satisfy μ i ≥δ3, then Φ i All the category labels that appear in the network image X i class label, and the network image X with class label i Add a multi-class label network dataset, where δ3 is used as a threshold to control the proportion of other foreground categories in image X i Otherwise, the multi-class labeled network dataset will not be added.

5. According to claim 1, a weakly supervised semantic segmentation method based on cascade decision making and interactive annotation self-boosting is characterized in that: In S4, the network structure in the weakly supervised semantic segmentation network training framework includes two parts, SSENet and AffinityNet, wherein SSENet first takes the training image as input and outputs a category activation map, and then AffinityNet takes the category activation map as input and outputs a pixel-level mask; a single-class label network dataset and a multi-class label network dataset are used together as training datasets, and after training on the weakly supervised semantic segmentation network training framework, a network domain segmentation network F is obtained. w .

6. According to claim 1, a weakly supervised semantic segmentation method based on cascade decision making and interactive annotation self-boosting is characterized in that: The S5 comprises the following sub-steps: S51. Use the target domain segmentation network F t , Network Domain Segmentation Network F w and salient object detection network respectively on the target domain dataset I t Infer and obtain the pixel-level mask M t , pixel-level mask M w and saliency map M b ; S52, initialize the pixel-level mask M without semantic label f All pixel values ​​of are 255, which is used to store the cascade decision results, and the pixel-level mask M f The resolution of is the same as the input image resolution; S53, traverse pixel-level mask M t For all pixels p1 in , if the semantic label of pixel p1 is foreground category l1, then the foreground category l1 is assigned to the pixel-level mask M f The pixel corresponding to pixel p1 in is taken as its semantic label; S54, traverse pixel-level mask M w For all pixels p2 in , if the semantic label of pixel p2 is foreground category l2 and the pixel-level mask M f The pixel corresponding to pixel p2 has not been assigned a semantic label, so the foreground category l2 is assigned a pixel-level mask M. f The pixel corresponding to pixel p2 in is taken as its semantic label; S55. Traverse the saliency map M b For all pixels p3 in , if pixel p3 is background and pixel-level mask M f The pixel corresponding to pixel p3 has been assigned a semantic label, so the pixel-level mask M f The pixel corresponding to pixel p3 is reassigned to 255 and is regarded as the first type of pixel; if pixel p3 is the foreground and the pixel-level mask M f The pixel value corresponding to pixel p3 is 255, so the pixel-level mask M is maintained. f The pixel value corresponding to the pixel p3 is 255 and is regarded as the second type of pixel; for the pixel-level mask M f Except for the first and second type pixels, the remaining pixel values ​​of 255 are uniformly regarded as background and the pixel value is assigned to 0, thus obtaining the final pixel-level mask M f .

7. According to claim 1, a weakly supervised semantic segmentation method based on cascade decision making and interactive annotation self-boosting is characterized in that: The S6 comprises the following sub-steps: S61. Based on target domain dataset I t And the final pixel-level mask of each image, the semantic segmentation network is trained using the cross entropy loss function and the multi-label classification loss function, and the pixels with a pixel value of 255 in the final pixel-level mask are not involved in the training. After the training is completed, the final segmentation network F is obtained. f ; S62: Input the target image into the final segmentation network to segment it and obtain a semantic segmentation result.

8. The weakly supervised semantic segmentation method based on cascade decision making and interactive annotation self-boosting as claimed in claim 1, characterized in that: The semantic segmentation network trained using the final pixel-level mask supervision in S6 is a semantic segmentation network based on ResNet38.

Images