Weakly supervised semantic segmentation method based on erasing and guiding cross-class pasting

Abstract

This invention discloses a weakly supervised semantic segmentation method based on erase-guided cross-class pasting, comprising: extracting class activation maps from training images, distinguishing foreground and background regions, generating initial pseudo-labels and binarized erase masks; calculating and filtering training images based on scale-aware uncertainty scores; performing connected component analysis based on the binarized erase masks, extracting connected components containing the highest activation values ​​as the target regions to be pasted; extracting foreground blocks from single-class images in the training images, constructing a foreground memory bank, and storing foreground blocks of each class and the global average class confidence in the foreground memory bank respectively; randomly selecting foreground blocks of a specific class, and pasting cross-class foreground features to the target regions to be pasted; completing the construction of a cross-class pasting model; and constructing an image-level global alignment loss to complete the training of the cross-class pasting model. This invention significantly improves the integrity of class activation maps and the quality of pseudo-labels, providing more accurate target perception for weakly supervised semantic segmentation.

Description

Technical Field

[0001] This invention relates to the field of weakly supervised semantic segmentation technology, specifically to a weakly supervised semantic segmentation method based on erasure-guided cross-class pasting. Background Technology

[0002] With the rapid development of deep learning, semantic segmentation technology has been widely used in fields such as autonomous driving and image editing. However, obtaining pixel-level segmentation annotations required for training deep learning models is highly labor-intensive and time-consuming. Therefore, weakly supervised semantic segmentation (WSSS) based on image-level labels has become an important research direction for alleviating annotation pressure.

[0003] Existing weakly supervised semantic segmentation processes typically follow three steps: converting image-level labels into pixel-level coarse labels, refining pseudo-labels, and training the final segmentation model using the refined labels. Regarding segmentation label generation, Class Activation Map (CAM) techniques have become the de facto paradigm for object localization using image-level labels. However, traditional CAMs can only highlight the most discriminative regions of a target, resulting in incomplete target discovery due to their small and sparse activations. Since the first step of seed generation is fundamental to subsequent processes, numerous works have emerged to expand CAM activations to obtain high-quality pseudo-labels. Among these, adversarial erasure is a mainstream method, aiming to discover new and complementary target regions by adversarially masking the currently detected regions.

[0004] However, this traditional erasure mechanism suffers from a serious and counterintuitive technical flaw: feature activation leakage. Specifically, after erasing highly activated target regions, the network's attention leaks to the erased blank areas, while generating almost no effective activation in other unexplored areas of the target object. The underlying reason is that the model doesn't explore the true target pattern but instead performs a "shortcut learning" by over-relying on the highly relevant features of the erased regions in memory. Although the erasure operation blocks salient regions at the input pixel level, when the model learns a large amount of semantics, it can successfully infer the overall reconstructed region, leading to unexpected activation leakage to the erased areas. This severe activation leakage significantly hinders the effective expansion of the class activation map and the discovery of complete targets. Summary of the Invention

[0005] The purpose of this invention is to provide a weakly supervised semantic segmentation method based on erase-guided cross-class pasting. This method aims to break the model's shortcut learning of features in the erased region, suppress feature activation leakage, and thus force the model to redistribute its attention to complementary regions of the target that have not been explored, thereby uncovering a more complete target response.

[0006] To achieve the above functions, this invention designs a weakly supervised semantic segmentation method based on erase-guided cross-class pasting, executing the following steps S1-S5 to construct a cross-class pasting model, and executing the following step S6 to apply the cross-class pasting model to complete the cross-class pasting of the image:

[0007] Step S1: Input the training image into the classification network to extract the class activation map. For the activation value of each pixel in the class activation map, a preset threshold is used to distinguish the foreground region and the background region, and an initial pseudo label and a binarized erasure mask are generated.

[0008] Step S2: Calculate the scale-aware uncertainty score based on the initial pseudo-label, set a dynamic threshold for the scale-aware uncertainty score, and select the training image set with the scale-aware uncertainty score higher than the dynamic threshold in the current training batch to construct a candidate sample set;

[0009] Step S3: For the training images in the candidate sample set, perform connected component analysis based on the binarized erasure mask to obtain several independent connected components, and extract the connected component containing the highest activation value as the target region to be pasted.

[0010] Step S4: Extract foreground blocks from single-class images in the training images, construct a foreground memory bank, extract the largest connected component of the foreground block from the single-class image as the discrimination region, and store the foreground block and the global average class confidence into the queues divided by class in the foreground memory bank respectively;

[0011] Step S5: Randomly select a foreground block of a specific category from the foreground memory bank, extract cross-category foreground features, and paste them into the target area to be pasted;

[0012] Step S6: Construct an image-level global alignment loss based on anchor branch and mask branch, train the cross-class pasting model based on the image-level global alignment loss to obtain the trained cross-class pasting model, and apply the trained cross-class pasting model to complete the cross-class pasting of the image.

[0013] As a preferred technical solution of the present invention, the specific steps of step S1 are as follows:

[0014] Step S1.1: For the ResNet-38 network model, remove the fully connected layers and set the number of output channels to [value missing]. ,in To determine the number of foreground categories, a backbone network is formed; training images are input into the backbone network, and class activation map features containing single-class features of each category are extracted. Class activation graph features Each channel in the text corresponds to a single-class feature;

[0015] By extracting class activation map features Medium category Single-class features Calculate and obtain the category Class activation graph The class activation maps of all categories are then concatenated along the channel dimension to obtain a channel count of [number missing]. Class activation graph :

[0016] ;

[0017] ;

[0018] in, express Activation functions are used to filter negative responses; This indicates global maximum value normalization processing, which will be applied to the category. Class activation graph Mapped to interval; Indicates will Concatenate activation maps of different classes; Indicates the first to the second Activation graph of each class;

[0019] Step S1.2: Targeting the class activation graph The activation values ​​of each pixel are preset with a high threshold. and low threshold Locate the foreground and background regions and generate initial pseudo-tags. The specific formula is as follows:

[0020] ;

[0021] in, Indicates coordinate position The initial pseudo-tags, Representation of class activation graph At coordinate position Activation vector extracted along the channel dimension; Representation of class activation graph features At coordinate position Feature vectors extracted along the channel dimension; Used to extract the highest activation value for each category; Used to extract the category with the highest activation value; 255 indicates the ignore label for uncertain regions;

[0022] Step S1.3: Activate the class graph The activation value is greater than or equal to the preset mask threshold. The pixel region is defined as the most discriminative region and subjected to binarization masking to obtain the binarized erase mask. :

[0023] ;

[0024] in, To preset the mask threshold, For class activation graph At coordinate position Binarized erase mask.

[0025] As a preferred technical solution of the present invention, the specific steps of step S2 are as follows:

[0026] Step S2.1: Construct the scale-aware uncertainty score as follows:

[0027] ;

[0028] in, The score represents the scale-perceived uncertainty. It is an indicator function; Indicates coordinate position; the numerator of the formula The denominator represents the total area of ​​uncertain pixels corresponding to the ignored labels in the uncertain regions of the initial pseudo-labels; This represents the total area of ​​pixels in the foreground region with activation values ​​between 0 and 255 in the initial pseudo-label;

[0029] Step S2.2: In each training image constituting a size of In the training batch, calculate the scale-aware uncertainty score set for all training images. And set a dynamic threshold for the scale-aware uncertainty score. :

[0030] ;

[0031] in, Indicates dynamic threshold. This represents the percentile in the training batch set. The preset scaling parameters;

[0032] Step S2.3: with The training images are high-uncertainty samples. High-uncertainty samples are selected from the training batch to construct a candidate sample set. As shown in the following formula:

[0033] ;

[0034] in, This represents the scale-aware uncertainty score of the i-th training image in the training batch; This is the training batch size.

[0035] As a preferred technical solution of the present invention, the specific steps of step S3 are as follows:

[0036] Step S3.1: For each training image, obtain the binarized erasure mask generated in step S1. By analyzing connected components, the binarized erase mask in the training image is segmented into several independent connected components;

[0037] Step S3.2: Select the connected component containing the highest activation value from the connected components as the high-response target region. This area will be used as the target area for pasting, and the specific formula is as follows:

[0038] ;

[0039] in, Representation of class activation graph At coordinate position The highest activation value in the activation vector; This represents connected component analysis; Representation of class activation graph The pixel coordinates with the highest global activation value; It represents a region in a set of connected components.

[0040] As a preferred embodiment of the present invention, the specific steps of step S4 are as follows:

[0041] Step S4.1: Based on the image-level weak labels attached to the training images themselves, determine the number of foreground categories contained in each training image, define the training images containing only one foreground category as single-class images, and extract foreground patches from single-class images;

[0042] Step S4.2: Obtain the unique category of a single-class image foreground mask of the foreground block Binarization is performed, and the connected component with the largest area is selected through connected component analysis. As a category The specific extraction formula for the discrimination region in the current training image is as follows:

[0043] ;

[0044] in, This indicates the calculation of the area of ​​a connected region; Represents a region within a set of connected components; This represents connected component analysis;

[0045] Step S4.3: Construct a foreground memory to store the foreground blocks for each category; the foreground memory adopts a first-in-first-out queue mechanism based on category and is updated in each training iteration; simultaneously, calculate the category... Global average class confidence corresponding to the discrimination region in the current training image It is also stored in the foreground memory, and the calculation formula is:

[0046] ;

[0047] in, The class activation map features of the current training image; This indicates that clipping corresponds to... Operations within a region; This is a global average pooling operation. express Activation function.

[0048] As a preferred embodiment of the present invention, the specific steps of step S5 are as follows:

[0049] Step S5.1: For the high-uncertainty samples selected in step S2, using the current training image as the anchor image, randomly select a category that does not exist in the anchor image. ;

[0050] Step S5.2: Randomly select a foreground block belonging to the category from the foreground blocks stored in the foreground memory constructed in step S4. Connected components and connected components Paste into the high-response target area identified in step S3 Above, obtain cross-class pasted images.

[0051] As a preferred technical solution of the present invention: the image-level global alignment loss constructed in step S6 based on anchor branch and mask branch. As shown in the following formula:

[0052] ;

[0053] in, This is the class activation map feature of the current training image, belonging to the anchor branch; Input the feature extraction results after cross-class pasting of the image into the mask branch; For categories retrieved from the foreground memory The global average class confidence corresponding to the discrimination region in the current training image.

[0054] The present invention also designs a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the weakly supervised semantic segmentation method based on erasure-guided cross-class pasting.

[0055] The present invention also designs a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the weakly supervised semantic segmentation method based on erasure-guided cross-class pasting.

[0056] Beneficial effects: Compared with the prior art, the advantages of the present invention include:

[0057] In suppressing feature leakage, this invention breaks through shortcut learning and significantly improves the completeness of feature mining and perceptual accuracy: By introducing deterministic cross-class foreground blocks, this invention blocks the model's attention leakage to the contextual features of erased blank areas. This mechanism physically disrupts the possibility of the model relying on local memory features for inference, forcing the model to shift its attention to mining more complete complementary target regions, thereby significantly improving the quality of class activation maps and the accuracy of the final weakly supervised semantic segmentation.

[0058] Regarding robustness in cross-class pasting, the category information is purely controllable, optimizing the stability of the global alignment framework: the foreground memory of this invention only intercepts and extracts foreground blocks from simple single-class images, ensuring that the category of the introduced physical interference information is pure and controllable. Simultaneously, by dynamically balancing the category confidence of the foreground blocks in the image-level global alignment loss, the rigor and logical consistency of the loss function update are guaranteed, enabling the segmentation model to enhance its complementary region mining capabilities while maintaining classification and localization stability.

[0059] In terms of computational efficiency, a dual filtering mechanism avoids redundant computation and invalid perturbations: This invention innovatively designs a dual filtering mechanism of image-level scale-aware uncertainty scoring and region-level high-response connected component analysis. This not only avoids over-expansion of already fully explored high-confidence images, but also prevents extensive and invalid pasting operations caused by the sparse distribution of activation maps. This mechanism greatly improves the execution efficiency and specificity of the algorithm, achieving an effective balance between suppressing activation leakage and controlling computational costs. Attached Figure Description

[0060] Figure 1 This is a flowchart of a weakly supervised semantic segmentation method based on erasure-guided cross-class pasting according to an embodiment of the present invention;

[0061] Figure 2 This is a comparison chart of the visual detection results of the method of the present invention and the existing benchmark method in a real scene, according to an embodiment of the present invention. Detailed Implementation

[0062] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0063] This invention provides a weakly supervised semantic segmentation method based on erasure-guided cross-class pasting, referring to... Figure 1 Perform the following steps S1-S5 to construct a cross-category pasting model, and then perform the following step S6 to apply the cross-category pasting model to complete the cross-category pasting of the image:

[0064] Step S1: Input the training image into the classification network to extract the class activation map. For the activation value of each pixel in the class activation map, a preset threshold is used to distinguish the foreground region and the background region, and an initial pseudo label and a binarized erasure mask are generated.

[0065] Step S1 aims to provide the necessary basic features, label supervision, and erasure region information for subsequent scale-aware uncertainty assessment and cross-class pasting mechanism. The specific steps are as follows:

[0066] Step S1.1: For the ResNet-38 network model, remove the fully connected layers and set the number of output channels to [value missing]. ,in To determine the number of foreground categories, a backbone network is formed; training images are input into the backbone network, and class activation map features containing single-class features of each category are extracted. Class activation graph features Each channel in the text corresponds to a single-class feature;

[0067] By extracting class activation map features Medium category Single-class features Calculate and obtain the category Class activation graph The class activation maps of all categories are then concatenated along the channel dimension to obtain a channel count of [number missing]. Class activation graph :

[0068] ;

[0069] ;

[0070] in, express Activation functions are used to filter negative responses; This indicates global maximum value normalization processing, which will be applied to the category. Class activation graph Mapped to interval; Indicates will Concatenate activation maps of different classes; Indicates the first to the second Activation graph of each class;

[0071] Step S1.2: Targeting the class activation graph The activation values ​​of each pixel are preset with a high threshold. and low threshold Locate high-confidence foreground and background regions to generate initial pseudo-labels for calculating scale-aware uncertainty scores. The specific formula is as follows:

[0072] ;

[0073] in, Indicates coordinate position The initial pseudo-tags, Representation of class activation graph At coordinate position Activation vector extracted along the channel dimension; Representation of class activation graph features At coordinate position Feature vectors extracted along the channel dimension; Used to extract the highest activation value for each category; Used to extract the category with the highest activation value; 255 indicates the ignore label for uncertain regions; high threshold. Set as Low threshold Set as ;

[0074] Step S1.3: Activate the class graph The activation value is greater than or equal to the preset mask threshold. The pixel region is defined as the most discriminative region and subjected to binarization masking to obtain the binarized erase mask. :

[0075] ;

[0076] in, To preset the mask threshold, set it to... , For class activation graph At coordinate position Binarized erase mask.

[0077] Step S2: Calculate the scale-aware uncertainty score based on the initial pseudo-label, set a dynamic threshold for the scale-aware uncertainty score, and select the training image set with the scale-aware uncertainty score higher than the dynamic threshold in the current training batch to construct a candidate sample set;

[0078] Step S2 aims to quantify the mining potential of images and filter out image samples that truly need activation expansion, avoiding over-pasting of high-confidence images that have already been fully mined. The specific steps are as follows:

[0079] Step S2.1: The key to selecting potential data lies in designing appropriate criteria. The dilemma of activation expansion based on erasure or paste is that it needs to cover the discriminative region while avoiding complete object removal. An intuitive insight is that fewer uncertain pixels imply higher-confidence pseudo-labels, and the corresponding image is more likely to be fully utilized. Images with high-quality pseudo-labels typically do not require further exploration and optimization. Conversely, images corresponding to high-uncertainty pseudo-labels are more worthy of further exploration through paste operations. Therefore, a scale-aware uncertainty (SAU) metric is constructed to determine the necessity of pasting the current image. Specifically, considering that the area of ​​uncertain pixels around the target object is usually positively correlated with the object size, the area of ​​uncertain pixels is normalized by the target area, and the scale-aware uncertainty score is constructed as follows:

[0080] ;

[0081] in, The score represents the scale-perceived uncertainty. It is an indicator function; Indicates coordinate position; the numerator of the formula This represents the total area of ​​uncertain pixels corresponding to the ignored labels (value 255) in the uncertain regions of the initial pseudo-labels; the denominator is... This represents the total area of ​​pixels in the foreground region with activation values ​​between 0 and 255 in the initial pseudo-label;

[0082] Step S2.2: In each training image constituting a size of In the training batch, calculate the scale-aware uncertainty score set for all training images. And set a dynamic threshold for the scale-aware uncertainty score. :

[0083] ;

[0084] in, Indicates dynamic threshold. This represents the percentile in the training batch set. The preset scaling parameter is set to... In the formula, the dynamic threshold is... The first score of the mesoscale perceived uncertainty in the current training batch percentile;

[0085] Step S2.3: with The training images are high-uncertainty samples. High-uncertainty samples are selected from the training batch to construct a candidate sample set. This is used for subsequent cross-category paste operations, as shown below:

[0086] ;

[0087] in, This represents the scale-aware uncertainty score of the i-th training image in the training batch; This is the training batch size.

[0088] Step S3: For the training images in the candidate sample set, perform connected component analysis (CCA) based on the binarized erasure mask to obtain several independent connected components, and extract the connected component containing the highest activation value as the target region to be pasted.

[0089] Step S3 aims to precisely locate the most discriminative target region that needs to be disturbed, in order to avoid complex and extensive invalid pasting operations caused by the sparsity and dispersion of the class activation map. The specific steps are as follows:

[0090] Step S3.1: For each training image, obtain the binarized erasure mask generated in step S1. By analyzing connected components, the binarized erase mask in the training image is segmented into several independent connected components;

[0091] Step S3.2: To accurately implement cross-category pasting and minimize activation leakage, select the connected component containing the highest activation value as the high-response target region. This area will be used as the target area for pasting, and the specific formula is as follows:

[0092] ;

[0093] in, Representation of class activation graph At coordinate position The highest activation value in the activation vector; This indicates connected component analysis, used to obtain a set of several disjoint connected components; Representation of class activation graph The pixel coordinates with the highest global activation value; Represents the regions in the set of connected components; the final result is That is, a specific pixel containing the globally highest activation value. Once the high-response target area is identified, subsequent pasting perturbations will be specifically executed against the high-response target area, thereby enhancing the suppression of activation leakage while retaining the original erasure operation logic.

[0094] Step S4: Extract foreground blocks from single-class images in the training images, construct a foreground memory bank, extract the largest connected component of the foreground block from the single-class image as the discrimination region, and store the foreground block and the global average class confidence into the queues divided by class in the foreground memory bank respectively;

[0095] Step S4 aims to extract cross-class target features with sufficient discriminative power and clear category, so as to subsequently cover and physically block the activation of the original erased region, thereby breaking the network's shortcut learning of the original erased region. The specific steps are as follows:

[0096] Step S4.1: Based on the image-level weak labels attached to the training images themselves, determine the number of foreground categories contained in each training image, define the training image containing only one foreground category as a single-class image, and extract foreground blocks from the single-class images; in order to make the category information of the introduced object region clear and controllable, extract foreground blocks only from single-class images.

[0097] Step S4.2: Obtain the unique category of a single-class image foreground mask of the foreground block Binarization is performed, and the connected component with the largest area is selected through connected component analysis. As a category The specific extraction formula for the discrimination region in the current training image is as follows:

[0098] ;

[0099] in, This indicates the calculation of the area of ​​a connected region; Represents a region within a set of connected components; This represents connected component analysis;

[0100] Step S4.3: Construct a foreground memory to store the foreground blocks for each category; the foreground memory adopts a first-in-first-out (FIFO) queue mechanism based on category and is updated in each training iteration; simultaneously, calculate the category... Global average class confidence corresponding to the discrimination region in the current training image It is also stored in the foreground memory, and the calculation formula is:

[0101] ;

[0102] in, The class activation map features of the current training image; This indicates that clipping corresponds to... Operations within a region; This is a global average pooling operation. express Activation function.

[0103] Step S5: Randomly select a foreground block of a specific category from the foreground memory bank, extract cross-category foreground features, and paste them into the target area to be pasted;

[0104] Step S5 aims to physically block the original high-response region using discriminative features from other categories, forcing the model to break its dependence on the memory features of the erased region (i.e., shortcut learning), and redistributing attention to complementary regions of the target that have not been explored, thereby suppressing feature activation leakage. The specific steps are as follows:

[0105] Step S5.1: For the high-uncertainty samples selected in step S2, using the current training image as the anchor image, randomly select a category that does not exist in the anchor image. ;

[0106] Step S5.2: Randomly select a foreground block belonging to the category from the foreground blocks stored in the foreground memory constructed in step S4. Connected components and connected components Paste into the high-response target area identified in step S3 Above, obtain cross-class pasted images.

[0107] Step S6: Construct an image-level global alignment loss based on anchor branch and mask branch, train the cross-class pasting model based on the image-level global alignment loss to obtain the trained cross-class pasting model, and apply the trained cross-class pasting model to complete the cross-class pasting of the image.

[0108] The image-level global alignment loss constructed in step S6 based on anchor branch and mask branch As shown in the following formula:

[0109] ;

[0110] in, This is the class activation map feature of the current training image, belonging to the anchor branch; Input the feature extraction results after cross-class pasting of the image into the mask branch; For categories retrieved from the foreground memory The global average class confidence corresponding to the discrimination region in the current training image. Finally, the updated image-level global alignment loss is used. By backpropagating the original classification loss and other basic losses of the joint network, the model parameters are updated, prompting the model to discover a more complete target response region.

[0111] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the weakly supervised semantic segmentation method based on erasure-guided cross-class pasting.

[0112] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the weakly supervised semantic segmentation method based on erasure-guided cross-class pasting.

[0113] The following is an application example of the present invention:

[0114] To verify the effectiveness and rationality of the method of this invention, simulation experiments were conducted on the standard PASCAL VOC 2012 dataset to verify its effect on suppressing activation leakage and improving pseudo-label quality in weakly supervised semantic segmentation tasks. Specifically, the method of this invention is implemented based on a deep learning framework, using ResNet38 pre-trained on the ImageNet dataset as the basic backbone network for classification and class activation map extraction, and Deeplab V2, with ResNet101 pre-trained on the ImageNet dataset as the backbone network, as the final segmentation network. Regarding experimental parameters, a high threshold for generating initial pseudo-labels was used. Set as low threshold Set as ;Preset mask threshold for generating a binary erase mask Set as ; Calculate the percentile of the scale-perceived uncertainty score Set as For model optimization, SGD was used as the optimizer, with momentum set to 0.9 and weight decay set to [value missing]. The initial learning rate is set to A multinomial learning rate decay strategy was employed. The batch size was consistently set to 16 throughout the experiment.

[0115] For weakly supervised semantic segmentation tasks, this invention employs the mean Intersection over Union (mIoU) as the core evaluation metric. In pixel-level classification tasks, mIoU comprehensively measures the overlap between the model's predicted segmentation results and the ground truth annotations, making it the most important indicator for evaluating the completeness of target response region mining. Specifically, for a single class c, its mIoU... The calculation method is the ratio of the intersection area to the union area of ​​the predicted mask and the true mask, defined as:

[0116] ;

[0117] in, This represents the number of pixels predicted as positive and correctly predicted. This represents the number of pixels that were predicted as positive but were mispredicted. This represents the number of pixels that are actually positive but were missed in the prediction.

[0118] Based on this, the average crossover ratio For all categories participating in the evaluation (assuming a total of...) (1 category, including background category) The arithmetic mean of the two methods is used to obtain the following result:

[0119] ;

[0120] Average crossover ratio The larger the value, the more complete the target response area and the closer the boundary is to the actual target outline.

[0121] The method of this invention was quantitatively compared with several other mainstream weakly supervised semantic segmentation methods. The quantitative comparison of different methods on the Pascal VOC 2012 validation and test sets is shown in Table 1 below:

[0122] Table 1. Quantitative comparison of different methods on the Pascal VOC 2012 validation and test sets.

[0123]

[0124] On the PASCAL VOC 2012 dataset, the method of this invention significantly outperforms existing comparative methods in segmentation accuracy on both the validation and test sets, achieving 77.2% and 76.0% respectively. Compared to existing masking or erasing-based methods (such as AEFT), this invention's method, through an innovative cross-class feature pasting mechanism, physically blocks "feature activation leakage," effectively compensating for the shortcomings of traditional erasing methods in mining complementary regions. During the fusion and learning process, this invention's method forces the model to focus on the complete context of the target object, thereby generating high-quality pseudo-labels and ultimately leading to a significant system-level performance improvement.

[0125] Figure 2 This paper presents a visual comparison of the detection results of the proposed method and existing benchmark methods in real-world scenarios. The visualization clearly shows that the proposed method outperforms existing methods in terms of target contour integrity and background noise suppression. For example, in detecting complex targets such as pedestrians, horses, or aircraft, existing methods often miss edges or non-discriminatory parts (such as horse legs or aircraft tails) due to feature dependence; while the proposed method, through its dual filtering mechanism and cross-class feature physical interference, successfully activates the complete physical contour of the target, and its prediction results are closer to the ground truth annotations. These results demonstrate that the proposed method exhibits stronger robustness and adaptability in highly challenging weakly supervised scenarios, significantly improving the accuracy of target perception.

[0126] Experimental results show that the method of this invention maintains extremely high feature activation consistency in typical weakly supervised semantic segmentation scenarios. It not only effectively mines more complete complementary target regions but also significantly suppresses background noise interference, greatly surpassing existing benchmark methods in both generating high-quality pseudo-labels and achieving final segmentation accuracy. This fully demonstrates the feasibility and superior performance of the cross-class pasting leakage suppression strategy proposed in this invention in real-world complex scenarios.

[0127] In summary, the method of this invention achieves state-of-the-art performance in weakly supervised semantic segmentation tasks. This invention successfully prevents the model from becoming fixated on features of erased regions, forcing the model to redistribute attention to unexplored areas of the target, thereby significantly improving the integrity of the class activation map and the quality of pseudo-labels.

[0128] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A weakly supervised semantic segmentation method based on erase-guided cross-class pasting, characterized in that, Perform steps S1-S5 to construct a cross-category pasting model, and then perform step S6 to apply the cross-category pasting model to complete the cross-category pasting of the image. Step S1: Input the training image into the classification network to extract the class activation map. For the activation value of each pixel in the class activation map, a preset threshold is used to distinguish the foreground region and the background region, and an initial pseudo label and a binarized erasure mask are generated. Step S2: Calculate the scale-aware uncertainty score based on the initial pseudo-label, set a dynamic threshold for the scale-aware uncertainty score, and select the training image set with the scale-aware uncertainty score higher than the dynamic threshold in the current training batch to construct a candidate sample set; Step S3: For the training images in the candidate sample set, perform connected component analysis based on the binarized erasure mask to obtain several independent connected components, and extract the connected component containing the highest activation value as the target region to be pasted. Step S4: Extract foreground blocks from single-class images in the training images, construct a foreground memory bank, extract the largest connected component of the foreground block from the single-class image as the discrimination region, and store the foreground block and the global average class confidence into the queues divided by class in the foreground memory bank respectively; Step S5: Randomly select a foreground block of a specific category from the foreground memory bank, extract cross-category foreground features, and paste them into the target area to be pasted; Step S6: Construct an image-level global alignment loss based on anchor branch and mask branch, train the cross-class pasting model based on the image-level global alignment loss to obtain the trained cross-class pasting model, and apply the trained cross-class pasting model to complete the cross-class pasting of the image.

2. The weakly supervised semantic segmentation method based on erasure-guided cross-class pasting according to claim 1, characterized in that, The specific steps of step S1 are as follows: Step S1.1: For the ResNet-38 network model, remove the fully connected layers and set the number of output channels to [value missing]. ,in To determine the number of foreground categories, a backbone network is formed; training images are input into the backbone network, and class activation map features containing single-class features of each category are extracted. Class activation graph features Each channel in the text corresponds to a single-class feature; By extracting class activation map features Medium category Single-class features Calculate and obtain the category Class activation graph The class activation maps of all categories are then concatenated along the channel dimension to obtain a channel count of [number missing]. Class activation graph : ； ； in, express Activation functions are used to filter negative responses; This indicates global maximum value normalization processing, which will be applied to the category. Class activation graph Mapped to interval; Indicates will Concatenation of activation maps of different classes Indicates the first to the second Activation graph of each class; Step S1.2: Targeting the class activation graph The activation values ​​of each pixel are preset with a high threshold. and low threshold Locate the foreground and background regions and generate initial pseudo-tags. The specific formula is as follows: ； in, Indicates coordinate position The initial pseudo-tags, Representation of class activation graph At coordinate position Activation vector extracted along the channel dimension; Representation of class activation graph features At coordinate position Feature vectors extracted along the channel dimension; Used to extract the highest activation value for each category; Used to extract the category with the highest activation value; 255 indicates the ignore label for uncertain regions; Step S1.3: Activate the class graph The activation value is greater than or equal to the preset mask threshold. The pixel region is defined as the most discriminative region and subjected to binarization masking to obtain the binarized erase mask. : ； in, To preset the mask threshold, For class activation graph At coordinate position Binarized erase mask.

3. The weakly supervised semantic segmentation method based on erase-guided cross-class pasting according to claim 2, characterized in that, The specific steps of step S2 are as follows: Step S2.1: Construct the scale-aware uncertainty score as follows: ； in, The score represents the scale-perceived uncertainty. It is an indicator function; Indicates coordinate position; the numerator of the formula The denominator represents the total area of ​​uncertain pixels corresponding to the ignored labels in the uncertain regions of the initial pseudo-labels; This represents the total area of ​​pixels in the foreground region with activation values ​​between 0 and 255 in the initial pseudo-label; Step S2.2: In each training image constituting a size of In the training batch, calculate the scale-aware uncertainty score set for all training images. And set a dynamic threshold for the scale-aware uncertainty score. : ； in, Indicates dynamic threshold. This represents the percentile in the training batch set. The preset scaling parameters; Step S2.3: with The training images are high-uncertainty samples. High-uncertainty samples are selected from the training batch to construct a candidate sample set. As shown in the following formula: ； in, This represents the scale-aware uncertainty score of the i-th training image in the training batch; This is the training batch size.

4. The weakly supervised semantic segmentation method based on erase-guided cross-class pasting according to claim 3, characterized in that, The specific steps of step S3 are as follows: Step S3.1: For each training image, obtain the binarized erasure mask generated in step S1. By analyzing connected components, the binarized erase mask in the training image is segmented into several independent connected components; Step S3.2: Select the connected component containing the highest activation value from the connected components as the high-response target region. This area will be used as the target area for pasting, and the specific formula is as follows: ； in, Representation of class activation graph At coordinate position The highest activation value in the activation vector; This represents connected component analysis; Representation of class activation graph The pixel coordinates with the highest global activation value; It represents a region in a set of connected components.

5. A weakly supervised semantic segmentation method based on erase-guided cross-class pasting according to claim 4, characterized in that, The specific steps of step S4 are as follows: Step S4.1: Based on the image-level weak labels attached to the training images themselves, determine the number of foreground categories contained in each training image, define the training images containing only one foreground category as single-class images, and extract foreground patches from single-class images; Step S4.2: Obtain the unique category of a single-class image foreground mask of the foreground block Binarization is performed, and the connected component with the largest area is selected through connected component analysis. As a category The specific extraction formula for the discrimination region in the current training image is as follows: ； in, This indicates the calculation of the area of ​​a connected region; Represents a region within a set of connected components; This represents connected component analysis; Step S4.3: Construct a foreground memory to store the foreground blocks for each category; the foreground memory adopts a first-in-first-out queue mechanism based on category and is updated in each training iteration; simultaneously, calculate the category... Global average class confidence corresponding to the discrimination region in the current training image It is also stored in the foreground memory, and the calculation formula is: ； in, The class activation map features of the current training image; This indicates that clipping corresponds to... Operations within a region; This is a global average pooling operation. express Activation function.

6. A weakly supervised semantic segmentation method based on erase-guided cross-class pasting according to claim 5, characterized in that, The specific steps of step S5 are as follows: Step S5.1: For the high-uncertainty samples selected in step S2, using the current training image as the anchor image, randomly select a category that does not exist in the anchor image. ; Step S5.2: Randomly select a foreground block belonging to the category from the foreground blocks stored in the foreground memory constructed in step S4. Connected components and connected components Paste into the high-response target area identified in step S3 Above, obtain cross-class pasted images.

7. A weakly supervised semantic segmentation method based on erase-guided cross-class pasting as described in claim 6, characterized in that, The image-level global alignment loss constructed in step S6 based on anchor branch and mask branch As shown in the following formula: ； in, This is the class activation map feature of the current training image, belonging to the anchor branch; Input the feature extraction results after cross-class pasting of the image into the mask branch; For categories retrieved from the foreground memory The global average class confidence corresponding to the discrimination region in the current training image.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the weakly supervised semantic segmentation method based on erasure-guided cross-class pasting as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the weakly supervised semantic segmentation method based on erasure-guided cross-class pasting as described in any one of claims 1 to 7.

Images