Feature Shuffling for Weakly Supervised Segmentation Bias Removal
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing weakly supervised semantic segmentation methods suffer from background bias due to classifiers overemphasizing background regions, leading to inaccurate class activation maps and pseudo-masks, as they exploit biases in datasets where specific objects frequently appear alongside specific backgrounds.
Innovation Solution
A data augmentation method involving feature separation and shuffling of object and background features within mini-batches, followed by contrastive learning to reduce similarity between these features, and generating synthetic features to create diverse representations for classifiers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If classifiers are trained to predict category using image-level class labels, then classification speed is improved, but background bias is generated causing inaccurate pixel-level segmentation
Solution Approach 1:
The patent segments features into object features and background features using separate aggregators (first aggregator for object features, second aggregator for background features). This segmentation allows the model to process and weigh different feature types independently, preventing background regions from dominating the classification decision while maintaining efficient image-level classification.
2Productivity
If classifiers exploit biases in datasets as shortcuts, then training efficiency is improved, but class activation maps become blurred and inaccurate
Solution Approach 1:
The patent introduces an intermediary mechanism (the shuffler and contrastive learning process) that mediates between the classifier's tendency to exploit background biases and the need for accurate object localization. By shuffling background features and performing contrastive learning, the system prevents direct exploitation of background shortcuts while maintaining training efficiency.
3Reliability
If deep learning models are trained with augmented data considering only object context, then object recognition is improved, but background bias remains unaddressed limiting segmentation accuracy
Solution Approach 1:
The patent applies local quality by treating object features and background features differently through dedicated aggregators. The first aggregator focuses on object-related features while the second aggregator handles background features, allowing each to be optimized for its specific function. This local specialization improves both object recognition and segmentation accuracy simultaneously.
Data Source
AI summary
A data augmentation method includes inputting multiple images constituting a mini-batch into an encoder and extracting features for respective images of the multiple images, inputting the extracted features of the respective images into a pre-trained first aggregator and second aggregator and separating the extracted features into object features, each being a feature of an object portion of each image, and background features, each being a feature of a background portion of each image, inputting the object feature and background feature of each of the images into a shuffler and shuffling either the object features or the background features within the mini-batch, generating a synthetic feature by synthesizing the shuffled feature and a non-shuffled feature among the object feature and the background feature in a synthesis unit, and generating a data-augmented image based on the synthetic feature.