Semantic Segmentation Training With Masked Weights for Domain Shift
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Image semantic segmentation networks trained on one distribution of images exhibit low accuracy when tested on images with different distributions due to sensitivity to feature variations, such as those from different cities, races, or environments.
Innovation Solution
A method involving a mask network to screen network weights insensitive to feature distribution, followed by training on both a training and test image set with different distributions, using adversarial training with a discriminator to enhance generalization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the image semantic segmentation network is trained using a training image set with a specific feature distribution, then the network achieves good segmentation accuracy on the training set, but the network exhibits low accuracy when tested on test images with different feature distributions
Solution Approach 1:
The network weights are segmented into two categories: distribution-insensitive weights and distribution-sensitive weights. The mask network identifies and separates these weights, allowing the distribution-insensitive weights to be retained for maintaining segmentation accuracy while the distribution-sensitive weights are replaced or adjusted to improve generalization to different distributions.
Solution Approach 2:
The invention changes the parameters (weights) of the neural network by using a mask network to identify and select weights that are insensitive to distribution changes. This parameter selection and replacement process transforms the network to be more robust against distribution shifts while maintaining its segmentation capabilities.
2Measurement precision
If all network weights are retained during training, then the network maintains high accuracy on the training distribution, but the network becomes sensitive to feature distribution variations and performs poorly on test sets with different distributions
Solution Approach 1:
The mask network extracts and identifies the distribution-insensitive weights from the complete set of network weights. By taking out only the essential weights that contribute to robust performance across different distributions, the invention maintains training accuracy while improving reliability on test sets with different distributions.
Solution Approach 2:
The mask network is trained preliminarily to identify distribution-insensitive weights before the final training phase. This preliminary action of weight selection and masking prepares the network in advance to handle distribution shifts, ensuring both high training accuracy and robustness when deployed on test sets with different distributions.
Data Source
AI summary
A method for training an image semantic segmentation network is provided. The method includes: training a first image semantic segmentation network based on a training image set, training images in the training image set including annotation information; performing mask processing on network weights of the first image semantic segmentation network by using a mask network, to obtain a second image semantic segmentation network, the mask network being configured to select network weights in the first image semantic segmentation network that are insensitive to a distribution of image features in the training image set; and training the second image semantic segmentation network based on the training image set and a test image set, the distribution of image features in the training image set being different from a distribution of image features in the test image set.