Semantic Segmentation With Multi-Level Prototypes for Few-Shot Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Semantic segmentation in domains with a small labeled image set is challenging due to the difficulty in training deep learning models, and the accuracy for narrow object regions is low.
Innovation Solution
A method involving the extraction of multi-level features from support images, generation of prototypes based on these features, and prediction of segmentation masks using these prototypes, which includes aggregating activation maps and correct segmentation masks to enhance accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If a deep learning model with multiple layers is used for semantic segmentation, then the model can capture global context information, but segmentation accuracy for narrow object regions deteriorates due to large receptive field
Solution Approach 1:
The patent segments the feature extraction process into two distinct pathways: a first feature extractor that captures global context information and a second feature extractor that focuses on local detailed information. This segmentation allows the model to preserve both global and local characteristics separately, then fuse them to achieve accurate segmentation of narrow object regions while maintaining global context understanding
Solution Approach 2:
The patent applies local quality by designing the second feature extractor specifically optimized for capturing local detailed information in narrow object regions. The model assigns different functional qualities to different feature extractors: the first for global context and the second for local precision, allowing each component to excel at its specific task
2Adaptability or versatility
If a deep learning model is trained in a few-shot environment with small labeled image set, then the model can be trained with limited data, but segmentation accuracy deteriorates due to insufficient training samples
Solution Approach 1:
The patent performs preliminary action by pre-training the feature extractors using prototype learning on the limited labeled data before performing actual semantic segmentation. The prototype generation step creates representative class prototypes from the small labeled dataset, which then guide the segmentation process. This preliminary prototyping phase allows the model to adapt to few-shot environments while maintaining segmentation accuracy through the guidance of learned prototypes
3Measurement precision
If multi-level features are extracted and prototypes are generated, then segmentation accuracy for narrow object regions is improved, but computational complexity increases
Solution Approach 1:
The patent extracts only the essential local detailed information needed for narrow object segmentation through the second feature extractor, rather than processing all possible features. The prototype generation step extracts representative class characteristics from limited labeled data, focusing computational resources on the most discriminative features. This selective extraction reduces unnecessary computational complexity while maintaining accuracy for narrow regions
Data Source
AI summary
A semantic segmentation method performed by at least one computing device is provided. The method may include extracting a plurality of support features, which include a first-level feature and a second-level feature, from a support image; generating a plurality of prototypes for a class associated with the support image, based on the plurality of support features; and predicting a segmentation mask for a query image using the plurality of prototypes, wherein the plurality of prototypes comprise a first prototype generated based on the first-level feature and a second prototype generated based on the second-level feature.