Stored Feature Vectors for Class-Incremental Semantic Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing semantic segmentation neural networks face the catastrophic forgetting phenomenon when performing class-incremental learning, leading to reduced identification performance for existing classes, and require large memory capacity for storing learning data or generating virtual data, which poses security and privacy issues.
Innovation Solution
A class-incremental semantic segmentation learning device and method that stores feature vectors of existing learning data and utilizes them for incremental learning, transforming these vectors using a rotation matrix to maintain performance without storing large amounts of data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If learning data is stored to prevent catastrophic forgetting, then identification performance for existing classes is maintained, but memory capacity requirements increase significantly
Solution Approach 1:
The patent extracts only the essential feature representations (logits and feature vectors) from the complete learning data, storing only these compressed representations instead of the full images. This extraction principle maintains the critical information needed for preventing catastrophic forgetting while dramatically reducing storage requirements from gigabytes to kilobytes per class.
Solution Approach 2:
The patent creates compressed copies of learning data in the form of feature vectors and logits that capture the essential characteristics needed for knowledge distillation. These copied representations serve as sufficient substitutes for the original learning data, enabling the model to retain knowledge of existing classes without storing the complete original datasets.
2Quantity of substance
If virtual learning data is generated using generative adversarial network, then memory capacity is reduced, but generation time and computational resources increase
Solution Approach 1:
The patent performs preliminary extraction of feature vectors and logits during the initial training phase, storing these pre-computed representations for later use in incremental learning. This preliminary action eliminates the need for time-consuming virtual data generation during incremental learning, as the essential features are already extracted and ready for knowledge distillation.
3Reliability
If complete retraining is performed for class-incremental learning, then identification performance is maintained, but training time and computational resources increase
Solution Approach 1:
The patent extracts and stores only the critical knowledge representations (logits and feature vectors) from the pre-trained model, enabling incremental learning to focus on acquiring new class knowledge without relearning existing classes. This extraction approach maintains identification performance while dramatically reducing training time compared to complete retraining.
Solution Approach 2:
The patent changes the learning parameters by using knowledge distillation with pre-extracted features instead of training from scratch or using standard incremental learning approaches. This parameter change enables the model to efficiently learn new classes while preserving knowledge of existing classes, achieving both high productivity and maintained reliability.
Data Source
AI summary
A class-incremental semantic segmentation learning method includes: obtaining an incremental semantic segmentation neural network by performing incremental learning on the basis of incremental learning data for an additional incremental class on a previously trained existing semantic segmentation neural network to classify an existing learning class according to existing learning data; after incremental learning with class-based feature vectors, extracted from a previous feature map estimated by the previously trained existing semantic segmentation neural network and pre-stored, obtaining a class-based representative feature vector by estimating a correlation between an existing feature map and an incremental feature map obtained from the existing semantic segmentation neural network and the incremental semantic segmentation neural network, respectively; performing transformation learning which sets a class-based rotation matrix on the basis of the class-based representative feature vector; and performing learning by inputting the transformed class-based feature vector into the semantic segmentation neural network using the class-based rotation matrix.