Few-Shot Learning Apparatus Using Merged Feature Vectors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing incremental few-shot learning methods, such as XtarNet, face difficulties in converging due to the need to train multiple modules in the meta-learning process, making it challenging to adapt to new tasks without forgetting old ones.
Innovation Solution
The proposed machine learning apparatus and method reduce the complexity by pre-training base and novel class feature extractors, using self-distillation to obtain a novel class feature extractor, merging feature vectors, and classifying query samples based on a projected space distance, thereby minimizing classification loss with fewer modules to train in the meta-learning process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple modules are trained in the meta-learning process for task-adaptive representation extraction, then the ability to learn new tasks is improved, but the device complexity and difficulty of convergence increase
Solution Approach 1:
The base class feature extractor is pre-trained on base class data before the meta-learning process. This preliminary training allows the novel class feature extractor to be initialized with useful features, reducing the number of modules that need to be trained from scratch during meta-learning and simplifying the overall system complexity while maintaining the ability to learn new tasks
Solution Approach 2:
The learning system is segmented into distinct components: a pre-trained base class feature extractor and a novel class feature extractor that is initialized from the base class extractor. This segmentation allows each module to have specific functions and reduces the complexity of training all modules simultaneously during meta-learning
2Measurement precision
If multiple modules are trained in the meta-learning process, then the classification accuracy may improve, but the learning time increases
Solution Approach 1:
By pre-training the base class feature extractor before meta-learning, the system reduces the time required during meta-learning since the novel class feature extractor can be initialized with pre-computed features. This preliminary action significantly decreases the overall learning time while maintaining classification accuracy through the continued training of the novel class extractor
Solution Approach 2:
Instead of training all possible modules to full convergence, the system uses partial training during meta-learning by initializing the novel class feature extractor from the base class extractor. This partial action approach achieves sufficient classification accuracy without the time cost of training all modules from scratch
Data Source
AI summary
A machine learning apparatus that continually learns a novel class with fewer samples than a base class is provided. A base class feature extraction unit extracts a feature vector of the base class. A novel class feature extraction unit extracts a feature vector of the novel class. A merged feature calculation unit merges the feature vector of the base class and the feature vector of the novel class to calculate a merged feature vector that merges the base class and the novel class. A learning unit classifies, on a projected space, a query sample of a query set based on a distance between a position of the merged feature vector of the query sample of the query set and a position of a classification weight vector of each class, and learns a classification weight vector of the novel class to minimize a loss incurred in classification.