Graph Model for Continual Learning Preventing Catastrophic Forgetting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for continual learning in classification tasks face challenges such as catastrophic forgetting, where knowledge acquired in the past is lost, and intertask confusion, where classification accuracy decreases when learning new classes.
Innovation Solution
A machine learning apparatus and method that employs a feature extraction unit pre-trained on base class data, a weight calculation unit for novel class weights, and a graph model trained through pseudo-continual learning to adapt and reconstruct classification weights, effectively learning dependencies between base and novel classes using meta-learning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If continual learning is used to learn novel classes while maintaining base class knowledge, then the ability to classify both base and novel classes is improved, but catastrophic forgetting occurs causing loss of base class knowledge
Solution Approach 1:
The method performs preliminary actions by generating pseudo novel class data from base class data before actual novel class learning begins. This pre-training of the graph model on pseudo data prepares the system to handle novel classes while maintaining base class knowledge, preventing catastrophic forgetting by establishing proper feature relationships in advance
Solution Approach 2:
The graph model serves as an intermediary that learns the dependency relationships between base class and novel class features. By training this intermediary model on pseudo continual learning tasks, the system can properly integrate novel class information without directly interfering with base class knowledge storage, thus preventing catastrophic forgetting
2Ease of manufacture
If a graph model is trained using only rotated base class images as pseudo novel class data, then the training process is simplified, but sufficient classification accuracy cannot be obtained
Solution Approach 1:
The method merges multiple data sources and processing steps: generating rotated images, creating synthetic images through blending, and training the graph model on this combined pseudo continual learning dataset. This combination of multiple approaches creates more diverse and effective training data while maintaining process feasibility
Solution Approach 2:
The training process is made dynamic by iteratively generating pseudo novel class data through rotation and synthesis, then using this dynamically created data to train the graph model. The system adapts the training data generation process based on learning progress, improving classification accuracy through multiple training iterations rather than static data
Data Source
AI summary
In a machine learning apparatus that learns data of a novel class with a smaller number of samples than data of a base class by continual learning, a feature extraction unit is pre-trained using first data and second data of the base class. The feature extraction unit receives an input of the data of the novel class to output a feature vector of the data of the novel class. A weight calculation unit calculates a classification weight of the novel class based on the feature vector. A graph model receives an input of the classification weight calculated and classification weights of all classes previously learned and outputs reconstructed classification weights. The graph model is trained by pseudo continual learning using third data of the base class. The first data, the second data, and the third data are different data.