Synthetic Outlier Regularization for Continual Learning Classifier Heads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In continual learning for machine learning models, existing methods face challenges in maintaining performance on old data while updating with new data without retraining from scratch, leading to catastrophic forgetting and high computational costs, especially in rehearsal-free class-incremental learning settings where distinguishing inputs from different training sessions is crucial.
Innovation Solution
The method involves generating synthetic outlier samples using Gaussian noise applied to feature vectors at the boundary of clusters from the same training session, which are used to regularize the classifier head, allowing for precise delineation of decision boundaries and reducing misclassification between training sessions, while keeping the pre-trained encoder fixed and updating only the classification head and prompt.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the model is updated with new data using standard continual learning methods, then the model learns new classes, but the model performance on old data deteriorates due to catastrophic forgetting
Solution Approach 1:
The method generates synthetic outlier samples in advance before the actual training process. These outlier samples are created by adding Gaussian noise to boundary samples from the support set, preparing regularization data that will prevent catastrophic forgetting during subsequent training sessions
Solution Approach 2:
Synthetic outlier samples serve as an intermediary element between the support set (old data) and the query set (new data). These outliers act as a buffer that helps the classifier maintain proper decision boundaries, preventing direct interference between old and new class representations
2Reliability
If a pool of prompts associated with different training sessions is used to prevent knowledge overwriting, then knowledge retention is improved, but extra computation is required to compose appropriate prompts
Solution Approach 1:
The method extracts and removes the prompt composition step from the inference process. Instead of maintaining a pool of prompts that require selection and composition, the approach uses a single fixed prompt combined with synthetic outlier regularization, eliminating the computational overhead of prompt management
Solution Approach 2:
The method creates synthetic copies of boundary samples by adding Gaussian noise to generate outlier samples. These copied and perturbed samples serve as regularizers that mimic the structure of real data boundaries without requiring actual access to multiple training sessions or prompt variations
3Productivity
If the classifier head is updated frequently to adapt to new data, then the model adapts quickly to new classes, but the decision boundaries become less precise and misclassification increases
Solution Approach 1:
The method performs preliminary generation of synthetic outlier samples before training. These pre-computed outliers are then used as regularizers during classifier updates, ensuring that decision boundaries are continuously refined even as the classifier adapts to new classes, thereby maintaining precision alongside adaptability
Data Source
AI summary
In some aspects, the techniques described herein relate to a method including: determining a first cross-entropy loss, wherein the first cross-entropy loss is determined based on a set of predictions, and wherein the set of predictions are based on a classifier head of a machine learning model generating the set of predictions based on a set of feature vectors; updating the classifier head and a prompt of the machine learning model with the first cross-entropy loss; generating outlier samples based on the set of feature vectors; providing, as input to the classifier head, the set of feature vectors and the outlier samples, wherein a second cross-entropy loss and an outlier regularization loss are computed by the classifier head based on the set of feature vectors and the outlier samples; and updating the classifier head with the second cross-entropy loss and the outlier regularization loss.