Covariance-Based Neural Network Classifier for Continual Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning models face challenges in visual classification due to catastrophic forgetting and distributional shifts when adapting to new classes and domains, failing to maintain performance on previous classes and generalize to unseen domains effectively.
Innovation Solution
A computer-implemented method for training a task-based neural network classifier using a center and covariance matrix for each class, which minimizes the impact of catastrophic forgetting by leveraging similarity measures between image features and the center-covariance pairs, allowing for domain generalizable continual learning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a standard neural network is trained on a fixed dataset with fixed classes, then it achieves good classification performance on that dataset, but it cannot adapt to learn new classes without degrading performance on previous classes (catastrophic forgetting)
Solution Approach 1:
The patent changes the parameter representation in the final layer from fixed weights to a similarity function based on class centers and covariance matrices. This allows the model to adapt to new classes by updating these parameters while maintaining the ability to generalize to unseen domains through domain-invariant feature learning in the convolutional base.
Solution Approach 2:
The patent introduces an intermediary similarity function that acts as a bridge between the fixed convolutional feature extractor and the adaptive classification layer. This similarity function using centers and covariances allows flexible adaptation to new classes while preserving knowledge of previous classes, resolving the catastrophic forgetting problem.
2Adaptability or versatility
If a model is trained to adapt to new classes, then it gains versatility, but it fails to generalize to unseen domains due to distributional shifts
Solution Approach 1:
The patent segments the neural network into two parts: a fixed convolutional base that learns domain-invariant features, and a separate adaptive final layer with centers and covariance matrices that handles class-specific adaptation. This segmentation allows the model to adapt to new classes without compromising generalization to unseen domains.
Solution Approach 2:
The patent changes the final layer parameters from traditional weights to a similarity-based representation using class centers and covariance matrices. This parameter change enables the model to adapt to distributional shifts and generalize better to unseen domains while maintaining adaptability to new classes.
3Reliability
If previous proposals match the output of two different models for continual learning, then they attempt to prevent forgetting, but they fail to handle distributional shifts effectively
Solution Approach 1:
The patent replaces the mechanical approach of matching outputs from two different models with a similarity-based classification layer using centers and covariance matrices. This substitution allows the model to prevent catastrophic forgetting while simultaneously handling distributional shifts through domain-invariant feature learning.
Data Source
AI summary
A computer-implemented method for model training is provided. The method includes receiving, by a hardware processor, sets of images, each set corresponding to a respective task. The method further includes training, by the hardware processor, a task-based neural network classifier having a center and a covariance matrix for each of a plurality of classes in a last layer of the task-based neural network classifier and a plurality of convolutional layers preceding the last layer, by using a similarity between an image feature of a last convolutional layer from among the plurality of convolutional layers and the center and the covariance matrix for a given one of the plurality of classes, the similarity minimizing an impact of a data model forgetting problem.