On-Chip Continual Learning via Streaming Linear Discriminant Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional edge CNN accelerators lack the ability to learn and adapt dynamically due to high computation and memory requirements, leading to catastrophic forgetting and reliance on remote server training, which is inefficient and privacy-constrained.
Innovation Solution
An on-chip continual learning system using a streaming linear discriminant analysis (SLDA) model with a novel edge accelerator, combining FPGA with a CNN, that updates weights locally without retraining the entire model, utilizing a pre-trained CNN to extract features and update covariance matrices and means for incremental learning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional CNN training based on backpropagation is used to enable continual learning on edge devices, then the ability to learn and adapt to new knowledge is improved, but the computation and memory requirements become prohibitively high for resource-constrained edge accelerators
Solution Approach 1:
The patent extracts only the essential learning components (mean and covariance matrix updates) from the full backpropagation training process. By separating the feature extraction function (performed by the pre-trained CNN) from the learning function (performed by SLDA on edge), the system achieves continual learning capability with minimal computation and memory resources on edge devices.
Solution Approach 2:
The patent introduces Streaming Linear Discriminant Analysis (SLDA) as an intermediary method between full backpropagation training and simple inference. SLDA serves as a lightweight learning algorithm that operates on extracted features, enabling edge devices to perform continual learning without the prohibitive resource requirements of conventional backpropagation.
2Adaptability or versatility
If the edge accelerator constantly transfers data and retrieves models from a remote server to accumulate knowledge, then the model can be updated with new knowledge, but the round-trip communication overhead increases latency and reduces responsiveness
Solution Approach 1:
The patent enables the edge accelerator to perform self-learning through SLDA by locally processing incoming data samples and updating its own model parameters. This self-service capability eliminates the need for constant server communication, allowing the system to accumulate knowledge autonomously and respond in real-time without communication delays.
3Adaptability or versatility
If conventional DNNs are used for continual learning, then new knowledge can be learned, but catastrophic forgetting occurs where previous knowledge is lost
Solution Approach 1:
The patent applies preliminary action by pre-training the CNN on comprehensive datasets before deployment. This pre-trained model serves as a knowledge base that is then refined through SLDA. The method of incremental learning with regularization terms in SLDA helps preserve previously learned information while adapting to new knowledge, preventing catastrophic forgetting.
4Adaptability or versatility
If the entire model is retrained to add new knowledge, then complete knowledge update is achieved, but the process becomes slow and requires large storage capacity on the edge device
Solution Approach 1:
The patent applies partial action by updating only the necessary components (mean and covariance matrices) through SLDA rather than retraining the entire CNN model. This selective updating approach achieves effective knowledge update while maintaining fast learning speed and minimizing storage requirements on the resource-constrained edge device.
Data Source
AI summary
A continual learning method, comprising: obtaining, for each of a plurality of classes, a mean of each of a plurality of features generated by passing a plurality of training samples to a pre-trained convolutional neural network, and a covariance matrix of the plurality of fea-tures, wherein each of the training samples belongs to one of the classes; obtaining extracted features from the pre-trained convolutional neural network for an additional training sample belonging to a class of said classes; generating a centred feature vector using the mean for the class and the extracted features; updating, using the centred feature vector and the covar-iance matrix, the covariance matrix, wherein said updating is performed for diagonal ele-ments of the covariance matrix but not for off-diagonal elements; updating the mean for the class using the extracted features; and outputting, based on the updated mean and updated covariance matrix, weights of a streaming linear discriminant analysis (SLDA) model.