Continual Learning Error Sensitivity Modulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) face catastrophic forgetting when learning from a stream of data with varying distribution shifts, leading to abrupt representation drifts and forgetting of previous tasks, unlike the human brain's ability to robustly adapt to dynamic environments.
Innovation Solution
A computer-implemented method for continual learning in artificial neural networks that maintains a memory of errors using an exponential moving average of supervised loss, employs a dual memory system with a fixed-size episodic memory and a stable model, and adjusts the contribution of each input sample based on its cross-entropy loss relative to the error memory, to modulate error sensitivity and reduce representation drift.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the model is exposed to varying degrees of input and/or output distribution shift to learn new tasks, then the model's adaptability to new tasks is improved, but catastrophic forgetting of previous tasks occurs
Solution Approach 1:
The patent applies preliminary action by maintaining a memory buffer that stores representations from previous tasks before new tasks are introduced. This buffer is prepared in advance and used to compute loss functions that prevent forgetting, allowing the model to adapt to new tasks while preserving knowledge of previous ones through proactive retention mechanisms.
Solution Approach 2:
The patent implements feedback mechanisms through multiple loss functions that continuously monitor and adjust the model's performance. The total loss combines classification loss for current tasks with regularization losses (L2 regularization, entropy regularization) and memory retrieval losses that provide feedback on how well previous task representations are preserved, enabling dynamic adjustment to balance learning new tasks and retaining old knowledge.
2Reliability
If replay of past activations is used to reduce forgetting, then catastrophic forgetting is mitigated, but the model undergoes drastic drift in representations when there is a distribution shift
Solution Approach 1:
The patent applies parameter changes by dynamically adjusting the weighting of different loss functions during training. The regularization coefficients (lambda parameters) for L2 regularization, entropy regularization, and memory retrieval losses are modified based on training progress and distribution shift detection, allowing the model to stabilize representations when drift is detected while still enabling adaptation when necessary.
Solution Approach 2:
The patent implements dynamics through adaptive learning rates and dynamic loss weighting that respond to the training process in real-time. The model transitions from emphasizing retention of previous tasks in early training to balancing retention with adaptation to new tasks, creating a dynamic training process that adjusts the strength of replay mechanisms based on the current training stage and observed distribution shifts.
3Quantity of substance
If a small buffer is used to approximate the joint distribution of tasks, then memory efficiency is improved, but the model cannot effectively capture the joint distribution and suffers from representation drift
Solution Approach 1:
The patent applies local quality by storing diverse representations from different tasks and time points in the memory buffer rather than uniform samples. The buffer selectively retains representations that are most valuable for preventing forgetting, using task identifiers and temporal information to ensure local quality and diversity of stored samples, which improves the approximation of the joint distribution even with limited buffer size.
Solution Approach 2:
The patent implements copying by creating and storing compressed representations (embeddings) of past activations and task characteristics in the memory buffer. Instead of storing full activation sequences, the model maintains compact copied representations that capture essential information from previous tasks, enabling efficient memory usage while preserving the necessary information for preventing catastrophic forgetting.
Data Source
AI summary
A computer-implemented method that maintains a memory of errors along the training trajectory and adjusts the contribution of each sample towards learning based on how far it is from the mean statistics of the error memory. The method may include the step of maintaining an additional semantic memory, called a stable model, which gradually aggregates the knowledge encoded in the weights of the working model. The stable model is utilized to select the low loss samples from the current task for populating the error memory. The different components of the method complement each other to effectively reduce the drift in representations at the task boundary and enables consolidation of information across the tasks.