Online Class-Incremental Learning with Adversarial Shapley Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks suffer from catastrophic forgetting when exposed to new data without reviewing previously seen data, and existing continual learning methods struggle with memory-based replay techniques for online class-incremental settings, particularly in resource-constrained environments.
Innovation Solution
The Adversarial Shapley Value (ASV) scoring method evaluates memory data samples based on their ability to preserve latent decision boundaries for previous classes while interfering with current class boundaries, using K-Nearest Neighbor Shapley values (KNN-SV) to select and update memory samples strategically.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If DNNs are continuously exposed to new data without reviewing previously seen data, then new knowledge is learned efficiently, but catastrophic forgetting occurs and previously learned knowledge is lost
Solution Approach 1:
The system performs preliminary action by proactively selecting and storing representative data samples from each task before moving to the next task. The replay buffer pre-stores these samples so that when catastrophic forgetting is detected or predicted, the system can immediately replay them without re-collecting or re-processing original data, thus maintaining knowledge retention while preserving continuous learning efficiency
Solution Approach 2:
The system implements continuity of useful action by continuously replaying selected samples from the replay buffer during training on new tasks. This continuous replay mechanism ensures that previously learned knowledge is continuously reinforced and maintained, preventing catastrophic forgetting while allowing the model to learn new tasks, thus maintaining both learning productivity and knowledge retention over time
2Reliability
If memory replay techniques are used in online class-incremental settings, then catastrophic forgetting is reduced, but computational resources and memory footprint increase
Solution Approach 1:
The system extracts only the most critical and representative samples from each task for storage in the replay buffer, rather than storing all training data. This selective extraction is based on criteria such as representation diversity, loss magnitude, and gradient norm, ensuring that a minimal set of samples suffices to prevent catastrophic forgetting, thus reducing memory footprint and computational overhead
Solution Approach 2:
The system applies local quality by differentiating which samples to replay based on their specific characteristics and importance. Different samples are selected with different priorities based on local properties such as their representativeness of particular classes, their contribution to decision boundaries, and their relevance to preventing forgetting of specific tasks, optimizing the replay process for minimal resource consumption
3Reliability
If task-incremental setting with multi-head evaluation is used, then knowledge retention is improved, but task identity supervision is required which limits applicability
Solution Approach 1:
The system implements universality by using a single unified head for all tasks rather than task-specific heads. This single head is trained to handle multiple tasks simultaneously using task-agnostic loss functions that combine current task loss with replay loss, enabling the model to generalize across tasks without requiring task identity information, thus achieving both knowledge retention and broad applicability to real-world scenarios
Solution Approach 2:
The system introduces an intermediary mechanism in the form of task-agnostic loss functions that mediate between current task learning and knowledge retention. These loss functions act as intermediaries that translate the requirement for knowledge retention into training objectives that can be achieved without task identity, bridging the gap between supervised task-incremental learning and unsupervised online learning settings
4Reliability
If all training data is stored in memory for replay, then complete knowledge retention is achieved, but memory footprint becomes unmanageable
Solution Approach 1:
The system applies partial action by replaying only a subset of training data - specifically, the most representative samples - rather than replaying all training data. This partial replay is sufficient to prevent catastrophic forgetting for the most critical knowledge, achieving acceptable knowledge retention with minimal memory usage, following the principle that complete action is unnecessary when partial action suffices
Solution Approach 2:
The system implements discarding and recovering by selectively discarding less important training samples from the replay buffer while retaining and periodically replaying critical samples. The replay buffer dynamically manages its contents by discarding samples that are less representative or less important for preventing forgetting, and recovering critical knowledge through targeted replay of remaining samples, thus maintaining knowledge retention with limited memory
Data Source
AI summary
A method for scoring training data samples according to an ability to preserve latent decision boundaries for previously observed classes while promoting learning from an input batch of new images from an online data stream, comprising: receiving the input batch of the new images from the online data stream, performing a memory retrieval process that retrieves data to be learned along with a new set of data from the memory to retain the previously learned knowledge, and performing a memory update process that selects and exchanges a small set of data to be saved in the memory in the memory update process. In addition, the method performs data valuation based on KNN-SV for both the memory retrieval and memory update processes to perform strategic and intuitive data selection based on the properties of KNN-SV.