Federated Continual Learning With Replay Buffers for Catastrophic Forgetting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models face catastrophic forgetting, where they forget how to solve prior tasks as their weights are updated with new data, and existing rehearsal methods are in need of improvement.
Innovation Solution
A method for federated continual learning across multiple decentralized training systems, where each system trains a machine learning algorithm using local and replay training datasets, sharing information and data to prevent forgetting while preserving privacy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the ML model weights are updated with new data to learn new tasks, then the model's ability to solve new tasks is improved, but the model forgets how to solve prior tasks (catastrophic forgetting)
Solution Approach 1:
The patent stores training data from previous tasks in a replay buffer before the model forgets them. This preliminary preservation of data allows the model to revisit and relearn from past tasks during subsequent training iterations, preventing catastrophic forgetting while still enabling learning of new tasks.
Solution Approach 2:
The replay buffer acts as an intermediary between past training data and the current model training process. It mediates the conflict between learning new tasks and retaining old knowledge by providing a mechanism to periodically review and reinforce previously learned information during the training of new tasks.
2Reliability
If rehearsal methods are used to prevent catastrophic forgetting, then retention of prior tasks is improved, but the training process becomes more complex and resource-intensive
Solution Approach 1:
The patent extracts only the essential training data from previous tasks and stores it in a replay buffer, rather than attempting to preserve the entire training process or model states. This extraction of critical information simplifies the rehearsal mechanism while maintaining its effectiveness in preventing catastrophic forgetting.
Solution Approach 2:
The system discards the assumption that models can retain knowledge indefinitely and instead implements a recovery mechanism where forgotten information is periodically retrieved from the replay buffer. This approach simplifies the overall system by accepting forgetting as inevitable and providing a straightforward recovery process rather than attempting complex prevention.
Data Source
AI summary
The present disclosure relates to a method comprising a training system iteratively training a machine learning algorithm using current training data. The current training data comprises a local dataset of a current task and a replay dataset and may be updated for a next iteration as follows. A training dataset may be received. If the training dataset is not a shared dataset and its task is different from the current task: information representing the local dataset may be shared with other training systems, the local dataset may be added to the replay dataset, and the received training dataset may be used as the local dataset for a next iteration. In case the task is the current task: the received training dataset may be added to the local dataset. If the training dataset is a shared dataset, the received training dataset may be added to the replay dataset.