Continuous Federated Learning Using Synthetic Data Retuning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning models face challenges in continuously learning and adapting with new data while retaining previously learned knowledge, especially in scenarios where data sharing is restricted by privacy concerns, leading to catastrophic forgetting and performance degradation due to site-specific variabilities.
Innovation Solution
A continuous federated learning framework utilizing synthetic datasets generated from actual datasets, allowing local models to be retuned without sharing actual data, which captures site-specific distributions and maintains global performance by using generative models for guided tissue contrast transformation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If deep learning models are retrained with incremental new data, then the models can adapt to new knowledge, but previously learned knowledge is lost (catastrophic forgetting)
Solution Approach 1:
The system performs preliminary actions by creating synthetic datasets that capture the distribution of actual training data before retraining occurs. These synthetic datasets are generated in advance and stored, allowing the model to be retrained on new data while referencing the pre-generated synthetic data to preserve previously learned knowledge patterns.
Solution Approach 2:
The system creates copies of the actual training data distribution through synthetic data generation. Instead of directly using the original data for retraining (which causes catastrophic forgetting), the system uses synthetic copies that replicate the essential characteristics and distributions of the original data, allowing retraining without direct access to the original data.
2Reliability
If deep learning models are retrained from scratch on entire datasets, then catastrophic forgetting is avoided, but data privacy concerns prevent sharing of training data
Solution Approach 1:
The system introduces synthetic data as an intermediary between the actual training data and the retraining process. Instead of directly sharing or retraining on sensitive actual data, the synthetic data serves as a mediator that captures the essential patterns and distributions without containing the actual private information, enabling retraining while preserving data privacy.
Solution Approach 2:
The system creates synthetic copies of the actual training data that preserve the statistical properties and distributions necessary for effective retraining. These copies allow the model to be retrained on entire datasets (preventing catastrophic forgetting) without requiring access to or sharing of the actual private training data.
3Adaptability or versatility
If deep learning models are trained on site-specific data, then local preferences are captured, but model performance degrades on diverse datasets from different sites
Solution Approach 1:
The system creates synthetic datasets that are designed to be universal across multiple sites and domains. The synthetic data generation process captures diverse patterns and distributions from various sources, creating a multi-functional training resource that can serve different local needs while maintaining generalizability across diverse datasets from different sites.
Solution Approach 2:
The system employs parameter changes in the synthetic data generation process to create variations that reflect different site-specific characteristics. By adjusting generation parameters, the system can produce synthetic data that adapts to local preferences while maintaining the underlying patterns necessary for good performance on diverse datasets.
Data Source
AI summary
A computer implemented method is provided. The method includes establishing, via multiple processors, a continuous federated learning framework including a global model at a global site and respective local models derived from the global model at respective local sites. The method also includes retraining or retuning, via the multiple processors, the global model and the respective local models without sharing actual datasets between the global site and the respective local sites but instead sharing synthetic datasets generated from the actual datasets.