GAN-Based Synthetic Data Generation for ML Retraining
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models lose generalization capabilities when presented with new data, especially when data from disconnected nodes is not available, leading to skewed retraining and catastrophic forgetting due to data imbalances and impracticality of unlimited storage for retraining with all data.
Innovation Solution
Implementing Generative Adversarial Networks (GANs) to generate synthetic data that resembles the underlying distribution of missing node data, ensuring balanced retraining and preventing forgetting by initializing a GAN for each node and using it to generate synthetic data when disconnected, thus maintaining model accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If retraining is performed using all available data, then model generalization capability is improved, but storage requirements become impractical and data availability becomes constrained
Solution Approach 1:
The patent creates synthetic copies of historical training data using pre-trained GANs when original data is unavailable. The generator produces synthetic data samples that replicate the statistical properties and patterns of the original training data, enabling model retraining without requiring access to the actual historical data, thus resolving the contradiction between maintaining generalization capability and managing storage constraints
Solution Approach 2:
The patent pre-trains GANs on historical data before the retraining phase. This preliminary action captures and preserves the essential characteristics of the training data distribution in the form of trained generator models. When retraining is needed and original data is unavailable, these pre-trained GANs can generate synthetic data on-demand, eliminating the need to store large volumes of historical data while ensuring data availability for continuous model improvement
2Loss of information
If imputation techniques such as interpolation or probabilistic modeling are used, then data availability is improved, but data biases are introduced and manufacturing precision deteriorates
Solution Approach 1:
Instead of using traditional imputation methods that assume simple probability distributions (like Gaussian), the patent employs GANs to create accurate synthetic copies of the original data distribution. The generator learns the complex, non-linear relationships and patterns in the data during pre-training, then reproduces realistic data samples during retraining, maintaining data accuracy without introducing the biases inherent in simpler imputation techniques
Solution Approach 2:
The patent transitions from fixed probabilistic assumptions (e.g., Gaussian distribution) to a flexible, data-driven approach where the GAN's generator parameters are trained to match the actual data distribution. This parameter adaptation allows the system to capture complex data characteristics and generate accurate synthetic samples that reflect the true underlying distribution, avoiding the biases introduced by rigid probabilistic models
3Productivity
If model parameters converge to a stable configuration, then training efficiency is improved, but generalization capability deteriorates due to catastrophic forgetting
Solution Approach 1:
The patent establishes a continuous retraining mechanism that periodically updates the model using synthetic data generated from pre-trained GANs. This continuous action prevents the model from stagnating at suboptimal convergence points and forgetting previously learned patterns. By continuously introducing new synthetic training samples that reflect the original data distribution, the model maintains its generalization capability while still benefiting from efficient training between retraining cycles
Solution Approach 2:
The pre-training of GANs on historical data serves as a preliminary action that preserves the essential characteristics of the training data distribution. This preliminary preparation ensures that when the model needs retraining to prevent catastrophic forgetting, accurate synthetic data is readily available, allowing the model to recover and maintain generalization capabilities without requiring access to the original historical data
Data Source
AI summary
Machine learning model training using real and/or synthetic data is disclosed. Nodes contribute data to a central machine learning service. The data is used to train corresponding models whose generators, when trained, are configured to generate synthetic data according to a node's distribution. When a node is unavailable or for other reasons, the data contributed by the node for retraining a machine learning model includes at least some synthetic data from an enabled generator.