Federated Learning Updates Evaluated with Privacy-Preserving Synthetic Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In federated learning, evaluating local updates of machine learned models is challenging due to data privacy regulations, limited representative test data, and increased computational and data traffic, which can lead to overfitting and reduced turnaround times.
Innovation Solution
A method using a generative AI function to create synthetic representations of local data based on parameterizations, allowing evaluation of local updates at a model aggregator device without exposing the original data, thus maintaining data privacy and reducing data transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a fixed centrally-hosted test data set is used, then data privacy is maintained, but the test data becomes difficult to obtain, limited in number, and not representative of real-world scenarios
Solution Approach 1:
The patent generates synthetic test data that copies the statistical properties and characteristics of real-world data without using actual patient data. The synthetic data is created by training a generative model on real data distributions and then sampling from that model, providing unlimited representative test instances while maintaining privacy.
Solution Approach 2:
The patent transforms real data into synthetic data by changing the representation parameters - instead of using actual patient records, it uses generated samples that match the statistical parameters (distributions, correlations, patterns) of the original data, thus maintaining representativeness without data privacy concerns.
2Reliability
If decentralized evaluation is used where clients download models and upload results, then models are tested on real-world data, but computational resources are increased and data traffic is increased
Solution Approach 1:
The patent introduces synthetic data as an intermediary that allows centralized evaluation to achieve the same reliability as decentralized evaluation. The synthetic data acts as a mediator between the central server and real-world data, enabling accurate model testing without requiring clients to download models or upload results, thus reducing computational burden and turnaround time.
3Reliability
If decentralized evaluation is used, then test data changes over time reducing overfitting, but it becomes difficult to compare current models with historical models
Solution Approach 1:
The patent generates synthetic test data that consistently copies the statistical properties of real-world data across different time periods. This allows models to be tested on realistic data distributions that change over time (reducing overfitting) while maintaining consistent evaluation criteria that enable comparison across historical models.
4Measurement precision
If local updates are evaluated at the model aggregator device using real data, then evaluation is accurate, but data privacy regulations prevent data transfer
Solution Approach 1:
The patent creates synthetic copies of real patient data that preserve the statistical properties and evaluation characteristics needed for accurate model assessment, while containing no actual patient information. This allows the model aggregator device to evaluate local updates with the same accuracy as if using real data, while fully complying with data privacy regulations.
Data Source
AI summary
A computer-implemented method comprises: receiving, from a first local site remote from a model aggregator device, a local update of a machine learned model and a parameterization of local data, wherein the local update was generated at the first local site based on the local data; generating a synthetic representation of the local data based on the parameterization using a generative AI function; evaluating the local update using the synthetic representation to obtain an evaluation result indicative of the performance of the local update; and updating the machine learned model based on the evaluation result and the local update.