Federated Model Training Using Synthetic Data for Client Privacy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training machine learning models on client data exposes sensitive information to malicious actors, particularly in federated learning systems, as models are transmitted and trained on client devices and central servers, risking data exposure and exploitation.
Innovation Solution
A data protection system divides training across multiple simple models, each learning a subset of client data, and aggregates predictions using an ensemble model on synthetic data to create a new model for deployment, ensuring client data is not directly exposed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine learning models are trained on client data in federated learning systems, then model training accuracy is improved, but client data security deteriorates due to exposure risks during transmission
Solution Approach 1:
The patent divides the training process into multiple iterations where models are trained on different subsets of client data across multiple rounds. Each model learns only a portion of the complete dataset, preventing any single model from containing the full client data picture. This segmentation approach maintains training effectiveness while reducing the information exposure risk in each transmitted model.
Solution Approach 2:
The patent introduces a coordinate transformation mechanism that acts as an intermediary between client data and the trained models. By transforming the data representation and using differential privacy techniques, the system mediates the information flow so that models can be trained effectively without directly exposing raw client data during transmission.
2Object-affected harmful factors
If multiple simple models are trained on subsets of client data, then data security is improved by limiting model understanding, but system complexity increases due to ensemble aggregation requirements
Solution Approach 1:
The patent combines multiple simple models trained on different data subsets into an ensemble model that aggregates their predictions. This merging approach allows the system to benefit from the security advantages of limited model understanding while achieving accurate predictions through the collective intelligence of multiple models working together.
Solution Approach 2:
The patent creates a multi-functional system where the ensemble model serves multiple purposes: it maintains data security by using simple component models, achieves high prediction accuracy through aggregation, and provides robustness against adversarial attacks. The same ensemble structure handles both training and inference tasks across different client devices.
3Object-affected harmful factors
If synthetic data is used to train new models, then data protection is improved by eliminating direct client data exposure, but information loss occurs due to synthetic data generation limitations
Solution Approach 1:
The patent creates synthetic copies of client data that preserve the statistical properties and patterns of the original data without containing actual sensitive information. These synthetic data copies are used to train additional models, allowing the system to generate diverse model perspectives while maintaining data protection. The copying process uses differential privacy to ensure synthetic data reflects real data distributions without exposing individual records.
Data Source
AI summary
Methods and systems are described herein for protecting client data while training machine learning models. The system may transmit, to client devices, simple models to be trained on a respective client device to generate predictions based on a respective subset of respective client data of the respective client device. The system may receive the trained simple models from the client devices. The system may input, into an ensemble model including the simple models, an unlabeled synthetic dataset. This may cause the ensemble model to aggregate a set of predictions generated by each simple model to generate labels for the unlabeled synthetic dataset. The system may then input, into a new model, the unlabeled synthetic dataset and the labels to train the new model to predict the labels for the unlabeled synthetic dataset.