Federated Learning Auxiliary Model for Sample Enhancement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In federated learning systems, the lack of sufficient labeled modeling samples can lead to overfitting of models, and existing methods do not effectively address the need for sample enhancement while ensuring data security and compliance.
Innovation Solution
A method where a server side training sample is obtained from candidate samples based on client side training samples, using a distance detection model to predict sample distances and obtain similar training samples, which are then used to train an auxiliary model. This model is used to provide intermediate data for the first participation device to train its own model, enabling joint prediction and reducing overfitting through sample enhancement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If federated learning systems use only local client side training samples for model training, then data security and compliance are maintained, but the model suffers from insufficient sample quantity leading to overfitting
Solution Approach 1:
The patent introduces an auxiliary model as an intermediary component that bridges the gap between data security requirements and sample quantity needs. The auxiliary model is trained on server side training samples obtained through privacy-preserving techniques (such as secure multi-party computation or differential privacy), then generates synthetic training data that can be used to enhance the main model without compromising the security of original data. This intermediary mechanism allows the system to benefit from additional training samples while maintaining data isolation and compliance.
Solution Approach 2:
The patent employs copying by creating synthetic training samples through the auxiliary model rather than directly sharing or accessing original client data. The auxiliary model generates copied representations of training data that capture the statistical properties and patterns needed for effective model training, while the original sensitive data remains isolated on each client device. This copying approach enables sample enhancement without violating data security constraints.
2Quantity of substance
If federated learning systems access and share data across multiple participation devices, then sample quantity increases and overfitting is reduced, but data security and compliance are compromised
Solution Approach 1:
The auxiliary model serves as a mediator that enables indirect access to training samples without direct data sharing. Instead of clients accessing each other's raw data, the auxiliary model processes server side training samples and generates synthetic data that is then used for training the main model. This intermediary process ensures that no client can directly access or infer information from other clients' data, maintaining compliance while achieving sample enhancement.
Solution Approach 2:
The patent replaces the mechanical approach of direct data sharing and aggregation with a computational approach using privacy-preserving technologies. Instead of physically or logically combining datasets from multiple devices, the system uses cryptographic methods, secure computation protocols, or differential privacy mechanisms to enable the auxiliary model to learn from distributed data without exposing the actual data. This substitution maintains security while achieving the benefits of multi-source training.
3Productivity
If the system trains models using only available local samples, then training process is simple and fast, but model generalization capability is poor due to overfitting
Solution Approach 1:
The patent applies preliminary action by pre-training the auxiliary model on server side training samples before using it to generate enhanced training data for the main model. This preliminary training phase prepares the auxiliary model to effectively synthesize realistic training samples that capture important patterns and variations. By performing this preparation work in advance, the system enables the main model training to proceed efficiently with high-quality synthetic samples, improving generalization without significantly increasing the main training timeline.
Data Source
AI summary
A data processing method including obtaining a server side training sample corresponding to a client side training sample of a first participation device from one or more candidate samples of a second participation device, calling a distance detection model to predict one or more sample distances each between one candidate sample and the server side training sample, obtaining a similar training sample from the one or more candidate samples based on the one or more sample distances, training an auxiliary model of the second participation device using the server side training sample and the similar training sample, to obtain a trained auxiliary model, and transmitting intermediate data generated during training of the auxiliary model to the first participation device, to enable the first participation device to train a model of the first participation device based on the client side training sample and the intermediate data, to obtain a trained model.