Federated Learning System with Preliminary Local Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In federated learning, setting appropriate training conditions for machine learning models across multiple devices is challenging, especially as the number of devices increases, due to the need for balancing parameters like learning rate, regularization strength, and optimizer settings.
Innovation Solution
A learning system that determines federated local training conditions on local devices and global training conditions on a server using preliminary local training information, allowing for optimal model updates and parameter aggregation, while considering data distribution and task difficulty levels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the number of local devices increases in federated learning, then the computational load distribution and privacy confidentiality are improved, but the difficulty of setting appropriate training conditions (balancing parameters) increases
Solution Approach 1:
The patent applies preliminary action by conducting preliminary local training before federated learning to obtain preliminary local training information. This preliminary training enables the determination of appropriate federated local training conditions and global training conditions without requiring complex manual configuration, thus resolving the contradiction between scalability and training condition setting complexity
Solution Approach 2:
The patent implements feedback by using the preliminary local training information (which includes training results from local devices) to determine the training conditions for federated learning. This feedback loop allows the system to automatically adjust training parameters based on actual local device performance, eliminating the need for complex manual parameter tuning as the number of devices increases
2Power
If the number of local devices increases in federated learning, then the computational load is distributed, but the difficulty of setting appropriate training conditions increases
Solution Approach 1:
The patent performs preliminary local training before federated learning to gather training information from local devices. This preliminary action enables the system to automatically determine appropriate training conditions for all devices, thereby maintaining ease of operation while distributing computational load across multiple devices
Solution Approach 2:
The system performs self-service by automatically determining training conditions based on preliminary local training information generated by the local devices themselves. This eliminates the need for external manual configuration and simplifies operation as the number of devices increases, while still distributing computational tasks effectively
Data Source
AI summary
According to one embodiment, a learning system includes a plurality of local devices and a server. The plurality of local devices each includes processing circuitry configured to determine a federated local training condition indicating a training condition in federated learning of a local model based on preliminary local training information including a preliminary local training condition and a preliminary local training result in a case where a model is preliminarily trained using local data. The server includes processing circuitry configured to determine a global training condition of a global model based on the preliminary local training information.