Federated Learning Parameter Update via Delta Transmission
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In federated learning, the large size of machine learning models leads to significant communication overhead as they and their parameters are transmitted between servers and clients, compromising data confidentiality and efficiency.
Innovation Solution
A method where a first information processing apparatus transmits parameter generation information to multiple second information processing apparatuses, which evaluate the machine learning model and send back evaluation values, aggregated by the first apparatus, to update the model, reducing data flow by using small-sized strings for parameter generation, evaluation, and aggregate results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the machine learning model and its parameters are transmitted between server and clients for federated learning, then the learning can be performed in a distributed manner protecting data confidentiality, but the communication amount increases significantly due to the large model size
Solution Approach 1:
The patent extracts only the essential parameter information needed for model updates from the complete machine learning model. Instead of transmitting the entire model structure and all parameters, the system identifies and transmits only the differential parameter values that need to be updated, significantly reducing the communication payload while maintaining the integrity of the federated learning process
Solution Approach 2:
The patent applies parameter changes by transmitting parameter difference values (deltas) rather than absolute parameter values. The server calculates the difference between old and new parameter values after local training, and only these differential values are communicated to clients. This approach reduces communication overhead while ensuring that the model evolves correctly through accumulated parameter changes
2Measurement precision
If the complete machine learning model is transmitted to each client for local training, then accurate model updates can be achieved, but the communication overhead and data flow between server and clients increase
Solution Approach 1:
The patent uses copying by creating and transmitting only the necessary parameter update information rather than replicating the entire model. Each client receives a copy of the updated parameters or parameter differences from the server, which are then integrated into their local model copies. This selective copying approach maintains model update accuracy while minimizing the amount of data that needs to be transmitted and stored
Data Source
AI summary
A first information processing apparatus transmits first information used for acquiring values of parameters of a machine learning model to a plurality of second information processing apparatuses. The plurality of the second information processing apparatuses acquire an evaluation value of the machine learning model when the value of the parameters acquired based on the first information is applied to the machine learning model, and transmit the evaluation value to the first information processing apparatus. The first information processing apparatus aggregates a plurality of evaluation values received from the plurality of the second information processing apparatuses, and transmits the aggregate result of the evaluation values to the plurality of the second information processing apparatuses. The first information processing apparatus and the plurality of the second information processing apparatuses update the machine learning model based on the aggregate result of the evaluation values.