Federated Model Pruning via Serial Agent Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Federated machine learning faces challenges in reducing communication overhead and local computation complexity due to large model sizes and geo-distributed devices, with existing pruning methods either requiring central data sharing or not addressing resource limitations effectively.
Innovation Solution
The method involves incremental and decentralized pruning of machine learning models, where a server initiates a serial sequence of model pruning among agents, allowing each agent to prune and refine the model locally without sharing data, and determines further pruning needs, maintaining model accuracy and security.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large deep neural networks are trained directly on agents, then model accuracy is improved, but resource limitations and communication overhead become infeasible
Solution Approach 1:
The patent segments the model pruning process into multiple stages and distributes it across different agents. Instead of requiring all agents to handle the full large model, the pruning process is divided into phases where agents work on progressively smaller model versions, reducing individual resource requirements while maintaining overall model accuracy.
Solution Approach 2:
The server performs preliminary model pruning before distributing models to agents. By pre-pruning the model on the server using aggregated data from multiple agents, the model size is reduced before it reaches individual agents, making it feasible for agents with limited resources to train and use the model effectively.
2Device complexity
If model pruning is performed centrally at the server, then communication overhead is reduced, but data privacy and decentralization are compromised
Solution Approach 1:
The patent implements local model pruning at each agent using locally available data. Each agent performs pruning operations on its own device using its local dataset, ensuring that data remains decentralized and private. This local quality approach maintains data privacy while still achieving model optimization without requiring central data aggregation.
3Productivity
If all agents participate in model pruning simultaneously, then pruning efficiency is improved, but coordination complexity and communication requirements increase
Solution Approach 1:
The patent segments agent participation into sequential rounds rather than simultaneous processing. Agents participate in pruning in successive rounds where each round builds upon the previous one, reducing coordination complexity. This sequential approach allows efficient pruning while simplifying communication protocols compared to requiring all agents to coordinate simultaneously.
Data Source
AI summary
A computer-implemented method, a computer program product, and a computer system for incremental and decentralized pruning of a machine learning model in federated learning. A federated learning system determines a serial sequence of participating in model pruning by agents in the federated learning system. A server in the federated learning system sends, to a first agent in the serial sequence, an initial model to trigger a federated pruning process for the machine learning model. The each of agents in the serial sequence prunes the machine learning model. The each of agents in the serial sequence generates an intermediately pruned model for an immediately next agent to prune. A final agent in the serial sequence prunes the machine learning model and generates a finally pruned model. The final agent sends the finally pruned model to the server.