Personalized Federated Learning via Partial Parameter Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional federated learning methods face inefficiencies in communication due to large data transfer and struggle to optimize local model performance when local data characteristics differ, necessitating a method for personalized model adaptation.
Innovation Solution
A method for personalized federated learning that shares only specific parameters of an artificial neural network model, where the electronic device trains a local model using both shared and unshared parameters, with the external device updating the shared parameters based on received global parameter sets, optimizing communication efficiency and model personalization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If all model parameters are shared in federated learning, then communication completeness is improved, but communication efficiency deteriorates due to large data transfer
Solution Approach 1:
The patent segments the model parameters into two distinct sets: global parameters (first parameter set) that are shared across all devices, and local parameters (second parameter set) that remain private to each device. This segmentation allows selective communication of only the global parameters, reducing communication overhead while maintaining model collaboration.
Solution Approach 2:
The patent extracts and isolates the global parameters from the complete model parameter set for communication purposes. By identifying and separating the parameters that need to be shared (global parameters) from those that should remain local, the system achieves efficient communication without sacrificing model effectiveness.
2Stability of the object's composition
If a global model is created as an average of local models, then federated learning convergence is improved, but local model performance deteriorates when local data characteristics differ
Solution Approach 1:
The patent applies local quality by allowing different parts of the model to have different update strategies. Global parameters are updated through federated averaging for convergence, while local parameters are updated independently based on local data characteristics, enabling each device to adapt to its specific data distribution.
Solution Approach 2:
The patent introduces dynamics by making the parameter update process flexible and adaptive. The system dynamically determines which parameters to share and which to keep local, allowing the model structure to adapt to varying local data characteristics while maintaining overall convergence through coordinated global parameter updates.
3Adaptability or versatility
If more parameters are shared for model personalization, then local model performance is improved, but communication overhead increases
Solution Approach 1:
The patent applies partial action by sharing only the necessary global parameters rather than all parameters. This selective sharing provides sufficient personalization capability through the global parameters while avoiding the communication overhead of transmitting the entire parameter set, achieving an optimal balance between personalization and efficiency.
Data Source
AI summary
Provided is a method of personalized federated learning performed by an electronic device. The method is performed by an electronic device including one or more processors, a communication circuit which communicates with an external device, and one or more memories storing at least one instruction executed by the one or more processors. The method may include, by the one or more processors, training a local model using local data, in which the local model as an artificial neural network model includes a first parameter set corresponding to a global parameter set and a second parameter set corresponding to a local parameter set, transmitting the first parameter set to the external device, receiving a 1-1st parameter set for renewing the first parameter set from the external device, changing the first parameter set included in the local model to the 1-1st parameter set, and training the local model including the 1-1st parameter set.