Personalized Federated Learning for IIoT Weight Divergence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing federated learning methods in industrial IoT face challenges in adapting to diverse client needs and addressing severe data distribution disparities, leading to weight divergence and suboptimal model performance.
Innovation Solution
A personalized federated learning method is introduced, decoupling the model into a shared and personalized layer, using an orthogonality constraint loss to train the shared layer, and a local class prototype vector to update the personalized layer, ensuring adaptability and accuracy across clients.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a unified global model is used in federated learning, then data privacy is protected, but the model cannot adapt to diverse personalized needs of different clients
Solution Approach 1:
The model is segmented into a shared layer and a personalized layer. The shared layer handles common features learned from aggregated data, while the personalized layer handles client-specific adaptations. This segmentation allows the model to protect privacy through federated learning while adapting to individual client needs through the personalized layer that learns from local data.
Solution Approach 2:
Different parts of the model have different functions: the shared layer uses globally aggregated knowledge for general patterns, while the personalized layer uses local client data for specific adaptations. This local quality approach enables each part of the model to optimize for its specific purpose, balancing privacy protection with personalized adaptability.
2Quantity of substance
If federated learning aggregates models from multiple clients, then data utilization is improved, but severe data distribution disparities cause weight divergence
Solution Approach 1:
By segmenting the model into shared and personalized layers, the aggregation process only combines shared layer weights from multiple clients. This segmentation prevents data distribution disparities from causing complete model divergence, as the personalized layer remains local to each client while the shared layer benefits from aggregated knowledge.
Solution Approach 2:
The personalized layer is extracted from the global aggregation process and kept local to each client. This extraction allows federated learning to aggregate useful shared patterns while isolating the personalized adaptations that would otherwise cause weight divergence due to data distribution disparities.
Data Source
AI summary
A personalized federated learning method for industrial Internet of Things targeting client needs is provided. The method includes: issuing, by a server, an initial model to each client as a local model, the local model including a shared layer and a personalized layer; for each client, freezing parameters of the personalized layer and locally training the shared layer based on local industrial data of the client using an orthogonality constraint loss; uploading the trained parameter of the shared layer of each client to the server for averaging and aggregation; sending the aggregated parameter to each client; updating the parameter of the shared layer of each client based on the aggregated parameter; repeating the training and parameter updating process of the shared layer until a count of iterations equals to a count of policy switching communications; training the shared layer and the personalized layer to obtain a trained local model of the client.