Multi-Task Federated Learning Clustering Nodes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional federated learning in transmission networks with node heterogeneity faces challenges of decreased model precision and overlong training time consumption, with existing solutions being less adaptive and unable to address both issues simultaneously.
Innovation Solution
A multi-task learning method based on federated learning that involves determining participating nodes, clustering them using the K-Means algorithm, calculating cluster models, identifying cluster key feature sets with the SHAP framework, and training global models to alleviate issues of data and device heterogeneity through feature masking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional federated learning is applied in transmission networks with node heterogeneity, then model training can be performed across distributed nodes, but model precision decreases and training time consumption increases
Solution Approach 1:
The patent segments the heterogeneous participating nodes into multiple clusters based on their device performance and data distribution characteristics. Each cluster is trained independently with tailored hyperparameters, avoiding the time waste of waiting for slow nodes while maintaining model precision through specialized training strategies for each cluster.
Solution Approach 2:
The patent dynamically adjusts training hyperparameters for each cluster based on their specific characteristics (device performance, data distribution). This dynamic adaptation allows fast clusters to train more aggressively while slow clusters receive more time and resources, resolving the contradiction between precision and training time.
2Adaptability or versatility
If conventional federated learning handles node heterogeneity, then distributed training can proceed, but the solution is less adaptive and can solve only a single problem
Solution Approach 1:
The patent creates a multi-functional training framework that can simultaneously address multiple issues: it handles device heterogeneity through clustering, data heterogeneity through adaptive hyperparameter tuning, and optimizes both precision and training time. This universal approach adapts to different cluster characteristics and solves multiple problems concurrently.
Solution Approach 2:
The patent changes training parameters (hyperparameters) dynamically for each cluster based on their characteristics. Fast clusters may use larger batch sizes, more epochs, or different learning rates, while slow clusters receive adjusted parameters. This parameter adaptation enables the system to be versatile in handling different node types while maintaining high model precision.
Data Source
AI summary
A multi-task learning method comprises: determining participating nodes; performing clustering to the participating nodes and determining several clusters; determining a global model for the clusters according to the several clusters and by means of the federated learning within the cluster; determining a cluster key feature set of any one of the clusters according to the cluster model and by means of calculation with the SHAP framework; determining a global model according to the cluster key feature set; and training the global model according to the any one of the clusters, and determining the cluster model of the any one of the clusters, wherein a plurality of the clusters are used for achieving multi-task learning. In the present application, by clustering, the participating nodes having similar device performance and data distribution are collected into the same group, and the participating nodes within the same cluster are trained together.