Federated AI Model Operation With Selective Weight Aggregation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge in federated learning with large AI models is high data transmission costs, network bandwidth limitations, and increased communication latency due to the considerable size of model parameters, along with unclear data protection and ownership issues.
Innovation Solution
A modified federated learning algorithm that calculates importance metrics for model weights, transmitting only significant weights within a predefined range to a central server for aggregation, reducing communication overhead while maintaining model performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If all model parameters are transmitted in federated learning, then model accuracy is maintained, but communication costs and data transmission volume increase significantly
Solution Approach 1:
The patent extracts and transmits only the important model parameters that contribute significantly to model performance, rather than transmitting all parameters. This is achieved by calculating importance metrics for each parameter and selectively transmitting only those above a threshold, thereby reducing communication overhead while maintaining model accuracy.
Solution Approach 2:
The patent applies different treatment to different model parameters based on their individual importance metrics. Parameters are categorized into important and unimportant groups, with only important parameters being transmitted to the server for aggregation, while unimportant parameters are discarded locally, optimizing the quality of transmitted data.
2Measurement precision
If all model parameters are transmitted in federated learning, then model performance is maintained, but communication latency increases
Solution Approach 1:
The patent extracts only the essential model parameters for transmission, reducing the total data volume that needs to be communicated between clients and server. This extraction based on importance metrics directly reduces communication latency while preserving model performance.
3Quantity of substance
If important model parameters are selectively transmitted, then communication costs are reduced, but model accuracy may deteriorate
Solution Approach 1:
The patent changes the parameter transmission strategy from transmitting all parameters to transmitting only important parameters identified through importance metrics. This parameter selection approach reduces communication overhead while maintaining model accuracy by focusing on parameters that have the greatest impact on performance.
Solution Approach 2:
The patent implements a feedback mechanism where the server aggregates important parameters from multiple clients and sends updated global model parameters back to clients. This iterative feedback process ensures that model accuracy is maintained across federated learning rounds despite selective parameter transmission.
4Reliability
If large amounts of model parameters are communicated, then comprehensive model updates are achieved, but network bandwidth is exceeded
Solution Approach 1:
The patent extracts and transmits only the important model parameters that are essential for effective model updates, rather than transmitting all parameters. This extraction approach ensures reliable model updates while operating within network bandwidth constraints by significantly reducing the volume of transmitted data.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A computer-implemented method for operating a technical device through a system, comprising the following steps: a) providing and initializing an initial model based on artificial intelligence and storing it as a global model; b) transferring the global model to at least one client and storing it as a respective local model; c) training the respective local model with training data provided by the respective client and calculating initial gradients over the behavior of the model weights of the respective local model during training; d) calculating the respective importance metrics of the local model based on the initial gradients provided by the respective client; e) transmitting the model parameters of the respective local model, along with the respective importance metrics, which lie within a predefined range of values, from the respective client to the server.f) Aggregate the previously received model parameters of the respective clients with the global model to form an aggregated model, and remove those model parameters in the aggregated model for which no respective importance metrics are available; g) Store the aggregated model as a global model; h) Calculate second gradients over the behavior of model weights of the aggregated model, and check whether the second gradient has met a predefined convergence criterion; if not, continue with step b); otherwise, continue with step i); i) Operate the technical device with the local model.