Federated AI Model Updates With Sensitive Parameter Separation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In federated learning scenarios, data distribution heterogeneity and privacy concerns hinder the effective aggregation of models across clients, leading to suboptimal performance and increased communication costs, especially in industrial applications like visual quality control.
Innovation Solution
A method that identifies and separates sensitive model parameters from non-sensitive ones, allowing clients to update their models selectively with a global model while retaining privacy, by checking gradients and accuracy thresholds, and only sharing non-sensitive parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If federated learning aggregates models from multiple clients to improve overall model performance, then model accuracy is improved, but data privacy is compromised and communication costs increase
Solution Approach 1:
The patent segments model parameters into sensitive and non-sensitive categories based on their contribution to model accuracy. Only non-sensitive parameters are shared with the server for aggregation, while sensitive parameters remain local. This segmentation allows model improvement through federated learning without compromising data privacy, as the server never receives or processes sensitive client data.
Solution Approach 2:
The patent extracts and identifies sensitive model parameters that should not be shared, separating them from non-sensitive parameters. By taking out the sensitive parameters from the sharing process, the system maintains privacy while still enabling beneficial knowledge transfer through aggregation of non-sensitive parameters across clients.
2Measurement precision
If all model parameters are shared in federated learning to improve model performance, then model accuracy is improved, but communication costs increase
Solution Approach 1:
Instead of sharing all model parameters (excessive action), the patent applies partial action by sharing only the non-sensitive parameters that are necessary for model improvement. This selective sharing reduces communication bandwidth and energy consumption while still achieving the goal of improving model accuracy through federated learning.
3Stability of the object's composition
If a global model is trained for all clients to standardize performance, then model consistency is improved, but individual client performance deteriorates due to non-IID data distribution
Solution Approach 1:
The patent applies local quality by allowing each client to maintain and optimize their own model parameters locally while still benefiting from the global model. Clients can adapt the global model to their specific non-IID data distributions by keeping sensitive parameters local, thus achieving both model consistency through global aggregation and individual performance optimization through local adaptation.
Data Source
Figure 1~2
Figure 3
AI summary
A computer-implemented method for operating a technical device (TD) with an artificial intelligence-based model by a client (C1-C3) of a client-server system, wherein each client (C1-C3) stores a client model (CM1-CM3) for operating a connected technical device (TD1), and the following steps are performed: a) providing the global model (GM) based on federated learning to a client (C1), b) the client (C1-C3) checking whether a sensitive model parameter is stored in memory (MEM) that is not included in the global model (GM), and if so, aggregating the provided global model (GM) with the sensitive model parameter, and updating the client model (CM1) with the provided global model (GM), c) providing a reference dataset, d) calculating an initial accuracy of the client model (CM1) using the Reference data set,e) Determine the gradients of the model parameters of the client model (CM1) and determine a selected gradient of the model parameters of the client model (CM1), f) Remove the model parameter corresponding to the selected gradient from the client model (CM1), g) Calculate a second accuracy of the client model (CM1) using the reference dataset, h) Check if the second accuracy is lower than the first accuracy; if so, designate the at least one model parameter corresponding to the at least one selected gradient as the sensitive parameter and provide the client model (CM1) to the server (S), i) Update the global model, j) Operate the technical device (TD1) with the client model (CM1) and the sensitive parameter.