Federated Learning Model Parameter Extraction for Privacy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data privacy concerns limit the availability and quality of data for training and updating machine learning models, as traditional centralized approaches require sharing sensitive information, while decentralized methods struggle to maintain model accuracy without direct access to private data.
Innovation Solution
A federated learning system that generates and updates master feature models and worker models based on private data inputs from entities, allowing for the creation of a federated model without accessing private data, using a dynamic virtual network to combine models from multiple entities and adjust weights based on accuracy metrics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional centralized machine learning techniques are used where all local datasets are uploaded to one server, then model training can be performed with complete data, but data privacy concerns arise and data security is compromised
Solution Approach 1:
The patent extracts only the essential model parameters (weights and biases) from the complete local datasets, transmitting only these extracted elements to the central server for aggregation. This extraction approach maintains data privacy by leaving sensitive information localized while still enabling collaborative model training through the transferred parameter summaries.
Solution Approach 2:
The patent introduces model parameters (weights and biases) as intermediaries between local datasets and the central server. Instead of directly sharing raw data, each entity processes its local data through its local model to generate parameter summaries, which then serve as the intermediary communication medium that preserves privacy while enabling centralized aggregation and global model improvement.
2Reliability
If federated learning is used where summaries of individual data are exchanged, then data privacy is partially protected, but the summaries may still reveal significant private or sensitive information
Solution Approach 1:
The patent extracts only the mathematical parameters (weights and biases) that define the model's learned patterns, deliberately excluding any direct representations of sensitive input data. This extraction ensures that even though information is shared for model improvement, the fundamental private data remains inaccessible and unrecoverable from the transmitted parameters.
Solution Approach 2:
The patent creates simplified copies of local models in the form of parameter summaries (weights and biases) that capture the essence of local data patterns without containing the actual sensitive information. These parameter copies enable the central server to aggregate knowledge from multiple sources while the original sensitive data remains localized and protected.
3Reliability
If decentralized federated learning is implemented without intermediate models, then data privacy is maintained, but model accuracy may be compromised
Solution Approach 1:
The patent implements a feedback mechanism where the central server aggregates parameter summaries from multiple entities, computes a global model update, and sends this aggregated feedback back to all participating entities. This feedback loop enables continuous improvement of local models based on collective knowledge from the federation, maintaining high accuracy without requiring centralized access to sensitive data.
Solution Approach 2:
The patent merges parameter summaries (weights and biases) from multiple local models into a single aggregated global model at the central server. This merging process combines the knowledge and patterns learned by each entity from its local data, creating a more accurate and robust global model that benefits from diverse data sources while preserving the decentralized and private nature of the training process.
Data Source
AI summary
The invention provides a federated model based on locally trained machine learning models. In embodiments, a method includes: monitoring, by a computing device, cached data of an entity in a networked group of entities for changes in data, wherein the cached data includes model output data from worker models and a master feature model of the entity, and wherein the worker models and the master model comprise machine learning models; iteratively updating, by the computing device, parameter weights of the worker models and the master feature model based on the monitoring, thereby generating updated worker models and an updated master feature model; and providing, by the computing device, the updated worker models and an updated master feature model to a remote federated server for use in a federated model incorporating the updated worker models and an updated master feature model of the entity with other updated master feature models and other updated worker models of other entities in the networked group of entities.