Federated Model Partitioning for Confidential Gradient Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Federated learning systems face challenges in ensuring the confidentiality of the machine learning model due to the potential corruption or untrustworthiness of distributed trainers, as the model is disclosed to them.
Innovation Solution
Implementing a data-processing model with hidden and open parts, where the hidden part's parameters are inaccessible to the trainers, and using trust scores to determine the extent of model disclosure, along with encryption and hierarchical configurations to maintain confidentiality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the machine learning model is disclosed to distributed trainers for federated learning, then the trainers can compute training gradients and contribute to model improvement, but the confidentiality of the machine learning model is compromised due to potential corruption or untrustworthiness of trainers
Solution Approach 1:
The machine learning model is divided into two distinct parts: a hidden part containing sensitive parameters that remain confidential, and an open part that is disclosed to distributed trainers for computation. This segmentation allows trainers to compute gradients on the open part without accessing the hidden part, thus maintaining model confidentiality while enabling training contribution.
Solution Approach 2:
Different parts of the model have different accessibility qualities. The hidden part has restricted access (parameters not accessible to trainers), while the open part has full access (parameters accessible to trainers). This local differentiation of quality allows the system to balance confidentiality requirements with training effectiveness.
2Reliability
If the hidden part of the model contains all sensitive parameters, then model confidentiality is maximized, but the trainers cannot effectively contribute to training the model
Solution Approach 1:
The model is segmented into hidden and open parts, allowing trainers to access and compute on the open part while the hidden part remains protected. This enables training effectiveness through gradient computation on the open part while maintaining confidentiality of the hidden part.
Solution Approach 2:
Instead of disclosing the entire model (excessive action), only the necessary open part is disclosed to trainers (partial action). This partial disclosure provides sufficient information for effective training contribution while avoiding the risk of full model exposure.
3Productivity
If the open part of the model is made larger, then trainers have more access for effective training, but the confidentiality of the model is reduced
Solution Approach 1:
The model exhibits local quality differentiation where the open part has high accessibility for training purposes while the hidden part maintains restricted accessibility for security. This allows optimization of training contribution capability in the open part without compromising the confidentiality of the hidden part.
Data Source
AI summary
An apparatus for federated training, the apparatus comprising means for:Transmitting a first implementation (22) of a data-processing model to a first distributed trainer, wherein the first implementation of the data-processing model comprises a first hidden part (221) and a first open part (222),Transmitting a second implementation (23) of the data-processing model to a second distributed trainer, wherein the second implementation of the data-processing model comprises a second hidden part (231) and a second open part (232),Receiving a first training gradient from the first distributed trainer and a second training gradient from the second distributed trainer, wherein the first gradient relates to the first open part of the first implementation of the data-processing model, wherein the second gradient relates to the second open part of the second implementation of the data-processing model,Updating the data-processing model using the first gradient and the second gradient.