Secure Model Averaging via Multi-Party Encryption
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current federated learning and blind learning systems face challenges in maintaining data privacy, as malicious servers can exploit clients' models during the averaging process, potentially extracting sensitive information from model weights.
Innovation Solution
The implementation of a secure multi-party averaging method that encrypts each client's model using the Diffie-Hellman key or similar protocols, ensuring the server cannot access the actual key and thus cannot exploit sensitive data, allowing clients to average models without revealing each other's weights.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the server averages client models directly, then the global model is updated efficiently, but the server can extract sensitive information from model weights
Solution Approach 1:
The patent introduces an intermediary encryption layer between the client models and the server. Each client encrypts their model weights using their private key before transmitting to the server. The server receives encrypted models, performs averaging on the encrypted data, and returns encrypted gradients. This intermediary encryption mechanism prevents the server from directly accessing and extracting sensitive information from model weights while still enabling efficient model averaging through the encrypted channel.
Solution Approach 2:
The patent changes the state of model weights from plaintext to encrypted form before they enter the averaging process. By transforming the data representation from unencrypted weights to encrypted ciphertext, the system maintains the mathematical structure needed for averaging while fundamentally altering the accessibility of the data. The server operates on encrypted parameters throughout, ensuring that even though averaging is performed efficiently, the sensitive information remains protected from extraction.
2Reliability
If clients share their models with the server for averaging, then the global model improves, but clients lose privacy protection
Solution Approach 1:
The encryption scheme acts as an intermediary that allows clients to share model information with the server for global model improvement while maintaining privacy. Each client encrypts their model weights with their own private key, creating a personal barrier that prevents the server from accessing the actual model parameters. The server can still perform averaging operations on the encrypted data and transmit encrypted gradients back, enabling collaborative model improvement without compromising client privacy.
Solution Approach 2:
The patent segments the model sharing process into encrypted components. Instead of sharing raw model weights directly, each client shares only the encrypted version of their weights. The encryption key remains with the client, segmenting the information access rights. This allows the server to receive and process model data for global model quality improvement while the client retains exclusive access to the decryption key, thus preserving privacy.
3Object-affected harmful factors
If the server accesses encrypted models, then the server cannot extract sensitive data, but the server cannot perform averaging operations
Solution Approach 1:
The encryption scheme serves as an intermediary that simultaneously provides protection and enables processing. By encrypting model weights with client-specific keys, the server cannot extract sensitive information. However, the server can still perform averaging operations on the encrypted data because the encryption preserves the mathematical structure. The server averages the encrypted models and returns encrypted gradients, maintaining processing capability while preventing information extraction.
Solution Approach 2:
The patent changes the parameter state from plaintext to encrypted form, which fundamentally alters both security and processing characteristics. The encryption transformation maintains the mathematical properties needed for averaging operations while adding a security layer that prevents information extraction. The server works with encrypted parameters throughout, and the encryption/decryption operations are performed at appropriate boundaries to enable both protection and processing.
Data Source
AI summary
A system and method are disclosed for providing an averaging of models for federated learning and blind learning systems. The method includes selecting, at a server, a generator g and a number p, transmitting, to at least two n client devices, the generator g and the number p, receiving, from each client device i of the at least two client devices, a respective value ki=gri mod p and transmitting the set of respective values ki to each client device i of the at least two client devices where respective added group of shares are generated on each client device i. The method includes receiving each respective added group of shares from each client device i of the at least two client devices and adding all the respective added group of shares to make a global sum of shares and dividing the global sum of shares by n.