Federated Learning Secure Aggregation via Mask Partitioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Federated learning techniques face challenges in ensuring the privacy of device information and efficiency due to computational bottlenecks and dropout resiliency, particularly in model inversion attacks and the need for costly mask-reconstruction operations.
Innovation Solution
The system employs a method for secure aggregation that partitions and encodes local model shares, allowing for one-shot aggregate-mask reconstruction at the server, reducing computational overhead and eliminating the need for pairwise random-seed reconstruction, while maintaining strong privacy and dropout-resiliency guarantees.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If secure aggregation protocols are used to protect device privacy in federated learning, then privacy protection is improved, but computational overhead increases significantly
Solution Approach 1:
The patent segments the mask reconstruction process into two parts: (1) pre-computed aggregation vectors that are stored and reused, and (2) simple vector additions performed during aggregation. This segmentation eliminates the need for costly pairwise random-seed reconstruction operations, reducing computational overhead from O(N^2) to O(N) while maintaining privacy protection through the segmented mask structure.
2Reliability
If mask-reconstruction operations are performed to ensure secure aggregation, then privacy guarantees are improved, but computational cost increases
Solution Approach 1:
The patent performs preliminary computation by pre-calculating aggregation vectors before the federated learning aggregation process. These pre-computed vectors are stored and reused during actual model aggregation, eliminating the need for expensive mask-reconstruction operations at aggregation time. This preliminary action reduces computational cost while maintaining the same privacy guarantees.
3Reliability
If traditional secure aggregation protocols are used, then device privacy is protected, but training time increases due to computational bottlenecks
Solution Approach 1:
The patent extracts the computationally intensive mask-reconstruction operations from the real-time aggregation process and replaces them with pre-computed aggregation vectors. By taking out the expensive operations and replacing them with simple vector additions, the system maintains device privacy protection while reducing training time by eliminating computational bottlenecks.
4Reliability
If pairwise random-seed reconstruction is performed, then dropout resiliency is improved, but computational overhead increases
Solution Approach 1:
The patent merges the dropout resiliency mechanism with the pre-computed aggregation vectors. Instead of performing separate pairwise random-seed reconstruction operations for each device combination, the system combines all device contributions into single aggregation vectors that are pre-computed and stored. This merging maintains dropout resiliency by allowing the server to reconstruct masks for any subset of devices while reducing computational overhead from O(N^2) to O(N).
Data Source
AI summary
Systems and methods to generate a model based on a subset of models generated at remote devices include a first device operatively coupled with a second device. The first device can generate, based on a model parameter and data restricted to the first device, a first model via machine learning, partition the first model into a plurality of local mask shares each including a distinct portion of the first model, encode one or more of the plurality of local mask shares into a corresponding first plurality of encoded shares, and generate an aggregation of encoded shares including a first encoded share having a first index among the first plurality of encoded shares and a second encoded share having the first index among a second plurality of encoded shares. The second encoded share includes a distinct portion of a second model generated by a second device via machine learning.