Asynchronous Federated Learning for Vertical Data Privacy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Federated learning systems face challenges in preserving privacy when data is exchanged among decentralized devices, particularly for vertically partitioned data, where existing algorithms are limited to synchronous computation and struggle with unbalanced computation/communication resources.
Innovation Solution
The development of asynchronous federated learning algorithms, such as AFSGD-VP, AFSVRG-VP, and AFSAGA-VP, which allow for parallel and independent training across active and passive computing devices using tree-structured communication schemes, ensuring privacy preservation and efficient computation by avoiding synchronization barriers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If synchronous federated learning algorithms are used, then privacy is preserved through coordinated computation, but computation efficiency deteriorates due to synchronization barriers and idle time
Solution Approach 1:
The patent transitions from static synchronous computation to dynamic asynchronous computation, where computing devices operate independently without waiting for others. Each device can process different instances at different times, dynamically adapting to available resources while maintaining privacy through secure multi-party computation protocols.
Solution Approach 2:
The asynchronous framework eliminates idle time by allowing all computing devices to continuously perform useful computation. Instead of waiting for synchronization points, devices can process instances whenever ready, ensuring continuous utilization of computational resources while preserving privacy through cryptographic protocols.
2Stability of the object's composition
If synchronous computation is used across distributed devices, then coordination is achieved, but resource utilization deteriorates due to unbalanced computation/communication resources
Solution Approach 1:
The system adopts dynamic asynchronous operation where each device operates independently based on its own pace and resource availability. This eliminates the rigid synchronization constraints that cause resource imbalance, allowing fast devices to process more instances and slow devices to catch up without holding back the entire system.
Solution Approach 2:
The computation is segmented into independent instance-level tasks that can be processed individually by different devices at different times. This segmentation allows each device to work on its own subset of instances without requiring coordination for every computation step, improving overall resource utilization.
3Productivity
If data is exchanged frequently among decentralized devices, then federated learning is achieved, but privacy preservation deteriorates
Solution Approach 1:
The patent introduces cryptographic protocols as intermediaries that enable computation across devices without direct data exchange. Secure multi-party computation and homomorphic encryption act as mediators, allowing devices to contribute to the federated model while their raw data never leaves their local devices, thus preserving privacy.
Solution Approach 2:
Instead of exchanging original data, the system exchanges cryptographic copies or transformations of data that preserve utility for learning but reveal no sensitive information. Encrypted gradients and model updates are transmitted rather than raw data, maintaining privacy while enabling federated learning.
Data Source
AI summary
System and method for training a federated learning model asynchronously. The system includes a coordinator, an active computing device and a passive computing device in communication with each other. The active computing device has a processor and a storage device storing computer executable code. The computer executable code is configured to: train the federated learning model in the active computing device using dimensions of an instance in the active computing device; and instruct the at least one passive computing device to train the federated learning model in the at least one passive computing device using dimensions of the instance in the at least one passive computing device. The training instances in the active and the at least one passive computing devices do not correspond to each other at the same training time.