Vertical Federated Learning via Differential Privacy and Multitask Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Vertical federated learning faces challenges with low security and inefficiency, particularly in data privacy protection and communication overhead, due to the difficulty in sharing data with different features among participants.
Innovation Solution
An efficient, secure, and low-communication vertical federated learning method is introduced, combining horizontal federated learning with differential privacy and multitask learning, where participants select features, add noise, and align data to train models, reducing the need for extensive data sharing and ensuring privacy through the BlinkML method and labeled-unlabeled multitask learning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If vertical federated learning is used to train models with data from multiple participants having different features, then model completeness is improved, but communication overhead and security risks increase
Solution Approach 1:
The patent extracts only the necessary components for model training: data indexes and gradients, rather than transmitting complete feature data. Participants select features from their held data feature sets, extract relevant information, and transmit only the extracted components (indexes and gradients) to other participants, significantly reducing communication volume while maintaining model training effectiveness
Solution Approach 2:
The patent segments the feature data into discrete selectable features. Each participant has a held data feature set containing multiple features, and they selectively transmit specific features to other participants based on training needs. This segmentation allows precise control over what data is shared, reducing unnecessary communication while ensuring model completeness
2Measurement precision
If participants share feature data with different features, then model accuracy is improved, but data security and privacy protection deteriorate
Solution Approach 1:
The patent introduces data indexes as intermediaries between participants. Instead of directly sharing raw feature data, participants use indexes to reference and select features from their held data feature sets. This intermediary mechanism allows model training with diverse features while maintaining data security, as the actual feature values remain local and are only referenced through secure index-based selection
Solution Approach 2:
The patent implements partial data sharing where participants share only the specific features needed for training rather than all their data. Each participant selects features from their held data feature set that are relevant to the training task, sharing only this partial subset. This partial action approach maintains model accuracy by including necessary features while enhancing security by excluding unnecessary data
3Productivity
If all participants hold label data, then training effectiveness is improved, but the system becomes vulnerable when some participants lack labels
Solution Approach 1:
The patent makes the training system universal by designing it to handle multiple scenarios: all participants holding labels, some participants holding labels, or no participants holding labels. The held data feature set mechanism and feature selection process work uniformly across all these scenarios, allowing the system to adapt to different label availability conditions while maintaining training effectiveness through consistent feature-based learning
Data Source
AI summary
An efficient, secure and low-communication vertical federated learning method, includes: all participants select part of features of a held data feature set and a small number of samples of the selected features; the participants add noise satisfying differential privacy to part of samples of the selected features, and then send them to other participants together with data indexes of the selected samples; all participants take the received feature data as a label, take each missing feature as a learning task, and train each model with the feature data originally held in the same data index, respectively; all participants predict the data of the other samples with the trained model to complete the missing feature; the participants jointly train a model through horizontal federated learning. The present disclosure can protect data privacy and provide quantitative support for data privacy protection while efficiently training the model with horizontal federated learning.