Vertical Federated Learning Encoder Customization for Faster Convergence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Vertical Federated Learning (VFL) architectures face challenges in supporting diverse participants with different ML models and data features, leading to increased training costs and slow convergence due to overlapped data features, lack of encoder customization, and inefficient data and encoder selection.
Innovation Solution
The proposed solution involves an encoder selector to choose appropriate encoders and data sources, an encoder customizer to mute neurons or links for overlapping features, and a VFL configurator to set up data exchange and gradient backpropagation routes, enabling customized VFL configurations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional VFL architectures are used with multiple participants having different ML models and data features, then data privacy is maintained, but training costs increase and convergence speed decreases due to overlapped data features
Solution Approach 1:
The patent segments the VFL system into distinct functional components: encoder selectors that choose appropriate encoders for different participants, encoder customizers that modify encoder structures by muting neurons or links for overlapping features, and VFL configurators that set up data exchange and gradient backpropagation routes. This segmentation allows each component to optimize specific aspects of the training process, reducing redundant computations from overlapped features while maintaining data privacy through the federated learning framework.
2Adaptability or versatility
If participants use different ML model structures and hyper-parameters, then adaptability to diverse data types is improved, but system complexity increases
Solution Approach 1:
The patent implements a universal VFL architecture that can accommodate diverse ML models from different participants. The encoder selector and encoder customizer components provide a standardized interface that works with various encoder types and structures. The VFL configurator establishes a common framework for data exchange and gradient backpropagation that is model-agnostic, allowing participants to use different ML model structures and hyper-parameters while maintaining system coherence and reducing architectural complexity.
Data Source
AI summary
Methods and servers for executing Vertical Federating Learning (VFL) over a communication network are disclosed. Some methods include encoder and data selection, encoder customization, and VFL configuration. Encoder and data selection includes receiving a VFL request and analyzing it to determine data requirements and encoder requirements, selecting, based on the encoder requirements, a first encoder and a second encoder, and selecting, based on the data requirements, data sources for the first encoder and for the second encoder. Encoder customization includes selecting a first encoder and a second encoder, and customizing the first encoder by muting a portion of the first encoder. The VFL configuration includes receiving labelled data information and an indication of a loss function, configuring an evaluator using the labelled data information and the indication of the loss function, configuring encoder outputs to the joint classifier, and configuring a privacy router.