Decentralized Training Aggregating Disparate Feature Sets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing decentralized training methods for predictive models, such as federated learning, assume uniform data and feature sets across participating sites, which is often not the case in practice, leading to challenges in aggregating model updates and determining feature importance.
Innovation Solution
A decentralized training method that allows multiple clients to contribute model updates and feature weights, enabling the server to aggregate and distribute an aggregated model, while clients assess the model's feature importance locally and report back to the server for global weight calculation, allowing for disparate feature sets and improving model accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If federated learning is used to enable decentralized training, then data privacy is protected and data does not need to leave hospitals, but the method assumes uniform data and feature sets across participating sites which is not the case in practice
Solution Approach 1:
The patent allows each client site to use its own local feature set and data characteristics while contributing to a global model. Each site trains locally with its available features and sends model updates to the server, which aggregates them into a global model that adapts to heterogeneous feature sets across different sites.
Solution Approach 2:
The global model is designed to be universal and adaptable to multiple different feature sets. The model architecture and training process are configured to handle varying features from different clients, allowing the same model to function effectively across diverse local environments with different available features.
2Manufacturing precision
If a central data set is collected at a central location, then data uniformity and quality can be ensured, but privacy concerns make it increasingly harder to collect the needed data at a central location
Solution Approach 1:
Instead of collecting central data and distributing it to clients, the patent inverts the approach by having clients train locally with their own data and send model updates to the server. The server aggregates these updates to create a global model, thus achieving centralized model training without centralized data collection, preserving privacy while maintaining model quality.
Solution Approach 2:
The server acts as an intermediary that receives model updates from clients and aggregates them into a global model. This intermediary process allows the system to achieve the benefits of centralized training (model aggregation, quality control) without requiring centralized data collection, thus resolving the privacy-quality trade-off.
3Stability of the object's composition
If high central control is maintained in federated learning to ensure uniformity of data, then model consistency can be achieved, but such high level of control may not always be feasible
Solution Approach 1:
The system dynamically adapts to the capabilities and constraints of each client site. Rather than imposing rigid centralized control, the server adjusts the aggregation process based on the model updates received from clients with different feature sets, allowing flexibility in how each client contributes while maintaining overall model coherence.
Solution Approach 2:
Each client site independently trains its local model using its own data and features, performing self-service training without requiring centralized data or model distribution. The server aggregates these independently trained models, reducing the need for active central control while maintaining model consistency through the aggregation process.
Data Source
AI summary
Some embodiments are directed to training a model, e.g., a medical model. The training uses multiple model updates received from multiple client systems. At least some of the multiple client train on training sets that indicate values for different features. The model updates are aggregated in an aggregated model, for which feature weights are obtained. The feature weights provide information on the relative importance of the multiple features for the aggregated model's output.