Privacy-Preserving Ensemble Learning via Missing Feature Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Entities face challenges in sharing private data for machine learning (ML) model collaboration due to privacy restrictions and intellectual property concerns, leading to suboptimal performance in domains like cyber security, where models developed by different companies often fail to detect threats effectively in real-time.
Innovation Solution
A privacy-preserving collaborative ensemble learning framework that allows entities to share trained ML models, important feature lists, and missing feature generators, enabling a centralized model aggregator to produce a single prediction result while protecting intellectual property, using techniques like Generative Adversarial Networks (GANs) to generate missing features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If entities share raw data for collaborative ML model training, then model performance and accuracy improve, but privacy and intellectual property are compromised
Solution Approach 1:
The patent extracts only the essential components needed for collaboration (trained models, feature importance lists, missing feature generators) while leaving the sensitive raw data behind at each entity. This allows the model aggregator to combine model outputs for improved accuracy without ever accessing or sharing the private raw datasets, thus resolving the contradiction between accuracy improvement and privacy protection.
Solution Approach 2:
The model aggregator acts as an intermediary that receives processed model components from multiple entities, combines them through ensemble methods, and returns predictions without handling any raw data. This intermediary architecture enables collaborative accuracy improvement while maintaining a clear boundary that protects each entity's private data, eliminating the need to share sensitive information.
2Reliability
If encryption methods like homomorphic encryption are used to protect data during sharing, then privacy is preserved, but computational and communication overhead increases significantly
Solution Approach 1:
The patent performs data processing and model training actions in advance at each entity's local system, producing pre-trained models and feature extraction components before any sharing occurs. This preliminary action eliminates the need for real-time encryption during data transmission and processing, as the sensitive raw data never leaves the local system, thus maintaining privacy without the computational overhead of continuous encryption operations.
Solution Approach 2:
Instead of sharing and processing encrypted copies of raw data, the patent shares lightweight model components (trained model parameters, feature importance lists) that are copies of the essential information needed for collaboration. These copies can be transmitted and processed efficiently without requiring heavy encryption mechanisms, as they contain only aggregated statistical information rather than sensitive raw data.
3Measurement precision
If multiple ML models are executed for collaborative predictions, then prediction accuracy improves, but computational resources and time consumption increase
Solution Approach 1:
The patent implements a feature importance filtering mechanism that identifies and executes only the most relevant models for each prediction request, rather than running all available models. By using feature importance lists to determine which models are most applicable to the current input data, the system achieves near-optimal accuracy with significantly reduced computational overhead, balancing precision and productivity.
Solution Approach 2:
The patent segments the prediction process into distinct stages: feature extraction using importance lists, selective model execution based on feature matching, and ensemble aggregation. This segmentation allows the system to parallelize independent operations and skip unnecessary model executions, improving prediction speed while maintaining accuracy through the structured multi-model approach.
Data Source
AI summary
Techniques described herein relate to a method for predicting results using ensemble models. The method may include receiving trained model data sets from a model source nodes, each trained model data set comprising a trained model, an important feature list, and a missing feature generator; receiving a prediction request data set; making a determination that the prediction request data set does not include an input feature for a trained model; generating, based on the determination and using a missing feature generator, a substitute feature to replace the input feature; executing the trained model using the prediction request data set and the substitute feature to obtain a first prediction; executing a second trained model using the prediction request data set to obtain a second prediction; and obtaining a final prediction using the first prediction, the second prediction, and an ensemble model.