Distributed ML Model Partitioning with Trusted Execution Environment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing distributed learning systems face challenges in distinguishing between legitimate and malicious updates, making them vulnerable to attacks such as malicious update attacks, model stealing, data extraction, and data poisoning.
Innovation Solution
Implementing a Trusted Execution Environment (TEE) at edge devices to securely process private classification layers, while using Byzantine Gradient Descent at the aggregator device to filter out extreme updates and ensure model integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If distributed learning is implemented across multiple edge devices, then training efficiency and scalability are improved, but the system becomes vulnerable to malicious updates and attacks
Solution Approach 1:
The patent introduces an aggregator device as an intermediary between edge devices. This aggregator collects model updates from multiple edge devices, applies Byzantine Gradient Descent to filter malicious updates, and generates a cleaned aggregated model. The intermediary structure allows distributed training to proceed efficiently while centralizing the security function to protect against malicious updates.
Solution Approach 2:
The patent implements a feedback mechanism where the aggregator device sends the aggregated model back to edge devices for further training. This closed-loop feedback system allows continuous improvement of the global model while enabling the aggregator to monitor and filter updates in each iteration, maintaining reliability across distributed training cycles.
2Loss of information
If private classification layers are processed at edge devices, then data privacy is improved, but security against model stealing and data extraction attacks deteriorates
Solution Approach 1:
The patent extracts the most sensitive classification layers and processes them separately at the aggregator device rather than at edge devices. By taking out these critical layers from the distributed edge environment, the system maintains data privacy through federated learning while reducing the attack surface for model stealing and data extraction attacks at the edge.
Solution Approach 2:
The patent segments the model into public layers (processed at edge devices) and private layers (processed at aggregator device). This segmentation allows the system to distribute processing for efficiency while concentrating security-critical operations in a controlled environment, balancing privacy benefits with security requirements.
3Reliability
If Byzantine Gradient Descent is used to filter malicious updates, then model robustness is improved, but computational complexity and processing time increase
Solution Approach 1:
The patent applies Byzantine Gradient Descent partially, only to the aggregation step at the aggregator device rather than at every edge device. This partial application provides sufficient robustness against malicious updates while avoiding the excessive computational complexity that would result from implementing full Byzantine resilience at all distributed nodes.
4Adaptability or versatility
If model updates are aggregated from multiple edge devices, then learning diversity is improved, but vulnerability to malicious update attacks increases
Solution Approach 1:
The patent converts the potential harm of malicious updates into a benefit by using Byzantine Gradient Descent to identify and filter them. The same aggregation mechanism that enables learning diversity from multiple edge devices also concentrates malicious updates, which the Byzantine algorithm can then detect and remove, turning the diversity vulnerability into a detectable anomaly.
Data Source
AI summary
Methods, apparatus, systems and articles of manufacture for distributed use of a machine learning model are disclosed. An example edge device includes a model partitioner to partition a machine learning model received from an aggregator into private layers and public layers. A public model data store is implemented outside of a trusted execution environment of the edge device. The model partitioner is to store the public layers in the public model data store. A private model data store is implemented within the trusted execution environment. The model partitioner is to store the private layers in the private model data store.