Shadow Model Training Pipeline for ML Reverse Engineering Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud-based machine learning models are vulnerable to reverse engineering attacks, where attackers reconstruct a surrogate model by submitting inference requests, compromising the confidentiality and privacy of the original model and its data.
Innovation Solution
A shadow model training pipeline is generated and monitored, with feature vectors from inference requests used to train the shadow model, allowing for comparison with the original model to detect and prevent reverse engineering attacks by filtering or blocking malicious requests.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If cloud-based machine learning models are made accessible over a network, then ease of operation and accessibility are improved, but vulnerability to reverse engineering attacks increases
Solution Approach 1:
The patent creates a shadow model that is a copy of the original machine learning model. This shadow model receives the same inference requests as the original model and generates responses that can be compared to detect reverse engineering attacks. The copying principle allows the system to monitor and detect attacks without affecting the original model's accessibility.
Solution Approach 2:
The shadow model acts as an intermediary between the original machine learning model and the attacker. By placing the shadow model in the inference path, the system can detect reverse engineering attempts through comparison of responses while maintaining the original model's accessibility to authorized users.
2Reliability
If a shadow model training pipeline is implemented to detect reverse engineering attacks, then security and reliability are improved, but device complexity increases
Solution Approach 1:
Instead of implementing complex security measures, the system uses a simplified approach by copying the original model to create a shadow model. This copy is trained in parallel and compares responses to detect attacks, providing reliable security monitoring with relatively simple system architecture.
Solution Approach 2:
The shadow model training pipeline uses the same inference requests that would normally go to the original model to train the shadow model. This self-service approach allows the system to monitor for attacks while utilizing existing computational resources and traffic patterns.
3Measurement precision
If the original model responses are compared with shadow model responses to detect attacks, then measurement precision of attack detection is improved, but computational resources and time consumption increase
Solution Approach 1:
The shadow model is trained to replicate the original model's behavior, allowing for direct comparison of responses. This copying approach enables precise attack detection by measuring deviations in responses while using efficient comparison methods that minimize processing time overhead.
Data Source
AI summary
Embodiments prevent a reverse engineering attack on a machine learning (“ML”) model. Embodiments receive a first set of requests from a plurality of users to the ML model. Based on the first set of requests, embodiments identify a first user attempting to attack the ML model and, in response to the identifying, generate a shadow model that is similar to the ML model. Embodiments receive a second set of requests from the first user to the ML model and, in response to the second set of requests, generate an ML model set of responses and a shadow model set of responses. Embodiments compare the ML model set of responses with the shadow model set of responses and, based on the comparison, determine whether the first user is attempting the reverse engineering attack on the ML model.