Backdoored Models Detect Adversarial Samples
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models, particularly neural networks, are vulnerable to adversarial samples that can cause misclassification, and existing defense mechanisms fail to effectively thwart adaptive adversaries and prevent the transferability of adversarial samples from surrogate to target models.
Innovation Solution
The method involves generating backdoored versions of a machine learning model by introducing carefully crafted triggers during the training phase, allowing the model to detect adversarial samples by comparing classifications from multiple backdoored models against a genuine model, thereby reducing the transferability of adversarial samples.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If model parameters are hidden to make it harder for adversaries to create adversarial samples, then security is improved, but adversarial samples can still transfer from surrogate models to target models with high probability
Solution Approach 1:
Instead of trying to hide model parameters to prevent adversarial attacks (conventional approach), the patent inverts the strategy by openly distributing backdoored models that are designed to detect adversarial samples. The backdoored models contain intentional vulnerabilities that trigger specific behaviors when exposed to adversarial inputs, allowing the system to identify and reject such samples without needing to conceal the detection mechanism.
Solution Approach 2:
The patent introduces backdoored models as intermediary components between the target model and adversarial samples. These backdoored models act as mediators that first interact with incoming samples, and if they detect adversarial characteristics, they signal the system to reject the sample before it reaches the main target model, thus preventing adversarial transfer without requiring parameter hiding.
2Measurement precision
If multiple backdoored models are used to detect adversarial samples, then detection accuracy is improved, but system complexity increases
Solution Approach 1:
The detection system is segmented into multiple independent backdoored models, each with a specific detection trigger or vulnerability profile. Rather than using a single complex detection system, the patent divides the detection function across multiple simpler models, where each model handles a specific aspect of adversarial detection. This segmentation improves detection accuracy through specialized detection while keeping individual model complexity low.
Solution Approach 2:
The system uses multiple backdoored models with potentially redundant detection capabilities. Rather than optimizing for minimal complexity, the patent employs excessive detection capacity by deploying multiple models that may overlap in their detection functions. This partial redundancy ensures high detection accuracy by catching adversarial samples that might evade individual models, while the cost is acceptable given the critical security function.
Data Source
AI summary
A method for securing a genuine machine learning model against adversarial samples includes receiving a sample, as well as receiving a classification of the sample using the genuine machine learning model or classifying the sample using the genuine machine learning model. The sample is classified using a plurality of backdoored models, which are each a backdoored version of the genuine machine learning model. The classification of the sample using the genuine machine learning model is compared to each of the classifications of the sample using the backdoored models to determine a number of the backdoored models outputting a different class than the genuine machine learning model. The number of the backdoored models outputting a different class than the genuine machine learning model is compared against a predetermined threshold so as to determine whether the sample is an adversarial sample.