Adversarial Detection of Backdoored Machine Learning Classifiers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning classification systems, particularly deep learning systems, can be compromised by backdoored classifiers that misclassify images or sounds when a specific trigger pattern is detected, raising concerns about security and reliability.
Innovation Solution
A computer-implemented method and system for identifying and mitigating backdoored classifiers by detecting trigger patterns through adversarial attacks on robustified versions of the classifiers, allowing for the identification of both original and unintended trigger patterns.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a backdoored classifier is trained with trigger patterns, then the classifier can be controlled to misclassify specific inputs, but the reliability and security of the classification system deteriorates
Solution Approach 1:
The patent applies adversarial examples to expose backdoor trigger patterns. By intentionally creating adversarial perturbations that trigger the backdoor mechanism, the system converts the harmful backdoor behavior into useful information for detection and mitigation. The adversarial examples serve as a means to uncover and neutralize the hidden trigger patterns, transforming the security vulnerability into a detection opportunity.
2Reliability
If adversarial attacks are performed to identify trigger patterns, then the security of the classifier improves, but the complexity of the system increases
Solution Approach 1:
The patent performs preliminary adversarial attacks during the training phase to identify and mitigate trigger patterns before the classifier is deployed. By conducting these security-related adversarial examples in advance, the system proactively identifies vulnerable trigger patterns and applies mitigations, thereby reducing the need for complex real-time detection mechanisms during actual operation.
3Reliability
If trigger patterns are identified through adversarial examples, then unintended misclassifications can be prevented, but the time required for training increases
Solution Approach 1:
The patent integrates trigger pattern identification through adversarial examples into the training process itself, performing this security-related computation during the training phase rather than requiring separate post-training analysis. This preliminary action allows the system to learn both normal classification tasks and security-related trigger detection simultaneously, reducing overall training time compared to sequential approaches.
Data Source
AI summary
A computer-implemented method for training a machine learning network includes receiving an input data from one or more sensors, selecting one or more batch samples from the input data, wherein the batch samples include one or more perturbed samples from a source class configured to be misclassified into a target class, identifying the one or more perturbed samples from the one or more batch samples, determining a trigger event in response to identification of a trigger pattern of the one or more batch samples, wherein the trigger pattern induces a pre-determined response on a classifier, outputting a classification in response to identification of the trigger pattern via the classifier, and outputting a set of trigger patterns extracted from the machine-learning network.