Backdoor Detection in Deep Neural Networks Using Clean Samples
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods are inadequate for detecting backdoor poisoning in Deep Neural Network (DNN) classifiers, particularly when the training set is inaccessible, as they fail to identify imperceptible backdoor patterns that can cause misclassification without degrading the classifier's performance on clean patterns, making them undetectable by conventional validation procedures.
Innovation Solution
A novel unsupervised framework that uses a set of clean data samples to estimate potential backdoor perturbations for each source-target class pair, determining if a DNN has been backdoor-poisoned by identifying candidate perturbations based on size and misclassification rates, and inferring the source and target classes associated with the attack, without requiring access to the training set or labeled examples.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional validation procedures are used to detect backdoor patterns, then the detection process is simple and accessible, but the backdoor patterns remain undetectable because they do not degrade classifier performance on clean patterns
Solution Approach 1:
The detection process is segmented into multiple independent optimization problems, one for each source-target class pair. This allows the system to systematically evaluate each potential backdoor mapping separately, identifying subtle misclassification patterns that conventional validation would miss while maintaining overall system reliability.
Solution Approach 2:
The system performs preliminary estimation of backdoor perturbations for each source-target class pair before making a final detection determination. By pre-computing these perturbations and evaluating their misclassification rates in advance, the system can identify backdoor patterns that would otherwise remain hidden during standard validation procedures.
2Reliability
If the training set is made inaccessible to protect against poisoning, then the security of the training process is improved, but the ability to detect backdoor patterns is reduced
Solution Approach 1:
The system introduces an intermediary detection mechanism that operates on clean test samples rather than requiring direct access to the training set. This intermediary process estimates backdoor perturbations and evaluates misclassification rates, enabling security verification without compromising training process security or requiring training data access.
Solution Approach 2:
The system creates a virtual representation of the backdoor attack by estimating potential backdoor perturbations for each source-target class pair. Instead of accessing the actual poisoned training data, the system generates and evaluates hypothetical perturbed versions of clean samples, effectively copying the attack pattern for analysis without needing the original poisoned data.
3Quantity of substance
If a small clean data set is used for detection, then the resource requirements are reduced, but the detection accuracy may be compromised
Solution Approach 1:
The system changes the evaluation parameter from requiring large quantities of clean data to evaluating the misclassification rate induced by estimated backdoor perturbations. By focusing on the effect of perturbations rather than the volume of clean samples, the system achieves reliable detection even with small clean data sets, as the key metric is how strongly the estimated backdoor pattern causes misclassification.
Data Source
AI summary
This patent concerns novel technology for detecting backdoors of neural network, particularly deep neural network (DNN), classifiers. The backdoors are planted by suitably poisoning the training dataset, i.e., a data-poisoning attack. Once added to input samples from a source class (or source classes), the backdoor pattern causes the decision of the neural network to change to a target class. The backdoors under consideration are small in norm so as to be imperceptible to a human, but this does not limit their location, support or manner of incorporation. There may not be components (edges, nodes) of the DNN which are dedicated to achieving the backdoor function. Moreover, the training dataset used to learn the classifier may not be available. In one embodiment of the present invention which addresses such challenges, if the classifier is poisoned then the backdoor pattern is determined through a feasible optimization process, followed by an inference process, so that both the backdoor pattern itself and the associated source class(es) and target class are determined based only on the classifier parameters and a set of clean (unpoisoned attacked) samples from the different classes (none of which may be training samples).