Classifier with abstain class for adversarial detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning networks lack provable performance guarantees for robustness against adversarial perturbations, leading to potential misclassification and failure in detecting adversarial examples.
Innovation Solution
A computer-implemented method and system for training a machine-learning network with an additional abstain class, where the classifier is trained to provide worst-case bounds on classification error and loss for perturbed inputs, ensuring robustness and detection of adversarial examples.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If adversarial training is applied to robustify classifier performance against perturbations, then robustness is improved, but provable performance guarantees are lost
Solution Approach 1:
The system performs preliminary bounding of hidden layer values before classification to establish worst-case guarantees. By computing bounds on activation values in advance and using them to determine the abstain class, the system proactively ensures robustness with provable guarantees rather than relying on post-hoc adversarial training.
2Reliability
If an abstain class is added to detect adversarial examples, then detection capability is improved, but device complexity increases
Solution Approach 1:
The abstain class acts as an intermediary between regular classification and adversarial detection. Rather than building a completely separate detection system, the patent introduces an additional output class that the classifier can select, providing detection capability while reusing the existing network architecture and weights.
Solution Approach 2:
The system changes the parameter space of the classifier by adding one additional output class. This minimal parameter change enables detection functionality without substantially increasing model complexity, as the added parameter is simply one more class probability output rather than a new network architecture.
3Reliability
If worst-case bounds are computed for all perturbed versions of input data, then robustness is improved, but computational cost increases
Solution Approach 1:
Instead of computing worst-case bounds uniformly for all possible perturbations, the system computes bounds locally at each hidden layer based on the actual activation values. This localized bounding approach provides robustness guarantees where needed while avoiding unnecessary computational overhead for regions of the input space that are not actually activated.
Solution Approach 2:
The system performs partial bounding by computing bounds only for the specific hidden layer values that are actually activated during forward propagation, rather than computing bounds for all possible perturbations across the entire input space. This partial action provides sufficient robustness guarantees for the actual input while reducing computational cost.
Data Source
AI summary
A computer-implemented method for training a machine-learning network includes receiving an input data from a sensor, wherein the input data includes a perturbation, wherein the input data is indicative of image, radar, sonar, or sound information, obtain a worst-case bound on a classification error and loss for perturbed versions of the input data, utilizing at least bounding of one or more hidden layer values, in response to the input data, train a classifier, wherein the classifier includes a plurality of classes, including an additional abstain class, wherein the abstain class is determined in response to at least bounding the input data, outputting a classification in response to the input data, and output a trained classifier configured to detect the additional abstain class in response to the input data classifier with a plurality of classes, including an additional abstain class.