Neural Network Prediction Screening for Adversarial Perturbations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks are vulnerable to adversarial perturbations, which can lead to incorrect classifications in safety-critical applications, necessitating mechanisms to detect and mitigate such perturbations.
Innovation Solution
A method that uses a processor and memory to implement a neural network, generating measurement quantities based on gradients of the network function and objective function to determine if inputs are perturbed, and performing remedial actions if thresholds are exceeded, such as rejecting predictions or issuing warnings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If neural networks are used for classification in sensing systems, then classification accuracy and productivity are improved, but vulnerability to adversarial perturbations increases
Solution Approach 1:
The system performs preliminary actions by computing measurement quantities (gradients of network function and objective function) before final classification decisions are made. This allows the system to detect adversarial perturbations in advance and take remedial actions such as rejecting predictions or issuing warnings, thereby maintaining both classification efficiency and robustness against perturbations.
2Reliability
If measurement quantities are computed to detect perturbations, then reliability is improved, but device complexity and computational cost increase
Solution Approach 1:
The system uses the neural network's own internal structures (network function gradients and objective function gradients) to generate measurement quantities for perturbation detection. This self-service approach leverages existing computational resources within the neural network framework, avoiding the need for entirely separate detection systems and thereby improving reliability while controlling complexity.
3Reliability
If remedial actions are performed for perturbed inputs, then reliability is improved, but loss of time and productivity occur
Solution Approach 1:
The system implements feedback by continuously monitoring measurement quantities and comparing them against threshold values. When perturbations are detected, the system provides feedback through remedial actions such as rejecting predictions or issuing warnings. This feedback mechanism maintains classification reliability while minimizing processing delays by only triggering remedial actions when necessary, rather than processing every input through multiple verification stages.
Data Source
AI summary
A system and method for processing predictions in the presence of adversarial perturbations in a sensing system. The processor receives inputs from sensors and runs a neural network having a network function that generates, as outputs, predictions of the neural network. The method generates from a plurality of outputs a measurement quantity (m) that may be, at or near a given input, either (i) a first measurement quantity M1 corresponding to a gradient of the given output, (ii) a second measurement quantity M2 corresponding to a gradient of a predetermined objective function derived from a training process for the neural network, or (iii) a third measurement quantity M3 derived from a combination of M1, and M2. The method determines whether the measurement quantity (m) is equal to or greater than a threshold. If greater than the threshold, one or more remedial actions are performed to correct for a perturbation.