Trojan Neural Network Detection via Adversarial Perturbation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for detecting Trojan neural networks are inadequate, especially in data-limited or data-free environments, as they require access to training or test data and are sensitive to trigger size and quality, making them impractical for ensuring model trustworthiness in online scenarios.
Innovation Solution
The proposed solution involves generating adversarial perturbations to identify neuron activations and maximize them, allowing for the detection of Trojan models without access to clean data, using a data-limited TrojanNet detector (DL-TND) and a data-free TrojanNet detector (DF-TND), which leverage universal and per-sample perturbations to reveal trigger patterns and detect poisoned models efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If current detection methods are used, then detection capability is provided, but access to training or test data is required and sensitivity to trigger size and quality occurs
Solution Approach 1:
The patent extracts the detection capability from the context of requiring training or test data. By formulating detection as an optimization problem that operates on the model itself without external data, it removes the data access requirement while maintaining detection reliability.
Solution Approach 2:
The detection method performs self-service by using the model's own internal representations and computations to detect Trojan triggers. The method leverages the model's forward propagation mechanics and neuron activations without requiring external training or test datasets.
2Productivity
If data-limited or data-free detection is implemented, then detection speed and trustworthiness are enhanced, but ability to detect small or subtle triggers deteriorates
Solution Approach 1:
The patent performs preliminary action by pre-computing and storing the influence matrices of neuron activations during a single forward pass. This preliminary computation enables rapid detection of both large and subtle triggers without requiring multiple data iterations, thus maintaining both speed and precision.
Solution Approach 2:
The patent transitions from data-space analysis to model-parameter-space analysis. By examining the internal neuron activation patterns and their influences on output predictions, it detects triggers in a different dimensional space that reveals subtle patterns invisible in traditional data-based approaches.
3Measurement precision
If multiple adversarial perturbations are generated and neuron activations maximized, then detection accuracy is improved, but computational complexity increases
Solution Approach 1:
The patent applies partial action by focusing computational effort only on the most informative neuron activations and their influences. Rather than exhaustively analyzing all possible perturbations and activations, it identifies and maximizes the key activations that provide the strongest detection signals, achieving high accuracy with reduced complexity.
Data Source
AI summary
One or more computer processors generate a plurality of adversarial perturbations associated with a model, wherein the plurality of adversarial perturbations comprises a universal perturbation and one or more per-sample perturbations. The one or more computer processors identify a plurality of neuron activations associated with the model and the plurality of generated adversarial perturbations. The one or more computer processors maximize the identified plurality of neuron activations. The one or more computer processors determine the model is a Trojan model by leveraging one or more similarities associated with the maximized neuron activations and the generated adversarial perturbations.