Neural Network Logit Analysis for Backdoor Trigger Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks are vulnerable to backdoor attacks during training, where malicious data with hidden triggers cause the model to produce incorrect outputs, compromising security and safety in critical applications like vehicle automation and security systems.
Innovation Solution
A method to detect backdoor triggers in neural networks by analyzing logit distributions, comparing the probability of the highest logit value against a threshold, and taking corrective actions when a backdoor is suspected.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If neural networks are trained using poisoned data containing backdoor triggers, then the model achieves normal classification performance on clean data, but the model produces incorrect outputs when triggered during inference
Solution Approach 1:
The patent applies preliminary action by computing and storing the distribution of logit values for all classes during a setup phase using benign data before the testing phase. This pre-computed distribution serves as a reference to detect backdoor triggers when data is presented for classification, allowing the system to identify anomalies without reprocessing the entire benign dataset during inference
Solution Approach 2:
The patent uses logit value distribution as an intermediary mechanism to detect backdoor triggers. Instead of directly analyzing input data for triggers or modifying the neural network architecture, the system computes logits for each class and compares their distribution against the pre-established benign distribution. This intermediary approach enables indirect detection of backdoor activity while maintaining normal classification functionality
2Measurement precision
If the system computes logit distributions for all classes to detect backdoors, then detection accuracy improves, but computational complexity increases
Solution Approach 1:
The patent reduces computational complexity during the testing phase by performing the computationally intensive logit distribution computation in advance during the setup phase. The pre-computed distribution is stored and reused for multiple detections, avoiding repeated heavy calculations while maintaining high detection accuracy through the established reference distribution
Solution Approach 2:
The system uses the pre-computed logit distribution to serve itself in detecting backdoor triggers. During testing, the system only needs to compute logits for the current input and compare against the stored distribution, rather than requiring complex external detection mechanisms or extensive reprocessing of training data
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Determination in an artificial intelligence system using a neural network whether a deep neural network (DNN) of the neural network based artificial intelligence system has a backdoor trigger, wherein the DNN is used by the artificial intelligence system to classify data objects into a set of classes. The method includes a setup phase having the steps of for all elements in a dataset of benign data, computing logits corresponding to all said classes and computing a distribution of the logit values for all classes wherein the distribution is a count of data objects corresponding to each logit value for each class. The method further includes a testing phase having the steps of for a presented data object, calculating logitvalues for each class, determining the probability that the highest logit value corresponds to a data object carrying a backdoor trigger; and when the probability that the highest logit value corresponds to a data object carrying a trigger, taking a corrective action.