Layer-Wise Neural Network Detection for AI Backdoor Triggers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks are vulnerable to backdoor attacks, where malicious triggers are embedded during training, causing misclassifications that compromise security, integrity, and trust in critical applications like vehicle automation and security systems.
Innovation Solution
A method involving topological data analysis of neuron activation values across layers of a neural network to detect anomalies indicative of backdoor triggers, using a pattern library of characteristic values from benign data to compare with test data, and taking corrective actions if a threshold probability of a backdoor is exceeded.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If neural networks are used to provide artificial intelligence solutions in critical applications, then functionality and intelligence are improved, but security and reliability deteriorate due to backdoor attacks
Solution Approach 1:
The system performs preliminary analysis of neuron activation patterns during the inference process itself. By establishing a pattern library from benign data during training and comparing test data activations against this library, the system proactively detects backdoor triggers before they can cause harmful misclassifications, rather than reacting after damage occurs
Solution Approach 2:
The patent introduces an intermediary detection layer that sits between the neural network's normal operation and the final classification output. This intermediary mechanism analyzes the internal neuron activation patterns and compares them against expected patterns from the pattern library, serving as a mediator that can identify and block backdoor attacks without interfering with legitimate network functionality
2Manufacturing precision
If backdoor triggers are embedded in training data, then the model learns targeted behavior, but the model's integrity and trustworthiness deteriorate
Solution Approach 1:
The system implements a feedback mechanism where the analysis of neuron activation patterns feeds back into the detection process. By continuously comparing actual activation patterns against the pattern library and adjusting detection thresholds based on observed deviations, the system can identify backdoor triggers while maintaining the ability to learn and adapt to legitimate variations in data
Solution Approach 2:
The patent extracts and isolates the problematic backdoor trigger patterns from the otherwise functional neural network. By separating the detection of anomalous activation patterns from the normal classification process, the system can identify and remove the influence of backdoor triggers while preserving the legitimate learning and classification capabilities of the network
3Measurement precision
If layer-wise neuron activation analysis is performed to detect backdoors, then detection accuracy is improved, but computational complexity increases
Solution Approach 1:
The patent segments the neural network into individual layers and analyzes neuron activation patterns layer by layer. By dividing the complex task of backdoor detection into manageable layer-specific analyses and comparing each layer's activations against corresponding layers in the pattern library, the system achieves high detection accuracy while making the computational process more tractable and efficient
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present invention relates to a method for protecting a neural network based artificial intelligence system having a deep neural network (DNN) against backdoor attacks, wherein the DNN is used by the artificial intelligence system to classify data elements into a set of classes, the method comprising: -a set-up phase (901) including: for a plurality of data elements in a dataset of benign data, for each layer of neurons of a plurality of layers of the DNN computing characteristic values based on neuron activation values of neurons of said each layer of the plurality of layers of the DNN thereby computing a layerwise time series library of characteristic values for each data element; - a testing phase (903) including: receiving a data element under test; for the input data element, for each layer of neurons of the DNN computing characteristic values based on neuron activation values of said each layer of the plurality of layers of the DNN thereby computing a time series of characteristic values for the data element under test; computing a probability that data element under test contains a backdoor trigger based on comparing the time series of characteristic values for the data element under test to the layerwise time series library of characteristic values for each data element; if the probability that data element under test contains a backdoor trigger exceeds a threshold, take a corrective action (917).