Backdoor Detection Model for Deep Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) deployed in security or safety-related tasks may be compromised with trojan or backdoor code, which can go undetected during normal testing and operation, posing a significant security concern.
Innovation Solution
A framework that includes a backdoor detection model to analyze DNNs for embedded trojans or backdoors by extracting characteristic features from their mathematical formulations and using statistical analysis, structure analysis, dynamic analysis, and inverse dynamic analysis to distinguish between compromised and uncompromised models, generating a detector model that can identify the presence of trojans or backdoors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If normal testing and operation are used to evaluate DNNs, then the DNNs can perform their intended tasks at an acceptable level, but the presence of embedded trojans or backdoor code cannot be detected
Solution Approach 1:
The patent applies preliminary action by performing backdoor detection analysis before the DNN is deployed for normal operation. The system extracts mathematical formulation features and performs statistical analysis in advance to identify compromised models, ensuring that only clean DNNs proceed to normal testing and deployment. This prevents the detection problem from arising during normal operation.
Solution Approach 2:
The patent introduces an intermediary detection system that acts as a mediator between DNN development and normal operation. This intermediary system uses mathematical formulation extraction and statistical analysis to bridge the gap between normal performance testing and security verification, enabling detection of trojans without interfering with the DNN's intended functionality.
2Measurement precision
If comprehensive analysis methods are used to detect trojans, then detection accuracy improves, but computational complexity and analysis time increase
Solution Approach 1:
The patent applies the extraction principle by isolating and analyzing only the mathematical formulation features of the DNN, separating the detection task from the full DNN operation. By extracting specific mathematical characteristics and applying statistical analysis only to these features rather than the entire model, the system achieves high detection accuracy while reducing computational complexity.
3Reliability
If multiple analysis techniques are applied to distinguish compromised from uncompromised models, then detection reliability improves, but the time and resources required for analysis increase
Solution Approach 1:
The patent performs preliminary mathematical formulation extraction and statistical analysis before deploying more complex detection techniques. This preliminary action filters out obviously clean or compromised models using computationally efficient methods, reducing the number of models that require more time-consuming comprehensive analysis and thereby reducing overall analysis time while maintaining high reliability.
Data Source
AI summary
A system and method including receiving a set of deep neural networks (DNN) including DNNs trained with an embedded trojan and DNNs trained without any embedded trojan, each of the trained DNNs being represented by a mathematical formulation learned by the DNNs and expressing a relationship between an input of the DNNs and an output of the DNNs; extracting at least one characteristic feature from the mathematical formulation of each of the trained DNNs; statistically analyzing the at least one characteristic feature to determine whether there is a difference between the DNNs trained with the embedded trojan and the DNNs trained without any embedded trojan; generating, in response to the determination indicating there is a difference, a detector model to execute the statistical analyzing on deep neural networks; and storing a file including the generated detector model in a memory device.