Detecting Backdoor Attacks in Convolutional Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting backdoor attacks in convolutional neural networks (CNNs) require prior knowledge of the targeted class or type of triggers and access to the poisoned training dataset, which is not feasible in scenarios where the user outsources model training to an untrusted third party or downloads pre-trained models from untrusted sources.
Innovation Solution
A system that uses Universal Litmus Patterns (ULPs) optimized through a process involving random images, training networks, and a classifier to detect backdoor attacks without prior knowledge of the targeted class or trigger type, by compiling CNN specifications into an executable model and feeding ULPs through it to determine the presence of backdoor attacks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If statistical analysis of poisoned training dataset or neural activations is used for backdoor detection, then detection capability is improved, but access to infected dataset is required which is not available in outsourced training scenarios
Solution Approach 1:
The patent introduces an intermediary detection mechanism that operates on the trained model itself rather than requiring access to training data. The system uses the model's own behavior and responses to detect backdoors, serving as a mediator between the unavailable poisoned data and the detection goal.
Solution Approach 2:
Instead of analyzing training data to detect backdoors (traditional approach), the patent inverts the approach by analyzing the trained model's outputs and behavior. The detection system feeds test images through the model and analyzes the neural activations and outputs to identify backdoor presence, reversing the conventional detection direction.
2Measurement precision
If Neural Cleanse approach is used to find minimal perturbations for backdoor detection, then detection accuracy is improved, but computational demand increases significantly
Solution Approach 1:
The patent applies partial action by using a fixed set of test images and a limited number of passes through the model, rather than performing exhaustive minimal perturbation searches. This partial approach achieves sufficient detection accuracy without the full computational burden of finding optimal perturbations for all possible image pairs.
Solution Approach 2:
The system uses simple, computationally inexpensive test images and basic neural activation analysis instead of complex optimization procedures. These lightweight detection mechanisms provide sufficient accuracy without requiring expensive computational resources.
3Measurement precision
If existing backdoor detection methods are used, then detection capability is improved, but prior knowledge of targeted class or trigger type is required
Solution Approach 1:
The patent creates a universal detection system that works across different backdoor types, target classes, and trigger configurations without requiring specific prior knowledge. The method analyzes general patterns in neural activations that are indicative of backdoors regardless of the specific attack variant, making the system universally applicable.
Solution Approach 2:
The system detects backdoors by analyzing changes in neural activation patterns and model outputs rather than requiring knowledge of specific trigger parameters or target classes. By focusing on behavioral changes in the model's processing rather than specific attack parameters, the method eliminates the need for prior knowledge about the backdoor configuration.
Data Source
AI summary
Described is a system for detecting backdoor attacks in deep convolutional neural networks (CNNs). The system compiles specifications of a pretrained CNN into an executable model, resulting in a compiled model. A set of Universal Litmus Patterns (ULPs) are fed through the compiled model, resulting in a set of model outputs. The set of model outputs are classified and used to determine presence of a backdoor attack in the pretrained CNN. The system performs a response based on the presence of the backdoor attack.