Adversarial Attack on Black Box Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing adversarial attack methods are ineffective against black box object detection algorithms, as they require knowledge of the algorithm's operational details, which are often unknown, and primarily focus on characterizing classification rather than detection aspects.
Innovation Solution
A substitute model is trained using initial training data from the black box object detection algorithm, allowing for the generation of adversarial input data that the algorithm will fail to detect accurately, without needing to know the algorithm's operational details.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If adversarial attack methods are applied to object detection algorithms, then the ability to identify vulnerable input data is improved, but the methods are ineffective when the algorithm's operational details are unknown (black box scenario)
Solution Approach 1:
The patent creates a substitute model that copies the input-output behavior of the black box object detection algorithm. This substitute model replicates the detection results without requiring knowledge of the internal algorithmic operations, enabling adversarial attacks to be performed on the copy and transferred to the original system.
Solution Approach 2:
The substitute model serves as an intermediary between the attacker and the black box algorithm. Instead of directly attacking the opaque algorithm, the attacker interacts with the substitute model, which mediates the attack process by providing a surrogage that mimics the target system's behavior.
2Measurement precision
If existing adversarial attack methods are used, then classification characterization is improved, but detection aspects remain unaddressed
Solution Approach 1:
The patent adapts adversarial attack methods dynamically to work with object detection algorithms by modifying the attack approach from classification-focused to detection-focused. The substitute model is specifically trained to replicate detection outputs (bounding boxes, object locations) rather than just class labels, making the attack method versatile for detection tasks.
Data Source
AI summary
Systems and methods to generate an adversarial attack on a black box object detection algorithm of a sensor involve obtaining an initial training data set from the black box object detection algorithm. The black box object detection algorithm performs object detection on initial input data to provide black box object detection algorithm output that provides the initial training data set. A substitute model is trained with the initial training data set such that output from the substitute model replicates the black box object detection algorithm output that makes up the initial training data set. Details of operation of the black box object detection algorithm are unknown and details of operation of the substitute model are known. The substitute model is used to perform the adversarial attack. The adversarial attack refers to identifying adversarial input data for which the black box object detection algorithm will fail to perform accurate detection.