Black-Box Adversarial Query Search Using Bayesian Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing black-box adversarial attacks on neural networks are inefficient and time-consuming due to large search spaces and limitations in physically realizable perturbations, which can be costly and impractical for real-world scenarios.
Innovation Solution
A method utilizing Bayesian optimization to update adversarial patterns by incorporating all past successful query updates, allowing arbitrary pixel values, and using a Gaussian process model to efficiently select the next query, reducing the number of required queries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If random search or sparse-search methods are used for black-box adversarial attacks, then the attack can be implemented with minimal information about the model, but the number of queries required becomes extremely large (hundreds or thousands)
Solution Approach 1:
The patent implements feedback mechanisms where the attacker observes model outputs and uses this information to update and refine adversarial perturbations iteratively. The system incorporates feedback from query results to adjust the search direction, allowing convergence with far fewer queries than random search by learning from each interaction with the model.
Solution Approach 2:
The patent changes the search strategy from random parameter updates to optimized parameter selection based on observed model behavior. By dynamically adjusting perturbation parameters based on feedback and incorporating gradient information when available, the system reduces the number of queries needed while maintaining black-box attack capability.
2Reliability
If lp-bounded perturbations are used where all pixels can be perturbed, then the attack coverage is comprehensive, but the attack is not physically realizable as it modifies the captured image before feeding to the model
Solution Approach 1:
The patent segments the attack into two distinct components: (1) digital perturbation generation that maintains comprehensive coverage through lp-bounded perturbations, and (2) physical patch creation that is printed and placed in the scene. This segmentation allows the system to achieve both complete attack coverage and physical realizability by separating the optimization process from the physical implementation.
Solution Approach 2:
The patent creates physical copies of the optimized adversarial patterns by printing them as patches and placing them in the scene. This copying approach allows the digitally optimized perturbations to be transferred to the physical domain, achieving physical realizability while maintaining the comprehensive coverage benefits of lp-bounded perturbations.
3Device complexity
If sparse-search method limits pixel values to extreme values (0 or 255), then the search space is reduced, but the number of allowed perturbations is significantly limited reducing attack strength
Solution Approach 1:
The patent makes the search space dynamic by adapting the perturbation values based on feedback from model queries. Instead of static extreme values, the system dynamically adjusts pixel values within the full range (0-255) based on observed model responses, maintaining a reduced search space while preserving attack strength through adaptive optimization.
Solution Approach 2:
The patent performs preliminary optimization of perturbation values before finalizing the attack pattern. By pre-computing optimal pixel values within the full range and then creating physical copies, the system reduces the effective search space during the physical attack phase while maintaining strong attack capability through the pre-optimized values.
Data Source
AI summary
A system includes a machine learning network input interface configured to receive input data from a sensor, one or more processors collectively programmed to receive an input data from the sensor, wherein the input data is indicative of image of a scene that includes a perturbation from a black-box attack with a physical perturbation at the scene, display an adversarial pattern at the scene, determine an objective function utilizing at least the adversarial pattern and a target classification of the machine-learning network, randomly select a plurality of data points associated with the adversarial pattern and the objective function, wherein the data points are associated with a number of queries of the objective function, obtain a machine-learning model output utilizing the data points displayed in the scene, and in response to meeting a criteria associated with the adversarial pattern and model output, identify a successful attack pattern.