Computer Vision Robustness Evaluation Using Learned Black-Box Attacks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing black-box attack techniques for evaluating the robustness of computer vision models are unreliable due to their dependence on manually specified schedules and update distributions, making it unclear whether the lack of adversarial examples found is because they do not exist or because the attack was ineffective.
Innovation Solution
A computer-implemented method using a trained strategy model for black-box attacks, which is trained with white-box access to source models, allowing it to adapt proposal distributions dynamically and provide a more accurate and versatile indication of model robustness without requiring gradient access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If manually specified schedules and update distributions are used in black-box attacks, then the attack can be applied to a wide range of models, but the reliability of robustness evaluation deteriorates due to inability to determine whether lack of adversarial examples is due to non-existence or attack ineffectiveness
Solution Approach 1:
The patent applies dynamics by replacing static manually specified schedules and update distributions with a learned strategy model that dynamically adapts its parameters based on feedback from the target model. The strategy model continuously refines its attack strategy through iterative optimization, allowing it to adjust proposal distributions and attack parameters automatically during the evaluation process, thereby resolving the contradiction between versatility and reliability.
Solution Approach 2:
The patent implements feedback mechanisms where the strategy model uses the responses from the target computer vision model to iteratively improve its attack strategy. By incorporating feedback loops that monitor attack effectiveness and adjust the strategy model's parameters accordingly, the system can reliably determine whether adversarial examples exist while maintaining broad applicability across different model types.
2Measurement precision
If gradient-based techniques are used for robustness evaluation, then the evaluation can be more precise, but the technique becomes inapplicable to models with gradient obfuscation or where gradients are unavailable
Solution Approach 1:
The patent introduces an intermediary approach by using a strategy model that mediates between the attacker and the target model. Instead of directly computing gradients (which fail on obfuscated models), the strategy model learns attack strategies through indirect interactions, using the model's output responses to guide the search for adversarial examples. This intermediary layer enables precise robustness measurement while maintaining applicability to models with gradient obfuscation.
Solution Approach 2:
The patent substitutes the mechanical gradient-based optimization system with a learnable strategy model that operates through functional relationships. Rather than relying on direct gradient computations, the system replaces the gradient mechanism with a data-driven approach where the strategy model learns optimal attack strategies by observing model responses, enabling precise measurement without requiring gradient access.
3Adaptability or versatility
If black-box attacks are used to evaluate robustness, then the evaluation is more versatile and applicable to models with gradient obfuscation, but the effectiveness of the attack becomes uncertain without manually tuned parameters
Solution Approach 1:
The patent applies dynamics by implementing a learnable strategy model that automatically adapts attack parameters based on feedback from the target model. Instead of relying on fixed manually tuned parameters, the strategy model dynamically adjusts its proposal distributions and search strategies during the evaluation process, thereby improving the accuracy of robustness indications while maintaining the versatility of black-box attack approach.
Solution Approach 2:
The patent implements parameter changes by automatically adjusting attack parameters through the strategy model's learned parameters. The strategy model modifies proposal distributions, search step sizes, and other attack parameters based on the specific characteristics of the target model, eliminating the need for manual parameter tuning while improving measurement precision through adaptive parameter optimization.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
The invention relates to a computer-implemented method (800) of evaluating a robustness of a target computer vision model using a black-box attack. The black-box attack comprises repeatedly determining an update to a current perturbation using a strategy model. The strategy model has been trained by: applying the black-box attack according to the strategy model to a source computer vision model; and using white-box access to the source model, updating the strategy model based on an effectiveness of the black-box attack. Using the trained strategy model, the robustness is evaluated by obtaining input images for the target model; applying the black-box attack according to the trained strategy model; and determining an effectiveness of the black-box attack by comparing model outputs of the target model for the input images and for the input images perturbed according to the obtained perturbations. A robustness measure is output based on the determined effectiveness.