Multi-Size Image Masking for Adversarial Patch Robustness
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning based image classifiers are vulnerable to adversarial patches, which are easier to implement in the physical world compared to full image adversarial perturbations, posing a significant threat to real-world computer vision systems.
Innovation Solution
A multi-size greedy cutout system is employed to generate and train machine learning models using a set of masks that cover adversarial patches, ensuring robustness by generating worst-case masked images and creating a training dataset with these masks to enhance the model's resistance to adversarial attacks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multi-size masking is applied to generate worst-case masked images, then certified robustness against adversarial patches is improved, but device complexity increases
Solution Approach 1:
The patent segments the masking process into multiple sizes (e.g., 50%, 75%, 90% masks) and applies them systematically to different regions of the image. This segmentation allows the system to handle adversarial patches of various sizes and positions, improving certified robustness while keeping each individual mask operation computationally manageable.
Solution Approach 2:
The patent performs preliminary masking operations during the training phase to generate worst-case masked images. By pre-computing these adversarial scenarios and incorporating them into the training dataset, the system prepares the neural network in advance to handle potential attacks, thereby improving robustness without adding complexity to the deployment phase.
2Reliability
If comprehensive masking strategies are used to cover all possible adversarial patches, then defense reliability is improved, but computational cost increases
Solution Approach 1:
The patent applies different masking strategies to different regions of the image based on their vulnerability to adversarial patches. Critical regions receive more aggressive masking (e.g., 90% masks), while less critical regions use lighter masking (e.g., 50% masks). This localized approach ensures comprehensive coverage where needed while reducing unnecessary computational overhead in other areas.
Solution Approach 2:
The patent uses a tiered masking approach where multiple levels of masking intensity are applied (partial masking for some regions, excessive masking for others). This allows the system to achieve comprehensive coverage without uniformly applying the most computationally expensive masking operations across the entire image, thus balancing defense reliability with computational cost.
3Reliability
If multiple masks and submasks are generated for training, then model robustness is improved, but training time increases
Solution Approach 1:
The patent segments the training process into multiple batches, each handling a specific mask size or type. Instead of generating all possible masked images simultaneously, the system processes them in organized segments (e.g., first 50% masks, then 75% masks, then 90% masks), which improves memory management and allows for better utilization of parallel computing resources, reducing overall training time.
Solution Approach 2:
The patent implements periodic masking operations during training, where masks are applied at regular intervals rather than continuously. This periodic approach reduces the total number of forward propagation operations required while still providing sufficient adversarial examples for robust training, thereby decreasing training time without significantly compromising model robustness.
Data Source
AI summary
A computer-implemented system and method relate to certified robust defenses against adversarial patches. A two-mask image is generated using a first mask and a second mask with respect to a source image. The two-mask image is associated with a highest prediction loss. A set of two-submask images are generated using a first submask and a second submask with respect to the source image. The first submask refers to a portion of the first mask. The second submask refers a portion of the second mask. A machine learning system generates a set of predictions upon receiving the set of two-submask images. A particular two-submask image with a highest prediction loss is selected from among the set of two-submask images. The machine learning system is trained via a training dataset, which includes the source image, the two-mask image, and the selected two-submask image.