Cohort-Based Adversarial Input Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning systems, particularly those based on deep learning, are vulnerable to adversarial attacks that manipulate input data to misclassify images, leading to security breaches and misidentification, with current defense mechanisms being fragile and requiring white-box access, adversarial samples, specific layers, or retraining.
Innovation Solution
A cohort-based adversarial input detection mechanism that uses a pair of machine learning classifiers, one trained on input images and another on visually similar images, to compare classification outputs and detect adversarial inputs by identifying discrepancies between feature and image spaces, without requiring white-box access or retraining.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing defense mechanisms are used, then adversarial attack detection is achieved, but the system requires white-box access, adversarial samples, specific layers, or retraining which increases complexity and reduces reliability
Solution Approach 1:
The patent creates a copy of the original image (the cohort image) and processes it through the same neural network to generate a corresponding classification output. This copy mechanism allows the system to detect adversarial attacks by comparing the classification outputs of the original and copied images, eliminating the need for white-box access or retraining while maintaining robustness against adversarial attacks.
Solution Approach 2:
The patent introduces an intermediary comparison mechanism that evaluates the difference between classification outputs of the original and copied images. This intermediary step serves as a mediator to detect adversarial perturbations without requiring direct access to the attack vectors or modification of the neural network architecture, thereby reducing system complexity while improving reliability.
2Reliability
If a single machine learning classifier is used, then processing speed is maintained, but the system cannot detect adversarial inputs without additional comparison mechanisms
Solution Approach 1:
The patent segments the image processing task into two independent parts: processing the original image through the neural network and processing the copied image through the same network. This segmentation allows each classifier to operate independently and efficiently, maintaining processing speed while enabling adversarial detection through the comparison of the two separate classification outputs.
Solution Approach 2:
The patent applies partial action by processing only the necessary portions of the image (creating a copy with modified pixels) rather than processing every possible variation. This selective copying approach enables adversarial detection without the computational overhead of exhaustively analyzing all possible image variations, thus maintaining processing speed while improving detection reliability.
3Reliability
If the system processes only the input image, then computational resources are minimized, but performance on clean images may be compromised when faced with adversarial attacks
Solution Approach 1:
The patent creates a copy of the input image with modified pixels and processes this copy through the neural network. This copying mechanism enables the system to maintain high performance on clean images even under adversarial attacks by comparing the classification outputs, while the computational overhead is minimized by processing only a single additional image rather than multiple variations.
Solution Approach 2:
The patent changes a specific parameter of the image (creating a copy with modified pixels) to enable adversarial detection. This parameter change approach allows the system to maintain efficient processing by modifying only the necessary image parameters rather than reprocessing the entire image spectrum, thus improving reliability while minimizing computational resource consumption.
Data Source
AI summary
Mechanisms are provided to provide an improved computer tool for determining and mitigating the presence of adversarial inputs to an image classification computing model. A machine learning computer model processes input data representing a first image to generate a first classification output. A cohort of second image(s), that are visually similar to the first image, is generated based on a comparison of visual characteristics of the first image to visual characteristics of images in an image repository. A cohort-based machine learning computer model processes the cohort of second image(s) to generate a second classification output and the first classification output is compared to the second classification output to determine if the first image is an adversarial image. In response to the first image being determined to be an adversarial image, a mitigation operation by a mitigation system is initiated.