MLaaS Confidence Thresholding Against Query-Based Adversarial Attacks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine-learning models are vulnerable to adversarial attacks, particularly in safety-critical domains like autonomous driving and facial recognition, where adversarial samples can cause misclassification with imperceptible perturbations, and current defenses are inadequate against adaptive black-box attacks.
Innovation Solution
Implement a defense mechanism that returns a random or pseudo-random output for low-confidence input samples, using a cryptographic key-based randomness to thwart gradient estimation in query-based attacks, and ensures consistent output for similar inputs, employing a confidence thresholding approach.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If machine-learning models are deployed in safety-critical domains, then automatic predictions can be made, but the models become vulnerable to adversarial attacks causing misclassification
Solution Approach 1:
The patent applies preliminary anti-action by implementing a defense mechanism that proactively counteracts adversarial attacks before they can successfully misclassify inputs. The system detects adversarial patterns and applies countermeasures such as randomizing predictions for low-confidence samples, thereby preventing the attack from achieving its goal of causing misclassification in safety-critical domains.
2Reliability
If current defense mechanisms are implemented, then some protection against adversarial samples is provided, but they are inadequate against adaptive black-box attacks
Solution Approach 1:
The patent applies dynamics by implementing a adaptive defense mechanism that changes its behavior based on the characteristics of incoming inputs. The system dynamically adjusts its response by detecting confidence levels and adversarial patterns, then adapting its countermeasures accordingly. This dynamic approach allows the defense to effectively counter adaptive black-box attacks by continuously adjusting its protection strategy based on real-time analysis of input samples.
3Reliability
If a defense mechanism returns random output for low-confidence inputs, then query-based attacks are thwarted, but the system complexity increases
Solution Approach 1:
The patent applies parameter changes by modifying the output behavior of the machine-learning model based on confidence threshold parameters. The system changes its output from deterministic predictions to randomized predictions when confidence levels fall below a threshold, thereby thwarting query-based attacks. This parameter-based approach allows the system to maintain simplicity in its core architecture while introducing targeted complexity only where needed to counter adversarial threats.
Data Source
AI summary
A method for thwarting attacks on a machine-learning (ML) model is provided. The method includes determining, by the ML model, a classification vector based on an input. The method further includes evaluation the classification vector based on a threshold parameter to determine a threshold result. The method also includes outputting a classification prediction based on the threshold result.