Extracted Model Adversaries For Black Box Attack Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Reading comprehension machine learning models, particularly black box models, are challenging to correct due to their non-linear parameter interactions, leading to errors in interpretation and a lack of insight into their decision-making processes.
Innovation Solution
A system is developed to train a white box reading comprehension model to simulate a black box model by capturing its behavior, allowing for the identification and analysis of adversarial attacks, which are then used to improve the black box model's performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a black box reading comprehension model is used, then the model can process complex non-linear parameter interactions, but it becomes difficult to analyze and correct errors due to lack of transparency
Solution Approach 1:
The patent creates a simplified copy (extracted model) of the black box reading comprehension model that replicates its behavior on reading comprehension tasks. This extracted model maintains the functional capabilities of the original while being transparent and analyzable, allowing error analysis and correction without dealing with the full complexity of the black box model's internal parameters
Solution Approach 2:
The extracted model serves as an intermediary between the black box model and the analysis process. Instead of directly analyzing the opaque black box model, the patent uses the extracted model as a mediator that provides the same functional output but with visible and understandable internal workings, enabling systematic error identification and correction
2Reliability
If the number of parameters in the reading comprehension model is increased to improve performance, then the model can handle more complex interpretations, but correcting errors becomes more challenging
Solution Approach 1:
By creating an extracted model that copies the essential functionality of the large-parameter black box model, the patent enables error analysis in a simplified environment. The extracted model maintains interpretation accuracy while having fewer, more manageable parameters that can be systematically analyzed and corrected
Solution Approach 2:
The patent segments the error correction process by working with the extracted model separately from the original black box model. This allows focused analysis and correction of specific interpretation errors in the extracted model without needing to tackle the entire complex parameter space of the original model
Data Source
AI summary
Techniques are described for identifying successful adversarial attacks for a black box reading comprehension model using an extracted white box reading comprehension model. The system trains a white box reading comprehension model that behaves similar to the black box reading comprehension model using the set of queries and corresponding responses from the black box reading comprehension model as training data. The system tests adversarial attacks, involving modified informational content for execution of queries, against the trained white box reading comprehension model. Queries used for successful attacks on the white box model may be applied to the black box model itself as part of a black box improvement process.