Classification Model Susceptibility Evaluation for Inversion Attacks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The susceptibility of machine-learned classification models to model inversion attacks is not well understood, posing risks to the security of data processing systems, as attackers can create replicas or infer decision boundaries, potentially leading to exploits or manipulated training data.
Innovation Solution
A method to determine the susceptibility of classification models to model inversion attacks by training a replica model using sample data, evaluating its properties, and providing a measure based on the number of samples and the replica's performance, allowing for iterative training and selection of security measures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a classification model is deployed with black-box access, then the model can be used by users to obtain classifications, but the model becomes vulnerable to model inversion attacks where attackers can create replicas or infer decision boundaries
Solution Approach 1:
The patent applies preliminary action by conducting model inversion attacks during the model development phase to evaluate susceptibility before deployment. The system proactively trains replica models and measures their accuracy to quantify vulnerability, allowing security measures to be implemented in advance rather than reacting to attacks after they occur.
Solution Approach 2:
The patent implements feedback by using the results of model inversion attacks to guide model improvement. The measured susceptibility (replica model accuracy) provides feedback that can be used to adjust the original model's training data, features, or architecture to reduce vulnerability while maintaining functionality.
2Ease of manufacture
If attackers obtain sufficient input-output pairs from the model, then they can train a replica of the classification model, but the cost and complexity of training a new model from scratch increases
Solution Approach 1:
The patent directly addresses the copying principle by studying how attackers create replica models from input-output pairs. The system evaluates the quality and quantity of data needed for successful replication, understanding the trade-off between the ease of model copying and the complexity of training a new model from scratch.
3Productivity
If the classification model is used to classify new data, then the model provides useful classifications, but the model may be used to infer details about decision boundaries and training data
Solution Approach 1:
The patent converts the harmful effect of information leakage into a beneficial evaluation mechanism. By deliberately measuring how much information about decision boundaries can be inferred from input-output pairs, the system quantifies vulnerability and uses this knowledge to improve model security without sacrificing classification functionality.
Data Source
AI summary
A computer implemented method, computer system and computer program are provided for determining a measure of susceptibility of a machine-learned classification model to model inversion attacks. The method obtains a set of sample data from the classification model, the set of sample data comprising a plurality of samples, each sample comprising a respective classification produced by the classification model for an associated set of input values. The method trains a replica of the classification model using the sample data. The method obtains a set of further sample data from the classification model, the set of further sample data comprising a plurality of further samples, each further sample comprising a respective classification produced by the classification model for an associated set of input values. The method evaluates a property of the replica of the classification model using the further sample data. The method provides a measure of the level of susceptibility of the classification model to model inversion attacks based on the number of samples in the sample data used to train the replica of the classification model and the property of the replica of the classification model.