AI Model Protection Using Don't Know Class Query Locking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI systems are vulnerable to model stealing attacks, where adversaries iteratively query the system to extract internal information and build a parallel model, causing potential business disadvantages, loss of confidential information, and intellectual property theft.
Innovation Solution
Introduce a 'Don't Know Class' (DKC) to handle unclassifiable inputs, increment a counter for DKC occurrences, and lock the system when a threshold is reached to prevent model stealing by labeling unclassifiable queries as DKC and dynamically adjusting the counter based on user interaction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the AI system processes all incoming queries to generate classification outputs, then the model functionality is fully utilized and trained, but adversaries can exploit this to extract internal information and build parallel models through iterative queries
Solution Approach 1:
The system applies preliminary anti-action by introducing a 'Don't Know Class' (DKC) as a preventive measure before adversaries can extract model information. When the DKC counter reaches a threshold, the system proactively locks the model, preventing further extraction attempts. This anticipatory defense stops model stealing attacks before they can succeed, while still allowing legitimate classification operations to proceed normally during the grace period.
Solution Approach 2:
The DKC counter functions as an intermediary mechanism between the model and adversaries. It mediates by monitoring query patterns and controlling access to the model. The counter acts as a buffer that allows legitimate operations while gradually restricting access when suspicious patterns emerge, ultimately blocking adversaries without affecting normal users who remain below the threshold.
2Reliability
If the system locks the model to prevent model stealing, then intellectual property is protected, but legitimate users may be affected by false positives and lose access to the AI system
Solution Approach 1:
The system applies dynamics by making the DKC counter threshold adjustable rather than fixed. The threshold can be dynamically configured based on the specific AI system, attack risk assessment, and legitimate usage patterns. This allows organizations to balance protection strength against false positive rates, adjusting the threshold to lock the model only when truly necessary while maintaining accessibility for legitimate users.
Solution Approach 2:
The system changes parameters by allowing the DKC counter threshold to be modified based on different operating conditions. The threshold parameter can be adjusted to accommodate varying levels of security risk and legitimate query volumes. This parameter change capability enables the system to adapt to different scenarios, preventing model stealing when threats are present while avoiding false positives that would block legitimate access.
3Object-affected harmful factors
If the DKC counter threshold is set low to quickly prevent model stealing, then protection is more effective, but legitimate queries may be blocked due to normal variability in classification outcomes
Solution Approach 1:
The system applies preliminary action by establishing a DKC counter threshold before attacks occur, creating a safety margin that accounts for normal classification variability. This pre-configured threshold serves as a buffer zone that absorbs legitimate DKC occurrences while still detecting genuine attack patterns. The threshold is set in advance to balance sensitivity and specificity, preventing both model stealing and false positive blockages.
Data Source
Figure 1
Figure 2
AI summary
The invention discloses a system and a method to prevent unauthorized capturing of models in an Artificial Intelligence based system (Al system) (100). The method is comprising the steps: receiving an input (103) from an user; checking whether the input (103) corresponds to at least one of classes (107) in a data set; returning a class (107) which corresponds to the input (103), from the data set, if the input (103) corresponds to at least one of the classes (107) in the data set; returning a p re-defined class (116) (Don't Know Class - DKC) from the data set if the input does not correspond to any of the said classes in the data set.