AI Model Poisoning Detection via Risk Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
AI models can become untrustworthy if poisoned training data is introduced, leading to computationally expensive re-training processes that disrupt inference generation and other services.
Innovation Solution
Implement a system that analyzes training data sets for risk using a risk analysis process, which generates a severity level of risk and a magnitude of impact, allowing for decisions on whether to re-train the AI model or discard suspicious data sets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If AI models are re-trained using new training data to improve model updates and adaptability, then the AI model can learn new patterns and improve performance, but poisoned training data can be introduced leading to untrustworthy models and computationally expensive re-training processes
Solution Approach 1:
The system performs preliminary risk analysis on training data sets before they are used to re-train AI models. The risk analysis process evaluates the training data set to generate a severity level of risk and magnitude of impact, allowing the system to identify and intercept poisoned training data before it can compromise the model, thus preventing untrustworthy model updates while maintaining adaptability
Solution Approach 2:
A risk analysis process acts as an intermediary between the training data set and the AI model re-training process. This intermediary evaluates the training data set, generates risk metrics, and provides recommendations on whether to proceed with re-training, thereby protecting the model from poisoned data while enabling safe updates
2Reliability
If risk analysis is performed on training data sets to detect malicious attacks, then the trustworthiness of AI model inferences is maintained, but additional computational overhead and processing time are required
Solution Approach 1:
The risk analysis process is segmented into distinct operational components: evaluating the training data set, generating a severity level of risk, generating a magnitude of impact, and providing recommendations. This segmentation allows the system to perform comprehensive security analysis while organizing complexity into manageable, modular steps that can be executed systematically
3Measurement precision
If comprehensive risk analysis is performed on training data sets to detect malicious attacks, then the severity level of risk and magnitude of impact can be determined, but the processing time and computational resources increase
Solution Approach 1:
The risk analysis process generates feedback in the form of a severity level of risk and magnitude of impact, along with recommendations on whether to proceed with re-training. This feedback mechanism enables precise risk detection and measurement while providing actionable insights that help decision-makers quickly determine the appropriate course of action, reducing the time lost in uncertainty
Data Source
AI summary
Methods and systems for detecting malicious attacks targeting an artificial intelligence (AI) model are disclosed. To detect malicious attacks targeting an AI model, data sets and data sources from which the data sets are obtained may be analyzed. A severity level of risk associated with using the data set to re-train the AI model may be obtained. If the severity level of risk exceeds a re-training risk threshold, the data set may not be used to re-train the AI model. To reduce the likelihood of generating tainted inferences that may significantly impact operation of a downstream consumer, a magnitude of impact of potential inferences derived from the data set may be obtained. If the magnitude of impact of potential inferences derived from the data set exceeds a magnitude threshold, the data set may be discarded without generating any inferences.