AI Model Trustworthiness via Anomalous Data Root Cause Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial intelligence models' trustworthiness is compromised by anomalous data, leading to unreliable predictions, as existing methods fail to effectively identify and address issues like class imbalance, confused classes, and low density regions within the training data.
Innovation Solution
A computer-implemented method that identifies anomalous data points, determines their root causes, and adjusts the training data by relabeling or adding new data points to improve model confidence, using techniques such as Local Interpretable Model-Agnostic Explanations (LIME) to understand and address adversarial, class imbalance, confused class, and low density issues.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If machine learning models are trained using existing methods, then training can be completed, but the predictions become unreliable when anomalous data is present
Solution Approach 1:
The patent segments the training data into normal and anomalous data points, and further segments anomalous data into different root cause categories (class imbalance, confused class, low density, adversarial). This segmentation allows targeted handling of each type of anomaly rather than treating all anomalies uniformly, improving prediction reliability while managing complexity through structured analysis.
Solution Approach 2:
The patent changes parameters of the training data by adjusting class weights, modifying data density distributions, and transforming adversarial examples. These parameter changes directly address the root causes of anomalies, converting unreliable training scenarios into reliable ones, thereby improving prediction reliability without requiring fundamentally new model architectures.
2Reliability
If anomalous data points are identified and addressed, then model trustworthiness improves, but the processing time and computational resources increase
Solution Approach 1:
The patent performs preliminary identification and categorization of anomalous data points before model training. By detecting and addressing anomalies in advance (relabelling, removing, or augmenting around them), the system prevents anomalies from degrading model performance during training, thereby improving trustworthiness while minimizing the need for lengthy post-training corrections or retraining cycles.
Solution Approach 2:
The system automatically identifies anomalous data points, determines their root causes, and applies appropriate corrections without requiring extensive manual intervention. This self-service approach to anomaly handling reduces processing time by eliminating manual data review and correction steps, while still achieving comprehensive anomaly addressing to maintain model trustworthiness.
3Manufacturing precision
If the training data is adjusted to remove anomalies, then prediction accuracy improves, but the complexity of data preparation increases
Solution Approach 1:
The patent applies different data preparation techniques to different types of anomalous data points based on their specific root causes. For example, class imbalance issues are addressed through selective data augmentation or re-sampling, while adversarial examples are handled through detection and removal or transformation. This localized approach to data preparation improves prediction accuracy by targeting specific issues without unnecessarily complicating the entire data preparation pipeline.
Solution Approach 2:
The patent modifies specific parameters of the training data such as class weights, data densities, and feature distributions to correct anomalies. These parameter changes are applied selectively to affected data regions rather than transforming the entire dataset, thereby improving prediction accuracy while keeping data preparation complexity manageable through targeted rather than universal modifications.
Data Source
AI summary
Methods, systems, and computer program products for improving trustworthiness of artificial intelligence models in presence of anomalous data are provided herein. A method includes obtaining a machine learning model and a set of training data; determining one or more anomalous data points in said set of training data; for a given one of said anomalous data points, identifying attributes that decrease confidence with respect to at least one output of said machine learning model; determining that a root cause of said decreased confidence corresponds to one of: a class imbalance issue related to said at least one attribute, a confused class issue related to said at least one attribute, a low density issue related to said at least one attribute, and an adversarial issue related to said at least one attribute; and performing step(s) to improve said confidence based at least in part on said determined root cause.