Explainable AI Model Selection Architecture for Stable Explanations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial intelligence systems lack interpretability, particularly in sensitive determinations like credit scoring, making it difficult to comply with regulations such as GDPR and ECOA, and existing explanatory models provide unstable explanations.
Innovation Solution
A computer system selectively uses individual explanatory models, such as SHAP and LIME, based on evaluation metrics and rules to generate accurate and precise explanations for machine learning model outputs, and adjusts the models as needed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If complex tree-based models or deep learning models are used to improve AI system performance, then model accuracy and predictive power are improved, but model interpretability deteriorates making reasoning difficult to interpret
Solution Approach 1:
The patent introduces explanatory models as intermediary components that mediate between the complex black-box AI model and the user/regulator. These explanatory models translate the complex internal reasoning of tree-based or deep learning models into interpretable formats, allowing the system to maintain high accuracy while providing understandable explanations for decisions.
Solution Approach 2:
The patent segments the explanation generation process into multiple components including local explanatory models, global explanatory models, and evaluation metrics. This segmentation allows the complex interpretability challenge to be broken down into manageable parts that can be independently optimized and combined to provide comprehensive explanations.
2Device complexity
If existing explanatory models are used to provide interpretations, then model interpretability is improved, but explanation stability deteriorates providing unstable explanations
Solution Approach 1:
The patent merges multiple explanatory models (local and global) with multiple evaluation metrics to create a comprehensive explanation framework. By combining multiple sources of explanation and evaluating them against multiple criteria, the system achieves more stable and reliable explanations compared to using single explanatory models in isolation.
Solution Approach 2:
The patent implements feedback mechanisms where explanation quality is continuously evaluated using multiple metrics, and the explanatory models are adjusted based on this evaluation. This feedback loop ensures that explanations remain stable and reliable by identifying and correcting inconsistencies or instabilities in the explanation generation process.
3Measurement precision
If multiple explanatory models are evaluated and combined, then explanation accuracy is improved, but system complexity increases
Solution Approach 1:
The patent implements a dynamic framework where the selection and combination of explanatory models is not fixed but adapts based on the specific AI model being explained, the data characteristics, and the evaluation results. This dynamic approach allows the system to use only the necessary explanatory models for each case, reducing unnecessary complexity while maintaining high explanation accuracy.
Solution Approach 2:
The patent changes parameters such as the type of explanatory model used, the evaluation metrics applied, and the combination strategy based on the specific requirements of each explanation task. By adjusting these parameters dynamically, the system achieves high explanation accuracy without permanently increasing system complexity.
Data Source
AI summary
A system includes one or more processors to store a first explanatory model (e.g., a SHAP model or a LIME model) and a second explanatory model; execute the machine learning model (e.g., a neural network) using a first set of data to generate a first classification data point; generate a first plurality of explanatory evaluation metrics for the first explanatory model by applying the first explanatory model to the first classification data point; and responsive to the first plurality of explanatory evaluation metrics satisfying an explanatory model selection policy, apply the first explanatory model and the second explanatory model to a second classification data point output by the machine learning model based on a second set of transaction data.