AI Explanation Evaluation by Feature Removal and Retraining
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional explainable artificial intelligence (XAI) methods face issues in accurately assessing feature importance and comparing different explanations for machine learning model decisions, leading to incorrect attribution of performance changes and difficulty in selecting reliable XAI techniques.
Innovation Solution
A method involving retraining a machine learning model on a modified dataset from which important features are dropped, allowing for improved evaluation of XAI techniques by comparing performance metrics before and after feature removal.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If data is replaced with perturbations to evaluate feature importance, then feature importance can be assessed, but model performance decreases due to drastic changes in data distributions
Solution Approach 1:
The patent extracts the feature to be evaluated from the dataset and creates a modified dataset without that feature. This allows assessment of feature importance by comparing model performance with and without the feature, avoiding the need to introduce perturbations that distort data distributions.
Solution Approach 2:
Instead of adding perturbations to existing data to assess feature importance, the patent inverts the approach by removing the feature entirely from the dataset. This inversion eliminates the harmful effect of perturbations on data distribution while still enabling feature importance evaluation.
2Loss of information
If conventional XAI techniques are used to explain model decisions, then explanations can be generated, but it is difficult to determine which explanation to trust when multiple techniques produce different results
Solution Approach 1:
The patent implements a feedback mechanism where multiple XAI techniques are applied to the same model and dataset, and their explanations are compared against each other. The technique whose explanations are most consistent with actual model behavior (as measured by performance changes when features are removed) is identified as the most reliable.
Solution Approach 2:
The patent creates a universal evaluation framework that can assess multiple different XAI techniques using the same methodology (feature removal and performance comparison). This multi-functional approach allows systematic comparison across different explanation techniques to identify the most reliable one.
Data Source
AI summary
A computing system may generate a first set of importance metrics (e.g., scores or values) for a model. The importance metrics may be generated using an explainable artificial intelligence technique, and an individual importance metric may indicate how influential a corresponding feature is for a decision made by a model. The computing system may determine an important feature and create a modified dataset by removing the important feature from the dataset. The computing system may train the model on the modified dataset and evaluate the performance of the model to determine the effect of removing the feature (e.g., which may indicate how important the feature is to output generated by the model). This process may be repeated for additional features and additional performance metrics may be obtained.