Local Permutation Importance for Stable ML Feature Attribution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Complex deep learning models are challenging to interpret due to their increased complexity, leading to a need for efficient local explanation techniques that can accurately explain predictions without requiring a large number of input instances, especially in high-dimensional spaces where global explanations have low accuracy and are computationally expensive.
Innovation Solution
A perturbation-based approach that samples feature values from the empirical marginal distribution of a reference dataset to generate perturbed tuples, allowing for linear-time local feature attribution, reducing the number of instances needed for explanations, and increasing accuracy by focusing on realistic data distributions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If perturbation-based local explanation techniques are used to explain complex deep learning models, then interpretability is improved, but computational cost increases exponentially with the number of features
Solution Approach 1:
The patent extracts only the necessary information for local explanation by sampling a small number of perturbed instances rather than evaluating all possible feature combinations. This extraction approach obtains sufficient explanatory power without computing the full exponential space of feature interactions.
Solution Approach 2:
The patent applies partial action by using a limited number of perturbed instances (k instances) rather than exhaustively exploring all possible perturbations. This partial sampling provides adequate local explanation while avoiding the exponential computational burden of complete evaluation.
2Adaptability or versatility
If global explanation techniques are used to explain model behavior, then general understanding is improved, but accuracy decreases in high-dimensional spaces with large spatial gaps
Solution Approach 1:
The patent applies local quality by providing explanations tailored to specific instances rather than global explanations. Each local explanation is generated by perturbing the specific instance being explained, ensuring high accuracy for that particular point in the high-dimensional space without being affected by spatial gaps between training instances.
3Measurement precision
If the number of input instances is increased to improve explanation accuracy, then measurement precision is improved, but the number of instances required grows exponentially with feature dimensionality
Solution Approach 1:
The patent uses partial action by sampling a small fixed number of perturbed instances (k) for each explanation rather than requiring a large number of instances that grows exponentially with features. This partial sampling provides sufficient statistical evidence for local feature importance without the exponential instance requirement.
Data Source
AI summary
In an embodiment, a computer hosts a machine learning (ML) model that infers a particular inference for a particular tuple that is based on many features. For each feature, and for each of many original tuples, the computer: a) randomly selects many perturbed values from original values of the feature in the original tuples, b) generates perturbed tuples that are based on the original tuple and a respective perturbed value, c) causes the ML model to infer a respective perturbed inference for each perturbed tuple, and d) measures a respective difference between each perturbed inference of the perturbed tuples and the particular inference. For each feature, a respective importance of the feature is calculated based on the differences measured for the feature. Feature importances may be used to rank features by influence and/or generate a local ML explainability (MLX) explanation.