Model-Agnostic Explanations With Adaptive Neighborhood Sampling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for providing local explanations for closed box analytical models are unfaithful and unstable, leading to high infidelity and inconsistency in explaining model outputs, as they fail to accurately replicate the behavior of the closed box model and are sensitive to distribution widths.
Innovation Solution
An adaptive neighborhood sampling (ANS) framework that determines an uncertainty score for a distribution of samples around a selected input, identifies a subset based on this score, and generates explanations using a local interpretable model, ensuring faithfulness and stability across different neighborhood widths.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a sparse model is built by perturbing the input or sampling around the input to create a distribution neighborhood, then explanations can be generated for closed box models, but the explanations are unfaithful and unstable, leading to high infidelity and high variation in quality
Solution Approach 1:
The patent changes the parameter of distribution width adaptively based on uncertainty scores. Instead of using a fixed distribution width, the system adjusts the width dynamically to achieve optimal balance between faithfulness and stability. This resolves the contradiction by making the explanation generation process adaptive to the specific characteristics of each input and model, thereby improving reliability while maintaining ease of operation.
Solution Approach 2:
The patent implements feedback through uncertainty scores that are calculated based on the distribution of samples and used to adjust the sampling process. The uncertainty score feedback loop allows the system to iteratively improve the quality of explanations, reducing infidelity and variation while maintaining the ability to generate explanations for closed box models.
2Adaptability or versatility
If the distribution width is varied to explore different neighborhood ranges, then more comprehensive sampling can be achieved, but the quality and consistency of explanations vary significantly
Solution Approach 1:
The patent applies dynamics by making the distribution width adjustable and adaptive rather than fixed. The system dynamically adjusts the distribution width based on uncertainty scores and input characteristics, allowing it to adapt to different sampling scenarios. This dynamic adjustment resolves the contradiction by enabling comprehensive sampling coverage while maintaining consistent explanation quality through adaptive control.
Solution Approach 2:
The patent changes the parameter of distribution width from a static value to a dynamic variable that adapts to different sampling scenarios. By implementing parameter changes based on uncertainty scores and input characteristics, the system achieves both comprehensive sampling coverage and consistent explanation quality, resolving the contradiction between adaptability and reliability.
3Productivity
If a fixed distribution width is used for sampling, then the sampling process is simple and fast, but the explanations show high variation in quality and infidelity across different inputs
Solution Approach 1:
The patent transitions from a static fixed distribution width to a dynamic adaptive width that adjusts based on uncertainty scores. This dynamic approach maintains computational efficiency while significantly improving the consistency of explanation quality. The system can quickly compute uncertainty scores and adjust the distribution width accordingly, preserving productivity while enhancing reliability.
Solution Approach 2:
The patent implements parameter changes in the distribution width based on uncertainty scores and input characteristics. This allows the system to maintain fast sampling speeds for simple cases while adapting to complex cases that require more comprehensive sampling. The parameter change mechanism resolves the contradiction by enabling the system to achieve both high productivity and consistent explanation quality through intelligent parameter adjustment.
Data Source
AI summary
One or more systems, devices, computer program products and/or computer-implemented methods of use provided herein relate to a process for providing an explanation result for an analytical model. A system can comprise a memory that stores computer executable components, and a processor that executes the computer executable components stored in the memory, wherein the computer executable components can comprise an uncertainty component that determines an uncertainty score for a distribution of samples that neighbor a selected input to an analytical model, a sampling component that identifies a subset of the distribution of samples based on the uncertainty score, and an explanation component that generates an explanation of an output of the analytical model, corresponding to the selected input, based on use of a sample from the subset of the distribution of samples.