Explainable AI for Non-Differentiable Models via Integrated Gradients
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial intelligence models, particularly non-differentiable ones, pose challenges in explainability due to their 'black box' nature, making it difficult to identify errors and improve models in applications like intent prediction, fraud detection, and cyber incident detection, as conventional explainable AI techniques are only applicable to differentiable models.
Innovation Solution
The integration of integrated gradients to generate numerical approximations for non-differentiable models, enabling the application of explainable AI by approximating gradients and integrals, and using a novel neural network architecture that computes conditional expectations to provide feature importance without assuming feature independence.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If non-differentiable AI models are used, then prediction accuracy is improved, but explainability deteriorates
Solution Approach 1:
The patent introduces integrated gradients as an intermediary mechanism that bridges non-differentiable models and explainability requirements. This intermediary computes approximate gradients through numerical differentiation, enabling XAI techniques to work with non-differentiable models without requiring actual differentiability of the model itself
Solution Approach 2:
The patent replaces the mechanical requirement of mathematical differentiability with a numerical approximation approach. Instead of relying on analytical gradients that require differentiable functions, the system uses finite difference methods to compute approximate gradients, substituting the mechanical constraint of differentiability with a computational approximation technique
2Loss of information
If conventional XAI techniques are applied, then explainability is improved, but model compatibility deteriorates
Solution Approach 1:
The patent changes the parameter requirements for XAI techniques by introducing numerical approximation methods. Instead of requiring models to have analytical gradients, the system changes the approach to gradient computation using finite differences, thereby expanding model compatibility to include non-differentiable models while maintaining explainability
3Adaptability or versatility
If numerical approximations are used, then model compatibility is improved, but computational complexity increases
Solution Approach 1:
The patent applies partial action by computing integrated gradients only at specific points in the model's input space rather than requiring full differentiability throughout the entire model. This selective approach to gradient computation enables compatibility with non-differentiable models while managing computational complexity through targeted approximation rather than comprehensive analysis
Data Source
AI summary
Methods and systems are described herein for novel uses and/or improvements to artificial intelligence applications. As one example, methods and systems are described herein related to adapting explainable artificial intelligence (XAI) to non-differentiable models (e.g., as used in intent prediction, fraud detection, and/or cyber incident detection). The systems and methods achieve this through the use of integrated gradients. For example, the systems and methods generate numerical approximations to gradients and integrals for non-differentiable models. These integrated gradients may then be used to apply XAI to non-differentiable models.