Layered Neural Network Explainability via Relevance Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current artificial intelligence systems, particularly in decision-making processes like misappropriation detection and alert processing, lack effective methods for explainability and interpretability, leading to inverse proportionality between performance and explainability, and regulatory mandates for improved transparency.
Innovation Solution
An attention-based layered neural network classification system that uses a sequence of neural networks with varying rigor levels to calculate and integrate relevance scores, generating a total relevance score output and visualization maps, and employs machine learning-based techniques for logical grouping and refinement to enhance interpretability and explainability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional neural networks are used for decision-making, then processing speed and accuracy are improved, but explainability and interpretability deteriorate
Solution Approach 1:
The patent segments the neural network into multiple layers, where earlier layers focus on accuracy and later layers focus on explainability. Each layer processes relevance scores independently, allowing the system to maintain high processing speed in early layers while generating interpretability information in later layers without compromising overall decision speed.
Solution Approach 2:
The patent introduces relevance scores as an intermediary mechanism between the neural network's internal processing and the final decision output. These relevance scores act as a bridge that translates complex neural network computations into interpretable information about feature importance, enabling both fast processing and explainability.
2Loss of information
If multiple layers of neural networks are added to improve explainability, then system complexity increases
Solution Approach 1:
The patent applies local quality by assigning different functions to different layers: earlier layers are optimized for accurate classification while later layers are optimized for generating explainability information. This localized functional differentiation allows the system to improve explainability without requiring complete redesign of the entire network architecture.
Solution Approach 2:
The patent implements partial action by having only the later layers compute and output relevance scores, while earlier layers focus solely on accurate classification. This partial application of explainability computation to specific layers reduces overall computational complexity while still providing the needed interpretability in the final output.
3Loss of information
If relevance scores are calculated at each layer, then interpretability is improved, but computational overhead increases
Solution Approach 1:
The patent performs preliminary computation of relevance scores in earlier layers, storing these intermediate results for use in later layers. This preliminary action allows subsequent layers to leverage previously computed information rather than recalculating from scratch, reducing redundant computational energy consumption while maintaining comprehensive interpretability.
Solution Approach 2:
The patent selectively discards less important computational paths in earlier layers while recovering and emphasizing relevant feature importance information in later layers. By discarding redundant computations early and recovering only the essential relevance information needed for interpretability, the system reduces overall computational overhead while maintaining explainability quality.
Data Source
AI summary
A system for attention-based layered neural network classification is provided. The system comprises: a sequence of layered neural networks; and a controller configured for controlling data routed through the sequence of layered neural networks, the controller configured to: receive interaction data comprising data features, wherein the data features are distinct characteristics of the interaction data; input data features into the sequence of layered neural networks, wherein each sequential layer of the sequence of layered neural networks comprises a heightened rigor level for at least one of the data features; calculate a relevance score output for at least one of the data features at each layer of the sequence of layered neural networks; and integrate the relevance score output from each layer of the sequence of layered neural networks to generate a total relevance score output.