Global Attribution Mapping for Neural Network Interpretability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current global attribution methods in neural networks reduce decisions to a single set of features, failing to provide comprehensive insights into feature importance and potential bias across subpopulations.
Innovation Solution
The proposed Global Attribution Mapping (GAM) technique generates global attributions by normalizing and comparing local attributions, applying clustering algorithms like K-medoids to identify clusters of similar attributions, and providing tunable granularity to capture various subpopulations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If global attribution methods reduce decisions to a single set of features, then the model complexity is reduced, but the measurement precision of feature importance across subpopulations deteriorates
Solution Approach 1:
The patent segments the global attribution analysis by dividing the data into multiple subpopulations based on feature thresholds. Instead of providing a single unified feature importance ranking, the system creates multiple segmented views (e.g., high-risk and low-risk subpopulations) each with its own feature attribution. This segmentation allows precise measurement of feature importance across different subpopulations while maintaining manageable model complexity through structured analysis.
Solution Approach 2:
The patent introduces an additional dimension to feature importance measurement by analyzing interactions between features across different subpopulations. The system extends traditional single-dimensional feature importance rankings to a multi-dimensional framework that captures how feature importance varies across different segments of the data space, thereby improving measurement precision without proportionally increasing model complexity.
2Measurement precision
If local attributions are generated for each sample, then the measurement precision of feature importance is improved, but the quantity of data processing increases
Solution Approach 1:
The patent merges similar local attributions across multiple samples into unified global attributions for each subpopulation. By clustering and aggregating local attribution results, the system reduces the total quantity of individual data processing while preserving the precision of feature importance measurements. The merging process consolidates redundant information and creates representative global attribution profiles that capture essential patterns across the entire dataset.
Solution Approach 2:
The patent changes the parameter representation from individual sample-level attributions to population-level aggregated attributions. By transforming local attribution data into global attribution parameters for each subpopulation, the system reduces the quantity of data points while maintaining measurement precision through statistical aggregation and representation of feature importance at the population level.
3Adaptability or versatility
If clustering algorithms are applied to local attributions, then the adaptability to different subpopulations is improved, but the device complexity increases
Solution Approach 1:
The patent applies clustering algorithms to a selective subset of local attributions rather than all data points. By identifying and clustering only the most informative or representative local attributions, the system achieves adaptability to different subpopulations while reducing processing complexity. This partial action approach allows the system to capture essential subpopulation patterns without the computational burden of processing every individual data point through clustering.
Data Source
AI summary
Embodiments include techniques to determine a set of credit risk assessment data samples, generate local credit risk assessment attributions for the set of credit risk assessment samples, and normalize each local credit risk assessment attribution of the local credit risk assessment attributions. Further, embodiments techniques to compare each pair of normalized local credit risk assessment attributions and assign a rank distance thereto proportional to a degree of ranking differences between the pair of normalized local credit risk assessment attributions. The techniques also include applying a K-medoids clustering algorithm to generate clusters of the local risk assessment attributions, generating global attributions, and determining insights for the neural network based on the global attributions.