Ensemble Output Explanation Using Contribution Matrices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Ensemble models operate as a 'black box,' lacking transparency in their decision-making processes, and current methods for explaining these models are slow and unsuitable for real-time applications, especially in the context of explainable artificial intelligence (XAI).
Innovation Solution
A stacked ensemble model with multiple layers of component models, generating feature importance scores through contribution matrices, and combining them to provide real-time explanations using a processor, which analyzes user data to identify and explain the decision-making process. The processor generates an overall feature importance matrix and selects top features to provide an explanation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If current explainer models are used to identify top contributing factors for ensemble models, then explanation capability is provided, but the explanation speed is slow and unsuitable for real-time applications
Solution Approach 1:
The patent segments the explanation generation process by computing contribution matrices separately for each component model within the ensemble, then combining them. This allows parallel computation of individual model contributions and enables real-time explanation generation while maintaining accurate attribution of predictions to specific features through structured matrix operations.
2Measurement precision
If ensemble models with stacked layers are used to improve prediction accuracy, then model performance is enhanced, but the complexity of explaining the decision-making process increases
Solution Approach 1:
The patent introduces contribution matrices as intermediary structures that mediate between the complex stacked ensemble model and the explanation output. Each component model's contribution to the final prediction is captured in a separate contribution matrix, which then combines these contributions systematically. This intermediary representation simplifies the explanation of complex multi-layer ensemble decisions by providing a structured view of feature importance at each layer.
3Reliability
If explainer models are retrained periodically to adapt to policy changes and data shifts, then explanation accuracy is maintained, but computational overhead and time consumption increase
Solution Approach 1:
The patent enables the explainer to adapt to policy changes and data shifts through self-service mechanisms where the contribution matrices are automatically updated by computing new feature importances on incoming data without requiring full retraining. The system continuously generates updated explanations by processing new data through the existing ensemble model structure, maintaining reliability while minimizing computational overhead through incremental updates rather than periodic retraining.
Data Source
AI summary
A method including applying a stacked ensemble model having a number of component models to a user profile. Values for the features are extracted from the user profile. A first contribution matrix, generated for the first model, contains first feature importance scores for the first subset of the features used in the first model. A second contribution matrix, generated for the second model, contains second feature importance scores for the second subset of the features used in the second model. An overall feature importance matrix is generated by combining the first contribution matrix and the second contribution matrix. A set of top features including a third subset of the features is selected from the overall feature importance matrix. An explanation for the final output is generated according to the set of top features. The explanation is presented.