Surrogate Hierarchical Neural Network for Interpretable ML Predictions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning models, especially black-box models, lack interpretability, making it difficult for non-technical humans-in-the-loop to understand their predictions, as existing explainability methods primarily provide feature-attribution explanations that are unintelligible to domain experts in sensitive contexts like Financial Crime and Healthcare.
Innovation Solution
A computer-implemented method and system that uses a surrogate hierarchical neural network to jointly mimic a machine-learning model classifier and provide concept explanations, employing a hierarchical multi-task network with a knowledge distillation task and an explainability task, where the network learns to predict domain concepts and align with the black-box model's predictions, using attention mechanisms and concept labels to generate high-level explanations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If complex machine learning models are used to improve prediction accuracy, then model performance is improved, but model interpretability deteriorates
Solution Approach 1:
The patent introduces a surrogate hierarchical model as an intermediary that translates the black-box model's predictions into human-interpretable concept explanations. This surrogate model acts as a mediator between the complex original model and human users, preserving prediction accuracy while providing interpretable concept-based explanations that reveal the reasoning behind predictions.
Solution Approach 2:
The patent creates a simplified copy (surrogate model) of the complex black-box model that replicates its prediction behavior while being structured to provide interpretable explanations. The surrogate model copies the predictive functionality but presents it in a human-understandable format using concept hierarchies, allowing users to understand the decision-making process without sacrificing accuracy.
2Loss of information
If feature-attribution explanations are provided to explain model predictions, then explanation detail is improved, but ease of understanding deteriorates
Solution Approach 1:
The patent transforms the explanation parameters from low-level feature attributions to high-level concept explanations. Instead of presenting raw feature importance scores that are technical and difficult to interpret, the system changes the parameter space to use domain-specific concepts and hierarchical relationships that are meaningful to human users, thereby improving ease of understanding while preserving explanatory power.
Solution Approach 2:
The patent adds a hierarchical dimension to explanations by organizing concepts in a hierarchy from general to specific. This dimensional transformation allows the system to provide both high-level conceptual understanding and detailed feature-level information when needed, making explanations accessible to non-technical users while maintaining comprehensive detail for those who require it.
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
The present document discloses a computer-implemented method for obtaining a surrogate hierarchical machine-learning model, trained to provide concept explanations for a machine-learning model classifier, the method comprising the steps of: obtaining input records and corresponding concept labels for each input record; feeding the obtained input records to the machine-learning model classifier; feeding the obtained input records to the surrogate hierarchical machine-learning model; comparing the determined class estimate from the output records of the surrogate hierarchical machine-learning model with the determined class estimate contained in the output records of the machine-learning model classifier to calculate a distillation loss; backpropagating a calculated gradient of the distillation loss to update weights of the surrogate hierarchical machine-learning model. It is also disclosed a respective computer system and a computer program product embodied in a non-transitory computer-readable medium.