Shapley Value Animation for Model Understandability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning models lack transparency and explainability, as users cannot easily understand the features and learnings that contribute to their predictions, leading to insufficient insight into model robustness and reliability.
Innovation Solution
A computer-implemented method that trains models using multiple portions of a training dataset, generates Shapley values for each portion, and creates animations based on these values to provide enhanced understandability, utilizing synthetic Shapley values for smoother transitions and embedding spaces to visualize model decisions over time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine learning models are trained to achieve accurate predictions, then prediction accuracy is improved, but model explainability deteriorates
Solution Approach 1:
The patent segments the model training process into multiple portions of the training dataset, generating Shapley values for each portion separately. This segmentation allows users to understand the contribution of different data portions to the model's predictions, thereby maintaining explainability while achieving accurate predictions through comprehensive training.
Solution Approach 2:
The patent introduces Shapley values as an intermediary mechanism between the trained model and the user. These values serve as a mediator that translates the complex internal workings of the machine learning model into interpretable information about feature contributions, enabling users to understand model decisions without sacrificing prediction accuracy.
2Reliability
If comprehensive training data is used to improve model reliability, then model robustness is improved, but user understanding of model decisions deteriorates
Solution Approach 1:
The patent divides the comprehensive training data into multiple portions and processes them separately to generate corresponding Shapley values. This segmentation enables users to understand the contribution of each data portion to model decisions, making comprehensive training data manageable and interpretable while maintaining model robustness.
Solution Approach 2:
The patent transforms the complex parameter space of model decisions into Shapley values that represent feature contributions. This parameter transformation changes the representation of model behavior from opaque internal states to interpretable contribution metrics, enabling users to understand decisions while the model utilizes comprehensive training data for robustness.
3Loss of information
If detailed model training processes are maintained for explainability, then model transparency is improved, but computational complexity deteriorates
Solution Approach 1:
The patent extracts the essential explainability information from the complex model training process by generating Shapley values that summarize feature contributions. This extraction separates the transparency-providing component (Shapley values) from the computationally intensive model training, maintaining model transparency while reducing the burden of analyzing detailed training processes.
Data Source
AI summary
Embodiments utilize interpolation to smooth animation frames by generating synthetic Shapley values utilized to generate the animation. Additionally, some embodiments provide for generation of an embedding space for improved understandability of the model training based on corresponding Shapley values. The embedding space is mapped to using activations derived from a specially configured LSTM for each model, where an embedded representation of the LSTM activations is generated by a specially configured autoencoder such that the embedded representation may be mapped to the embedding space. The embedding space and animations are renderable to an interface for user investigation and may be further customizable to narrow investigation on particular data thereof.