Locally Interpretable Model Training via Instance Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
There is a trade-off between performance and interpretability in machine learning models, where high interpretability leads to degraded performance, and existing locally interpretable models suffer from underfitting due to representational capacity differences compared to black-box models.
Innovation Solution
A reinforcement learning-based method is employed to train locally interpretable models by selecting high-value training instances using an instance-wise weight estimator, which determines selection probabilities based on a reinforcement signal to match the performance of black-box models while maintaining interpretability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If black-box machine learning models are used to achieve high prediction performance, then performance is improved, but interpretability deteriorates
Solution Approach 1:
The training dataset is segmented into two distinct sets: auxiliary training samples generated from the black-box model, and probe training samples used for final model training. This segmentation allows the system to leverage the performance of black-box models while training interpretable models on carefully selected subsets, thereby resolving the contradiction between performance and interpretability.
Solution Approach 2:
The instance-wise weight estimator model assigns different selection probabilities to different training samples based on their individual characteristics and relevance to the black-box model's predictions. This local quality approach ensures that each sample is weighted appropriately, allowing the interpretable model to capture local decision-making patterns while maintaining overall performance.
2Ease of operation
If locally interpretable models are trained with all training samples to improve interpretability, then interpretability is maintained, but performance deteriorates due to underfitting
Solution Approach 1:
The system extracts the most valuable training information from the black-box model by generating auxiliary training samples that capture the decision-making patterns. These extracted samples are then used to train the interpretable model, allowing it to achieve high performance without being overwhelmed by the full complexity of the original training dataset, thus preventing underfitting while maintaining interpretability.
Solution Approach 2:
The black-box model is trained first on the complete training dataset to learn complex patterns. Then, auxiliary training samples are generated from this pre-trained model before training the interpretable model. This preliminary action allows the interpretable model to benefit from the patterns already learned by the black-box model, improving its performance without requiring access to all original training samples.
3Reliability
If the representational capacity of locally interpretable models is increased to match black-box models, then performance is improved, but model complexity and loss of interpretability increase
Solution Approach 1:
Instead of increasing the representational capacity of the interpretable model to match the black-box model, the system creates copies of the decision-making patterns through auxiliary training samples. The interpretable model learns from these copied patterns rather than attempting to replicate the complex structure of the black-box model, thereby maintaining simplicity and interpretability while achieving comparable performance.
Data Source
AI summary
A method for training a locally interpretable model includes obtaining a set of training samples and training a black-box model using the set of training samples. The method also includes generating, using the trained black-box model and the set of training samples, a set of auxiliary training samples and training a baseline interpretable model using the set of auxiliary training samples. The method also includes training, using the set of auxiliary training samples and baseline interpretable model, an instance-wise weight estimator model. For each auxiliary training sample in the set of auxiliary training samples, the method also includes determining, using the trained instance-wise weight estimator model, a selection probability for the auxiliary training sample. The method also includes selecting, based on the selection probabilities, a subset of auxiliary training samples and training the locally interpretable model using the subset of auxiliary training samples.