Training Example Selection for Interpretable ML Decisions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The complexity of autonomous and semi-autonomous systems has led to a lack of understanding in their decision-making processes, causing trust issues and difficulties in correcting errors, as existing methods are inefficient and often impossible for non-probabilistic models.
Innovation Solution
A method and system that generate a subset of training data examples using Bayesian inference to explain decisions made by machine learning programs, selecting subsets based on probability and suitability scores to effectively teach users about the decision-making process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine learning systems become more complex to improve decision accuracy, then decision accuracy is improved, but understandability of the decision-making process deteriorates
Solution Approach 1:
The patent introduces an intermediary system that translates complex machine learning decisions into understandable examples. The system selects and presents training examples that mediate between the complex internal decision-making process and user comprehension, allowing users to understand decisions without needing to grasp the underlying complex algorithms.
Solution Approach 2:
The patent segments the complex decision-making process into discrete, understandable examples. By breaking down the decision-making into individual training examples that can be presented separately, the system makes the complex process comprehensible while maintaining the accuracy of the underlying complex model.
2Loss of information
If traditional example-based analysis methods are used to understand system decisions, then some understanding can be gained, but the process becomes extremely inefficient requiring vast time and numerous examples
Solution Approach 1:
The patent performs preliminary action by pre-selecting and organizing training examples that are most informative for understanding system decisions. Instead of requiring users to manually analyze numerous examples over time, the system beforehand identifies and presents the most relevant examples, dramatically reducing the time needed to gain understanding.
Solution Approach 2:
The patent replaces the mechanical process of manual example analysis with an automated system that uses algorithms to select and present informative examples. This substitution transforms the inefficient manual process into an automated system that quickly provides understanding through strategically selected examples.
3Productivity
If non-probabilistic learning models are used to improve system performance, then system performance is improved, but the ability to directly interpret decision-making processes deteriorates
Solution Approach 1:
The patent introduces an intermediary interpretation layer that translates the decisions of non-probabilistic models into understandable examples. This mediator allows users to interpret decisions from high-performance non-probabilistic models without needing to understand the complex internal mechanics, maintaining both performance and interpretability.
Data Source
AI summary
A method of generating a set of examples for explaining decisions made by a machine learning program, involving receiving a set of training data for training the program, and for given subsets of the training data, determining each of (a) a probability of a user correctly inferring a future decision of the program after observing the respective decisions of the program for the given subset of the training data, (b) a suitability of a size of the given subset, and (c) an average probability of the user correctly inferring a future decision of the program after observing the respective decisions of the program for an unspecified subset of the training data. The determinations (a), (b) and (c) are used to score each of the given subsets of training data, and a subset of training data is selected as the generated set of examples based on the scores.