LIME Explanation System for Black-Box AI Without Training Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Decision systems, particularly those using AI and machine learning, are difficult to interpret due to their opaque nature, making it challenging for explanation systems like LIME to determine why specific outputs are generated when training data is unavailable or not used.
Innovation Solution
A process is introduced to generate modified input data that acts as training data, allowing LIME systems to analyze decision systems by perturbing the original input data and using these modified inputs to determine feature importance, even when training data is not accessible.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If decision systems use AI and machine learning models to provide automated decisions, then productivity and decision-making efficiency are improved, but the system becomes difficult to interpret and understand
Solution Approach 1:
The patent introduces LIME (Local Interpretable Model-agnostic Explanations) as an intermediary system that sits between the complex decision system and users. LIME generates simplified explanations by creating perturbed versions of input data and analyzing how changes affect outputs, making the black-box decision system interpretable without modifying the original AI model's core functionality.
2Device complexity
If explanation systems like LIME are applied to decision systems, then transparency and interpretability are improved, but training data availability is required which may not be accessible
Solution Approach 1:
The patent creates synthetic training data by generating perturbed copies of the original input data. Instead of requiring access to the actual training data used to train the decision system, LIME creates artificial samples by adding noise and perturbations to input features, then uses these copies to train local interpretable models that explain the decision system's behavior.
3Measurement precision
If LIME systems use perturbed input data to generate explanations, then explanation accuracy is improved, but additional data processing and computation are required
Solution Approach 1:
The patent applies partial action by focusing explanations on local regions of the input space rather than attempting to explain the entire decision system globally. LIME generates perturbations only around specific input points of interest, creating locally accurate explanations without the computational burden of global interpretation, thus achieving sufficient precision with reduced complexity.
Data Source
AI summary
A method includes receiving input data that is used by a machine learning model to generate a first inference. The machine learning model was previously trained using a set of training data. The method also includes generating a set of modified input data based on the input data. The set of modified input data is generated without using the training data. The method further includes obtaining, from an explanation system, an explanation array. The explanation system is to generate the explanation array based on the machine learning model and the set of modified input data.