Counterfactual Explanations Without Training Data Using Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing counterfactual explanation (CFE) methods require access to training datasets, which can be inaccessible due to privacy concerns, especially in high-impact domains like healthcare and finance, and lack flexibility in specifying user preferences for feature adjustments.
Innovation Solution
A reinforcement learning-based method, Counterfactual Explanation Without Training Datasets (CFWoT), that adjusts model inputs iteratively using a neural network to generate actionable, valid, and plausible counterfactual examples without requiring training data, applicable to both static and multivariate time-series data with continuous and discrete features, including non-differentiable models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing counterfactual explanation methods are used, then model explanations can be generated, but training datasets are required which may be inaccessible due to privacy concerns
Solution Approach 1:
The patent extracts and removes the dependency on training datasets from the counterfactual explanation process. By formulating CFE as a reinforcement learning problem that only requires the trained model itself, the method extracts the essential explanation capability while leaving behind the privacy-sensitive training data requirement, thus resolving the contradiction between explanation availability and data accessibility
Solution Approach 2:
The patent introduces an intermediary reinforcement learning agent that mediates between the user's explanation request and the trained model. This agent learns to generate counterfactual explanations by interacting with the model through its API, without direct access to training data, thus enabling explanation generation while maintaining data privacy boundaries
2Adaptability or versatility
If reinforcement learning is used for counterfactual explanations, then model-agnostic and scalable explanations can be generated, but the method requires access to training datasets
Solution Approach 1:
The patent extracts the training data dependency from the reinforcement learning-based CFE process. By designing the RL agent to learn solely from interactions with the trained model's predictions, the method removes the need for training dataset access while preserving model-agnostic capabilities and scalability across different model types
Solution Approach 2:
The trained model serves itself by providing predictions to the reinforcement learning agent during the explanation generation process. The RL agent learns to generate counterfactuals by querying the model's own behavior, enabling the system to be self-sufficient without external training data, thus reducing data access complexity
Data Source
AI summary
When ML methods are responsible for making critical decisions, stakeholders often require insights into how to alter these decisions. Counterfactual explanations (CFEs) have emerged as a solution, offering interpretations of opaque ML models and providing a pathway to transition from one decision to another. However, most existing CFE methods require access to a training dataset which was used to train the underlying model and from which an explanation is drawn. Counterfactual explanations can be successfully generated without training dataset through the use of a neural network to determine adjustments to inputs. The neural network can be trained using reinforcement learning techniques.