Stable Prescriptive Policy Construction via Counterfactual Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Decision trees in prescriptive analytics are highly unstable, leading to challenges in implementing and managing rules, as slight changes in input data result in drastically different tree structures, and existing notions of stability do not consider the tree structure or policy content, causing interpretability and trust issues.
Innovation Solution
A method is developed to quantify stability for policies, optimizing the underlying objective while maintaining stability by using a computer-implemented approach that identifies decision trees, detects new data, identifies counterfactual data, and generates updated decision trees based on new and counterfactual data, incorporating a similarity metric to ensure consistency and interpretability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If decision trees are used for prescriptive analytics, then interpretability is improved, but stability deteriorates due to high sensitivity to input data changes
Solution Approach 1:
The patent applies preliminary action by generating counterfactual data before updating the decision tree. When new data arrives, the system first identifies what counterfactual data would be needed to maintain stability, then generates this counterfactual data in advance. This preliminary preparation allows the system to proactively adjust the decision tree in a controlled manner, preventing unstable changes while maintaining interpretability.
Solution Approach 2:
The patent changes the parameter of data representation by introducing counterfactual data as an additional dimension. Instead of simply adding new data points to the training set, the system transforms the data by creating counterfactual versions that represent alternative outcomes. This parameter transformation stabilizes the decision tree by providing a more comprehensive view of data relationships, reducing sensitivity to small changes while preserving the tree's interpretability.
2Adaptability or versatility
If decision trees are updated with new data, then adaptability is improved, but stability deteriorates due to structural changes
Solution Approach 1:
The patent implements feedback by using counterfactual data to evaluate the impact of new data on the decision tree before applying updates. The counterfactual data serves as a feedback mechanism that simulates potential changes and allows the system to assess whether updates would maintain stability. This feedback loop enables the system to adapt to new data while controlling structural changes, preventing instability.
Solution Approach 2:
The system performs preliminary analysis by generating counterfactual data before actually updating the decision tree. This preliminary action allows the system to pre-evaluate the potential impact of new data and plan updates in a way that maintains stability. By preparing counterfactual scenarios in advance, the system can make informed decisions about how to update the tree structure, balancing adaptability with stability.
3Device complexity
If existing stability notions are used, then computational simplicity is maintained, but policy content and tree structure are not considered
Solution Approach 1:
The patent segments the stability evaluation process into distinct components: identifying new data, generating counterfactual data, evaluating policy outcomes, and updating the decision tree. This segmentation allows the system to handle complexity in manageable parts, maintaining computational simplicity while improving reliability. Each segment can be processed independently, making the overall system more reliable without requiring a complete redesign of stability notions.
Data Source
AI summary
A computer implemented method includes identifying, by one or more processors, a decision tree corresponding to an artificial intelligence model, detecting, by one or more processors, new data associated with an update to the identified decision tree, identifying, by one or more processors, counterfactual data corresponding to the new data, identifying, by one or more processors, one or more expected outcomes corresponding to the counterfactual data and the new data, and generating, by one or more processors, an updated decision tree based on the identified new data and the identified counterfactual data. A computer program product and computer system corresponding to the method are also disclosed.