Process-Aware Neighborhood Sampling for Black-Box Model Explanation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing explanation methods for AI business process applications, such as LIME and SHAP, provide potentially misleading explanations when applied to business process management models due to their inability to account for process-specific constraints and rules, leading to out-of-distribution issues.
Innovation Solution
A process-aware neighborhood sampling procedure is introduced, which uses business process rules to determine a suitable neighborhood around a data sample and computes proximity using a specific distance metric, allowing for the derivation of a local linear model that mirrors the behavior of a black-box machine learning model, thereby providing more accurate explanations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If existing explanation methods (LIME, SHAP) are applied to business process management models, then explanation capability is provided, but the explanations become potentially misleading due to inability to account for process-specific constraints
Solution Approach 1:
The patent applies local quality by making the explanation method adaptive to local process characteristics. Instead of using a uniform sampling approach, the system defines process-aware neighborhoods that respect local business process constraints and rules. This allows the explanation to be accurate locally within process boundaries while maintaining adaptability to different process domains.
Solution Approach 2:
The patent changes the parameters of the sampling procedure by introducing process-aware distance metrics and process-aware neighborhood definitions. These parameter changes transform the standard LIME/SHAP approaches into process-aware versions that account for business process constraints, thereby improving explanation accuracy without sacrificing adaptability.
2Measurement precision
If process-aware neighborhood sampling is used to improve explanation accuracy, then coefficient mismatch scores are reduced, but computational complexity increases due to process rule evaluation
Solution Approach 1:
The patent applies preliminary action by pre-defining process-aware distance metrics and neighborhood structures before performing the explanation task. By preparing the process-aware sampling framework in advance, the system reduces the computational burden during actual explanation generation, as the complex process rule evaluations are performed once during neighborhood definition rather than repeatedly during model approximation.
3Productivity
If standard neighborhood sampling is used, then computational efficiency is maintained, but explanation accuracy deteriorates due to out-of-distribution issues
Solution Approach 1:
The patent applies dynamics by making the neighborhood sampling procedure adaptive to the local data distribution and process constraints. Instead of using fixed, static neighborhoods, the system dynamically defines process-aware neighborhoods that adjust to the specific characteristics of the data sample and business process context, thereby improving explanation accuracy while maintaining reasonable computational efficiency through targeted sampling.
Data Source
AI summary
A computer-implemented method, a computer program product, and a computer system for explaining black-box machine learning models. A computer or server determines a process-aware neighborhood around a data sample, using one or more business process rules. The computer or server computes proximity between the process-aware neighborhood and the data sample, using a process-aware distance metric. The computer or server finds, from a family of linear functions, a local linear model, by minimizing losses of respective ones of the linear functions and a black-box machine learning model. The computer or server provides the local linear model for explanation of output of the black-box model on the data sample.