Reason Code Generation for Ensemble Models via Black Box Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generating reason codes for ensemble computer models is impractical due to their complex nature, treating them as black boxes, which leads to organizations opting for simpler models with lower performance, as existing methods struggle to effectively explain the contributions of input variables to the model score.
Innovation Solution
The proposed approach involves treating the ensemble model as a black box, identifying trivial values for input variables, replacing them, and evaluating the impact on the model score to generate reason codes, with two variants: one round of replacement for top drops and multiple rounds to analyze incremental drops, allowing for effective reason code generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If ensemble models are used to improve prediction performance, then model accuracy is improved, but generating reason codes becomes impractical due to model complexity
Solution Approach 1:
The patent introduces an intermediary system that sits between the ensemble model and the reason code generation process. This intermediary automatically extracts and analyzes input variable contributions by treating the ensemble model as a black box, injecting modified input values and measuring score changes. This mediator enables reason code generation without requiring direct examination of the complex ensemble model structure, thus resolving the contradiction between using complex ensemble models and generating interpretable reason codes.
2Ease of operation
If ensemble models are treated as black boxes to simplify reason code generation, then ease of operation is improved, but understanding of variable contributions is reduced
Solution Approach 1:
The patent replaces the mechanical approach of examining ensemble model structures with an automated computational system. Instead of manually analyzing model architectures, the system automatically injects modified input values, captures score changes, and generates reason codes through systematic computation. This substitution maintains operational simplicity while preserving comprehensive information about variable contributions through the automated black box analysis method.
3Ease of manufacture
If simpler models are used to enable reason code generation, then ease of manufacture is improved, but prediction performance is reduced
Solution Approach 1:
The patent changes the parameter of approach from examining model structure to measuring input-output relationships. By systematically varying input parameters (injecting modified values) and measuring the resulting score changes, the system enables reason code generation for any ensemble model regardless of its internal complexity. This parameter change allows organizations to maintain high-performance ensemble models while still generating interpretable reason codes, eliminating the need to choose simpler models.
Data Source
AI summary
A new approach is proposed to support two variants of the approach to effectively generate reason codes for an ensemble model. Both variants involve treating the ensemble model as a black box, identifying trivial values for input variables to the ensemble model, replacing each of the input variables to the ensemble model with its corresponding trivial values and then evaluating the impact on a score of the model after the trivial value replacement. The evaluation result of the impact is then used to generate the reason codes for the ensemble model. Specifically, the first variant is configured to perform one round of replacement, wherein variables with top drops in the score will be the reason codes. The second variant is configured to perform multiple rounds of replacement, which in each round, keeps the identified reason codes variables replaced with trivial values and analyzes incremental drops of replacing remaining variables.