Gradient Boosting Reason Code Identification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Gradient boosting machines build complex classification models that are difficult to interpret, making it impractical to identify which features have the greatest effect on classification outcomes due to their ensemble nature comprising hundreds or thousands of smaller models, each depending on multiple features.
Innovation Solution
A combined classification and reason code determination model is created, where feature contributions are estimated during an offline phase using averaged feature values, allowing for accurate reason code identification in real-time during the online phase with a single traversal of the model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If gradient boosting machines build accurate classification models using ensembles of hundreds or thousands of smaller models, then classification accuracy is improved, but the complexity of identifying which features had the greatest effect on the classification outcome worsens
Solution Approach 1:
The patent pre-calculates and stores feature contribution values for each terminal node in the decision trees during the offline phase. These feature contribution values are computed based on the difference between the average feature value for entities reaching that terminal node and the overall average feature value. By performing this computation in advance and storing it in a lookup table, the system enables rapid feature importance identification during online classification without requiring complex post-hoc analysis of the ensemble model.
Solution Approach 2:
The patent introduces feature contribution values as an intermediary metric that bridges the gap between the complex ensemble model and interpretable feature importance. Instead of directly analyzing the hundreds or thousands of decision trees to determine feature effects, the system uses pre-computed feature contribution values associated with each terminal node as a mediator. These values aggregate the cumulative effect of features along the path from root to terminal node, providing a simplified representation of feature importance that can be quickly retrieved and interpreted.
2Measurement precision
If the classification model includes hundreds or thousands of smaller models where each model depends on multiple features, then classification accuracy is improved, but the time and computational resources required to determine reason codes worsens
Solution Approach 1:
The patent pre-computes feature contribution values for every terminal node in every decision tree of the ensemble during the offline training phase. These values are stored in a structured format that allows efficient retrieval. During online classification, when an entity is classified, the system simply looks up the feature contribution values associated with the terminal nodes reached by the entity's feature values, sums them up, and identifies the top contributing features. This eliminates the need for time-consuming iterative analysis of the ensemble model at classification time.
Solution Approach 2:
The patent extracts the feature contribution information from the complex ensemble model structure and separates it into pre-computed values that can be independently stored and retrieved. Instead of analyzing the entire ensemble of hundreds or thousands of decision trees to determine feature importance, the system extracts and stores the relevant feature contribution data in advance, allowing rapid extraction of reason codes during online operation without re-processing the full model.
3Measurement precision
If feature contributions are calculated in real-time for each classification, then accurate reason codes are obtained, but computational resources and processing time increase
Solution Approach 1:
The patent performs the computationally intensive feature contribution calculations during the offline training phase, when computational resources are abundant and not constrained by real-time requirements. The feature contribution values are calculated once for each terminal node based on the training data and stored for reuse. During online classification, the system only performs simple lookup and summation operations, dramatically reducing the computational resources and energy required at classification time while maintaining accurate reason code generation.
Data Source
AI summary
A classification server perform a method for classifying an entity and identifying reason codes for the classification. The classification server can use a gradient boosting machine to build a classification model using training data. The classification model can be an ensemble of decision trees where each terminal node in the decision tree is associated with a response. The responses from each decision tree can be aggregated by the classification server in order to determine a classification for a new entity. The classification server can determine feature contribution values based on expected feature values. These feature contribution values can be associated with each of the responses in the classification model. These feature contribution values can be used to determine reason codes for the classification of the entity. As such, the classification server can perform a single traversal of the classification model to classify the entity and identify reason codes.