GBDT Feature Interpretation via Leaf Node Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for interpreting Gradient Boosting Decision Tree (GBDT) models lack effective solutions for local feature contribution analysis, which is crucial for understanding the prediction performance of individual user features.
Innovation Solution
A method and apparatus that determine feature interpretations of predicted label values by analyzing the scores and split features of decision trees in a GBDT model, calculating local increments, and aggregating relevance measures across multiple decision trees to identify features relevant to the predicted label value, using existing parameters and prediction results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If existing GBDT interpretation methods (remodeling or eigenvalue adjustment) are used to analyze local feature contribution, then feature interpretation capability is improved, but computational complexity and time consumption increase significantly
Solution Approach 1:
The patent extracts only the necessary information (leaf node scores and split features) from the trained GBDT model to perform feature interpretation, rather than retraining or adjusting the entire model. This extraction approach obtains local feature contributions by tracing prediction paths through the decision trees, significantly reducing computational overhead while maintaining interpretation accuracy
Solution Approach 2:
The patent creates a simplified interpretation structure that copies only the essential elements (prediction paths, leaf node scores, and split features) needed for feature analysis. This copying mechanism avoids the computational burden of working with the full model while preserving the information necessary for accurate feature interpretation
2Measurement precision
If comprehensive feature interpretation is performed across all decision trees in the GBDT model, then interpretation accuracy is improved, but device complexity and computational resources increase
Solution Approach 1:
The patent segments the GBDT model into individual decision trees and further segments each tree into prediction paths from root to leaf nodes. By processing each path independently and aggregating results, the method maintains high interpretation accuracy while reducing the complexity of analyzing the entire model at once. The segmentation allows parallel processing and modular computation
3Adaptability or versatility
If local feature contribution analysis is implemented for individual users, then model interpretability is improved, but ease of operation and implementation difficulty increase
Solution Approach 1:
The patent implements a self-service mechanism where the GBDT model's own internal structures (prediction paths, leaf node scores, and split features) are utilized to generate feature interpretations. The model interprets itself by tracing the prediction paths through its own decision trees and aggregating feature contributions, eliminating the need for external interpretation tools or complex additional processing
Data Source
AI summary
Implementations of the present specification disclose methods, devices, and apparatuses for determining a feature interpretation of a predicted label value of a user generated by a GBDT model. In one aspect, the method includes separately obtaining, from each of a predetermined quantity of decision trees ranked among top decision trees, a leaf node and a score of the leaf node; determining a respective prediction path of each leaf node; obtaining, for each parent node on each prediction path, a split feature and a score of the parent node; determining, for each child node on each prediction path, a feature corresponding to the child node and a local increment of the feature on the child node; obtaining a collection of features respectively corresponding to the child nodes; and obtaining a respective measure of relevance between the feature corresponding to the at least one child node and the predicted label value.