GBDT Uncertainty Scoring with Virtual Tree Ensembles
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Gradient Boosted Decision Tree (GBDT) models lack effective methods to quantify knowledge uncertainty in their predictions, which is crucial for high-risk applications like healthcare and finance, where inaccurate predictions can lead to severe consequences.
Innovation Solution
Develop a method to generate an uncertainty score for GBDT model outputs by employing ensemble-based algorithms, utilizing stochastic gradient boosting (SGB) and stochastic gradient Langevin boosting (SGLB) techniques to form virtual ensembles from truncated sequences of trees within a single GBDT model, quantifying knowledge uncertainty.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If GBDT models are used for high-risk applications, then prediction accuracy is improved, but the ability to quantify knowledge uncertainty is insufficient
Solution Approach 1:
The patent segments the GBDT model into multiple sub-sequences of trees, creating virtual ensembles that can independently evaluate predictions. This segmentation allows the system to quantify uncertainty by comparing results across different tree subsequences while maintaining the overall prediction accuracy of the full model.
Solution Approach 2:
The patent creates virtual ensembles by copying and truncating sequences of trees from the original GBDT model. These copied subsequences serve as surrogate models that can be used to estimate knowledge uncertainty without requiring training of entirely new models, thus preserving accuracy while enabling reliability assessment.
2Reliability
If virtual ensembles are formed from truncated sequences of trees, then knowledge uncertainty can be quantified, but computational complexity increases
Solution Approach 1:
The patent applies partial action by using truncated sequences of trees rather than complete ensembles. Each virtual ensemble uses only a portion of the available tree sequence, which is sufficient to quantify uncertainty without requiring the full computational resources of training and evaluating multiple complete models from scratch.
Solution Approach 2:
The patent enables the single GBDT model to serve multiple functions: it generates predictions using the full tree sequence and simultaneously provides uncertainty quantification through its own truncated subsequences. This self-service approach eliminates the need for separate ensemble models, reducing overall computational complexity.
3Reliability
If uncertainty scores are generated for GBDT outputs, then decision-making robustness is improved, but model complexity increases
Solution Approach 1:
The patent makes the GBDT model universal by enabling it to perform both prediction and uncertainty quantification through its existing tree structure. The same model that generates predictions also provides uncertainty scores via its truncated subsequences, eliminating the need for separate uncertainty modeling components and reducing overall system complexity.
Solution Approach 2:
The patent changes the parameter of tree sequence length to create virtual ensembles. By varying the truncation points of tree subsequences, the system generates diversity in predictions for uncertainty estimation without fundamentally changing the model architecture or training process, thus maintaining simplicity while improving robustness.
Data Source
AI summary
A method of generating an uncertainty score for an output of a Gradient Boosted Decision Tree (GBDT) model is disclosed. The output is a prediction of the GBDT model for an in-use dataset. The method comprises acquiring the GBDT model including a sequence of trees beginning with an initial tree and ending with a last tree, a given one of the sequence of trees having been stochastically built during a current training iteration of the GBDT model, and defining a plurality of sub-sequences of trees in the sequence of trees as sub-models of the GBDT model. During a given in-use iteration of the GBDT model executable for the in-use dataset, the method comprises generating a plurality of sub-outputs using the respective sub-models and generating the uncertainty score using the plurality of sub-outputs, the uncertainty score being indicative of how different sub-outputs from the plurality of sub-outputs are amongst each other.