Parallel Gradient Boosting Layered Architecture for Risk Modeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Gradient boosting tree models for risk prediction in online transactions require extensive time to build due to sequential training of multiple decision trees, making frequent updates necessary to adapt to changing behavior patterns, which is inefficient and time-consuming.
Innovation Solution
Implementing an enhanced gradient boosting technique that allows multiple models to be built and trained in parallel, reducing the time required to generate the machine learning model while maintaining accuracy performance by compiling models in layers with non-overlapping or partially overlapping subsets of features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If gradient boosting tree is used for high accuracy risk predictions, then prediction accuracy is improved, but model building time increases significantly
Solution Approach 1:
The patent divides the model building process into multiple independent stages or iterations, where each stage trains a subset of trees on a portion of the data. This segmentation allows parallel processing of different tree subsets while maintaining the sequential boosting logic within each stage, thereby reducing overall building time without sacrificing prediction accuracy.
Solution Approach 2:
The patent performs preliminary actions by pre-processing the training data and pre-defining feature subsets before the actual model training begins. This preliminary preparation enables more efficient parallel processing during the boosting stages, reducing the time required for each iteration while preserving the accuracy benefits of gradient boosting.
2Measurement precision
If multiple decision trees are built sequentially to maintain accuracy, then prediction quality is improved, but productivity decreases
Solution Approach 1:
The patent segments the ensemble of decision trees into multiple parallel batches or stages. Each batch contains multiple trees that can be trained simultaneously on different data subsets, while the batches themselves follow the sequential boosting paradigm. This segmentation enables significant speedup in model building while maintaining prediction quality through the preserved sequential error-correction mechanism.
Solution Approach 2:
The patent implements partial action by training multiple trees in parallel within each boosting stage rather than strictly one tree at a time. This partial parallelization provides a trade-off where some approximation is introduced but overall prediction quality is maintained while productivity increases significantly.
3Adaptability or versatility
If model is rebuilt frequently to adapt to changing behavior patterns, then adaptability is improved, but time consumption increases
Solution Approach 1:
The patent segments the model into modular components that can be independently updated. When behavior patterns change, only specific segments or stages of the model need to be retrained rather than the entire model, enabling frequent adaptations to new patterns while minimizing the time cost of each rebuilding operation.
Solution Approach 2:
The patent introduces dynamic elements that allow the model to adapt to changing patterns more efficiently. This includes dynamic stage configuration where the number and size of parallel tree batches can be adjusted based on data characteristics, enabling faster rebalancing when patterns change while maintaining adaptability.
Data Source
AI summary
Methods and systems are presented for generating a machine learning model using enhanced gradient boosting techniques. The machine learning model is configured to receive inputs corresponding to a set of features and to produce an output based on the inputs. The machine learning model includes multiple layers, wherein each layer includes multiple models. To generate the machine learning model, multiple models are built and trained in parallel for each layer of the machine learning model. The multiple models use different subsets of features to produce corresponding output values. After a layer in built and trained, a collective error may be determined for the layer based on the output values from the different models in the layer. An additional layer of models may be added to the machine learning model to reduce the collective error of a previous layer.