Gradient Boosting Category Feature Processing Circuit Scale Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Gradient Boosting Decision Trees (GBDT) face challenges in processing speed and circuit scale when handling category feature amounts, particularly due to the need for increased feature dimensions and parallel processing, which complicates learning and discrimination tasks.
Innovation Solution
A learning and discrimination device is configured with a histogram memory unit, category and non-category accumulated gradient calculators, and a cumulative sum selector to efficiently process and store gradient information, allowing for parallel processing of feature amounts and reducing circuit scale by integrating gradient information and calculating cumulative sums for threshold-based classification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If One Hot Encoding is used to convert category feature amount into binary feature amount, then the feature amount can be processed numerically, but the feature dimension increases corresponding to the number of categories, requiring more learning modules and increasing circuit scale
Solution Approach 1:
The patent segments the processing of category features by introducing a dedicated category feature processing unit that handles category features differently from numerical features. Instead of converting all category features to one-hot encoding, the system divides the feature processing into separate pathways: one for numerical features and one for category features, thereby avoiding the circuit expansion that would result from one-hot encoding all category features.
Solution Approach 2:
The patent applies local quality by treating category features and numerical features differently within the same system. The category feature processing unit implements specialized processing logic tailored to category features, while the numerical feature processing unit handles numerical features using traditional methods. This localized specialized processing avoids the need to expand the entire system's circuit scale to accommodate category feature processing.
2Productivity
If learning processing is performed in parallel on increased feature amount from One Hot Encoding, then learning speed may improve, but the number of learning modules increases, complicating the system architecture
Solution Approach 1:
The patent segments the learning processing into distinct units: a category feature processing unit and a numerical feature processing unit. Each unit is optimized for its specific feature type, allowing parallel processing without requiring a proportional increase in the number of learning modules. The segmentation enables efficient resource utilization while maintaining manageable system complexity.
Solution Approach 2:
The category feature processing unit is designed to handle multiple category features through a unified processing mechanism rather than requiring separate modules for each category. This multi-functional approach allows the system to process increased feature amounts without linearly increasing the number of learning modules, thereby improving productivity while controlling device complexity.
3Measurement precision
If GBDT is used for structured data with category features, then accuracy is improved, but processing time increases due to the need for multiple learning iterations and hyperparameter adjustment
Solution Approach 1:
The patent applies preliminary action by pre-processing category features through the category feature processing unit before they enter the main gradient boosting learning process. This preliminary processing organizes and optimizes category feature data in advance, reducing the computational burden during the multiple learning iterations and hyperparameter adjustment phases, thereby decreasing processing time while preserving accuracy.
Solution Approach 2:
The category feature processing unit acts as an intermediary between the raw category feature data and the main learning algorithm. It transforms category features into a format optimized for gradient boosting, facilitating more efficient processing during the iterative learning process and reducing the overall processing time required to achieve high accuracy.
Data Source
AI summary
A learning and discrimination device configured to perform learning by gradient boosting, including: a histogram memory unit configured to store, for each value of a feature amount, a histogram obtained by integrating gradient information corresponding to the value in sample data; a category accumulated gradient calculator configured to calculate a histogram corresponding to a value of the feature amount corresponding to a threshold for the feature amount as a cumulative sum of the gradient information; a non-category accumulated gradient calculator configured to calculate, as a cumulative sum of the gradient information, a sum of a histogram corresponding to a value of the feature amount that is classified using a classification method with the threshold different from the category accumulated gradient calculator; and a cumulative sum selector configured to output the cumulative sum calculated by the category accumulated gradient calculator or the non-category accumulated gradient calculator.