Parallel Learning Device for Gradient Boosting Decision Trees
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning technologies, particularly gradient boosting decision trees (GBDT), face challenges in processing speed and accuracy when dealing with large datasets, especially in real-time applications like robotics and finance, due to limitations in data parallelization and memory access efficiency.
Innovation Solution
A learning device configured with multiple learning units and model memories that perform parallel learning of decision trees using divided data, utilizing Field-Programmable Gate Arrays (FPGAs) to enhance processing speed by optimizing memory access and parallel processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If gradient boosting is used to improve learning accuracy, then accuracy is improved, but processing speed deteriorates due to sequential learning requirement
Solution Approach 1:
The patent divides the learning data into multiple pieces and assigns each piece to a separate learning unit. Multiple decision trees are learned in parallel by different learning units simultaneously, rather than sequentially. This segmentation of data and parallel processing resolves the contradiction by maintaining gradient boosting accuracy while dramatically improving processing speed through concurrent execution of multiple learning units.
2Speed
If learning data is divided to enable parallel learning, then processing speed is improved, but memory access efficiency deteriorates
Solution Approach 1:
The patent merges the model data of multiple decision trees learned in parallel into a single model memory structure. This unified memory structure allows efficient access and integration of results from multiple learning units, reducing the overhead of managing separate memory structures and improving overall memory access efficiency while maintaining parallel processing benefits.
3Productivity
If multiple memories are used to store models of respective decision trees, then parallel learning capability is improved, but device complexity increases
Solution Approach 1:
The patent designs a unified model memory structure that serves multiple functions: storing individual decision tree models, storing aggregated ensemble models, and facilitating efficient data exchange between learning units. This multi-functional memory design maintains parallel learning capability while reducing device complexity by eliminating the need for separate specialized memory structures for each learning unit.
Data Source
AI summary
A learning device is configured to perform learning of a decision tree by gradient boosting. The learning device includes a plurality of learning units and a plurality of model memories. The plurality of learning units are configured to perform learning of the decision tree using learning data divided to be stored in a plurality of data memories. The plurality of model memories are each configured to store data of the decision tree learned by corresponding one of the plurality of learning units.