Parallel Gradient Processing for Histogram Creation in Learning Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current techniques for creating histograms of gradient information and feature amounts in gradient boosting methods are limited, as they cannot handle parallel input of multiple variables and increase the circuit scale of memory storage, hindering processing speed improvements.
Innovation Solution
A learning device configured with multiple data memory units, gradient output units, an addition unit, and a histogram memory unit allows for parallel input and processing of feature amounts and gradient information, reducing the circuit scale by integrating added values for each bin, enabling efficient histogram creation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the number of memories that store the histogram is increased in proportion to the number of division of the memory that holds the learning data, then the processing speed of learning processing is improved, but the circuit scale is increased
Solution Approach 1:
The patent merges multiple gradient output units into a single unified histogram memory unit. Instead of having separate memories for each divided data portion, the invention combines all gradient information from multiple data divisions into one histogram memory, eliminating the need for proportional increase in memory数量 while maintaining parallel processing capability
Solution Approach 2:
The histogram memory unit is designed to handle gradient information from multiple data divisions universally. A single histogram memory unit can accumulate and process gradients from all data portions through the addition unit, making it a multi-functional component that serves all parallel processing streams without requiring dedicated memory for each
2Measurement precision
If learning processing is performed several hundreds of times for adjustment of hyperparameter and feature selection, then the accuracy and generalization performance are improved, but the calculation time becomes a problem
Solution Approach 1:
The patent performs preliminary parallel calculation of gradient information for all data divisions before final histogram accumulation. By pre-computing gradients in parallel across multiple data portions and then efficiently combining them through the addition unit, the system reduces the total calculation time required for repeated learning processing while maintaining accuracy
Solution Approach 2:
The invention enables continuous parallel processing of gradient information across multiple data divisions without interruption. The addition unit continuously accumulates gradients from all data portions into the histogram memory, maintaining uninterrupted useful action throughout the learning process and reducing overall calculation time
Data Source
AI summary
A learning device is configured to perform learning by gradient boosting, and includes: data memory units configured to store learning data including a type of feature amount and corresponding gradient information; gradient output units each configured to receive an input of the feature amount and the corresponding gradient information from a corresponding one of the data memory units, and output the gradient information through an output port corresponding to each value of the input feature amount; an addition unit configured to add up one or more pieces of the gradient information corresponding to the same value of the feature amount, and output an added value of the gradient information corresponding to each value of the feature amount; and a histogram memory unit configured to store a histogram obtained by integrating added values of the gradient information corresponding to each value of the feature amount as a bin.