Reduction Tree Quantization for Lower-Cost BNN Accumulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Reduction trees in binary neural networks (BNNs) are costly due to the dominant role of population counters, which require extensive hardware resources, especially in fully connected and convolutional layers, as the width of reduction operations increases logarithmically with each level of the tree.
Innovation Solution
Incorporating intermediate quantizer circuits between reduction operators in the reduction tree to eliminate intermediate lossless accumulations, reducing hardware requirements by quantizing sums based on trained thresholds, thereby reducing the precision needed for calculations and hardware complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If population counters are used to reduce vectors of products to scalar values in a binary neural network, then the reduction operation can be performed, but the hardware costs become dominant with tens of thousands of LUTs required
Solution Approach 1:
The reduction tree is segmented into multiple levels with intermediate quantization stages. Instead of using a single large population counter, the system divides the reduction process into hierarchical levels where each level processes a subset of products and passes quantized results to the next level, significantly reducing the hardware resources required at each stage.
Solution Approach 2:
The system changes the precision parameter of intermediate results by applying quantization at each level of the reduction tree. By reducing the bit-width of intermediate sums through quantization based on trained thresholds, the hardware complexity of subsequent population counters is dramatically reduced while maintaining acceptable accuracy.
2Productivity
If the width of reduction operations increases to handle more products, then the reduction capability improves, but the width increases logarithmically with each level of the tree
Solution Approach 1:
Quantization is applied as a preliminary action at each level of the reduction tree before the results are passed to the next level. This preliminary quantization reduces the width of intermediate sums, preventing the logarithmic increase in bit-width that would otherwise occur as the reduction tree progresses through multiple levels.
3Measurement precision
If intermediate lossless accumulations are performed to maintain precision, then calculation accuracy is preserved, but hardware requirements increase significantly
Solution Approach 1:
The system changes the precision parameter of intermediate results by applying quantization at each level of the reduction tree. By reducing the bit-width of intermediate sums through quantization based on trained thresholds, the hardware complexity of subsequent population counters is dramatically reduced while maintaining acceptable accuracy.
Solution Approach 2:
The system uses low-precision quantized representations for intermediate results in the reduction tree. These quantized values are sufficient for the accumulation operation and can be discarded after the reduction is complete, eliminating the need for expensive high-precision accumulators that would be required for lossless calculations.
Data Source
AI summary
A circuit arrangement includes a reduction operator circuits arranged in a first level of a reduction tree. Each reduction operator circuit accumulates respective products into a respective sum. Quantizer circuits are configured to quantize the sums from the reduction operator circuits into quantized sums, respectively, based on values of the sums relative to respective first thresholds. Another reduction operator circuit is arranged in a second level of the reduction tree and is configured to accumulate the quantized sums and provide a first sum. A second-level quantizer circuit is configured to quantize the first sum into a quantized first sum based on a value of the first sum relative to a second threshold.