Deep Learning Accelerator Chunking GEMM Architecture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current dot-product architectures in deep learning neural networks face precision issues due to accumulation errors caused by mismatched bit-widths in multipliers and accumulators, leading to inaccurate computations and increased complexity, with existing solutions failing to mitigate these errors without increasing computation time or architectural complexity.
Innovation Solution
The implementation of a modified dot-product architecture with a two-segment configuration, where input vectors are chunked and processed using a compute matrix with variable bit-width FPUs, allowing for reduced accumulator bit-width and minimized accumulation errors through chunking, enabling more accurate and efficient computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If higher precision computations are used in dot-product operations, then computation accuracy is improved, but computation time increases
Solution Approach 1:
The patent divides the dot-product computation into multiple segments by introducing intermediate accumulation steps. The compute unit performs partial accumulations with controlled precision, and these partial results are then combined in a second accumulation stage. This segmentation allows the system to achieve high final precision without requiring all intermediate computations to use high precision, thus reducing overall computation time while maintaining accuracy.
2Device complexity
If accumulator bit-width is reduced to simplify architecture, then device complexity is reduced, but accumulation errors increase
Solution Approach 1:
The accumulation process is divided into two stages: a first accumulation in the compute unit with reduced precision (using smaller bit-width accumulators) and a second accumulation that combines partial results. This segmented approach allows the use of simpler, lower-bit-width accumulators in the compute unit while still achieving high overall precision through the final combination step, thereby reducing device complexity without significantly increasing accumulation errors.
Solution Approach 2:
The patent introduces intermediate partial results as mediators between the reduced-precision compute unit and the final high-precision output. These intermediate accumulations serve as stepping stones that break the direct path from low-precision operations to final result, allowing error control at each stage while enabling the use of simpler hardware components.
3Device complexity
If mismatched bit-widths are used in multipliers and accumulators to reduce complexity, then device complexity is reduced, but computation precision deteriorates
Solution Approach 1:
The computation is segmented into multiple operational stages with different precision requirements. In the first stage, reduced-precision multipliers and accumulators are used to compute partial products and intermediate sums. In the second stage, these intermediate results are combined with higher precision. This segmentation allows bit-width mismatch in the first stage without compromising final precision, as the intermediate errors are corrected during the combination phase.
Data Source
AI summary
A compute matrix is configured to include a set of compute units, each compute unit including a multiplier and an accumulator, each of the multiplier and the accumulator formed using at least one floating point unit (FPU). An accumulator array is configured to include a set of external accumulators. The compute matrix is operated to produce a chunk dot-product using a first chunk of a first input vector and a first chunk of a second input vector. The accumulator array is operated to output a dot-product of the first input vector and the second input vector using the chunk dot-product.