Systolic Array Dot-Product Circuitry for ML Training Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing programmable integrated circuits face challenges in efficiently supporting machine learning training due to insufficient floating-point multiplier capacity and bandwidth constraints, leading to fitting and timing closure problems.
Innovation Solution
The implementation of a hybrid floating-point dot-product circuitry that combines hard DSP blocks with custom soft multiplier blocks, utilizing both 16-bit BFLOAT16 and single-precision formats to enhance dot-product density and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional floating-point multipliers are used to support machine learning training operations, then the device can perform training functions, but the floating-point throughput is insufficient and timing closure problems occur
Solution Approach 1:
The patent segments the floating-point multiplication function into two parts: integer multiplication using DSP blocks and fractional multiplication using soft logic. This segmentation allows each part to be optimized independently - the DSP blocks handle the integer portion with high speed and reliability, while the soft logic handles the fractional portion with sufficient precision, collectively achieving the required floating-point throughput without timing closure issues.
Solution Approach 2:
The patent merges two different computational approaches (integer multiplication in DSP blocks and fractional multiplication in soft logic) into a unified floating-point multiplication system. By combining these two specialized components, the system achieves both the high throughput needed for machine learning training and the timing closure reliability, as neither component alone could satisfy both requirements.
2Productivity
If more soft logic is used to support training operations, then the training capability is enhanced, but fitting and timing closure problems arise
Solution Approach 1:
The patent segments the computational workload by assigning integer multiplication to hardwired DSP blocks and fractional multiplication to soft logic. This segmentation limits the amount of soft logic required while enhancing training capability, as the bulk of the computational burden (integer multiplication) is handled by the efficient DSP blocks, reducing overall device complexity and avoiding fitting and timing closure problems.
Solution Approach 2:
The patent changes the parameter representation by separating the floating-point multiplication into integer and fractional components. This parameter change allows the system to use fixed-point arithmetic in DSP blocks combined with fractional arithmetic in soft logic, achieving enhanced training capability with controlled device complexity that avoids fitting and timing closure issues.
3Productivity
If external memory bandwidth is increased to support complex training operations, then the training performance improves, but internal bandwidth management constraints are exceeded
Solution Approach 1:
The patent changes the arithmetic representation to use integer multiplication for the dominant portion of floating-point operations. This parameter change improves training performance by enabling more efficient use of available memory bandwidth, as integer arithmetic in DSP blocks is more resource-efficient and reduces the overall bandwidth requirements compared to full floating-point operations.
Data Source
AI summary
A programmable device may be configured to support machine learning training operations using matrix multiplication circuitry. In some embodiments, the multiplication is implemented on a systolic array. The systolic array includes an array of processing elements, each of which includes hybrid floating-point dot-product circuitry.