Tiled Matrix Multiply Engine With Cell-Array Accumulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing matrix multiplication techniques are computationally intensive, particularly for large matrices, and there is a need for more efficient methods to accelerate these computations.
Innovation Solution
A matrix multiply engine is designed with a first and second operand buffer and a cell array that computes dot products of multiple elements in an operating cycle, utilizing a cell array with accumulator circuitry to increase arithmetic intensity, and includes readout circuitry for selective row or column readout of tile state elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional parallel processing or vector processors are used for matrix multiplication, then computational speed is improved, but arithmetic intensity decreases as operand width increases
Solution Approach 1:
The matrix multiplication is divided into smaller sub-matrix operations that can be processed in parallel. The operand matrices are segmented into tiles, and each tile is processed by dedicated cell arrays, allowing simultaneous computation of multiple dot products while maintaining high arithmetic intensity through fine-grained parallelism
Solution Approach 2:
The patent introduces a tile-based dimension to organize matrix operations. By transforming the traditional scalar-by-scalar processing into tile-by-tile processing with accumulated results stored in tile state memory, the system achieves both high throughput and high arithmetic intensity through multi-dimensional data organization
2Quantity of substance
If operand width is increased to improve computational efficiency, then arithmetic intensity increases, but computational complexity and resource requirements increase
Solution Approach 1:
Large operand widths are segmented into smaller tile dimensions. Instead of processing entire large matrices at once, the system divides them into manageable tiles that can be handled by fixed-size cell arrays, reducing the complexity of individual processing units while maintaining high overall arithmetic intensity through parallel tile processing
Solution Approach 2:
The system dynamically adjusts tile dimensions and processing parameters based on the operand size and required arithmetic intensity. By changing the tile configuration parameters, the system can optimize performance for different computational workloads without requiring proportional increases in hardware complexity
3Productivity
If dedicated matrix multiplication circuits are used, then computational speed is improved, but device complexity increases
Solution Approach 1:
The dedicated matrix multiplication circuit is segmented into multiple identical cell arrays that can operate in parallel. Each cell handles a specific tile of the matrix multiplication, and the results are accumulated in tile state memory. This segmentation allows the system to achieve high computational speed through parallelism while keeping individual cell complexity low and modular
Solution Approach 2:
The cell arrays are designed as universal, reusable units that can process different tile sizes and operand configurations. The same basic cell structure serves multiple functions in the matrix multiplication process, including data loading, dot product computation, and result accumulation, thereby reducing overall circuit complexity through functional consolidation
Data Source
AI summary
A matrix multiply engine can include a first operand buffer and a second operand buffer, each of which can store multiple operand elements arranged in rows and columns. A cell array can be formed of cells, where each cell includes a memory and accumulator circuitry to receive operand elements column-wise from each of the first operand buffer and the second operand buffer, to compute a dot product of the received operand elements, and to accumulate the dot product into a corresponding tile state element in the memory. Matrix elements of the operand matrices to be multiplied can be loaded row-wise into rows of the operand buffers and read column-wise into the cells. The number of elements for which a dot product is computed can be selected depending on operand element width.