Systolic Array Tile Processing for High Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systolic arrays face challenges in increasing throughput without increasing the number of read/write ports, which leads to circuit area increases and higher costs.
Innovation Solution
Implementing a matrix operation instruction that operates on multiple tiles per matrix dimension, such as the MTPMD-matrix instruction, which specifies multiple source and destination tiles, allowing for increased compute throughput without expanding port numbers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the number of read/write ports in systolic array is increased to increase throughput, then compute throughput is improved, but circuit area increases and costs increase
Solution Approach 1:
The patent segments the matrix operation into multiple tiles along each dimension (M, N, K dimensions). Instead of processing one large matrix at a time, the system divides matrices into multiple smaller tiles that can be processed in parallel through the same ports, increasing throughput without adding more ports.
Solution Approach 2:
The patent extends the tile processing from 2D to 3D by adding multiple tiles along each dimension (M dimension, N dimension, and K dimension). This dimensional extension allows the system to process more data through the same port infrastructure by utilizing the additional degree of freedom in tile indexing.
2Productivity
If multiple tiles per matrix dimension are processed, then throughput is enhanced and port utilization is maximized, but instruction complexity increases
Solution Approach 1:
The patent creates a universal tile processing mechanism that can handle multiple tiles along M, N, and K dimensions using the same port infrastructure. The single instruction format with extended indexing capabilities provides multi-functionality, allowing the same hardware to process different numbers of tiles dynamically without requiring separate dedicated circuits for each configuration.
3Area of stationary object
If the same number of ports is used for multiple tiles, then circuit area is reduced, but port utilization efficiency must be optimized
Solution Approach 1:
The patent ensures continuous utilization of ports by pipeline-izing the tile processing. While one tile is being processed through the ports, the system prepares the next tile data, ensuring that ports remain continuously occupied with useful work. The multiple tiles along each dimension provide a continuous stream of data to keep ports busy without idle cycles.
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
An embodiment of an apparatus comprises a systolic array to perform a matrix operation on two input tiles to produce an output tile result, and circuitry coupled to the systolic array to cause the systolic array to perform respective full matrix operations on more than one tile per matrix dimension in response to a single request. Other embodiments are disclosed and claimed.