General-Purpose Systolic Array Cells for Sparse and Dynamic Workloads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing parallel computing systems, such as systolic arrays, GPUs, and many-core CPUs, are limited in their ability to perform operations beyond dense linear algebra, and lack flexibility in handling tasks like sparse linear algebra, sorting, and dynamic programming efficiently.
Innovation Solution
A systolic array design incorporating general-purpose arithmetic logic units (ALUs) and a register file, with crossbar switches connecting cells, allowing for flexible execution of loop-blocks that can perform operations like dense or sparse matrix multiplication, sorting, and dynamic programming without contention or arbitration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If fixed-function systolic arrays (MXU, TPU, GPU tensor cores) are used, then matrix multiplication performance is improved, but versatility for other operations deteriorates
Solution Approach 1:
The patent implements a universal systolic array cell that can perform multiple operations including dense matrix multiplication, sparse matrix multiplication, sorting, and dynamic programming. Each cell contains general-purpose ALUs and a crossbar switch that can be configured through loop-block programming to execute different algorithms, eliminating the need for specialized hardware for each operation type.
Solution Approach 2:
The systolic array employs dynamic configuration through loop-block programming that can change the behavior of cells during execution. The crossbar switches and ALUs can be reconfigured between operations, allowing the same hardware to adapt to different computational tasks without physical reconfiguration, achieving both high performance and versatility.
2Productivity
If many-core CPUs are used, then parallel processing capability is improved, but core count and parallelism scale deteriorate
Solution Approach 1:
The patent divides the computational system into a large array of simple, identical systolic cells that can be scaled systematically. Each cell is a standardized unit containing ALUs and crossbar switches, allowing the system to achieve high parallelism through numerical scaling of the array dimensions rather than through complex many-core CPU architectures.
Solution Approach 2:
The systolic array achieves scalable parallelism by changing the array dimensions (N×M cells) as a parameter. This allows systematic scaling from small to large configurations while maintaining the same simple cell design, avoiding the complexity of many-core CPU designs and enabling efficient parallel execution of operations across thousands of cells.
3Device complexity
If vector machines are used, then computational simplicity is improved, but dimensionality and processing capability deteriorate
Solution Approach 1:
The patent transitions from one-dimensional vector machine architecture to a two-dimensional systolic array architecture. This dimensional expansion enables cells to receive inputs from multiple directions (top and left neighbors), perform operations, and propagate outputs to multiple destinations (bottom and right neighbors), thereby supporting a broader range of operations including sorting and dynamic programming while maintaining simple cell designs.
4Device complexity
If traditional systolic arrays with small node count are used, then hardware simplicity is improved, but parallel processing power deteriorates
Solution Approach 1:
The patent segments the computational system into a large number of simple, identical systolic cells arranged in an N×M grid. Each cell contains basic ALUs and crossbar switches, keeping individual node complexity low while achieving high overall parallel processing power through the large number of nodes working simultaneously on different data elements.
Data Source
AI summary
A systolic array cell is described, the cell including two general-purpose arithmetic logic units (ALUs) and register-file. A plurality of the cells may be configured in a matrix or array, such that the output of the first ALU in a first cell is provided to a second cell to the right of the first cell, and the output of the second ALU in the first cell is provided to a third cell below the first cell. The two ALUs in each cell of the array allow for processing of a different instruction in each cycle.