Systolic Array Fine-Grained Sparsity Emulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Systolic arrays face inefficiencies when performing matrix multiplications on sparse weight matrices due to unnecessary computations and storage requirements for zero values, leading to wasted computational capacity and increased power consumption.
Innovation Solution
The architecture is optimized by decomposing sparse weight matrices into constrained fine-grained sparse matrices, which are stored in a compressed manner, allowing only non-zero values to be loaded into the systolic array, and using expanded feature map input buses to select appropriate input elements for each processing element, thereby minimizing the number of rows used and reducing power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If sparse weight matrices are processed using conventional systolic arrays, then matrix multiplication can be performed, but computational capacity is wasted and power consumption increases due to unnecessary computations on zero values
Solution Approach 1:
The patent extracts and processes only the non-zero elements of sparse weight matrices, separating them from the zero elements. By loading only non-zero values into the systolic array and using compressed sparse row (CSR) format storage, the system eliminates unnecessary computations on zero values, thereby improving computational efficiency and reducing power consumption.
Solution Approach 2:
The patent applies different processing strategies to different parts of the weight matrix based on their properties. Non-zero elements are processed through the systolic array with full computational resources, while zero elements are skipped entirely. This local differentiation in processing quality optimizes resource allocation and reduces wasted energy on insignificant computations.
2Quantity of substance
If conventional matrix storage formats are used, then all weight values including zeros can be stored, but memory footprint increases and memory bandwidth is wasted
Solution Approach 1:
The patent extracts only the non-zero elements from the weight matrix for storage and processing. By using compressed sparse row (CSR) format, the system stores only the necessary data (non-zero values and their positions), eliminating the storage overhead of zero elements. This reduces memory footprint and prevents waste of memory bandwidth on transferring zero values.
3Productivity
If the systolic array processes sparse matrices using traditional methods, then computations can be completed, but the number of active processing elements increases unnecessarily
Solution Approach 1:
The patent extracts and processes only the relevant non-zero elements through the systolic array, allowing the system to use fewer processing elements effectively. By mapping non-zero weight elements to appropriate processing elements based on their row and column indices, the system activates only the necessary subset of the array, reducing effective device complexity while maintaining computational throughput.
Data Source
AI summary
To take advantage of the architecture of a systolic array tailored to perform sparse matrix multiplications, a weight matrix can be converted into a set of constrained fine-grained sparse weight matrices. The conversion process may include receiving a request to perform a matrix multiplication operation with a weight matrix, and determining that the weight matrix satisfies a sparsity condition to convert the weight matrix into a set of constrained fine-grained sparse weight matrices. The weight matrix can then be converted into a set of constrained fine-grained sparse weight matrices. Computer instructions can then be generated for an integrated circuit device to perform the requested matrix multiplication operation as a set of sparse matrix multiplication operations using the set of constrained fine-grained sparse weight matrices.