Systolic Array Fine-Grained Sparsity Computation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Systolic arrays face inefficiencies when performing matrix multiplications on sparse weight matrices, as many processing elements perform unnecessary computations with zero values, leading to wasted capacity and increased power consumption, and storage of these matrices requires significant memory for zero values.
Innovation Solution
The architecture is optimized by decomposing sparse weight matrices into constrained fine-grained sparse matrices, which are stored without zero values and loaded into a systolic array in a compressed form, allowing processing elements to select appropriate input elements based on metadata, thereby minimizing rows used and reducing power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If sparse weight matrices are stored and processed in conventional form, then complete matrix data is available, but significant memory is consumed for zero values and processing elements waste capacity performing unnecessary computations
Solution Approach 1:
The patent extracts only the non-zero values from sparse weight matrices and stores them in a compressed format with metadata indicating their positions. This eliminates the need to store zero values in memory, significantly reducing memory consumption while maintaining the ability to perform matrix multiplication operations.
Solution Approach 2:
The patent segments the weight matrix into non-zero values and zero values, processing only the non-zero values through the systolic array. The metadata provides information about the positions of non-zero values, allowing the system to reconstruct the full matrix behavior without processing zero values, thus improving computational efficiency.
2Reliability
If all processing elements perform matrix multiplication operations, then complete computations are executed, but power consumption increases due to unnecessary operations with zero values
Solution Approach 1:
The patent applies local quality by enabling processing elements to selectively operate on non-zero values based on metadata information. Each processing element can identify whether its corresponding weight value is non-zero and adjust its operation accordingly, performing multiplication only when necessary and skipping operations when the weight is zero, thus reducing power consumption while maintaining computational reliability.
3Ease of operation
If conventional matrix storage is used, then all data is readily accessible, but memory bandwidth is wasted loading zero values into the systolic array
Solution Approach 1:
The patent performs preliminary action by pre-processing the weight matrix to identify and separate non-zero values from zero values before loading into the systolic array. The metadata is prepared in advance to indicate the positions of non-zero values, allowing the system to load only the necessary non-zero values and their position information, eliminating wasted memory bandwidth on zero values while maintaining ease of operation through the metadata guidance.
Data Source
AI summary
A systolic array can implement an architecture tailored to perform matrix multiplications on sparse matrices. Each processing element in the systolic array may include a register configured to store a value, and a multiplexor configured to select an input element from multiple input data buses based on metadata associated with the value. Each processing element may also include a multiplier configured to multiply the selected input element with the value to generate a multiplication result, and an adder configured to add the multiplication result to a partial sum input to generate a partial sum output.