Sparse Matrix Multiplication Processor Using Index Tuples
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing processors are inefficient in performing matrix multiplication, especially with sparse matrices, as they waste time, memory bandwidth, and power due to the presence of zero elements in the matrices.
Innovation Solution
A processor architecture is introduced that includes a vector unit capable of executing index tuples in parallel, generating result values by multiplying input vector elements with kernel weights, and adding these results to accumulators, optimized for sparse matrices by encoding non-zero elements and their indices into a binary format called Sparse Matrix Compiled Object (SMCO).
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional processors perform matrix multiplication by processing all elements including zeros, then complete matrix operations are achieved, but computational overhead and resource utilization increase significantly
Solution Approach 1:
The patent extracts and processes only the non-zero elements of the sparse matrix by using index tuples to identify positions of non-zero kernel weights. This extraction principle eliminates unnecessary processing of zero elements, directly reducing computational overhead and power consumption while maintaining correct matrix multiplication results.
Solution Approach 2:
The patent applies partial action by performing matrix multiplication operations only on the necessary non-zero elements rather than all elements. The index tuples enable the processor to selectively compute only the partial products that contribute to the final result, avoiding excessive computation on zero elements and improving overall efficiency.
2Productivity
If traditional processors perform matrix multiplication by processing all elements including zeros, then complete matrix operations are achieved, but time consumption increases
Solution Approach 1:
The patent extracts only the non-zero elements using index tuples that store the positions of non-zero kernel weights. By extracting and processing only these essential elements, the computation time is significantly reduced compared to processing the entire matrix including all zero elements.
Solution Approach 2:
The patent performs preliminary action by pre-computing and storing index tuples that identify the positions of non-zero kernel weights before the actual matrix multiplication. This preliminary organization of data enables faster access and processing during the multiplication operation, reducing overall computation time.
3Productivity
If traditional processors perform matrix multiplication by processing all elements including zeros, then complete matrix operations are achieved, but memory bandwidth utilization increases
Solution Approach 1:
The patent extracts only the non-zero kernel weights and their positions using index tuples, thereby reducing the quantity of data that needs to be transferred between memory and processor. This extraction approach significantly reduces memory bandwidth utilization compared to loading and processing entire matrix rows including zero elements.
Solution Approach 2:
The patent applies partial action by loading and processing only the necessary non-zero kernel weights rather than all kernel weights. This selective processing reduces the volume of data transferred through memory bandwidth, improving system efficiency and reducing memory subsystem load.
Data Source
AI summary
A processor and method for efficiently implementing matrix multiplication. The processor comprises: a first register (vC) for storing elements of an input vector (X); a second register (vB) for storing a plurality of index tuples, each index tuple comprising at least an input index addressing an element of the input vector (X) in the first register (vC); an output register (vA) comprising a plurality of accumulators for storing elements of an output vector (V); a vector unit configured to execute each index tuple in the second register (vB) in parallel by, for each index tuple: i) generating a respective result value by multiplying the element of the input vector (X) in the first register (vC) addressed by the input index of that index tuple by a corresponding kernel weight in a memory; and ii) adding the result value for that index tuple to one of the accumulators in the output register (vA).