Sparse Matrix Multiplication Hardware Using Row Lookup and Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current hardware systems are inefficient in performing matrix multiplication, especially for large-scale computations in AI applications like CNN, DNN, and RNN, due to high compute resource requirements and inability to handle sparse matrix multiplication effectively, leading to increased power consumption and costs.
Innovation Solution
A hardware system capable of performing matrix multiplication, sparse matrix multiplication, and convolution operations using the same compute units, which compresses matrices by storing only non-zero values and using a row lookup unit and multiplier-accumulator configuration to reduce computation by skipping zero multiplications, thereby optimizing resource usage and reducing latency and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If dedicated AI hardware is used for specific computations, then computation speed is improved, but device complexity and cost increase
Solution Approach 1:
The patent implements a universal hardware architecture that can perform multiple types of computations (matrix multiplication, sparse matrix multiplication, and convolution operations) using the same compute units. This eliminates the need for separate dedicated hardware for each operation type, reducing device complexity and cost while maintaining high computation speed through optimized algorithms and data structures
2Productivity
If sparse matrix multiplication is handled with special logic, then computation efficiency is improved, but device complexity increases
Solution Approach 1:
The patent extracts and handles only the non-zero elements of sparse matrices for computation, while implicitly treating zero elements as skipped operations. This approach achieves high computation efficiency by avoiding unnecessary zero multiplications without requiring complex special logic hardware, as the sparsity handling is achieved through algorithmic optimization rather than hardware complexity
3Power
If large-scale matrix operations are performed, then computational capability is improved, but power consumption increases
Solution Approach 1:
The patent extracts and processes only the essential non-zero data elements for large-scale matrix operations, avoiding computation on zero elements. This significantly reduces power consumption while maintaining computational capability, as the hardware performs fewer active operations and data transfers, directly addressing the energy efficiency challenge of large-scale computations
4Productivity
If traditional matrix multiplication methods are used, then algorithm simplicity is maintained, but processing speed decreases
Solution Approach 1:
The patent changes the fundamental parameters of matrix operation by representing matrices in compressed formats (storing only non-zero elements with their positions) and changing the computation approach from element-by-element to block-based and pattern-based processing. This dramatically improves processing speed for sparse and structured matrices while the increased algorithm complexity is offset by hardware optimizations
Data Source
AI summary
A system and method of multiplying a first matrix and a second matrix is provided, the method comprising compressing the second matrix into a third matrix to process primarily non-zero values. For each row in the first matrix, a row may be loaded into a row lookup unit. For each entry in the third matrix, a row address may be extracted, a row value may be obtained from a corresponding loaded row of the first matrix based on the extracted row address, the row value from the loaded row may be multiplied with the matrix value from the third matrix for each column, and the multiplied value may be added to an accumulator corresponding to the each column. Lastly, a multiplied matrix may be output for the loaded row.