Sparse Matrix Multiplication Acceleration via Column Folding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current hardware accelerators for neural networks inefficiently handle matrix multiplication due to the presence of zeros in matrices, leading to poor performance and power utilization, especially when dealing with sparse matrices.
Innovation Solution
The implementation of matrix folding and squeezing techniques to pack sparse matrices, avoiding multiplication by zero elements, and using routing circuitry to optimize the processing of sparse matrices, allowing for efficient execution on hardware accelerators.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If hardware accelerators perform matrix multiplication in dense format, then hardware resources are utilized and power is consumed, but performance deteriorates due to inefficient multiplication with zero elements
Solution Approach 1:
The patent extracts and removes zero elements from sparse matrices before performing matrix multiplication. By identifying and eliminating zero elements, the system processes only non-zero elements, thereby avoiding unnecessary multiplication operations and reducing power consumption while maintaining computational accuracy.
Solution Approach 2:
The patent changes the data representation parameters by converting sparse matrices from dense format to compressed sparse format. This parameter transformation reduces the number of elements to be processed, optimizing the balance between hardware utilization and energy efficiency.
2Loss of energy
If hardware accelerators process sparse matrices, then power efficiency improves, but device complexity increases due to need for sparsity detection and handling mechanisms
Solution Approach 1:
The patent implements a universal matrix multiplication unit that can handle both dense and sparse matrices through a single architecture. The system uses configurable processing elements that can adapt to different matrix densities, eliminating the need for separate hardware paths and reducing overall device complexity.
Solution Approach 2:
The patent introduces dynamic configuration capabilities where the hardware architecture can adapt its behavior based on matrix sparsity levels. The system dynamically adjusts processing strategies during execution, allowing the same hardware to efficiently handle varying matrix densities without requiring complex static redesigns.
3Power
If conventional matrix multiplication approaches are used, then hardware resources are utilized, but flexibility to support various data formats and both dense and sparse matrices is limited
Solution Approach 1:
The patent designs a universal matrix multiplication accelerator that supports multiple data formats (integer, floating-point) and matrix types (dense, sparse) through a single hardware architecture. The system uses configurable processing elements and flexible data paths that can be adapted to different computational requirements without requiring separate specialized units.
Data Source
AI summary
Disclosed embodiments relate to sparse matrix multiplication (SMM) acceleration using column folding and squeezing. In one example, a processor, in response to a SMM instruction having fields to specify locations of first, second, and output matrices, the second matrix being a sparse matrix, uses execution circuitry to pack the second matrix by replacing one or more zero-valued elements with non-zero elements yet to be processed, each of the replaced elements further including a field to identify its logical position within the second matrix, and, the execution circuitry further to, for each non-zero element at row M and column K of the specified first matrix, generate a product of the element and each corresponding non-zero element at row K, column N of the packed second matrix, and accumulate each generated product with a previous value of a corresponding element at row M and column N of the specified output matrix.