Sparse Matrix Memory Pre-allocation via Symbolic Sampling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Efficient memory allocation for sparse matrix multiplications is challenging due to the uncertainty in the number of non-zero elements in the output matrix, leading to issues such as memory waste or excessive dynamic allocations.
Innovation Solution
A hardware accelerator with sampling and memory management circuitry estimates the number of non-zero elements and floating-point operations through symbolic multiplication, allowing for pre-allocation of memory space before the sparse matrix-matrix multiplication, and dynamically allocates additional memory as needed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If memory space is pre-allocated based on estimated non-zero elements, then memory efficiency is improved, but estimation accuracy is worsened due to uncertainty in predicting the exact number of non-zero values
Solution Approach 1:
The patent performs symbolic multiplication in advance to estimate the number of non-zero elements and compute compression ratios before actual spGEMM execution. This preliminary estimation enables proactive memory space allocation, reducing memory waste while avoiding the need for expensive dynamic reallocations during runtime.
Solution Approach 2:
The patent samples a subset of rows (e.g., 100 rows) from the sparse matrix to perform symbolic multiplication and estimate compression ratios, rather than processing the entire matrix. This partial action provides sufficiently accurate estimates for memory allocation while significantly reducing the computational overhead of the estimation process.
2Adaptability or versatility
If dynamic memory allocation is used during spGEMM, then memory utilization is improved, but computational overhead is worsened due to frequent reallocations
Solution Approach 1:
The patent performs symbolic multiplication and memory space estimation before the actual spGEMM computation. By determining the required memory size in advance based on sampled compression ratios, the system can allocate sufficient contiguous memory space upfront, eliminating the need for frequent dynamic reallocations during runtime and thus maintaining high computational efficiency.
3Measurement precision
If symbolic multiplication is performed on the entire matrix, then estimation accuracy is improved, but computational cost is worsened
Solution Approach 1:
The patent performs symbolic multiplication on a sampled subset of rows (e.g., 100 rows) rather than the entire sparse matrix. This partial computation provides sufficiently accurate compression ratio estimates for memory allocation purposes while reducing the computational cost by a factor proportional to the sampling ratio, making the estimation process practical for large-scale applications.
Data Source
AI summary
This application describes an accelerator, a computer system, and a method for memory optimization in sparse matrix-matrix multiplications (spGEMM). The memory optimization includes accurate memory pre-allocation for a to-be-generated output matrix of spGEMM between two sparse matrices. An exemplary method may include: sampling a plurality of first rows in the first sparse matrix; identifying, based on indices of non-zero data in the plurality of first rows, a plurality of second rows in a second sparse matrix; performing symbolic multiplication operations between the non-zero data in the plurality of first and second rows; determining an estimated compression ratio of the output matrix; determining an estimated mean row size for each row in the output matrix based on the estimated compression ratio; and allocating, according to the estimated mean row size and a total number of rows of the output matrix, a memory space in a hardware memory.