Sparse Matrix Multiplication Memory-Efficient Accumulation Method
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing SpGEMM algorithms are memory-bounded and focus on computation throughput rather than memory efficiency, leading to suboptimal performance due to irregular memory access patterns and high memory overheads.
Innovation Solution
A memory-efficient accumulation method for SpGEMM that utilizes cache-based processors to allocate buffers for intermediate products, performs iterative merging of lists, and optimizes buffer sizes based on symbolic computations to minimize memory consumption and maximize reuse, allowing for low-latency execution of memory-intensive steps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing SpGEMM algorithms focus on computation throughput optimization, then computation speed is improved, but memory consumption increases and memory efficiency deteriorates
Solution Approach 1:
The patent segments the accumulation process into multiple iterative merging steps, where intermediate lists are progressively merged in stages rather than all at once. This segmentation allows memory to be reused across iterations, reducing peak memory consumption while maintaining computation throughput.
Solution Approach 2:
The patent introduces a temporal dimension to memory usage by implementing iterative merging across multiple passes. Instead of holding all intermediate results in memory simultaneously, the algorithm processes lists in waves, using the same memory buffers across different time steps, thus transforming a space-intensive approach into a time-efficient one.
2Quantity of substance
If memory-efficient accumulation is implemented, then memory consumption is reduced, but computation complexity increases
Solution Approach 1:
The patent performs preliminary sorting of intermediate lists by column index before the merging process. This preliminary action organizes the data in advance, enabling efficient merging in subsequent iterations without requiring complex comparison logic during the accumulation phase, thus balancing memory efficiency with computational simplicity.
Solution Approach 2:
The merging process is designed to be self-organizing, where lists automatically merge in order of their column indices without requiring external sorting operations during each iteration. The algorithm leverages the inherent ordering properties of the data structures to perform merges efficiently, reducing the need for additional computational overhead.
3Quantity of substance
If iterative merging process is used, then memory reuse is maximized, but execution time increases
Solution Approach 1:
The patent implements periodic merging actions where lists are merged in regular iterations rather than continuously. Each iteration completes a full cycle of merging and buffer swapping, allowing memory to be fully utilized and reused in each period. This periodic structure optimizes the balance between memory reuse and execution time by avoiding unnecessary intermediate operations.
4Speed
If buffer sizes are increased to store all intermediate lists, then memory access latency is reduced, but memory overhead increases
Solution Approach 1:
The patent implements dynamic buffer management where buffer sizes and allocations are adjusted across different iterations of the merging process. Instead of allocating fixed large buffers to store all intermediate lists simultaneously, the algorithm dynamically reuses the same buffers in different iterations, adapting memory usage to the actual needs of each merging stage, thus reducing overall memory overhead while maintaining access efficiency.
Data Source
AI summary
This application describes an hardware acceleration design for improving SpGEMM efficiency. An exemplary method may include: obtaining a first sparse matrix and a second sparse matrix for performing SpGEMM; allocating a pair of buffers respectively pointed by a first pointer and a second pointer; for each first row in the first sparse matrix that comprises a plurality of non-zero elements, identifying a plurality of second rows in the second sparse matrix that correspond to the plurality of non-zero elements; obtaining a plurality of intermediate lists computed based on each of the plurality of non-zero elements in the first row and one of the plurality of second rows that corresponds to the non-zero element; performing accumulation of the intermediate lists using the pair of buffers; and migrating the one final merged list to a system memory as a row of an output matrix of the SpGEMM.