Sparse Matrix Block Storage for GPU Parallel Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing sparse matrix data formats fail to meet the high concurrency requirements of modern computing devices like GPUs and matrix units, leading to inefficiencies in sparse computing.
Innovation Solution
A new matrix storage method involving blocking and compression of sparse matrices into sub-blocks, followed by tiling and sequential storage of these sub-blocks, optimizing subscript data and improving data access locality and parallelism.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If existing sparse compression formats (COO, CSR, CSC, ELL) are used, then storage space is reduced, but computing efficiency and memory access performance deteriorate due to poor adaptability to high concurrency hardware
Solution Approach 1:
The patent segments the sparse matrix into multiple sub-blocks with uniform dimensions, transforming the traditional global compression approach into a localized block-based structure. Each sub-block is independently processed and stored, enabling better parallelization and hardware utilization while maintaining compression benefits.
Solution Approach 2:
The patent introduces a new dimension of organization by tiling sub-blocks into a structured grid layout with explicit row and column offset metadata. This dimensional transformation enables efficient random access and multi-concurrency operations on modern hardware architectures.
2Device complexity
If traditional sparse formats are used, then data storage is simplified, but memory access efficiency deteriorates due to poor data locality
Solution Approach 1:
By dividing the matrix into fixed-size sub-blocks, the patent creates localized data regions that improve spatial locality. Related elements are grouped together in contiguous memory regions, reducing cache misses and improving access speed.
Solution Approach 2:
The patent pre-organizes matrix data into tiled sub-blocks with embedded offset information during the storage phase. This preliminary structuring enables direct computation of memory addresses during access, eliminating the need for complex index lookups and reducing access latency.
3Quantity of substance
If high compression rates are achieved through aggressive compression, then storage efficiency improves, but adaptability to different sparse matrix distributions deteriorates
Solution Approach 1:
The patent creates a universal storage format that handles various sparse matrix distributions (random, structured, dense, sparse) through its block-based approach. The fixed-size sub-block structure with offset metadata provides a unified interface that adapts to different data patterns without requiring format changes.
Solution Approach 2:
The patent adjusts compression parameters at the block level rather than globally, allowing different sub-blocks to be compressed with optimal parameters suited to their local density characteristics. This localized parameter adjustment maintains high compression rates while adapting to varying sparsity patterns across the matrix.
Data Source
AI summary
A matrix storage method and apparatus are provided. In the method, a plurality of first sub-blocks corresponding to a to-be-stored original matrix are obtained. The plurality of first sub-blocks are obtained after blocking is performed on the original matrix by using a sub-block with a scale of M1×N1 as a unit and a non-zero element in each submatrix obtained through blocking is compressed in a specified direction, where M1 and N1 are positive integers. Blocking is performed on each first sub-block by using a sub-block with a scale of M2×N2 as a unit to obtain a plurality of second sub-blocks, where M2 is a positive integer not greater than M1, and N2 is a positive integer not greater than N1. Data of the plurality of second sub-blocks is sequentially stored by using a second sub-block as a unit.