Sparse Matrix Bitmask Compression Without Indirect Addressing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional data compression techniques for sparse matrices, such as CSR and CSC formats, are inefficient due to indirect addressing steps in scalar operations, leading to slow processing and wastage of resources, particularly in large sparse neural networks.
Innovation Solution
A mask-based compression scheme that uses bit masks to represent zero and non-zero values, allowing for efficient storage and operation skipping, with optional hierarchical pyramid mask formats to further reduce storage and computational requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional CSR or CSC formats are used to store sparse matrices, then the matrix can be stored with some compression, but indirect addressing steps are required for every scalar operation leading to slow processing
Solution Approach 1:
The patent extracts and removes zero elements from the sparse matrix storage entirely. By using a bitmask to indicate zero positions and only storing non-zero elements with their indices, the format eliminates the need to process zero values during matrix operations, directly improving processing speed while reducing addressing complexity.
Solution Approach 2:
The patent applies different storage strategies to different parts of the matrix data. Non-zero elements are stored with full precision and direct addressing, while zero elements are represented by compact bitmask indicators. This local differentiation optimizes both processing speed and addressing efficiency by treating non-zero and zero elements differently.
2Productivity
If traditional CSR or CSC formats are used to store sparse matrices, then the matrix structure is preserved, but processing and memory are wasted on zero values
Solution Approach 1:
The patent extracts zero elements from active processing by representing them only through a compact bitmask. This eliminates memory waste by not allocating full element storage for zeros while preserving the ability to identify their positions, thereby improving processing efficiency and reducing energy consumption.
Solution Approach 2:
Instead of storing all elements and filtering zeros during processing (traditional approach), the patent inverts the strategy by storing only non-zero elements with their positions indicated by bitmask and index arrays. This inversion eliminates redundant processing of zero values and reduces memory waste.
3Quantity of substance
If bit masks are used to represent zero and non-zero values, then storage needs are reduced and operations can be skipped, but the data structure becomes more complex
Solution Approach 1:
The patent segments the sparse matrix storage into three distinct components: a bitmask indicating zero positions, an index array storing column positions of non-zero elements, and a values array storing only non-zero values. This segmentation reduces overall storage requirements while managing complexity through clear separation of concerns for each data structure component.
Data Source
AI summary
A method, system and program product includes examining elements of a first matrix in a sequential fashion. Values of the examined elements are determined. A corresponding bit of a first mask is set to a first value if a determined value is zero. A corresponding bit of a first mask is set to a second value if a determined value is non-zero. The non-zero values are packed in a first vector, wherein bits of at least the first mask determine operations on packed values.