Sparse Matrix Multiply Using Bit Masks for Zero-Skip Computation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for multiplying sparse matrices are inefficient due to the need to compute dot products involving zero-valued elements, leading to increased power consumption and computational overhead.
Innovation Solution
The method involves partitioning vectors into sub-vectors and using bit masks to identify non-zero elements, allowing only these elements to be read and multiplied, thereby reducing unnecessary computations and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If traditional matrix multiplication is performed on sparse matrices, then complete computation is achieved, but power consumption increases due to unnecessary multiplications by zero-valued elements
Solution Approach 1:
The patent segments the input vectors into sub-vectors and uses bit masks to identify and process only non-zero elements. This segmentation allows the system to divide the computation into meaningful units (sub-vectors) and selectively process only relevant portions based on bit mask indicators, thereby avoiding multiplications by zero-valued elements while maintaining computational completeness.
Solution Approach 2:
The patent applies partial action by computing only the necessary dot products involving non-zero elements rather than performing complete multiplication on all elements. The bit mask mechanism enables the system to perform exactly the right amount of computation needed for sparse matrices, avoiding both complete computation (excessive) and insufficient computation (partial), thus optimizing power consumption while maintaining accuracy.
2Loss of energy
If all elements are read and processed in matrix multiplication, then complete computation is ensured, but bandwidth consumption increases due to reading zero-valued elements
Solution Approach 1:
The patent performs preliminary action by computing bit masks that identify non-zero elements before the actual multiplication process. This preliminary step allows the system to prepare selection criteria in advance, so that during the multiplication phase, only non-zero elements are read and processed, thereby reducing bandwidth consumption while ensuring computation accuracy through the pre-computed bit mask guidance.
Solution Approach 2:
The bit mask serves as an intermediary between the input vectors and the multiplication process. It mediates the selection of elements to be processed, acting as a filter that guides the system to read and multiply only non-zero elements. This intermediary mechanism ensures that bandwidth is consumed only for relevant data while maintaining computational accuracy through the mediating selection process.
3Loss of energy
If bit masks are computed and packed format is used to store only non-zero elements, then power consumption is reduced, but device complexity increases
Solution Approach 1:
The patent merges the storage of non-zero elements with bit mask information in a packed format. Instead of separate structures for element values and their positions, the system combines them into a compact representation where non-zero elements are stored contiguously with associated bit mask bits. This merging reduces the overall data structure complexity compared to traditional sparse formats while enabling efficient processing and reducing power consumption.
Data Source
AI summary
A sparse dot product and/or matrix multiply is computed by subdividing each vector and simultaneously performing operations to generate output matrix elements. In an embodiment, a bit mask is computed that includes one bit for each element of an input matrix, each bit indicating whether the element is non-zero or zero. In an embodiment, the element values are stored in a packed format, where all of the non-zero values are packed together and the remaining storage for the matrix contains zeros (or any other values). The bit mask can then be used to determine the location of each non-zero element in the packed storage. Rather than reading all of the elements, only the non-zero elements that will be multiplied by a non-zero element from the other input vector should be read. Any multiplication by a zero element from either input vector or matrix is unnecessary.