Sparse Matrix Multiplication via Bitmap Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing matrix multiplication techniques, particularly for artificial neural networks, are inefficient due to high 'multiply by zero' conditions when dealing with sparse matrices, leading to significant processing and power consumption on CPUs, especially in convolutional neural networks.
Innovation Solution
A system and method that utilize a matrix multiply accelerator to multiply dense and sparse matrices by generating a bitmap based on non-zero elements, compressing the sparse matrix, and using processing engines to perform dot product calculations efficiently, reducing the number of unnecessary multiplications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If standard matrix multiplication techniques are used on sparse matrices, then complete matrix operations are performed, but processing efficiency deteriorates due to large number of multiply-by-zero operations
Solution Approach 1:
The patent extracts only the non-zero elements from sparse matrices and performs multiplication operations exclusively on these extracted elements. The bitmap data structure identifies and extracts non-zero element positions, eliminating the need to process zero elements that would otherwise consume processing resources and energy.
Solution Approach 2:
The patent applies different processing strategies to different parts of the matrix based on their properties. Non-zero elements are processed with full multiplication operations, while zero elements are identified through the bitmap and skipped entirely. This local differentiation optimizes processing efficiency and reduces energy consumption.
2Productivity
If bitmap-based compression is applied to sparse matrices, then processing efficiency improves by reducing multiply-by-zero operations, but device complexity increases due to additional data structures
Solution Approach 1:
The patent segments the sparse matrix representation into two distinct components: the bitmap data structure that tracks non-zero element positions, and the compressed matrix data that stores only non-zero values. This segmentation allows efficient processing by separating the indexing function (bitmap) from the computation function (compressed matrix), improving overall efficiency despite adding structural complexity.
Data Source
AI summary
A system and method for multiplying matrices are provided. The system includes a processor coupled to a memory and a matrix multiply accelerator (MMA) coupled to the processor. The MMA is configured to multiply, based on a bitmap, a compressed first matrix and a second matrix to generate an output matrix including, for each element i,j of the output matrix, a calculation of a dot product of an ith row of the compressed first matrix and a jth column of the second matrix based on the bitmap. Or, the MMA is configured to multiply, based on the bitmap, the second matrix and the compressed first matrix and to generate the output matrix including, for each element i,j of the output matrix, a calculation of a dot product of an ith row of the second matrix and a jth column of the compressed first matrix based on the bitmap.