Sparse Matrix Multiplication Accelerator Using Non-Zero Bitmasks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional approaches lack processors and instructions for accelerating multiplication of sparse matrices, and they do not have the flexibility to handle sparsity in one or both matrices, as well as varying formats and sizes of matrix elements.
Innovation Solution
The implementation of a system that utilizes matrix operations accelerators, which include instructions for performing sparse matrix multiplication (SMM) using configured tiles. These tiles can be loaded from and stored to memory, and the system supports various configurations and data types, allowing for efficient multiplication of sparse matrices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional matrix multiplication is used, then dense matrix operations can be performed, but performance deteriorates when matrices are sparse due to unnecessary computations involving zero-valued elements
Solution Approach 1:
The patent extracts only the non-zero elements from sparse matrices and processes them separately using specialized instructions. The system identifies non-zero elements through bitmasking and loads only these relevant elements into processing units, eliminating unnecessary computations involving zero-valued elements and thereby improving productivity while reducing energy loss.
Solution Approach 2:
The patent applies different processing qualities to different parts of the matrix. By using bitmasking to identify non-zero elements and applying specialized sparse matrix multiplication instructions only to these local non-zero regions, the system optimizes computation for the most critical parts while skipping redundant operations on zero elements.
2Adaptability or versatility
If conventional matrix multiplication instructions are used, then general matrix operations are supported, but flexibility to handle varying formats and sizes of matrix elements is insufficient
Solution Approach 1:
The patent implements universal sparse matrix multiplication instructions that can handle multiple matrix formats and sizes through a single instruction set. The system uses configurable tile dimensions and bitmasking mechanisms that adapt to different sparsity patterns, allowing the same instruction architecture to process various matrix configurations without requiring separate specialized instructions for each format.
Solution Approach 2:
The patent introduces dynamic configuration capabilities where the tile dimensions and processing parameters can be adjusted based on the specific matrix being processed. The system dynamically determines the number of tiles, their dimensions, and the bitmask patterns needed, allowing flexible adaptation to varying matrix formats and sizes while maintaining a relatively simple base instruction set.
3Reliability
If all matrix elements are processed, then complete matrix multiplication is achieved, but time is wasted computing with zero-valued elements
Solution Approach 1:
The patent performs preliminary action by first identifying and masking non-zero elements before the actual multiplication computation. Through bitmasking operations that pre-determine which elements are non-zero, the system prepares a filtered set of elements to process, ensuring that subsequent multiplication operations only handle relevant data and thereby reducing computation time without compromising accuracy.
Solution Approach 2:
The patent implements skipping by rapidly identifying zero-valued elements through bitmasking and excluding them from processing. The specialized instructions allow the system to skip directly to the next non-zero element, rushing through the computation process by eliminating redundant operations on zero elements while maintaining complete computational accuracy for the remaining elements.
Data Source
AI summary
Disclosed embodiments relate to accelerating multiplication of sparse matrices. In one example, a processor is to fetch and decode an instruction having fields to specify locations of first, second, and third matrices, and an opcode indicating the processor is to multiply and accumulate matching non-zero (NZ) elements of the first and second matrices with corresponding elements of the third matrix, and executing the decoded instruction as per the opcode to generate NZ bitmasks for the first and second matrices, broadcast up to two NZ elements at a time from each row of the first matrix and each column of the second matrix to a processing engine (PE) grid, each PE to multiply and accumulate matching NZ elements of the first and second matrices with corresponding elements of the third matrix. Each PE further to store an NZ element for use in a subsequent multiplications.