Sparse-Dense Matrix Accelerator for Skipping Zero-Value Cycles
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional CPU and GPU instruction set architectures require symmetric inputs for sparse-dense matrix multiplication, limiting the ability to leverage the sparsity of sparse input matrices and hindering performance gains in deep learning applications.
Innovation Solution
An accelerator for sparse-dense matrix multiplication (SDMM) instructions that process asymmetric inputs, utilizing execution circuitry with SIMD processing lanes, multi-bank memories, and compressed sparse formats to efficiently handle sparse and dense matrices, thereby avoiding processing cycles on zero-valued elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional CPU and GPU instruction set architectures are used for sparse-dense matrix multiplication, then symmetric inputs with the same density are required, but this limits the ability to leverage sparsity of sparse input matrices and hinders performance gains
Solution Approach 1:
The patent applies asymmetry by designing execution circuitry that handles asymmetric sparse-dense matrix multiplication inputs. The circuitry processes a first input matrix with a first density and a second input matrix with a second density, where the densities can differ. This allows the system to efficiently process sparse matrices with varying densities without requiring symmetric inputs, directly resolving the contradiction between throughput improvement and adaptability to asymmetric inputs
2Productivity
If traditional symmetric matrix multiplication circuits are used, then processing cycles are wasted on zero-valued elements, but modifying the circuit architecture to handle sparsity increases complexity
Solution Approach 1:
The patent applies local quality by making different parts of the execution circuitry have different functions optimized for their specific tasks. The circuitry includes separate components for identifying non-zero elements, retrieving corresponding elements from the second input matrix, and accumulating products. This localized specialization allows efficient handling of sparse matrices without requiring complete architectural redesign, balancing productivity improvement with manageable complexity
3Loss of time
If all elements of sparse matrices are processed including zero-valued elements, then the processing time increases proportionally to the matrix size, but skipping elements requires additional logic to identify non-zero elements
Solution Approach 1:
The patent applies preliminary action by pre-processing the sparse input matrix to identify and store locations of non-zero elements before the multiplication operation. The execution circuitry uses these pre-identified locations to directly access and process only non-zero elements during multiplication. This preliminary identification step eliminates the need for complex real-time zero-element detection during processing, reducing processing time while keeping the added logic manageable
Data Source
AI summary
Disclosed embodiments relate to an accelerator for sparse-dense matrix instructions. In one example, a processor to execute a sparse-dense matrix multiplication instruction, includes fetch circuitry to fetch the sparse-dense matrix multiplication instruction having fields to specify an opcode, a dense output matrix, a dense source matrix, and a sparse source matrix having a sparsity of non-zero elements, the sparsity being less than one, decode circuitry to decode the fetched sparse-dense matrix multiplication instruction, execution circuitry to execute the decoded sparse-dense matrix multiplication instruction to, for each non-zero element at row M and column K of the specified sparse source matrix generate a product of the non-zero element and each corresponding dense element at row K and column N of the specified dense source matrix, and generate an accumulated sum of each generated product and a previous value of a corresponding output element at row M and column N of the specified dense output matrix.