Sparse-Dense Matrix Accelerator for Asymmetric Input Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional CPU and GPU instruction set architectures require symmetric inputs for sparse-dense matrix multiplication, limiting the ability to leverage the sparsity of sparse input matrices and hindering performance gains in deep learning applications.
Innovation Solution
An accelerator for sparse-dense matrix multiplication (SDMM) instructions that process asymmetric inputs, utilizing SIMD processing lanes, multi-bank memories, and fused multiply-add circuits to improve throughput by up to 20-fold, specifically designed for deep learning contexts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional CPU and GPU instruction set architectures are used for sparse-dense matrix multiplication, then the computation can be performed, but the symmetric input requirement limits the ability to leverage sparsity and reduces throughput
Solution Approach 1:
The patent applies asymmetry by designing separate processing paths for sparse and dense matrices. The execution circuitry includes dedicated sparse matrix processing units that handle sparse inputs differently from dense matrices, allowing the system to exploit sparsity patterns while maintaining compatibility with traditional matrix multiplication operations. This asymmetric treatment enables throughput improvement proportional to the sparsity of the input matrix.
2Adaptability or versatility
If symmetric input requirements are enforced for matrix multiplication, then traditional CPU/GPU architectures can be used, but performance advantage from sparsity is lost
Solution Approach 1:
The execution circuitry is segmented into multiple specialized units: sparse matrix processing units, dense matrix processing units, and accumulation units. This segmentation allows different parts of the system to handle different input types optimally, with sparse matrix units exploiting sparsity patterns through specialized data structures and computation paths, while dense matrix units handle the corresponding dense operations.
3Productivity
If traditional matrix multiplication instructions are used, then symmetric inputs are processed, but the sparsity of sparse input matrices cannot be exploited
Solution Approach 1:
The patent merges multiple processing units into a unified execution circuitry that can handle both sparse and dense matrix operations. The sparse matrix processing units, dense matrix processing units, and accumulation units are integrated to work together on the same hardware platform, allowing the system to achieve high throughput for sparse-dense matrix multiplication while maintaining a cohesive device architecture.
Data Source
AI summary
Disclosed embodiments relate to an accelerator for sparse-dense matrix instructions. In one example, a processor to execute a sparse-dense matrix multiplication instruction, includes fetch circuitry to fetch the instruction having fields to specify an opcode, a dense output matrix, a dense source matrix, and a sparse source matrix having a sparsity of non-zero elements, the sparsity being less than one, decode circuitry to decode the instruction, execution circuitry to execute the decoded instruction to, for each non-zero element at row M and column K of the specified sparse source matrix generate a product of the non-zero element and each corresponding dense element at row K and column N of the specified dense source matrix, and generate an accumulated sum of each generated product and a previous value of a corresponding output element at row M and column N of the specified dense output matrix.