Systolic Sparse Matrix Multiplication That Skips Zero Multiply-Adds
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning algorithms involving neural networks face inefficiencies due to unnecessary computations from zero-valued elements in sparse matrices, which degrade performance.
Innovation Solution
Implementing a systolic matrix multiplication array with sparse matrix acceleration logic to eliminate unnecessary multiply-add operations by identifying zero operands, optimizing resource and time usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional dense matrix multiplication is used for sparse matrices, then the computation is straightforward and consistent, but unnecessary multiply-add operations are performed on zero elements leading to degraded performance
Solution Approach 1:
The patent extracts and removes zero elements from the matrix multiplication process. The sparsity acceleration logic identifies zero operands and eliminates them from computation, extracting only the necessary non-zero operations to perform, thereby avoiding wasted energy on zero-element multiplications while maintaining computational correctness
Solution Approach 2:
The patent applies local quality by making the computation process adaptive to local characteristics of the data. The sparsity acceleration logic dynamically identifies regions with zero elements and applies optimization selectively to those regions, while maintaining standard computation for non-zero regions, thus improving overall productivity without compromising accuracy
2Loss of energy
If sparsity optimization techniques are applied to eliminate zero operations, then energy efficiency improves, but the device complexity increases due to additional sparsity acceleration logic
Solution Approach 1:
The patent applies universality by designing the sparsity acceleration logic to perform multiple functions within a unified structure. The same logic unit identifies zero elements, determines which operations to skip, and manages the computation flow, thereby reducing the need for separate dedicated circuits for each optimization function and limiting the increase in device complexity
Solution Approach 2:
The patent introduces an intermediary sparsity acceleration logic layer between the input matrices and the systolic array computation units. This intermediary component analyzes the input data, identifies sparsity patterns, and controls the computation flow, thereby enabling energy efficiency improvements while containing complexity increases to a manageable intermediary layer rather than throughout the entire system
3Loss of time
If all multiply-add operations are performed including those with zero operands, then the computation is simple and consistent, but the time required for matrix multiplication increases
Solution Approach 1:
The patent applies preliminary action by performing sparsity analysis and identifying zero elements before the main matrix multiplication computation begins. The sparsity acceleration logic pre-processes the input matrices to determine which operations can be skipped, thereby eliminating time waste during the actual computation phase while containing the complexity overhead to the preliminary analysis stage
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
An apparatus to facilitate acceleration of matrix multiplication operations. The apparatus comprises a systolic array including matrix multiplication hardware to perform multiply-add operations on received matrix data comprising data from a plurality of input matrices and sparse matrix acceleration hardware to detect zero values in the matrix data and perform one or more optimizations on the matrix data to reduce multiply-add operations to be performed by the matrix multiplication hardware.