Sparse Matrix Scheduler for Skipping Zero-Value Operands
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current graphics processing units (GPUs) face inefficiencies in processing sparse matrices due to the need to handle zero-value operands and dense computations, which can lead to increased processing time and resource utilization.
Innovation Solution
Implementing a sparse matrix processing mechanism that includes a scheduler to identify and skip zero-value operands, a pattern tracker to detect sparse segments, and compression logic to store frequently used sparse matrices in a buffer for efficient execution, allowing for the partitioning of execution units to handle neural network layers effectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If dense computation is performed on sparse matrices, then computational completeness is maintained, but processing time and resource utilization increase
Solution Approach 1:
The patent extracts and identifies zero-value operands from sparse matrix computations using a pattern tracker that detects sparse segments. By separating zero-value operations from non-zero operations, the system eliminates unnecessary computations while maintaining computational correctness for the meaningful data portions.
Solution Approach 2:
The patent applies partial action by performing computations only on non-zero segments of sparse matrices. The pattern tracker identifies sparse segments and the scheduler selectively executes computations only where needed, rather than performing complete dense matrix operations, thus reducing processing time while maintaining necessary computational accuracy.
2Measurement precision
If zero-value operands are processed in matrix multiplication, then computational accuracy is maintained, but processing load increases
Solution Approach 1:
The pattern tracker extracts zero-value operands from the computation stream and the scheduler excludes them from execution. This removal of redundant zero-value operations reduces processing load significantly while maintaining computational accuracy for all non-zero contributions to the matrix multiplication result.
Solution Approach 2:
The patent changes the execution parameter by dynamically scheduling computations based on operand values. Instead of uniformly processing all operands, the system adjusts execution to skip zero-value operands, thereby reducing processing load while preserving the accuracy of non-zero computational contributions.
3Quantity of substance
If sparse matrix compression is implemented, then storage efficiency improves, but memory access complexity increases
Solution Approach 1:
The patent segments sparse matrices into compressed formats that store only non-zero elements and their positions. This segmentation approach improves storage efficiency by eliminating redundant zero-value storage while the associated metadata structures manage memory access patterns to handle the increased access complexity systematically.
Solution Approach 2:
The pattern tracker acts as an intermediary between the compressed sparse storage format and the computation units. It translates compressed storage representations into computation-ready formats, managing the complexity of memory access while enabling efficient utilization of the compressed data structure.
4Productivity
If execution units are partitioned for neural network layers, then resource utilization optimizes, but system complexity increases
Solution Approach 1:
The patent partitions execution units into dedicated groups for processing different neural network layers. This segmentation improves resource utilization by assigning specific computational resources to specific computational tasks, reducing contention and optimizing throughput. The increased system complexity is managed through structured partitioning that maintains modularity and scalability.
Data Source
AI summary
An apparatus to facilitate processing of a sparse matrix is disclosed. The apparatus includes a plurality of processing units each comprising one or more processing elements, including logic to read operands, a multiplication unit to multiply two or more operands and a scheduler to identify operands having a zero value and prevent scheduling of the operands having the zero value at the multiplication unit.