Sparse Matrix Accelerator Scheduling for Zero-Skipping Computation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current graphics processing units (GPUs) face inefficiencies in processing sparse matrix operations due to the need to perform computations on zero-value operands, which wastes resources and reduces performance in deep learning applications.
Innovation Solution
Implementing a sparse matrix processing mechanism that includes a scheduler to identify zero-value operands, a sparse pattern tracker to detect sparse segments, and a compression logic to store frequently used sparse matrices in a compressed buffer, allowing only non-zero operands to be processed and optimizing computations by bypassing zero-value multiplications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional dense matrix computation is used in GPUs, then general processing capability is maintained, but processing efficiency is reduced due to unnecessary computations on zero-value operands
Solution Approach 1:
The patent extracts and identifies zero-value operands from the input matrix using a sparsity detection mechanism, then excludes these operands from subsequent multiplication operations. This extraction of unnecessary computational elements directly resolves the contradiction by eliminating waste without sacrificing general processing capability.
Solution Approach 2:
The patent implements dynamic sparsity detection and adaptive computation scheduling that adjusts the processing pipeline based on the actual sparsity pattern of input matrices. This dynamic adaptation allows the system to optimize for sparse operations when applicable while maintaining dense computation capability when needed, resolving the efficiency-energy contradiction adaptively.
2Speed
If sparse matrix operations are optimized by skipping zero-value computations, then processing speed increases, but device complexity increases due to additional sparsity detection and management mechanisms
Solution Approach 1:
The patent merges the sparsity detection function with the existing computation pipeline by integrating pattern tracking logic into the scheduler. This combination allows sparsity optimization to be achieved without adding completely separate detection and management systems, thus increasing speed while limiting the growth of device complexity through functional integration.
Solution Approach 2:
The patent implements self-organizing data structures and automatic sparsity pattern recognition that enable the system to identify and handle sparse operations without extensive external control mechanisms. This self-service approach increases processing speed through intelligent optimization while reducing the complexity overhead by leveraging automated rather than manually managed solutions.
3Measurement precision
If sparsity pattern tracking is implemented to identify non-zero segments, then computational precision is improved by avoiding unnecessary operations, but measurement complexity increases due to additional tracking requirements
Solution Approach 1:
The patent performs preliminary sparsity pattern tracking and segmentation of the input matrix before the main computation phase. By pre-identifying non-zero segments and organizing them into optimized data structures, the system achieves precise computation control while reducing the complexity of real-time detection during the actual multiplication operations.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
An apparatus, a method, a data processing system and a computer program are disclosed. For example, the apparatus comprises a memory to store a set of input matrices and an accelerator coupled to the memory. The accelerator comprises a hardware unit to detect a sparsity pattern of the set of input matrices, a buffer to store elements of the set of input matrices, and an array of processing elements to perform an operation on the elements of the input matrices. Each processing element in the array of processing element includes one or more registers to store operands for the algebraic operation. The array of processing elements is to: receive the elements of the set of input matrices from the buffer, store the elements received from the buffer within the one or more registers of the processing elements within the array of processing elements, perform the algebraic operation on non-zero elements received from the buffer, the non-zero elements determined based on the detected sparsity pattern, and output a result of the algebraic operation.