Zero Skipping Sparsity for Memory-Accelerator Data Movement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing sparsity techniques primarily focus on processor and accelerator implementations, neglecting memory subsystems, which leads to performance bottlenecks due to low bandwidth in off-chip memory devices, especially for high-speed processors and accelerators performing sparse scalar, vector, and matrix computations.
Innovation Solution
The introduction of a zero skipping sparsity computation system that includes a processor and memory with sparsity control units, which generate and manage zero and near-zero element masks to optimize data movement by skipping unnecessary computations and data transmission, thereby reducing bandwidth utilization and computation workload.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional sparsity techniques are implemented on processor or accelerator, then computation efficiency is improved, but data movement bottleneck remains due to low bandwidth of off-chip memory devices
Solution Approach 1:
The memory sparsity control unit performs preliminary actions by generating zero element masks before data transmission occurs. It scans memory data, identifies zero and near-zero elements, and creates masks that indicate which elements should be skipped during transmission and computation, thereby reducing data movement volume before the bottleneck occurs
Solution Approach 2:
The invention extracts only the necessary information (non-zero elements and their positions) from the full dataset and transmits only this extracted information to the processor. The zero element masks enable the system to take out and transmit only essential data, reducing the quantity of data movement while maintaining computation accuracy
2Quantity of substance
If compressed sparse row/column data structures are used, then storage requirements are reduced, but compression and de-compression overhead increases computational workload
Solution Approach 1:
Instead of fully compressing and de-compressing data structures, the invention implements a skipping mechanism where zero element masks allow the processor to skip over zero elements directly during data transmission and computation. This rushing through of zero elements eliminates the need for complex compression/de-compression operations while maintaining storage efficiency
3Productivity
If all data is transmitted from memory to processor, then computation can proceed, but bandwidth utilization increases proportionally to data size
Solution Approach 1:
The system applies partial action by transmitting only the necessary non-zero elements and their position information rather than all data. The zero element masks enable selective transmission of only essential data, reducing bandwidth utilization while maintaining sufficient information for accurate computation
Data Source
AI summary
Zero skipping sparsity techniques for reduced data movement between memory and accelerators and reduced computational workload of accelerators. The techniques include detection of zero and near-zero values on the memory. The non-zero values are transferred to the accelerator for computation. The zero and near-zero values are written back within the memory as zero values.