Sparsity Acceleration Logic for Multi-Precision DNN Dataflow
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural network accelerators face challenges in efficiently handling multi-precision dataflow due to high overhead in processing and storage requirements, particularly when dealing with sparse data and varying precision values, which hinders performance and energy efficiency.
Innovation Solution
The implementation of low overhead sparsity acceleration logic that leverages block sparsity and multi-byte bitmaps to rearrange and group non-zero values, allowing for efficient operation across different precision levels within a single input vector or matrix, using precision-based queues and buffers to optimize dataflow in deep neural network accelerators.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multi-precision dataflow is implemented in deep neural network accelerators, then computational precision and versatility are improved, but processing overhead and storage requirements increase
Solution Approach 1:
The system segments data into sparse and non-sparse portions, processing them through different pathways. Sparsity acceleration logic identifies and processes only non-zero elements, while standard processing handles remaining data, thereby reducing overall processing overhead while maintaining multi-precision capability
Solution Approach 2:
Different precision levels are applied locally to different data elements based on their sparsity characteristics. Non-zero sparse elements may use higher precision while zero elements or less critical data use lower precision, optimizing the balance between computational accuracy and processing overhead
2Productivity
If sparsity acceleration logic is implemented, then processing speed is improved, but device complexity increases
Solution Approach 1:
Sparsity information is pre-computed and stored in bitmaps before the main computation phase. The sparsity acceleration logic uses these pre-computed bitmaps to quickly identify and process only non-zero elements, achieving speedup without adding significant runtime complexity
Solution Approach 2:
Sparsity bitmaps serve as intermediary data structures that mediate between the input data and the computation engine. These bitmaps enable the system to selectively activate computation units based on sparsity patterns, improving throughput while keeping the added complexity localized to the bitmap generation and management
3Productivity
If block sparsity and multi-byte bitmaps are used to group non-zero values, then dataflow efficiency is improved, but memory overhead increases
Solution Approach 1:
Multiple sparsity bitmaps are merged into multi-byte bitmaps that represent sparsity patterns at a coarser granularity. This merging reduces the total number of bitmap entries needed, decreasing memory overhead while still enabling efficient block-level sparsity exploitation for improved dataflow efficiency
Data Source
AI summary
Methods, apparatus, systems, and articles of manufacture to perform low overhead sparsity acceleration logic for multi-precision dataflow in deep neural network accelerators are disclosed. An example apparatus includes a first buffer to store data corresponding to a first precision; a second buffer to store data corresponding to a second precision; and hardware control circuitry to: process a first multibit bitmap to determine an activation precision of an activation value, the first multibit bitmap including values corresponding to different precisions; process a second multibit bitmap to determine a weight precision of a weight value, the second multibit bitmap including values corresponding to different precisions; and store the activation value and the weight value in the second buffer when at least one of the activation precision or the weight precision corresponds to the second precision.