Sparsity-Aware Datastore for Centralized DNN Tensor Decoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network hardware systems face significant energy costs and performance drawbacks due to high energy consumption in data movement and decoding processes, particularly in aligning sparsity bitmaps and sparse compressed data within processing element arrays.
Innovation Solution
A sparsity-aware datastore is introduced that aligns sparsity bitmaps and sparse compressed data before distributing them to a processing element array, thereby reducing the need for internal alignment and decoding within each processing element, and maximizing data reuse across the array.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If sparsity decoding is performed within each processing element, then data can be decoded locally without centralized alignment, but power consumption increases significantly due to redundant decoding logic across all processing elements
Solution Approach 1:
The patent extracts the sparsity decoding function from individual processing elements and concentrates it in a centralized datastore. The datastore performs sparsity alignment on compressed data before distribution, removing the need for each processing element to have its own decoding logic, thereby eliminating redundant power consumption while maintaining local processing capability
Solution Approach 2:
The patent merges the sparsity decoding functionality into a single centralized datastore component. Instead of having distributed decoding across multiple processing elements, all sparsity alignment operations are combined and performed in one location, reducing overall system power consumption while preserving the ability to serve multiple processing elements
2Adaptability or versatility
If each processing element includes internal alignment and decoding logic, then data can be processed independently, but silicon area increases due to redundant decoding circuits across the array
Solution Approach 1:
The patent extracts the sparsity alignment and decoding logic from individual processing elements and relocates it to a centralized datastore. This removes redundant decoding circuits from each processing element, significantly reducing total silicon area while maintaining the independence of processing elements through the centralized service model
Solution Approach 2:
The centralized datastore serves multiple processing elements simultaneously, providing sparsity alignment and decoding services to the entire array. This universal approach allows one component to perform the function that would otherwise require many separate instances, reducing overall silicon area while maintaining system versatility
3Speed
If compressed data is distributed without pre-alignment, then data movement can begin immediately, but processing efficiency decreases due to lack of sparsity alignment
Solution Approach 1:
The patent performs sparsity alignment as a preliminary action in the centralized datastore before data is distributed to processing elements. By pre-aligning the compressed data with the sparsity bitmap and reordering elements according to the sparsity pattern, the system ensures that subsequent processing is highly efficient while data movement can proceed immediately after the brief alignment phase
Data Source
AI summary
Systems, apparatuses and methods may provide for technology that prefetches compressed data and a sparsity bitmap from a memory to store the compressed data in a decode buffer, where the compressed data is associated with a plurality of tensors, wherein the compressed data is in a compressed format. The technology aligns the compressed data with the sparsity bitmap to generate decoded data, and provides the decoded data to a plurality of processing elements.