DCiM Sub-Macro Partitioning for Flexible N:M Sparsity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing digital compute-in-memory (DCiM) architectures face challenges in accommodating flexible structured sparsity patterns, leading to increased area overhead and compromising the regular architecture, which affects computational throughput and efficiency.
Innovation Solution
The FlexCiM design introduces a flexible DCiM architecture that supports multiple N:M sparsity ratios by partitioning the DCiM macro into sub-macros, using two-to-one multiplexers in compute-in-memory cells, and incorporating input activation buffers and distribution networks to manage input activations and support various sparsity patterns efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If DCiM macro is partitioned into sub-macros to support flexible N:M sparsity patterns, then adaptability improves, but device complexity increases
Solution Approach 1:
The DCiM macro is divided into multiple sub-macros, each capable of handling specific sparsity patterns. This segmentation allows the system to support flexible N:M sparsity patterns by routing different data streams to appropriate sub-macros, resolving the contradiction between adaptability and complexity through modular organization.
Solution Approach 2:
The architecture employs dynamic routing mechanisms that can adaptively direct input data to appropriate sub-macros based on the required sparsity pattern. This dynamic configuration enables the system to maintain flexibility while managing complexity through intelligent data flow control rather than static hardwired connections.
2Adaptability or versatility
If two-to-one multiplexers are added to compute-in-memory cells to support sparsity patterns, then adaptability improves, but area overhead increases
Solution Approach 1:
Two-to-one multiplexers are integrated into the compute-in-memory cell structure to combine multiple input data streams. This merging approach enables support for various sparsity patterns including 1:2, 2:4, and 4:8 patterns while minimizing area overhead by sharing resources across multiple operations rather than duplicating full processing units.
3Productivity
If input activation buffers and distribution networks are incorporated to manage input activations, then productivity improves, but device complexity increases
Solution Approach 1:
Input activation buffers are implemented to pre-load and organize input data before it reaches the compute units. This preliminary organization of data in the distribution network enables efficient routing to appropriate sub-macros, improving computational throughput by ensuring data is ready and properly configured before processing begins.
Solution Approach 2:
The distribution network acts as an intermediary layer between input buffers and compute units, managing data flow and routing decisions. This intermediary structure improves productivity by centralizing data management functions and enabling efficient resource utilization while containing complexity in a dedicated intermediate layer rather than distributing it throughout the entire system.
Data Source
AI summary
To support flexible N:M sparsity pattern in a DCiM macro, the DCiM macro is subdivided into multiple sub-macros according to a partitioning factor P. Each sub-macro can support 1:2 sparsity ratio. Leveraging the partitioned design, the sub-macros can be grouped together to support different N:M sparsity patterns. To determine optimal N:N sparsity pattern for each layer of a neural network, an algorithm can determine the value A of a sparsity ratio A/B is based on the number of outliers in a layer, and the value B of the sparsity ratio A/B is based on the locality measure of the outliers representing the spatial distribution of the outliers. Moreover, the optimal N:M sparsity pattern that is aligned with the determined sparsity ratio A/B can be selected based on whether to prioritize latency or accuracy, or to balance both latency and accuracy.