Banked Memory Architecture for DNN Accelerator Scalability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep Neural Network (DNN) accelerators face scalability issues due to memory bandwidth limitations when increasing data parallelism, as each datapath must access a single logically unified memory structure, leading to prohibitively expensive hardware overhead and complexity.
Innovation Solution
Implementing a banked memory structure where each data channel has its own register memory bank, using a memory decoder to map addresses and isolate dataflows, allowing independent concurrent memory access patterns and reducing hardware overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the number of independent datapaths is increased to scale compute throughput, then data parallelism is improved, but the complexity of supporting independent accesses to a single unified memory structure becomes prohibitively expensive
Solution Approach 1:
The unified memory structure is segmented into multiple independent memory banks, where each memory bank is associated with a specific datapath. This segmentation allows each datapath to access its dedicated memory bank independently, eliminating the complexity of coordinating access to a single shared memory structure while maintaining high compute throughput through parallel datapath operations.
2Ease of operation
If each datapath accesses data from a single logically unified memory structure, then memory access is simplified, but hardware overhead becomes prohibitively expensive
Solution Approach 1:
Each datapath is given access to a locally-associated memory bank rather than a distant shared memory structure. This local quality optimization reduces the hardware overhead for memory access control and arbitration, as each datapath has direct, dedicated access to its local memory bank without requiring complex global coordination mechanisms.
3Productivity
If data parallelism is increased by scaling datapaths, then compute throughput is improved, but memory bandwidth becomes a bottleneck
Solution Approach 1:
The memory system is segmented into multiple independent memory banks that can be accessed in parallel by different datapaths. This segmentation enables memory bandwidth to scale linearly with the number of datapaths, as each datapath can simultaneously access its dedicated memory bank without contending for bandwidth, thus preventing memory bandwidth from becoming a bottleneck.
Data Source
AI summary
The present disclosure relates to devices and methods for using a banked memory structure with accelerators. The devices and methods may segment and isolate dataflows in datapath and memory of the accelerator. The devices and methods may provide each data channel with its own register memory bank. The devices and methods may use a memory address decoder to place the local variables in the proper memory bank.