Two-Level Memory Topology for DNN Accelerator Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) face significant computational and memory bandwidth limitations due to their high computation demands and large memory requirements, leading to inefficiencies in resource-constrained devices like mobile and edge devices, where memory bandwidth becomes a bottleneck.
Innovation Solution
A DNN accelerator with a two-level memory topology is introduced, where memory banks are grouped into bank groups, and a group selection module manages data transfer requests, allowing for improved bandwidth utilization by sending consecutive requests to different groups, thus reducing backpressure and enhancing PE utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a conventional single-level memory topology is used, then the memory structure is simple, but memory bandwidth utilization is low and backpressure occurs during data transfer
Solution Approach 1:
The memory is segmented into multiple bank groups (first bank groups and second bank groups) with separate selection modules. This segmentation allows concurrent data transfer requests to be distributed across different bank groups, preventing backpressure and improving memory bandwidth utilization without requiring a complete redesign of the memory architecture.
Solution Approach 2:
The patent introduces a hierarchical dimension to the memory topology by organizing banks into bank groups with multiple selection levels. This dimensional organization enables requests to be routed through different hierarchical levels, increasing the effective memory bandwidth without proportionally increasing overall system complexity.
2Productivity
If memory bandwidth is increased to meet DNN computation demands, then computation efficiency improves, but energy consumption increases
Solution Approach 1:
The dual selection module architecture enables continuous data transfer operations by simultaneously serving different bank groups. This continuity reduces idle compute cycles and keeps the memory subsystem actively utilized, improving computation efficiency without requiring proportional increases in energy consumption.
Solution Approach 2:
The patent changes the memory access parameters by introducing hierarchical bank group selection, which optimizes the data transfer pattern. This parameter change in the memory access architecture allows more efficient utilization of existing bandwidth resources, improving computation efficiency without additional energy expenditure.
3Productivity
If more memory banks are added to increase bandwidth, then memory bandwidth utilization improves, but the memory area and complexity increase
Solution Approach 1:
Instead of adding more individual memory banks, the patent segments existing banks into multiple bank groups with hierarchical selection. This segmentation approach increases effective bandwidth utilization by organizing existing resources more efficiently, avoiding the need to expand the physical memory area.
Solution Approach 2:
The selection modules serve multiple functions by routing requests to different bank groups based on hierarchical criteria. This multi-functionality allows the same structural elements to handle more complex access patterns, improving bandwidth utilization without requiring additional dedicated hardware for each function.
Data Source
AI summary
A deep neural network (DNN) accelerator includes one or more compute blocks that perform deep learning operations in DNNs. A compute block includes a memory and one or more processing elements. The memory may include bank groups, each of which includes memory banks. The memory may also include a group selection module, buffers, interconnects, and bank selection modules. The group selection module may select a bank group for a data transfer request from a processing element and store the data transfer request in a buffer associated with the bank group. The memory address in the data transfer request may be transmitted from the buffer to a bank selection module associated with the bank group through an interconnect. The bank selection module may select a memory bank in the bank group based on the memory address. Data can be read from or written into the selected memory bank.