Bank-Specific CDC Buffers for Neural Network Accelerator Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
DNN accelerators face performance bottlenecks due to memory bandwidth limitations, particularly in edge devices with limited computational and memory resources, exacerbated by the increasing complexity and compute demands of DNN models, leading to inefficiencies in memory utilization and power consumption.
Innovation Solution
Implementing DNN accelerators with bank-specific clock domain crossing (CDC) buffers in on-chip memory, allowing simultaneous access from faster clock domain PEs to slower clock domain SRAM banks, reducing area footprint and power consumption while improving latency and bandwidth utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If DNN models with high accuracy are used, then computational accuracy is improved, but computation cost and memory bandwidth requirements increase significantly
Solution Approach 1:
The memory system is divided into multiple banks (first memory bank, second memory bank, etc.), each operating in different clock domains. This segmentation allows independent optimization of each memory bank's access pattern and timing, enabling the system to serve high-accuracy DNN computations while managing bandwidth constraints through selective access to different memory banks.
Solution Approach 2:
The patent introduces a temporal dimension to memory access by using asynchronous clock domains with different frequencies. Instead of only spatially distributing memory access across multiple banks, the system also utilizes temporal asynchrony in clock domains to enable overlapping read and write operations, effectively increasing the memory bandwidth dimension to meet high computational demands.
2Productivity
If multiple memory banks are accessed simultaneously from different clock domains, then memory bandwidth is improved, but clock domain crossing complexity increases
Solution Approach 1:
A CDC buffer is introduced as an intermediary component between the processing elements (PEs) and the memory banks. This buffer absorbs the complexity of clock domain crossings by providing a standardized interface that handles timing differences automatically, allowing multiple memory banks to be accessed simultaneously without proportionally increasing overall system complexity.
Solution Approach 2:
Instead of implementing complex synchronizers for every clock domain crossing, the patent uses CDC buffers that replicate the timing characteristics of the slower clock domain into the faster domain. This copying approach simplifies the crossing mechanism by preserving timing information rather than attempting to synchronize every signal transition.
3Loss of time
If CDC buffers are added to enable simultaneous multi-domain memory access, then latency is reduced, but area footprint increases
Solution Approach 1:
CDC buffers are placed locally at each memory bank interface rather than centrally or distributed throughout the entire memory subsystem. This localized placement minimizes the total area footprint by limiting buffer presence only to the necessary crossing points between clock domains and memory banks, while still providing effective latency hiding for each individual access path.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An accelerator may include one or more data processing units that perform deep learning operations in neural networks. A data processing unit includes a memory and a compute engine. The memory includes memory banks and clock domain crossing (CDC) buffers. Each memory bank has its own CDC buffer(s). The memory banks may be grouped into bank groups. The memory may also include a group selection module and includes bank selection modules, each of which is associated with a different bank group. The group selection module selects a bank group for a data transfer request from the compute engine. The bank selection module of the selected bank group may select a memory bank from the selected bank group and store the data transfer request in a CDC buffer of the selected memory bank. The data transfer request is transmitted to the selected memory bank from the CDC buffer.