Write Combine Buffer for DNN Accelerator Memory Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
DNN accelerators face performance constraints due to limited memory bandwidth, which is exacerbated by the high number of write transactions, leading to inefficient memory access and increased power consumption.
Innovation Solution
Implementing a Write Combine Buffer (WCB) that combines write transactions to reduce the number of memory accesses and optimize memory bandwidth usage, allowing for faster write operations and improved system performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the number of write transactions is increased to handle high-accuracy DNN computations, then computation accuracy is improved, but memory bandwidth utilization deteriorates and power consumption increases
Solution Approach 1:
The WCB merges multiple write transactions targeting the same memory address into a single combined write transaction. When multiple MAC operations produce write transactions for overlapping memory locations, the WCB combines these transactions, reducing the total number of memory writes while preserving all data values. This directly improves memory bandwidth utilization without compromising computation accuracy.
Solution Approach 2:
The WCB acts as an intermediary component between the MAC array and external memory. It intercepts write transactions, processes them through combination logic, and forwards the reduced set of combined transactions to memory. This mediator approach allows the system to maintain high computation accuracy while significantly reducing memory bandwidth consumption.
2Measurement precision
If the number of write transactions is increased to handle high-accuracy DNN computations, then computation accuracy is improved, but power consumption increases
Solution Approach 1:
By combining multiple write transactions into fewer memory operations, the WCB reduces the total number of memory access cycles. Since each memory transaction consumes power, reducing the transaction count directly lowers power consumption while maintaining the same computational accuracy requirements.
Solution Approach 2:
The WCB identifies redundant write transactions that can be discarded (merged with previous writes) and recovers the memory bandwidth and power that would have been consumed by those redundant operations. This approach eliminates wasted energy on duplicate memory writes.
3Productivity
If write transactions are combined in the WCB, then memory bandwidth utilization is improved, but device complexity increases
Solution Approach 1:
The WCB is implemented as a separate, dedicated module within the DNN accelerator, distinct from the MAC array and memory interfaces. This segmentation isolates the combination logic complexity into a specific component, making the overall system easier to design and maintain while achieving improved memory bandwidth utilization.
Data Source
AI summary
A compute tile includes a WCB that receives a workload of writing an output tensor of a convolution into a local memory of the compute tile. The local memory may be a SRAM. The WCB receives write transactions. A write transaction includes a data block, which is a part of the output tensor, and metadata describing one or more attributes of the data block. The WCB may store write transactions in its internal buffers. The WCB may determine whether to combine two write transactions, e.g., based on an operation mode or metadata in the write transactions. In embodiments where the WCB determines to combine the two write transactions, the WCB may combine the two write transactions into a new write transaction and write the new write transaction into the local memory or an internal memory of the WCB. The total number of write transactions for the workload can be reduced.