CNN Hardware Accelerator Memory Bandwidth Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current FPGA accelerators for deep convolutional neural networks face inefficiencies due to mismatched computation throughput and memory bandwidth, leading to underutilization of resources and performance limitations, especially with the increasing complexity of deep learning applications.
Innovation Solution
An analytical design scheme using the roofline model to quantify computing throughput and memory bandwidth, combined with optimization techniques like loop tiling and transformation, is proposed to identify the best performance and resource-efficient solutions for CNN designs, implemented on a VC707 FPGA board.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If standard scratchpad memories are used in FPGA accelerators, then memory bandwidth is sufficient, but FPGA resources increase significantly
Solution Approach 1:
The accelerator is divided into multiple processing elements (PEs) that can be independently configured and scaled. Each PE contains its own local memory, allowing the system to segment memory resources across multiple units rather than requiring a single large memory block, thus reducing overall FPGA resource requirements while maintaining sufficient bandwidth.
Solution Approach 2:
The design implements a hierarchical memory structure where small local SRAM buffers are nested within each processing element, which are then nested within the larger accelerator architecture. This nested approach allows efficient data sharing between PEs without requiring duplicate full-size memories in each PE, optimizing the trade-off between bandwidth and resource usage.
2Productivity
If computation throughput is increased in CNN accelerators, then processing speed improves, but memory bandwidth requirements exceed what can be provided by the FPGA platform
Solution Approach 1:
The design pre-loads and caches frequently accessed weight and activation data into local SRAM buffers within each processing element before computation begins. This preliminary action reduces the need for high-speed external memory access during the actual computation phase, allowing higher throughput without proportionally increasing memory bandwidth requirements.
Solution Approach 2:
The accelerator implements overlapping computation and data transfer operations, where while one set of processing elements is computing, other PEs are loading data from external memory. This continuous utilization of resources ensures that computation throughput is maximized without requiring peak memory bandwidth to be sustained at all times, smoothing out the bandwidth requirements.
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
A hardware acceleration component is provided for implementing a convolutional neural network. The hardware acceleration component includes a N-row and M-column array of functional units, N input data buffers to store input data, and M weights data buffers to store weights data. Each of the N input data buffers is coupled to a corresponding one of the N rows of functional units. Each of the M weights data buffers is coupled to a corresponding one of the M columns of functional units. Each functional unit in a row receives a same set of input data. Each functional unit in a column receives a same set of weights data from the weights data buffer coupled to the row. Each of the functional units performs a convolution of the received input data and the received weights data, and the M columns of functional units provide M planes of output data.