Block-Based CNN Inference Reducing DRAM Bandwidth via Feature Reuse
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) require high DRAM bandwidth and computing power for high-performance image processing, leading to inefficiencies in memory usage and computing resources.
Innovation Solution
A block-based inference method for a memory-efficient CNN implementation, which divides the input image into blocks and performs a multi-layer convolution operation, reusing features along the block scanning direction to reduce recomputing overheads and eliminating the global line buffer by recomputing features between different scan lines.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If feature recomputing is performed for each block, then memory overhead is reduced, but effective pixel throughput is lowered
Solution Approach 1:
The input image is divided into multiple blocks that are processed independently. Each block undergoes feature extraction and convolution operations separately, allowing memory-efficient processing while maintaining parallel execution capability to preserve throughput.
Solution Approach 2:
Feature maps from previous layers are pre-computed and stored in on-chip buffers before being reused in subsequent convolution operations. This preliminary computation eliminates the need for repeated calculations across blocks, maintaining high throughput while reducing memory overhead.
2Power
If feature reusing is performed by storing features in on-chip buffers, then computing overhead is reduced, but huge line buffers are required
Solution Approach 1:
Instead of uniformly storing all feature maps in large on-chip buffers, the patent selectively retains only the necessary feature maps required for the current block processing. This localized memory retention strategy reduces buffer requirements while maintaining computational efficiency for active blocks.
Solution Approach 2:
The buffer allocation is dynamically adjusted based on the processing stage and block characteristics. During feature extraction phases, minimal buffering is used, while during convolution phases, buffers are allocated only for the specific feature maps needed for the current operation, optimizing the trade-off between computing overhead and memory usage.
3Measurement precision
If high-performance CNN inference is implemented, then image processing quality is improved, but DRAM bandwidth and computing power requirements increase
Solution Approach 1:
The image processing is segmented into independent block-level operations, allowing the system to process multiple blocks in parallel or sequentially with reduced memory access requirements. This segmentation maintains processing quality while reducing peak DRAM bandwidth demands compared to full-image processing.
Solution Approach 2:
Instead of repeatedly accessing large feature maps from external memory, the patent creates compact copies of necessary feature data in on-chip buffers for each block processing. This copying strategy reduces DRAM access frequency and bandwidth requirements while preserving the computational accuracy needed for high-quality image processing.
Data Source
AI summary
A block-based inference method for a memory-efficient convolutional neural network implementation is performed to process an input image. A block-based inference step is performed to execute a multi-layer convolution operation on each of a plurality of input block data to generate an output block data and includes selecting a plurality of ith layer recomputing features according to a position of the output block data along a scanning line feed direction, and then selecting an ith layer recomputing input feature block data according to the position of the output block data and the ith layer recomputing features, and selecting a plurality of ith layer reusing features according to the ith layer recomputing input feature block data along a block scanning direction, and then combining the ith layer recomputing input feature block data with the ith layer reusing features to generate an ith layer reusing input feature block data.