Convolutional Neural Network Image Blocking to Reduce DRAM Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing size of input images in convolutional neural network (CNN) processing requires a significant increase in dynamic random-access memory (DRAM) bandwidth, which is a bottleneck in efficient image processing.
Innovation Solution
The proposed solution involves storing image groups and parts of images in the processing device during CNN modeling, allowing subsequent processing of input images with stored data to reduce the required DRAM bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the size of input images is increased to improve processing capability, then the processing capability is improved, but the required DRAM bandwidth is increased
Solution Approach 1:
The input image is divided into multiple image blocks, which are processed in a pipeline manner. The processing device receives a first image block group, processes it to generate output image blocks, then receives a second image block while the first image block group is still being processed. This segmentation allows the system to handle larger images without proportionally increasing DRAM bandwidth requirements, as not all image blocks need to be loaded simultaneously.
Solution Approach 2:
The system performs preliminary processing of the first image block group to generate output image blocks before the second image block is fully received. By starting processing early and maintaining a pipeline, the system prepares data in advance, reducing the peak bandwidth requirements during subsequent processing stages.
2Productivity
If the DRAM bandwidth is increased to handle larger images, then the processing capability is improved, but the device complexity is increased
Solution Approach 1:
The system implements a dynamic processing pipeline where the reception and processing of image blocks occur in an overlapping, time-varying manner. The processing device dynamically transitions between receiving new image blocks and processing previously received blocks, allowing efficient resource utilization without requiring excessive bandwidth infrastructure.
3Quantity of substance
If all image blocks are processed sequentially to reduce bandwidth, then the DRAM bandwidth is reduced, but the processing time is increased
Solution Approach 1:
The processing pipeline maintains continuous useful action by ensuring that while one image block group is being processed, the next image block is already being received and prepared. This overlapping of reception and processing operations eliminates idle time in the system, maintaining high utilization without requiring peak bandwidth for all blocks simultaneously.
Solution Approach 2:
The system employs periodic reception and processing of image blocks in a structured pipeline rhythm. Image blocks are received and processed in periodic intervals, with the processing device alternating between reception phases and processing phases, creating an efficient periodic workflow that balances bandwidth usage and processing throughput.
Data Source
AI summary
A method includes: generating, by a processing device, at least one first output image block based on a first image block group; storing stored image blocks corresponding to a first part of the first image block group in the processing device; and after the at least one first output image block is generated, generating, by the processing device, at least one second output image block based on a first image block and the stored image blocks, wherein the first image block group and the first image block are arranged in order along a first direction, and the at least one first output image block and the at least one second output image block are arranged in order along the first direction. A system is also disclosed herein.