CNN Tile Boundary Caching for Bandwidth Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural network processing systems require significant bandwidth for reading image information from system memory due to the need for overlapping tiles and extensive data transfer across the system bus, which increases overhead and power consumption.
Innovation Solution
A dedicated CNN engine with a cache that stores both input and output map information for each tile, allowing for reduced overlap between tiles by using pre-stored output map pixels from previous layers to minimize bandwidth requirements, specifically storing output map pixels after pooling steps to reduce the overlap from 32 to 16 pixels, thereby optimizing processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If overlapping tiles are used in conventional neural network processing, then processing accuracy is maintained, but bandwidth requirements increase significantly
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing output map pixels at the boundaries of tiles before they are needed for subsequent tile processing. The cache memory stores output map pixels from previous layers at tile boundaries, so when processing adjacent tiles, these pre-stored pixels can be directly retrieved without requiring overlapping input tiles, thereby reducing bandwidth requirements while maintaining processing accuracy.
Solution Approach 2:
The patent introduces an intermediary mechanism by using cache memory as a mediator between the input image data and the neural network processing layers. The cache stores output map pixels from previous layers at tile boundaries, acting as an intermediary that provides boundary pixel information without requiring the full overlapping tile data to be transferred across the system bus, thus reducing bandwidth requirements.
2Loss of information
If extensive data transfer across the system bus is performed, then complete image information is available for processing, but power consumption increases
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing output map pixels at tile boundaries in the cache memory before they are needed for subsequent processing. This allows the system to retrieve boundary pixel information from the cache rather than transferring complete overlapping tiles across the system bus, significantly reducing data transfer volume and power consumption while maintaining image information completeness.
Solution Approach 2:
The patent extracts only the necessary boundary output map pixels from the full tile data and stores them in the cache memory. Instead of transferring complete overlapping tiles across the system bus, the system extracts and stores only the boundary pixels that are needed for adjacent tile processing, thereby reducing data transfer requirements and power consumption while maintaining information completeness.
3Area of stationary object
If more tiles are used to process larger images, then processing coverage is improved, but processing time increases
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing output map pixels at tile boundaries in the cache memory. This allows adjacent tiles to share boundary pixel information from the cache rather than requiring full overlap, reducing the number of tiles needed to cover the same image area and thereby reducing total processing time while maintaining complete image coverage.
Solution Approach 2:
The patent segments the image processing into tiles with reduced overlap by using cache memory to store boundary output pixels. Instead of using many small overlapping tiles to cover the entire image, the system can use fewer larger tiles with minimal overlap, as the cache provides the necessary boundary information, thereby reducing processing time while maintaining complete coverage.
Data Source
AI summary
An apparatus for processing a neural network comprises an image memory into which an input image is written tile-by-tile, each tile overlapping a previous tile to a limited extent; a weights memory for storing weight information for a plurality of convolutional layers of a neural network, including at least two pooling layers; and a layer processing engine configured to combine information from the image and weights memories to generate an output map and to write the output map to image memory. The apparatus is configured to store a limited number of values from adjacent a boundary of an output map for a given layer. The layer processing engine is configured to combine the output map values from a previously processed image tile with the information from the image memory and the weights when generating an output map for a layer of the neural network following the given layer.