Pipelined CNN Image Cache for Memory Bandwidth Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional Neural Networks (CNNs) face challenges in responsiveness and memory requirements when incorporated into image processing systems, necessitating improved memory management and processing efficiency for real-time feature classification.
Innovation Solution
A CNN engine with a pipelined image cache architecture that allows for rapid provision of input map information, enabling 3D convolution operations in a minimal number of clock cycles and simultaneous reading of blocks of pixels of varying sizes for efficient processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a CNN engine is incorporated into an image processing system, then feature classification capability is improved, but memory requirements and responsiveness challenges worsen
Solution Approach 1:
The image cache is segmented into multiple banks (first bank, second bank, third bank) that can operate independently and in parallel. This segmentation allows the system to store and retrieve different portions of image data simultaneously, reducing the total memory required at any given time while maintaining the capability to process large images through staged loading and processing.
Solution Approach 2:
The patent introduces a temporal dimension to memory access through pipelined architecture, where data is loaded, processed, and retired in overlapping stages across different clock cycles. This allows the system to handle large memory requirements by spreading data access across multiple time steps rather than requiring all data to be available simultaneously.
2Speed
If a pipelined image cache architecture is used, then processing speed is improved, but device complexity worsens
Solution Approach 1:
The pipelined image cache is divided into multiple banks with dedicated control logic for each bank. This segmentation allows parallel processing of different data portions simultaneously, achieving high processing speeds while keeping each individual bank's complexity manageable and modular.
Solution Approach 2:
The patent implements dynamic control signals (RWB0-RWB7, RB0-RB7, WB0-WB7) that adaptively manage data flow between cache banks and processing units based on real-time processing needs. This dynamic control enables efficient resource utilization and maintains high processing speeds while avoiding the need for overly complex static interconnection structures.
3Productivity
If 3D convolution operations are performed in minimal clock cycles, then productivity is improved, but memory access bandwidth requirements worsen
Solution Approach 1:
The system performs preliminary loading of image data into the image cache banks before the actual convolution processing begins. This pre-loading allows the convolution engine to access data from the fast cache memory during processing rather than reading directly from slow external memory, thereby achieving high productivity while reducing the bandwidth burden on the external memory interface.
Solution Approach 2:
The pipelined architecture ensures continuous data flow through the system with overlapping load-process-retire cycles. Data is continuously supplied to the convolution engine from the cache banks without idle cycles, maintaining high productivity while the cache structure amortizes the memory bandwidth requirements across continuous operation rather than burst access.
Data Source
AI summary
A convolutional neural network (CNN) for an image processing system comprises an image cache responsive to a request to read a block of N×M pixels extending from a specified location within an input map to provide a block of N×M pixels at an output port. A convolution engine reads blocks of pixels from the output port, combines blocks of pixels with a corresponding set of weights to provide a product, and subjects the product to an activation function to provide an output pixel value. The image cache comprises a plurality of interleaved memories capable of simultaneously providing the N×M pixels at the output port in a single clock cycle. A controller provides a set of weights to the convolution engine before processing an input map, causes the convolution engine to scan across the input map by incrementing a specified location for successive blocks of pixels and generates an output map within the image cache by writing output pixel values to successive locations within the image cache.