CNN Engine Pipelined Image Cache Memory Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional Neural Networks (CNNs) face challenges in responsiveness and memory requirements when incorporated into image processing systems for real-time feature classification, necessitating improved memory management and processing efficiency.
Innovation Solution
A CNN engine with a pipelined image cache architecture that allows for rapid provision of input map information, enabling 3D convolution in a minimum number of clock cycles and simultaneous reading of blocks of pixels of varying sizes for efficient processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a conventional memory architecture is used in CNN processing, then memory capacity is sufficient, but memory bandwidth requirements are high and processing speed is slow
Solution Approach 1:
The memory system is divided into multiple banks (first bank, second bank, third bank) that can operate in parallel. Each bank handles specific data access patterns, allowing simultaneous reading of input feature maps, weights, and bias values without contending for the same memory resources. This segmentation reduces the effective memory bandwidth requirement per bank while maintaining overall high throughput.
Solution Approach 2:
The patent introduces a temporal dimension to memory access by implementing a pipelined architecture where different stages of convolution operations access different memory banks at the same time. This transforms a single-dimension memory bottleneck into a multi-dimensional parallel access system, effectively reducing the bandwidth burden on any single memory interface.
2Reliability
If more memory is provided for CNN processing, then data availability improves, but memory bandwidth requirements increase
Solution Approach 1:
By segmenting memory into specialized banks, the system ensures that critical data (input features, weights, bias) are always available in the appropriate bank without requiring excessive total memory bandwidth. Each bank is optimized for its specific data type, improving data availability while controlling bandwidth usage.
Solution Approach 2:
The memory system is pre-configured with data in the appropriate banks before convolution operations begin. Input feature maps, weights, and bias values are loaded into their respective banks in advance, ensuring immediate data availability when needed without requiring high bandwidth during the actual computation phases.
3Productivity
If parallel processing is implemented in CNN, then processing speed increases, but memory access complexity increases
Solution Approach 1:
The parallel processing architecture is supported by segmented memory banks, where each bank handles a specific data stream. This segmentation simplifies the control logic for parallel access compared to a unified memory system, as each bank has a dedicated access pattern and can be controlled independently, reducing overall access complexity despite increased parallelism.
Data Source
AI summary
A convolutional neural network (CNN) for an image processing system comprises an image cache responsive to a request to read a block of N×M pixels extending from a specified location within an input map to provide a block of N×M pixels at an output port. A convolution engine reads blocks of pixels from the output port, combines blocks of pixels with a corresponding set of weights to provide a product, and subjects the product to an activation function to provide an output pixel value. The image cache comprises a plurality of interleaved memories capable of simultaneously providing the N×M pixels at the output port in a single clock cycle. A controller provides a set of weights to the convolution engine before processing an input map, causes the convolution engine to scan across the input map by incrementing a specified location for successive blocks of pixels and generates an output map within the image cache by writing output pixel values to successive locations within the image cache.