CNN Image Cache Segmentation for Real-Time Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current Convolutional Neural Network (CNN) engines face challenges in responsiveness and memory requirements, making it difficult to perform real-time feature classification within image processing systems.
Innovation Solution
A CNN engine with a pipelined image cache architecture that rapidly provides input map information for 3D convolution, allowing for minimal clock cycles and incorporating a weights cache with efficient weight representation and compression techniques to reduce memory access requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional memory architecture is used in CNN engines, then memory capacity is sufficient, but memory bandwidth and access time are insufficient for real-time processing
Solution Approach 1:
The memory system is segmented into multiple independent memory banks (first memory bank, second memory bank, third memory bank, fourth memory bank) that can operate in parallel. This segmentation allows simultaneous access to different portions of the input map, effectively increasing memory bandwidth without requiring a single large high-speed memory unit.
Solution Approach 2:
The patent introduces a temporal dimension to memory access by implementing a pipelined architecture where different stages of convolution operations access different memory banks at different clock cycles. This allows the system to sustain high throughput by overlapping memory access with computation across multiple pipeline stages.
2Quantity of substance
If more memory is provided for storing input maps and weights, then capacity is increased, but memory access time and bandwidth requirements increase
Solution Approach 1:
Large memory capacity is achieved through segmentation into multiple banks rather than using a single large memory unit. This allows the system to maintain high capacity while keeping access times low by accessing smaller portions of data in parallel across different banks.
Solution Approach 2:
Weights are pre-loaded into dedicated weight storage units before convolution operations begin. The input map is loaded into the memory banks in advance, organized into tiles that can be quickly accessed during the convolution process. This preliminary preparation eliminates the need for repeated memory access during computation.
3Productivity
If parallel memory access is implemented, then processing speed is improved, but system complexity increases
Solution Approach 1:
The memory system is divided into four independent banks with dedicated read ports and control logic. This segmentation enables parallel access while keeping each individual bank relatively simple in design. The complexity is distributed across multiple identical units rather than concentrated in a single complex unit.
Solution Approach 2:
Multiple memory banks are merged into a unified memory system with a single control interface from the perspective of the CNN engine. The control logic coordinates access to all banks, presenting a simplified interface to higher-level components while maintaining parallel operation internally.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A convolutional neural network (CNN) for an image processing system comprises an image cache responsive to a request to read a block of NxM pixels extending from a specified location within an input map to provide a block of NxM pixels at an output port. A convolution engine reads blocks of pixels from the output port, combines blocks of pixels with a corresponding set of weights to provide a product, and subjects the product to an activation function to provide an output pixel value. The image cache comprises a plurality of interleaved memories capable of simultaneously providing the NxM pixels at the output port in a single clock cycle. A controller provides a set of weights to the convolution engine before processing an input map, causes the convolution engine to scan across the input map by incrementing a specified location for successive blocks of pixels and generates an output map within the image cache by writing output pixel values to successive locations within the image cache.