CNN Line Memory Layout for Bufferless Overlap Pooling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing CNN-IPs do not support Overlap Pooling processing, necessitating the use of general-purpose programmable processors like CVe, which have lower processing performance and require additional buffers, leading to increased circuit scale and power consumption.
Innovation Solution
An image processing device that performs Overlap Pooling processing without additional buffers by using a line memory with a register group for each channel, a shift data generation circuit, a filtering processing circuit, and a pooling processing circuit to extract maximum values across channels, dividing the processing into Max Filtering and Max Pooling operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If additional buffers (registers) are provided for overlap processing in CNN-IP, then Overlap Pooling processing can be supported, but the circuit scale of the CNN-IP is increased
Solution Approach 1:
The patent merges the overlap buffer function with the existing line memory structure by storing data along the channel direction instead of requiring separate overlap buffers. The line memory's register groups are utilized to hold both the spatial overlap data and channel data, combining multiple functions into a single existing component without increasing circuit scale.
Solution Approach 2:
The patent changes the data storage dimension by storing data along the channel direction in the line memory rather than allocating separate spatial buffers for overlap. This dimensional reorganization allows the same memory structure to handle overlap requirements without additional circuit resources.
2Adaptability or versatility
If Overlap Pooling processing is assigned to general-purpose programmable processors, then processing flexibility is maintained, but processing performance is lower than specialized CNN-IP
Solution Approach 1:
The patent makes the specialized CNN-IP multi-functional by enabling it to perform both standard pooling and Overlap Pooling operations. By reorganizing the data storage and processing flow within the existing CNN-IP architecture, the processor gains versatility to handle different pooling types without losing its specialized high-performance capabilities.
3Adaptability or versatility
If switching allocation of processing from CNN-IP to another IP is performed, then Overlap Pooling can be implemented, but implementation time including system support and data transfer is increased
Solution Approach 1:
The patent combines multiple processing functions within a single CNN-IP by reorganizing the data flow and memory usage. This eliminates the need for switching between different IP blocks, reducing implementation time and removing the overhead of system support and data transfer between separate processing units.
Data Source
AI summary
An image processing device includes: a reception circuit receiving image data spread in a height direction and in a horizontal direction; a line memory having a register group for each channel, the resister group being capable of holding data spread in the horizontal direction in units of row along the height direction; a shift data generation circuit generating a plurality of pieces of first intermediate data in which spatial positions including the height direction of the image data are shifted by different shift amounts, and storing them in a plurality of register groups mutually corresponding to a plurality of channels of the line memory; a filtering processing circuit extracting, in the plurality of pieces of first intermediate data, data indicating the maximum value among a plurality of pieces of data having the same spatial position and having different channels; and a pooling processing circuit extracting, in second intermediate data indicating the maximum value among the plurality of pieces of data for each predetermined spatial region, and generating output data.