CNN Feature Plane Memory Allocation for Skip Connection Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing CNN calculation systems face inefficiencies due to simultaneous data reading and writing conflicts in single port memories, leading to increased circuit scale or restricted CNN configurations when processing skip connections and ping-pong buffer schemes.
Innovation Solution
A memory allocation method that allocates input and output feature planes to different memories using a dedicated output memory and alternating input memories, allowing simultaneous reading and writing operations without increasing memory count.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If N+1 independent memories are allocated to process connection of N input feature planes, then memory access conflicts are avoided, but circuit scale increases
Solution Approach 1:
The patent divides the memory system into a dedicated output memory and multiple input memories, where input memories are segmented and allocated to different input feature planes. This segmentation allows simultaneous read operations from multiple input memories without conflict, while the dedicated output memory handles write operations separately, avoiding the need for N+1 independent memories and reducing circuit scale.
Solution Approach 2:
The patent merges multiple input feature plane storages into a set of shared input memories that can be simultaneously accessed. By combining the storage function for multiple input planes into fewer physical memory units with coordinated access control, the system avoids the proliferation of N+1 separate memories while maintaining access efficiency.
2Device complexity
If the number of memories is reduced to avoid increasing circuit scale, then device complexity decreases, but memory access conflicts occur
Solution Approach 1:
The patent implements dynamic memory allocation and access control where input memories are cyclically or alternately assigned to different input feature planes based on the processing stage. This dynamic assignment allows the same physical memory units to serve multiple functions at different times, maintaining high processing efficiency with fewer memories by avoiding simultaneous access conflicts through temporal separation.
Solution Approach 2:
The patent introduces a memory management unit that acts as an intermediary between the CNN calculation unit and the memory system. This intermediary coordinates read and write operations, managing the allocation of input memories to different input feature planes and ensuring that access operations are scheduled to avoid conflicts, thereby maintaining processing efficiency with reduced memory count.
3Productivity
If ping-pong buffer scheme is used with two memories, then continuous data processing is enabled, but memory allocation becomes complex for skip connections
Solution Approach 1:
The patent makes the input memories universal by designing them to handle both sequential layer processing and skip connection processing. The same set of input memories can be allocated to different input feature planes depending on the processing stage, whether it's continuous layer processing or skip connection processing. This multi-functionality simplifies the memory allocation scheme compared to dedicated memory allocation for different processing modes.
Data Source
AI summary
An apparatus includes memories hold feature planes each corresponding to a corresponding layer of layers in a neural network, a calculation unit performs calculation processing on the feature planes, and a memory control unit reads a feature plane from any of the memories and input the feature plane to the calculation unit, and writes a feature plane output from the calculation unit to any of the memories. In a case where feature planes corresponding to different layers are connected and the calculation processing is performed, the memory control unit writes the feature planes to be connected, in memories other than a specific memory among the memories, reads the feature planes to be connected, from the memories other than the specific memory and inputs the feature planes to the calculation unit, and writes the feature plane output from the calculation unit in the specific memory.