CNN Layer Output Partitioning for Memory-Limited Image Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional technologies do not effectively address the issue of processing results at a certain layer in a convolutional neural network being input into the next layer and further subsequent layers in the network structure.
Innovation Solution
A processing circuit is provided with an internal memory, acquisition unit, and control unit to divide data for subnetworks within a convolutional neural network, ensuring the total size of outputs fits within the storage capacity of the internal memory, allowing efficient processing across multiple layers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the output data of multiple layers in a convolutional neural network is stored in external memory, then the storage capacity is sufficient, but the processing speed decreases due to repeated read/write operations
Solution Approach 1:
The patent segments the neural network layers into two groups: layers whose output data is stored in internal memory (first group) and layers whose output data is stored in external memory (second group). This segmentation allows frequently accessed layer outputs to be kept in fast internal memory while less frequently accessed data remains in external memory, resolving the contradiction between storage capacity and processing speed.
Solution Approach 2:
The patent applies local quality by assigning different storage locations (internal memory vs. external memory) to different layer outputs based on their specific access patterns and requirements. Layers with outputs needed by multiple subsequent layers are placed in internal memory, while others use external memory, optimizing the system for local access characteristics.
2Productivity
If all layer outputs are kept in internal memory for fast access, then processing speed improves, but the device complexity increases due to larger memory requirements
Solution Approach 1:
The patent segments the memory hierarchy into internal memory for critical layer outputs and external memory for others, avoiding the need for a single large internal memory while maintaining fast access where needed.
Solution Approach 2:
The patent applies partial action by keeping only the necessary layer outputs in internal memory (those required by multiple subsequent layers) rather than keeping all outputs in internal memory, thus reducing memory capacity requirements while maintaining processing speed for critical operations.
3Quantity of substance
If data is divided and acquired to fit within internal memory capacity, then memory usage is optimized, but the difficulty of data management increases
Solution Approach 1:
The patent applies preliminary action by pre-determining which layer outputs should be stored in internal memory versus external memory based on the network architecture and access patterns. This preliminary classification simplifies data management during execution, as the storage allocation is decided beforehand rather than requiring dynamic management.
Data Source
AI summary
Even when the processing result at a certain layer in a convolutional neural network is input to the next layer and further subsequent layers, the processing can be executed more appropriately. A subnetwork included in a convolutional neural network includes a first layer, a second layer, a third layer, and a fourth layer, the output of the first layer is input to the second layer, the output of the second layer is input to the third layer, and the output of the first layer and the output of the third layer are input to the fourth layer, the acquisition unit divides and acquires the data to be processed so that the total size of the input data to each layer is equal to or less than the storage capacity of the internal memory.