Arithmetic Processing Apparatus Buffer Control for Neural Network Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks require large memory sizes to hold feature plane data and filter kernel coefficients, leading to increased memory capacity needs, especially in hardware implementations, which can be costly and inefficient, especially when implemented in embedded devices.
Innovation Solution
An arithmetic processing apparatus with a storage unit and a buffer control unit that switches buffer systems based on the configuration of the hierarchical network, optimizing memory usage by selectively allocating memory for each layer, using either a ring buffer or frame buffer system to reduce memory size.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a deep network is implemented with a large number of connections and feature planes, then the processing capability and accuracy are improved, but the memory size required to hold feature planes and filter kernels increases significantly
Solution Approach 1:
The patent divides the deep network into multiple layers and processes each layer separately. Feature planes are generated and processed layer by layer, allowing the system to handle large-scale networks without requiring all feature planes to be held in memory simultaneously. This segmentation enables processing of deep networks with many connections while controlling memory usage.
Solution Approach 2:
The patent performs preliminary generation of feature planes for each layer before proceeding to the next layer. By generating and processing feature planes in a predetermined sequence layer by layer, the system can allocate and release memory resources efficiently, avoiding the need to hold all feature planes in memory at once while maintaining the ability to process complex deep networks.
2Quantity of substance
If sequential processing of convolution filter calculation is used to reduce memory size, then the memory capacity is reduced, but the processing time increases
Solution Approach 1:
The patent segments the sequential processing into layer-based units, where each layer's feature planes are generated and processed independently. This allows for efficient memory management within each layer while maintaining overall processing speed through the layered structure. The segmentation enables parallel processing opportunities within layers while keeping memory requirements low.
Solution Approach 2:
The patent maintains continuous processing by generating feature planes layer by layer in sequence without idle time. Each layer's processing immediately feeds into the next layer, ensuring that the processing pipeline remains full and efficient. This continuous action minimizes processing time while the layer-by-layer approach keeps memory capacity requirements low.
3Measurement precision
If feature plane data is held over multiple layers to support deep hierarchy, then the processing accuracy is improved, but the memory size increases
Solution Approach 1:
The patent segments feature plane data retention to specific layers rather than holding data across all layers simultaneously. Each layer generates and processes its feature planes independently, releasing memory after processing. This segmentation maintains processing accuracy for deep hierarchies while preventing memory size from increasing proportionally with network depth.
Solution Approach 2:
The patent implements periodic generation and processing of feature planes layer by layer. Each layer periodically generates its feature planes, processes them, and releases the memory before the next layer begins. This periodic action ensures that accurate feature plane data is available when needed for each layer while preventing accumulation of feature plane data in memory across multiple layers.
Data Source
AI summary
To allow arithmetic processing using a plurality of processing nodes to be executed with a smaller memory size, an arithmetic processing apparatus for executing processing using a hierarchical type network formed by the plurality of processing nodes, comprises: a storage unit configured to store a parameter used by each of the plurality of processing nodes for arithmetic processing and a calculation result of the arithmetic processing in each of the plurality of processing nodes; and a buffer control unit configured to switch, based on a configuration of the hierarchical type network, a buffer system of the parameter and the calculation result in the storage unit in at least one layer of the hierarchical type network.