Intelligence Processor Memory Management Circuit Reducing Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network models with multiple layers of convolution operations require large memory bandwidth due to repeated reading and writing of data tiles between DRAM and the processing device.
Innovation Solution
An intelligent processing device with a memory management circuit and multiple memory layers splits convolution operations into multiple stages, temporarily storing results in a cache, and only writing final results back to DRAM, thereby reducing memory bandwidth requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If convolution operations with multiple layers are performed sequentially with data tiles repeatedly read and written between DRAM and processing device, then the convolution operation can be completed, but the memory bandwidth requirement increases significantly
Solution Approach 1:
The patent segments the convolution operation into multiple stages, where intermediate results are stored in a cache memory rather than being repeatedly read from and written to DRAM. This segmentation reduces the number of memory accesses and the corresponding bandwidth requirements while maintaining the sequential processing of convolution layers.
Solution Approach 2:
The patent introduces a cache memory as an intermediary between DRAM and the processing device. This intermediary stores intermediate results of convolution operations, reducing the frequency of data transfers to and from DRAM, thereby decreasing the memory bandwidth requirement while preserving the functionality of multi-layer convolution processing.
2Productivity
If data tiles are repeatedly read and written to DRAM for each convolution layer, then the neural network model can be processed, but the number of memory accesses increases
Solution Approach 1:
The patent divides the convolution processing into stages with intermediate results cached in memory. This segmentation allows the system to process multiple convolution layers without repeatedly accessing DRAM for the same data tiles, reducing the total number of memory accesses and the time spent on memory operations.
Solution Approach 2:
The patent performs preliminary storage of intermediate convolution results in cache memory before proceeding to the next convolution layer. This preliminary action ensures that data is readily available for subsequent processing without requiring repeated access to DRAM, thereby reducing memory access time and improving overall processing efficiency.
Data Source
AI summary
An intelligent processing device includes a first memory, a second memory, a memory management circuit and a convolution operation circuit. The memory management circuit transfers an input data from an external memory to the first memory. The convolution operation circuit reads the input data from the first memory, and performs multiple stages of calculations to generate multiple sets of feature map data. After a first data tile of a first feature map data is generated, the memory management circuit stores the first data tile to the second memory. When a data amount of the first data tile stored satisfies a predetermined value, the memory management circuit transfers the first data tile from the second memory to the first memory, and the convolution operation circuit reads the first data tile from the first memory and accordingly performs a second-stage calculation to generate a second data tile of a second feature map data.