Neural Network Convolution Data Reuse via Fetch Unit Buffering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional deep learning accelerators face inefficiencies in convolution operations due to high memory read frequency and limited data reuse, leading to reduced processing speed and energy efficiency, especially when handling various input tensors and convolution parameters.
Innovation Solution
A device and method that reuse data read from memory by using a fetch unit to sequentially read and store data groups in a queue, and an operation unit to perform convolution operations multiple times using identical operators, optimizing data throughput and reducing memory read instances.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If one activation is read K2 times from memory to process convolution operation, then convolution operation can be completed, but number of memory read instances increases causing unnecessary energy consumption
Solution Approach 1:
The patent applies preliminary action by reading multiple activation values from memory in advance and storing them in buffers before the convolution operation begins. This allows the operation unit to reuse these pre-loaded data without additional memory reads during the convolution process, thereby reducing total memory read instances and energy consumption while maintaining complete convolution operation functionality.
2Productivity
If one activation is read K2 times from memory, then convolution operation can be processed, but memory read bandwidth is exceeded causing bottleneck in activation read speed
Solution Approach 1:
The patent reads multiple activation values from memory in advance and stores them in buffers before the convolution operation begins. This preliminary data loading distributes the memory read operations over time, preventing bandwidth saturation during the actual convolution processing and eliminating the bottleneck in activation read speed.
Solution Approach 2:
The patent introduces buffers as intermediary storage between memory and the operation unit. These buffers act as mediators that receive pre-loaded activation data and supply it to the operation unit during convolution, decoupling the memory read operations from the computation timeline and enabling sustained high-speed processing without bandwidth constraints.
3Productivity
If conventional deep learning accelerator is optimized for specific input type, then processing performance for that type is improved, but data reuse rate for other input types is lowered
Solution Approach 1:
The patent implements universality by designing a convolution processing system with generic buffer structures and control logic that can handle multiple input tensor types and convolution parameters. The same buffer management mechanism works across different data formats, enabling high data reuse rates and processing performance regardless of the specific input type or convolution configuration.
Data Source
AI summary
A device for processing convolution operations includes: a processor that executes, in a neural network, a convolution operation on input data in a form of width×height×input channel and on a filter in a form of K×K×input channel or K×K to correspond to a form of the input data, K being an integer greater than or equal to one, and that generates output data in a form of width×height×output channel; and a reader that sequentially reads, from a memory storing the input data, a data group having more pieces of data than unit data throughput of an operator, and provides the data group to the operator to reuse at least one piece of data constituting the data group in the convolution operation. The processor executes, by using one or more operators identical to the operator, the convolution operation multiple times based on the unit data throughput.