Neural Network Tile-Based Execution Engine
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network computing is computation-intensive and incurs heavy memory access, necessitating improvements in system performance, particularly in convolution operations which require extensive data access to external system memory.
Innovation Solution
A neural network accelerator with hardware engines and a buffer memory that performs convolution operations on input tiles, allowing for data reuse by overlapping tiles and reducing system memory access through a selected tiling and tile traversal scheme, enabling parallel processing and efficient data movement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If convolution operations are performed using external system memory, then data capacity is sufficient, but memory access time and energy consumption increase
Solution Approach 1:
The input feature map is divided into multiple non-overlapping tiles, allowing the system to process smaller data units that can be loaded into and processed by on-chip buffer memory, reducing the need for repeated external memory access
Solution Approach 2:
Input tiles are pre-loaded into on-chip buffer memory before processing, and overlapping tiles are pre-positioned in the buffer to enable seamless continuation of convolution operations without external memory access during processing
2Productivity
If tile-based processing is implemented, then computational efficiency improves, but memory buffer requirements increase
Solution Approach 1:
The buffer memory is organized with multiple banks that can store overlapping tiles in a spatial arrangement, allowing efficient utilization of buffer space while maintaining the ability to quickly access both current and subsequent input tiles for continuous processing
3Loss of time
If overlapping tiles are stored in buffer memory, then data reuse efficiency increases, but memory management complexity increases
Solution Approach 1:
Overlapping tiles are stored in the buffer memory to ensure that when one tile is being processed, the next overlapping tile is already available in the buffer, eliminating idle time and ensuring continuous utilization of the convolution engine without gaps in processing
Data Source
AI summary
An accelerator for neural network computing includes hardware engines and a buffer memory. The hardware engines include a convolution engine and at least a second engine. Each hardware engine includes circuitry to perform neural network operations. The buffer memory stores a first input tile and a second input tile of an input feature map. The second input tile overlaps with the first input tile in the buffer memory. The convolution engine is operative to retrieve the first input tile from the buffer memory, perform convolution operations on the first input tile to generate an intermediate tile of an intermediate feature map, and pass the intermediate tile to the second engine via the buffer memory.