Convolution Computation Unit Tile Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network accelerators, such as TPU and EIE, are not suitable for low-end edge devices due to high on-chip memory requirements or significant off-chip memory access, and they have low utilization rates of multiply-accumulate units (MACs) due to inefficient processing unit architecture and filter mapping.
Innovation Solution
An execution method for convolution computation that divides input images into tiles and reuses data to minimize repeated memory access, utilizing a convolution computation unit with processing units and a controller to perform sequential multiply-accumulate operations with kernel shifting to optimize data reuse and MAC utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional neural network accelerators (TPU, EIE) are used, then convolution computation can be performed, but large on-chip memory or significant off-chip memory access is required
Solution Approach 1:
The input image is divided into multiple tiles (e.g., 4 tiles) that are processed sequentially. Each tile is further divided into feature tiles of size T×T, allowing the system to process data in manageable chunks that fit within limited on-chip memory while minimizing off-chip memory access
Solution Approach 2:
The controller pre-divides the input image into tiles and manages the sequential processing order. By planning the processing sequence in advance and reusing data across multiple convolution operations, the system reduces the need for large memory capacity
2Productivity
If traditional filter mapping on MACs is used, then convolution computation can be performed, but the utilization rate of multiply-accumulate units is low
Solution Approach 1:
The kernel dynamically shifts position across the feature tile after each multiply-accumulate operation. Instead of fixed filter mapping, the kernel moves to different positions (e.g., from top-left to bottom-right) to process different portions of the input data, maximizing MAC utilization
Solution Approach 2:
The kernel continues to shift and perform multiply-accumulate operations across all feature tiles without interruption. This continuous processing ensures that MAC units remain actively utilized throughout the convolution computation, eliminating idle time
3Productivity
If data is accessed repeatedly from off-chip memory, then complete convolution computation can be performed, but computation efficiency is reduced
Solution Approach 1:
Multiple convolution operations are merged by reusing the same input image data and weight values across different feature tiles. The controller manages data reuse so that results from one tile can be used in subsequent tiles, reducing redundant memory access
Solution Approach 2:
The system recovers and reuses data that would otherwise be discarded. Input image data and weight values that are needed for multiple convolution operations are retained in on-chip memory and reused across tiles, minimizing the need to re-access off-chip memory
Data Source
AI summary
An execution method for convolution computation is disclosed, which includes: dividing an input image of N channels into a first tile to an X-th tile according to a feature tile; sequentially performing convolution computations on the data in the first tile to the X-th tile of the input image of the N channels, and storing the computation results as output data; mapping the data in each of the tiles by a kernel, and performing multiply-accumulate operations on the mapped data in each of the tiles, wherein each time the multiply-accumulate operation performed on the data mapped by the kernel is complete, the kernel is shifted to change the mapped data in said tile, and multiply-accumulate operation is performed on the changed mapped data until the multiply-accumulate operations performed on all of the data in said tile are complete, thereby finishing the convolution computation of said tile.