CNN Accelerator Data Reuse via Tile Scanning and Buffering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional Neural Network (CNN) accelerators face inefficiencies in memory bus bandwidth and power consumption due to limited cache size, requiring effective data reuse methods to optimize performance.
Innovation Solution
A data reuse method for CNN accelerators involving a tile scanning module that partitions data into tile blocks, allocates them to processing units, and implements data reuse within an input activation weight buffer unit to minimize memory access and maximize cache utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the cache size is increased to store more input activation and weight data, then the data reuse rate is improved, but the device complexity and power consumption increase
Solution Approach 1:
The patent divides the input activation data and weight data into smaller tile blocks that can be processed in batches. The tile scanning module segments the computational workload into manageable chunks, allowing the limited cache to hold multiple tile blocks simultaneously. This segmentation enables the system to achieve high data reuse rates without requiring a single large cache, thus resolving the contradiction between data reuse rate and device complexity.
2Productivity
If the memory bus bandwidth is increased to provide more input IA and weight data, then the computing performance is improved, but the power consumption and system complexity increase
Solution Approach 1:
The patent loads multiple tile blocks of input activation and weight data into the cache memory in advance before the actual computation begins. The tile scanning module pre-fetches and organizes the data tiles, so that when computation starts, all necessary data is already available in the cache. This preliminary action eliminates the need for high-speed memory bus access during computation, reducing power consumption while maintaining high computing performance.
3Loss of time
If the tile block size is increased to reduce the number of memory access operations, then the memory access overhead is reduced, but the cache memory requirements increase
Solution Approach 1:
The patent dynamically adjusts the tile block dimensions (width, height, and depth parameters) to optimize the balance between memory access efficiency and cache memory usage. By changing the tile block size parameters, the system can process larger data volumes with fewer memory access operations while ensuring that the total cache memory requirement remains within available limits. This parameter optimization resolves the contradiction between reducing memory access overhead and limiting cache memory usage.
Data Source
AI summary
A data reuse method based on a convolutional neural network accelerator includes a tile scanning module receiving command information of a command module, the command information comprising a size of a CNN job to be divided into tile blocks; a tile scanning module according to a tile. The size of the tile generates the coordinates of the tile block and sends it to the memory request module; the memory request module generates a memory read request and sends the memory read request to the memory module; the memory module sequentially returns the tile block data to the input activation In the weight buffer unit, the input activation weight buffer unit saves the received tile block data to implement data reuse and transmits the received tile block data to the calculation processing unit PE.