Hardware Accelerator Tile Caching for Neural Network Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network hardware accelerators face significant computational delays and increased processing power consumption when handling image enhancement tasks due to the high volume of feature data and frequent data interactions with off-chip memory.
Innovation Solution
A hardware accelerator with a data scheduler that schedules MAC operations between an internal buffer unit and a processing element array, utilizing a line-input method where image lines are longer in the X-dimension and shorter in the Y-dimension, with overlapping pixel lines subjected to MAC operations and operation results from previous tiles cached for use in subsequent tiles, reducing the need for frequent off-chip memory access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If frequent data interactions with off-chip memory are performed for image enhancement tasks, then feature data can be processed, but computational delays and processing power consumption increase significantly
Solution Approach 1:
The patent segments the image data into multiple tiles and processes them in parallel using a PE array. Each tile is independently processed, allowing simultaneous computation on different parts of the image, which reduces overall processing time and computational delay while maintaining high productivity
Solution Approach 2:
The patent loads multiple image lines and tiles into the internal buffer unit before processing begins. This preliminary data loading ensures that data is ready for immediate processing, reducing the need for frequent off-chip memory accesses during computation and thereby minimizing computational delays
2Productivity
If frequent data interactions with off-chip memory are performed for image enhancement tasks, then feature data can be processed, but processing power consumption increases
Solution Approach 1:
The patent merges multiple data access operations into fewer, more efficient transactions. By organizing data loading and processing operations strategically, the system reduces the frequency of off-chip memory interactions, thereby lowering power consumption while maintaining processing capability
Solution Approach 2:
The patent implements continuous processing of image data through the PE array without interruption. Once data is loaded into the internal buffer, processing continues seamlessly across multiple tiles and image lines, eliminating idle periods and reducing the total energy consumption associated with frequent data access cycles
3Productivity
If a data scheduler is added to manage MAC operations between internal buffer unit and PE array, then computational efficiency improves, but device complexity increases
Solution Approach 1:
The data scheduler acts as an intermediary component that coordinates data flow between the internal buffer unit and the PE array. This mediator manages the complexity of data scheduling and tile processing, improving computational efficiency while containing the increase in device complexity through a dedicated control unit
Data Source
AI summary
A hardware accelerator comprises a PE array, an internal buffer unit, and a data scheduler. The data scheduler obtains multiple image lines from the internal buffer unit and schedules the PE array to sequentially perform MAC (multiply-accumulate) operations on the multiple image lines. There are overlapping pixel lines between adjacent image lines, and the overlapping pixel lines are subjected to MAC operations in both of their adjacent image lines to which they belong. During the MAC operations on each image line, the PEs of the PE array are scheduled to perform MAC operations in tiles on multiple tiles included in each image line. For adjacent tiles, the operation result of the overlapping portion between the previous tile and the subsequent tile is cached, and combined with the operation result of the non-overlapping portion of the subsequent tile to form the MAC operation result of the subsequent tile.