Matrix Transfer Accelerator for CNN Feature Map Tiling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Efficient data transfer between external and local data memories with different access times is hindered by memory alignment and transfer length restrictions, particularly in processing large feature maps within convolutional neural networks, leading to suboptimal performance due to increased data transfer overhead.
Innovation Solution
A matrix transfer accelerator system that utilizes a data transfer processor to interface external and local data memories, employing column tile processing, padding, and zero-fill DMA controllers to optimize data movement, allowing for efficient transfer of large feature maps by breaking data into manageable tiles and overlapping computation and data transfer operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is transferred between external and local data memories using conventional methods, then data transfer is possible, but data transfer overhead increases and performance decreases due to memory alignment and transfer length restrictions
Solution Approach 1:
The patent segments large feature maps into smaller tiles that can be efficiently transferred between external and local data memories. By dividing the data into manageable chunks, the system avoids the performance penalties associated with transferring large contiguous blocks of data, thereby reducing data transfer overhead and improving overall data transfer efficiency.
Solution Approach 2:
The patent implements preliminary action by pre-loading data tiles into local data memory before they are needed for computation, and by preparing transfer buffers in advance. This allows the system to minimize idle time and reduce data transfer overhead by ensuring that data is ready for immediate processing when computation begins.
2Productivity
If large feature maps are processed in conventional memory systems, then processing is possible, but local memory requirements increase leading to suboptimal performance
Solution Approach 1:
The patent applies segmentation by dividing large feature maps into smaller tiles that can be processed in stages. This allows the system to work with smaller data blocks in local memory at any given time, reducing the volume of stationary memory required while maintaining high processing speed through efficient tile-by-tile computation.
Solution Approach 2:
The patent transforms the traditional memory access pattern by introducing a tile-based dimensional organization. Instead of processing entire feature maps as single large blocks, the system processes data in two-dimensional tiles, allowing for more efficient memory utilization and reduced local memory requirements while maintaining processing throughput.
3Productivity
If data transfer operations are performed without optimization, then simplicity is maintained, but data movement overhead increases reducing overall system performance
Solution Approach 1:
The patent introduces an intermediary data transfer mechanism that manages the movement of tiled data between external and local data memories. This intermediary layer optimizes transfer operations by handling alignment and buffering, thereby reducing data movement overhead and improving system performance while adding manageable complexity to the data transfer mechanism.
Data Source
AI summary
A matrix transfer accelerator (MTA) system/method that coordinates data transfers between an external data memory (EDM) and a local data memory (LDM) using matrix tiling and/or grouping is disclosed. The system utilizes foreground/background buffering that overlaps compute and data transfer operations and permits EDM-to-LDM data transfers with or without zero pad peripheral matrix filling. The system may incorporate an automated zero-fill direct memory access (DMA) controller (ZDC) that transfers data from the EDM to the LDM based on a set of DMA controller registers including data width register (DWR), transfer count register (TCR), fill count register (FCR), EDM source address register (ESR), and LDM target address register (LTR). The ZDC transfers matrix data from the EDM[ESR] to the LDM[LTR] such that EDM matrix data of DWR row data width is automatically zero-filled around a periphery of a matrix written to the LDM matrix based on the FCR value.