Accelerator Data Packing for Memory Access Cost Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current hardware accelerators for deep neural networks face performance bottlenecks due to inefficient storage and memory access costs, particularly in handling complex input data, which limits their operational efficiency.
Innovation Solution
The proposed solution involves packing input data with a data layout determined by the memory word width and filter size, performing operations between portions of the packed data and weights stored in the same memory word, and utilizing im2col transformation to optimize convolution operations, thereby improving storage efficiency and reducing memory access costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If input data is stored in conventional memory layout, then memory access is simple, but storage efficiency is low and memory access costs are high
Solution Approach 1:
The patent applies preliminary action by pre-packing input data into an optimized layout before storing it in memory. The data is transformed from conventional layout to a packed layout that groups data elements according to filter spatial sizes and memory word widths, so that when convolution operations are performed, multiple data elements can be accessed from the same memory word, reducing memory access costs and energy consumption.
2Reliability
If multiple convolution operations are performed with frequent memory access, then computational accuracy is maintained, but power consumption increases
Solution Approach 1:
The patent applies merging by combining multiple data elements that are needed for different convolution operations into the same memory word through data packing. This allows the accelerator to fetch multiple data elements in a single memory access operation, maintaining computational accuracy for multiple convolutions while significantly reducing the number of memory access operations and associated power consumption.
3Productivity
If data is packed according to memory word width and filter size, then memory efficiency improves, but data layout complexity increases
Solution Approach 1:
The patent applies parameter changes by transforming the data layout parameters to match the memory system parameters (word width) and computational parameters (filter spatial size). The packing process reorganizes data according to these specific parameters, creating a layout where data elements are grouped in a way that maximizes memory efficiency. While the layout becomes more complex, this complexity is justified by the significant improvement in memory efficiency and the reduction in memory access operations.
Data Source
AI summary
An accelerator, a method of operating the accelerator, and an electronic device including the accelerator. A method of operating the accelerator configured to perform a target operation includes packing input data with a data layout determined based on a word width of a memory in the accelerator and a spatial size of a filter to be applied to the target operation and storing the packed input data in the memory, and performing the target operation between a portion of the input data stored in a same word in the memory and weights of the filter.