IM2COL Transform for CNN Accelerator Memory Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) are challenging to implement in constrained devices due to the high number of MAC operations required, which consume large amounts of data and increase power consumption, especially in hardware accelerators where the IM2COL transform results in inefficiencies like multiple SRAM reads.
Innovation Solution
Optimized hardware implementations of the IM2COL transform, such as those suitable for SIMD accelerators and systolic arrays, reduce internal memory bandwidth by generating a single output column per cycle, minimizing SRAM reads and power consumption through staggered dataflow and aligned memory patterns.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the IM2COL transform is implemented in hardware accelerators, then CNN operations can be performed, but local SRAM bandwidth consumption increases due to multiple reads
Solution Approach 1:
The patent applies preliminary action by performing the IM2COL transform before the main CNN computation. The transform converts the input feature map into a format suitable for convolution operations, organizing data into columns that can be processed efficiently. This preliminary data reorganization enables subsequent MAC operations to proceed with reduced memory access requirements.
Solution Approach 2:
The patent implements dimensionality change by transforming the three-dimensional input feature map (height, width, channels) into a two-dimensional column-major representation. The IM2COL operation reorganizes spatial data into column vectors, changing the data layout from a 3D tensor to a 2D matrix format that optimizes for the specific hardware architecture's memory access patterns.
2Productivity
If the IM2COL transform is implemented in hardware accelerators, then CNN operations can be performed, but power consumption increases due to multiple SRAM reads
Solution Approach 1:
The patent applies preliminary action by performing the IM2COL transform before the main CNN computation. The transform converts the input feature map into a format suitable for convolution operations, organizing data into columns that can be processed efficiently. This preliminary data reorganization enables subsequent MAC operations to proceed with reduced memory access requirements.
Solution Approach 2:
The patent implements dimensionality change by transforming the three-dimensional input feature map (height, width, channels) into a two-dimensional column-major representation. The IM2COL operation reorganizes spatial data into column vectors, changing the data layout from a 3D tensor to a 2D matrix format that optimizes for the specific hardware architecture's memory access patterns.
3Speed
If the IM2COL transform generates multiple output columns per cycle, then processing speed increases, but internal memory bandwidth increases
Solution Approach 1:
The patent applies local quality by optimizing the dataflow architecture to generate exactly one output column per clock cycle rather than multiple columns. This localized optimization at the dataflow level matches the computational capacity of the MAC array, ensuring that memory bandwidth consumption is synchronized with processing capability. The staggered dataflow pattern ensures that each cycle's memory access is precisely matched to the single column generation rate.
Data Source
AI summary
A system, apparatus and method for utilizing a transpose function to generate a two-dimensional array from three-dimensional input data. The use of the transpose function reduces redundant elements in the resultant two-dimensional array thereby increasing efficiency and decreasing power consumption.