Parallel Neural Network Convolutions with Im2Col Data Reuse
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Im2Col implementations in convolutional neural networks (CNNs) with dilation 1 and stride smaller than the kernel size result in duplicate data loading, leading to inefficient memory bandwidth usage and increased computational demands on microcontrollers with limited resources.
Innovation Solution
A method for parallelized calculation of two convolutions by reordering kernel elements and data access to avoid duplicate data loading, utilizing existing data storage for multiple output pixels without additional memory access, optimized for microcontrollers with limited resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If Im2Col implementation is used for convolution calculation, then the number of multiplications is reduced, but memory bandwidth usage increases due to duplicate data loading
Solution Approach 1:
The patent applies preliminary action by pre-loading data values into the Im2Col buffer in a specific order before the convolution calculation begins. This pre-organization of data ensures that when the filter is applied, the required data is already available in the buffer without needing to reload duplicate values from memory, thus reducing memory bandwidth usage while maintaining calculation efficiency
Solution Approach 2:
The patent changes the parameter of data organization in the Im2Col buffer by storing data in a specific sequential order that matches the filter application pattern. Instead of traditional Im2Col ordering, the patent reorders the buffer storage to contain data that will be accessed sequentially by the filter, eliminating the need to reload duplicate data and optimizing memory bandwidth utilization
2Device complexity
If partial Im2Col implementation is used on microcontrollers, then computational operations are reduced, but memory bandwidth consumption increases
Solution Approach 1:
The patent applies preliminary action by pre-loading data values into the Im2Col buffer in a specific order before the convolution calculation begins. This pre-organization of data ensures that when the filter is applied, the required data is already available in the buffer without needing to reload duplicate values from memory, thus reducing memory bandwidth usage while maintaining calculation efficiency
Solution Approach 2:
The patent changes the parameter of data organization in the Im2Col buffer by storing data in a specific sequential order that matches the filter application pattern. Instead of traditional Im2Col ordering, the patent reorders the buffer storage to contain data that will be accessed sequentially by the filter, eliminating the need to reload duplicate data and optimizing memory bandwidth utilization
Data Source
AI summary
A method is for parallelized calculation of two convolutions of a filter having first and second receptive fields of the filter on input data. The first and second receptive fields correspond to a filter shifted by one step on the input data. The method includes initializing first and second output variables each having an initial value, and executing a loop for each line of the filter. The loop performs loading a first kernel element of the filter of the line and the corresponding data values to the first kernel element of the first and second receptive fields of the input data.