Convolution Layer Processing Method Reducing Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing convolution neural network methods for image and video processing face inefficiencies in memory usage and performance due to overlapping data points and increased memory requirements, particularly with larger filters, leading to poor performance and high power consumption.
Innovation Solution
A processing method and apparatus for single-channel and multi-channel convolution layers that divide data into blocks, allowing graphic processor thread groups to read and calculate convolutions directly without unfolding data, reducing memory access and data movement, and utilizing a register dividing method to minimize memory access calculations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the filter acts on the image directly to calculate the convolution with graphic processor thread group in two-dimension mode, then the convolution calculation can be performed, but input data points corresponding to adjacent output data points overlap with each other, causing a large number of data to be read into the local memory repeatedly, resulting in poor performance
Solution Approach 1:
The patent segments the input image data into distinct non-overlapping blocks and assigns each block to a specific thread group for processing. This segmentation eliminates the repeated reading of overlapping data points that occurs in traditional 2D mode convolution, where adjacent output data points share common input data. By dividing the work into independent segments, each thread group processes its assigned block without redundant memory accesses, thereby improving both performance and reducing power consumption.
2Productivity
If all image data is unfolded one data point by one data point according to the size of filter, then the convolution calculation is transferred to a dense matrix multiplication, but the needed memory space is in direct proportion to the size of convolution kernel, greatly increasing the global memory cost of the graphic processor
Solution Approach 1:
The patent divides the image data into multiple blocks that are processed independently by different thread groups. Instead of unfolding the entire image data point by point as in traditional matrix multiplication approaches, each thread group processes its assigned block directly with the filter. This segmentation approach maintains computational efficiency while dramatically reducing the memory space required, as each thread group only needs to store its assigned block and the filter, rather than storing the entire unfolded image.
Solution Approach 2:
The patent transitions from the traditional 2D mode where threads are arranged in a two-dimensional grid to a block-based processing mode that effectively adds a spatial dimension of organization. By organizing computation around data blocks rather than individual data points, the patent creates a more efficient memory hierarchy where frequently accessed data resides in local memory blocks, reducing global memory access requirements.
3Speed
If the graphic processor threads store the data to be processed and convolution kernel data in the local memory of a high-speed graphic processor, then the read number of the global memory of a low-speed graphic processor thread is decreased, but the data movement needs to be reduced further to decrease power consumption and increase the ratio of performance to power consumption
Solution Approach 1:
The patent segments data into blocks that are loaded once into local memory for processing by multiple threads. This segmentation strategy reduces the frequency of global memory reads by keeping segmented data blocks in faster local memory, thereby decreasing power consumption associated with global memory access while maintaining high processing speed.
Data Source
AI summary
A processing method and a processing apparatus for a single-channel convolution layer, and a processing method and apparatus for a multi-channel convolution layer are provided. The processing method for a single-channel convolution layer includes following steps. Data to be processed is divided into a plurality of data blocks, the plurality of data blocks are read by a plurality of graphic processor thread groups into local memories of the plurality of graphic processor thread groups, a plurality of filters are read by the plurality of graphic processor thread groups into the local memories of the plurality of graphic processor thread groups and convolutions of corresponding data points in the plurality of data blocks and the plurality of filters simultaneously are calculated by a plurality of threads in the plurality of graphic processor thread groups.