Parallel Convolution Tiling With Precomputed Offsets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing convolutional neural networks (CNNs) face inefficiencies in multi-convolution operations due to high memory usage and lengthy execution times, particularly when using matrix-based techniques, which can strain available memory and require excessive clock cycles for address calculations.
Innovation Solution
A method involving a convolution preprocessor that precomputes offsets to optimize matrix multiplication routines, reducing latency by expanding image tiles on the fly and performing multi-convolution operations in a parallel processing system, thereby minimizing memory usage and clock cycles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If matrix-based techniques are used for multi-convolution operations, then execution time is reduced, but memory usage increases significantly
Solution Approach 1:
The patent segments the large image matrix into smaller image tiles that can be processed independently. Each tile is expanded and processed separately through the convolution operation, allowing the system to benefit from matrix-based computation speed while using only the memory required for small tiles rather than the entire image matrix. This segmentation enables efficient parallel processing of multiple tiles across available computational resources.
2Productivity
If image data is expanded into matrix form for convolution operations, then computation efficiency improves, but the size of data required exceeds available memory
Solution Approach 1:
The patent divides the large image data into smaller image tiles, each of which is expanded into a manageable matrix form for convolution operations. This segmentation allows the system to maintain computation efficiency through matrix operations while keeping the expanded data size within available memory constraints. The system processes multiple tiles sequentially or in parallel batches, effectively handling large images without requiring excessive memory.
Solution Approach 2:
The patent performs preliminary expansion of individual image tiles into matrix form before executing convolution operations, rather than expanding the entire image at once. This preliminary action on a tile-by-tile basis allows the system to prepare computation-efficient matrix representations while maintaining memory usage within acceptable limits. The expansion is performed just-in-time for each tile, enabling efficient computation without overwhelming memory resources.
3Quantity of substance
If stencil-based techniques are used for multi-convolution operations, then memory usage is reduced, but execution time increases unacceptably
Solution Approach 1:
The patent changes the computational approach by transforming the convolution operation into matrix multiplication form, fundamentally altering the mathematical parameters of the operation. This parameter change enables the system to achieve both reduced memory usage (by processing tiles) and improved execution time (through optimized matrix multiplication routines). The system leverages highly optimized linear algebra libraries that provide superior performance compared to traditional stencil-based approaches.
4Productivity
If the entire image matrix is expanded for convolution operations, then computation speed improves, but available memory is completely consumed
Solution Approach 1:
The patent segments the image into smaller tiles that are processed independently, allowing the system to maintain high computation speeds through matrix operations while using only a fraction of the memory that would be required to expand the entire image. The segmentation approach enables the system to process large images by dividing them into manageable chunks that fit within available memory, achieving both speed and memory efficiency.
Solution Approach 2:
The patent performs preliminary expansion and processing of image tiles in a staged manner, expanding only the necessary portions of the image at any given time. This preliminary action on subsets of the image allows the system to achieve computation speed benefits of matrix operations while avoiding the memory consumption problem of expanding the entire image simultaneously. The system manages memory resources efficiently by controlling the expansion scope.
Data Source
AI summary
In one embodiment of the present invention, a convolution engine configures a parallel processing pipeline to perform multi-convolution operations. More specifically, the convolution engine configures the parallel processing pipeline to independently generate and process individual image tiles. In operation, for each image tile, the pipeline calculates source locations included in an input image batch based on one or more start addresses and one or more offsets. Subsequently, the pipeline copies data from the source locations to the image tile. The pipeline then performs matrix multiplication operations between the image tile and a filter tile to generate a contribution of the image tile to an output matrix. To optimize the amount of memory used, the pipeline creates each image tile in shared memory as needed. Further, to optimize the throughput of the matrix multiplication operations, the values of the offsets are precomputed by a convolution preprocessor.