Parallel Processing Pipeline for Multi-Convolution Tile Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for performing multi-convolution operations in Convolutional Neural Networks (CNNs) are inefficient, particularly when dealing with a wide range of dimensions and parameters, leading to unacceptably long execution times and memory constraints due to the need for large on-chip memory allocation.
Innovation Solution
A computer-implemented method that calculates source locations, copies data between memory tiles, and performs matrix multiplication operations between image and filter tiles, optimizing on-chip memory usage by processing each image tile independently and minimizing memory requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If direct calculation techniques are used for multi-convolution operations, then execution time is reduced for small subsets of dimensions, but execution time becomes unacceptably long for wide ranges of dimensions and parameters
Solution Approach 1:
The patent segments the multi-convolution operation into distinct phases: data loading phase and computation phase. It divides the image batch and filter stack into tiles that can be processed independently, allowing the system to handle wide ranges of dimensions by processing smaller manageable segments rather than requiring optimization for all possible dimension combinations simultaneously
Solution Approach 2:
The patent implements dynamic tiling where the image batch and filter stack are divided into tiles of configurable dimensions. This dynamic segmentation allows the system to adapt to different input sizes and parameter ranges by adjusting tile dimensions, enabling efficient processing across a wide range of dimensions without requiring separate optimization for each case
2Productivity
If matrix-based operations are used to reduce execution time, then CNN execution becomes significantly faster, but on-chip memory requirements increase to the point of consuming all available memory
Solution Approach 1:
The patent applies segmentation by dividing both the image batch and filter stack into smaller tiles that fit within available on-chip memory. By processing one tile pair at a time and computing partial output tiles, the system maintains the speed benefits of matrix operations while reducing memory requirements to manageable levels that can be accommodated on-chip
Solution Approach 2:
The patent implements partial action by computing only the necessary portions of the convolution operation at each step. Instead of loading entire image batches and filter stacks into on-chip memory, it loads only the required tiles, performs matrix multiplication on these partial datasets, and accumulates results, thereby using minimal on-chip memory while maintaining overall computation accuracy
3Productivity
If FFT-based techniques are used for multi-convolution operations, then some operations execute more efficiently, but execution time becomes prohibitively long when horizontal stride or vertical stride is greater than one
Solution Approach 1:
The patent implements a universal tiling approach that works efficiently across different stride values and parameter combinations. The tile-based matrix multiplication method provides consistent performance whether strides are 1 or greater, eliminating the performance degradation that FFT-based techniques suffer when strides exceed one, thereby achieving multi-functionality across various operational parameters
Data Source
AI summary
In one embodiment of the present invention a convolution engine configures a parallel processing pipeline to perform multi-convolution operations. More specifically, the convolution engine configures the parallel processing pipeline to independently generate and process individual image tiles. In operation, for each image tile, the pipeline calculates source locations included in an input image batch. Notably, the source locations reflect the contribution of the image tile to an output tile of an output matrix--the result of the multi-convolution operation. Subsequently, the pipeline copies data from the source locations to the image tile. Similarly, the pipeline copies data from a filter stack to a filter tile. The pipeline then performs matrix multiplication operations between the image tile and the filter tile to generate data included in the corresponding output tile. To optimize both on-chip memory usage and execution time, the pipeline creates each image tile in on-chip memory as-needed.