Indirect Sample Access for Memory-Efficient Multi-Convolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing convolutional neural networks (CNNs) face inefficiencies in multi-convolution operations due to high memory usage and lengthy computation times, particularly when using matrix-based techniques, which can strain available memory and require excessive clock cycles for address calculations.
Innovation Solution
A method involving a convolution preprocessor that precomputes offsets to optimize memory usage by expanding image tiles on the fly, reducing latency in address calculations and enabling efficient matrix multiplication operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If matrix-based techniques are used to perform multi-convolution operations, then computation time is reduced, but memory usage increases significantly
Solution Approach 1:
The patent segments the image batch into multiple image tiles and processes them independently. Instead of expanding the entire image batch into a full matrix, the system divides it into manageable tiles that can be processed separately, reducing peak memory usage while maintaining the computational efficiency of matrix-based convolution operations.
Solution Approach 2:
The patent precomputes offset sequences that map source addresses in the image batch to destination addresses in the expanded image matrix. By calculating these offsets in advance, the system avoids performing expensive address calculations during the actual convolution operation, reducing computation time while using minimal additional memory.
2Productivity
If image batch is expanded into full matrix form, then convolution operations can be performed efficiently, but available memory is completely consumed
Solution Approach 1:
The patent divides the image batch into multiple image tiles and processes each tile independently through the convolution operation. This segmentation allows the system to maintain the efficiency of matrix-based operations on smaller data subsets rather than requiring memory for the entire expanded matrix at once.
Solution Approach 2:
The patent processes image tiles in a streaming manner, where each tile is expanded, processed, and then discarded before moving to the next tile. This approach recovers memory space after each tile processing cycle, allowing the system to handle large image batches without consuming all available memory simultaneously.
3Reliability
If address calculations are performed during convolution operations, then data can be accessed correctly, but excessive clock cycles are required
Solution Approach 1:
The patent precomputes offset sequences that define the mapping between source addresses in the image batch and destination addresses in the expanded image matrix. By calculating these offsets beforehand, the system eliminates the need for complex address calculations during the actual convolution operation, significantly reducing clock cycle consumption while ensuring correct data access.
Solution Approach 2:
The patent introduces an offset sequence as an intermediary data structure that simplifies address mapping. Instead of performing direct address calculations during convolution, the system uses precomputed offsets as a mediator to translate source addresses to destination addresses, reducing computational overhead and improving execution speed.
Data Source
AI summary
In one embodiment of the present invention, a convolution engine configures a parallel processing pipeline to perform multi-convolution operations. More specifically, the convolution engine configures the parallel processing pipeline to independently generate and process individual image tiles. In operation, for each image tile, the pipeline calculates source locations included in an input image batch based on one or more start addresses and one or more offsets. Subsequently, the pipeline copies data from the source locations to the image tile. The pipeline then performs matrix multiplication operations between the image tile and a filter tile to generate a contribution of the image tile to an output matrix. To optimize the amount of memory used, the pipeline creates each image tile in shared memory as needed. Further, to optimize the throughput of the matrix multiplication operations, the values of the offsets are precomputed by a convolution preprocessor.