Convolution Operation Efficiency via Matrix Decompression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional Convolutional Neural Networks (CNNs) face inefficiencies in convolution operation speed, leading to prolonged training and reasoning times due to the slow process of multiplying elements in a convolution kernel filter with input feature maps and accumulating results.
Innovation Solution
A method that involves loading target feature tiles from a preset memory layout, decompressing them according to convolution parameters, and performing matrix multiplication to improve convolution operation efficiency, allowing for simultaneous execution of matrix decompression and multiplication without needing to convert the entire feature map into a super large matrix.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the entire feature map is converted into a super large matrix for matrix multiplication, then the convolution operation can be performed, but the storage requirements and processing complexity increase significantly
Solution Approach 1:
The patent divides the feature map into multiple smaller feature tiles (e.g., 4x4, 8x8, or 16x16 blocks) instead of processing the entire feature map as one large matrix. This segmentation allows the system to perform matrix multiplication on manageable sub-matrices, reducing memory requirements and processing complexity while maintaining overall convolution operation efficiency.
Solution Approach 2:
The patent transforms the 2D feature map into a multi-dimensional structure by organizing feature tiles in a specific memory layout with dimensions including batch processing, channel, and position. This dimensional reorganization enables efficient matrix multiplication operations without requiring the entire feature map to be stored as a single large matrix, thus reducing storage requirements.
2Productivity
If conventional element-by-element multiplication is used for convolution, then the operation can be performed, but the training and reasoning time increases
Solution Approach 1:
The patent replaces the conventional mechanical element-by-element multiplication process with a matrix multiplication approach. By organizing feature tiles and convolution kernels into matrix structures and using optimized matrix multiplication algorithms, the system achieves significantly faster computation compared to traditional nested loop implementations, thereby reducing training and reasoning time.
Solution Approach 2:
The patent changes the fundamental parameters of the convolution operation by transforming it from element-wise operations to matrix multiplication operations. This parameter change enables parallel processing and utilizes optimized linear algebra libraries, resulting in substantial performance improvements and reduced computation time.
3Quantity of substance
If the feature map is tiled and stored in a specific memory layout, then the storage requirements are reduced, but the data arrangement complexity increases
Solution Approach 1:
The patent segments the feature map into smaller feature tiles and organizes them in a specific memory layout that groups related tiles together. This segmentation allows the system to store only the necessary portions of the feature map in memory at any given time, reducing overall storage requirements while maintaining efficient access patterns for matrix multiplication operations.
Solution Approach 2:
The patent performs preliminary organization of feature tiles into the desired memory layout before the actual convolution computation. By pre-arranging the data in an optimized structure with proper padding and alignment, the system reduces the complexity of data access during computation and minimizes the need for additional data manipulation steps.
Data Source
AI summary
Convolution operation method and apparatus, matrix decompression device and graphics processor are provided. The method includes: loading, from a preset memory layout, at least one target feature tile constituting any sub-feature map in an original feature map for the any sub-feature map; the memory layout being obtained by writing at least one feature tile into memory according to preset way of data arrangement; the at least one feature tile being obtained by tiling the original feature map; decompressing a feature map which is composed of the at least one target feature tile according to a convolution parameter of a convolutional layer to obtain a destination decompressed matrix; performing a matrix multiplication operation on the destination decompressed matrix and the decompressed matrix corresponding to a convolution kernel to obtain a convolution operation result of the original feature map. The present disclosure may improve the convolution operation efficiency.