Convolution Kernel Layout for Matrix Accelerator Utilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The utilization ratio of matrix operation units in hardware accelerators for convolutional neural networks is low due to mismatched computation requirements, leading to increased memory space, transmission time, and underutilization of computing power, particularly in the first-layer convolution operations.
Innovation Solution
The method involves determining an operation convolution kernel based on an initial kernel, adjusting the layout of input data to match the number of channels of the operation kernel, and performing a convolution operation to achieve a result equivalent to the initial kernel, thereby optimizing data layout and utilizing the matrix operation unit's full computing power.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the initial convolution kernel layout is used directly, then the computation logic is simple, but the utilization ratio of matrix operation units is low
Solution Approach 1:
The patent segments the initial convolution kernel into multiple operation convolution kernels through layout transformation. The input data is divided into multiple feature maps, and the single convolution kernel is split into multiple smaller kernels that can be processed in parallel by matrix operation units, thereby improving utilization ratio while maintaining computational equivalence
Solution Approach 2:
The patent changes the dimensionality organization of the convolution kernel by transforming from a standard layout to a transposed layout. This dimensional reorganization allows the kernel to be viewed and processed from a different perspective, enabling better alignment with matrix operation unit requirements and improving parallel processing efficiency
2Loss of time
If data layout is not adjusted, then the operation process is simple, but the computation time is increased
Solution Approach 1:
The patent performs preliminary layout adjustment of the convolution kernel and input data before the actual convolution operation. By pre-processing the data and kernel into the transposed layout format, the computation is optimized in advance, reducing the actual computation time during the convolution operation without significantly increasing overall processing complexity
3Loss of time
If the standard convolution kernel layout is used, then the implementation is straightforward, but the data transmission time is increased
Solution Approach 1:
The patent changes the layout parameter of the convolution kernel from the standard configuration to a transposed configuration. This parameter change optimizes the data arrangement in memory, reducing the distance and steps required for data transmission between memory and processing units, thereby decreasing data transmission time
Data Source
AI summary
A convolution operation method, a convolution operation apparatus, an electronic device and a storage medium. The method includes: determining an operation convolution kernel, wherein the operation convolution kernel is obtained on the basis of an initial convolution kernel; adjusting an arrangement mode of input data on the basis of the number of channels of the operation convolution kernel, so as to obtain target data, wherein the size of the target data and the number of channels of the target data are different from the size of the input data and the number of channels of the input data, and the number of channels of the target data is equal to the number of channels of the operation convolution kernel; and performing a convolution operation on the basis of the target data and the operation convolution kernel, so as to obtain a convolution operation result.