Transposed Convolution Kernel Division to Eliminate Zero Multiplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current transposed convolution solutions suffer from reduced matrix multiplication efficiency due to zero-padding and insertion of zeros in input data, leading to low computational efficiency and high memory footprint.
Innovation Solution
Optimize input data by reducing zero padding and dividing kernels into sub-kernels for direct convolution, eliminating the need for zero multiplication and flattening, thereby improving matrix multiplication efficiency and reducing memory footprint.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of moving object
If zero-padding and insertion of additional rows and columns of zeros is applied to input data for transposed convolution, then spatial dimensions are increased, but matrix multiplication efficiency is reduced due to sparse input tensor with large number of zeros
Solution Approach 1:
The kernel is divided into multiple sub-kernels (e.g., 2x2 sub-kernels from a 4x4 kernel). Each sub-kernel operates on a specific region of the input tensor, processing only non-zero elements. This segmentation eliminates multiplication by zero while maintaining the up-sampling function, improving matrix multiplication efficiency by 30-70%.
2Productivity
If input data is flattened into linearly-stored non-zero data to eliminate zeros before matrix multiplication, then computational efficiency is improved, but memory footprint increases and the process is time-consuming
Solution Approach 1:
Instead of globally flattening the entire input tensor, the solution applies local processing where each sub-kernel operates only on its relevant region of the input tensor. This maintains the spatial structure locally while eliminating zero multiplications, improving efficiency without increasing memory footprint.
3Ease of operation
If standard convolution operation is applied over sparse input tensor with many zeros, then transposed convolution function is achieved, but computational resources are wasted on multiplying and accumulating zeros
Solution Approach 1:
The solution extracts and processes only the non-zero elements of the input tensor using sub-kernels. By taking out the essential computational work (multiplying non-zero elements) and eliminating the redundant operations (multiplying zeros), the method achieves the transposed convolution function with significantly reduced computational resource consumption.
Data Source
AI summary
A new approach is proposed that contemplates system and method to support efficient implementation of transposed convolution for machine learning (ML). Under the proposed approach, input data/tensor to a transposed convolution operation is optimized before the transposed convolution operation and each of a plurality of original kernels used for the transposed convolution operation is divided into a plurality of smaller sub-kernels. A plurality of direct sub-convolutions are then performed by sequentially applying each sub-kernel of the plurality of sub-kernels of each of the original kernels over the optimized input tensor without flattening either the input tensor or the plurality of sub-kernels. The output from the sub-convolutions using the plurality of sub-kernels are then combined as the final output tensor for each of the original kernels for the transposed convolution operation.