Decomposing Deconvolution into Multiple Convolutions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep neural networks (DNNs) face inefficiencies in deconvolution operations due to the need for upsampling, which is poorly implemented on hardware and increases memory footprint, leading to computational resource constraints.
Innovation Solution
Decomposing deconvolution into multiple convolutions with reduced kernels, allowing for efficient hardware implementation and reduced memory usage by generating upsampled input tensors and interleaving output activations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If deconvolution is implemented using traditional upsampling operations, then the output tensor size is correctly restored, but hardware implementation efficiency deteriorates and memory footprint increases
Solution Approach 1:
The patent segments the deconvolution operation into multiple separate convolution operations with reduced kernels. Instead of using a single deconvolution layer that requires upsampling, the method divides the computation into several convolution layers that process the input tensor multiple times with progressively smaller kernels, eliminating the need for inefficient upsampling hardware implementation while maintaining correct output tensor size restoration.
Solution Approach 2:
The patent changes the kernel size parameter dynamically across multiple convolution operations. Starting with a larger kernel in early convolutions and progressively reducing the kernel size in subsequent convolutions, the method achieves the same effect as deconvolution with upsampling but using standard convolution operations that are more efficiently implementable on hardware and require less memory.
2Length of moving object
If deconvolution uses upsampling operations, then the output tensor dimensions are correctly expanded, but memory footprint doubles
Solution Approach 1:
The patent segments the dimension expansion into multiple steps, each handled by a separate convolution operation with reduced kernel. By processing the tensor through multiple convolutions with progressively smaller kernels, the method achieves the same dimensional expansion as deconvolution without requiring the large temporary buffers that upsampling operations need, thus reducing memory footprint.
Solution Approach 2:
The patent uses multiple partial convolution operations with reduced kernels instead of a single full deconvolution operation. Each convolution processes only a portion of the computation needed for the final output, and by accumulating results across multiple passes with smaller kernels, the method achieves the same dimensional expansion with significantly reduced memory requirements.
3Device complexity
If traditional deconvolution is used, then the operation is implemented as a single layer, but computational resource requirements increase
Solution Approach 1:
The patent segments the deconvolution computation into multiple convolution layers with reduced kernels. Instead of implementing deconvolution as a single complex operation that requires substantial computational resources, the method divides the work across multiple simpler convolution layers, each consuming fewer resources individually but collectively achieving the same transformation with reduced total computational burden.
Solution Approach 2:
The patent dynamically changes the kernel size parameter across multiple convolution operations, using progressively smaller kernels in subsequent convolutions. This parameter change strategy reduces the computational complexity of each individual convolution operation, thereby reducing the total computational resource requirements compared to a single deconvolution layer while maintaining the correct output tensor dimensions.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A deconvolution can be decomposed into multiple convolutions. Results of the convolutions constitute an output of the deconvolution. Zeros may be added to an input tensor of the deconvolution to generate an upsampled input tensor. Subtensors having the same size as the kernel of the deconvolution may be identified from the upsampled input tensor. A subtensor may include one or more input activations and one or more zeros. Subtensors having same distribution patterns of input activations may be used to generate a reduced kernel. The reduced kernel includes a subset of the kernel. The position of a weight in the reduced kernel may be the same as the positions of an input activation in the subtensor. Multiple reduced kernels may be generated based on multiple subtensors having different distribution patterns of activations. Each of the convolutions may use the input tensor and a different one of the reduced kernels.