Deconvolution Processing Using Optimized Convolution Hardware
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
General-purpose processors are inefficient for large-scale convolution, pooling, and deconvolution operations in neural networks due to unsuitable hardware architecture, leading to low operating efficiency and high energy consumption.
Innovation Solution
The method involves splitting a deconvolution kernel into multiple convolution kernels, optimizing them by removing invalid weights, and performing convolution operations using dedicated convolution hardware with a multiply-add array and on-chip memory, thereby reducing hardware complexity and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If general-purpose processors are used to perform deconvolution processing, then versatility is maintained, but operating efficiency deteriorates and energy consumption increases
Solution Approach 1:
The patent segments the deconvolution operation into multiple convolution operations by splitting the deconvolution kernel into multiple convolution kernels. This allows the use of dedicated convolution hardware to perform the segmented operations, achieving high efficiency while maintaining the original deconvolution functionality through software-level decomposition.
Solution Approach 2:
The patent makes convolution hardware perform multiple functions by using it to execute deconvolution operations through kernel splitting. The same convolution hardware that processes standard convolution operations is now also capable of performing deconvolution, eliminating the need for separate dedicated deconvolution hardware and improving resource utilization.
2Ease of manufacture
If general-purpose processors are used to perform deconvolution processing, then implementation simplicity is maintained, but energy consumption increases
Solution Approach 1:
The deconvolution kernel is segmented into multiple convolution kernels, enabling the use of energy-efficient convolution hardware. The segmentation transforms a single high-energy deconvolution operation into multiple lower-energy convolution operations that can be executed on optimized hardware.
Solution Approach 2:
The patent changes the operational parameters by transforming deconvolution parameters into equivalent convolution parameters through kernel splitting. This parameter transformation allows the system to operate in a more energy-efficient mode by utilizing convolution hardware optimized for such operations.
3Productivity
If deconvolution hardware is dedicatedly designed, then processing efficiency is improved, but device complexity and hardware area increase
Solution Approach 1:
The patent makes convolution hardware universal by enabling it to perform both convolution and deconvolution operations. This eliminates the need for separate dedicated deconvolution hardware, reducing device complexity and hardware area while maintaining high processing efficiency through the kernel splitting technique.
4Speed
If deconvolution hardware is dedicatedly designed, then processing speed is improved, but chip area overhead increases
Solution Approach 1:
The patent achieves high processing speed using existing convolution hardware by splitting the deconvolution kernel into multiple convolution kernels. This approach utilizes the existing high-speed convolution processing units without requiring additional dedicated deconvolution hardware, thereby avoiding chip area overhead while maintaining fast processing speeds.
5Stability of the object's composition
If invalid weights are not removed from convolution kernels, then kernel completeness is maintained, but operating efficiency deteriorates
Solution Approach 1:
The patent extracts and removes invalid weights (rows and columns with all zero values) from the convolution kernels derived from the deconvolution kernel. This extraction process eliminates unnecessary computations while preserving the essential functional components of the kernels, thereby improving operating efficiency without compromising the validity of the deconvolution operation.
Data Source
AI summary
The present disclosure provides a method and apparatus for deconvolving feature data using convolution hardware. The method includes: reading a feature map and deconvolution kernel into on-chip memory, and padding zeroes to the feature map; determining convolution kernels based on the deconvolution kernel; removing a row and/or column of each convolution kernel whose elements all are invalid weights, to obtain an optimized convolution kernel, and removing a corresponding row and/or column in the zero-padded feature map to obtain an corresponding optimized feature map; convolving each optimized convolution kernel with corresponding optimized feature map using the multiply-add array, to obtain convolutional outputs; and interleaving and synthesizing the convolutional outputs to obtain an interleaving synthetic output including at least a deconvolutional output corresponding to the feature map and deconvolution kernel. The method reduces hardware complexity, chip area and power consumption, and many invalid operations, improving operating efficiency of convolution hardware.