Dilated Convolution Hardware Implementation via Kernel Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network accelerator hardware faces inefficiencies in implementing dilated convolutions, which are essential for tasks like object detection and image segmentation, due to the need for large kernel sizes and increased computational overhead.
Innovation Solution
The method involves splitting the kernel into constituent convolutions with smaller kernels or dividing the input data into parts, allowing existing hardware to evaluate these constituent convolutions and combine their results to produce the dilated convolution output, optimizing performance by selecting the best strategy based on dilation rate and convolution type.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If dilated convolution is implemented directly on existing neural network accelerator hardware, then the hardware can process dilated convolutions, but the computational overhead and execution time increase significantly
Solution Approach 1:
The dilated convolution operation is segmented into multiple constituent convolutions with smaller kernels. Instead of applying one large dilated kernel directly, the method divides the computation into several smaller convolution operations that can be executed more efficiently by existing hardware, reducing the overall execution time while maintaining the same receptive field.
Solution Approach 2:
The patent transforms the problem from the spatial domain to the channel dimension by performing multiple constituent convolutions with different input channel subsets. This dimensional transformation allows the hardware to process dilated convolutions more efficiently by utilizing its existing parallel processing capabilities across channels rather than across the expanded spatial receptive field.
2Area of stationary object
If dilated convolution uses larger kernel sizes to increase receptive field, then the coverage area increases, but the computational complexity and memory usage increase
Solution Approach 1:
The large dilated kernel is segmented into multiple smaller constituent kernels, each operating on a subset of input channels. This segmentation reduces the memory footprint and computational complexity while achieving the same effective receptive field through the combination of multiple smaller convolutions.
Solution Approach 2:
Each constituent convolution operates with local quality on a specific subset of input channels with a smaller kernel size, rather than applying a single large kernel globally across all channels. This local processing approach reduces the overall computational complexity while maintaining the global receptive field coverage through aggregation of results.
3Adaptability or versatility
If dilated convolution is implemented with standard convolution hardware, then existing hardware can be used, but the power consumption and energy efficiency decrease
Solution Approach 1:
The power-consuming dilated convolution operation is segmented into multiple smaller constituent convolutions that can be executed using existing hardware efficiently. By breaking down the computation, the method reduces redundant calculations and memory accesses, thereby lowering overall power consumption while maintaining hardware compatibility.
Solution Approach 2:
Instead of performing a complete dilated convolution with all its computational overhead, the method performs partial convolutions through multiple constituent operations, each handling a subset of the computation. This partial action approach reduces energy consumption by avoiding redundant computations that would occur in a direct dilated convolution implementation.
Data Source
AI summary
A method and data processing system implement dilated convolution operations in hardware. Embodiments provide various ways to implement a dilated convolution based on a number of constituent convolutions, by either splitting the kernel to construct a set of constituent convolutions with smaller kernels, or dividing the input data into multiple parts and applying a convolution to each part separately. The constituent convolutions are evaluated in hardware and their results are combined to produce the result of the dilated convolution.