Dilated Convolution Feature Map Splitting for Computational Complexity Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing methods for processing data using dilated convolution layers in convolutional neural networks face increased computational complexity and memory access bottlenecks, limiting their application scope due to the need for weight zero filling and additional data fetching logic.
Innovation Solution
The proposed method involves splitting the input feature map into local feature maps, performing zero padding and convolution processing on these maps, and then fusing the output to generate an output feature map, thereby reducing computational complexity and enabling data reuse between adjacent sliding windows.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If weight zero filling is performed on the holes in the dilated convolution kernel to form a new convolution kernel, then the dilated convolution can be implemented, but computational complexity is increased significantly
Solution Approach 1:
The patent divides the feature map into multiple local feature maps corresponding to different holes in the dilated convolution kernel. Each local feature map is processed independently through convolution operations, and the results are then combined. This segmentation approach avoids the need to create and process a large sparse convolution kernel with zeros, significantly reducing computational complexity while maintaining the ability to implement dilated convolution with various dilation rates.
2Device complexity
If the second processing manner is used to directly extract local feature maps with holes by taking the dilation rate as a stride, then the computational complexity is reduced, but a memory access bottleneck occurs
Solution Approach 1:
The patent merges the extraction of multiple local feature maps into a unified processing framework. Instead of extracting each local feature map separately and accessing memory independently, the method combines their processing through a coordinated convolution operation that shares computational resources and memory access patterns, thereby reducing the memory access bottleneck while maintaining low computational complexity.
3Adaptability or versatility
If down-sampling is used to expand the awareness range, then the feature extraction capability is improved, but resolution of the feature map is lost and accuracy of target detection and semantic segmentation tasks is affected
Solution Approach 1:
The patent changes the parameter of the convolution operation by introducing the dilation rate, which controls the spacing between elements in the convolution kernel. This parameter change allows the kernel to cover a larger area (expanding awareness range) without reducing the resolution of the feature map, as no down-sampling is performed. The dilation rate can be adjusted to achieve different awareness ranges while maintaining full resolution throughout the network.
Data Source
AI summary
A processing method includes: obtaining an input feature map; processing the input feature map by using a dilated convolution layer of the convolutional neural network, to obtain a plurality of local feature maps; obtaining a plurality of local output feature maps by performing zero padding on the plurality of local feature maps performing convolution processing on the plurality of zero-padded local feature maps; and fusing the plurality of local output feature maps, to obtain an output feature map processed by the dilated convolution layer. A plurality of consecutive local feature maps can be split from the input feature map. The local feature map can be performed with convolution processing by using a compact convolution kernel. Performing dilated convolution processing on the input feature map under a premise of not increasing computational complexity overcomes limitation of holes on a dilated convolution algorithm, and can realize data reuse between adjacent sliding windows.