Multi-size Convolutional Layer for Hardware Accelerators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional convolutional neural networks with octave convolution layers are inefficient on hardware accelerators due to increased computational requirements and data movement, especially when using dedicated hardware with coarse operation granularity, which hinders performance.
Innovation Solution
An unconventional convolutional layer configures input feature maps with channels of differing sizes, resizes them, and combines them for a single convolution operation to generate output feature maps, reducing the need for additional resizing and addition operations, thus optimizing performance on hardware accelerators.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional convolutional layers with multiple resizing operations are used, then processing flexibility is maintained, but computational efficiency and hardware compatibility deteriorate
Solution Approach 1:
The input feature map is segmented into multiple input maps, each corresponding to a different channel group. This segmentation allows the convolutional layer to process different channel groups separately and efficiently, reducing the overall computational complexity while maintaining processing flexibility.
Solution Approach 2:
The patent performs preliminary resizing of input channels to match target channel sizes before the convolution operation. By pre-processing the input feature map to align channel dimensions, the system eliminates the need for multiple subsequent resizing operations during the convolution process, thereby improving computational efficiency and reducing the number of operations required.
2Productivity
If conventional convolutional layers with multiple resizing operations are used, then processing flexibility is maintained, but data movement increases
Solution Approach 1:
The patent performs preliminary resizing of input channels to match target channel sizes before the convolution operation. By pre-processing the input feature map to align channel dimensions, the system eliminates the need for multiple subsequent resizing operations during the convolution process, thereby improving computational efficiency and reducing the number of operations required.
Solution Approach 2:
The patent combines multiple channel groups into a unified convolution operation after preliminary resizing. By merging the processing of different channel groups into a single convolution pass, the system reduces redundant data movement between memory and processing units, thereby lowering energy consumption associated with data transfer.
3Adaptability or versatility
If conventional convolutional layers are used, then compatibility with standard architectures is maintained, but compatibility with dedicated hardware accelerators deteriorates
Solution Approach 1:
The patent changes the operational parameters of the convolutional layer by organizing channels into groups with predetermined sizes and performing preliminary resizing to match these sizes. This parameter reorganization aligns the convolutional layer's operations with the coarse granularity of dedicated hardware accelerators, improving hardware compatibility without significantly increasing operational complexity.
Data Source
AI summary
Improved convolutional layers for neural networks can obtain an input feature map comprising groups of channels. Each group of channels can include one or more channels having a predetermined size. The predetermined sizes can differ between the groups. The convolutional layer can generate, for each one of the groups of channels, an output channel. Generation of the output channel can include resizing the channels in the remaining groups of channels to match the predetermined size of the each one of the groups of channels. Generation can further include combining the channels in the each one of the groups with the resized channels and applying the combined channels to a convolutional sub-layer to generate the output channel.