Butterfly Transform Layer for CNN Channel Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) face computational complexity challenges, particularly in resource-constrained edge devices, due to the high computational requirements of point-wise convolutional layers, which hinder efficient execution and resource utilization.
Innovation Solution
The introduction of a lightweight butterfly transform (BFT) layer, which reduces computational complexity by using a separable depth-wise convolution and channel fusion through log(n) sub-layers, mimicking the Fast Fourier Transform (FFT) algorithm structure, thereby reducing the number of edges and operations needed for channel fusion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If point-wise convolutional layers are used for channel fusion, then channel fusion can be achieved, but computational complexity becomes quadratic in the number of channels
Solution Approach 1:
The patent segments the channel fusion operation into multiple stages using depth-wise convolutions followed by point-wise convolutions, rather than performing a single quadratic point-wise convolution. This segmentation reduces the computational complexity from O(n²) to O(nk²) where k is the reduction ratio, making channel fusion feasible on resource-constrained devices
Solution Approach 2:
The patent introduces an intermediate dimension by expanding channels through depth-wise convolutions before fusion, creating a bottleneck layer with fewer channels. This dimensional transformation allows the subsequent point-wise convolution to operate on a reduced dimension, significantly lowering computational complexity while maintaining fusion capability
2Adaptability or versatility
If CNNs are deployed on resource-constrained edge devices, then wider accessibility is achieved, but computational complexity challenges arise
Solution Approach 1:
The convolutional layer is segmented into depth-wise and point-wise operations, allowing efficient implementation on edge devices with limited computational resources. This segmentation enables the model to run on mobile and embedded devices while maintaining performance
Solution Approach 2:
The patent changes the parameter configuration by using different kernel sizes and channel reduction ratios in depth-wise and point-wise convolutions, optimizing the balance between computational complexity and performance for resource-constrained devices
Data Source
AI summary
Input data having multiple channels may be received and passed through a convolutional neural network model to generate output data. Passing the input data through the convolutional neural network model may include passing the input data through a depth-wise convolutional layer configured to perform a convolution on the input data for each channel of the input data to generate first data. The first data is passed from the depth-wise convolutional layer through a butterfly transform layer comprising multiple sub-layers configured to perform a linear transformation of the first data to fuse the channels of the first data and generate second data, wherein the output data is based on the generated second data. The output data may be provided for further processing on a computing device.