CNN Encoder-Decoder Fusion for Memory Bandwidth Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing CNN architectures, particularly U-Nets, face inefficiencies due to significant memory read/write operations across layers, limiting computational speed and efficiency in applications like image and voice recognition.
Innovation Solution
The proposed solution involves a fusion approach within encoder and decoder blocks of CNNs, integrating feature networks with filter networks to process operations in parallel, reducing the need for external memory reads and writes, and utilizing a novel U-Net architecture where output feature maps from these blocks are fused and directly provided to corresponding stages of the filter network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is processed through each layer of the network with separate memory read/write operations, then each layer can be processed independently, but the volume of memory operations increases significantly, limiting computational efficiency
Solution Approach 1:
The patent merges multiple separate memory operations into a single unified memory operation. Specifically, it combines the reading of input data and the writing of output data for multiple layers into single read and write operations, thereby reducing the total volume of memory operations while maintaining the ability to process data through multiple layers independently.
2Speed
If traditional CNN architectures are used with sequential layer processing, then the network structure is simpler to implement, but the processing speed is limited by frequent memory access
Solution Approach 1:
The patent implements preliminary action by pre-loading all necessary input data into high-speed memory before processing begins. This allows multiple layers to access their required data from fast memory without requiring frequent accesses to slower system memory, thereby increasing processing speed while managing architectural complexity through structured data preparation.
3Productivity
If separate processing of encoder and decoder blocks is performed, then each block can be optimized independently, but the overall computational efficiency is reduced due to additional memory operations
Solution Approach 1:
The patent merges the processing of encoder and decoder blocks by implementing a unified processing architecture where both blocks share common memory resources and can access data simultaneously. This reduces the total number of memory operations compared to completely separate processing, while still allowing independent optimization of each block's internal operations.
Data Source
AI summary
Techniques are described for processing data in a convolutional neural network (CNN) via fused operations within encoder and/or decoder blocks of a feature network to increase computational efficiency and reduce memory usage. Padding units are added to the input data for each convolutional operation within the encoder/decoder blocks. In certain embodiments, the feature network is coupled to a filter network, forming a combined U-Net architecture in which each of one or more decoder blocks of the feature network is coupled to a corresponding block of the filter network, enabling parallel processing of the output feature maps via the corresponding blocks of the filter network.