U-Net Skip-Connection Compression for Lower Memory Overhead
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The U-Net neural network model faces significant storage space overhead due to skip connections, which is exacerbated by the large memory required for feature maps during hardware deployment, and existing methods to reduce this overhead, such as removing or merging connections, lead to performance degradation.
Innovation Solution
A method involving compressing and enhancing feature maps in the encoding stage of a U-Net model to reduce storage space while maintaining performance, by fusing multi-resolution feature maps into single-resolution maps and enhancing them back to original resolution and channel counts, using techniques like channel-wise merging, addition, multiplication, and convolution operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If skip connections are retained in the U-Net model to preserve detailed information, then model performance is improved, but storage space overhead increases significantly
Solution Approach 1:
The patent merges multiple feature maps from different encoding stages into a single feature map using channel-wise operations. Instead of storing separate feature maps E1, E2, E3, E4 for skip connections, the patent combines them into one consolidated feature map that contains all necessary information, significantly reducing storage space while maintaining the integrity of detailed information needed for decoding
Solution Approach 2:
The patent transforms the feature maps by changing their parameters - specifically, it adjusts the channel dimensions and applies compression operations. The feature maps are compressed to reduce their size while preserving essential information, and then enhanced during decoding to restore the necessary detail level, effectively managing storage requirements without sacrificing model performance
2Quantity of substance
If feature maps are compressed to reduce storage space, then storage space overhead is reduced, but detailed information may be lost
Solution Approach 1:
The patent performs preliminary compression of feature maps in the encoding stage before they are needed in the decoding stage. By pre-compressing and storing the consolidated feature map, the system reduces storage requirements upfront while maintaining all necessary information. During decoding, the compression is reversed or undone to restore the full detail level when needed
Solution Approach 2:
The patent applies reversible parameter changes to feature maps - compressing them for storage by adjusting channel dimensions and applying compression operations, then enhancing them during decoding by reversing these operations. This allows the same feature map to serve both storage efficiency and information preservation requirements at different stages
Data Source
AI summary
The present disclosure provides a generation method, an application method, a training apparatus and an application apparatus for a neural network model, a storage medium, and a computer program product. The generation method comprises: compressing feature maps generated in an encoding stage of a U-shaped neural network model or a variant thereof, wherein the feature maps are connected to a decoding stage of the U-shaped neural network model or the variant thereof via skip connections, wherein the U-shaped neural network model or the variant thereof includes at least an encoding stage and the decoding stage for processing image data; compressing, in the encoding stage, the generated feature map to be connected to the decoding stage; and generating, in the decoding stage, enhanced feature maps from the compressed feature maps.