Bracket-Style CNN for Balanced Local and Global Context
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image segmentation methods struggle to balance local and global context information effectively, leading to suboptimal pixel-wise labeling in applications like autonomous driving and augmented reality, where semantic segmentation is crucial.
Innovation Solution
The proposed solution involves a bracket-style convolutional neural network (B-Net) with an encoding module using residual blocks and a decoding module featuring Attention-embedded Threefold Fusion (ATF) modules, which combine high-resolution and low-resolution feature maps through upsampling, attention mechanisms, and summation units to generate prediction maps with balanced local and global context.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If shallow layers of CNN are used to capture finely patterned features, then local information is preserved, but semantic properties are weakly learned
Solution Approach 1:
The decoder merges feature maps from multiple encoder layers with different resolutions, combining finely patterned features from shallow layers with semantically rich features from deep layers through upsampling and concatenation operations
Solution Approach 2:
The network transitions from 2D spatial feature maps to 3D feature volumes by adding the channel dimension through multi-scale feature concatenation, allowing simultaneous representation of local and global context
2Loss of information
If deep layers of CNN are used to acquire semantically rich information, then global context is captured, but spatial resolution is decreased
Solution Approach 1:
The encoder performs preliminary feature extraction at multiple resolution levels, preserving semantically rich information from deep layers while maintaining corresponding spatial details through multi-scale feature map generation
Solution Approach 2:
The decoder restores spatial resolution by upsampling low-resolution feature maps and concatenating them with high-resolution feature maps from shallower layers, effectively recovering spatial details in the channel dimension
3Measurement precision
If traditional upsampling strategies are used to produce densely labeled output, then output resolution is restored, but balance between local and global context is not achieved
Solution Approach 1:
The network dynamically adapts feature fusion at each decoding level by selectively combining feature maps from different encoder layers based on their complementary strengths, achieving balanced context integration
Solution Approach 2:
Different parts of the feature pyramid are utilized for different purposes: shallow layers provide local detail information while deep layers provide global semantic context, with each layer optimized for its specific function
Data Source
AI summary
The disclosure relates to an image segmentation method and apparatus, and a computer program thereof. The image segmentation apparatus according to the disclosure may include an encoding module configured to obtain a plurality of feature maps having different resolutions with respect to an input image, using an artificial neural network including one or more residual blocks, and a decoding module configured to generate one prediction map from pairs of adjacent feature maps among the plurality of feature maps in bracket-shaped manner, wherein the decoding module performs one or more decoding rounds, each decoding round includes one or more ATF modules that generate a combined feature map using a high-resolution feature map having a higher resolution among an adjacent feature map pair constituting the feature map generated in the previous round and a low-resolution feature map having a lower resolution thereof, and the decoding round is repeatedly performed until the one prediction map is generated. According to the disclosure, it is possible to combine local information and global context in a balanced manner.