Multi-Scale LSTM Image Deblurring With Fewer Network Parameters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network-based image deblurring methods require high computational power and large memory due to a large number of parameters, limiting their use on devices with lower capabilities, and often assume linear convolution despite real-world blurring being nonlinear and shift-variant.
Innovation Solution
A compact neural network using dilated convolutions and LSTM-Inception blocks with residual connections to share context between scales, reducing the number of trainable parameters by 96% compared to SRN-DeblurNet, while achieving comparable deblurring performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If neural network-based deblurring methods are used, then image deblurring performance is improved, but computational power and memory requirements increase significantly
Solution Approach 1:
The network processes images at multiple scales (coarse to fine) by segmenting the deblurring task into hierarchical stages. Each scale processes a down-sampled version of the image, reducing computational load while maintaining overall deblurring effectiveness through progressive refinement.
Solution Approach 2:
The network embeds multiple processing scales within a unified architecture where coarser scale results are up-sampled and integrated into finer scale processing. This nested structure allows information from different resolutions to be combined, achieving high deblurring performance with reduced computational requirements compared to processing full-resolution images directly.
2Measurement precision
If standard convolutional neural networks are used for deblurring, then deblurring capability is achieved, but the number of parameters increases to millions
Solution Approach 1:
The network divides the deblurring task across multiple scales with shared parameters at each scale. By processing down-sampled images at coarser scales and sharing weights across scales, the total number of parameters is dramatically reduced while maintaining deblurring capability through hierarchical feature learning.
Solution Approach 2:
The same network parameters are used across multiple scales of image processing. Each scale uses identical convolutional layers with shared weights, allowing the network to perform deblurring at different resolutions without requiring separate parameter sets, thus reducing overall parameter count while maintaining versatility.
3Loss of information
If multi-scale processing with up-convolution is used, then context sharing between scales is improved, but computational complexity increases
Solution Approach 1:
Instead of using computationally intensive up-convolution operations to upsample coarse scale features, the network uses bilinear interpolation to create up-sampled versions of coarse scale deblurring results. This copying approach maintains context information between scales while significantly reducing computational complexity compared to learned up-convolution layers.
Data Source
AI summary
A scale-based, convolutional, long-short term memory (LSTM) network is developed for image deblurring. Multi-scale information is obtained using dilated convolutions shared between scales using recurrent connections resulting in low-parameter count and to deblur an image without the use of prior information. Effectiveness is evaluated with industry standard datasets. Results show that a comparable sharp image can be recovered more efficiently even with a significant reduction in the total number of network parameters.