Neural Network Image Deblurring for Mobile Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image deblurring methods are computationally complex, lead to artifacts, struggle with dynamic scenes, and are not suitable for low-power mobile devices.
Innovation Solution
An electronic device equipped with a processor and memory that uses a neural network model to perform image deblurring by inputting blurred images and camera motion information to obtain feature information, filter it using a recurrent neural network, and reconstruct a deblurred image.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If conventional image deconvolution or filtering methods are used, then deblurring can be achieved, but computational complexity increases and processing speed decreases
Solution Approach 1:
The patent replaces traditional iterative deconvolution algorithms with a neural network-based approach. The neural network learns the deblurring transformation during training and can perform rapid inference on blurred images, substituting complex mechanical computation with learned pattern recognition that achieves both high quality and speed.
Solution Approach 2:
The neural network model is pre-trained offline using extensive training data and computations. During deployment, the pre-computed network weights enable rapid processing without requiring time-consuming iterative calculations, effectively performing the heavy lifting in advance during model development rather than during actual image processing.
2Manufacturing precision
If iterative deconvolution algorithms are used, then deblurring quality improves, but energy consumption increases
Solution Approach 1:
The patent replaces energy-intensive iterative deconvolution computations with a neural network inference system. Once trained, the network performs deblurring through efficient convolutional operations that consume significantly less energy compared to repeated iterative algorithms, making real-time processing feasible on mobile devices.
3Productivity
If existing neural networks for deblurring are used, then processing speed improves, but device complexity and computational requirements increase
Solution Approach 1:
The patent employs a U-Net architecture with skip connections that preserve local spatial information while processing images. The network maintains different levels of detail through encoder-decoder structures, allowing efficient processing by focusing computational resources where needed rather than uniformly across the entire image, thus reducing overall complexity.
Solution Approach 2:
The neural network is divided into encoder and decoder sections with intermediate skip connections. This segmentation allows the model to process images in a hierarchical manner, extracting features at multiple scales and then reconstructing the image step-by-step, which simplifies the computational graph compared to monolithic approaches.
4Adaptability or versatility
If conventional deblurring methods are used, then static scenes can be processed, but dynamic scenes with motion blur in different directions cannot be handled
Solution Approach 1:
The patent uses a U-Net architecture that processes images through multiple convolutional layers with different kernel sizes and stride values. The network adapts to various blur types and motion directions by learning diverse patterns during training, effectively handling dynamic scenes without requiring separate processing paths for different blur conditions.
Data Source
AI summary
An image processing device includes an encoder configured to receive a blurry image and generate a global feature map of the image, a merging unit configured to merge the global feature map and blur information, a decoder configured to generate a feature tensor and weight tensors, a recurrent refinement module configured to perform recurrent feature filtering, and an image reconstruction module configured to reconstruct a deblurred image where the image processing device is configured to estimate an image global shift and to activate or deactivate the recurrent refinement module based on the estimation.