Super-Resolution Unfolding for Non-Uniform Motion Blur
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional super-resolution techniques struggle with non-uniform spatial blur, particularly in action camera footage, due to the arbitrary and anisotropic nature of motion blur, which cannot be effectively represented in closed-form, leading to resource-intensive processing and undesirable artifacts.
Innovation Solution
A method involving the synthesis of training data using pixel-wise overlays of spatially variant blur operators, combined with a linearized alternating direction method of multipliers (ADMM) for convolutional neural networks (CNNs), allowing for efficient super-resolution on embedded devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If conventional super-resolution techniques (arithmetic interpolation/extrapolation) are used, then processing is simpler, but image quality deteriorates with undesirable artifacts
Solution Approach 1:
The patent replaces conventional arithmetic interpolation/extrapolation methods with a neural network-based approach. The neural network learns optimal super-resolution transformations from training data, substituting mechanical/mathematical algorithms with a learned model that produces higher quality results without visible artifacts while maintaining computational feasibility through optimized network architecture.
2Productivity
If simple neural network implementations are used, then processing is faster, but ability to handle non-uniform spatial blur deteriorates
Solution Approach 1:
The patent implements spatially variant blur kernels that allow different regions of the image to have different blur characteristics. The neural network is trained with these localized, position-dependent blur patterns rather than uniform blur, enabling the model to adapt to local image variations and handle non-uniform spatial blur effectively while maintaining processing efficiency.
Solution Approach 2:
The patent introduces dynamic blur kernels that vary across space and can adapt to different regions of the image. The system uses spatially varying parameters that change depending on location, allowing the neural network to handle dynamic, non-uniform blur patterns rather than static uniform blur, thereby improving super-resolution capability for action camera footage.
3Manufacturing precision
If spatially variant blur is handled using existing neural network approaches, then more realistic blur representation is achieved, but computational complexity and resource requirements increase
Solution Approach 1:
The patent segments the blur representation into spatially variant components, dividing the image into regions with different blur characteristics. This segmentation allows the neural network to process each region with appropriate local parameters rather than applying a single global model, achieving accurate blur representation while reducing overall computational complexity through localized processing.
Solution Approach 2:
The patent changes the parameters of the blur kernel from uniform to spatially variant, introducing position-dependent parameters that accurately represent non-uniform blur. By formulating the optimization problem with these changed parameters and deriving closed-form solutions, the system achieves realistic blur representation while reducing computational burden compared to iterative numerical methods.
Data Source
AI summary
Systems, apparatus, and methods for super-resolution of non-uniform spatial blur. Non-uniform spatial blur presents unique challenges for conventional neural network processing. Existing implementations attempt to handle super-resolution with a “brute force” optimization. Various embodiments of the present disclosure subdivide the super-resolution function into sub-steps. “Unfolding” super-resolution into smaller closed-form functions allows for operation using generic plug-and-play convolutional neural network (CNN) logic. Additionally, each step can be optimized with its own step-specific hyper parameters to improve performance.