Adaptive Kernel Network for Video Deblurring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video processing systems face challenges in effectively deblurring motion in videos due to high memory demand and limited performance in capturing significant motion and low-light conditions, with existing techniques being cumbersome and generating artifacts.
Innovation Solution
A multi-kernel adaptive separable convolution network (MKASCN) is used to predict different sized kernels for deblurring, leveraging spatial information across frames and reducing resource demand through separable filters, allowing for improved deblurring performance without causing a resource bottleneck.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple different sized kernels are used to capture larger motions, then deblurring performance is improved, but memory demand increases
Solution Approach 1:
The patent divides the kernel processing into multiple different sized kernels (e.g., 3x3, 5x5, 7x7) that can be processed independently and then combined. This segmentation allows the system to handle complex motion deblurring by breaking it down into manageable kernel operations, improving deblurring performance while controlling memory usage through incremental processing.
Solution Approach 2:
The patent introduces a temporal dimension by processing kernels sequentially across multiple frames rather than loading all kernels simultaneously. By leveraging the temporal structure of video data and processing frames in sequence, the system achieves the benefits of multiple kernel sizes without requiring all kernels to be stored in memory at once.
2Measurement precision
If existing optical flow-based techniques are used, then deblurring can be performed, but the techniques are cumbersome and time consuming with limited improvement
Solution Approach 1:
The patent performs preliminary actions by pre-defining a set of standard kernel sizes and their corresponding operations. During video processing, the system can directly apply these pre-configured kernels without performing complex optical flow calculations, significantly reducing processing time while maintaining effective deblurring performance.
Solution Approach 2:
The patent replaces the mechanical optical flow-based approach with a direct kernel convolution method. Instead of calculating complex optical flow fields and using them to guide deblurring, the system directly applies multiple fixed-size kernels to the video frames, simplifying the process and reducing computational overhead.
3Measurement precision
If existing deep learning-based techniques are used, then deblurring can be performed, but results are subpar due to memory demand limitations
Solution Approach 1:
The patent segments the deep learning model into multiple processing stages, each handling a specific kernel size. This allows the model to process different spatial frequencies and motion scales separately, achieving better deblurring performance while keeping the memory footprint of each individual processing stage manageable.
Solution Approach 2:
The patent introduces dynamic kernel selection that adapts to the local motion characteristics in different regions of the video frame. By dynamically adjusting which kernel sizes are applied where, the system optimizes deblurring performance for varying motion patterns while efficiently utilizing available memory resources.
Data Source
AI summary
Embodiments are disclosed for deblurring motion in video. A method of deblurring motion in video can include receiving an input frame from a digital video, extracting a plurality of features of the input frame using an encoder network, determining, using a neural network, a plurality of spatial alignment kernels and a plurality of deblur kernels each corresponding to a feature of the input frame, wherein the plurality of spatial alignment kernels include different sizes of spatial alignment kernels and wherein the plurality of deblur kernels include different sizes of deblur kernels, generating, by the neural network, a plurality of output features for the input frame using the plurality of spatial alignment kernels and the plurality of deblur kernels, and generating a deblurred output frame from the plurality of output features using a decoder network.