Video Denoising via Optical Flow Patch Averaging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Video denoising is a challenging problem in digital videos captured with low-end devices, where noise degrades image quality and clarity, especially in low light conditions or during compression, making it difficult to watch on larger monitors.
Innovation Solution
The method employs optical flow to estimate noise in video frames by selecting a reference frame, determining matching patches, and using a k-nearest neighbor search and integral image algorithm to compute noise estimates, which are then used to denoise the frames by averaging similar motion patches from previous and subsequent frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If video denoising is performed using traditional methods, then noise reduction is achieved, but image detail and clarity are lost
Solution Approach 1:
The image is divided into multiple patches, and denoising is performed on each patch individually using k-nearest neighbor search. This allows noise reduction while preserving local image details and structures, as each patch can be processed independently with its own characteristics maintained.
Solution Approach 2:
The patent introduces temporal dimension by utilizing multiple video frames (previous and subsequent frames) in addition to the current frame. By computing noise estimates across temporal dimensions and using optical flow to track patches across frames, the method gathers more statistical information for denoising while preserving spatial details.
2Object-affected harmful factors
If more video frames are used for denoising, then noise reduction quality improves, but processing complexity increases
Solution Approach 1:
The patent dynamically adjusts the temporal window size based on the computed noise estimates. When noise levels are high, a larger temporal window (more frames) is used to gather more data for denoising. When noise levels are low, a smaller window is used to reduce processing complexity. This adaptive approach optimizes the trade-off between denoising quality and processing cost.
Solution Approach 2:
The patent performs preliminary noise estimation using optical flow and patch matching before committing to full denoising processing. This preliminary action allows the system to determine how much processing is actually needed, avoiding unnecessary computation when noise levels are already low.
3Manufacturing precision
If adaptive temporal window selection is used, then denoising quality is optimized, but computational overhead increases
Solution Approach 1:
The patent computes noise estimates for all frames in the temporal window to determine the optimal window size, which may seem like excessive computation. However, these noise estimate computations are relatively lightweight compared to full denoising operations, and they enable the system to avoid performing full denoising when not needed, resulting in net computational savings.
Data Source
AI summary
In techniques for video denoising using optical flow, image frames of video content include noise that corrupts the video content. A reference frame is selected, and matching patches to an image patch in the reference frame are determined from within the reference frame. A noise estimate is computed for previous and subsequent image frames relative to the reference frame. The noise estimate for an image frame is computed based on optical flow, and is usable to determine a contribution of similar motion patches to denoise the image patch in the reference frame. The similar motion patches from the previous and subsequent image frames that correspond to the image patch in the reference frame are determined based on the optical flow computations. The image patch is denoised based on an average of the matching patches from reference frame and the similar motion patches determined from the previous and subsequent image frames.