Video Denoising via Motion-Compensated Patch Averaging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video denoising techniques fail to effectively remove structured noise introduced by low-end camcorders and digital cameras, which has strong spatial correlations and is not addressed by existing methods that focus on additive white Gaussian noise.
Innovation Solution
A method that uses motion estimation to project pixel locations into adjacent frames, selecting patches based on content similarity, and replacing pixel values with an average value from combined sets of patches in adjacent frames to reduce structured noise.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If conventional local neighborhood filters are used to remove additive white Gaussian noise, then noise is reduced, but edges in the image are blurred
Solution Approach 1:
The image is divided into multiple local patches around each pixel. Instead of filtering the entire image uniformly, the method segments the image into smaller regions and processes each patch independently, allowing for more precise noise removal while preserving edge information through local analysis.
Solution Approach 2:
The filtering operation adapts to local image characteristics by computing weights based on the similarity between patches. Regions with edges or high-frequency content receive different treatment compared to smooth regions, maintaining local quality and preventing edge blurring while still removing noise effectively.
2Shape
If non-linear filters such as median filters are used to reduce blurring, then edge sharpness is improved, but some a priori knowledge about noise spectra and original signal is required
Solution Approach 1:
The algorithm automatically adapts to the local image structure and noise characteristics without requiring manual specification of noise spectra or signal models. The patch-based approach with similarity weighting enables the filter to self-adjust its behavior based on the content being processed, eliminating the need for a priori knowledge while maintaining edge sharpness.
3Device complexity
If conventional NLM approaches are used for video denoising without motion estimation, then processing is simpler, but visual consistency between frames deteriorates
Solution Approach 1:
Motion estimation is performed beforehand to determine the displacement of pixels between frames. This preliminary step allows the algorithm to correctly identify corresponding patches across frames, ensuring that noise reduction operations maintain temporal coherence and visual consistency without requiring complex real-time adjustments during the filtering process.
4Device complexity
If patches are selected without considering motion estimation, then the method is simpler to implement, but the denoising quality decreases
Solution Approach 1:
The patch selection process becomes dynamic by incorporating motion compensation. Patches are selected based on their temporal coherence after accounting for motion, allowing the algorithm to adapt to moving objects and maintain high denoising quality. The similarity metric is computed in the motion-compensated domain, ensuring accurate matching despite scene motion.
Data Source
AI summary
A method of operating a computer system to reduce structured noise in a video is described. Patches in a frame of a sequence of video frames are selected using a modified non-local means (NLM) search algorithm with a reduced search complexity compared to conventional NLM search algorithms. The set of spatial neighboring patches in a frame is extended to include patches in adjacent frames of the video sequence. Patches in the adjacent frames are selected based, at least in part, on a projection of a location of a pixel in the frame into the adjacent frames using a motion estimation algorithm. An average value determined from the combined set of patches is used to replace the pixel currently being processed.