Temporal Noise Reduction Architecture With Early Motion Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing temporal noise reduction methods struggle with accurately distinguishing between static and moving objects in video streams, leading to smearing and ghosting effects, especially when motion analysis is performed after image processing, which complicates noise modeling and increases resource usage.
Innovation Solution
A temporal noise reduction architecture that separates motion analysis from blending, performing motion analysis on a downscaled, unprocessed image early in the processing pipeline, using a deep-learning-based classifier to identify static and moving pixels, and blending the processed images at the end to ensure accurate noise reduction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If motion analysis is performed on full-resolution processed images, then motion detection accuracy is improved, but computational resource consumption increases and processing time is extended
Solution Approach 1:
The patent segments the motion analysis process by performing it on downsampled images rather than full-resolution images. This segmentation allows the system to process motion detection at a reduced resolution level, significantly decreasing computational resource consumption and processing time while still maintaining adequate motion detection accuracy for the temporal noise reduction task.
Solution Approach 2:
The patent changes the resolution dimension by performing motion analysis on downsampled images (lower resolution) rather than full-resolution images. This dimensional change reduces the computational complexity from O(N²) to O((N/k)²) where k is the downsampling factor, thereby improving processing throughput while preserving the essential motion information needed for noise reduction.
2Device complexity
If motion analysis is performed after image processing, then noise modeling is simplified, but processing time increases and resource usage increases
Solution Approach 1:
The patent performs motion analysis as a preliminary action before the main image processing and temporal blending operations. By downsampling the image first and performing motion analysis early in the processing pipeline, the system identifies motion regions in advance, allowing subsequent processing steps to focus computational resources only on static regions where noise reduction is applicable, thereby reducing overall processing time.
3Reliability
If temporal noise reduction is applied to moving objects, then noise reduction effectiveness is improved, but ghosting and smearing artifacts increase
Solution Approach 1:
The patent applies local quality by performing temporal blending with different weights for different regions of the image. Static regions are blended with higher weights to maximize noise reduction effectiveness, while moving regions are blended with lower weights or excluded from blending to prevent ghosting and smearing artifacts. This region-specific approach allows the system to optimize noise reduction for each local area based on its motion characteristics.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems and methods for improving a temporal noise reducer (TNR) architecture that improves TNR performance and IQ. Temporal noise reduction is a core feature of a video processing pipeline, where TNR can be used to decrease noise in video streams. TNRs generally includes two main steps: motion analysis and blending. Motion analysis includes identifying moving elements, and can include generating a motion map indicating regions of the input image that are static versus regions with movement. Blending includes blending the current input image frame with the previous temporally-denoised frame. An architecture is provided that separates the motion analysis from the blending step. In particular, the architecture includes a motion analysis block that operates on the raw image at the start of the pipeline, while the blending operation is completed on the processed image at the end of the image processing pipeline.