Motion-Compensated Frame Denoising for Low-Light Video
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing digital image capture technologies struggle with noise, particularly color noise in low-light conditions, which can result in distracting visual artifacts and the blurring of fine details, and current denoising solutions often introduce additional issues like smudging or ghosting.
Innovation Solution
A method involving frame denoising techniques that create synthetic frames using motion data from neighboring frames, combine these with captured frames to form composite frames, and apply masking to reduce noise while preserving sharp edges and details, using optical flow analysis and depth maps to handle occlusions and anomalies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If denoising solutions are applied to reduce noise in video footage, then noise is reduced, but fine details and textures are blurred
Solution Approach 1:
The patent applies different processing treatments to different regions of the video frame based on local characteristics. Motion-compensated filtering is applied selectively to regions with low motion activity, while regions with high motion or complex textures preserve more detail. This local adaptation allows noise reduction in uniform areas without blurring important edges and textures.
Solution Approach 2:
The filtering strength and parameters are dynamically adjusted based on local motion estimates, texture complexity, and noise characteristics. The system adapts the denoising intensity in real-time for different regions and frames, rather than applying a static filter, thereby preserving temporal and spatial details while reducing noise.
2Object-affected harmful factors
If traditional denoising filters are applied, then noise is reduced, but artifacts like banding and smudging are introduced
Solution Approach 1:
The patent replaces traditional spatial-domain filtering mechanisms with a motion-compensated temporal filtering approach. By aligning pixels across frames using motion estimation and compensation, the system filters noise in the temporal domain after motion alignment, avoiding the spatial smoothing that causes banding and smudging artifacts.
Solution Approach 2:
The patent introduces motion compensation as an intermediary step between frame capture and denoising. Motion vectors serve as a mediator to align corresponding pixels across frames before filtering, ensuring that temporal noise reduction does not create spatial artifacts. This intermediary alignment process prevents the direct mixing of unrelated pixel values that causes banding.
3Illumination intensity
If ISO setting is increased to improve low-light performance, then light sensitivity is improved, but sensor noise is amplified
Solution Approach 1:
The patent performs motion compensation and frame alignment as preliminary actions before denoising. By pre-aligning the video frames using motion estimation, the system prepares the data structure for effective temporal filtering, ensuring that noise reduction operates on properly registered pixel correspondences across frames.
Solution Approach 2:
The patent merges multiple frames together through motion-compensated averaging or stacking. By combining information from multiple frames after motion alignment, the signal-to-noise ratio is improved, allowing effective noise reduction while preserving the benefits of high ISO settings for low-light capture.
4Object-affected harmful factors
If frame stacking is performed to reduce noise, then noise is reduced, but ghosting effects occur due to motion
Solution Approach 1:
The patent dynamically adjusts the stacking process by incorporating motion compensation. Frames are warped to a common reference frame using motion vectors before stacking, and the contribution of each frame is weighted based on motion magnitude and occlusion detection. This dynamic approach prevents static ghosting artifacts while maintaining noise reduction benefits.
Solution Approach 2:
The patent segments the video frame into regions based on motion activity and occlusion status. Different stacking strategies are applied to different segments: highly dynamic regions use motion-compensated alignment with selective frame inclusion, while static regions can use simpler averaging. This segmentation prevents ghosting in motion-prone areas while maintaining noise reduction elsewhere.
Data Source
AI summary
Systems, apparatus, and methods for post-processing video e.g. frame denoising. Noise reduction techniques may be employed to improve the quality of digital video. Frames may be extracted from a video. Synthetic frames may be created using motion data between the extracted frames. Synthetic frames may be masked to exclude pixels from the composite frame. Thresholds used in masking may vary based on the temporal distance of the extracted frame used to create the synthetic frame and the extracted frame. Masking may be based on frame differences between extracted and synthetic frames (e.g., sub-pixel/luminance differences), areas of lower quality motion data (e.g., occlusions), or edge detection in the extracted frames. Synthetic and extracted frames may be composited generating frames having less noise. The composited frame may be based on averaging pixel values across the synthetic and extracted frames. Composited frames may be compiled and encoded into denoised video.