Adaptive Video Denoising for Mobile Conferencing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Video conferencing on mobile devices faces challenges with noisy video capture due to small image sensors, especially in low-light conditions, and existing real-time denoising algorithms are processor-intensive and codec-specific, failing to meet the processing requirements effectively.
Innovation Solution
An adaptive denoising method that buffers and filters video frames into blocks, performs temporal denoising, and determines whether to replace or keep denoised blocks based on quantization parameters and motion vectors, independent of the codec used, allowing for efficient noise reduction in real-time video conferencing applications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If conventional denoising algorithms are used to reduce noise in video captured by mobile devices, then video quality is improved, but processor intensity increases and real-time processing capability is lost
Solution Approach 1:
The video frame is divided into multiple blocks, and each block is processed independently through temporal denoising operations. This segmentation allows parallel processing of blocks, reducing overall processing time while maintaining denoising quality, thus enabling real-time performance on mobile devices
Solution Approach 2:
The patent applies selective denoising based on motion detection and block classification. Not all blocks are denoised equally - some blocks are processed more aggressively while others are skipped or lightly processed, reducing total computational load while maintaining acceptable video quality
2Productivity
If codec-specific real-time denoising algorithms are used, then processing speed is improved for real-time applications, but adaptability to different codecs is lost
Solution Approach 1:
The denoising algorithm operates on raw video frames before encoding, using universal image processing operations (temporal filtering, motion compensation) that are independent of any specific video codec. This allows the same denoising process to be applied regardless of which codec (H.264, H.265, VP9, etc.) is used for encoding, providing broad codec compatibility while maintaining real-time performance
3Area of stationary object
If small image sensors are used in mobile devices, then device size is reduced, but noise in captured video increases
Solution Approach 1:
The patent introduces temporal information from previous and future frames as an intermediary to help reconstruct the current frame. By using motion-compensated temporal filtering, noise in the current frame is reduced by averaging with corresponding blocks from adjacent frames, effectively compensating for the limited light-gathering capability of small sensors
Solution Approach 2:
Motion compensation and block matching are performed in advance to identify corresponding blocks across frames before applying temporal denoising. This preliminary action ensures that the temporal averaging process aligns correctly with moving objects, preventing artifacts while effectively reducing noise
Data Source
AI summary
A method and apparatus for adaptive denoising of source video in a video conference application is provided. Video captured is analyzed on a frame by frame basis to determine whether denoising of the frame should be performed prior to providing the source frame to an encoder. If the frame is to be denoised, the frame is divided into a plurality of blocks and a local denoising process is performed on a block per block basis.