CNN Video Denoising Without Motion Estimation Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video denoising methods based on deep learning require large computing times and memory, making them impractical for use in video denoising.
Innovation Solution
A convolutional neural network-based approach that utilizes multiple denoisers to process central and temporal frames, eliminating the need for explicit motion estimation and compensation, and employs a cascaded U-Net architecture for efficient video denoising.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional deep learning methods are used for video denoising, then denoising performance can be achieved, but computing time and memory requirements become excessively large
Solution Approach 1:
The video denoising task is segmented into multiple independent denoising operations applied to different frame groups. Instead of processing the entire video sequence as a single deep learning task, the method divides frames into groups and applies denoising operations to each group separately, significantly reducing computational time while maintaining denoising performance
Solution Approach 2:
The patent replaces complex mechanical deep learning processing with a simplified computational approach. By substituting the traditional end-to-end deep learning architecture with a multi-denoiser pipeline that processes frame groups independently, the system achieves the same denoising effect with fraction of the computing time
2Measurement precision
If traditional deep learning methods are used for video denoising, then denoising performance can be achieved, but memory footprint becomes excessively large
Solution Approach 1:
The video sequence is segmented into multiple frame groups that can be processed independently with smaller memory footprints. Each denoiser processes only its assigned frame group rather than loading the entire video into memory, reducing the memory requirement from O(N) to O(1) where N is the total number of frames
Solution Approach 2:
Multiple denoisers are designed with universal functionality to handle different frame groups using the same computational resources. This allows the system to reuse memory buffers and processing pipelines across different frame groups, maximizing memory utilization efficiency and reducing peak memory requirements
3Productivity
If multiple denoisers process different frame groups, then computing speed improves, but system complexity increases
Solution Approach 1:
Instead of designing a complex single denoiser that handles all frames, the system creates multiple simplified denoiser copies, each handling a specific frame group. This copying approach reduces the complexity of individual denoisers while achieving parallel processing speedups, as each copy uses identical or similar simplified architecture
Data Source
AI summary
An image capture device includes an image sensor and a processor. The image sensor is configured to capture a first plurality of frames, a second plurality of frames, and a third plurality of frames. The processor includes a first denoising layer and a second denoising layer. The first denoising layer includes a first denoiser, a second denoiser, and a third denoiser. The first denoiser is configured to denoise the first plurality of frames and output a first denoised frame. The second denoiser is configured to denoise the second plurality of frames and output a second denoised frame. The third denoiser is configured to denoise the third plurality of frames and output a third denoised frame. The second denoising layer includes a fourth denoiser. The fourth denoiser is configured to output a denoised frame based on the first denoised frame, the second denoised frame, and the third denoised frame.