Low Light Video Enhancement via Dynamic Neural Network Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for enhancing videos captured under low light conditions or using low-quality sensors face challenges such as high noise, low brightness, color artifacts, flickering, and computational intensity, which are not effectively addressed by spatial or temporal filters and deep Convolutional Neural Networks (CNNs).
Innovation Solution
The proposed solution involves switching between three neural networks - a high complexity neural network (HCN), a temporally guided lower complexity neural network (TG-LCN), and another neural network - based on shot boundary detection and artificial light flickering, using a multi-frame Siamese training method to enhance video quality by correcting brightness and reducing noise and flicker.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If spatial or temporal filters are used to denoise video, then noise reduction is achieved, but the filters fail to efficiently remove noise from low light videos
Solution Approach 1:
The patent transforms the input video frames by applying gamma correction and brightness adjustment to enhance the low light conditions. This parameter transformation allows subsequent neural network processing to be more effective at noise removal, as the enhanced brightness reveals more structural information that guides the denoising process while preserving detail in dark regions.
Solution Approach 2:
The patent introduces an intermediary brightness enhancement step between the raw low light input and the neural network denoising process. This intermediary transformation serves as a bridge that converts the problematic low light conditions into a form that is more amenable to effective noise removal, allowing the network to operate on enhanced data while the final output maintains natural appearance.
2Manufacturing precision
If deep Convolutional Neural Networks are used to enhance video, then video quality improvement is achieved, but the networks are too computationally heavy and memory intensive for real-time deployment
Solution Approach 1:
The patent segments the video processing task into distinct stages: brightness enhancement, neural network denoising, and temporal consistency refinement. By dividing the complex enhancement task into manageable segments, each processing stage can be optimized independently, reducing overall computational burden while maintaining high video quality through specialized processing at each stage.
Solution Approach 2:
The patent implements dynamic complexity adaptation by adjusting the neural network processing based on scene characteristics and motion detection. In low-motion or static scenes, more aggressive denoising with higher computational effort is applied, while in high-motion scenes, lighter processing is used to maintain real-time performance. This dynamic approach allows the system to adapt computational resources to actual needs.
3Object-affected harmful factors
If deep Convolutional Neural Networks are used for video enhancement, then denoising is improved, but the output video suffers from flickering due to inconsistent denoising of consecutive frames
Solution Approach 1:
The patent implements feedback mechanisms where previous frame outputs are used as references for current frame processing. The temporal consistency module compares consecutive frames and adjusts denoising parameters to maintain consistency, using feedback from motion detection and similarity metrics to ensure that noise removal patterns remain stable across frames, thereby eliminating flickering artifacts.
Solution Approach 2:
The patent performs preliminary brightness enhancement and alignment of consecutive frames before applying denoising operations. By pre-processing the video to correct brightness variations and align frames, the subsequent denoising process operates on consistent, pre-aligned data, which prevents the inconsistent denoising that causes flickering. This preliminary action ensures that the neural network receives uniform input across all frames.
4Illumination intensity
If maximum exposure time is increased to improve low light video quality, then brightness is improved, but frame rate decreases due to noise constraints
Solution Approach 1:
The patent applies continuous temporal smoothing and multi-frame processing to maintain brightness enhancement across consecutive frames. By processing multiple frames together and applying temporal consistency constraints, the system achieves effective brightness improvement without requiring individually long exposure times, thus maintaining high frame rates while still achieving low light enhancement through cumulative signal integration.
Data Source
AI summary
A method for enhancing media includes: receiving, by an electronic device, a media stream; performing, by the electronic device, an alignment of a plurality of frames of the media stream; correcting, by the electronic device, a brightness of the plurality of frames; selecting, by the electronic device, one of a first neural network, a second neural network, or a third neural network, by analyzing parameters of the plurality of frames having the corrected brightness, wherein the parameters include at least one of shot boundary detection and artificial light flickering; and generating, by the electronic device, an output media stream by processing the plurality of frames of the media stream using the selected one of the first neural network, the second neural network, or the third neural network.