Video Stabilization Neural Network for Flicker-Free HDR Output
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image enhancement algorithms for video stabilization fail to address continuity between consecutive images, resulting in visible flickering when adjusted images are played back as video, leading to unstable video playback.
Innovation Solution
A method involving the conversion of low dynamic range (LDR) images to high dynamic range (HDR) images using a first neural network, followed by training a second neural network for video stabilization based on features extracted from both LDR and HDR images, with a loss function that minimizes flickering artifacts and incorporates a reward value from an HDR classifier to generate stabilized HDR images in a time-dependent manner.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If single-image enhancement algorithms are used to adjust each image individually, then image enhancement quality is improved, but video stability deteriorates due to visible flickering between consecutive images
Solution Approach 1:
The patent applies continuity of useful action by ensuring that enhancement adjustments maintain temporal consistency across consecutive video frames. The system processes images in sequence while preserving continuity information, so that enhancement parameters evolve smoothly over time rather than changing abruptly between frames, thereby eliminating flickering while maintaining enhancement quality.
Solution Approach 2:
The patent implements feedback mechanisms where the system monitors enhancement adjustments across consecutive frames and uses this information to regulate future adjustments. By analyzing the temporal patterns of enhancement changes and providing feedback to the enhancement algorithm, the system prevents excessive or abrupt adjustments that would cause flickering, thus maintaining both enhancement quality and video stability.
2Adaptability or versatility
If different adjustments are applied to different images based on individual color and light intensity, then image-specific optimization is improved, but temporal consistency deteriorates causing visible flickering
Solution Approach 1:
The patent applies dynamics by making the enhancement adjustments adaptive and time-varying rather than static. The system dynamically adjusts enhancement parameters based on both the current image characteristics and the temporal context from previous frames. This dynamic approach allows each image to be optimized for its specific content while ensuring that adjustments evolve smoothly over time, preventing flickering caused by abrupt changes.
Data Source
AI summary
A training method for video stabilization and an image processing device using the same are proposed. The method includes the following steps. An input video including low dynamic range (LDR) images is received. The LDR images are converted to high dynamic range (HDR) images by using a first neural network. A feature extraction process is performed to obtain features based on the LDR images and the HDR images. A second neural network for video stabilization is trained according to the LDR images and the HDR images based on a loss function by minimizing a loss value of the loss function to generate stabilized HDR images in a time-dependent manner, where the loss value of the loss function depends upon the features. An HDR classifier is constructed according to the LDR images and the HDR images. The stabilized HDR images are classified by using the HDR classifier to generate a reward value, where the loss value of the loss function further depends upon the reward value.