Video Processor Tone Mapping Frequency Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current tone mapping algorithms struggle to preserve highlight detail and color accuracy when converting high dynamic range (HDR) video frames to lower dynamic range for display, often resulting in loss of texture detail and hue shifts, especially on consumer displays with limited luminance capabilities.
Innovation Solution
The proposed solution involves separating each video frame into low and high frequency components, applying tone mapping only to the low frequency components, and then recombining them to preserve highlight detail, along with additional techniques like HDR Hue Tweak for fire and explosions, dynamic scene detection, HDR Luminance Channel Repair, Histogram Shaped Tone Mapping, and neural network-based decisions for optimal image quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If tone mapping is applied to convert HDR video frames to lower dynamic range, then the video can be displayed on consumer displays with limited luminance capabilities, but highlight detail and texture information are lost
Solution Approach 1:
The video frame is divided into multiple frequency bands (low, mid, and high frequencies). Tone mapping is selectively applied only to the low frequency components, while high frequency components containing highlight details are preserved. This segmentation allows the system to adapt to display limitations while maintaining critical visual information.
Solution Approach 2:
Different processing strategies are applied to different frequency components of the same image. Low frequency components undergo tone mapping to ensure display compatibility, while high frequency components are preserved to maintain highlight detail. This local quality approach ensures that each part of the signal is treated according to its specific role in the final image.
2Adaptability or versatility
If conventional tone mapping algorithms are used to convert HDR to lower dynamic range, then display compatibility is achieved, but color accuracy deteriorates with hue shifts
Solution Approach 1:
The color information is processed by separating it into frequency components. Tone mapping is applied selectively to low frequency components where hue shifts are less perceptible, while high frequency color information is preserved to maintain color accuracy. This segmentation resolves the contradiction between display compatibility and color precision.
3Adaptability or versatility
If full tone mapping is applied to all frequency components, then dynamic range compression is achieved for display compatibility, but texture detail and visual quality are degraded
Solution Approach 1:
The image is segmented into frequency bands, and tone mapping is applied only to the low frequency band. This selective application ensures that dynamic range compression is achieved where necessary for display compatibility, while texture details in the high frequency band remain intact and visually sharp.
Solution Approach 2:
Instead of applying tone mapping to the entire signal (excessive action), the method applies tone mapping only to the low frequency components (partial action). This partial application is sufficient to achieve display compatibility while avoiding the degradation of texture details that would result from full-signal tone mapping.
Data Source
AI summary
Various improvements to video processing are described, including highlight recovery by dividing an image into different frequency ranges and compressing the lower range before recombining, tone mapping to avoid hue shifts, dynamic tone mapping based on detected scene changes in the video, dynamic tone mapping using the shape of a tone mapping curve that is changed based on the histogram of each video frame, HDR Luminance Channel Repair of artifacts, changing the shape of the tone mapping curve based on a histogram of each video frame, upscaling Chroma by using Luma channel information motion interpolation using Neural Networks, motion compensated noise reduction using Neural Networks to filter random noise and film grain, grain/noise agnostic upscaling using Neural Networks, and hiding video frames by strategically dropping or repeating them in unnoticeable places to prevent a visible video stutter based on scene characteristics to accommodate clock variations.