Spatiotemporal Tone Mapping for HDR Video
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current tone mapping techniques for high dynamic range (HDR) video fail to account for the temporal nature of video, resulting in undesirable temporal artifacts and 'halo' effects when trying to display HDR images on consumer-level displays with limited dynamic range.
Innovation Solution
A method involving spatiotemporal filtering based on forward and backward optical flow to separate HDR images into a base layer and a detail layer, followed by tone mapping to reduce dynamic range while preserving local details, using iterative filtering to minimize halo artifacts and enhance contrast preservation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Illumination intensity
If still image tone mapping techniques are applied to HDR video, then the dynamic range is reduced to accommodate consumer displays, but temporal artifacts are introduced due to ignoring the temporal nature of video
Solution Approach 1:
The patent applies dynamic temporal filtering that adapts to motion in the video sequence. The filter characteristics change over time based on detected motion, allowing the system to maintain temporal coherence during motion while still performing tone mapping. This resolves the contradiction by making the tone mapping process dynamic rather than static, ensuring temporal reliability is maintained throughout the video sequence.
Solution Approach 2:
The patent performs preliminary decomposition of the HDR video into base layer and detail layer before tone mapping. By separating the video content into these layers first, the system can apply different processing strategies to each layer, preserving temporal information in the base layer while enhancing local contrast in the detail layer, thus avoiding temporal artifacts.
2Illumination intensity
If HDR video is decomposed into base layer and detail layer for tone mapping, then local contrast is enhanced, but halo artifacts are introduced at edges
Solution Approach 1:
The patent applies different filtering strengths to different regions of the image based on local characteristics. In regions with strong edges, the filtering is reduced to prevent halo artifacts, while in uniform regions, stronger filtering is applied to enhance local contrast. This local adaptation resolves the contradiction by making the contrast enhancement selective rather than uniform.
Solution Approach 2:
The patent introduces an intermediate permeability map that controls the interaction between base layer and detail layer at edges. This permeability map acts as a mediator that prevents excessive contrast enhancement at edges by reducing the influence of the detail layer in regions where halos would occur, thus eliminating the harmful artifact while preserving local contrast elsewhere.
3Illumination intensity
If strong filtering is applied to enhance local details in HDR video, then local contrast is improved, but temporal artifacts increase due to motion sensitivity
Solution Approach 1:
The patent dynamically adjusts filtering strength based on detected motion. When motion is detected, the filtering strength is reduced to prevent temporal artifacts, while when the scene is static, stronger filtering is applied to enhance local contrast. This dynamic adaptation resolves the contradiction by making filtering intensity dependent on temporal conditions.
Solution Approach 2:
The patent applies spatially varying filtering strength that adapts to local motion characteristics. Regions with high motion have reduced filtering to maintain temporal coherence, while regions with little or no motion receive stronger filtering for enhanced local contrast. This local adaptation resolves the contradiction at the spatial level as well as the temporal level.
Data Source
AI summary
Approaches are described for filtering a first image frame in a sequence of image frames. A tone-mapping pipeline applies a spatiotemporal filter to each pixel of the first image frame, based on a forward optical flow and a backward optical flow, to produce a base layer image. The tone-mapping pipeline applies a temporal filter to each pixel of the first image frame, based on the forward optical flow and the backward optical flow, to produce a temporally filtered frame. The tone-mapping pipeline produces a detail layer image based on the base layer image and the temporally filtered frame.