Semantic Segmentation for HDR Tone Mapping Detail Preservation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing HDR tone mapping algorithms lack semantic awareness, leading to reduced contrast and detail loss in images, particularly in areas like sharp edges and semantic objects, due to their inability to understand the meaning of the data they process.
Innovation Solution
A HDR tone mapping system that incorporates a semantic segmentation module to extract semantic information from the input image, followed by image decomposition, statistics generation, curve computation, compression, detail adjustment, and image reconstruction, all guided by semantic information to preserve contrast and details.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If typical tone mapping algorithms are used, then the HDR image can be displayed on SDR medium, but the contrast and details are lost due to lack of semantic awareness
Solution Approach 1:
The patent segments the image processing into multiple stages: semantic segmentation to identify meaningful regions, image decomposition to separate base and detail layers, and selective tone mapping applied differently to each segment. This allows preservation of important semantic boundaries while adapting to SDR display constraints.
Solution Approach 2:
The patent applies different processing strategies to different regions of the image based on their semantic content. Semantic-aware regions receive specialized handling to preserve their characteristics, while non-semantic regions undergo standard tone mapping. This local differentiation maintains contrast and details in important areas.
2Shape
If edge-preserving filters are applied, then sharp boundaries are retained, but semantic boundaries are not properly preserved leading to contrast reduction
Solution Approach 1:
The patent performs semantic segmentation before the actual tone mapping process. By identifying semantic boundaries in advance, the system can prepare appropriate processing strategies for each region, ensuring that important semantic boundaries are preserved during subsequent compression and tone mapping operations.
Solution Approach 2:
The patent uses the extracted semantic information as feedback to guide the tone mapping process. The semantic segmentation results inform the decomposition and compression stages, creating a closed-loop system where semantic understanding continuously influences processing decisions to preserve meaningful boundaries.
3Adaptability or versatility
If global curve compression is applied, then the overall image can be adapted to SDR, but there is a tradeoff between bright objects and dark objects appearance
Solution Approach 1:
The patent replaces global curve compression with local, region-specific tone mapping curves generated based on semantic information. Each semantic region receives a customized compression curve optimized for its specific characteristics, eliminating the need to compromise between bright and dark objects in a global sense.
Solution Approach 2:
The patent dynamically adjusts compression parameters and tone mapping curves based on the semantic content and statistics of different regions. By changing parameters locally rather than applying a fixed global curve, the system can optimize appearance quality for each region independently, resolving the tradeoff between bright and dark objects.
Data Source
AI summary
A HDR tone mapping system includes several modules. A semantic segmentation module is used to extract semantic information from the input image. An image decomposition module is used to decompose the input image to a high-bit base layer and a detail layer. A statistics module is used to generate statistics of pixels of the input image according to the semantic information. A curve computation module is used to generate a tone curve from the statistics. A compression module is used to compress the high-bit base layer to a low-bit base layer according to the tone curve, the statistics and the semantic information. A detail adjustment module is used to tune the detail layer according to the semantic information and the statistics to generate an adjusted detail layer. An image reconstruction module is used to combine the adjusted detail layer and the low-bit base layer to generate an output image.