Hierarchical Image Tone Compensation for Backlit Sky Preservation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional dodging processing techniques for image tone compensation often result in loss of contrast and unnatural images due to compression of tone characteristics in luminance ranges other than dark parts, especially affecting high luminance areas like skies.

Innovation Solution

An image processing apparatus and method that generates hierarchical images with different frequency bands, calculates gains using tone conversion curves prioritizing specific luminance ranges, and combines these gains for tone conversion, prioritizing the high luminance range in lower frequency bands to enhance tone characteristics.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Illumination intensity

If gain characteristics are directly used for local luminance compensation, then dark parts are brightened effectively, but tone characteristics in other luminance ranges are compressed and contrast is lost

Engineering Contradiction:

Improvebrightness of dark partsVSAvoidtone characteristics in high luminance ranges

Core Design Contradiction:

Illumination intensityVSLoss of information

Solution Approach 1:

The image is divided into multiple hierarchical images with different frequency bands. Each hierarchical image processes specific frequency components separately, allowing selective tone compensation without affecting other luminance ranges. The low-frequency hierarchical image handles overall brightness while high-frequency hierarchical images preserve local contrast and tone characteristics.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Different tone conversion curves are applied to different hierarchical images based on their frequency characteristics. Low-frequency hierarchical images use curves optimized for overall brightness compensation, while high-frequency hierarchical images use curves that preserve local contrast. This localized approach to tone processing prevents compression of tone characteristics in specific luminance ranges.

Inventive Principle:

Principle #3Local quality

2Ease of manufacture

If a single tone conversion curve is used for the entire image, then processing is simple, but it cannot simultaneously optimize dark parts and preserve high luminance tone characteristics

Engineering Contradiction:

Improveprocessing simplicityVSAvoidtone quality across different luminance ranges

Core Design Contradiction:

Ease of manufactureVSReliability

Solution Approach 1:

The image processing is segmented into multiple hierarchical levels, each with its own tone conversion curve. This segmentation allows complex multi-curve processing to be organized systematically, maintaining manageability while achieving superior tone quality across all luminance ranges compared to a single curve approach.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The final output image is a composite of multiple hierarchical images that have been processed with different tone conversion curves. By combining the processed hierarchical images, the system achieves comprehensive tone optimization across all luminance ranges while maintaining processing organization through the hierarchical structure.

Inventive Principle:

Principle #40Composite materials

3Illumination intensity

If gain is increased for dark parts, then backlit objects become brighter, but contrast at boundaries becomes unnatural and artifacts appear

Engineering Contradiction:

Improvebrightness of backlit objectsVSAvoidboundary artifacts and unnatural contrast

Core Design Contradiction:

Illumination intensityVSObject-affected harmful factors

Solution Approach 1:

The image is segmented into hierarchical frequency bands, allowing boundary regions to be processed differently from uniform regions. High-frequency hierarchical images preserve sharp edges and boundaries, while low-frequency hierarchical images provide smooth overall brightness adjustment. This prevents the mixing of gain adjustments that causes boundary artifacts.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system dynamically adjusts the weight of low-frequency and high-frequency hierarchical images based on local image characteristics. In boundary regions, the weight of high-frequency images is increased to preserve edge sharpness and avoid unnatural contrast. In uniform dark regions, the weight of low-frequency images is increased for effective brightness compensation. This dynamic adaptation prevents boundary artifacts while maintaining brightness improvement.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS10218953B2Image processing apparatus, image processing method, and storage medium

Publication Date: 2019.02.26 CANON KK

AI summary

An image processing apparatus includes a generation unit configured to generate, from an input image, a plurality of hierarchical images having different frequency bands; a gain calculation unit configured to calculate, for each of the hierarchical images, a gain based on a luminance value for each image area by using a tone conversion curve that is set according to the corresponding frequency band, wherein the tone conversion curve assigns tone with priority to different luminance ranges according to the frequency bands of the hierarchical images; a determination unit configured to determine a combined gain by combining gains that are set for the plurality of hierarchical images; and a conversion unit configured to perform tone conversion on the input image by using the combined gain determined by the determination unit.