Method and apparatus for inverse tone mapping

Abstract

Inverse tone mapping (ITM) aims at generating a single high dynamic range (HDR) image from a low dynamic range (LDR) image. While ITM was frequently used for graphics rendering in the HDR space, the advent of HDR consumer displays (e.g., HDR TV) and the consequent need for HDR multimedia contents open up new horizons for the consumption of ultra-high quality video contents. However, due to the lack of HDR-filmed contents, the legacy LDR videos must be up-converted for viewing on these HDR displays. Unfortunately, the previous ITM methods are not appropriate for HDR consumer displays, and their inverse-tone-mapped results are not visually pleasing with noise amplification or lack of details. In this paper, we propose a convolutional neural network (CNN) based architecture designed for the ITM to HDR consumer displays, called ITM-CNN, and its training strategy for enhancing the performance based on image decomposition using the guided filter. We demonstrate the benefits of decomposing the image by experimenting with various architectures and also compare the performance for different training strategies. To the best of our knowledge, this paper first presents the ITM problem using CNNs for HDR consumer displays, where the network is trained to restore lost details and local contrast. Our ITM-CNN can readily up-convert LDR images for direct viewing on an HDR consumer medium, and is a very powerful means to solve the lack of HDR video contents with legacy LDR videos.

Description

TECHNICAL FIELD[0001]At least one example embodiment relates to a method for inverse tone mapping and apparatuses performing the method.BACKGROUND ART[0002]The human visual system perceives the world as much brighter, with stronger contrasts and more details than is typically presented in standard dynamic range (SDR) displays. In comparison, recently available high dynamic range (HDR) consumer displays allow users to enjoy videos closer to reality as seen by the naked eye, with the brightness of at least 1,000 cd / m2 (as opposed to 100 cd / m2 for SDR displays), higher contrast ratio, increased bit depth of 10 bits or more, and wide color gamut (WCG). However, although HDR TVs are readily available in the market, there is a severe lack of HDR contents.[0003]Inverse tone mapping (ITM), also referred to as reverse tone mapping, is a popular area of research in computer graphics that aims to predict HDR images from low dynamic range (LDR) images for better graphics rendering. Another fiel...

AI Technical Summary

Benefits of technology

[0006]Therefore, we formulate a slightly different problem where we aim to generate HDR images that can be directly viewed on commercial HDR TVs. In this way, LDR legacy videos may be up-converted to be viewed on available HDR displays without additional information required. We propose an effective convolutional neural network (CNN) based structure and its learning strategy for up-converting a single LDR image of 8 bits / pixel, gamma-corrected [20], in the BT.709 color container [25], to an HDR image of 10 bits / pixel through the perceptual quantization (PQ) transfer function [27] in the BT.2020 color container [26], that may be directly viewed with commercial HDR TVs.

Problems solved by technology

In [23], Masia et al. evaluated a number of ITM algorithms and found that the performance of the algorithms decreased for overexposed input images.

Even when the transfer function and the color container are converted manually, HDR images converted by the previous methods are not suitable for viewing on consumer HDR displays.

three YUV channels is largely in part due to the color container and transfer function mismatch of LDR and HDR images if the U and V channels are not trained.

Since no previous method was explicitly trained for viewing with consumer HDR displays, fair comparison with our method is difficult.

Another complication is tone mapping for viewing on paper or SDR displays.

Although this is not the exact application of our problem, the result images still support our approach to be valid, and show that the existing methods are not directly applicable for viewing on HDR consumer displays.

Images