60 results about "Inverse tone mapping" patented technology

The present invention provides a generative adversarial network-based high dynamic range inverse tone mapping method and system. The method includes the following steps that: an original high dynamicrange video is read, and the original high dynamic range video is cut and converted into data sets which can be used for training and are corresponding to a standard dynamic range and a high dynamic range respectively; a generative adversarial network is established based on a convolutional neural network and a hopping type connection is established, and standard dynamic range images are convertedinto high dynamic range images, namely inverse tone mapping is performed; and the entire generative adversarial network is continuously optimized according to a set comprehensive objective function,and a finally obtained network can complete mapping from the standard dynamic range to the high dynamic range. With the method of the invention adopted, the problems such as non-linearity insufficiency and complicated parameter adjustment of an existing non-learning method can be solved; the one-dimensional characteristic and gradient characteristic of the high dynamic range images are considered,so that inverse dynamic mapping of the high dynamic range can be better realized.

Method comprising, for each pixel, obtaining a pixel expansion exponent value (E′(p)) by low pass filtering luminance values of colors of pixels in its spatial neighborhood, obtaining a pixel luminance-enhancement value (Yenhance(p)) by extraction of high frequencies of luminance values of colors of pixels in its spatial neighborhood, and then inverse tone mapping the luminance (Y(p)) according to the equation Yexp(p)=Y(p)E′(p)×[Yenhance(p)]c.

A method and system for inverse tone mapping and decoder. The method performs inverse tone mapping of an image in a decoder. For each block of each color channel of the image the following steps are performed. A scaling factor is determined for a current block of the image by adding a predicted scaling factor for the current block to a difference between the predicted scaling factor and the scaling factor of an adjacent block. An offset value for the current block is determined by adding a predicted offset for the current block to a difference between the predicted offset value and the offsetvalue of the adjacent block. The scaling factor and the offset value are applied to pixel intensity values of the current block to produce a mapped block in which a bit-depth of the mapped block is greater than the bit-depth of the current block.

The present invention proposes to provide methods, apparatus and systems for inverse tone mapping of a high dynamic range image. The method may include the operations of obtaining a luminance component of the high dynamic range image. The method may further include determining an HDR to HDR inverse tone mapper operator. The method may further include determining a tone expanded image by applying the HDR to HDR inverse tone mapper to the luminance component of the high dynamic range image. The method may further include wherein the HDR to HDR inverse tone mapper curve comprises a linear part for dark and mid-tone levels, and an expanding non-linear part for highlights. The system or apparatus may be configured to perform the operations of the method.

The invention provides a multi-exposure high-dynamic range inverse tone mapping model construction method and device. The multi-exposure high-dynamic range inverse tone mapping model construction method comprises the following steps: intercepting a high dynamic range image from an original high dynamic range video, converting the standard dynamic range images into standard dynamic range images, adjusting exposure time to generate multi-exposure standard dynamic range images, and forming a supervised data set by the standard dynamic range images with different exposures and a high dynamic rangeimage with the same normal exposure to serve as a training data set; establishing a generative adversarial network based on a convolutional neural network and skip connection; and establishing a target loss function integrated by the image content features, the intrinsic features and the perception features for the generative adversarial network, and continuously training and optimizing by adopting the training data set to obtain a final model. According to the multi-exposure high-dynamic range inverse tone mapping model construction method, the brightness of the overexposure or underexposureimage can be adjusted, and the effect of the generated high-dynamic-range image is improved, and the brightness characteristic and the chrominance characteristic of the high-dynamic-range image are considered, and inverse tone mapping of the high dynamic range is better achieved.

Methods and apparatus are provided for bit depth scalable video encoding and decoding utilizing tone mapping and inverse tone mapping. An apparatus includes an encoder (100) for encoding picture data for at least a portion of a picture by generating an inter-layer residue prediction for the portion using a tone mapping function and an inverse tone mapping function. The tone mapping function and the inverse tone mapping function perform a conversion of the inter-layer residue prediction from a residue domain to a pixel domain for bit depth scalability while maintaining a same bit depth representation.

A scalable video bitstream may have an H.264 / AVC compatible base layer and a scalable enhancement layer, where scalability refers to color bit-depth. According to the invention, BL information is bit-depth upsampled using separate look-up tables for inverse tone mapping on two or more hierarchy levels, such as picture level, slice level or MB level. The look-up tables are differentially encoded and included in header information. Bit-depth upsampling is a process that increases the number of values that each pixel can have, corresponding to the pixels color intensity. The upsampled base layer data are used to predict the collocated enhancement layer, based on said look-up tables. The upsampling is done at the encoder side and in the same manner at the decoder side, wherein the upsampling may refer to temporal, spatial and bit depth characteristics. Thus, the bit-depth upsampling is compatible with texture upsampling.

A method is described for inverse tone mapping the luminance (Yt(p)) of each pixel (p) of at least one image (It) of this sequence into an expanded luminance Ytexp(p)=Yt(p)Et(p). Ytenh(p), wherein Et(p) is an expansion exponent value extracted from an expansion exponent map built from low pass motion-compensated temporal filtering of said image (It) and wherein Ytenh(p) is a luminance-enhancement value extracted from a luminance-enhancement map built from high pass motion-compensated temporal filtering of said image (It).

The invention provides a high dynamic range image generation method based on inverse tone mapping and saturation adjustment. The method is based on visual characteristics of human eyes, and comprisesthe following steps: firstly, converting an original low-dynamic-range image from an RGB color space to an HSV color space, and separating out a brightness component and a saturation component of thelow-dynamic-range image; secondly, performing inverse tone mapping expansion on the brightness component, separating out a highlight area, performing gamma correction on the highlight area, and fusinga low light area to obtain a new brightness component; then, carrying out linear pull-up on the saturation component, calculating correlation coefficients of the brightness component and the saturation pair component of the original image at the same time, and adjusting the pulled-up saturation component according to the new brightness component and the correlation coefficients to obtain a new saturation component; and finally, fusing the new brightness component, the new saturation component and the tone component to obtain a high-dynamic-range image of the HSV space, and converting the image into an RGB color space to obtain a final high-dynamic-range image.

A video coding system may perform inter-layer processing by simultaneously performing inverse tone mapping and color gamut conversion scalability processes on a base layer of a video signal. The video coding system may then perform upsampling on the processed base layer. The processed base layer may be used to code an enhancement layer. Bit depth may be considered for color gamut conversion modules. Luma and / or chroma bit depths may be aligned with respective larger or smaller bit depth values of luma and / or chroma.

Inverse tone mapping (ITM) aims at generating a single high dynamic range (HDR) image from a low dynamic range (LDR) image. While ITM was frequently used for graphics rendering in the HDR space, the advent of HDR consumer displays (e.g., HDR TV) and the consequent need for HDR multimedia contents open up new horizons for the consumption of ultra-high quality video contents. However, due to the lack of HDR-filmed contents, the legacy LDR videos must be up-converted for viewing on these HDR displays. Unfortunately, the previous ITM methods are not appropriate for HDR consumer displays, and their inverse-tone-mapped results are not visually pleasing with noise amplification or lack of details. In this paper, we propose a convolutional neural network (CNN) based architecture designed for the ITM to HDR consumer displays, called ITM-CNN, and its training strategy for enhancing the performance based on image decomposition using the guided filter. We demonstrate the benefits of decomposing the image by experimenting with various architectures and also compare the performance for different training strategies. To the best of our knowledge, this paper first presents the ITM problem using CNNs for HDR consumer displays, where the network is trained to restore lost details and local contrast. Our ITM-CNN can readily up-convert LDR images for direct viewing on an HDR consumer medium, and is a very powerful means to solve the lack of HDR video contents with legacy LDR videos.

A method for inverse tone mapping is provided. The method comprising obtaining a digital image in a color space wherein the luminance is separate from the chrominance, determining a base luminance of pixels in the digital image, determining a detail enhancement map, determining a pixel expansion exponent map, determining an edge map of the image, inverse tone mapping luminance of image based on edge map, pixel expansion map and base luminance, and providing an expanded dynamic range image based on the inverse tone mapped luminance.

There are provided methods and apparatus for inter-layer residue prediction for scalable video. An apparatus is described for an encoder (200) for encoding a block of a picture, or a decoder (300) for decoding a block of a picture, by applying inverse tone mapping to an inter-layer residue prediction process for the block, wherein the inverse tone mapping is performed in the pixel domain. Methods for encoding (440, 460) or decoding (540, 560) a block of a picture are also described; and performed by applying inverse tone mapping to an inter-layer residue prediction process for the block, wherein the inverse tone mapping is performed in the pixel domain.

Inverse tone mapping based on luminance zones is provided. A digital image can be obtained and luminance values of pixels of the digital image can be determined. One or more values of luminance zone boundaries can be determined. A zone expansion exponent map based on the luminance values and the one or more luminance zone boundaries can be determined. The one or more luminance zone boundaries divide the zone expansion exponent map into a plurality of luminance zones. The luminance values of the image can be inverse tone mapped based on the zone expansion exponent map. An expanded dynamic rangeimage can be provided based on the inverse tone mapped luminance values.

There are provided methods and apparatus for artifact removal for bit depth scalability. The method and apparatus utilize an encoder for encoding an enhancement layer for at least a portion of a picture. A deblocking filter is applied at the enhancement layer for bit depth scalability. A decoding method and apparatus are described for decoding an enhancement layer for at least a portion of a picture, wherein the during the decoding process, a deblocking filter is applied at the enhancement layer for bit depth scalability. Furthermore, an encoder and method are described for encoding image data for at least one block of a picture, wherein a deblocking filter (117) removes coding artifacts caused by local inverse tone mapping for intra-layer texture prediction for bit depth scalability. A decoding method and apparatus are described for decoding image data for at least one block of a picture, wherein a deblocking filter (230, 237, 435) removes coding artifacts caused by local inverse tone mapping for intra-layer texture prediction for bit depth scalability.

The invention provides a video high dynamic range inverse tone mapping model construction method, which comprises the following steps of: cutting an original high dynamic range video into a pluralityof high dynamic range videos, converting the high dynamic range videos into standard dynamic range videos, and forming a supervised data set by the standard dynamic range videos and the high dynamic range videos to serve as a subsequent training data set; establishing a video generation network based on a three-dimensional convolutional neural network and skip connection; and establishing a targetloss function integrated by spatial features, time domain features, intrinsic features and perception features for the video generation network, and continuously training and optimizing by adopting the training data set to obtain a final network model. The invention also provides a corresponding construction device and a video high dynamic range inverse tone mapping method. According to the method, the problem of video flicker is solved, and the spatial characteristics, intrinsic characteristics and time domain characteristics of the high-dynamic-range video are considered, so that the inverse tone mapping of the high-dynamic-range video is better realized.

To encode High Dynamic Range (HDR) images, the HDR images can be converted to Low Dynamic Range (LDR) images through tone mapping operation, and the LDR images can be encoded with an LDR encoder. The present principles formulates a rate distortion minimization problem when designing the tone mapping curve. In particular, the tone mapping curve is formulated as a function of the probability distribution function of the HDR images to be encoded and a Lagrangian multiplier that depends on encoding parameters. At the decoder, based on the parameters indicative of the tone mapping function, an inverse tone mapping function can be derived to reconstruct HDR images from decoded LDR images.

Methods and apparatus are provided for motion compensation with a smooth reference frame in bit depth scalability. An apparatus includes an encoder (100) for encoding picture data for at least a portion of a picture by generating an inter-layer residue prediction for the portion using an inverse tone mapping operation performed in the pixel domain for bit depth scalability. The inverse tone mapping operation is shifted from a residue domain to the pixel domain.

A method for converting a standard dynamic range (SDR) sequence into a high dynamic range (HDR) sequence is described. The method includes determining (S1) successive shots in the SDR sequence by cut detection, detecting (S2) if the SDR sequence comprises a sub-sequence of at least two successive short shots having a duration lower than D1, said subsequence comprising at least one bright shot having a luminance and a size greater than L1 and SZ1 respectively, and mapping (S3) the luminance range of bright shots of the detected subsequence from the low dynamic range to a first high dynamic range and mapping the other shots of the SDR sequence from the standard dynamic range to a second high dynamic range in order to generate the HDR sequence, the first high dynamic range having a maximal value Lmax1 lower than that of the second high dynamic range Lmax2.

Methods and apparatus are provided for motion compensation with a smooth reference frame in bit depth scalability. An apparatus includes an encoder (100) for encoding picture data for at least a portion of a picture by generating an inter-layer residue prediction for the portion using an inverse tone mapping operation performed in the pixel domain for bit depth scalability. The inverse tone mapping operation is shifted from a residue domain to the pixel domain.

Method comprising, for each pixel, obtaining a pixel expansion exponent value (E′(p)) by low pass filtering luminance values of colors of pixels in its spatial neighborhood, obtaining a pixel luminance-enhancement value (Yenhance(p)) by extraction of high frequencies of luminance values of colors of pixels in its spatial neighborhood, and then inverse tone mapping the luminance (Y(p)) according to the equation Yexp(p)=Y(p)E′(p)×[Yenhance(p)]c.

There are provided methods and apparatus for artifact removal for bit depth scalability. The method and apparatus utilize an encoder (100) for encoding an enhancement layer for at least a portion of a picture. A deblocking filter is applied at the enhancement layer for bit depth scalability (500). A decoding method and apparatus are described for decoding an enhancement layer for at least a portion of a picture, wherein the during the decoding process, a deblocking filter is applied at the enhancement layer for bit depth scalability (600). Furthermore, an encoder and method are described for encoding image data for at least one block of a picture, wherein a deblocking filter (117) removes coding artifacts caused by local inverse tone mapping for intra-layer texture prediction for bit depth scalability. A decoding method and apparatus are described for decoding image data for at least one block of a picture, wherein a deblocking filter (230, 237, 435) removes coding artifacts caused by local inverse tone mapping for intra-layer texture prediction for bit depth scalability.

The invention provides a super-resolution, color gamut expansion and inverse tone mapping combined method and device, and the method comprises the steps: S1, carrying out down-conversion on video frames with high resolution, high color gamut and high dynamic range into video frames with low resolution, low color gamut and standard dynamic range, and forming a training data set and a test data set;S2, designing a convolutional neural network based on local residual learning and global residual learning; S3, continuously training and optimizing the convolutional neural network by using the training data set to obtain a convolutional neural network capable of completing joint super-resolution, color gamut extension and inverse tone mapping; and S4, inputting the low-resolution, low-color-gamut and standard dynamic range video frames in the test data set into the convolutional neural network obtained after training in S3 to obtain high-resolution, high-color-gamut and high-dynamic range video frames. According to the invention, the artifact problem of the existing method is improved, and the subjective and objective quality is improved.

There are provided methods and apparatus for inter-layer residue prediction for scalable video. An apparatus is described for an encoder (200) for encoding a block of a picture, or a decoder (300) for decoding a block of a picture, by applying inverse tone mapping to an inter-layer residue prediction process for the block, wherein the inverse tone mapping is performed in the pixel domain. Methods for encoding (440, 460) or decoding (540, 560) a block of a picture are also described; and performed by applying inverse tone mapping to an inter-layer residue prediction process for the block, wherein the inverse tone mapping is performed in the pixel domain.

The invention discloses an SDR to HDR conversion method, relates to an inverse tone mapping model based on machine learning, and belongs to the field of image processing. The SDR to HDR conversion method disclosed in the invention comprises the steps: predicting a high-brightness threshold value of a single-lens image through a machine learning method, and dividing the image into a diffuse reflection region and a high-light reflection region through the high-brightness threshold value; enabling the brightness of the dark areas of different lenses to be relatively stable, so that the brightness threshold of the dark areas is given according to experience; respectively establishing different brightness mapping functions for the dark part region, the diffuse reflection region, the high reflection region and the transition region among different brightness regions; and in a small interval which is close to the dark part threshold value and takes the high brightness threshold value as the center, taking a cubic spline interpolation function as a brightness mapping function, so that the brightness among the brightness areas is in smooth transition. The method can be applied to the fields of video popularization, televisions and movies and the like.

A lower dynamic range visual content (21) of a virtual reality or VR scene represented in a curved shape form is adapted to a higher dynamic range display of the virtual reality scene. At least two views planar key views (230) of the VR scene area obtained, at least two parameter values (240) respectively associated with those planar key views are obtained, corresponding to at least one parameter adapted to expand a dynamic range of a visual content, and a higher dynamic range visual content (22) corresponding to the lower dynamic range visual content in at least part of the VR scene extending beyond the planar key views is determined, based on the parameter values. Applications to Inverse Tone Mapping.

A method of producing at least one output image signal representing, at least, a plurality of output luminance values in an output range of luminance values is disclosed. The method involves: identifying a mapping function based at least in part on a perceptually scaled distribution of a plurality of input luminance values in an input range of luminance values different from the output range of luminance values; transforming the plurality of input luminance values into respective perceptually scaled output luminance values according to the mapping function; transforming the perceptually scaled output luminance values into respective ones of the plurality of output luminance values; and producing the at least one output signal representing, at least, the plurality of output luminance values. Computer-readable media and apparatuses are also disclosed.

A lower dynamic range visual content (21) of a virtual reality or VR scene represented in a curved shape form is adapted to a higher dynamic range display of the virtual reality scene. At least two planar key views (230) of the VR scene area obtained, at least two parameter values (240) respectively associated with those planar key views are obtained, corresponding to at least one parameter adapted to expand a dynamic range of a visual content, and a higher dynamic range visual content (22) corresponding to the lower dynamic range visual content in at least part of the VR scene extending beyond the planar key views is determined, based on the parameter values. The invention further provides applications to Inverse Tone Mapping.