A video distribution system with adjustable dynamic range.

Abstract

A video delivery system has a memory storing a plurality of chroma offset values corresponding to grid points of a 5D grid, and a processor for converting an input SDR image into a corresponding HDR output image, the processor being configured to: generate an intermediate HDR image by reshaping the input image, the reshaping being performed using a reshaping function index map having respective indexes identifying corresponding reshaping functions for each pixel of the intermediate image; estimate display management metadata values corresponding to the intermediate image; and generate the output image by applying respective chroma offsets to each pixel of the intermediate image, each respective chroma offset being determined from the plurality of chroma offset values by specifying the grid point using the respective index, the display management metadata value, and three respective pixel values of the corresponding pixel of the input image.

Description

[Technical Field]

[0001] 1. Cross-references to related applications This application claims priority from European Patent Application No. 22178928.2 and U.S. Provisional Patent Application No. 63 / 351,855 (both filed on June 14, 2022), each of which is incorporated herein by reference in whole.

[0002] 2. Areas of Disclosure Various exemplary embodiments relate to image processing operations, and more specifically, to video codecs, but are not limited thereto. [Background technology]

[0003] 3. Background This section presents aspects that may help facilitate a better understanding of this disclosure. Therefore, the statements in this section should be read in this context and should not be understood as an acknowledgment of what is or is not in the prior art.

[0004] As used herein, the term “dynamic range” (DR) may relate to the human visual system’s (HVS) ability to perceive a range of intensity (e.g., luminance, luma) within an image, such as from the darkest black (shades) to the brightest white (highlights). In this sense, DR relates to “scene-based” intensity. DR may also relate to the ability of a display device to render a given range of intensity well or approximately. In this sense, DR relates to “display-based” intensity. Unless explicitly specified that a particular meaning has a particular significance in any part of the description herein, the term should be presumed to be interchangeable in either sense, for example.

[0005] As used herein, the term “High Dynamic Range” (HDR) refers to a DR width of 14 to 15 orders of magnitude or more within the human visual system (HVS). In practice, the DR over which humans can simultaneously perceive a wide range of intensity may be somewhat truncated compared to HDR. As used herein, the terms “Enhanced Dynamic Range” (EDR) or “Visual Dynamic Range” (VDR) may individually or interchangeably relate to the DR perceptible within a scene or image by the human visual system, including eye movements, taking into account any adaptive light changes across the scene or image. As used herein, EDR may relate to a DR of 5 to 6 orders of magnitude. While perhaps somewhat narrower compared to true scene-based HDR, EDR nevertheless represents a wide DR width and is sometimes referred to as HDR.

[0006] In practice, an image contains one or more color components in a color space (e.g., lumens Y, chromins Cb, and Cr), each color component represented with n bits per pixel (e.g., n=8). Using nonlinear luminance coding (e.g., gamma coding), an image with n ≤ 8 (e.g., a 24-bit color JPEG image) is considered a standard dynamic range (SDR) image, while an image with n > 8 is considered an improved dynamic range image.

[0007] The reference electro-optic transfer function (EOTF) of a given display characterizes the relationship between the color values ​​(e.g., luminance) of an input video signal and the output screen color values ​​(e.g., screen luminance) produced by the display. For example, ITU Rec. ITU-R BT.1886, “Reference Electro-Optical Transfer Functions for Flat Panel Displays Used in HDTV Studio Production” (March 2011), defines a reference EOTF for flat panel displays. Given a video stream, information regarding its EOTF may be embedded in the bitstream as (image) metadata. The term “metadata” here refers to any auxiliary information transmitted as part of the encoded bitstream. Such metadata may be used to assist a decoder in rendering the decoded image and may include, but are not limited to, color space or color gamut information, reference display parameters, and auxiliary signal parameters, as further described herein.

[0008] As used herein, the term “PQ” refers to perceptual luminance amplitude quantization. The human visual system responds to increases in light levels in a highly nonlinear manner. A person’s ability to see a stimulus is influenced by the stimulus’s luminance, magnitude, the spatial frequencies that make up the stimulus, and the luminance level to which the eye has adapted at a particular moment in time when the stimulus is being viewed. In some cases, a PQ function maps a linear input gray level to an output gray level that better matches the contrast sensitivity threshold in the human visual system. An exemplary PQ mapping function is described in SMPTE ST 2084:2014 “High Dynamic Range EOTF for Mastering Reference Displays” (hereinafter, “SMPTE”), which is incorporated herein by reference in its entirety.

[0009] 200-1000 cd / m² 2Alternatively, displays that support nitrile luminance are typical of low dynamic range (LDR), also known as standard dynamic range (SDR), in contrast to EDR (or HDR). EDR content may be displayed on EDR displays that support a higher dynamic range (e.g., 1,000 nits to 5,000 nits or more). Such displays may be defined using alternative EOTFs that support high luminance capabilities (e.g., 0 to 10,000 nits or more). Examples of such EOTFs are defined in SMPTE 2084 and Rec. ITU-R BT.2100 “Image parameter values ​​for high dynamic range television for use in production and international program exchange” (06 / 2017). These are incorporated herein by reference in their entirety.

[0010] Patent Document 1 discloses an adaptive local reshaping method for SDR-HDR upconversion. Using a luma codeword in the input image, a global index value is generated for selecting a global reshaping function for an input image with a relatively low dynamic range. Image filtering is applied to the input image to generate a filtered image. The filtered value of the filtered image provides a measure of the local luma level in the input image. Using the global index value and the filtered value of the filtered image, a local index value is generated for selecting a specific local reshaping function for the input image. By reshaping the input image with the specific local reshaping function selected using the local index value, a reshaped image with a relatively high dynamic range is generated. [Patent Document 1] WO2022 / 072884A1

[0011] Patent Document 2 discloses a method for color correction in high dynamic range video (HDR) using a 2D lookup table (LUT). Color correction may be applied in the decoder after decoding the HDR video signal. For example, color correction may be applied before, during, or after chroma upsampling of the HDR video signal. The 2D LUT may include a representation of the color space of the HDR video signal. Color correction may include applying triangular interpolation to the sample values ​​of the color components of the color space. The 2D LUT may be estimated by the encoder and signaled to the decoder. The encoder may decide to reuse a previously signaled 2D LUT or to use a new 2D LUT. [Patent Document 2] WO2017 / 059415A1 [Overview of the project] [Means for solving the problem]

[0012] This specification discloses various embodiments of a video distribution system capable of performing color shift correction on HDR images generated from SDR images. In exemplary embodiments, color shift correction is performed using a pre-calculated lookup table (LUT) representing a five-dimensional (5D) grid, the LUT being addressable using reshaping function index values, metadata values, hue values, saturation values, and intensity values. Linear interpolation can be used to obtain chroma offset values ​​for any point in the corresponding 5D parameter space that is not a grid point. This specification also discloses exemplary sequential iterative minimization methods using a well-constructed cost function that can be used to fill the LUT. Beneficially, the exemplary embodiments of color shift correction disclosed are compatible with existing display-management functions and do not require modification.

[0013] According to one exemplary embodiment, a video distribution system is provided that can change the dynamic range of an input image. The distribution system includes: a memory that stores a plurality of chroma offset values ​​corresponding to grid points of a five-dimensional grid; a processor that converts an input image having a first dynamic range into a corresponding output image having a larger second dynamic range, wherein the processor performs the steps of: generating an intermediate image having the second dynamic range by reshaping the input image, wherein the reshaping is performed using a reshaping function index map having a respective index for each pixel of the intermediate image that identifies a corresponding reshaping function applied to that pixel; estimating a display management metadata value corresponding to the intermediate image; and generating the output image by applying a respective chroma offset to each pixel of the intermediate image, wherein the respective chroma offset is determined from the plurality of chroma offset values ​​by addressing the grid points using the respective index, the display management metadata value, and the respective pixel value of the corresponding pixel of the input image.

[0014] According to another exemplary embodiment, a method for changing the dynamic range of an input image is provided. The method comprises transforming an input image having a first dynamic range into a corresponding output image having a larger second dynamic range, wherein the transformation is performed using a plurality of pre-calculated chroma offset values ​​corresponding to grid points of a five-dimensional grid; the transformation includes: a step of generating an intermediate image having the second dynamic range by reshaping the input image, wherein the reshaping is performed using a reshaping function index map having a respective index for each pixel of the intermediate image that identifies a corresponding reshaping function applied to that pixel; a step of estimating a display management metadata value corresponding to the intermediate image; and a step of generating the output image by applying a respective chroma offset to each pixel of the intermediate image, wherein the respective chroma offset is determined from the plurality of chroma offset values ​​by addressing the grid points using the respective index, the display management metadata value, and the respective pixel value of the corresponding pixel of the input image.

[0015] In yet another exemplary embodiment, when executed by an electronic processor, a non-temporary computer-readable medium is provided which stores instructions causing the electronic processor to perform an operation including a method for changing the dynamic range of an input image. The method includes transforming an input image having a first dynamic range into a corresponding output image having a larger second dynamic range, the transformation being performed using a plurality of pre-calculated chroma offset values ​​corresponding to grid points of a five-dimensional grid; the transformation includes: a step of generating an intermediate image having the second dynamic range by reshaping the input image, the reshaping being performed using a reshaping function index map having a respective index for each pixel of the intermediate image that identifies a corresponding reshaping function applied to that pixel; a step of estimating a display management metadata value corresponding to the intermediate image; and a step of generating the output image by applying a respective chroma offset to each pixel of the intermediate image, the respective chroma offset being determined from the plurality of chroma offset values ​​by addressing the grid points using the respective index, the display management metadata value, and the respective pixel value of the corresponding pixel of the input image.

[0016] A further exemplary embodiment provides a method for generating multiple chroma offset values ​​for performing color shift correction in an output video generated by changing the dynamic range of an input image. This method includes defining a five-dimensional grid having first, second, third, fourth, and fifth dimensions representing reshaping function index values, metadata values, hue values, saturation values, and intensity values, respectively; defining a cost function for quantifying at least one cost for the hue difference between an input image and an output image; determining each set of chroma offset values ​​by performing sequential iterative minimization of the cost function for each grid point of the five-dimensional grid; and arranging each set of chroma offset values ​​in an electronic lookup table addressable using discrete value sets corresponding to the first, second, third, fourth, and fifth dimensions. [Brief explanation of the drawing]

[0017] Other aspects, features, and advantages of the various embodiments disclosed will become more fully apparent, for example, from the detailed description and accompanying drawings below.

[0018] [Figure 1] This illustrates an exemplary process for a video distribution pipeline.

[0019] [Figure 2] An exemplary process that can be used in the video distribution pipeline of Figure 1, according to one embodiment, is shown.

[0020] [Figure 3A] An indexing scheme that can be used in the process shown in Figure 2, according to one embodiment, is illustrated. [Figure 3B] An indexing scheme that can be used in the process shown in Figure 2, according to one embodiment, is illustrated. [Figure 3C] An indexing scheme that can be used in the process shown in Figure 2, according to one embodiment, is illustrated.

[0021] [Figure 4A] An example of calculating interpolation weights that can be used in the process shown in Figure 2, according to one embodiment, is graphically illustrated. [Figure 4B] An example of calculating interpolation weights that can be used in the process shown in Figure 2, according to one embodiment, is graphically illustrated. [Figure 4C] An example of calculating interpolation weights that can be used in the process shown in Figure 2, according to one embodiment, is graphically illustrated.

[0022] [Figure 5A] The graph illustrates an exemplary effect of scaling the probability distribution function of luminance Y according to one embodiment. [Figure 5B] The graph illustrates an exemplary effect of scaling the probability distribution function of luminance Y according to one embodiment.

[0023] [Figure 6A] Exemplary grids for the YCbCr color space, according to various embodiments, are shown in the figure. [Figure 6B] The diagram illustrates exemplary grids for the RGB color space according to various embodiments.

[0024] [Figure 7] This flowchart shows a method for embedding a 5D LUT for the process shown in Figure 2, according to one embodiment.

[0025] [Figure 8] This is a flowchart showing a sequential and iterative minimization process of the method shown in Figure 7, according to one embodiment. [Modes for carrying out the invention]

[0026] The disclosure and its aspects can be embodied in various forms, including hardware, devices or circuits controlled by computer implementations, computer program products, computer systems and networks, user interfaces and application programming interfaces, and hardware implementations, signal processing circuits, memory arrays, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), etc. The foregoing is intended only to give a general concept of the various aspects of the disclosure and is not intended to limit the scope of the disclosure.

[0027] The following description includes numerous details, such as optical device configuration, timing, and operation, to provide an understanding of one or more aspects of this disclosure. It will be readily apparent to those skilled in the art that these specific details are merely illustrative and not intended to limit the scope of this application.

[0028] Furthermore, while this disclosure primarily focuses on examples of how various circuits are used in digital projection systems, it should be understood that these are merely examples. It should also be understood that the disclosed systems and methods can be used in any device that needs to project light, such as cinemas, consumer and other commercial projection systems, head-up displays, virtual reality displays, etc. The disclosed systems and methods may be implemented in additional display devices having OLED displays, LCD displays, quantum dot displays, etc.

[0029] Many consumer desktop displays have a brightness of 200-300 cd / m². 2 Alternatively, it can support luminance in nits. Many consumer HDTVs are in the 300-500 nit range, and newer models are 1000 nits (cd / m²). 2As advancements in both image capture equipment (e.g., cameras) and HDR displays (e.g., the PRM-4200 professional reference monitor from Dolby Laboratories) increase the availability of HDR content, HDR content may be color graded and displayed on HDR displays that support a higher dynamic range (e.g., 1000 nits to 5000 nits or more).

[0030] Some embodiments may benefit from at least some features disclosed in the international patent application “ADAPTIVE LOCAL RESHAPING FOR SDR-TO-HDR UP-CONVERSION,” PCT / US2021 / 053241, filed 1 October 2021, which is incorporated herein by reference in its entirety.

[0031] In this specification, the term “metadata” refers to any auxiliary information transmitted as part of an encoded bitstream to assist the decoder in rendering the corresponding image. For television broadcasts and video streaming, video metadata can be used to provide side information about a particular video and audio stream or file. Metadata may be either directly embedded in the video or contained as a separate file within a container such as MP4 or MKV. Metadata may include information about the entire video stream or file, or information about a particular video frame. Metadata is created by the camera, encoder, and other video processing elements (see, for example, 115, 120 in Figure 1) and may include, but is not limited to, timestamps, video resolution, digital film grain parameters, color space or color gamut information, reference display parameters, auxiliary signal parameters, file size, closed captions, audio language, ad insertion points, color space, error messages, etc. Additional examples of metadata related to the disclosed embodiments are described below in this specification.

[0032] In some embodiments disclosed herein, image metadata includes L1 metadata. As used herein, the term “L1 metadata” represents one or more of the minimum (L1-min), intermediate (L1-mid), and maximum (L1-max) luminance values ​​associated with a particular portion of video content, such as an input frame or image. The L1 metadata is associated with the video signal. To generate the L1 metadata, a pixel-level, frame-by-frame analysis of the video content is preferably performed on the encoding side. Alternatively, the analysis may be performed on the decoding side. The analysis describes the distribution of luminance values ​​across a defined portion of the video content covered by an analysis pass, such as a single frame or a series of frames like a scene. The L1 metadata may be computed in an analysis pass covering a single video frame and / or a series of frames like a scene. The L1 metadata has various values ​​derived during the analysis pass, associated with each portion of the video content from which the L1 metadata was computed, and together form the L1 metadata associated with the video signal. Such L1 metadata may include at least one of the following: (i) an L1-min value representing the lowest black level in each part of the video content; (ii) an L1-mid value representing the average luminance level across each part of the video content; and (iii) an L1-max value representing the highest luminance level in each part of the video content. L1 metadata may be generated for each video frame and / or each scene encoded in the video signal and attached to them. L1 metadata may also be generated for areas of an image, and such L1 metadata may be called local L1 values. L1 metadata may be calculated by converting RGB data to a lumens-chroma format (e.g., YCbCr) and then calculating one or more of the minimum, midpoint (mean), and maximum values ​​in the Y plane, or they may be calculated directly in RGB space.

[0033] In some embodiments, the L1-min value may represent the minimum PQ-encoded min(RGB) value of each portion of video content (e.g., video frame or image), taking only the active area into consideration (by excluding, for example, gray bars or black bars, letterbox bars, etc.), where min(RGB) represents the minimum color component value {R,G,B} of the pixel. The L1-mid and L1-max values ​​can be calculated similarly. In particular, in exemplary embodiments, L1-mid may represent the average of the PQ-encoded max(RGB) values ​​of the image, and L1-max may represent the maximum PQ-encoded max(RGB) value of the image, where max(RGB) represents the maximum color component value {R,G,B} of the pixel. In some embodiments, the L1 metadata may be normalized to fall within the range [0,1].

[0034] Video coding according to an exemplary embodiment Figure 1 shows an exemplary process of a video distribution pipeline (100) illustrating various stages from video capture to video content display according to one embodiment. A sequence of video frames (102) may be captured or generated using an image generation block (105). The video frames (102) may be digitally captured (for example, by a digital camera) or generated by a computer (for example, using computer animation) to provide video data (107). Alternatively, the video frames (102) may be captured on film by a film camera. The film may then be converted to a digital format to provide video data (107).

[0035] In the production phase (110), video data (107) can be edited to provide a video production stream (112). The data in the video production stream (112) can then be provided to a processor (or one or more processors such as a central processing unit, CPU, etc.) in a post-production block (115) for post-production editing. Post-production editing in block (115) may include, for example, adjusting or correcting the color or brightness in specific areas of the image to improve image quality or to achieve a specific look for the image, according to the creative intent of the video producer. Other edits (e.g., scene selection and sequencing, image cropping, addition of computer-generated visual special effects, etc.) may be performed in block (115) to produce a "final" version (117) of the production for distribution. During post-production editing (115), the video image can be viewed on a reference display (125).

[0036] Following post-production (115), the final version (117) of the video data may be delivered to an encoding block (120) for downstream distribution to decoding and playback devices such as television sets, set-top boxes, and cinemas. In some embodiments, the encoding block (120) may include audio and video encoders, such as those defined by ATSC, DVB, DVD, Blu-ray, and other distribution formats, to generate an encoded bitstream (122). Several methods described below herein may be performed in the encoding block (120) by corresponding processors. For example, the encoding block (120) may be configured to perform SDR-HDR local reshaping and color shift correction, as described in more detail below. At the receiver, the encoded bitstream (122) may be decoded by a decoding unit (130) to produce a corresponding decoded signal (132) representing a copy or close approximation of the signal (117). The receiver may be mounted on a target display (140) which may have characteristics that are somewhat or completely different from those of a reference display (125). In such a case, a display management (DM) block (135) may be used to map the decoded signal (132) to the characteristics of the target display (140) by generating a display-mapped signal (137). Several methods described below herein may be performed by a decoding unit (130) and / or a display management block (135). Depending on the embodiment, the decoding unit (130) and the display management block (135) may include individual processors or be based on a single integrated processing unit.

[0037] In one exemplary embodiment, SDR-HDR local reshaping can take a 3-channel (Y,Cb,Cr) input SDR image and predict a 3-channel (Y,Cb,Cr) output HDR image using a pre-trained reshaping function. To view the HDR image on a target display (140), a DM block (135) may further process the HDR image based on the corresponding metadata to generate a display-mapped signal (137) representing the DM image according to the target display luminance.

[0038] In most cases, the chromaticity of HDR and DM images is equal to or close to that of SDR images, even if the luminance differs due to upconversion and enhancement. However, in some cases, the difference in chromaticity can be significant, and such a difference is called SDR-HDR color shift. Color shift from SDR to HDR can occur for several reasons. For example, since pre-trained reshaping functions and DMs are typically trained on natural images, such training can introduce larger errors and / or color shifts for colors that are not very prevalent in natural images. Also, DM blocks (135) may perform clipping on pixels near color space boundaries, thereby amplifying existing color shifts or introducing new color shifts.

[0039] In one exemplary embodiment, the DM block (135) may remain unmodified and, for example, identical to that of a legacy video distribution pipeline. Rather, the exemplary embodiment of the proposed color shift correction framework is designed to correct end-to-end color shift between an SDR image and a final DM image by adding a chroma offset to the Cb and Cr channels of the HDR image. Since the end-to-end process from an SDR image to a DM image is typically highly nonlinear, the required chroma offset is determined using a 5D lookup table (LUT) on a 5D grid, where the 5Ds are the SDR pixel values ​​(3D), the reshaping function index, and the DM metadata L1-mid. The resulting 5D space is referred to herein as the HSWLM space, where H represents hue, S represents saturation, W represents the scaled intensity value, L represents the local reshaping function index, and M represents the metadata. The 5D LUT can be populated with values ​​using an appropriate cost function by sequential iterative minimization, for example, as will be further detailed below with reference to Figures 7 and 8.

[0040] Color shift correction Figure 2 shows an exemplary process (200) of a video distribution pipeline (100) according to one embodiment. Process (200) can typically receive an input SDR image (117) and generate a corresponding output DM image (137) (see also Figure 1). During process (200), a chroma offset can be added to the initial HDR image (216) so that any color shifts that may occur for the reasons described above can be mitigated.

[0041] The process (200) includes a reshaping block (210) and a chroma offset processing block (220). The reshaping block (210) is configured to generate an initial HDR image (216) based on the input SDR image (117). The reshaping block (210) includes generating a reshaping function index map (212) and performing processing directed to applying a pre-trained reshaping function (214) to the input SDR image (117). In some embodiments, the reshaping block (210) may be implemented as disclosed in the above-cited international patent application PCT / US2021 / 053241.

[0042] In one exemplary embodiment, the reshaping block (210) includes SDR-HDR local reshaping, and the 3-channel (Y, Cb, Cr) output HDR image (216) is predicted using the 3-channel (Y, Cb, Cr) input SDR image (117) and a set of pre-trained reshaping functions (214). The reshaping function index map (212) is created to indicate which reshaping function (214) is used for different pixels.

[0043] Let the Y, Cb, and Cr channels of the i-th pixel in the input SDR image S(117) be s i YCbCr =(s i Y ,s i Cb ,s i Cr ), and let the reshaping function index of the i-th pixel in the reshaping function index map L(212) be l i . The initial HDR image V (init) (216) generated by the SDR-HDR local reshaping of the reshaping block (210), where the i-th pixel v i (init),YCbCr =(v i (init),Y ,v i (init),Cb ,v i (init),Cr ) can be expressed as follows.

number

[0044] As a non-limiting example, the processing of a single image / frame is described below. Based on the description provided, a person skilled in the art will be able to implement the corresponding processing of multiple frames, for example, without any excessive experimentation, since the disclosed processing does not typically or explicitly rely on temporal information. For the sake of simplicity and without any limiting implications, the description is given for normalized pixel values, i.e., values ​​belonging to the range [0,1]. It is also described that conversions between color spaces can be performed as needed. For example, in the YCbCr color space, s i YCbCr If a value exists, the corresponding value in the RGB color space is s i RGB It is also assumed that such conversions between color spaces can be performed. Those skilled in the art will readily understand how to perform such conversions between color spaces.

[0045] To view an HDR image, for example, an initial HDR image (216) or an output HDR image (240), on a target display (140), processing of a DM block (135) is typically applied to the HDR image, the result of which is the output DM image (137). Such DM processing may be controlled by the metadata L1-min, L1-mid, and L1-max described above, which represent the minimum, mean, and maximum values ​​of the RGB channels of the HDR image, respectively. However, in exemplary SDR-HDR upconversion scenarios, L1-min and L1-max may be set to constant values ​​in at least some cases. Thus, some embodiments may rely on the L1-mid value alone.

[0046] If a chroma offset processing block (220) does not exist, the initial HDR image (216) is applied directly to the DM block (135). Initial HDR image V (init) The R, G, and B channels of the i-th pixel in (216) are v i (init),RGB =(v i (init),R ,v i (init),G ,v i (init),B This is expressed as follows. The DM metadata L1-mid of the initial HDR image can be calculated as follows:

number

number

number

[0047] In some cases, ^s i(init),YCbCr and s i YCbCr They may have similar chromaticity. However, in other cases, ^s i(init),YCbCr and s i YCbCr The chromaticity difference, or color shift, between the two can become noticeable to the observer. Therefore, the processing implemented in the chroma offset processing block (220) is aimed at significantly reducing or completely eliminating such differences. In one exemplary embodiment, the chroma offset processing block (220) uses a 5D grid and a corresponding 5D LUT (228).

[0048] A grid of dimension D (for example, D=5) corresponds to a multidimensional space R D It can be constructed by sampling along each dimension in . For computational efficiency, a uniformly spaced grid can be used, and coordinate values ​​in the same dimension are sampled uniformly. For the dth dimension, N d The number of values, initial value p d Starting from interval b d Let's assume it's sampled at i. d The second sample is p d +i d b d It can be calculated as follows: Here, i d =0,1,…,N d -1, and d=0,1,…,D-1. This sampling defines grid X. Index i=(i0,i1,…,i D-1 ) lattice point X i It can be expressed as follows:

number

[0049] In general, LUTs such as 5D LUT(228) can be constructed to model any function φ on a grid. More specifically, the LUT function Φ on grid X is defined by each input grid point X i The output value Φ corresponds to the LUT. i =φ(X i It can be defined so that it can return ). For example, in one embodiment, function Φ may be a LUT representing a function φ that takes SDR pixel values, a reshaping function index, and DM metadata L1-mid as inputs and then provides a chroma offset as an output. In various embodiments, the outputs of functions φ and Φ may be in scalar or vector form. In the non-restrictive example described below, the output is a 2D vector for the chroma offset in the Cb and Cr channels.

[0050] In one exemplary embodiment, linear interpolation may be used to handle input values ​​that do not lie on the grid. For example, such linear interpolation may follow a definition similar to that used in ordinary bilinear or trilinear interpolation, and the output value is based on the corresponding linear interpolation in each dimension. To facilitate computation, one exemplary embodiment may rely on normalized grid coordinates. This approach provides a normalized grid starting from 0 and progressing at unit intervals. Input x = (x0, x1, ..., x D-1 Given ), the corresponding normalized grid coordinates are obtained by shifting and scaling.

number

number

number

[0051] Furthermore, normalized grid

number

number

number

number

[0052] Figures 3A to 3C illustrate the application of the above-described indexing method to an exemplary 4x4 2D grid (with a tilde X) according to one embodiment. More specifically, Figure 3A shows two grid dimensions, represented as dimension 0 and dimension 1, respectively. Figure 3B shows the indexing, where the grid points (represented as nodes) are indexed as described above. Figure 3C shows the indexing, where the hypercube is indexed as described above.

[0053] Since linear interpolation is linear in each dimension, such linear interpolation can be performed using normalized coordinates (tilde x) and normalized grids (tilde X) instead of the corresponding denormalized entities. In one exemplary embodiment, the processing step of performing linear interpolation may include a substep of finding the unit hypercube containing tilde x, and then performing interpolation using the distance between tilde x and the vertices of the unit hypercube. The index of the unit hypercube containing tilde x is

number

[0054] Let ^φ(x) be the interpolation result of the input x. Based on the above definition of linear interpolation, the interpolation result ^φ(x) can be expressed as follows:

number

[0055] The weights of the linear interpolation above are x with a tilde and X with a tilde within the same unit hypercube. i' Since it depends only on the distance between the two points, for computational efficiency, the weights can be pre-calculated for multiple possible distances, allowing their lookup at runtime. For example, a unit hypercube can be quantized, and the corresponding interpolation weights can be stored in a LUT. When the quantization is relatively dense, the output of the LUT is typically relatively close to the corresponding unquantized interpolation result.

[0056] Consider a unit hypercube located at the origin. There are 2 such unit hypercubes. D The number of vertices V = {0,1} D It has. In one exemplary embodiment, the vertices are their coordinates, i.e., V k = vector k=(k0,k1,…,k D-1 ) can be indexed by M in the closed interval [0,1] for each dimension d=0,1,...D-1. d If each point is quantized uniformly, the quantization grid Q is given by index j=(j0,j1,...,j D-1 A quantization point with ) can be defined as follows:

number

[0057] Node Q j The linear interpolation weights for vertex k can be calculated as follows:

number

[0058] Figures 4A to 4C illustrate an example of calculating interpolation weights in a 2D unit hypercube using a quantization grid Q of size 4x5 according to one embodiment. More specifically, Figure 4A shows the two dimensions of the unit hypercube, represented as dimension 0 and dimension 1, respectively. Figure 4B shows the quantization grid Q for the two dimensions of the unit hypercube, with the indices V of the four corresponding vertices. k =k=(k0,k1,…,k D-1 ) is explicitly shown. Figure 4C shows node Q for the four vertices of the unit hypercube shown in Figure 4B. 2,1The coordinates and their corresponding calculated weights are shown.

[0059] To use LUT W, the input q in the unit hypercube is q=(q0,q1,…,q D-1 Regarding the input, the input is the nearest node Q located in the direction toward the origin. j It is mapped to:

number

number

number

number

[0060] In one embodiment, the 5D grid X can be defined within the HSWLM space described above. To obtain better numerical stability and interpolation quality, the 5D grid X can be aligned with the boundary of the valid input parameter space. Such alignment can typically help to properly perform interpolation for input points located near the boundary. For the reshaping function index and DM metadata L1-mid, their original values ​​are independent (separated) from other dimensions, so these values ​​can be used for the grid X. For example, changing the reshaping function index and DM metadata L1-mid does not invalidate inputs that are valid at other points. On the other hand, for SDR pixel values, when using the YCbCr color space, 0≦s i Y ,s i Cb ,s i CrNot all combinations of ≤1 are valid. One way to prevent invalid inputs in the YCbCr color space is to cross-reference the YCbCr input to other color spaces. For example, in the RGB or HSV color spaces, all combinations of [0,1] values are valid (see also Figure 6B). Here, RGB represents red, green, and blue, and HSV represents hue, saturation, and value.

[0061] In one exemplary embodiment, the scaled HSV color space for grid creation can be designed such that the density of the grid is proportional to the perceived color difference. For example, the V component can be scaled in a non-linear manner such that the density of the grid remains approximately the same for different luminance values. Let the H, S, and V channels of the i-th pixel in the input SDR image S(117) be s i HSV =(s i H ,s i S ,s i V ). Then, the HSW color space can be defined as follows.

Equation

[0062] Figures 5A and 5B graphically illustrate the effect of VW scaling on the probability distribution function (PDF) of luminance Y according to one embodiment. Scaling can be performed, for example, in the HSW channel block (224) of process (200). In this particular example, Figure 5A graphically shows the PDF as a function of luminance Y for a grid created in the HSV color space and then converted to the YCbCr color space. Figure 5B graphically shows the PDF as a function of luminance Y for a grid created in the HSW color space and then converted to the YCbCr color space. Comparing the two PDFs, it is clear that the PDF in Figure 5B is more uniform than the PDF in Figure 5A. Here, luminance Y is within the typical SMPTE range (SMPTE stands for Society of Motion Picture and Television Engineers).

[0063] Figures 6A and 6B show an exemplary grid X created in the HSW color space and converted to the YCbCr and RGB color spaces, respectively, according to one embodiment. In this particular example, the grid size is 13 × 5 × 9, with a range of [0,1] for each of the HSW dimensions. From Figure 6A, it can be seen that grid Q in the YCbCr color space occupies only a portion of the [0,1][0,1][0,1] cube. In contrast, grid Q in the RGB color space occupies the entire [0,1][0,1][0,1] cube. In both cases, the grid points are appropriately aligned with the respective valid color space boundaries. Edges connecting R=G=B=0 or R=G=B=1 on the RGB color space boundaries have hue values ​​{0,1 / 6,2 / 6,3 / 6,4 / 6,5 / 6}. Therefore, to precisely place the grid points on these edges, the grid size in the H dimension can be selected according to equation 6n+1. Here, n is a positive integer. Other edges on the RGB color space boundary have a saturation value of 1, and accordingly, those edges have grid points on them, regardless of any particular grid size.

[0064] Typically, the DM metadata L1-mid of an HDR image represents the average of the RGB channels of the image. However, in process (200), the RGB channels of the output HDR image (240) depend on the chroma offset (230) (see Figure 2). As a result, it is necessary to estimate the DM metadata L1-mid. According to one exemplary embodiment, such an estimate (222) is obtained using the initial HDR image (216). More specifically, the estimate of the DM metadata L1-mid (222) is denoted as ^m [^m] and can be calculated as the average of the Y channels of the initial HDR image (216) as follows:

number

[0065] Referring back to Figure 2, in one exemplary embodiment, the 5D LUT Φ(228) used in the chroma offset processing block (220) of process (200) can be defined on a 5D grid in HSWLM space. In operation, the 5D LUT Φ(228) outputs a chroma offset (230) in response to an input vector (226) defined in HSWLM space. The input vector (226) is constructed using an HSW channel block (224), a reshaping function index map (212), and an estimate of the DM metadata L1-mid (222). An exemplary training process that can be used to populate the 5D LUT(228) is described in more detail below (see, for example, Figures 7-8). The chroma offset (230) obtained using the 5D LUT(228) is added (232) to the initial HDR image (216), thereby generating the output HDR image (240). The DM block (135) then processes the output HDR image (240) to generate the output DM image (137).

[0066] In one exemplary embodiment, the chroma offset (230) for different pixels may be determined, for example, for each pixel, using the linear interpolation described above. Here, the linear interpolation operation is represented as ^φ. For the i-th pixel, the chroma offset r i CbCr =(r i Cb ,r i Cr ) can be calculated as follows.

Equation

Equation

Equation

Equation

[0067] Parameter learning Figure 7 is a flowchart illustrating a method (700) for populating a 5D LUT (228) according to one embodiment. In one exemplary embodiment, method (700) depends on a cost function (704), which may typically include a color shift term and one or more regularization terms. For each selected grid point (706) on a suitable (e.g., predefined as described above) 5D grid (702), method (700) includes a sequential iterative minimization process (708) configured to find a chroma offset (710) corresponding to an approximate minimum of the cost function (704), and updating the emerging 5D LUT (228) using the found chroma offset (712). Method (700) further includes repeating a set of processing operations (708), (710), and (712) for a plurality of different selected grid points (706). The end of this iterative cycle occurs when an exit condition (714) is met. After completion, the method (700) includes outputting the 5D LUT with values ​​entered (716) and saving it as a 5D LUT (228) (see also Figure 2).

[0068] In one exemplary embodiment, the cost function (704) may be constructed to drive a sequential iterative minimization process (708) to find a nearly optimal chroma offset (710) that can compensate for the aforementioned color shift on the 5D grid X (702) for multiple grid points. For computational efficiency, the processing operations (708), (710), and (712) may be configured to process one grid point at a time. Below, the selected grid point X at index i. i Regarding (706), the H, S, and W channels at that point are s i HSW =(s i H ,s i S ,s i W )=(X i,0 ,X i,1 ,X i,2 ) is expressed as; the reshaping function index is l i =X i,3It is expressed as; DM metadata L1-mid(222) is m i =X i,4 This is expressed as follows: The corresponding HDR value for the initial HDR image (216) is v i (init),YCbCr =f l (B) (s i YCbCr )

[0069] If chroma offset (230) is not applied, the HDR value v i (init),YCbCr Pass this to the DM block (135) and set the initial DM value ^s i (init),YCbCr =f (DM) (v i (init),YCbCr ,m i ) is obtained. On the other hand, chroma offset r CbCr =(r Cb ,r Cr ) is the HDR value v i (init),YCbCr When applied to the Cb and Cr channels, the new HDR value v i YCbCr =v i (init),YCbCr +(0,r Cb ,r Cr ) is generated for the HDR image (240), and the final output DM value for the DM image (137) is ^s i YCbCr =f (DM) (v i YCbCr ,m i )

[0070] In one exemplary embodiment, it is desirable that the chroma offset (710) can significantly reduce (e.g., to an imperceptible level) or completely eliminate the SDR-HDR color shift. However, it is also desirable that the chroma offset (710) does not generate artifacts or alter the "look" of the image. Therefore, the cost function (704) may be constructed to include a color shift term and one or more regularization terms to ensure stability. Since the cost function (704) considers only one grid point (706) at a time, the cost function (704) may be constructed to be independent of the position of the grid point (706) on the 5D grid (702), and furthermore, independent of any particular topological features of the 5D grid (702).

[0071] For illustrative purposes only, and without any implied limitations, the example of the cost function (704) described below includes the following term: hue difference cost E hue , offset cost E off Luminance change cost E lum , saturation change cost E sat , and effective range cost E valid Total cost function E total (704) is defined as follows: E total =E hue +λ off E off +λ lum E lum +λ sat E sat +E valid (18) Here, .'' off , λ lum , and λ sat is a weighting constant. Term E valid For this particular term, the output value is 0 or infinity, so the weighting constant is 1. Examples of other weighting constants are λ off =0.01, λ lum =0.04, and λ sat= 0.0025. In various other embodiments, the cost function (704) may have more or fewer cost terms. Some of the terms of such other cost functions (704) may differ from the exemplary cost terms listed above.

[0072] Hue difference cost E hue This is the color shift term. In one exemplary embodiment, the color shift may be measured by the difference in hue in the HSV color space. Note that if the SDR image has neutral colors, its hue may be undefined and its saturation may be 0. Measuring the color shift by the change in saturation in the HSV color space can typically help to appropriately handle such situations. The H, S, and V channels of the input SDR value and the final output DM value are respectively:

number

number

[0073] Offset cost E off This is a regularization term configured to regularize the chroma offset within a reasonable range and avoid overfitting. Such an offset cost E off It can be defined as follows:

number

[0074] Luminance change cost E lum This is a regularization term configured to regularize the luminance change caused by the chroma offset. The cost of such a luminance change is E. lum It can be defined as follows:

number

[0075] Saturation change cost E sat This is a regularization term configured to regularize the saturation changes caused by chroma offset. Such a saturation change cost E sat It can be defined as follows:

number

[0076] Effective range cost E valid This is a regularization term configured to limit the effective range of the new HDR value. Such an effective range cost E valid It can be defined as follows:

number

[0077] Figure 8 is a flowchart of a sequential iterative minimization process (708) according to one embodiment. The sequential iterative minimization process (708) is performed using a cost function (704), for example, the total cost function E of equation (18). total The input to the sequential iterative minimization process (708) is the grid point X. i (804) includes the initial values ​​(802) for the chroma offset and step size. The output of the sequential iterative minimization process (708) includes the chroma offset (710) corresponding to the minimum value of the cost function (704). The chroma offset (710) thus obtained may typically be stored in a 5D LUT (228). For computational efficiency, the sequential iterative minimization process (708) is configured to find the chroma offset (710) within a relatively small local range specified for the computation block (806). The corresponding processing loop, which includes the block (808) for computing the value of the cost function (704), converges (814) or reaches a maximum number of iterations t. max The process continues until (812) is reached. The step size can be modified (typically reduced) in the modification block (816) to better correspond the chroma offset (710) to the actual minimum value of the cost function (704) within the local range used. For computational efficiency, the step size is not allowed to be smaller than a specified fixed minimum step size checked in the step size check block (818).

[0078] In one exemplary embodiment, the initial value of the chroma offset (802) is r0 CbCr It may be set to =(0,0). For t≧1, in the tth iteration, the local range R for the processing block (806) t CbCr It can be set as follows:

number

number

[0079] The sequential iterative minimization process (708) in the counter block (812) has a maximum number of iterations t max If it reaches r CbCr* =r t CbCr It is set to . Otherwise, if it is determined that the estimated chroma offset is at the local minimum (814), i.e., r t CbCr =r t-1 CbCr If so, the step size for the next iteration is Δr t+1 =αΔr t It may be reduced to (816). Here, α < 1 is a constant. The value of α can typically be chosen to achieve the desired convergence rate. t CbCr ≠r t-1 CbCr If so, the process is Δr t+1 =Δr t You may proceed with this. The precision of the chroma offset (710) is typically a constant Δr min It can be controlled by Δr t+1 <Δr min If so, the convergence criteria are considered to be met, and the output chroma offset (710) is r CbCr* =r t CbCrIt is set to . In one exemplary embodiment, the following parameter value may be used: Δr1 = 10 -3 , Δr min =10 -6 α=0.5, t max =100. In some embodiments, Δr min It is about 10 -3 ~10 -6 It may also be within the range between these values, and the visual quality of the corresponding output HDR image (137) may still be acceptable for certain applications even at the top of this range.

[0080] For example, in the Overview section and / or by reference to any one or any combination of any part or all of Figures 1 to 8, according to the exemplary embodiments disclosed above, a device is provided which includes a video distribution system capable of changing the dynamic range of an input image, the distribution system comprising: a memory (e.g., 228 in Figure 2) that stores a plurality of chroma offset values ​​corresponding to grid points of a five-dimensional grid; and a processor (e.g., 120 in Figure 1) that converts an input image (e.g., 117 in Figure 2) having a first dynamic range (e.g., SDR in Figure 2) to a corresponding output image (e.g., 240 in Figure 2) having a larger second dynamic range (e.g., HDR in Figure 2), the processor converts the second dynamic range by reshaping the input image (e.g., 210 in Figure 2). The system is configured to generate an intermediate image with a range (e.g., 216 in Figure 2) (reshaping is performed using a reshaping function index map (e.g., 212 in Figure 2) which has a corresponding index for each pixel of the intermediate image that identifies the corresponding reshaping function applied to that pixel); estimate the display management metadata value corresponding to the intermediate image (e.g., 222 in Figure 2); and generate an output image by applying a corresponding chroma offset (e.g., 230 in Figure 2) to each pixel of the intermediate image (e.g., 232 in Figure 2), where each chroma offset is determined from a plurality of chroma offset values ​​by addressing a grid point using the respective pixel values ​​of the corresponding pixel in the input image: the respective index, the display management metadata value, and the respective pixel value of the corresponding pixel in the input image.

[0081] In some embodiments of the above-described apparatus, the first dynamic range is the standard dynamic range (e.g., SDR, Figure 2), and the second dynamic range is the high dynamic range (e.g., HDR, Figure 2).

[0082] In some embodiment of the above apparatus, the three pixel values ​​are the hue, saturation, and intensity values ​​of the corresponding pixels in the input image.

[0083] In some embodiment of the apparatus described above, the processor is further configured to non-linearly rescale the intensity values ​​of the input image (e.g., V-to-W, 224 in Figure 2), where the three respective pixel values ​​are the hue, saturation, and rescaled intensity values ​​of the corresponding pixels in the input image.

[0084] In some embodiment of the above-described apparatus, the video distribution system is configured to generate a display-adapted image (e.g., 137 in Figure 2) by applying display management processing to the output image.

[0085] In some embodiment of the above-described apparatus, the video distribution system includes a video encoder (e.g., 120, Figure 1) which includes at least a portion of a processor.

[0086] In some embodiment of the above apparatus, the plurality of chroma offset values ​​are stored in memory in a lookup table that can be addressed using reshaping function index values, metadata values, hue values, saturation values, and intensity values.

[0087] In some embodiment of the above-described device, the display management metadata value corresponding to the intermediate image is the L1-mid luminance value.

[0088] In some embodiment of the apparatus described above, the processor is further configured to perform linear interpolation of chroma offset values ​​(for example, equations (6) to (11)) to determine each chroma offset.

[0089] For example, in the Overview section and / or by reference to any one or any combination of any part or all of Figures 1 to 8, a method for changing the dynamic range of an input image is provided according to another exemplary embodiment disclosed above, the method comprising the steps of transforming an input image (e.g., 117 in Figure 2) having a first dynamic range (e.g., SDR in Figure 2) to a corresponding output image (e.g., 240 in Figure 2) having a larger second dynamic range (e.g., HDR in Figure 2), the transformation being performed using a plurality of pre-calculated chroma offset values ​​corresponding to grid points of a five-dimensional grid, the transformation being performed by reshaping the input image (e.g., 210 in Figure 2) to an intermediate image having the second dynamic range (e.g., The process involves generating an intermediate image (216 in Figure 2) (reshaping is performed using a reshaping function index map (e.g., 212 in Figure 2) which has a corresponding index for each pixel of the intermediate image that identifies the corresponding reshaping function applied to that pixel); estimating the display management metadata value corresponding to the intermediate image (e.g., 222 in Figure 2); and generating the output image by applying a corresponding chroma offset (e.g., 230 in Figure 2) to each pixel of the intermediate image (e.g., 232 in Figure 2), where each chroma offset is determined from a plurality of chroma offset values ​​by addressing a grid point using three respective pixel values: their respective index, the display management metadata value, and the corresponding pixel of the input image.

[0090] In some embodiments of the above method, the method further includes non-linearly rescaling the intensity values ​​of the input image (e.g., V-to-W, 224, Figure 2), where the three respective pixel values ​​are the hue, saturation, and rescaled intensity values ​​of the corresponding pixels in the input image.

[0091] In some embodiments of the above method, the method further includes generating a display-adapted image (e.g., 137 in Figure 2) by applying display management processing to the output image.

[0092] In some embodiment of the above method, the display management metadata value corresponding to the intermediate image is the L1-mid luminance value.

[0093] In some embodiment of any of the above methods, the transformation further includes performing linear interpolation of the chroma offset values ​​(for example, equations (6) to (11)) to determine each chroma offset.

[0094] In some embodiment of any of the methods described above, multiple chroma offset values ​​are placed in an addressable lookup table using reshaping function index values, metadata values, hue values, saturation values, and intensity values.

[0095] For example, in the Overview section and / or with respect to any one or any combination of parts or all of Figures 1 to 8, according to yet another exemplary embodiment disclosed above, a non-temporary computer-readable medium is provided for storing instructions causing an electronic processor to perform an operation including a method for changing the dynamic range of an input image, the method comprising the steps of transforming an input image (e.g., 117 in Figure 2) having a first dynamic range (e.g., SDR in Figure 2) to a corresponding output image (e.g., 240 in Figure 2) having a larger second dynamic range (e.g., HDR in Figure 2), the transformation being performed using a plurality of pre-calculated chroma offset values ​​corresponding to grid points of a five-dimensional grid, the transformation including reshaping the input image (e.g., 21 in Figure 2) 0) generate an intermediate image having a second dynamic range (e.g., 216 in Figure 2) (reshaping is performed using a reshaping function index map (e.g., 212 in Figure 2) which has a corresponding index for each pixel of the intermediate image that identifies the corresponding reshaping function applied to that pixel); estimate the display management metadata value corresponding to the intermediate image (e.g., 222 in Figure 2); and generate an output image by applying a corresponding chroma offset (e.g., 230 in Figure 2) to each pixel of the intermediate image (e.g., 232 in Figure 2), where each chroma offset is determined from a plurality of chroma offset values ​​by addressing a grid point using three respective pixel values: their respective index, the display management metadata value, and the corresponding pixel of the input image.

[0096] For example, in the Overview section and / or by reference to any one or any combination of any part or all of Figures 1 to 8, a method is provided for generating a plurality of chroma offset values ​​for performing color shift correction in an output image produced by changing the dynamic range of an input image, the method comprising a 5-dimensional grid (e.g., 702 in Figure 7) having first, second, third, fourth, and fifth dimensions representing a reshaping function index value, metadata value, hue value, saturation value, and intensity value, respectively. This involves defining; defining a cost function (e.g., 704 in Figure 7) to quantify at least the cost (e.g., equation (19)) of the hue difference between the input and output images; determining each set of chroma offset values ​​by performing sequential iterative minimization of the cost function for each grid point (e.g., 804 in Figure 8) of a five-dimensional grid; and placing each set of chroma offset values ​​in an addressing electronic lookup table (e.g., 228 in Figure 2) using discrete sets of values ​​corresponding to the first, second, third, fourth, and fifth dimensions.

[0097] In some embodiments of the above method, defining the cost function involves including in the cost function one or more regularization terms configured to keep sequential iterative minimization within a valid range.

[0098] In some embodiments of any of the methods described above, sequential iterative minimization is performed within a local range of the parameter (e.g., 806 in Figure 8) that is narrower than the entire range of the parameter.

[0099] In some embodiments of any of the methods described above, sequential iterative minimization is performed using a variable step size (e.g., 818 in Figure 8).

[0100] With respect to the processes, systems, methods, heuristics, etc. described herein, the steps of such processes, etc. are described as occurring in a certain ordered sequence, but it should be understood that such processes may be carried out using described steps performed in an order other than that described herein. Furthermore, it should be understood that certain steps may be performed simultaneously, other steps may be added, or certain steps described herein may be omitted. In other words, the descriptions of processes herein are provided for the purpose of illustrating certain embodiments and should not be construed as limiting the scope of the claims.

[0101] Therefore, it should be understood that the above description is illustrative and not restrictive. Many embodiments and uses other than those provided will be apparent from reading the above description. The scope should not be determined by reference to the above description, but rather by reference to the appended claims, together with the full scope of equivalents to which such claims qualify. It is expected and intended that future developments will occur in the art described herein, and that the disclosed systems and methods will be incorporated into such future embodiments. In short, it should be understood that this application is modifiable and can be modified.

[0102] All terms used in the claims are intended to be given their broadest reasonable interpretation and their ordinary meaning as understood by a person familiar with the art described herein, unless expressly indicated otherwise herein. In particular, the use of singular articles such as “a,” “the,” and “said” should be read as describing one or more of the elements indicated, unless the claim expressly states the opposite limitation.

[0103] An abstract of this disclosure is provided to allow readers to quickly confirm the nature of the technical disclosure. The abstract is submitted with the understanding that it is not to be used to interpret or limit the scope or meaning of the claims. In addition, it is found that in the preceding detailed description, various features are grouped together in various embodiments for the purpose of improving the flow of the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the claimed embodiments incorporate more features than are expressly described in each claim. Rather, as reflected in the following claims, the subject matter of the invention consists of fewer features than all the features of a single disclosed embodiment combined. Thus, the following claims are incorporated hereby incorporated into the detailed description, and each claim stands alone as separately claimed subject matter.

[0104] This disclosure includes references to exemplary embodiments, but this specification is not intended to be constrained. Various modifications of the described embodiments, as well as other embodiments within the scope of this disclosure, which are apparent to those skilled in the art to which this disclosure belongs, are considered to be within the principles and scope of this disclosure, as expressed, for example, in the following claims.

[0105] Some embodiments may be implemented as a circuit-based process, including possible implementations on a single integrated circuit.

[0106] Some embodiments may be embodied in the form of methods and apparatus for carrying out those methods. Some embodiments may also be embodied in the form of program code recorded on tangible media such as magnetic recording media, optical recording media, solid-state memory, floppy disks, CD-ROMs, hard drives, or any other non-temporary machine-readable storage media, and when the program code is loaded into a machine such as a computer and executed by the machine, the machine becomes an apparatus for carrying out the patented invention. Some embodiments may also be embodied in the form of program code stored on, for example, a non-temporary machine-readable storage medium, which includes being loaded into and / or executed by a machine, and when the program code is loaded into a machine such as a computer or processor and executed by the machine, the machine becomes an apparatus for carrying out the patented invention. When implemented on a general-purpose processor, the program code segment, in combination with the processor, provides a unique device that operates similarly to a specific logic circuit.

[0107] Unless explicitly stated otherwise, each number and range should be interpreted as an approximation, as if the words “about” or “approximately” preceded the value or range.

[0108] The use of figure numbers and / or reference labels in the claims is intended to identify one or more possible embodiments of the subject matter described in the claims in order to facilitate the interpretation of the claims. Such use should not be construed as necessarily limiting the scope of those claims to the embodiments shown in the corresponding figures.

[0109] The elements in the following method claims are described in a specific order, using corresponding labels if any, but the elements are not necessarily intended to be limited to being implemented in that specific order unless the description of the claim otherwise implies a specific order for implementing some or all of those elements.

[0110] Any reference in this specification to “one embodiment” or “a particular embodiment” means that certain features, structures, or characteristics described in relation to that embodiment may be included in at least one embodiment of this disclosure. The phrase “in a particular embodiment” appearing in various parts of this specification does not necessarily refer to the same embodiment, and separate or alternative embodiments do not necessarily exclude other embodiments. The same applies to the term “implementation.”

[0111] Unless otherwise specified herein, the use of ordinal adjectives such as “first,” “second,” “third,” etc., to refer to one of several similar objects merely indicates that different instances of such similar objects are being referred to, and does not imply that the similar objects referred to in this way must be in a corresponding order or sequence, temporally, spatially, in ranking, or in any other way.

[0112] Unless otherwise specified herein, in addition to its simple meaning, the conjunction "...case" may also be interpreted to mean, or alternatively to mean, "...when," "...on the occasion of," "...in response to determining," or "...in response to detecting," and such interpretation may depend on the corresponding specific context. For example, the phrase "when determined" or "[the stated condition] is detected" may be interpreted to mean "when determined," "in response to having determined," or "[the stated condition or event] is detected," or "[the stated condition or event] has been detected."

[0113] Furthermore, for the purposes of this explanation, the terms “to combine,” “to combine,” “combined,” “connected,” or “connected” refer to any configuration known or to be developed in the Art in which energy is permitted to be transferred between two or more elements, and the intervention of one or more additional elements is possible, though not necessary. Conversely, terms such as “directly combined” or “directly connected” mean that no such additional elements are present.

[0114] As used herein in relation to elements and standards, the term "compatible" means that an element communicates with another element in the manner fully or partially specified by the standard, and that the other element recognizes as sufficiently capable of communicating with the other element in the manner specified by the standard. Compatible elements are not required to operate internally in the manner specified by the standard.

[0115] The functionality of the various elements shown in the diagram, including any functional blocks labeled “Processor” and / or “Controller,” may be provided through the use of dedicated hardware, as well as hardware capable of running software in conjunction with appropriate software. Where provided by a processor, functionality may be provided by a single dedicated processor, a single shared processor, or by multiple individual processors, some of which may be shared. Furthermore, the explicit use of the terms “Processor” or “Controller” should not be interpreted as referring only to, but not limited to, hardware capable of running software, but implicitly may include, digital signal processor (DSP) hardware, network processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), read-only memory (ROM), random-access memory (RAM), and non-volatile storage for storing software. Other conventional and / or custom hardware may also be included. Similarly, any switches shown in the diagram are conceptual only. These functions may be performed through the operation of program logic, through dedicated logic, through the interaction between program control and dedicated logic, or even manually, and the specific technique is selectable by the implementer so that it can be understood more concretely from the context.

[0116] As used in this application, the terms “circuit” and “circuits” may mean one, more, or all of the following: (a) a hardware-only circuit implementation (such as an implementation consisting only of analog and / or digital circuits); (b) (where applicable) (i) a combination of analog and / or digital hardware circuits and software / firmware; and (ii) any part of a hardware processor (including a digital signal processor), software, and memory having software that works together to cause a device such as a mobile phone or server to perform various functions; and (c) a hardware circuit and / or processor, such as a microprocessor or part of a microprocessor, which requires software (e.g., firmware) for operation, but which may not be present when not required for operation. The definition of circuit applies to all uses of this term in this application, including in any claim. As a further example, as used in this application, the term circuit also covers implementations of a hardware circuit or processor (or more processors), or a part of a hardware circuit or processor, and their (or their) accompanying software and / or firmware. The term "circuit" also covers, for example, a baseband integrated circuit or processor integrated circuit for a mobile device, or a similar integrated circuit in a server, cellular network device, or other computing or network device, where applicable to the elements of a particular claim.

[0117] It will be understood by those skilled in the art that any block diagram in this specification represents a conceptual diagram of an exemplary circuit embodying the principles of this disclosure. Similarly, any flowchart, flow diagram, state transition diagram, pseudocode, etc., which are substantially represented in a computer-readable medium, will be understood to represent various processes that may be performed by such a computer or processor, whether or not such a computer or processor is explicitly indicated.

[0118] The “Summary of the Invention” in this specification is intended to present several exemplary embodiments, and additional embodiments are described in the “Modes for Carrying Out the Invention” and / or with reference to one or more drawings. The “Summary of the Invention” is not intended to identify essential elements or features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0119] Various aspects of the present invention can be understood from the following enumerated example embodiments (EEE). [EEE1] A video distribution system that can change the dynamic range of an input image, wherein the distribution system: Memory for storing multiple chroma offset values ​​corresponding to grid points of a 5-dimensional grid; The system comprises a processor that converts the input image having a first dynamic range into a corresponding output image having a larger second dynamic range, wherein the processor: A step of generating an intermediate image having a second dynamic range by reshaping the input image, wherein the reshaping is performed using a reshaping function index map having a corresponding index for each pixel of the intermediate image that identifies the corresponding reshaping function applied to that pixel; The step of estimating the display management metadata value corresponding to the aforementioned intermediate image; The system is configured to perform the steps of: generating the output image by applying a chroma offset to each pixel of the intermediate image, wherein each chroma offset is determined from a plurality of chroma offset values ​​by specifying the grid point using the respective index, the display management metadata value, and the corresponding pixel value of the input image; Video distribution system. [EEE2] The first dynamic range is a standard dynamic range; The second dynamic range is a high dynamic range. The video distribution system according to EEE1. 〔EEE3〕 The video distribution system according to EEE1 or 2, wherein each of the three pixel values is a hue value, a saturation value, and a luminance value of a corresponding pixel of the input image. 〔EEE4〕 The processor is further configured to non-linearly rescale the intensity value of the input image; The video distribution system according to any one of EEE1 to 3, wherein each of the three pixel values is a hue value, a saturation value, and a rescaled intensity value of the corresponding pixel of the input image. The video distribution system according to any one of EEE1 to 3. 〔EEE5〕 The video distribution system according to any one of EEE1 to 4, wherein the video distribution system is configured to generate a display-adapted image by applying display management processing to the output image. 〔EEE6〕 The video distribution system according to any one of EEE1 to 5, wherein the video distribution system includes a video encoder including at least a part of the processor. 〔EEE7〕 The video distribution system according to any one of EEE1 to 6, wherein the plurality of chroma offset values are arranged in the memory in a lookup table addressable using a reshaping function index value, a metadata value, a hue value, a saturation value, and an intensity value. 〔EEE8〕 The video distribution system according to any one of EEE1 to 7, wherein the display management metadata value corresponding to the intermediate image is an average luminance value. 〔EEE9〕 The video distribution system according to any one of EEE1 to 8, wherein the processor is further configured to perform linear interpolation of the chroma offset values ​​in order to determine each of the chroma offsets. [EEE10] A method for changing the dynamic range of an input image, wherein the method is: The process includes transforming the input image having a first dynamic range into a corresponding output image having a larger second dynamic range, the transformation being performed using a plurality of pre-calculated chroma offset values ​​corresponding to grid points of a five-dimensional grid; The aforementioned conversion is: A step of generating an intermediate image having a second dynamic range by reshaping the input image, wherein the reshaping is performed using a reshaping function index map having a corresponding index for each pixel of the intermediate image that identifies the corresponding reshaping function applied to that pixel; The step of estimating the display management metadata value corresponding to the aforementioned intermediate image; A step of generating the output image by applying a chroma offset to each pixel of the intermediate image, wherein each chroma offset is determined from a plurality of chroma offset values ​​by specifying the grid point using the respective index, the display management metadata value, and the corresponding pixel value of the input image. method. [EEE11] The process further includes the step of non-linearly rescaling the intensity values ​​of the input image, The three pixel values ​​mentioned above are the hue value, saturation value, and rescaled intensity value of the corresponding pixel in the input image. Methods described in EEE10. [EEE12] The method according to EEE10 or 11, further comprising the step of generating a display-adapted image by applying display management processing to the output image. [EEE13] The method according to any one of EEE10 to 12, wherein the display management metadata value corresponding to the intermediate image is the average luminance value. [EEE14] The method according to any one of EEE10 to 13, wherein the conversion further comprises performing linear interpolation of the chroma offset values ​​to determine each of the chroma offsets. [EEE15] The method according to any one of EEE10 to 14, wherein the plurality of chroma offset values ​​are placed in a lookup table that can be addressed using a reshaping function index value, metadata value, hue value, saturation value, and intensity value. [EEE16] A non-temporary computer-readable medium that, when executed by an electronic processor, stores instructions causing the electronic processor to perform an operation including the method described in any one of EEE10 to 15. [EEE17] A method for generating multiple chroma offset values ​​for performing color shift correction in an output video generated by changing the dynamic range of an input image, the method being: Define a 5-dimensional grid having first, second, third, fourth, and fifth dimensions, each representing a reshaping function index value, metadata value, hue value, saturation value, and intensity value, respectively; Define a cost function to quantify at least one cost for the hue difference between the input image and the output image; For each grid point of the 5-dimensional grid, the cost function is iteratively minimized to determine each set of chroma offset values; This includes arranging each set of chroma offset values ​​in an electronic lookup table addressable using discrete sets of values ​​corresponding to the first, second, third, fourth, and fifth dimensions, method. [EEE18] The method according to EEE17, wherein defining the cost function includes including one or more regularization terms in the cost function that are configured to keep the sequential iterative minimization within a valid range. [EEE19] The method according to EEE17 or 18, wherein the sequential iterative minimization is performed within a local range of the parameter that is narrower than the entire range of the parameter. [EEE20] The method according to any one of EEE17 to 19, wherein the sequential iterative minimization is performed using a variable step size. [EEE21] For the tth iteration of the sequential iterative minimization, the best chroma offset r t CbCr but

number

Claims

1. A video distribution system capable of changing the dynamic range of an input image, wherein the distribution system: A memory that stores multiple chroma offset values ​​corresponding to grid points of a 5-dimensional grid; The system comprises a processor that converts the input image having a first dynamic range into a corresponding output image having a larger second dynamic range, wherein the processor: A step of generating an intermediate image having a second dynamic range by reshaping the input image, wherein the reshaping is performed using a reshaping function index map having a corresponding index for each pixel of the intermediate image that identifies the corresponding reshaping function applied to that pixel; The step of estimating the display management metadata value corresponding to the aforementioned intermediate image; The system is configured to perform the following steps: a step of generating the output image by applying a chroma offset to each pixel of the intermediate image, wherein each chroma offset is determined from a plurality of chroma offset values ​​by specifying the grid point using the respective index, the display management metadata value, and the corresponding pixel value of the input image; Video distribution system.

2. The first dynamic range is the standard dynamic range; The second dynamic range is a high dynamic range. The video distribution system according to claim 1.

3. The video distribution system according to claim 1, wherein each of the three pixel values ​​is the hue value, saturation value, and luminance value of the corresponding pixel in the input image.

4. The processor is further configured to non-linearly rescale the intensity values ​​of the input image; The three pixel values ​​mentioned above are the hue value, saturation value, and rescaled intensity value of the corresponding pixel in the input image. The video distribution system according to claim 1.

5. The video distribution system according to claim 1, wherein the video distribution system is configured to generate a display-adapted image by applying display management processing to the output image.

6. The video distribution system according to claim 1, wherein the video distribution system comprises a video encoder including at least a portion of the processor.

7. The video distribution system according to any one of claims 1 to 6, wherein the plurality of chroma offset values ​​are arranged in memory in a lookup table that can be addressed using reshaping function index values, metadata values, hue values, saturation values, and intensity values.

8. The video distribution system according to claim 1, wherein the display management metadata value corresponding to the intermediate image is the average luminance value.

9. The video distribution system according to claim 1, wherein the processor is further configured to perform linear interpolation of the chroma offset values ​​in order to determine each of the chroma offsets.

10. A method for changing the dynamic range of an input image, wherein the method is: The process includes transforming the input image having a first dynamic range into a corresponding output image having a larger second dynamic range, the transformation being performed using a plurality of pre-calculated chroma offset values ​​corresponding to grid points of a five-dimensional grid; The aforementioned conversion is: A step of generating an intermediate image having a second dynamic range by reshaping the input image, wherein the reshaping is performed using a reshaping function index map having a corresponding index for each pixel of the intermediate image that identifies the corresponding reshaping function applied to that pixel; The step of estimating the display management metadata value corresponding to the aforementioned intermediate image; A step of generating the output image by applying a chroma offset to each pixel of the intermediate image, wherein each chroma offset is determined from a plurality of chroma offset values ​​by specifying the grid point using the respective index, the display management metadata value, and the corresponding pixel value of the input image. method.

11. The process further includes the step of non-linearly rescaling the intensity values ​​of the input image, The three pixel values ​​mentioned above are the hue value, saturation value, and rescaled intensity value of the corresponding pixel in the input image. The method according to claim 10.

12. The method according to claim 10, further comprising the step of generating a display-adapted image by applying display management processing to the output image.

13. The method according to claim 10, wherein the display management metadata value corresponding to the intermediate image is an average luminance value.

14. The method according to claim 10, wherein the conversion further comprises performing linear interpolation of the chroma offset values ​​to determine each of the chroma offsets.

15. The method according to any one of claims 10 to 14, wherein the plurality of chroma offset values ​​are placed in a lookup table that can be addressed using a reshaping function index value, metadata value, hue value, saturation value, and intensity value.

Images