Video delivery system capable of dynamic range variation

Abstract

A video delivery system includes: a memory for storing a plurality of chroma offset values corresponding to grid points of a five-dimensional grid; and a processor to convert the input SDR image into a corresponding HDR output image, the processor configured to: generate an intermediate HDR image by shaping the input image, the shaping being performed using a shaping function index map having, for each pixel of the intermediate image, a respective index identifying a corresponding shaping function applied to that pixel; estimating a display-management metadata value corresponding to the intermediate image; and generating an output image by applying a respective chroma offset to each pixel of the intermediate image, the respective chroma offset being determined from the plurality of chroma offset values by addressing the grid points using the respective index, the display-management metadata value, and the three respective pixel values of the corresponding pixel of the input image.

Description

[0001] 1. Cross-reference to related applications

[0002] This application claims priority to European Patent Application 22 178 928.2 and U.S. Provisional Patent Application Serial No. 63 / 351,855, both filed on June 14, 2022, both of which are incorporated herein by reference in their entirety. 2. Technical field

[0003] Various exemplary embodiments relate to image processing operations and, more particularly, but not exclusively, to video codecs. 3. Background technology

[0004] This section introduces aspects that may help promote a better understanding of the present disclosure. Therefore, the statements in this section should be read in this light and should not be understood as an admission that there is something in the prior art or not in the prior art.

[0005] As used herein, the term "dynamic range" (DR) can relate to the ability of the human visual system (HVS) to perceive a range of intensities (e.g., luminance (luma)) in an image, e.g., from the darkest black (dark) to the brightest white (highlight). In this sense, DR relates to "scene-referred" intensities. DR can also relate to the ability of a display device to adequately or approximately render a range of intensities of a particular width. In this sense, DR relates to "display-referred" intensities. Unless a particular meaning is explicitly specified at any point in the specification herein as having a special meaning, it should be inferred that the term can be used in either sense, e.g., interchangeably.

[0006] As used herein, the term "high dynamic range" (HDR) refers to a DR width that spans 14-15 or more orders of magnitude across the human visual system (HVS). In practice, the DR over a wide range of intensities that humans may simultaneously perceive may be truncated to some extent compared to HDR. As used herein, the terms "enhanced dynamic range" (EDR) or "visual dynamic range" (VDR) may be used, either individually or interchangeably, to relate to the DR that is perceptible in a scene or image by the human visual system, including eye movement, allowing for some light adaptation variations across scenes or images. In this article, EDR may refer to a DR that spans 5 to 6 orders of magnitude. While the EDR may be somewhat narrower compared to real scene referenced HDR, it still represents a wide DR width and may also be referred to as HDR.

[0007] In practice, an image includes one or more color components (e.g., luminance Y and chrominance Cb and Cr) of a color space, where each color component is represented with n-bit precision per pixel (e.g., n=8). Using nonlinear luminance coding (e.g., gamma coding), images with n≤8 (e.g., 24-bit color JPEG images) are considered to be images with a standard dynamic range (SDR), while images with n>8 can be considered to be images with an enhanced dynamic range.

[0008] The reference electro-optical transfer function (EOTF) for a given display characterizes the relationship between the color values ​​(e.g., brightness) of an input video signal and the output screen color values ​​(e.g., screen brightness) produced by the display. For example, ITU Rec. ITU-R BT.1886, "Reference electro-optical transfer function for flat panel displays used in HDTV studio production" (March 2011), which is incorporated herein by reference in its entirety, defines a reference EOTF for flat panel displays. Given a video stream, information about its EOTF can be embedded in the bitstream as (image) metadata. The term "metadata" herein refers to any auxiliary information transmitted as part of the encoded bitstream. Such metadata can be used to assist a decoder in presenting a decoded image, and may include, but is not limited to, color space information or color gamut information, reference display parameters, and auxiliary signal parameters, such as those further described herein.

[0009] As used herein, the term "PQ" refers to perceived brightness amplitude quantization. The human visual system reacts to increasing brightness levels in a very nonlinear manner. The ability of humans to see a stimulus is affected by the brightness of the stimulus, the size of the stimulus, the spatial frequencies that make up the stimulus, and the brightness level to which the human eye adapts at a particular moment of viewing the stimulus. In some cases, a PQ function can map a linear input grayscale level to an output grayscale level that better matches the contrast sensitivity threshold in the human visual system. An example PQ mapping function is described in SMPTE ST 2084:2014 "High Dynamic Range EOTF of Mastering Reference Displays" (hereinafter referred to as "SMPTE"), which is incorporated herein by reference in its entirety.

[0010] Compared to EDR (or HDR), the supported brightness is 200 to 1,000cd / m 2A display with a resolution of 1000 nits or nits represents a lower dynamic range (LDR), also known as a standard dynamic range (SDR). EDR content can be displayed on an EDR display that supports a higher dynamic range (e.g., from 1,000 nits to 5,000 nits or more). Such a display may be defined using an alternative EOTF that supports high brightness capabilities (e.g., 0 to 10,000 nits or more). Examples of such EOTFs are defined in SMPTE 2084 and Rec. ITU-R BT.2100 "Image parameter values ​​for high dynamic range television for use in production and international programme exchange" (06 / 2017), which are incorporated herein by reference in their entirety.

[0011] WO 2022 / 072884 A1 discloses an adaptive local shaping method for SDR to HDR upconversion. Using the brightness codeword in the input image, a global index value is generated for selecting a global shaping function for an input image with a relatively low dynamic range. The input image is image filtered to generate a filtered image. The filtered value of the filtered image provides a measure of the local brightness level in the input image. A local index value is generated for selecting a specific local shaping function for the input image using the global index value and the filtered value of the filtered image. By shaping the input image with a specific local shaping function selected using the local index value, a shaped image with a relatively high dynamic range is generated.

[0012] WO 2017 / 059415 A1 discloses a method for color correction in high dynamic range video (HDR) using a 2D lookup table (LUT). After decoding an HDR video signal, color correction can be applied in a decoder. For example, color correction can be applied before, during, or after chroma upsampling of the HDR video signal. The 2D LUT may include a representation of a color space of the HDR video signal. Color correction may include applying triangular interpolation to sample values ​​of color components of the color space. The 2D LUT may be estimated by an encoder and transmitted to a decoder using a signal. The encoder may decide to reuse a previously signaled 2D LUT or to use a new 2D LUT. Summary of the invention

[0013] Various embodiments of a video transmission system capable of performing color shift correction in an HDR image generated from an SDR image are disclosed herein. In an exemplary embodiment, color shift correction is performed using a pre-computed lookup table (LUT) representing a five-dimensional (5-D) grid, the LUT being addressable using an integer function index value, a metadata value, a hue value, a saturation value, and an intensity value. Linear interpolation can be used to obtain a chromaticity offset value for any point in the corresponding 5-D parameter space that is not a grid point. Also disclosed herein is an example iterative minimization method using a properly constructed cost function that can be used to populate the LUT. Beneficially, the disclosed exemplary embodiments of color shift correction are compatible with existing display-management functionality and do not require any modification thereto.

[0014] According to an exemplary embodiment, a video transmission system capable of changing the dynamic range of an input image is provided, the transmission system comprising: a memory for storing a plurality of chroma offset values ​​corresponding to grid points of a five-dimensional grid; and a processor for converting an input image having a first dynamic range into a corresponding output image having a larger second dynamic range, the processor being configured to: generate an intermediate image having a second dynamic range by shaping the input image, the shaping being performed using a shaping function index map, the shaping function index map having, for each pixel of the intermediate image, a corresponding index identifying a corresponding shaping function applied to the pixel; estimate a display-management metadata value corresponding to the intermediate image; and generate an output image by applying a corresponding chroma offset to each pixel of the intermediate image, the corresponding chroma offset being determined from the plurality of chroma offset values ​​by addressing the grid point using a corresponding index, a display-management metadata value, and three corresponding pixel values ​​of the corresponding pixel of the input image.

[0015] According to another exemplary embodiment, a method for changing the dynamic range of an input image is provided, the method comprising: converting an input image having a first dynamic range into a corresponding output image having a larger second dynamic range, performing the conversion using a plurality of pre-calculated chroma offset values ​​corresponding to grid points of a five-dimensional grid; and, wherein the conversion comprises: generating an intermediate image having the second dynamic range by shaping the input image, the shaping being performed using a shaping function index map having a corresponding index identifying a corresponding shaping function applied to each pixel of the intermediate image; estimating a display-management metadata value corresponding to the intermediate image; and generating an output image by applying a corresponding chroma offset to each pixel of the intermediate image, the corresponding chroma offset being determined from the plurality of chroma offset values ​​by addressing the grid point using a corresponding index, a display-management metadata value, and three corresponding pixel values ​​of the corresponding pixel of the input image.

[0016] According to another exemplary embodiment, a non-transitory computer-readable medium storing instructions is provided, which, when executed by an electronic processor, causes the electronic processor to perform operations including a method of changing the dynamic range of an input image, the method comprising: converting an input image having a first dynamic range to a corresponding output image having a larger second dynamic range, performing the conversion using a plurality of pre-calculated chroma offset values ​​corresponding to grid points of a five-dimensional grid; wherein the conversion comprises: generating an intermediate image having the second dynamic range by shaping the input image, the shaping being performed using a shaping function index map having a corresponding index identifying a corresponding shaping function applied to each pixel of the intermediate image for each pixel; estimating a display-management metadata value corresponding to the intermediate image; and generating an output image by applying a corresponding chroma offset to each pixel of the intermediate image, the corresponding chroma offset being determined from the plurality of chroma offset values ​​by addressing the grid point using a corresponding index, a display-management metadata value, and three corresponding pixel values ​​of the corresponding pixel of the input image.

[0017] According to another exemplary embodiment, a method for generating multiple chromaticity offset values ​​for performing color shift correction in an output image generated by changing the dynamic range of an input image is provided, the method comprising: defining a five-dimensional grid, whose first, second, third, fourth and fifth dimensions respectively represent shaping function index values, metadata values, hue values, saturation values ​​and intensity values; defining a cost function for at least quantifying the cost of the hue difference between the input image and the output image; for each grid point of the five-dimensional grid, performing iterative minimization of the cost function to determine a corresponding set of chromaticity offset values; and arranging the corresponding set of chromaticity offset values ​​in an electronic lookup table that can be addressed using discrete value sets corresponding to the first, second, third, fourth and fifth dimensions. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Other aspects, features, and benefits of the various disclosed embodiments will become more fully apparent from the following detailed description and accompanying drawings, by way of example, in which:

[0019] Figure 1 An example process for a video delivery pipeline is depicted.

[0020] Figure 2 Describes an embodiment of the present invention. Figure 1 An exemplary process for use in a video delivery pipeline.

[0021] Figures 3A-3C The pictorial diagram illustrates an example of a method for Figure 2 The indexing scheme used in the process.

[0022] Figures 4A-4C The pictorial diagram illustrates that the computing according to the embodiment can be Figure 2 Example of interpolation weights used in the process of .

[0023] Figure 5A-5B An example effect of scaling the probability distribution function of brightness Y according to an embodiment is graphically illustrated.

[0024] Figure 6A-6B Diagrammatically illustrated are example grids for YCbCr and RGB color spaces, respectively, according to various embodiments.

[0025] Figure 7 is a diagram illustrating a method for filling a Figure 2 Flowchart of the method of processing the 5-D LUT.

[0026] Figure 8 is a diagram illustrating a method for Figure 7 Flowchart of iterative minimization process using the method. DETAILED DESCRIPTION

[0027] The present disclosure and its aspects may be embodied in a variety of forms, including hardware, devices or circuits controlled by computer-implemented methods, as well as computer program products, computer systems and networks, user interfaces and application programming interfaces; as well as hardware-implemented methods, signal processing circuits, memory arrays, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), etc. The above content is intended only to give a general idea of ​​the various aspects of the present disclosure and does not limit the scope of the present disclosure in any way.

[0028] In the following description, in order to provide an understanding of one or more aspects of the present disclosure, many details are listed, such as optical device configuration, timing, operation, etc. It is obvious to those skilled in the art that these specific details are merely exemplary and are not intended to limit the scope of the present application.

[0029] In addition, although the present disclosure focuses primarily on examples of using various circuits in digital projection systems, it will be understood that these are merely examples. It will be further understood that the disclosed systems and methods can be used in any device that requires projecting light, such as movie theaters, consumer and other commercial projection systems, head-up displays, and virtual reality displays. The disclosed systems and methods can be implemented in additional display devices, such as with OLED displays, LCD displays, quantum dot displays, and the like.

[0030] Many consumer desktop monitors support 200 to 300 cd / m 2Brightness is measured in nits or nits. Many consumer high-definition televisions (HDTVs) range from 300 to 500 nits, with newer models reaching 1000 nits (cd / m2). The availability of HDR content continues to increase due to advances in image capture equipment (e.g., cameras) and HDR displays (e.g., the PRM-4200 professional reference monitor from Dolby Laboratories), and HDR content can be color graded and displayed on HDR displays that support a higher dynamic range (e.g., from 1000 nits to 5000 nits or more).

[0031] Some embodiments may benefit from at least some of the features disclosed in the international patent application “ADAPTIVE LOCAL RESHAPING FOR SDR-TO-HDR UP-CONVERSION”, PCT / US2021 / 053241, filed by TW. Huang et al. on October 1, 2021, which is incorporated herein by reference in its entirety.

[0032] In this document, the term "metadata" refers to any auxiliary information that is transmitted as part of the coded bitstream and assists the decoder in rendering the corresponding image(s). For television broadcasting and video streaming, video metadata can be used to provide side information about specific video and audio streams or files. Metadata can be either embedded directly in the video or included as a separate file in a container, such as MP4 or MKV. Metadata can include information about the entire video stream or file or about specific video frames. Metadata is collected by cameras, encoders, and other video processing elements (see, for example, Figure 1 The metadata created by the video processor 115, 120) may include, but is not limited to, timestamp, video resolution, digital film grain parameters, color space information or color gamut information, reference display parameters, auxiliary signal parameters, file size, closed captions, audio language, advertising insertion points, color space, error messages, etc. Other examples of metadata related to the disclosed embodiments are described below.

[0033] In some embodiments disclosed below, image metadata includes L1 metadata. As used herein, the term "L1 metadata" refers to one or more of the minimum (L1-min) brightness value, the middle (L1-mid) brightness value, and the maximum (L1-max) brightness value associated with a specific portion of video content, such as an input frame or image. The L1 metadata is associated with a video signal. In order to generate L1 metadata, the video content is analyzed frame by frame at the pixel level, preferably at the encoding end. Alternatively, the analysis can be performed at the decoding end. The analysis describes the distribution of brightness values ​​on a defined portion of the video content covered by the analysis process (analysis pass), and the defined portion of the video content is, for example, a single frame or a series of frames (such as a scene). L1 metadata can be calculated in an analysis process covering a single video frame and / or a series of frames (such as a scene). L1 metadata may include various values ​​derived during the analysis process, which together form L1 metadata, and the L1 metadata is associated with the corresponding portion of the video content for which the L1 metadata is calculated, and is associated with the video signal. Such L1 metadata may include at least one of (i) an L1-min value representing the lowest black level in the corresponding portion of the video content, (ii) an L1-mid value representing the average brightness level across the corresponding portion of the video content, and (iii) an L1-max value representing the highest brightness level in the corresponding portion of the video content. The L1 metadata may be generated for each video frame and / or each scene encoded in the video signal and attached to these video frames and / or scenes. The L1 metadata may also be generated for a region of an image, and such L1 metadata may be referred to as a local L1 value. The L1 metadata may be calculated by converting the RGB data to a luminance-chrominance format (e.g., YCbCr) and then calculating one or more of a minimum value, a middle value (average value), and a maximum value on the Y plane, or may be calculated directly in the RGB space.

[0034] In some embodiments, when only active areas are considered (e.g., by excluding grayscale or blackscale bars, adding horizontal letterbox bars, etc.), the L1-min value may represent the minimum value of the PQ-encoded min(RGB) values ​​of the corresponding portion of the video content (e.g., a video frame or image), where min(RGB) represents the minimum value of the color component values ​​{R, G, B} of the pixel. The L1-mid and L1-max values ​​may also be calculated in a similar manner. Specifically, in an exemplary embodiment, L1-mid may represent the average value of the PQ-encoded max(RGB) values ​​of the image, and L1-max may represent the maximum value of the PQ-encoded max(RGB) values ​​of the image, where max(RGB) represents the maximum value of the color component values ​​{R, G, B} of the pixel. In some embodiments, the L1 metadata may be normalized to be in the range [0,1].

[0035] Video encoding according to an exemplary embodiment

[0036] Figure 1 An example process of a video delivery pipeline (100) according to an embodiment is depicted, showing various stages from video capture to video content display. A sequence of video frames (102) may be captured or generated using an image generation block (105). The video frames (102) may be captured digitally (e.g., by a digital video camera) or generated by a computer (e.g., using computer animation) to provide video data (107). Alternatively, the video frames (102) may be captured on film by a film camera. The film may then be converted to a digital format to provide video data (107).

[0037] During the production stage (110), the video data (107) may be edited to provide a video production stream (112). The data of the video production stream (112) may then be provided to a processor (or one or more processors, such as a central processing unit (CPU)) at a post-production block (115) for post-production editing. Post-production editing at this block (115) may include, for example, adjusting or modifying the color or brightness of specific areas of an image to enhance image quality or achieve a specific look for the image, in accordance with the creative intent of the video creator. This portion of post-production editing is sometimes referred to as "colortiming" or "color grading." Other edits (e.g., scene selection and sequencing, image cropping, addition of computer-generated visual effects, etc.) may be performed at block (115) to produce a "final" version (117) of the production for distribution. During post-production editing (115), the video image may be viewed on a reference display (125).

[0038] After post-production (115), the final version (117) of the video data may be transmitted to an encoding block (120) for downstream transmission to a decoding and playback device, such as a television, set-top box, movie theater, etc. In some embodiments, the encoding block (120) may include an audio and video encoder, such as an encoder defined by ATSC, DVB, DVD, Blu-ray, and other transmission formats, to generate an encoded bitstream (122). Some of the methods described below may be performed by a corresponding processor at the encoding block (120). For example, as described in more detail below, the encoding block (120) may be configured to perform local shaping and color shift correction for SDR to HDR. In the receiver, the encoded bitstream (122) is decoded by a decoding unit (130) to generate a corresponding decoded signal (132) representing a replica or approximation of the signal (117). The receiver may be connected to a target display (140), which may have characteristics that are somewhat or completely different from those of the reference display (125). In this case, a display management (DM) block (135) may be used to map the decoded signal (132) to the characteristics of the target display (140) by generating a display mapping signal (137). Some of the methods described below may be performed by the decoding unit (130) and / or the display-management block (135). Depending on the embodiment, the decoding unit (130) and the display management block (135) may include separate processors, or may be based on a single integrated processing unit.

[0039] In an exemplary embodiment, the SDR to HDR local shaping may take a three-channel (Y, Cb, Cr) input SDR image and predict a three-channel (Y, Cb, Cr) output HDR image using a pre-trained shaping function. In order to view the HDR image on a target display (140), the DM block (135) may further process the HDR image based on corresponding metadata to generate a display mapping signal (137) representing the DM image according to a target display brightness.

[0040] In most cases, the chromaticity of the HDR image and the DM image is equal to or close to the chromaticity of the SDR image, although the brightness may differ due to upconversion and enhancement. However, in some cases, the difference in chromaticity may become obvious, which is called the color shift from SDR to HDR. The color shift from SDR to HDR can be caused by several reasons. For example, since the pre-trained shaping functions and DM are usually trained on natural images, this training can cause larger errors and / or color shifts on colors that are less common in natural images. In addition, the DM block (135) can perform cropping on pixels near the color space boundary, thereby amplifying existing color shifts or introducing new color shifts.

[0041] In an exemplary embodiment, the DM block (135) is not modified, for example, it can remain the same as in the legacy video delivery pipeline. Instead, an exemplary embodiment of the proposed color shift correction framework is designed to correct the end-to-end color shift between the SDR image and the final DM image by adding chroma offsets to the Cb and Cr channels of the HDR image. Since the end-to-end process from SDR image to DM image is typically highly non-linear, the required chroma offset is determined using a 5-D lookup table (LUT) on a five-dimensional (5-D) grid, where the 5 dimensions are the SDR pixel value (three dimensions), the shaping function index, and the DM metadata L1-mid. The resulting 5-D space is referred to herein as the HSWLM space, where H represents hue, S represents saturation, W represents scaled intensity value, L represents local shaping function index, and M represents metadata. The 5-D LUT can be filled using a suitable cost function in an iterative minimization manner, for example, as described below with reference to Figures 7 and 8 Further detailed description.

[0042] Color shift correction

[0043] Figure 2 An example process (200) of a video delivery pipeline (100) according to an embodiment is depicted. The process (200) may generally receive an input SDR image (117) and may generate a corresponding output DM image (137) (see also Figure 1 During this process (200), a chroma offset may be added to the initial HDR image (216), so that possible color shifts due to the reasons indicated above may be mitigated.

[0044] The process (200) includes a shaping block (210) and a chroma offset processing block (220). The shaping block (210) is configured to generate an initial HDR image (216) based on an input SDR image (117). The shaping block (210) includes processing to generate a shaping function index map (212) and apply a pre-trained shaping function (214) to the input SDR image (117). In some embodiments, the shaping block (210) can be implemented as disclosed in the international patent application PCT / US2021 / 053241 cited above.

[0045] In an exemplary embodiment, a shaping block (210) includes local shaping from SDR to HDR, wherein a three-channel (Y, Cb, Cr) input SDR image (117) and a set of pre-trained shaping functions (214) are used to predict a three-channel (Y, Cb, Cr) output HDR image (216). A shaping function index map (212) is created to indicate which shaping functions (214) are used for different pixels.

[0046] The Y, Cb and Cr channels of the i-th pixel in the input SDR image S(117) are expressed as And the shaping function index of the i-th pixel in the shaping function index map L(212) is l i The shaping block (210) is locally shaped from SDR to HDR to generate an initial HDR image V (init) The i-th pixel in (216) It can be expressed as:

[0047]

[0048] in For the first i It is noted that various embodiments are not limited to certain shaping functions. For example, the function It can be any suitable shaping function, such as a function implemented by a LUT and a multivariate multiple regression function.

[0049] As a non-limiting example, the processing of a single image / frame is described below. Based on the description provided, one of ordinary skill in the relevant art will be able to implement corresponding processing of multiple frames without any undue experimentation, for example, because the disclosed processing does not generally or explicitly rely on temporal information. For the purpose of simplifying the description and without any implied limitations, the description is provided for normalized pixel values, i.e., values ​​in the range [0,1]. Furthermore, the description assumes that conversions between color spaces can be performed when necessary. For example, if one has a pixel in the YCbCr color space , it is assumed that in the RGB color space there is also A person skilled in the art will readily understand how to perform such conversions between color spaces.

[0050] In order to view an HDR image, such as an initial HDR image (216) or an output HDR image (240) on a target display (140), processing by a DM block (135) is typically applied to the HDR image, the result of the DM processing being an output DM image (137). Such DM processing may be controlled by the metadata L1-min, L1-mid and L1-max mentioned above, which represent the minimum, average and maximum values ​​of the RGB channels of the HDR image, respectively. However, in the example SDR to HDR upconversion scenario, at least in some cases, L1-min and L1-max may be set to constant values. Therefore, some embodiments may rely solely on the L1-mid value.

[0051] When the chroma offset processing block (220) does not exist, the initial HDR image (216) is directly applied to the DM block (135). (init) The R, G, and B channels of the i-th pixel in (216) are expressed as The DM metadata L1-mid of the initial HDR image can be calculated as:

[0052]

[0053] Where N is the number of pixels. The output DM image generated from the initial HDR image (216) is represented as Then you can The i-th pixel in

[0054] Among them, f (DM) Denotes a DM function. In various embodiments, the function f (DM) May be the same as the function in the legacy video delivery pipeline.In general, the various embodiments are not limited to only certain kinds of DM functionality.

[0055] In some cases, and can have similar chromaticity. However, in some other cases, and The chromaticity differences between the two, i.e., color shifts, can be noticeable to a viewer. Therefore, the processing implemented in the chromaticity shift processing block (220) is intended to significantly reduce or completely eliminate such differences. In an exemplary embodiment, the chromaticity shift processing block (220) uses a 5-D grid and a corresponding 5-D LUT (228).

[0056] By in the corresponding multidimensional space By sampling along each dimension, a grid of dimension D (e.g., D=5) can be constructed. For computational efficiency, a uniformly spaced grid can be used, where the coordinate values ​​of the same dimension are uniformly sampled. Assume that for the dth dimension, N d The value is from the initial value p d Start with interval b d is sampled. Then, the i-th d The samples can be calculated as p d +i d b d , where i d =0, 1, ..., N d -1 and d = 0, 1, ..., D-1. This sampling defines a grid X such that the grids with indices i = (i0, i1, ..., i D-1) of the grid points X i is expressed as:

[0057]

[0058] In general, a LUT, such as a 5-D LUT (228), can be constructed to model an arbitrary function φ on a grid. More specifically, a LUT function Φ on a grid X can be defined such that for each input grid point X i , the LUT can return the corresponding output value Φ i =φ(X i ). For example, in an embodiment, the function Φ may be a LUT representing a function φ, which takes as input an SDR pixel value, a shaping function index, and DM metadata L1-mid, and then provides a chroma shift as an output. In various embodiments, the outputs of the functions φ and Φ may be in scalar form or in vector form. In the non-limiting example described below, the output is a 2-D vector of chroma shifts in the Cb and Cr channels.

[0059] In an exemplary embodiment, to handle input values ​​that do not fall on the grid, linear interpolation may be used. For example, such linear interpolation may follow a definition similar to that used in conventional bilinear or trilinear interpolation, where the output values ​​are based on the corresponding linear interpolation in each dimension. For ease of computation, exemplary embodiments may rely on normalized grid coordinates. This approach provides a normalized grid starting at 0 and with uniform spacing. Given an input x=(x0, x1, ..., x D-1 ), the corresponding normalized grid coordinates can be defined by shifting and scaling as So that:

[0060]

[0061] From equation (5), it can be seen that the grid converted to normalized grid coordinates It starts at 0 and has a spacing of 1 in all dimensions. That is, the grid It can be expressed mathematically as where i is an integer counter.

[0062] It can also be noted that the normalized grid Formed (N0-1)×(N1-1)×...×(N D-1 -1) unit hypercubes. These hypercubes can be indexed in the same way as grid points. For example, The hypercube of can be the hypercube whose lower left vertex (i.e. the vertex closest to the origin) is i. Therefore, the hypercube of 2 DThe number of vertices is i+{0, 1} D .

[0063] Figures 3A-3C pictorially illustrates an example 2-D grid of size 4×4 as indexed by the above described scheme according to an embodiment More specifically, Figure 3A Two grid dimensions are illustrated, which are denoted as dimension 0 and dimension 1 respectively. Figure 3B An index is illustrated, where the grid points (shown as nodes) are indexed as described above. Figure 3C An index is illustrated wherein the hypercube is indexed as described above.

[0064] Since linear interpolation is linear in each dimension, normalized coordinates can be used and the normalized grid To perform this linear interpolation, rather than using the corresponding non-normalized entities x and X. In an exemplary embodiment, the processing step of performing linear interpolation may include looking up the value The substep of the unit hypercube that falls within it, and then using the vertices of the unit hypercube with The distance between them is used to perform interpolation. The index of the unit hypercube is represented as The clipping function clip3 is defined as clip3(x, a, b) = min(max(x, a), b). It can be noted that when When on the boundary between hypercubes, the specific is assigned to the hypercube that is located in a direction pointing away from the origin.

[0065] The interpolation result of input x is expressed as Based on the definition of linear interpolation above, the interpolation result is It can be expressed as:

[0066]

[0067] in, yes The interpolation weight of i′ is expressed as It can be expressed as:

[0068]

[0069] in

[0070]

[0071] It can be noted that the normalization factor in this linear interpolation has been processed because the normalized grid has a spacing of 1. In an exemplary embodiment, when the grid is relatively dense, the interpolation result It can usually be very close to the actual function output φ(x).

[0072] Since the weights of the above linear interpolation depend only on the and The distance between the two, so for computational efficiency, the weights of multiple possible distances can be pre-computed for lookup at runtime. For example, the unit hypercube can be quantized and the corresponding interpolation weights can be stored in a LUT. When the quantization is relatively dense, the output of the LUT will usually be relatively close to the corresponding non-quantized interpolation result.

[0073] Consider a unit hypercube located at the origin. Such a unit hypercube has 2 D Vertices In an exemplary embodiment, vertices may be indexed by their coordinates, i.e. Assume that the unit hypercube is uniformly quantized to M in the closed interval [0,1] for each dimension d=0,1,...,D-1 d point, then a quantization grid Q can be defined so that the quantization grid Q has an index j=(j0, j1, ...j D-1 ) is expressed as:

[0074]

[0075] in

[0076] For node Q j , the linear interpolation weight of vertex k can be calculated as:

[0077]

[0078] Using equation (9), all possible weights W can be pre-computed j，k , and save these weights in a file with size M0×M1×...×M D-1 The output of LUTW can include all 2 D Linear interpolation weights for each vertex.

[0079] Figures 4A-4C An example of calculating interpolation weights in a 2-D unit hypercube with a quantization grid Q of size 4×5 according to an embodiment is illustrated. More specifically, Figure 4A The two dimensions of the unit hypercube are shown, denoted as dimension 0 and dimension 1. Figure 4BThe quantized grid Q for the two dimensions of the unit hypercube is shown, with the corresponding four vertices explicitly shown. The index of . Figure 4C Shows Figure 4B Node Q in the unit hypercube shown 2，1 The coordinates of the four vertices and the corresponding calculated weights.

[0080] To use LUTW, for an input q=(q0, q1, ..., q D-1 ), the input is mapped to the nearest node Q in the direction towards the origin j ,in:

[0081]

[0082] For any input and Mapping operations can include converting the input and The hypercube where the hypercube is located is transformed into a unit hypercube located at the origin, so that and Equation (6) and equation (7) can be transformed into the following equation:

[0083]

[0084] in

[0085]

[0086] In an exemplary embodiment, a 5-D grid X may be defined in the HSWLM space mentioned above. In order to have better numerical stability and interpolation quality, the 5-D grid X may be aligned with the boundary of the valid input parameter space. Such alignment generally helps to perform proper interpolation for input points located close to the boundary. For the shaping function index and DM metadata L1-mid, their original values ​​may be used for the grid X, since these values ​​are independent (decoupled) from other dimensions. For example, changing the shaping function index and DM metadata L1-mid will not invalidate the originally valid input. On the other hand, for SDR pixel values, if the YCbCr color space is used, not all One way to prevent invalid input in the YCbCr color space is to cross-reference the YCbCr input to other color spaces. For example, in the RGB or HSV color spaces, all combinations of [0,1] values ​​are valid (see also Figure 6B ). Herein, RGB stands for Red, Green, and Blue; and HSV stands for Hue, Saturation, and Value.

[0087] In an exemplary embodiment, the scaled HSV color space used for grid creation can be designed so that the density of the grid is proportional to the perceived hue difference. For example, the V component can be scaled in a nonlinear manner so that the density of the grid remains approximately the same at different brightness values. The H, S, and V channels of the i-th pixel in the input SDR image S (117) are represented as Then, the HSW color space can be defined such that in:

[0088]

[0089] and

[0090]

[0091] Figure 5A-5B The effect of V to W scaling on the probability distribution function (PDF) of luminance Y according to an embodiment is graphically illustrated. For example, the scaling may be performed in the HSW channel block (224) of the process (200). In this specific example, Figure 5A Graphically illustrates the PDF as a function of luminance Y for a grid created in HSV color space and then transformed to YCbCr color space. Figure 5B The PDF as a function of luminance Y for a grid created in HSW color space and then transformed to YCbCr color space is graphically illustrated. A comparison of the two PDFs shows that Figure 5B PDF Ratio Figure 5A The PDF of is more uniform. In this paper, the luminance Y is within the typical SMPTE range, where SMPTE stands for Society of Motion Picture and Television Engineers.

[0092] Figure 6A-6B An example grid X created in the HSW color space and transformed to the YCbCr color space and the RGB color space, respectively, according to an embodiment is diagrammatically illustrated. In this specific example, the grid size for each HSW dimension is 13×5×9, ranging from [0, 1]. Fig. 6AIt is clear that the grid Q in the YCbCr color space occupies only a portion of the [0,1],[0,1],[0,1] cube. In contrast, the grid Q in the RGB color space occupies the entire [0,1],[0,1],[0,1] cube. In both cases, the grid points are properly aligned with the corresponding valid color space boundaries. The edges connecting R=G=B=0 or R=G=B=1 on the RGB color space boundary have hue values ​​{0,1 / 6,2 / 6,3 / 6,4 / 6,5 / 6}. Therefore, in order to place the grid points accurately on these edges, the grid size in the H dimension can be selected according to the formula 6n+1, where n is a positive integer. Other edges on the RGB color space boundary have a saturation value of 1, which causes these edges to have grid points on them regardless of any particular grid size.

[0093] Typically, the DM metadata L1-mid of an HDR image represents the average value of the RGB channels of the image. However, in the process (200), the RGB channels of the output HDR image (240) depend on the chroma offset (230) (see Figure 2 ). Therefore, it is necessary to estimate the DM metadata L1-mid. According to an exemplary embodiment, such an estimate (222) is obtained using the initial HDR image (216). More specifically, the estimate (222) of the DM metadata L1-mid is represented as It can be calculated as the average value of the Y channel of the initial HDR image (216) as follows:

[0094]

[0095] Reference Figure 2 In an exemplary embodiment, a 5-D LUTΦ (228) used in a chroma offset processing block (220) of the process (200) may be defined on a 5-D grid X in HSWLM space. In operation, the 5-D LUTΦ (228) outputs chroma offsets (230) in response to an input vector (226) defined in HSWLM space. The input vector (226) is composed of an HSW channel block (224), a shaping function index map (212), and an estimate (222) of the DM metadata L1-mid. An example training process that may be used to populate the 5-D LUT (228) is described in more detail below (e.g., see Figures 7 and 8 ). The chroma shift (230) obtained using the 5-D LUT (228) is added (232) to the initial HDR image (216), thereby producing an output HDR image (240). The DM block (135) then processes the output HDR image (240) to generate an output DM image (137).

[0096] In an exemplary embodiment, the above-described linear interpolation may be used to determine, for example, pixel-by-pixel, the chrominance shift for different pixels (230). In this document, the linear interpolation operation is represented as For the i-th pixel, the chromaticity shift can be calculated as:

[0097]

[0098] Equation (14) can be used to program the chroma offset processing block (220) to determine the chroma offset (230). The i-th pixel in the output HDR image (240) can be calculated as:

[0099]

[0100] Equation (15) can be used to configure the adder (232) of the chroma offset processing block (220). If the R, G, and B channels of the i-th pixel in the output HDR image V (240) are represented as Then the DM metadata L1-mid, defined as the average value of all R, G and B channels, can be expressed as:

[0101]

[0102] If the output DM image (137) is represented as but The i-th pixel in can be expressed as:

[0103]

[0104] where f (DM) is the DM function applied by the aforementioned DM block (135).

[0105] Parameter Learning

[0106] Figure 7is a flow chart illustrating a method (700) of populating a 5-D LUT (228) according to an embodiment. In an exemplary embodiment, the method (700) relies on a cost function (704), which may generally include a color shift term and one or more regularization terms. For each selected grid point (706) on a suitable (e.g., predefined as described above) 5-D grid (702), the method (700) includes an iterative minimization process (708) configured to find a chroma shift (710) corresponding to an approximate minimum of the cost function (704), and to update (712) a nascent 5-D LUT (228) using the found chroma shift. The method (700) also includes repeating the set of processing operations (708), (710), and (712) for a plurality of different selected grid points (706). Exiting from the repeated loop occurs when an exit condition (714) is satisfied. Upon exit, the method (700) includes outputting (716) the populated 5-D LUT and saving the populated 5-D LUT as the 5-D LUT (228) (see also Figure 2 ).

[0107] In an exemplary embodiment, a cost function (704) may be constructed to drive an iterative minimization process (708) to find an approximately optimized chrominance shift (710) that can correct the aforementioned color shifts for multiple grid points on the 5-D grid X (702). For computational efficiency, the processing operations (708), (710), and (712) may be configured to process one grid point at a time. Thereafter, for the selected grid point X of index i, i (706), the H, S and W channels of this point are represented as Integer function indexes are denoted by l i =X i，3 ; and the DM metadata L1-mid (222) is represented as m i =X i，4 The HDR value corresponding to the initial HDR image (216) is

[0108] If the chroma shift (230) is not applied, then the HDR value is passed to the DM block (135) to obtain the initial DM value On the other hand, if the chromaticity shift r CbCr =(r Cb , r Cr ) is applied to the HDR value The Cb and Cr channels of the HDR image (240) are then used to generate new HDR values ​​for the HDR image (240). And the final output DM value for the DM image (137) is

[0109] In an exemplary embodiment, it is desirable that the chroma shift (710) be able to significantly reduce (e.g., to an imperceptible level) or completely eliminate SDR to HDR color shift. However, it is also desirable that the chroma shift (710) not produce any artifacts or change the "look" of the image. Therefore, the cost function (704) can be constructed to include a color shift term and one or more regularization terms to ensure stability. Because the cost function (704) only considers one grid point (706) at a time, the cost function (704) can be constructed to be insensitive to the position of the grid point (706) on the 5-D grid (702), and also to be insensitive to any particular topological features of the 5-D grid (702).

[0110] For purposes of illustration, and without any implied limitation, the example cost function (704) described below includes the following terms: Hue difference cost E hue , offset cost E off , brightness change cost E lum , Saturation change cost E sat and effective range cost E valid The total cost function E total (704) is defined as:

[0111] E total =E hue +λ off E off +λl um E lum +λ sat E sat +E valid (18)

[0112] Among them, λ off , lum and λ sat is a weight constant. valid term, the weight constant is 1, since the output value for this particular term is either 0 or infinity. Other examples of weight constants could be λ off =0.01,λ lum = 0.04 and λ sat =0.0025. In various other embodiments, the cost function (704) may have more or fewer cost terms. Some of the terms of such other cost functions (704) may differ from the example cost terms listed above.

[0113] Hue difference cost E hueis the color shift term. In an exemplary embodiment, the color shift can be measured by the difference in hue in the HSV color space. It is also important to note that when the SDR image is a neutral color, its hue can be undefined and its saturation can be 0. Measuring the color shift as a change in saturation in the HSV color space usually helps to properly handle this situation. The H, S, and V channels of the input SDR value and the final output DM value are represented as and Then, the hue difference cost E hue Can be defined as:

[0114]

[0115] Among them, θ hue and θ sat is the threshold for acceptable color shift. Function diff hue (a, b) = min(|ab|, 1-|ab|) can be used to measure the difference in hue, for example, because the maximum difference in hue is 0.5. From equation (19), it can be seen that for non-neutral color SDR values, that is, when When the hue difference cost E hue is 0. On the other hand, for the neutral color SDR value, that is when When the hue difference cost E hue is also 0. In an exemplary embodiment, the parameter value of equation (19) can be θ hue β0.008 2 and θ sat =0.008 2 .

[0116] Offset cost E off is a regularization term that is configured to regularize the chromaticity offset to a reasonable range and avoid overfitting. Such an offset cost E off Can be defined as:

[0117]

[0118] From equation (20), we can see that when r Cb =r Cr When =0, the offset cost is minimum.

[0119] Brightness change cost E lum is a regularization term that is configured to regularize the brightness change caused by chromaticity shift. Such a brightness change cost E lum Can be defined as:

[0120]

[0121] Among them, β neg and β pos are the weights of the brightness change in the darker direction and the brighter direction, respectively. In an exemplary embodiment, β neg >β pos This means that if the chromaticity shift makes the output DM value relatively dark, the corresponding cost is relatively high. lum The presence of usually helps to maintain the highlight appearance in the HDR image (240). From equation (21), it can be seen that when the chroma shift is not applied, the brightness change cost is the smallest, that is, when r Cb =r Cr = 0. In an exemplary embodiment, the parameter value of equation (21) may be β neg =4 and β pos =1.

[0122] Saturation change cost E sat is a regularization term that is configured to regularize the saturation change caused by chromaticity shift. Such a saturation change cost E sat Can be defined as:

[0123]

[0124] It can be seen from equation (22) that the greater the saturation change, the higher the saturation change cost. sat The presence of usually helps to maintain the relative appearance of the HDR image (240). When chroma shift is not applied, the saturation change cost is minimal, i.e., r Cb =r Cr =0.

[0125] Effective range cost E valid is a regularization term that is configured to constrain new HDR values ​​to be within the valid range. Such a valid range cost E valid Can be defined as:

[0126]

[0127] in are the R, G and B channels of the corresponding HDR values. When , the HDR value is within the valid range, and the corresponding valid range cost is 0. In addition, for numerical stability, if there is no chromaticity shift, that is, r Cb =r Cb = 0, the effective range cost is also set to 0. Otherwise, the effective range cost is set to infinity.

[0128] Figure 8is a flow chart illustrating an iterative minimization process (708) according to an embodiment. The iterative minimization process (708) uses a cost function (704), for example, the total cost function E of equation (18) total The input to the iterative minimization process (708) includes the grid points X i (804) and initial values ​​of the chroma offset and step size (802). The output of the iterative minimization process (708) includes a chroma offset (710) corresponding to the minimum value of the cost function (704). The chroma offset (710) obtained in this manner can typically be stored in the 5-D LUT (228). For computational efficiency, the iterative minimization process (708) is configured to find the chroma offset (710) within a relatively small local range specified for the calculation block (806). The corresponding processing loop of the block (808) for calculating the value of the cost function (704) is run until convergence (814) or a maximum number of iterations t is reached. max (812). The step size may be changed (typically reduced) at a change block (816) to make the chroma shift (710) better correspond to the actual minimum of the cost function (704) in the local range used. For computational efficiency, the step size is not allowed to be less than a specified fixed minimum step size, which is checked in a step size check block (818).

[0129] In an exemplary embodiment, the initial value of the chromaticity shift (802) may be set to At the t-th iteration, t≥1, the local range of the processing block (806) Can be set as follows:

[0130]

[0131] in, is the estimated chromaticity shift from the previous iteration; Δr t is the current step size; and k is a constant that controls the size of the local range. Then, the optimal chroma offset (810) at the current iteration can be expressed as:

[0132]

[0133] If the iterative minimization process (708) reaches the maximum number of iterations t at the counter block (812) max , then set the chroma offset (710) to Otherwise, if the estimated chroma offset (814) is determined to be a local minimum, i.e. Then the step size (816) of the next iteration can be reduced to Δr t+1 =αΔr t , where α < 1 is a constant. The value of α can usually be chosen to achieve the desired convergence rate. If Then the treatment can be done with Δr t+1 =Δr t Continue. The accuracy of the chromaticity shift (710) can usually be determined by the constant Δr min Control. If Δr t+1 <Δr min , the convergence criterion is considered to be met, and the output chrominance offset (710) is set to In an exemplary embodiment, the following parameter values ​​may be used: Δr1=10 -3 , Δr min =10 -6 , α = 0.5 and t max =100. In some embodiments, Δr min The value can be around 10 -3 to 10 -6 , because the visual quality of the corresponding output HDR image (137) is still acceptable for some applications even at the top of this range.

[0134] According to the exemplary embodiments disclosed above, for example, in the Summary of the Invention section and / or reference Figures 1 to 8 Any one or any combination of some or all of the above, providing a device comprising a video transmission system capable of changing the dynamic range of an input image, the transmission system comprising: a memory (e.g., Figure 2 228), the memory is used to store a plurality of chroma offset values ​​corresponding to grid points of the five-dimensional grid; and a processor (e.g., Figure 1 120), the processor is used to have a first dynamic range (eg, Figure 2 SDR) input image (for example, Figure 2 117) to a second dynamic range having a larger dynamic range (e.g., Figure 2 The corresponding output image of HDR (for example, Figure 2 240), the processor is configured to: reshape the input image (for example, Figure 2 210), generating an intermediate image having a second dynamic range (eg, Figure 2 216), the shaping uses a shaping function index map (e.g., Figure 2 212) to perform; estimate (for example, Figure 2 222) a display-management metadata value corresponding to the intermediate image; and applying to each pixel of the intermediate image (e.g., Figure 2 232) corresponding chromaticity shift (for example, Figure 2230) to generate an output image, the corresponding chrominance offset is determined from a plurality of chrominance offset values ​​by addressing a grid point using a corresponding index, a display-management metadata value, and three corresponding pixel values ​​of a corresponding pixel of the input image.

[0135] In some embodiments of the above apparatus, the first dynamic range is a standard dynamic range (eg, Figure 2 SDR); and wherein the second dynamic range is a high dynamic range (e.g., Figure 2 HDR).

[0136] In some embodiments of any of the above apparatus, the three corresponding pixel values ​​are a hue value, a saturation value, and an intensity value of a corresponding pixel of the input image.

[0137] In some embodiments of any of the above apparatus, the processor is further configured to nonlinearly rescale (e.g., V to W, Figure 2 224) intensity value of the input image; and wherein the three corresponding pixel values ​​are the hue value, saturation value and rescaled intensity value of the corresponding pixel of the input image.

[0138] In some embodiments of any of the above apparatus, the video delivery system is configured to generate an image adapted for display by applying a display-management process to the output image (e.g., Figure 2 137).

[0139] In some embodiments of any of the above apparatus, the video delivery system includes a video encoder (e.g., Figure 1 120), the encoder includes at least a portion of a processor.

[0140] In some embodiments of any of the above apparatus, the plurality of chroma offset values ​​are arranged in a lookup table in a memory, the lookup table being addressable using an integer function index value, a metadata value, a hue value, a saturation value, and an intensity value.

[0141] In some embodiments of any of the above apparatus, the display-management metadata value corresponding to the intermediate image is an L1-mid luminance value.

[0142] In some embodiments of any of the above apparatus, the processor is further configured to perform linear interpolation of the chroma offset values ​​(eg, equations (6)-(11)) to determine the corresponding chroma offset.

[0143] According to another exemplary embodiment disclosed above, for example, in the abstract section and / or reference Figures 1 to 8 A method for changing the dynamic range of an input image is provided, the method comprising the following steps: converting an input image having a first dynamic range (e.g., Figure 2SDR) input image (for example, Figure 2 117) to a second dynamic range having a larger dynamic range (e.g., Figure 2 The corresponding output image of HDR (for example, Figure 2 240), the conversion is performed using a plurality of pre-computed chroma offset values ​​corresponding to grid points of a five-dimensional grid; and wherein the conversion includes: reshaping the input image (e.g., Figure 2 210) to generate an intermediate image having a second dynamic range (eg, Figure 2 216), the shaping uses a shaping function index map (e.g., Figure 2 212) to perform; estimate (for example, Figure 2 222) a display-management metadata value corresponding to the intermediate image; and applying to each pixel of the intermediate image (e.g., Figure 2 232) corresponding chromaticity shift (for example, Figure 2 230) to generate an output image, the corresponding chrominance offset is determined from a plurality of chrominance offset values ​​by addressing a grid point using a corresponding index, a display-management metadata value, and three corresponding pixel values ​​of a corresponding pixel of the input image.

[0144] In some embodiments of the above method, the method further comprises non-linearly rescaling the intensity values ​​of the input image (eg, V to W, Figure 2 224); and wherein the three corresponding pixel values ​​are the hue value, saturation value and rescaled intensity value of the corresponding pixel of the input image.

[0145] In some embodiments of any of the methods above, the method further comprises generating an image adapted for display by applying a display-management process to the output image (e.g., Figure 2 137).

[0146] In some embodiments of any of the above methods, the display-management metadata value corresponding to the intermediate image is an L1-mid luminance value.

[0147] In some embodiments of any of the methods above, converting further comprises performing linear interpolation of the chroma offset values ​​(eg, equations (6)-(11)) to determine the corresponding chroma offsets.

[0148] In some embodiments of any of the methods above, a plurality of chroma offset values ​​are arranged in a lookup table addressable using an integer function index value, a metadata value, a hue value, a saturation value, and an intensity value.

[0149] According to yet another exemplary embodiment disclosed above, for example, in the abstract section and / or reference Figures 1 to 8 any one or any combination of some or all of the above, providing a non-transitory computer readable medium storing instructions, which, when executed by an electronic processor, causes the electronic processor to perform operations including a method of changing the dynamic range of an input image, the method comprising the steps of: converting a first dynamic range (e.g., Figure 2 SDR) input image (for example, Figure 2 117) to a second dynamic range having a larger dynamic range (e.g., Figure 2 The corresponding output image of HDR (for example, Figure 2 240), the conversion is performed using a plurality of pre-computed chroma offset values ​​corresponding to grid points of a five-dimensional grid; and wherein the conversion comprises: reshaping the input image (e.g., Figure 2 210) to generate an intermediate image having a second dynamic range (eg, Figure 2 216), the shaping uses a shaping function index map (e.g., Figure 2 212) to perform; estimate (for example, Figure 2 222) a display-management metadata value corresponding to the intermediate image; and applying to each pixel of the intermediate image (e.g., Figure 2 232) corresponding chromaticity shift (for example, Figure 2 230) to generate an output image, the corresponding chrominance offset is determined from a plurality of chrominance offset values ​​by addressing a grid point using a corresponding index, a display-management metadata value, and three corresponding pixel values ​​of a corresponding pixel of the input image.

[0150] According to yet another exemplary embodiment disclosed above, for example, in the abstract section and / or reference Figures 1 to 8 A method for generating a plurality of chromaticity offset values ​​for performing color shift correction in an output image, the output image being generated by changing the dynamic range of an input image, the method comprising the steps of: defining a five-dimensional grid (e.g., Figure 7 702), wherein the first dimension, the second dimension, the third dimension, the fourth dimension and the fifth dimension represent the shaping function index value, the metadata value, the hue value, the saturation value and the intensity value respectively; defining a cost function (e.g., equation (19)) for at least quantizing the cost of the hue difference between the input image and the output image Figure 7 704); for each grid point of the five-dimensional grid (for example, Figure 8 804), performing iterative minimization of a cost function to determine a corresponding set of chromatic aberration offset values; and in an electronic lookup table (e.g., Figure 2 A corresponding set of chroma offset values ​​are arranged in 228).

[0151] In some embodiments of the above method, defining the cost function includes including one or more regularization terms in the cost function, the regularization terms configured to keep the iterative minimization within a valid range.

[0152] In some embodiments of any of the above methods, the iterative minimization is performed over a local parameter range narrower than the full parameter range (e.g., Figure 8 806).

[0153] In some embodiments of any of the above methods, an iterative minimization is performed using a variable step size (e.g., Figure 8 818).

[0154] With respect to the processes, systems, methods, heuristic steps, etc. described herein, it should be understood that although the steps of these processes, etc. are described as occurring according to a certain ordered sequence, these processes may be implemented with the described steps performed in a sequence different from that described herein. It should also be understood that certain steps may be performed simultaneously, other steps may be added, or certain steps described herein may be omitted. In other words, the descriptions of the processes provided herein are for the purpose of illustrating certain embodiments and should not be construed in any way as limiting the claims.

[0155] Therefore, it should be understood that the above description is intended to be illustrative, not restrictive. Many embodiments and applications other than the examples provided will be apparent upon reading the above description. The determination of the scope should not be made with reference to the above description, but with reference to the appended claims and the full scope of equivalents to which these claims are assigned. It is anticipated and expected that future developments will occur in the technology discussed herein, and the disclosed systems and methods will be incorporated into these future embodiments. In short, it should be understood that the present application is capable of modification and variation.

[0156] Unless expressly indicated to the contrary herein, all terms used in the claims are intended to be given their broadest reasonable construction and ordinary meaning as understood by those familiar with the technology described herein. In particular, unless an explicit limitation to the contrary is stated in a claim, use of singular articles such as a, the, and said, etc. should be construed to recite one or more of the indicated elements.

[0157] The Abstract of the Disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. The Abstract is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, it can be seen that various features are grouped together in various embodiments for the purpose of simplifying the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the claimed embodiments include more features than are expressly recited in each claim. On the contrary, as reflected in the following claims, inventive subject matter resides in fewer features than all the features of a single disclosed embodiment. Therefore, the following claims are hereby incorporated into the Detailed Description, with each claim independently serving as a separately declared subject matter.

[0158] Although the present disclosure includes references to illustrative embodiments, the description is not intended to be interpreted in a limiting sense. Various modifications of the described embodiments that are obvious to those skilled in the art to which the present disclosure belongs, as well as other embodiments within the scope of the present disclosure, are considered to be within the principles and scope of the present disclosure, for example, as expressed in the following claims.

[0159] Some embodiments may be implemented as a circuit-based process, including possible implementation on a single integrated circuit.

[0160] Some embodiments may be embodied in the form of methods and devices for implementing these methods. Some embodiments may also be embodied in the form of program code recorded in a tangible medium (e.g., a magnetic recording medium, an optical recording medium, a solid-state memory, a floppy disk, a CD-ROM, a hard drive, or any other non-transitory machine-readable storage medium), wherein, when the program code is loaded into and executed by a machine such as a computer, the machine becomes a device for implementing the authorized (multiple) inventions. Some embodiments may also be embodied in the form of program code, such as stored in a non-transitory machine-readable storage medium, which includes being loaded and / or executed by a machine, wherein, when the program code is loaded and executed by a machine such as a computer or a processor, the machine becomes a device for implementing the authorized (multiple) inventions. When implemented on a general-purpose processor, the program code segment is combined with the processor to provide a unique device that operates similarly to a specific logic circuit.

[0161] Unless expressly stated otherwise, each numerical value and range should be interpreted as approximate as if the word "about" or "approximately" preceded the value or range.

[0162] The use of figure numbers and / or reference numerals of the drawings in the claims is intended to identify one or more possible embodiments of the claimed subject matter to facilitate the interpretation of the claims. Such use should not be interpreted as necessarily limiting the scope of these claims to the embodiments shown in the corresponding drawings.

[0163] Although the elements in the following method claims, if any, are recited in a specific order with corresponding labels, these elements are not necessarily intended to be limited to being implemented in this specific order unless the claim recitation otherwise implies a specific order for implementing some or all of these elements.

[0164] References herein to "one embodiment" or "embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present disclosure. The phrase "in one embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment, nor is it necessarily a separate or alternative embodiment that is mutually exclusive of other embodiments. The same applies to the term "implementation".

[0165] Unless otherwise stated herein, the use of ordinal adjectives "first," "second," "third," etc. to refer to multiple similar objects merely indicates that different instances of such similar objects are referred to and are not intended to imply that the similar objects referred to must have a corresponding order or sequence in time, space, order, or in any other manner.

[0166] Unless otherwise indicated herein, in addition to its express meaning, the conjunction "if" may also or alternatively be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting", and its interpretation may depend on the corresponding specific context. For example, the phrase "if it is determined" or "if [the stated condition] is detected" may be interpreted to mean "after determining" or "in response to determining" or "after detecting [the stated condition or event]" or "in response to detecting [the stated condition or event]".

[0167] Also for descriptive purposes, the terms "couple", "coupling" and "coupled" or "connect", "connecting" and "connected" refer to any means known in the art or later developed that allow energy to be transferred between two or more elements and that take into account the insertion of one or more additional elements, although not required. In contrast, the terms "directly coupled", "directly connected", etc., imply the absence of such additional elements.

[0168] As used herein to refer to components and standards, the term compliant means that the component communicates with other components in a manner specified in whole or in part by the standard, and will be recognized by other components as being sufficiently capable of communicating with other components in a manner specified by the standard. A compliant component is not required to operate internally in a manner specified by the standard.

[0169] The functions of the various elements shown in the drawings, including any functional blocks marked as "processors" and / or "controllers", can be provided by using dedicated hardware and hardware capable of executing software associated with appropriate software. When provided by a processor, the functions can be provided by a single dedicated processor, a single shared processor, or multiple separate processors, some of which can be shared. In addition, the explicit use of the term "processor" or "controller" should not be interpreted as referring only to hardware capable of executing software, and may implicitly include, but are not limited to, digital signal processor (DSP) hardware, network processors, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), read-only memories (ROMs), random access memories (RAMs), and non-volatile storage devices for storing software. Other hardware, conventional and / or customized, may also be included. Similarly, any switches shown in the figures are conceptual only. Their functions can be implemented by the operation of program logic, by dedicated logic, by the interaction of program control and dedicated logic, or even manually, and the specific technology can be selected by the implementer from a more specific understanding of the context.

[0170] As used in this application, the terms "circuit" and "circuitry" may refer to one or more or all of the following: (a) a pure hardware circuit implementation (e.g., an implementation solely in analog and / or digital circuits); (b) a combination of hardware circuitry and software, such as (as applicable): (i) a combination of (multiple) analog and / or digital hardware circuits and software / firmware, and (ii) any portion of (multiple) hardware processors (including (multiple) digital signal processors), software and (multiple) memories having software, which work together to enable a device such as a mobile phone or server to perform various functions); and (c) hardware circuitry and / or (multiple) processors, such as (multiple) microprocessors or portions of (multiple) microprocessors, which require software (e.g., firmware) to operate, but the software may not be present when the software is not required to operate. This definition of "circuitry" applies to all uses of this term in this application, including in any claims. As a further example, as used in this application, the term circuitry also covers an implementation of merely a hardware circuit or a processor (or multiple processors) or a portion of a hardware circuit or a processor and its (or their) accompanying software and / or firmware. The term circuitry also covers, for example and if applicable to the particular claim element, a baseband integrated circuit or processor integrated circuit for a mobile device or a similar integrated circuit in a server, cellular network device, or other computing or network device.

[0171] It should be understood by those skilled in the art that any block diagram herein represents a conceptual view of an illustrative circuit that embodies the principles of the present disclosure. Similarly, it should be understood that any flow charts (flowcharts and flowdiagrams), state transition diagrams, pseudocodes, etc. represent various processes that can be substantially represented in a computer-readable medium and thus executed by a computer or processor, whether or not such computer or processor is explicitly shown.

[0172] The "Summary" in this specification is intended to introduce some exemplary embodiments, and additional embodiments are described in the "Detailed Description" and / or with reference to one or more drawings. The "Summary" is not intended to identify essential elements or features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0173] Various aspects of the present invention may be understood from the following enumerated exemplary embodiments (EEE):

[0174] EEE1. A video transmission system capable of changing the dynamic range of an input image, the transmission system comprising:

[0175] a memory for storing a plurality of chroma offset values ​​corresponding to grid points of a five-dimensional grid; and

[0176] a processor that converts the input image having a first dynamic range into a corresponding output image having a second, larger dynamic range, the processor being configured to:

[0177] generating an intermediate image having the second dynamic range by shaping the input image, the shaping being performed using a shaping function index map having, for each pixel of the intermediate image, a respective index identifying a corresponding shaping function applied to the pixel;

[0178] estimating a display-management metadata value corresponding to the intermediate image; and

[0179] The output image is generated by applying a corresponding chromaticity offset to each pixel of the intermediate image, wherein the corresponding chromaticity offset is determined from the multiple chromaticity offset values ​​by addressing the grid points using the corresponding index, the display-management metadata value and three corresponding pixel values ​​of the corresponding pixels of the input image.

[0180] EEE2. The video transmission system as described in EEE 1,

[0181] wherein the first dynamic range is a standard dynamic range; and

[0182] Wherein, the second dynamic range is a high dynamic range.

[0183] EEE3. A video transmission system as described in EEE1 or 2, wherein the three corresponding pixel values ​​are the hue value, saturation value and intensity value of the corresponding pixel of the input image.

[0184] EEE4. A video transmission system as described in any one of EEE1 to EEE3,

[0185] wherein the processor is further configured to non-linearly rescale the intensity values ​​of the input image; and

[0186] The three corresponding pixel values ​​are the hue value, the saturation value and the rescaled intensity value of the corresponding pixel of the input image.

[0187] EEE5. A video delivery system as described in any one of EEE1 to EEE4, wherein the video delivery system is configured to generate a display-adapted image by applying display-management processing to the output image.

[0188] EEE6. A video transmission system as described in any one of EEE1 to EEE5, wherein the video transmission system includes a video encoder, and the video encoder includes at least a portion of the processor.

[0189] EEE7. A video transmission system as described in any one of EEE1 to EEE6, wherein the multiple chrominance offset values ​​are arranged in a lookup table in a memory that can be addressed using a shaping function index value, metadata value, hue value, saturation value and intensity value.

[0190] EEE8. A video transmission system as described in any one of EEE1 to EEE7, wherein the display-management metadata value corresponding to the intermediate image is an average brightness value.

[0191] EEE9. A video transmission system as described in any one of EEE1 to EEE8, wherein the processor is further configured to perform linear interpolation of the chroma offset values ​​to determine the corresponding chroma offset.

[0192] EEE10. A method for changing the dynamic range of an input image, the method comprising:

[0193] converting the input image having a first dynamic range to a corresponding output image having a second, larger dynamic range, the converting being performed using a plurality of pre-computed chroma shift values ​​corresponding to grid points of a five-dimensional grid; and

[0194] The conversion includes:

[0195] generating an intermediate image having the second dynamic range by shaping the input image, the shaping being performed using a shaping function index map having, for each pixel of the intermediate image, a respective index identifying a corresponding shaping function applied to the pixel;

[0196] estimating a display-management metadata value corresponding to the intermediate image; and

[0197] The output image is generated by applying a corresponding chromaticity offset to each pixel of the intermediate image, wherein the corresponding chromaticity offset is determined from the multiple chromaticity offset values ​​by addressing the grid points using the corresponding index, the display-management metadata value and three corresponding pixel values ​​of the corresponding pixels of the input image.

[0198] EEE11. The method as described in EEE 10 further includes nonlinearly rescaling the intensity values ​​of the input image;

[0199] as well as

[0200] The three corresponding pixel values ​​are the hue value, the saturation value and the rescaled intensity value of the corresponding pixel of the input image.

[0201] EEE12. The method as described in EEE 10 or 11 also includes generating a display-adapted image by applying a display-management process to the output image.

[0202] EEE13. A method as described in any one of EEE10 to EEE12, wherein the display-management metadata value corresponding to the intermediate image is an average brightness value.

[0203] EEE14. A method as described in any one of EEE10 to EEE13, wherein the conversion also includes performing linear interpolation of the chroma offset values ​​to determine the corresponding chroma offset.

[0204] EEE15. A method as described in any one of EEE10 to EEE14, wherein the multiple chrominance offset values ​​are arranged in a lookup table that can be addressed using an integer function index value, a metadata value, a hue value, a saturation value, and an intensity value.

[0205] EEE16. A non-transitory computer-readable medium storing instructions which, when executed by an electronic processor, cause the electronic processor to perform operations including the method according to any one of EEE10 to EEE15.

[0206] EEE17. A method of generating a plurality of chromaticity offset values ​​for performing color shift correction in an output image generated by changing a dynamic range of an input image, the method comprising:

[0207] A five-dimensional grid is defined, wherein the first dimension, the second dimension, the third dimension, the fourth dimension and the fifth dimension represent an integer function index value, a metadata value, a hue value, a saturation value and an intensity value respectively;

[0208] defining a cost function for quantifying at least the cost of a difference in tones between the input image and the output image;

[0209] For each grid point of the five-dimensional grid, performing iterative minimization of the cost function to determine a corresponding set of chroma offset values; and

[0210] The respective sets of chromaticity offset values ​​are arranged in an electronic lookup table addressable using sets of discrete values ​​corresponding to the first dimension, the second dimension, the third dimension, the fourth dimension, and the fifth dimension.

[0211] EEE18. The method of EEE 17, wherein defining the cost function comprises including one or more regularization terms in the cost function, the one or more regularization terms being configured to keep the iterative minimization within a valid range. ...

[0212] EEE19. A method as described in EEE 17 or 18, wherein the iterative minimization is performed within a local parameter range narrower than the full parameter range.

[0213] EEE20. A method as described in any one of EEE 17 to EEE19, wherein the iterative minimization is performed using a variable step size.

[0214] EEE21. A method as described in any one of EEE17 to EEE 20, wherein, for the tth iteration of the iterative minimization, the corresponding optimal chromaticity offset is determined for in, To minimize the range, E total is the cost function.

[0215] EEE22. A method as described in any one of EEE17 to EEE 21, wherein the cost function includes a hue difference cost E hue , offset cost E off , brightness change cost E lum , Saturation change cost E sat and effective range cost E valid The weighted sum of .

Claims

1. A video transmission system capable of changing the dynamic range of an input image, the transmission system comprising: A memory, the memory being used to store a plurality of chromaticity offset values ​​corresponding to grid points of a five-dimensional grid; and a processor that converts the input image having a first dynamic range into a corresponding output image having a second, larger dynamic range, the processor being configured to: generating an intermediate image having the second dynamic range by shaping the input image, the shaping being performed using a shaping function index map having, for each pixel of the intermediate image, a respective index identifying a corresponding shaping function applied to the pixel; estimating a display-management metadata value corresponding to the intermediate image; as well as The output image is generated by applying a corresponding chromaticity offset to each pixel of the intermediate image, wherein the corresponding chromaticity offset is determined from the multiple chromaticity offset values ​​by addressing the grid points using the corresponding index, the display-management metadata value and three corresponding pixel values ​​of the corresponding pixels of the input image.

2. The video transmission system according to claim 1, in, The first dynamic range is a standard dynamic range; and Wherein, the second dynamic range is a high dynamic range.

3. The video transmission system according to claim 1 or 2, wherein: The three corresponding pixel values ​​are a hue value, a saturation value and an intensity value of the corresponding pixel of the input image.

4. The video transmission system according to any one of claims 1 to 3, in, The processor is further configured to non-linearly rescale the intensity values ​​of the input image; and The three corresponding pixel values ​​are the hue value, the saturation value and the rescaled intensity value of the corresponding pixel of the input image.

5. The video transmission system according to any one of claims 1 to 4, wherein: The video delivery system is configured to generate a display adapted image by applying a display-management process to the output image.

6. The video transmission system according to any one of claims 1 to 5, wherein: The video delivery system includes a video encoder that includes at least a portion of the processor.

7. The video transmission system according to any one of claims 1 to 6, wherein: The plurality of chroma offset values ​​are arranged in a lookup table in a memory that is addressable using an integer function index value, a metadata value, a hue value, a saturation value, and an intensity value.

8. The video transmission system according to any one of claims 1 to 7, wherein: The display-management metadata value corresponding to the intermediate image is an average luminance value.

9. The video transmission system according to any one of claims 1 to 8, wherein: The processor is further configured to perform linear interpolation of the chroma offset values ​​to determine the corresponding chroma offset.

10. A method for changing the dynamic range of an input image, the method comprising: converting the input image having a first dynamic range to a corresponding output image having a second larger dynamic range, the converting being performed using a plurality of pre-computed chroma shift values ​​corresponding to grid points of a five-dimensional grid; and The conversion includes: generating an intermediate image having the second dynamic range by shaping the input image, the shaping being performed using a shaping function index map having, for each pixel of the intermediate image, a respective index identifying a corresponding shaping function applied to the pixel; estimating a display-management metadata value corresponding to the intermediate image; and The output image is generated by applying a corresponding chromaticity offset to each pixel of the intermediate image, wherein the corresponding chromaticity offset is determined from the multiple chromaticity offset values ​​by addressing the grid points using the corresponding index, the display-management metadata value and three corresponding pixel values ​​of the corresponding pixels of the input image.

11. The method of claim 10, further comprising non-linearly rescaling intensity values ​​of the input image; and in, The three corresponding pixel values ​​are a hue value, a saturation value, and a rescaled intensity value of the corresponding pixel of the input image.

12. The method of claim 10 or 11, further comprising generating a display adapted image by applying a display-management process to the output image.

13. The method according to any one of claims 10 to 12, wherein: The display-management metadata value corresponding to the intermediate image is an average luminance value.

14. The method according to any one of claims 10 to 13, wherein: The converting also includes performing linear interpolation of the chroma offset values ​​to determine the corresponding chroma offsets.

15. The method according to any one of claims 10 to 14, wherein: The plurality of chroma offset values ​​are arranged in a lookup table addressable using an integer function index value, a metadata value, a hue value, a saturation value, and an intensity value.

Images