Video delivery using dynamic range conversion with perceptually lossless backward compatibility

Abstract

Video delivery methods and apparatus providing dynamic range conversion with perceptually lossless backward compatibility. An example video delivery method includes generating a first lookup table (LUT) configured for converting images of a first dynamic range into images of a second dynamic range larger than the first dynamic range, the generating including adjustment of the first LUT based on differences between first and second color vectors of the first dynamic range, the second color vector being generated by applying a second LUT to a third color vector, the second LUT being configured for converting images of the second dynamic range into images of the first dynamic range, the third color vector being generated by applying the first LUT to the first color vector. The method further includes using the first LUT to convert a first video having the first dynamic range into a second video having the second dynamic range.

Description

D24042W001VIDEO DELIVERY USING DYNAMIC RANGE CONVERSION WITH PERCEPTUALLY LOSSLESS BACKWARD COMPATIBILITY1. Cross-Reference to Related Applications

[0001] This application claims the benefit from U. S. Provisional Patent Application No.63 / 737,115, filed on 20 December 2024, and European Patent Application No. 25176490.8, filed on 14 May 2025, each of which is incorporated by reference herein in its entirety.2. Field of the Disclosure

[0002] Various example embodiments relate generally to video delivery and, more specifically but not exclusively, to image-processing operations for dynamic range changes.3. Background

[0003] Standard Dynamic Range (SDR) refers to a relatively narrow range of brightness and contrast used to create and broadcast content on legacy devices, such as older monitors and TV sets. SDR techniques have relatively wide compatibility and are cost-effective, making them an economical choice for many consumers. SDR videos are well suited, e.g., for casual viewing and legacy displays of sufficient quality. In contrast, High Dynamic Range (HDR) is a higher end technology used in professional photography and more-advanced displays. As such, HDR supports a larger range of brightness levels and color gamut and can be used to enhance the color and contrast of images and videos, e.g., to produce more-lifelike videos. HDR videos are well suited, e.g., for home theater viewing and for creating realistic visuals in gaming applications.BRIEF SUMMARY OF SOME SPECIFIC EMBODIMENTS

[0004] One embodiment disclosed herein provides a computationally efficient method for SDR-to-HDR up-conversion using a three-dimensional (3D) lookup table (LUT) system characterized by perceptually lossless backward compatibility. Some examples focus on the important need for preserving the original quality of SDR content while enhancing it into the HDR format, thereby ensuring that the original visual and creative intent is maintained in both media formats. In one example, a multistage LUT construction strategy is implemented to effectively resolve non-monotonic discrepancies often inherent to the use of a backward HDR-to-SDR LUT, thereby providing a corresponding forward SDR-to-HDR LUT capableD24042W001of delivering high-quality HDR images substantially without any perceivable artifacts. The developed forward-backward (roundtrip) conversion capability has been rigorously tested to confirm that the HDR-converted content can be reverted substantially back to its original SDR form (i.e., with the effects of the roundtrip conversion being perceptually insignificant). The methods and apparatus disclosed herein beneficially provide significant improvements to the current state of the art in the HDR technologies in general, and to the dynamic range conversion techniques in particular.

[0005] According to an example embodiment, a video delivery apparatus comprises: at least one processor; and at least one memory including program code, wherein the at least one memory and the program code are configured to, with the at least one processor, cause the apparatus at least to: generate a first lookup table (LUT) configured for converting images of a first dynamic range into images of a second dynamic range that is larger than the first dynamic range, the generating including adjustment of the first LUT based on differences between first and second color vectors of the first dynamic range, the second color vector being generated by applying a second LUT to a third color vector, the second LUT being configured for converting images of the second dynamic range into images of the first dynamic range, the third color vector being generated by applying the first LUT to the first color vector; and use the first LUT to convert a first video having the first dynamic range into a second video having the second dynamic range.

[0006] According to another example embodiment, a video delivery method comprises: generating a first LUT configured for converting images of a first dynamic range into images of a second dynamic range that is larger than the first dynamic range, the generating including adjustment of the first LUT based on differences between first and second color vectors of the first dynamic range, the second color vector being generated by applying a second LUT to a third color vector, the second LUT being configured for converting images of the second dynamic range into images of the first dynamic range, the third color vector being generated by applying the first LUT to the first color vector; and using the first LUT to convert a first video having the first dynamic range into a second video having the second dynamic range.

[0007] According to yet another example embodiment, provided is a non-transitory computer-readable medium storing instructions that, when executed by an electronicD24042W001processor, cause the electronic processor to perform operations comprising the above video delivery method.BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Other aspects, features, and benefits of various disclosed embodiments will become more fully apparent, by way of example, from the following detailed description and the accompanying drawings, in which:

[0009] FIG. 1 is a block diagram illustrating a roundtrip SDR-HDR workflow according to one example.

[0010] FIG. 2 is a flowchart illustrating a core-LUT adjustment process used in the workflow of FIG. 1 according to some examples.

[0011] FIG. 3 is a block diagram illustrating a perceptual adjustment process used in the workflow of FIG. 1 according to some examples.

[0012] FIG. 4 graphically illustrates an initial error distribution across the 33x33x33 color cube at Stage 1 of the LUT adjustment process used in the workflow of FIG. 1 according to one example.

[0013] FIG. 5 graphically illustrates a refined error distribution across the 33x33x33 color cube after Stage 2 of the LUT adjustment process used in the workflow of FIG. 1 according to one example.

[0014] FIG. 6 graphically illustrates a further-refined error distribution across the 33x33x33 color cube after Stage 3 of the LUT adjustment process used in the workflow of FIG. 1 according to one example.

[0015] FIGS. 7A-7B pictorially illustrate reduction in visible artifacts after Stage 4 of the LUT adjustment process used in the workflow of FIG. 1 according to one example.

[0016] FIG. 8 is a block diagram illustrating a video delivery system in which various embodiments can be practiced according to some examples.

[0017] FIG. 9 is a flowchart of a video delivery method implemented in the video delivery system of FIG. 8 according to some examples.D24042W001

[0018] FIG. 10 is a block diagram illustrating a computing device used in the video delivery system of FIG. 8 according to some examples.DETAILED DESCRIPTION

[0019] As used herein, the term “dynamic range” (DR) may relate to a capability of the human visual system (HVS) to perceive a range of intensity (e.g., luminance, luma) in an image, e.g., from darkest blacks (darks) to brightest whites (highlights). In this sense, DR relates to a “scene-referred” intensity. DR may also relate to the ability of a display device to render, adequately or approximately, an intensity range of a particular breadth. In this sense, DR relates to a “display-referred” intensity. Unless a particular sense is explicitly specified to have particular significance at any point in the description herein, it should be inferred that the term may be used in either sense, e.g., interchangeably.

[0020] As used herein, the term “high dynamic range” (HDR) relates to a DR breadth that spans 14-15 or more orders of magnitude of the HVS. In practice, the DR over which a human may simultaneously perceive an extensive breadth in intensity range may be somewhat truncated, in relation to HDR. As used herein, the terms “enhanced dynamic range” (EDR) or “visual dynamic range” (VDR) may individually or interchangeably relate to the DR that is perceivable within a scene or image by a human visual system that includes eye movements, allowing for some light adaptation changes across the scene or image.Herein, EDR may relate to a DR that spans 5 to 6 orders of magnitude. While perhaps somewhat narrower in relation to the true scene referred HDR, EDR nonetheless represents a wide DR breadth and sometimes may also be referred to as HDR.

[0021] In practice, images comprise one or more color components (e.g., luma Y and chroma Cb and Cr) of a color space, where each color component is represented with a precision of n-bits per pixel (e.g., n=8). Using non-linear luminance coding (e.g., gamma encoding), images where n < 8 (e.g., 24-bit color JPEG images) are considered images of standard dynamic range (SDR), while images where n > 8 may be considered images of EDR.

[0022] With wide deployment of HDR displays, being able to have an HDR viewing experience has been providing a motivation for the consumers to purchase new, more-advanced display hardware. However, a lack of sufficient volume of content in the HDR domain often serves as an impediment to further penetration of the display market becauseD24042W001the bulk of the existing content is still in the SDR domain. As a result, development of computationally efficient SDR-to-HDR up-conversion techniques is an important and perhaps even urgent need. In addition, the reversibility for converting the up-converted HDR back to original SDR to preserve the original creative intent associated with the source SDR content is also desirable. These needs motivate the development of “round trip” SDR-to-HDR up-conversion algorithms, preferably operating in an ultra-computationally efficient manner.

[0023] Accordingly, some embodiments are directed at solving an inverse problem in which by being given a backward three-dimensional lookup table (3D-LUT) configured to map an HDR image to a corresponding SDR image, a workflow can be implemented to build a forward 3D-LUT. Such forward 3D-LUT enables taking the SDR input to generate the corresponding HDR output such that this HDR output can be converted back to the SDR domain via the given backward 3D-LUT to produce an SDR image that is substantially perceptually indistinguishable from the original SDR image. In some examples, the backward 3D-LUT can be generated through parametric methods, one example of which is the Dolby display management (DM) process. One objective is to devise a method for generating an efficient 3D-LUT that facilitates the reverse conversion from SDR to HDR such that the up-converted HDR is not prone to artifacts and still visually perceived as possessing pertinent HDR characteristics. Another objective is to provide a “roundtrip” conversion capability, where an SDR image, when converted to HDR and then back to SDR, retains its original visual characteristics and quality, thereby making the final (roundtrip) SDR image perceptually indistinguishable from the source SDR image.

[0024] In the above-outlined approach, initial steps in the process of creating the forward 3D-LUT include operations configured to generate the backward 3D-LUT. In one example, these operations are based on the Dolby DM process. On challenge associated with this process is that, in some cases, the backward 3D-LUT so constructed may exhibit relatively ill-defined monotonic non-decreasing nature of the corresponding conversion function, which may have some relatively pronounced spikes and jumps in local neighborhoods. At least some embodiments disclosed herein successfully overcome this challenge.

[0025] Some embodiments provide solutions to the above-indicated problems in the state of the art using a multi-step LUT construction workflow. For example, in one embodiment, the final 3D-LUT is constructed via cascading through multiple 3D-LUTs to achieve theD24042W001objectives of providing: (1) the roundtrip perceptually lossless reconstructed SDR; (2) substantially artifact-free HDR with visually perceptible HDR-ness; and (3) ultra-computationally efficient SDR-to-HDR up-conversion.

[0026] FIG. 1 is a block diagram illustrating a roundtrip SDR-HDR workflow 100 according to one example. The workflow 100 employs a forward LUT 120 and a backward LUT 140. When applied to a first SDR RGB image 110, the forward LUT 120 converts that image into a corresponding HDR RGB image 130. When applied to the HDR RGB image 130, the backward LUT 140 converts that image into a second SDR RGB image 150. When the forward LUT 120 and the backward LUT 140 are appropriately constructed, the differences between the first and second SDR RGB images 110, 150 are relatively small, e.g., visually imperceptible to the human eye.

[0027] In one example, the backward 3D-LUT 140 is constructed based on a set of DM parameters used for putative display on which the various images can be displayed. A design goal is to build the forward 3D-LUT 120 based on the backward 3D-LUT 140 so constructed such the first and second SDR RGB images 110, 150 are substantially perceptually identical. Towards this goal, the generation of the forward 3D-LUT 120 can be structured as a four-stage process as follows:• A first stage involves creating a first approximation of the forward 3D-LUT 120 by inversing the backward 3D-LUT 140 (which is formulated using the original 3D LUT derived from the Display Management (DM) metadata).• Second and third stages include iterative fine-tuning processes controlled in an “objective” manner. During these stages, the LUT is methodically adjusted to reduce the error at each node of the forward 3D-LUT 120. This methodical calibration is needed for improving the precision of color mapping, ensuring that each color is represented as accurately and faithfully as possible within the confines of the target color space.• A fourth stage is configured to focus on local color regions from the “subjectively perceptual” point of view. One goal in the fourth stage is to reduce any subjectively visible artifacts and discrepancies that might not be fully resolved by objective node adjustments alone. Towards this goal, the process is configured to employ advanced algorithms designed to specifically target and mitigate errors in color representation that elude the resolution capabilities of the LUT nodes. By addressing theseD24042W001perceptual inconsistencies, the overall visual quality of the converted images can be significantly enhanced, thereby bringing them closer to the original content’s true color fidelity.

[0028] With the forward 3D-LUT 120, one can convert the incoming SDR video to the corresponding HDR video. Since the DM parameters are fixed, one can directly append the corresponding (e.g., Dolby Vision) metadata bitstream without involving any other composer or DM computation. The composer can be set to the bypass 1: 1 mode, and the DM can be set to the value that creates the backward 3D-LUT 140. In this manner, the entire process tends to be computationally ultra-efficient.

[0029] With a fixed set of DM parameters, one can build a fixed static backward 3D-LUT 140 to convert HDR to SDR using the following concepts and definitions.

[0030] The HDR cube, CHDR, is defined as an HDR space where each color is represented by a triplet (r' g' b', with values from 0 to 1, accommodating enhanced luminance and color depth. The SDR color cube, CSDR, is defined with each color represented by a triplet (r g b), with each component constrained between 0 and 1, reflecting standard color intensities.

[0031] The transformation is facilitated by a LUT and involves using 3D interpolation to map colors from the HDR space to the SDR space effectively. We mathematically define this transformation as:LHDR-to-SDR= {(r'ig'ib'i), (rigibi) | i = 1,..., N} (1) where N denotes the number of entries in the LUT. In one example, this table is used in conjunction with the interp3 function of MATLAB to interpolate between known data points in a non-uniform grid, ensuring smooth transitions.

[0032] Interpolation Formula: In some examples, using the MATLAB’s interp3 functionality, one can perform linear interpolation within the 3D LUT to compute the SDR values from a given HDR input. Let us assume that (X, Y, Z, F) represent the grid coordinates and value arrays in the HDR space, and (Xq, Yq, Zq) are the query points in the SDR space. Then, the following operations can be defined:• Query Preparation: Create grid arrays Xq, Yq, Zq corresponding to the desired SDR output resolution using meshgrid.D24042W001• Interpolation Execution: Compute the interpolated values Vq that represent the SDR colors:Vq = interp3(X, Y, Z, V, Xq, Yq, Zq,’ linear’) (2)• The function defined by Eq. (2) interpolates the values at the grid points (X, Y, Z) with known values V, at the query points (Xq, Yq, Zq), using linear interpolation.

[0033] The above-mentioned four stages used to construct the forward 3D-LUT 120 according to some examples are described in more detail below.Stage 1 - Initial Construction of 3D-LUT

[0034] In one example, defining the transformation matrix LSDR-t0-HDRbegins by deriving its inverse from LHDR-t0-SDR, which provides a computational framework to reverse-engineer the color mapping process.

[0035] Input SDR Color Vector: In some examples, each node i in the SDR dataset is represented by a color vector Vj, which is structured as follows:where rf, gt, bte [0,1] (3)

[0036] Output HDR Matrix: In some examples, the HDR dataset is represented by the matrix M, containing m HDR color vectors:where r'k,g'k,b'ke [0,1] (4)r'i - where each m, = g'i is equal to its corresponding Vj = 9ib’i. bi.

[0037] In some examples of our methodology and coding practices, we utilize the same dataset as referenced in the foregoing description. However, it is important to note that variations between datasets are possible depending on specific requirements or contexts.

[0038] Replication of SDR Vector for Distance Calculation: The SDR vector for each node is replicated to align with each HDR vector in M, creating Vrepexpressed as follows:D24042W001V *rep 91 9m-b bm-Jmx3

[0039] Euclidean Distance Squared Calculation: In some examples, the squared distance between the replicated SDR vector and each HDR vector in M is computed as follows:Diik= (ri - r'k)2+ (^ - g'k)2+ (bt - b'^2, k = l,...,m (6) This computation results in a distance vector Dj for each SDR node i.

[0040] This step is designed to ensure consistency and accuracy in the processing of image data. Although HDR and SDR images originate from different domains, our approach is directed at maintaining a relatively high degree of similarity between them. Such similarity is important, as it ensures that irrespective of the dynamic range of the display medium, the visual representation remains consistent and true to the source content.

[0041] Sorting and Selecting Nearest Neighbors: The distances Dj are sorted, and the indices of the nearest neighbors are selected, e.g., as follows:sortedlndices = argsort(Dj), sortedDistances = sort(Dj) (7) The closest neighbors (up to a maximum of 8 or as specified) are used for the interpolation.

[0042] Weight Calculation for Interpolation: Weights are calculated inversely relative to the adjusted distances, ensuring no zero denominators, e.g., as follows:WkK= - — - — -, k = 1,...,min(neighbours, 8) (8) sortedDistances[fc]+0.0001& v’ where the neighbors are defined above in reference to Eq. (7).

[0043] Weighted Average for HDR Output: The interpolated HDR values are calculated using the weighted sums method, e.g., as follows:Smin(neighbours,8)_ k=l ^ / c’^'sortedlndicesf / c]i v^min(neighbours,8)K=iWkFormulas similar to Eq. (9) are used for g'i and b'j, ensuring a smooth transition from SDR values to HDR values by averaging based on proximity and color intensity, any values out of the [0,1] range in either domain are clipped to 0 and 1 as follows:r = clip3( r, 0, 1 ) (10a)g = clip3( g, 0, 1 ) (10b)b = clip3( b, 0, 1 ) (10c)D24042W001

[0044] In some examples, the LUT is mathematically defined as a set of ordered pairs, as follows:LSDR-to-HDR= {((rigibi), r'ig'ib'i) | i = 1,..., N} (11) where (rfgi b^ are the components of the color vectors in the SDR space, each ranging from 0 to 1; (r'i g'i b'i) are the components of the corresponding color vectors in the HDR space, also each ranging from 0 to 1; and N represents the total number of mappings in the LUT, providing a granular mapping from the SDR to the HDR.Stage 2 - Global Iterative Core-LUT Adjustment

[0045] In this stage, the previously generated 3D-LUT (as the core 3D-LUT) is refined via adjustment of the node values based on the error between the input SDR value and the corresponding output SDR value of the roundtrip SDR-HDR workflow 100.

[0046] Let V represent the set of SDR color vectors, and H represent the set of HDR color vectors. Each vector can mathematically be expressed as:9i.where ri, gi, bi E [0,1] (12)b,-Hi = g\.where r’ i, g’ i, b’ i E [0,1] (13)The transformations from SDR to HDR and from HDR to SDR are managed by the 3D-LUTs, LSDR-to-HDRand LHDR-to-SDR, respectively.

[0047] FIG. 2 is a flowchart illustrating a core-LUT adjustment process 200 used in the workflow 100 according to some examples. The process 200 is an iterative process conducted over K cycles, which are indexed from 0 to K₁-1. In one example, K₁ is 161. In other examples, other suitably selected numbers of iterations can also be used. Each iteration aims to progressively enhance the accuracy of the roundtrip transformations.

[0048] Within each iteration, SDR color vectors 202 are transformed in a block 210 into corresponding HDR color vectors 212 by applying the specified SDR-to-HDR LUT mathematically expressed as follows:H = f(V, LSDR-to-HDR) = LSDR-to-HDR(V) (14) where f is the function applied using the LSDR-to-HDRLUT for the input triplets V (also see Eq. (12)). In the first iteration, an initial SDR-to-HDR LUT 204 is applied in the block 210.D24042W001For each subsequent iteration, a newly updated SDR-to-HDR LUT 206 is applied in the block 210.

[0049] Let k be the iteration index. Then, Eqs. (12) and (14) can be used to obtain the following:Hi,k= [f((ri,kgi,kbi,k), LSDR-to-HDR)] (15) To validate the roundtrip predictions, the HDR vectors 212 are inversely transformed in a block 220 back to SDR vectors 222 using the HDR-to-SDR LUT 140 as follows:E' = f (H, ^HDR-to-SDR) (16)'i,k=[ / ((r / t,fc d'i.k b'i,k)’ ^HDR-to-SDR)] (1 )This transformation is important as it allows for the assessment of the fidelity of the HDR predictions by comparing the inversely predicted SDR vectors 222 against the original SDR vectors 202.

[0050] The differences D between V and V are calculated, and adjustments are made based on their magnitudes in a block 230 to obtain adjusted differences 232 as follows:Di,k= Vi,k- V'i,k(18)Di,kif|A,k | > 0.02Di,k / 2(19)0 if|Di,k| < 0.002where |Di,k| = (|Di,k / 1|, |Di,k / 2|, |Di,k / 3|,... ).

[0051] The adjustments performed in the block 230 are designed to scale down only significant discrepancies and to refine the LUT incrementally without overfitting to minor noise. Based on the adjusted differences 232, the updated SDR-to-HDR LUT 206 is computed in a block 240 as follows:^SDR-to-HDR = ^SDR-to-HDR + reshape(Dadjusted, dimensions) X 0.5 (20) In this example, the update is configured refine the LUT by integrating the learned corrections, scaled down by a factor of 0.5 to moderate the update intensity and ensure stability throughout the iterative process. In other examples, other factor values can also be used. By following the iterative process 200, the transformation from SDR to HDR (and vice versa) is refined, thereby enhancing the fidelity of the resulting images and ensuring a more accurate representation of colors in different dynamic ranges.Stage 3 - Global Iterative Pre-LUT AdjustmentD24042W001

[0052] In this stage, we introduce another 3D-LUT (hereafter referred to as pre-LUT) before the core 3D-LUT to fine-adjust the end-to-end combined forward 3D-LUT 120. In one example, the pre-LUT is generated iteratively over 165,600 cycles aimed at adjusting the predictions by cascading both pre- and core 3D-LUTs. The corresponding process is designed to minimize discrepancies and enhance the fidelity of image transformations across three stages: (i) SDR to modified SDR, (ii) modified SDR to High Dynamic Range (HDR), and (iii) HDR back to SDR.

[0053] In various examples, the above-outlined process begins with SDR color vectors, which are defined for each pixel i in the SDR dataset as:Vi= (rigibi), where ri, gi, bi∈ [0,1]b,- After the SDR to HDR transformation, the HDR image data for each pixel i are represented by HDR color vectors expressed as follows:, where ∈ [0,1] (22)The inverse transformation converts HDR back to SDR, resulting in inverse SDR color vectors expressed as follows:V'i= (r'ig'ib'i), where ∈ [0,1] (23)

[0054] The initial step involves transforming SDR vectors into modified SDR vectors using a dedicated pre-LUT, which can mathematically be expressed as follows:M = f(V, ksDR-to-Modified SDR) (24) where M denotes the matrix of modified SDR vectors. Modified SDR vectors are subsequently transformed to HDR using the previously mentioned core-LUT as follows:H = f(M, LModified SDR-to-HDR) (25)Finally, HDR vectors are converted back to SDR to assess the accuracy of the HDR predictions:V' = f(M, LHDR-to-SDR) (26)

[0055] Discrepancies D between the original and the inversely predicted SDR vectors are calculated as:D24042W001D = V - V (27)Adjustments are made based on the magnitude of the discrepancies D, where significant discrepancies are identified and adjusted:Dn> (0 if|Dn| < 0.001 or (iter > 160,000 and |Dn| > 0.001))Dadjusted= Dn, if |Dn| ≥otherwise t where |D| = (|D1|, |D2|, |D3|,...).

[0056] The LUT is updated by integrating half the adjusted discrepancies to refine the color mapping accuracies, e.g., as follows:^SDR-to-Modified SDR=-^SDR-to-Modified SDR + reshape(Dadjusted, dimensions) x 0.5 (29)

[0057] Since both Pre-LUT (LSDR-to-Modified SDR) and Core-LUT (LModified SDR-to-HDR) are 3D-LUTs, we cascade them into a single 3D-LUT denoted as LSDR-to-HDR.

[0058] The above-described fine-grain iterative process of Stage 3 is designed to ensure that the SDR-to-HDR transformation (and vice versa) accurately represents the intended color dynamics of the original data. The structured approach involves direct adjustments and back-transformations to continuously improve the prediction accuracy, which is important for achieving high fidelity in HDR imaging.Stage 4 - Local Perceptual Adjustment

[0059] In the fourth stage, we refine parts of the lookup table (LUT) to minimize visible artifacts in different local node neighborhoods. Initially, we evaluate and classify the RGB color fidelity across SDR and HDR imaging formats by employing the LUTs developed in the Stages 1-3. This assessment is carried out through a sequence of steps. Each step is documented using a uniform mathematical notation, which underscores the precise transformations and analytical methods utilized. This approach ensures a systematic and transparent examination of how each RGB component is handled through various stages of the color range conversion.

[0060] FIG. 3 is a block diagram illustrating a perceptual adjustment process 300 according to some examples. The process 300 includes a block 310 in which an SDR-to-HDR LUT 302 is subjected to perceptual adjustment at problematic grid points to generate an adjusted SDR-to-HDR LUT 312. Parameters for the adjustment operations of the block 310 are determined as described in more detail below.D24042W001

[0061] An initial step of the perceptual adjustment process 300 includes defining RGB vectors 314, 316 for each color combination within an 8-bit color space. The vectors are normalized to facilitate further transformations as follows:r iVi= (rigibi), where ri, gi, bi∈ [0,1] (30) bi.The normalized RGB vectors are subsequently transformed in a block 320 into corresponding HDR color vectors 322 using the LUT 302 for the conversion from SDR to HDR:Hi= f(Vi, LSDR-to-HDR) = (r'ig'ib'i), where ∈ [0,1] (31)To validate the predictions, the HDR vectors 322 are inversely transformed in a block 330 into SDR vectors 332 using the current HDR-to-SDR LUT as follows:'ff 1 n~ / (^ ^HDR-to-SDR) — (32)

[0062] The impact of the transformations applied in the blocks 320, 310 is evaluated and quantified in blocks 340, 350, 360. In the example shown, the evaluation includes computing the Euclidean distance between the SDR vectors 316 and the inversely transformed SDR vectors 332 as follows:distances =√((ri- r''i)2+ (gi- g''i)2+ (bi- b''i)2) (33) Difference color vectors 342 are also computed. The vectors showing significant alteration (e.g., exceeding a selected fixed threshold) are identified for further analysis, e.g., using the following criterion:indices = {i| di stances [i] > 0.05} (34)

[0063] For each LUT node, operations of the block 350 are used to identify the colors that are closer to the colors of this node than any other nodes. Then, the number of problematic “indices” associated with these colors is calculated and saved as the parameter 'count':countj= |{i ∈ indices | Viis closest to nj}| (35) Operations of the block 350 also include computing the “frequency” 'count' of each matched reference vector and categorizing the vectors based on the computed frequencies:Categorize based on frequency thresholds, e.g., count < 400 and count > 400 (36)D24042W001

[0064] Operations of the block 310 include performing direct adjustments for each color channel at the specified indices, e.g., in accordance with the following formula:Lchannel[indices, channel] = V[indices, channel] - (H[indices] × V[indices, channel] × α)(37)In one example, a = 0.06. The “channel” can be one of the red, green, and blue channels. The operation represented by Eq. (37) adjusts the LUT’s prediction directly based on the scaled product of the differences between the original and modified image intensities.

[0065] A series of predefined offsets a> is applied to introduce non-linear adjustments, thereby enhancing the LUT’s adaptability and sensitivity:offsets = {0.5,0.95,0.05,0.85,0.15,0.75,0.25,0.65,0.35,0.90,0.10,0.80,0.20,0.70,0.30} (38)For each offset m, the nonlinear adjustment can be applied in the block 310, e.g., as follows:Lchannel[indices, channel] -= L[indices, channel] - ω × (H[indices] × α) ×(F [indices, channel] + 0.0) (39)The operation defined by Eq. (39) is configured to fine-tune the prediction adjustments by incorporating a cubic root function, thereby introducing a non-linear scale to the adjustments.

[0066] Next, we describe a process used to isolate and modify purple artifacts within images using a 3D-LUT and selective masking based on the color grid properties.

[0067] In one example, a mask, purpleMask, is created based on the intensity of the blue and green color channels:purpleMask=( (blueChannellntensity > 0.8) &( greenChannellntensity < 0.2 )) (40) The SDR image data at these indices are modified as follows:purpleMask(indices) = / / (indices) • purpleMask(indices) (41) The mask entries whose values are exactly “one” are reset as follows:purpleMask(purpleMask = 1) = 0 (42) Indices where significant modifications have occurred are identified as follows:significantindices = fmd(abs(purpleMask) > 0) (43)

[0068] Given the nonlinear adjustment of Eq. (39), we can now apply this formula to the indices identified as significant due to substantial modifications in the image data, asD24042W001quantified above. We denote these indices as “significantindices” and set a to 0.06 for this operation, where a is a selectable fixed value.

[0069] In one example, we apply the non-linear adjustment to the color channel at significant indices with the following settings:a = 0.06 (46)The adjustment is performed in accordance with Eq. (47):Lchannei[significantlndices, channel]— = (\jV[significantIndices, channel] — XH[significantlndices] x a) x (V[significantlndices, channel] + 0.0) (47) Note that Eq. (47) modifies the LUT values by applying cubic root scaling, thereby introducing a nonlinear effect tailored to the characteristics of the image at the positions indexed by significantindices. The cubic root function modifies the scale of adjustments based on the difference between the original and target color values, thereby enhancing the adaptability and sensitivity of the color transformation to accommodate specific image features.Example Improvements

[0070] This subsection describes example improvements achieved at different stages of the above-described four stages used to construct the forward 3D-LUT 120 according to one example. In this example, the four-stage process is applied to a uniform 33x33x33 color cube, wherein the indices are orderly transformed into a one-dimensional array, and the color values range from 0 to 1. As graphically and pictorially illustrated below, each of the four stages incrementally improves the accuracy of the LUT 120, thereby improving the color representation while reducing overall errors.

[0071] FIG. 4 graphically illustrates an initial error distribution 400 across the 33x33x33 color cube at Stage 1 of the LUT adjustment process used in the workflow 100 of FIG. 1 according to one example. Recall that, in Stage 1, we first establish a baseline LUT that maps the color cube into a usable format for further refinement. In the example shown, the reconstruction errors at this point are widely distributed along the ID index values (that run from 0 to 35,936) representing the 3D color cube. The magnitude of the errors is in the range from 0 to 0.6, with a significant number of errors notably being above the 0.2 level. This error range is representative of a typical preliminary mapping and sets the scale forD24042W001subsequent LUT refinements. The following mathematical expression approximately describes the error distribution 400:i = {e|0 < e < 0.6 and majority e > 0.2} (48)

[0072] FIG. 5 graphically illustrates a refined error distribution 500 across the 33x33x33 color cube at Stage 2 of the LUT adjustment process used in the workflow 100 of FIG. 1 according to one example. Recall that Stage 2 mainly focuses on reducing the spread of errors such that, upon completion, the majority of the color grid exhibits errors between 0 and 0.1, with a smaller residual portion still exhibits errors between 0.1 and 0.6. The abovedescribed adjustments implemented in Stage 2 beneficially enhance the precision of the LUT, thereby leading to a more-accurate color representation for the majority of the color grid, as evidenced by a visual comparison of the error distributions 400 and 500. The following mathematical expression approximately describes the error distribution 500:2= {e|0 < e < 0.6, with majority 0 < e < 0.1} (49)

[0073] FIG. 6 graphically illustrates a further-refined error distribution 600 across the 33x33x33 color cube after Stage 3 of the LUT adjustment process used in the workflow 100 of FIG. 1 according to one example. As described above, during Stage 3, the LUT undergoes further refinements, thereby achieving further improved (e.g., nearly perfect) color accuracy. More specifically, approximately one half of the color grid matches the respective target color with zero error, while the remainder shows relatively small errors, predominantly clustered near zero. As illustrated by the error distribution 600, Stage 3 performs well in terms of eliminating significant discrepancies and achieving a nearly optimal color representation. In the example show, the significant improvement achieved at Stage 3 is readily apparent from a visual comparison of the error distributions 600 and 500. The following mathematical expression approximately describes the error distribution 600:E3= {e|0 < e < 0.08, with approximately 50% e = 0} (50)

[0074] FIGS. 7A-7B pictorially illustrate reduction in visible artifacts after Stage 4 of the LUT adjustment process used in the workflow 100 of FIG. 1 according to one example. More specifically, FIG. 7A shows a roundtrip SDR image 150 generated using the forward LUT 120 obtained after Stage 3. FIG. 7B shows a roundtrip SDR image 150 generated using the forward LUT 120 obtained after Stage 4.D24042W001

[0075] It should be noted that Stage 4 does not produce a significant improvement in the numerical distribution of errors across the color grid compared to the error distribution 600 (FIG. 6). However, it does significantly reduce the occurrence of visually discernible artifacts, thereby significantly enhancing the visual quality of the resulting images 130, 150. For example, improvements associated with Stage 4 are readily apparent in the image areas marked by the two dashed-line boxes in FIG. 7A, where hair-outline and cheek-blush artifacts, respectively, are present in FIG. 7A but substantially absent in FIG. 7B. As already explained above, Stage 4 mainly focuses on fine-tuning the color adjustments directed at attenuating any residual SDR-to-HDR conversion impacts that might detrimentally affect the aesthetic aspects of the image.Video Delivery

[0076] FIG. 8 is a block diagram illustrating a video delivery system 800 in which various embodiments can be practiced according to some examples. The system includes a transmitter (e.g., a server) 802 and a receiver (e.g., a video player) 808 communicatively coupled via a communication link 806. In operation, the transmitter 802 generates a video bitstream 804, which is transmitted via the communication link 806 to the receiver 808 for playback.

[0077] The transmitter 802 is provided with a plurality of 3D-LUTs 120, each corresponding to a different respective set of DM parameters and having been previously generated using the above-described roundtrip SDR-HDR workflow 100 and the four-stage LUT adjustment process. Based on the actual or presumed display device used at the receiver 808, one of the plurality of 3D-LUTs 120 can be selected by a selection block 810 and provided to the SDR-to-HDR conversion pipeline of the transmitter 802, e.g., as indicated in FIG. 8. The selected 3D-LUT 120 is applied to an SDR video sequence 820 to generate a corresponding HDR video sequence 830. A video encoder 832 and a DM metadata injection module 834 are then used to generate, e.g., in a conventional manner, the video bitstream 804 having encoded therein the HDR video sequence 830 and the set of DM parameters corresponding to the selection made in the selection block 810.

[0078] The receiver 808 includes a DM metadata extraction module 836 and a video decoder configured to process, e.g., in a conventional manner, the received video bitstream 804 to recover therefrom (i) the DM metadata 837 inserted by the DM metadata injectionD24042W001module 832 and (ii) the HDR video sequence 830. Depending on the display device used at the receiver 808 and the settings thereof, the recovered HDR video sequence 830 can be played back in an HDR format or in an SDR format. For example, the display device can directly play the HDR video sequence 830 in the HDR format based on the settings defined by the DM metadata 837. Alternatively, when an SDR playback is needed, the receiver 808 operates to: (i) generate the backward 3D-LUT 140 based on the DM metadata 837; (ii) convert the HDR video sequence 830 into a corresponding roundtrip SDR video sequence 820' using the backward 3D-LUT 140; and (iii) play back the roundtrip SDR video sequence 820'.

[0079] FIG. 9 is a flowchart of a video delivery method 900 implemented in the video delivery system 800 according to some examples. The method 900 is described below with continued reference to FIGS. 1, 2, 8, and 9.

[0080] Operations of a block 902 of the method 900 include generating a plurality of first LUTs (e.g., LUTs 120, FIG. 1) corresponding to different respective sets of DM parameters used for changing a second dynamic range (e.g., HDR) to a first dynamic range (e.g., SDR) that is smaller than the second dynamic range. Each of the first LUTs is configured for converting images of the first dynamic range into images of the second dynamic range. The operations used for generating an individual first LUT include adjustment of the first LUT based on differences between first and second color vectors (e.g., 202 and 222, FIG. 2) of the first dynamic range, wherein the second color vector is generated by applying a second LUT (e.g., 140, FIG. 1) to a third color vector (e.g., 212, FIG. 2). The second LUT is configured for converting images of the second dynamic range into images of the first dynamic range. The third color vector is generated by applying the first LUT to the first color vector. In some examples, each of the first and second LUTs is a three-dimensional (3D) LUT, with each of the three dimensions thereof representing a different respective color of a color space (e.g., RGB space) used for the first and second videos.

[0081] In some examples, generating an individual first LUT includes sequentially generating first, second, third, and fourth approximations of the first LUT, e.g., in accordance with the above-described Stages 1-4, respectively.

[0082] In some examples, the first approximation of the first LUT is generated by inversing the second LUT.D24042W001

[0083] In some examples, the second approximation of the first LUT is generated by iteratively refining the first approximation via adjustment of each node value thereof based on a respective difference between the first and second color vectors. The respective difference can be quantified, e.g., using a Euclidean distance between the first and second color vectors in the color space used for the first and second videos. In a step of the iterative refining, determining a corresponding adjustment value to the node value includes (e.g., in accordance with Eq. (19)): (i) applying a smaller-than-one scaling factor to the Euclidean distance if the Euclidean distance is greater than a first threshold; and (ii) setting the corresponding adjustment value to zero if the Euclidean distance is smaller than a second threshold that is smaller than the first threshold. The smaller-than-one scaling factor may be incrementally decreased as the iterative refinement progresses to a next refinement step.

[0084] In some examples, the third approximation of the first LUT is generated by cascading a third LUT and the second approximation of the first LUT. The third LUT is configured to improve fidelity of image transformations directed at obtaining the third image from the first image and at obtaining the second image from the third image. The third LUT is iteratively refined via adjustment of each node value thereof based on a respective Euclidean distance between a pixel value of the first image and a corresponding pixel value of the second image obtained using the first, second, and third LUTs. In a step of iteratively refining the third LUT, determining a corresponding adjustment value to the node value includes setting the corresponding adjustment value to zero if the respective Euclidean distance is smaller than a fixed threshold value (e.g., in accordance with Eq. (28)).

[0085] In some examples, the fourth approximation of the first LUT is generated by adjusting node values of the cascaded third and first LUTs in a subset of nodes thereof identified by finding nodes for which respective Euclidean distances between corresponding ones of the first and second color vectors exceed a selected threshold value. For each node in the subset of nodes, a respective node value is adjusted based on a scaled product of the respective Euclidean distance between the corresponding ones of the first and second color vectors obtained using the cascaded third and first LUTs and the second LUT.

[0086] Operations of a block 904 of the method 900 include selecting one of the first LUTs from the plurality of first LUTs generated in the block 902. In some examples, such selection is based on a set of characteristics of the display used for video playback.D24042W001

[0087] Operations of a block 906 of the method 900 include converting a first video (e.g., 820, FIG. 8) having the first dynamic range into a second video (e.g., 830, FIG. 8) having the second dynamic range. The conversion is performed using the first LUT selected in the block 904.

[0088] Operations of a block 908 of the method 900 include generating a video bitstream (e.g., 804, FIG. 8) having encoded therein the second video and the metadata representing the pertinent set of DM parameters.

[0089] Operations of a block 910 of the method 900 include transmitting a communication signal carrying the video bitstream generated in the block 908 through a communication channel (e.g., 806, FIG. 8).

[0090] FIG. 10 is a block diagram illustrating a computing device 1000 one or more instances of which can be used in or in conjunction with the system 800 according to some examples. In some examples, the computing device 1000 can be used to implement the transmitter 802 or the receiver 808 (FIG. 8). In some examples, the computing device 1000 is programmed to implement at least some parts of the workflow 100 or the method 900 (FIG. 9).

[0091] The computing device 1000 of FIG. 10 is illustrated as having a number of components, but any one or more of these components may be omitted or duplicated, as suitable for the application and setting. In some embodiments, some or all of the components included in the computing device 1000 may be attached to one or more motherboards and enclosed in a housing. In some embodiments, some of those components may be fabricated onto a single system-on-a-chip (SoC) (e.g., the SoC may include one or more electronic processing devices 1002 and one or more storage devices 1004). Additionally, in various embodiments, the computing device 1000 may not include one or more of the components illustrated in FIG. 10, but may include interface circuitry for coupling to the one or more components using any suitable interface (e.g., a Universal Serial Bus (USB) interface, a High-Definition Multimedia Interface (HDMI) interface, a Controller Area Network (CAN) interface, a Serial Peripheral Interface (SPI) interface, an Ethernet interface, a wireless interface, or any other appropriate interface). For example, the computing device 1000 may not include a display device 1010, but may include display device interface circuitry (e.g., a connector and driver circuitry) to which an external display device 1010 may be coupled.D24042W001

[0092] The computing device 1000 includes a processing device 1002 (e.g., one or more processing devices). As used herein, the terms “electronic processor device” and “processing device” interchangeably refer to any device or portion of a device that processes electronic data from registers and / or memory to transform that electronic data into other electronic data that may be stored in registers and / or memory. In various embodiments, the processing device 1002 may include one or more digital signal processors (DSPs), application-specific integrated circuits (ASICs), central processing units (CPUs), graphics processing units (GPUs), server processors, or any other suitable processing devices.

[0093] The computing device 1000 also includes a storage device 1004 (e.g., one or more storage devices). In various embodiments, the storage device 1004 may include one or more memory devices, such as random-access memory (RAM) devices (e.g., static RAM (SRAM) devices, magnetic RAM (MRAM) devices, dynamic RAM (DRAM) devices, resistive RAM (RRAM) devices, or conductive-bridging RAM (CBRAM) devices), hard drive-based memory devices, solid-state memory devices, networked drives, cloud drives, or any combination of memory devices. In some embodiments, the storage device 1004 may include memory that shares a die with the processing device 1002. In such an embodiment, the memory may be used as cache memory and include embedded dynamic random-access memory (eDRAM) or spin transfer torque magnetic random-access memory (STT-MRAM), for example. In some embodiments, the storage device 1004 may include non-transitory computer readable media having instructions thereon that, when executed by one or more processing devices (e.g., the processing device 1002), cause the computing device 1000 to perform any appropriate ones of the methods disclosed herein below or portions of such methods.

[0094] The computing device 1000 further includes an interface device 1006 (e.g., one or more interface devices 1006). In various embodiments, the interface device 1006 may include one or more communication chips, connectors, and / or other hardware and software to govern communications between the computing device 1000 and other computing devices. For example, the interface device 1006 may include circuitry for managing wireless communications for the transfer of data to and from the computing device 1000. The term “wireless” and its derivatives may be used to describe circuits, devices, systems, methods, techniques, communications channels, etc., that may communicate data via modulated electromagnetic radiation through a nonsolid medium. The term does not imply that theD24042W001associated devices do not contain any wires, although in some embodiments they might not. Circuitry included in the interface device 1006 for managing wireless communications may implement any of a number of wireless standards or protocols, including but not limited to Institute for Electrical and Electronic Engineers (IEEE) standards including Wi-Fi (IEEE 802.11 family), IEEE 802.16 standards, Long-Term Evolution (LTE) project along with any amendments, updates, and / or revisions (e.g., advanced LTE project, ultramobile broadband (UMB) project (also referred to as “3GPP2”), etc.). In some embodiments, circuitry included in the interface device 1006 for managing wireless communications may operate in accordance with a Global System for Mobile Communication (GSM), General Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS), High Speed Packet Access (HSPA), Evolved HSPA (E-HSPA), or LTE network. In some embodiments, circuitry included in the interface device 1006 for managing wireless communications may operate in accordance with Enhanced Data for GSM Evolution (EDGE), GSM EDGE Radio Access Network (GERAN), Universal Terrestrial Radio Access Network (UTRAN), or Evolved UTRAN (E-UTRAN). In some embodiments, circuitry included in the interface device 1006 for managing wireless communications may operate in accordance with Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Digital Enhanced Cordless Telecommunications (DECT), Evolution-Data Optimized (EV-DO), and derivatives thereof, as well as any other wireless protocols that are designated as 3G, 4G, 5G, and beyond. In some embodiments, the interface device 1006 may include one or more antennas (e.g., one or more antenna arrays) configured to receive and / or transmit wireless signals.

[0095] In some embodiments, the interface device 1006 may include circuitry for managing wired communications, such as electrical, optical, or any other suitable communication protocols. For example, the interface device 1006 may include circuitry to support communications in accordance with Ethernet technologies. In some embodiments, the interface device 1006 may support both wireless and wired communication, and / or may support multiple wired communication protocols and / or multiple wireless communication protocols. For example, a first set of circuitry of the interface device 1006 may be dedicated to shorter-range wireless communications such as Wi-Fi or Bluetooth, and a second set of circuitry of the interface device 1006 may be dedicated to longer-range wireless communications such as global positioning system (GPS), EDGE, GPRS, CDMA, WiMAX, LTE, EV-DO, or others. In some other embodiments, a first set of circuitry of the interfaceD24042W001device 1006 may be dedicated to wireless communications, and a second set of circuitry of the interface device 1006 may be dedicated to wired communications.

[0096] The computing device 1000 also includes battery / power circuitry 1008. In various embodiments, the battery / power circuitry 1008 may include one or more energy storage devices (e.g., batteries or capacitors) and / or circuitry for coupling components of the computing device 1000 to an energy source separate from the computing device 1000 (e.g., to AC line power).

[0097] The computing device 1000 also includes a display device 1010 (e.g., one or multiple individual display devices). In various embodiments, the display device 1010 may include any visual indicators, such as a heads-up display, a computer monitor, a projector, a touchscreen display, a liquid crystal display (LCD), a light-emitting diode display, or a flat panel display.

[0098] The computing device 1000 also includes additional input / output (I / O) devices 1012. In various embodiments, the I / O devices 1012 may include one or more data / signal transfer interfaces, audio I / O devices (e.g., microphones or microphone arrays, speakers, headsets, earbuds, alarms, etc.), audio codecs, video codecs, printers, sensors (e.g., thermocouples or other temperature sensors, humidity sensors, pressure sensors, vibration sensors, etc.), image capture devices (e.g., one or more cameras), human interface devices (e.g., keyboards, cursor control devices, such as a mouse, a stylus, a trackball, or a touchpad), etc.

[0099] Depending on the specific embodiment, various components of the interface devices 1006 and / or I / O devices 1012 can be configured to output suitable control signals, receive suitable control / telemetry signals, and receive and transmit data streams. In some examples, the interface devices 1006 and / or I / O devices 1012 include one or more analog-to-digital converters (ADCs) for transforming received analog signals into a digital form suitable for operations performed by the processing device 1002 and / or the storage device 1004. In some additional examples, the interface devices 1006 and / or I / O devices 1012 include one or more digital-to-analog converters (DACs) for transforming digital signals provided by the processing device 1002 and / or the storage device 1004 into an analog form suitable for being transmitted through a communication channel.D24042W001

[0100] According to an example embodiment disclosed above, e.g., in the summary section and / or in reference to any one or any combination of some or all of FIGs. 1-10, provided is an apparatus comprising: at least one processor; and at least one memory including program code, wherein the at least one memory and the program code are configured to, with the at least one processor, cause the apparatus at least to: generate a first lookup table (LUT) configured for converting images of a first dynamic range into images of a second dynamic range that is larger than the first dynamic range, the generating including adjustment of the first LUT based on differences between first and second color vectors of the first dynamic range, the second color vector being generated by applying a second LUT to a third color vector, the second LUT being configured for converting images of the second dynamic range into images of the first dynamic range, the third color vector being generated by applying the first LUT to the first color vector; and use the first LUT to convert a first video having the first dynamic range into a second video having the second dynamic range.

[0101] According to another example embodiment disclosed above, e.g., in the summary section and / or in reference to any one or any combination of some or all of FIGs. 1-10, provided is a video delivery method comprising: generating a first lookup table (LUT) configured for converting images of a first dynamic range into images of a second dynamic range that is larger than the first dynamic range, the generating including adjustment of the first LUT based on differences between first and second color vectors of the first dynamic range, the second color vector being generated by applying a second LUT to a third color vector, the second LUT being configured for converting images of the second dynamic range into images of the first dynamic range, the third color vector being generated by applying the first LUT to the first color vector; and using the first LUT to convert a first video having the first dynamic range into a second video having the second dynamic range.

[0102] In some embodiments of the above method, each of the first and second LUTs is a three-dimensional LUT, with each of the three dimensions representing a different respective color of a color space used for the first and second videos.

[0103] In some embodiments of any of the above methods, the first dynamic range is a standard dynamic range; and wherein the second dynamic range is a high dynamic range.D24042W001

[0104] In some embodiments of any of the above methods, the method further comprises constructing the second LUT based on a set of display management (DM) parameters for changing the second dynamic range to the first dynamic range.

[0105] In some embodiments of any of the above methods, the method further comprises generating a video bitstream having encoded therein the second video and metadata representing the set of DM parameters.

[0106] In some embodiments of any of the above methods, the method further comprises transmitting a communication signal carrying the video bitstream through a communication channel.

[0107] In some embodiments of any of the above methods, the method further comprises repeating the constructing and the generating to produce a plurality of LUTs corresponding to different respective sets of DM parameters used for changing the second dynamic range to the first dynamic range.

[0108] In some embodiments of any of the above methods, the method further comprises selecting the first LUT from the plurality of LUTs based on a set of characteristics of a display used for video playback.

[0109] In some embodiments of any of the above methods, the generating comprises inversing the second LUT to obtain a first approximation of the first LUT.

[0110] In some embodiments of any of the above methods, the generating further comprises generating a second approximation of the first LUT by iteratively refining the first approximation via adjustment of each node value thereof based on a respective difference between the first and second color vectors.

[0111] In some embodiments of any of the above methods, the respective difference is quantified using a Euclidean distance between the first and second color vectors in a color space used for the first and second videos.

[0112] In some embodiments of any of the above methods, in a step of the iteratively refining, determining a corresponding adjustment value to the node value includes (e.g., in accordance with Eq. (19)): applying a smaller-than-one scaling factor to the Euclidean distance if the Euclidean distance is greater than a first threshold; and setting theD24042W001corresponding adjustment value to zero if the Euclidean distance is smaller than a second threshold that is smaller than the first threshold.

[0113] In some embodiments of any of the above methods, the smaller-than-one scaling factor is incrementally decreased as the iterative refinement progresses to a next step.

[0114] In some embodiments of any of the above methods, the generating further comprises generating a third approximation of the first LUT by cascading a third LUT and the second approximation of the first LUT, the third LUT being configured to improve fidelity of image transformations directed at obtaining the third image from the first image and at obtaining the second image from the third image.

[0115] In some embodiments of any of the above methods, the third LUT is iteratively refined via adjustment of each node value thereof based on a respective Euclidean distance between a pixel value of the first image and a corresponding pixel value of the second image obtained using the first, second, and third LUTs.

[0116] In some embodiments of any of the above methods, in a step of iteratively refining the third LUT, determining a corresponding adjustment value to the node value includes setting the corresponding adjustment value to zero if the respective Euclidean distance is smaller than a fixed threshold value (e.g., in accordance with Eq. (28)).

[0117] In some embodiments of any of the above methods, the generating further comprises generating a fourth approximation of the first LUT by adjusting node values of the cascaded third and first LUTs in a subset of nodes thereof identified by finding nodes for which respective Euclidean distances between corresponding ones of the first and second color vectors exceed a selected threshold value.

[0118] In some embodiments of any of the above methods, for each node in the subset of nodes, a respective node value is adjusted based on a scaled product of the respective Euclidean distance between the corresponding ones of the first and second color vectors obtained using the cascaded third and first LUTs and the second LUT.

[0119] A non-transitory computer-readable medium storing instructions that, when executed by an electronic processor, cause the electronic processor to perform operations comprising any of the above methods.D24042W001

[0120] With regard to the processes, systems, methods, heuristics, etc. described herein, it should be understood that, although the steps of such processes, etc. have been described as occurring according to a certain ordered sequence, such processes could be practiced with the described steps performed in an order other than the order described herein. It further should be understood that certain steps could be performed simultaneously, that other steps could be added, or that certain steps described herein could be omitted. In other words, the descriptions of processes herein are provided for the purpose of illustrating certain embodiments and should in no way be construed so as to limit the claims.

[0121] Accordingly, it is to be understood that the above description is intended to be illustrative and not restrictive. Many embodiments and applications other than the examples provided would be apparent upon reading the above description. The scope should be determined, not with reference to the above description, but should instead be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. It is anticipated and intended that future developments will occur in the technologies discussed herein, and that the disclosed systems and methods will be incorporated into such future embodiments. In sum, it should be understood that the application is capable of modification and variation.

[0122] All terms used in the claims are intended to be given their broadest reasonable constructions and their ordinary meanings as understood by those knowledgeable in the technologies described herein unless an explicit indication to the contrary is made herein. In particular, use of the singular articles such as “a,” “the,” “said,” etc. should be read to recite one or more of the indicated elements unless a claim recites an explicit limitation to the contrary.

[0123] The Abstract of the Disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, it can be seen that various features are grouped together in various embodiments for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments incorporate more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in fewer than all features of a single disclosed embodiment.D24042W001Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separately claimed subject matter.

[0124] While this disclosure includes references to illustrative embodiments, this specification is not intended to be construed in a limiting sense. Various modifications of the described embodiments, as well as other embodiments within the scope of the disclosure, which are apparent to persons skilled in the art to which the disclosure pertains are deemed to lie within the principle and scope of the disclosure, e.g., as expressed in the following claims.

[0125] Some embodiments may be implemented as circuit-based processes, including possible implementation on a single integrated circuit.

[0126] Some embodiments can be embodied in the form of methods and apparatuses for practicing those methods. Some embodiments can also be embodied in the form of program code recorded in tangible media, such as magnetic recording media, optical recording media, solid state memory, floppy diskettes, CD-ROMs, hard drives, or any other non-transitory machine-readable storage medium, wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the patented invention(s). Some embodiments can also be embodied in the form of program code, for example, stored in a non-transitory machine-readable storage medium including being loaded into and / or executed by a machine, wherein, when the program code is loaded into and executed by a machine, such as a computer or a processor, the machine becomes an apparatus for practicing the patented invention(s). When implemented on a general-purpose processor, the program code segments combine with the processor to provide a unique device that operates analogously to specific logic circuits.

[0127] Unless explicitly stated otherwise, each numerical value and range should be interpreted as being approximate as if the word “about” or “approximately” preceded the value or range.

[0128] The use of figure numbers and / or figure reference labels in the claims is intended to identify one or more possible embodiments of the claimed subject matter in order to facilitate the interpretation of the claims. Such use is not to be construed as necessarily limiting the scope of those claims to the embodiments shown in the corresponding figures.D24042W001

[0129] Although the elements in the following method claims, if any, are recited in a particular sequence with corresponding labeling, unless the claim recitations otherwise imply a particular sequence for implementing some or all of those elements, those elements are not necessarily intended to be limited to being implemented in that particular sequence.

[0130] Reference herein to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the disclosure. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments necessarily mutually exclusive of other embodiments. The same applies to the term “implementation.”

[0131] Unless otherwise specified herein, the use of the ordinal adjectives “first,” “second,” “third,” etc., to refer to an object of a plurality of like objects merely indicates that different instances of such like objects are being referred to, and is not intended to imply that the like objects so referred-to have to be in a corresponding order or sequence, either temporally, spatially, in ranking, or in any other manner.

[0132] Unless otherwise specified herein, in addition to its plain meaning, the conjunction “if’ may also or alternatively be construed to mean “when” or “upon” or “in response to determining” or “in response to detecting,” which construal may depend on the corresponding specific context. For example, the phrase “if it is determined” or “if [a stated condition] is detected” may be construed to mean “upon determining” or “in response to determining” or “upon detecting [the stated condition or event]” or “in response to detecting [the stated condition or event] ”

[0133] Also, for purposes of this description, the terms “couple,” “coupling,” “coupled,” “connect,” “connecting,” or “connected” refer to any manner known in the art or later developed in which energy is allowed to be transferred between two or more elements, and the interposition of one or more additional elements is contemplated, although not required. Conversely, the terms “directly coupled,” “directly connected,” etc., imply the absence of such additional elements.

[0134] As used herein in reference to an element and a standard, the term compatible means that the element communicates with other elements in a manner wholly or partially specified by the standard and would be recognized by other elements as sufficiently capableD24042W001of communicating with the other elements in the manner specified by the standard. The compatible element does not need to operate internally in a manner specified by the standard.

[0135] The functions of the various elements shown in the figures, including any functional blocks labeled as “processors” and / or “controllers,” may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared. Moreover, explicit use of the term “processor” or “controller” should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (DSP) hardware, network processor, application specific integrated circuit (ASIC), field programmable gate array (FPGA), read only memory (ROM) for storing software, random access memory (RAM), and nonvolatile storage. Other hardware, conventional and / or custom, may also be included. Similarly, any switches shown in the figures are conceptual only. Their function may be carried out through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, the particular technique being selectable by the implementer as more specifically understood from the context.

[0136] As used in this application, the terms “circuit,” “circuitry” may refer to one or more or all of the following: (a) hardware-only circuit implementations (such as implementations in only analog and / or digital circuitry); (b) combinations of hardware circuits and software, such as (as applicable): (i) a combination of analog and / or digital hardware circuit(s) with software / firmware and (ii) any portions of hardware processor(s) with software (including digital signal processor(s)), software, and memory(ies) that work together to cause an apparatus, such as a mobile phone or server, to perform various functions); and (c) hardware circuit(s) and or processor(s), such as a microprocessor s) or a portion of a microprocessor s), that requires software (e.g., firmware) for operation, but the software may not be present when it is not needed for operation.” This definition of circuitry applies to all uses of this term in this application, including in any claims. As a further example, as used in this application, the term circuitry also covers an implementation of merely a hardware circuit or processor (or multiple processors) or portion of a hardware circuit or processor and its (or their) accompanying software and / or firmware. The termD24042W001circuitry also covers, for example and if applicable to the particular claim element, a baseband integrated circuit or processor integrated circuit for a mobile device or a similar integrated circuit in server, a cellular network device, or other computing or network device.

[0137] It should be appreciated by those of ordinary skill in the art that any block diagrams herein represent conceptual views of illustrative circuitry embodying the principles of the disclosure. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudo code, and the like represent various processes which may be substantially represented in computer readable medium and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.

[0138] “BRIEF SUMMARY OF SOME SPECIFIC EMBODIMENTS” in this specification is intended to introduce some example embodiments, with additional embodiments being described in “DETAILED DESCRIPTION” and / or in reference to one or more drawings. “BRIEF SUMMARY OF SOME SPECIFIC EMBODIMENTS” is not intended to identify essential elements or features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

Claims

D24042W001CLAIMSWhat is claimed is:

1. A video delivery method, comprising:generating a first lookup table (LUT) configured for converting images of a first dynamic range into images of a second dynamic range that is larger than the first dynamic range, the generating including adjustment of the first LUT based on differences between first and second color vectors of the first dynamic range, the second color vector being generated by applying a second LUT to a third color vector, the second LUT being configured for converting images of the second dynamic range into images of the first dynamic range, the third color vector being generated by applying the first LUT to the first color vector; and using the first LUT to convert a first video having the first dynamic range into a second video having the second dynamic range.

2. The method of claim 1, wherein each of the first and second LUTs is a three-dimensional LUT, with each of the three dimensions representing a different respective color of a color space used for the first and second videos.

3. The method of claim 1 or 2,wherein the first dynamic range is a standard dynamic range; andwherein the second dynamic range is a high dynamic range.

4. The method of any preceding claim, further comprising constructing the second LUT based on a set of display management (DM) parameters for changing the second dynamic range to the first dynamic range.

5. The method of claim 4, further comprising generating a video bitstream having encoded therein the second video and metadata representing the set of DM parameters.

6. The method of claim 5, further comprising transmitting a communication signal carrying the video bitstream through a communication channel.D24042W0017. The method of claim 4, further comprising repeating the constructing and the generating to produce a plurality of LUTs corresponding to different respective sets of DM parameters used for changing the second dynamic range to the first dynamic range.

8. The method of claim 7, further comprising selecting the first LUT from the plurality of LUTs based on a set of characteristics of a display used for video playback.

9. The method of claim 4, wherein the generating comprises inversing the second LUT to obtain a first approximation of the first LUT.

10. The method of claim 9, wherein the generating further comprises generating a second approximation of the first LUT by iteratively refining the first approximation via adjustment of a node value thereof based on a respective difference between the first and second color vectors.

11. The method of claim 10, wherein the respective difference is quantified using a Euclidean distance between the first and second color vectors in a color space used for the first and second videos.

12. The method of claim 11, wherein, in a step of the iteratively refining, determining a corresponding adjustment value to the node value includes (e.g., in accordance with Eq. (19)):applying a smaller-than-one scaling factor to the Euclidean distance if the Euclidean distance is greater than a first threshold; andsetting the corresponding adjustment value to zero if the Euclidean distance is smaller than a second threshold that is smaller than the first threshold.

13. The method of claim 12, wherein the smaller-than-one scaling factor is incrementally decreased as the iterative refinement progresses to a next step.

14. The method of claim 10, wherein the generating further comprises generating a third approximation of the first LUT by cascading a third LUT and the second approximation of the first LUT, the third LUT being configured to improve fidelity of image transformationsD24042W001directed at obtaining the third image from the first image and at obtaining the second image from the third image.

15. The method of claim 14, wherein the third LUT is iteratively refined via adjustment of each node value thereof based on a respective Euclidean distance between a pixel value of the first image and a corresponding pixel value of the second image obtained using the first, second, and third LUTs.

16. The method of claim 15, wherein, in a step of iteratively refining the third LUT, determining a corresponding adjustment value to the node value includes setting the corresponding adjustment value to zero if the respective Euclidean distance is smaller than a fixed threshold value (e.g., in accordance with Eq. (28)).

17. The method of claim 14, wherein the generating further comprises generating a fourth approximation of the first LUT by adjusting node values of the cascaded third and first LUTs in a subset of nodes thereof identified by finding nodes for which respective Euclidean distances between corresponding ones of the first and second color vectors exceed a selected threshold value.

18. The method of claim 17, wherein, for each node in the subset of nodes, a respective node value is adjusted based on a scaled product of the respective Euclidean distance between the corresponding ones of the first and second color vectors obtained using the cascaded third and first LUTs and the second LUT.

19. A non-transitory computer-readable medium storing instructions that, when executed by an electronic processor, cause the electronic processor to perform operations comprising the method of any one of claims 1-18.

20. A video delivery apparatus, comprising:at least one processor; andat least one memory including program code,wherein the at least one memory and the program code are configured to, with the at least one processor, cause the apparatus at least to:D24042W001generate a first lookup table (LUT) configured for converting images of a first dynamic range into images of a second dynamic range that is larger than the first dynamic range, the generating including adjustment of the first LUT based on differences between first and second color vectors of the first dynamic range, the second color vector being generated by applying a second LUT to a third color vector, the second LUT being configured for converting images of the second dynamic range into images of the first dynamic range, the third color vector being generated by applying the first LUT to the first color vector; and use the first LUT to convert a first video having the first dynamic range into a second video having the second dynamic range.

Images