Intermediate Color Preservation for a Single-Layer Backward-Compatible Codec

Abstract

A novel method and system for processing a single-layer backward compatibility codec with multiple-channel multiple regression coefficients provided in or pointed to within metadata such that the coefficients are biased to prevent intermediate color shifts. The pseudo-intermediate color patches are used with saturation weighting coefficients to bias the coefficients.

Description

Technical Field

[0001] [Cross - Reference to Related Applications] This application claims the benefit of priority of European Patent Application No. 2217596.1 (Reference No.: D22021EP) and US Provisional Patent Application No. 63 / 345,161 (Reference No.: D22021USP1), both filed on May 24, 2020, which are hereby incorporated by reference in their entirety.

[0002] [Technical Field] This disclosure is generally related to images. More specifically, embodiments of the present invention relate to the preservation of intermediate colors in a single - layer backward - compatibility codec.

Background Art

[0003] As used in this specification, the term "dynamic range (DR)" may relate to the ability of the human visual system (HVS) to perceive the intensity (e.g., luminance, luma) range within an image, for example, from the darkest gray (black) to the brightest white (highlight). In this scene, DR relates to the "scene - reference" intensity. DR may also relate to the ability of a display device to appropriately or approximately render a specific width of intensity range. In this scene, DR relates to the "display - reference" intensity. At any point in the description of this specification, unless explicitly specified that a particular scene has a particular significance, it should be presumed that the term may be used interchangeably, for example, synonymously, in any scene.

[0004] As used in this specification, the term "dynamic range (DR)" may relate to the ability of the human visual system (HVS) to perceive the intensity (e.g., luminance, luma) range within an image from, for example, the darkest gray (black) to the brightest white (highlight). In this scene, the DR relates to the "scene reference" intensity. The DR may also relate to the ability of a display device to appropriately or approximately render a specific width of intensity range. In this scene, the DR relates to the "display reference" intensity. At any point in the description of this specification, unless explicitly specified that a particular scene has a particular importance, it should be presumed that the terms may be used synonymously in any scene, for example.

[0005] As used in this specification, the term "high dynamic range (HDR)" relates to a DR width that spans 14 to 15 times or more the magnitude of the human visual system (HVS). In fact, the DR that humans can simultaneously perceive over a wide range within the intensity range may be omitted in some way in relation to HDR. As used in this specification, the term "visual dynamic range (VDR)" or "enhanced dynamic range (EDR)" relates, individually or synonymously, to the DR perceivable within a scene or image by the human visual system (HVS) including eye movement, and enables light adaptation from any source to vary across the scene or image. As used in this specification, the EDR may relate to a DR that spreads to a magnitude of 5 to 6 digits. Therefore, although perhaps somewhat narrower compared to the actual scene called HDR, nevertheless, the VDR or EDR represents a wide DR width and may be called HDR.

[0006] In practice, an image includes one or more color components (e.g., luma Y, and chroma Cb and Cr), and each color component is represented with an accuracy of n bits per pixel (e.g., n = 8). For example, using gamma luminance coding, an image with n ≤ 8 (e.g., a 24-bit color JPEG image) is considered to be an image with a standard dynamic range. On the other hand, an image with n ≥ 10 may be considered to be an image with an extended dynamic range. HDR images may be stored and distributed using a high-definition (e.g., 16-bit) floating-point format such as the OpenEXR (trademark) file format developed by Industrial Light and Magic (trademark).

[0007] Most consumer desktop displays currently support a luminance of 200 - 300 cd / m 2 ² or nits. Most consumer HDTVs are in the range of 300 - 500 nits, and new models have reached 1000 nits (cd / m 2 ²). Such conventional displays exhibit characteristics of a lower dynamic range (LDR), also referred to as a standard dynamic range (SDR) with respect to HDR. As the availability of HDR content increases with advancements in both capture devices (e.g., cameras) and HDR displays (e.g., the Dolby Laboratories (trademark) PRM-4200 (trademark) professional reference monitor), HDR content may come to be displayed on HDR displays that support color grading and a higher dynamic range (e.g., 1000 nits to 5000 nits or higher).

[0008] When used in this specification, the term "PQ" represents Perceptual Quantization of Luminance Amplitude. The human visual system responds to increasing light levels in a highly non-linear way. A human's ability to perceive a stimulus is affected by the luminance of the stimulus, the size of the stimulus, the spatial frequency that makes up the stimulus, and the luminance level at which the eye is adapted at a particular instant while viewing the stimulus. In some embodiments, the perceptual quantization function maps linear input gray levels to output gray levels that better match the contrast sensitivity threshold in the human visual system. An exemplary PQ mapping function is described in SMPTE ST2084:2014, "High Dynamic Range EOTF of Mastering Reference Displays" (hereinafter "SMPTE"), which is hereby incorporated by reference in its entirety. Given a fixed stimulus size, for each luminance level (e.g., stimulus level, etc.), the minimum visible contrast step at that luminance level is selected according to the most sensitive adaptation level and the most sensitive spatial frequency at that luminance level (in accordance with the HVS model).

[0009] As used herein, multiple-channel multiple regression (MMR) refers to a method that enables an encoder to approximate / predict a higher dynamic range image (such as an HDR image) with respect to a given low dynamic range image (such as an SDR image) and an MMR model. Examples are provided in U.S. Patent No. 8,811,490, "Multiple Color Channel Multiple Regression Predictor" by Guan-Ming Su et al., which is hereby incorporated by reference in its entirety.

[0010] As used herein, single-layer backward compatible (SLBC) refers to a single-layer encoding system that supports both higher and lower dynamic range displays. This can be achieved using MMR coefficients provided in metadata, either directly (e.g., as data) or indirectly (e.g., a pointer to one of several pre-generated MMR models). Examples are provided in U.S. Patent No. 11,277,627 to Qing Song et al., "High-Fidelity Full Reference And High-Efficiency Reduced Reference Encoding In End-To-End Single-Layer Backward Compatible Encoding Pipeline", which is incorporated herein by reference in its entirety.

[0011] As used herein, "d3DMT" refers to a dynamic 3D mapping table. The d3DMT is constructed from HDR and SDR images to forward reshape chroma codewords in an HDR image to reshaped chroma codewords in an SDR image, to achieve a relatively high (e.g., highest) fidelity of perceived color. An example is shown in the aforementioned U.S. Patent No. 11,277,627.

[0012] WO2021 / 076822A1 discloses a method for encoding forward reconstructed image data in a video signal. The reverse reconstruction mapping table is first generated as the inverse of the forward reconstruction mapping table. The reverse reconstruction mapping table is updated by replacing the content map luminance codewords with the forward reconstructed luminance codewords generated by applying the luminance forward mapping to the sampled luminance codewords. The luminance forward mapping is composed of the forward reconstruction mapping table. The reverse reconstruction mapping table and the luminance forward mapping are used to generate a reverse reconstruction mapping for creating a reconstructed image from the forward reconstructed image. The forward reconstructed image is encoded in the video signal together with image metadata specifying the reverse reconstruction mapping. A receiving device of the video signal applies the reverse reconstruction mapping to the forward reconstructed image to create a reconstructed image of a second dynamic range.

[0013] The approaches described in this chapter are approaches that can be pursued, but are not necessarily approaches that have been previously devised or pursued. Accordingly, unless otherwise indicated, none of the approaches described in this chapter should be regarded as prior art merely by virtue of their inclusion in this chapter. Similarly, problems identified with respect to one or more approaches should not be assumed to have been recognized in any prior art based on this chapter unless otherwise indicated. SUMMARY OF THE INVENTION

[0014] The present invention is defined in the independent claims. The dependent claims relate to any features of some embodiments. Embodiments for achieving preservation of intermediate colors are constructing a dynamic 3D mapping table (d3DMT) from a first reference image and a second reference image, wherein the first reference image has a higher dynamic range than the second reference image; calculating a content color matrix based on the d3DMT; calculating content chroma from the d3DMT; Calculating at least one weighting coefficient from the content chroma; Constructing an intermediate color set; Calculating an intermediate color matrix based on the intermediate color set; Combining the content color matrix, the at least one weighting coefficient, and the intermediate color matrix to solve for multi-channel multiple regression (MMR) coefficients; Providing metadata including data related to the MMR coefficients to a decoder to enable backward compatibility of a single-layer bitstream; including a method comprising.

[0015] The method can be programmed / embedded in an encoder such as a codec.

[0016] The system and method are not limited to the above embodiments, and further details and embodiments are provided in the description and drawings provided herein.

Brief Description of the Drawings

[0017]

Figure 1

[0018]

Figure 2

[0019]

Figure 3

[0020]

Figure 4

[0021]

Figure 5

[0022]

Figure 6

[0023]

Figure 7

DETAILED DESCRIPTION OF THE INVENTION

[0024] As used herein, "intermediate color" refers to the center (0.5 on a normalized range from 0 to 1) of all color axes (e.g., Cb = 0.5 and Cr = 0.5 in the normalized YCbCr domain).

[0025] The SLBC algorithm includes two passes: a forward pass that maps an input high dynamic range image (e.g., HDR in this specification) to a low dynamic range image (e.g., SDR here), and a reverse pass that maps SDR to HDR. Figure 1 shows an example of SLBC based on using d3DMT to calculate the MMR coefficients used for image prediction in the SLBC codec. A dynamic 3D mapping table (130) is constructed using a reference HDR image (110) and a corresponding reference SDR image (120). From this, a matrix (described herein as A) and a vector (described herein as b) are calculated (140), which solve for the MMR coefficients (described herein as m) (150).

[0026] Construction of d3DMT Represent the i-th pixel of the reference HDR image frame for the three color channels (y, cb, cr) as (v i y , v i cb , v i cr ), and the corresponding reference SDR image as (s i y , s i cb , si cr ) is represented as (here the frame index t is removed but added back when solving the MMR coefficient). The pixel values are normalized to

[0001] , and there are P pixels in one image. Here, the forward path (from HDR to SDR) is used as an example, but the reverse path (from SDR to HDR) can also be derived by the same process.

[0027] The number of bins for each component is Q y , Q Cb , Q Cr is represented as. (Q y ×Q Cb ×Q Cr ) Calculate the 3D histogram. The bins of the 3D histogram of HDR are shown by the following formula:

Number

Number

[0028] For each channel, calculate the minimum value (L v,ch ) and the maximum value (H v,ch ) of the HDR image signal as follows:

Number

[0029] Each channel is uniformly quantized into Q ch bins based on the minimum value and the maximum value. The range of bin j is as follows:

Number

[0030] For a given input HDR value, the bin index is determined as follows:

Number

[0031] Calculate the HDR total for each HDR3D bin as follows:

Number

Number

[0032] An example of this operation is shown below.

[0033] STEP1: Initialization of the 3D SDR histogram and 3D mapped HDR chroma values:

Number

[0034] STEP2: Scan each pixel of HDR and SDR chroma from each of the P color patches (i).

Number

[0035] Here, find the 3D HDR histogram bins with a non-zero number of pixels. That is, discard all bins with no pixels.

Number

[0036] For each bin, for each of the K bins (i), the SDR mapping value: [Number] is set as follows: [Number]

[0037] All of the following mapping pairs: [Number] can be collected to construct d3DMT.

[0038] Configuration of the MMR coefficient The forward path is for conversion from HDR to SDR (from high dynamic range to low dynamic range), and the reverse path is for conversion from low dynamic range to high dynamic range. It is possible to use different MMR coefficients for the forward and reverse paths, or to specify the same MMR coefficient for both paths based on the more important path (for example, if the conversion from HDR to SDR is prioritized, only the forward path is calculated and used for both paths).

[0039] Forward path The i-th HDR entry of d3DMT in frame t: [Number] The MMR extended format of is shown as follows: [Number]

[0040] All K other than zero Ft Entries can be collected together as follows:

Number

[0041] The observed chroma signal (e.g., ch can be the Cb channel or the Cr channel) can be expressed as follows:

Number

[0042] The ch-th channel of the MMR coefficient is expressed as follows:

Number

Number

[0043] The optimal solution is obtained via the least squares solution:

Number

Number

[0044] To simplify the discussion, the following notation is further shown:

Number

Number

[0045] Reverse path The MMR coefficient of the reverse path can basically be calculated by the same process as the forward path. Using the predicted SDR image, construct d3DMT at frame t as follows:

Number

Number

[0046] All K other than 0 t B Grouping the entries gives the following:

Number

[0047] The observed chroma signal (e.g., ch can be the Cb channel or the Cr channel) can be expressed as follows:

Number

[0048] The ch-th channel of the MMR coefficient is expressed as follows:

Number

Number

[0049] The optimal solution is obtained via the least squares solution:

Number

[0050] This can be solved using the least squares solution algorithm.

Number

[0051] To simplify the discussion, the following notation is further presented:

Number

[0052] The above process generates an MMR coefficient that can bias the majority color of the reference image according to the color distribution. This can cause tinting of the intermediate colors, which means that the intermediate colors are not accurately preserved. Generally, the higher the saturation of the image, the higher the possibility of intermediate color shift. This can be corrected by introducing a set of virtual intermediate color patches into the MMR optimization procedure to provide an efficient solution to mitigate the problem of intermediate color shift.

[0053] Preservation of Intermediate Colors FIG. 2 shows an embodiment of an improved SLBC algorithm for addressing the problem of intermediate color shift. Similar to the previous process, an HDR image (210) and an SDR image (220) are used to create a d3DMT (230). However, two additional elements are added: 1) Calculate the chroma of the d3DMT (233), and calculate a matrix of weighting values w from its chroma (235). And 2) Construct a set of intermediate colors from the intermediate color patches added to the image (243), and calculate additional A and b matrices from those patches (245). Next, combine the additional matrices (w, A NC , b NC ) with the normal content color matrices (A CC and b CC ) (250), and solve for the MMR coefficient that is biased to retain the intermediate colors as a result above (260).

[0054] In other words, the left side of the flowcharts (210, 220, 230, 233, 235, 240) shows the "natural image" path (content color, or "CC"). Given a HDR-SDR natural image pair (210, 220), we first construct (230) a d3DMT. Based on the forward d3DMT, we calculate (240):

number

number

[0055] The right side of the flow chart (243, 245) shows the (added) neutral color (NC) pass. As explained here, we build (245):

number

number

[0056] In some embodiments, the system or method includes an intermediate color pass without weighting factors. In some embodiments, the system or method includes both a natural image pass and an intermediate color pass with weighting factors, as shown in FIG.

[0057] Next, for the forward and reverse paths separately, the following equations are used to combine the natural image and the intermediate color patches (250):

Number

[0058] The forward and reverse MMR coefficients can be solved as follows:

Number

[0059] To bias the optimization of the MMR coefficient towards the preservation of intermediate colors, the following matrix is modified:

Number

[0060] Forward path bias For the forward path, assign the i-th color patch, i = 0,..., N - 1, as follows:

Number

[0061] The MMR extended form of the i-th HDR intermediate color patch is as follows:

Number

[0062] All N color patches can be collected together as follows.

Number

[0063] The observed chroma signal (e.g., ch can be the Cb channel or the Cr channel) can be expressed as follows:

Number

[0064] Given the following matrix:

Number

Number

[0065] The MMR coefficient of the forward path can be solved as follows:

Number

[0066] Reverse path bias For the reverse path, assign the i-th color patch, i = 0,..., N - 1, as follows:

Number

[0067] The reverse path is the same process. Represent the corresponding reference SDR image as follows:

Number

Number

[0068] All N color patches can be collected together as follows.

Number

[0069] The observed chroma signal (e.g., ch can be the Cb channel or the Cr channel) can be expressed as follows:

Number

[0070] Given the following matrix:

Number

Number

[0071] The MMR coefficient of the reverse path can be solved as follows:

Number

[0072] Determination of the weighting coefficient As described in the previous section, it is necessary to determine the weighting coefficient (w t (F) ) of the forward path or the weighting coefficient (w t (B) ) of the reverse path. As shown here, an empirical method (experiment) can be used to find the optimal solution.

[0073] Deviation between content chroma and intermediate color The chroma of the content plays a role in determining the weighting coefficient. Note that in the YCbCr domain, the intermediate color has (0.5, 0.5) (Cb, Cr) values. For each color entry of d3DMT:

Number

Number

[0074] This is the saturation of one entry. All valid non-zero K t F For a cube, the average saturation can be calculated as follows: [Number]

[0075] The average saturation (β (F) ) of the full-grid RGB color points uniformly sampled in a specific color space (such as R.709 / P3 / R.2020) can be determined within the container (e.g., R.2020). The wider the color gamut, the higher the average saturation.

[0076] To check the influence of different values of the weighting factor, place a gray-level bar containing N b pixels at a known location in each test image. Since the location is known, measure the deviation from the intermediate color of the gray-level bar between the finally reconstructed HDR image and the intermediate color point 0.5. Adopt the mean of absolute difference (MAD) of the chroma channel of the intermediate color patch.

[0077] To simplify the design, set w t = w t (F) = w t (B) When applying the weighting factor as w t , the finally reconstructed HDR pixel values are expressed as follows: [Number] For the pixels in the gray-level bar, MAD can be calculated as follows: [Number]

[0078] d t ch (w t ) The larger the value of (w

[0079] When the following is applied and:

Number

Number

Number

[0080] Negative values actually mean that the chroma accuracy is improved and the distortion is small. Positive values mean that the chroma accuracy is reduced by the introduction of the weighting factor.

[0081] Data analysis - Empirical method for determining the weighting factor First, consider the case where the intermediate color is not saved, that is, when w t is set to 0. Figure 3 shows an example of the correlation between the content chroma β t (F) and the distortion of the gray level bar d t ch (0) of both channels. The content chroma plays a role in affecting the deviation of the intermediate color. The higher the content chroma (β t (F) ), the higher the chroma distortion (d t ch (0)), and the higher the possibility of showing artifacts other than the intermediate color. Note that in this example, the image is of 12-bit accuracy.

[0082] For each t-test image, search for the minimum value of the weighting factor so that the deviation of the intermediate color is smaller than the threshold δ: [Number]

[0083] Figure 4 shows an example of w required for δ = 8 (12-bit precision) versus content chroma β. t (F) for t opt Note that the higher the chroma of the content, the higher the weighting factor required to retain intermediate colors. In some situations, as in Figure 4 where the chroma is less than 0.1, there is no need to retain intermediate colors (i.e., w t opt = 0). In other situations, different weighting values may be required. For example, as in Figure 4, a weighting value of 5 can be used for chroma values less than 0.13, but higher chroma values may require a weighting value of 10 or 15.

[0084] Artifacts other than intermediate colors are rarely visible in images depicting natural scenes, but are more frequently detected in highly saturated synthetic datasets, particularly in certain color spaces (e.g., R.2020).

[0085] Method for selecting the adopted weighting factor Using a plot of chroma versus minimum W (e.g., Figure 3), a piecewise linear function can be found that envelopes / upper bounds the data dots. When measuring the content chroma β t (F) the required weighting factor β t (F) can be determined. An example is shown below: [Number]

[0086] Or, it can simply be represented as a clipping function: [Number]

[0087] Figure 5 shows this function (510) as a line clipped below 0.05 and above 0.2. Different profiles can generate different curves / functions.

[0088] Figure 6 shows the MAD, d t = f(β t (F) ) using the piecewise linear model w t ch (w t ) for each frame. The MAD is within the tolerance (δ = 8) at 12-bit precision. The subjective test reveals that this small intermediate color shift is not visible.

[0089] Figure 7 shows the MAD, D t = f(β t (F) ) using the optimal w t ch (w t ) for each frame. As shown in Figure 7, most frames show negative values, which means that the chroma is actually improved by this intermediate color preservation method. There are few frames where D t ch (w t ) is greater than 0, but they are still less than (δ = 8). The performance degrades in terms of MAD, but it is within the invisible range.

[0090] In some embodiments of this specification, the weighting factor (W) is multiplied by a plurality of intermediate color patches (N). In some embodiments, N can be fixed and W can be adjusted to provide intermediate color preservation. In other embodiments, W can be fixed and the number of intermediate color patches can be increased (e.g., to W * N). Typically, fixing N results in less computational load. Fixing W bypasses the step of determining W, and fixing W to 1 eliminates the need to multiply W by N in the calculation.

[0091] Hardware embodiments One embodiment of the system is as part of a codec used in an encoder-decoder system. The MMR coefficients can be resolved in an encoder (a processor / machine configured to receive an input of an image (individual and / or video) and output a data stream including the encoded image and metadata) using metadata transmitted to a decoder that includes the coefficients themselves or a pointer to one of a predetermined set of coefficients stored in the decoder. For example, the decoder can include a library of MMR coefficient sets that cover various expected bias levels, and the metadata notifies the decoder of the set to use for compatibility.

[0092] As described herein, embodiments of the present invention may thus be related to one or more of the exemplary embodiments listed below. Thus, the present invention may be embodied in any of the forms described herein including, but not limited to, the following exemplary embodiments (EEEs) that describe the structure, features, and functions of some parts of the present invention.

[0093] (EEE1) A method of preserving intermediate colors and encoding image data, the method comprising: constructing a dynamic 3D mapping table (d3DMT) from a first reference image and a second reference image, the first reference image having a different dynamic range from the second reference image; calculating a content color matrix based on the d3DMT; calculating content chroma from the d3DMT; calculating at least one weighting coefficient from the content chroma; constructing a set of intermediate colors; calculating an intermediate color matrix based on the set of intermediate colors; combining the content color matrix, the at least one weighting coefficient, and the intermediate color matrix to resolve a multi-channel multiple regression (MMR) coefficient. To enable backward compatibility of the single-layer bitstream, providing metadata including data related to the MMR coefficients to a decoder; A method including the above.

[0094] (EEE2) The method is configured to provide MMR coefficients including data related to both forward-path MMR coefficients and backward-path MMR coefficients, and the forward-path MMR coefficients are determined independently of the backward-path MMR coefficients. The method according to EEE1.

[0095] (EEE3) The method is configured to provide MMR coefficients including data related to only backward-path MMR coefficients. The method according to EEE1.

[0096] (EEE4) The content color matrix is in the form of A and b, where A is based on a vector of non-zero bins and b is based on a vector of non-zero bins and an observed chroma signal. The method according to any one of EEE1 to EEE3.

[0097] (EEE5) The MMR coefficients are evaluated using a least-squares solution algorithm. The method according to any one of EEE1 to EEE4.

[0098] (EEE6) The MMR is in the form of A -1 b. The method according to EEE5.

[0099] (EEE7) The step of combining includes multiplying one of the intermediate color matrices by one of the at least one weight coefficient and adding the product to the corresponding one of the content color matrices. The method according to any one of EEE1 to EEE6.

[0100] (EEE8) The method according to any one of EEE1 to 7, wherein the step of constructing the intermediate color set includes a step of constructing a plurality of pseudo-intermediate color patches that do not exist in either the first reference image or the second reference image.

[0101] (EEE9) The method according to any one of EEE1 to 8, wherein the at least one weighting coefficient includes a forward pass weighting coefficient and a backward pass weighting coefficient.

[0102] (EEE10) The method according to any one of EEE1 to 9, wherein the metadata includes a pointer to a set of MMR coefficients stored in the decoder.

[0103] (EEE11) The method according to any one of EEE1 to 10, wherein the step of calculating the at least one weighting coefficient includes a step of comparing the content chroma with experimental data.

[0104] (EEE12) The method according to EEE11, further including a step of setting a linear function that envelopes data points in the experimental data.

[0105] (EEE13) An encoder configured to execute the method according to any one of EEE1 to 12, the encoder including a processor, a signal input configured to receive the image, a signal output configured to transmit the metadata, and an encoder including the above.

[0106] (EEE14) A method of storing intermediate colors and encoding image data, the method including a step of constructing a dynamic 3D mapping table (d3DMT) from a first reference image and a second reference image, wherein the first reference image has a different dynamic range from the second reference image, a step of calculating a content color matrix based on the d3DMT, a step of constructing an intermediate color set, Based on the intermediate color set, calculating an intermediate color matrix; Combining the content color matrix and the intermediate color matrix to solve multi-channel multiple regression (MMR) coefficients; Providing metadata including data related to the MMR coefficients to a decoder to enable backward compatibility of a single-layer bitstream; A method comprising.

[0107] <Equivalents, Extensions, Alternatives and Miscellaneous> Exemplary embodiments regarding color conversion under coding efficiency constraints for coding of HDR video are thus described. In the foregoing specification, embodiments of the invention have been described with reference to numerous specific details that may vary from implementation to implementation. Accordingly, the sole and exclusive indication of what the invention is, and what the applicant intends the invention to be, is set forth in the claims, which are issued in a specific form including any later corrections. Any definition explicitly set forth in this specification for terms included in such claims should govern the meaning of such terms as used in the claims. Accordingly, no limitation, element, feature, advantage, or attribute not explicitly recited in the claims should in any way limit the scope of the claims. The specification and drawings are accordingly to be regarded in an illustrative rather than a limiting sense.

Claims

1. A method for encoding image data while preserving intermediate colors, the method comprising: constructing a dynamic 3D mapping table (d3DMT) from a first reference image and a second reference image, the first reference image having a different dynamic range from the second reference image; calculating a content color matrix based on the d3DMT; calculating a content chroma from the d3DMT; calculating at least one weighting coefficient from the content chroma; constructing a set of intermediate colors including a plurality of intermediate color image patches; calculating an intermediate color matrix based on the set of intermediate colors; combining the content color matrix, the at least one weighting coefficient, and the intermediate color matrix to solve for multi-channel multiple regression (MMR) coefficients; providing metadata including data related to the MMR coefficients to a decoder to enable backward compatibility of a single-layer bitstream; A method comprising the above steps.

2. The method according to claim 1, wherein the step of constructing a dynamic 3D mapping table (d3DMT) from a first reference image and a second reference image includes determining 3D histogram bins having a non-zero number of pixels.

3. The content color matrix is A CC and b CC in the form of, A CC is based on the vector of all non-zero bins, b CC is based on the vector of all non-zero bins and the observed chroma signal, the method according to claim 1.

4. The method according to claim 1, wherein the step of constructing the set of intermediate colors includes generating an intermediate color patch vector that collects a plurality of intermediate color patches, and in the intermediate color patches, the luminance (Y) value is within [0, 1) and the chroma values (Cb, Cr) are fixed at 0.

5.

5. The intermediate color matrix is A NC and b NC in the form of, and A NC is based on the intermediate color patch vector, and b NC is based on the intermediate color patch vector and the observed chroma signal of the intermediate color represented by the center of all color axes. The method according to claim 1.

6. The method according to claim 1, wherein the combining step includes multiplying one of the intermediate color matrices by one of the at least one weighting coefficient and adding the product to the corresponding one of the content color matrices.

7. The method according to claim 1, configured to provide MMR coefficients including data related to both forward path MMR coefficients and backward path MMR coefficients, the forward path MMR coefficients being determined independently of the backward path MMR coefficients.

8. The method according to claim 1, configured to provide MMR coefficients including data related to only the backward path MMR coefficients.

9. The method according to claim 1, wherein the MMR coefficients are determined using a least squares solution algorithm.

10. The MMR coefficient is A -１ The method according to claim 9, which is in the form of b.

11. The method according to claim 1, wherein the at least one weighting factor includes a forward path weighting factor and a reverse path weighting factor.

12. The method according to claim 1, wherein the metadata includes a pointer to a set of MMR coefficients stored in the decoder.

13. The method according to claim 1, wherein the step of calculating the at least one weighting factor includes comparing the content chroma with experimental data.

14. The method according to claim 13, wherein the step of calculating the at least one weighting factor from the content chroma includes finding a piecewise linear function for enveloping data points in the experimental data, and the piecewise linear function maps the measured content chroma to the at least one weighting factor.

15. An encoder configured to execute the method according to claim 1, the encoder comprising: a processor; a signal input configured to receive the image; a signal output configured to transmit the metadata; and an encoder including the above.

Images