Preserving neutral colors for single-layer backward compatible codecs

Abstract

A novel method and system for processing a single-layer backward compatibility codec with multiple-channel multiple regression coefficients provided in or pointed to within metadata such that the coefficients are biased to prevent intermediate color shifts. The pseudo-intermediate color patches are used with saturation weighting coefficients to bias the coefficients.

Description

[Technical Field]

[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application claims the benefit of priority to European Patent Application No. 2217596.1 (Reference No. D22021EP) and U.S. Provisional Patent Application No. 63 / 345,161 (Reference No. D22021USP1), both filed on May 24, 2020, which are incorporated herein by reference in their entireties.

[0002] [Technical field] FIELD OF THE DISCLOSURE This disclosure relates generally to images, and more particularly to preserving neutral colors in single-layer backward-compatible codecs. [Background technology]

[0003] As used herein, the term "dynamic range (DR)" may relate to the ability of the human visual system (HVS) to perceive a range of intensities (e.g., luminance, luma) within an image, e.g., from darkest gray (black) to brightest white (highlight). In this scenario, DR relates to "scene-referred" intensity. DR may also relate to the ability of a display device to properly or approximately render an intensity range of a particular width. In this scenario, DR relates to "display-referred" intensity. At any point in the description herein, unless it is explicitly specified that a particular scene has particular importance, it should be presumed that the terms may be used synonymously for either scene, e.g.,

[0004] As used herein, the term "dynamic range (DR)" may relate to the ability of the human visual system (HVS) to perceive a range of intensities (e.g., luminance, luma) within an image, e.g., from darkest gray (black) to brightest white (highlight). In this scenario, DR relates to "scene-referred" intensity. DR may also relate to the ability of a display device to properly or approximately render an intensity range of a particular width. In this scenario, DR relates to "display-referred" intensity. At any point in the description herein, unless it is explicitly specified that a particular scene has particular importance, it should be presumed that the terms may be used synonymously for either scene, e.g.,

[0005] As used herein, the term "high dynamic range (HDR)" refers to a DR width that spans 14-15 times or more the magnitude of the human visual system (HVS). Indeed, DR, where humans can simultaneously perceive a wide range of intensities, may be omitted in some manner in connection with HDR. As used herein, the terms "visual dynamic range (VDR)" or "enhanced dynamic range (EDR)," individually or synonymously, refer to the DR perceivable within a scene or image by the human visual system (HVS), including eye movements, and allowing for light adaptation to vary across a scene or image. As used herein, EDR may refer to a DR that spans 5-6 orders of magnitude. Thus, although perhaps somewhat narrower than the actual scene referred to as HDR, VDR or EDR may nevertheless represent a wide DR width and be referred to as HDR.

[0006] In practice, an image comprises one or more color components (e.g., luma Y, and chroma Cb and Cr), each represented with n bits of precision per pixel (e.g., n=8). For example, using gamma luminance coding, an image with n≦8 (e.g., a color 24-bit JPEG image) may be considered a standard dynamic range image, while an image with n≧10 may be considered an extended dynamic range image. HDR images may be stored and distributed using high-definition (e.g., 16-bit) floating-point formats such as the OpenEXR™ file format developed by Industrial Light and Magic™.

[0007] Most consumer desktop displays currently have a brightness of 200-300 cd / m 2 Most consumer HDTVs are in the 300-500 nits range, with newer models capable of 1000 nits (cd / m 2 ). Such conventional displays exhibit lower dynamic range (LDR), also known as standard dynamic range (SDR) as opposed to HDR. As the availability of HDR content increases due to advances in both capture devices (e.g., cameras) and HDR displays (e.g., Dolby Laboratories™ PRM-4200™ Professional Reference Monitor), HDR content can be color graded and displayed on HDR displays that support a higher dynamic range (e.g., 1000 nits to 5000 nits, or higher).

[0008] The term "PQ" as used herein stands for perceptual luminance amplitude quantization. The human visual system responds to increasing light levels in a highly nonlinear manner. A human's ability to see a stimulus is affected by the luminance of the stimulus, the size of the stimulus, the spatial frequencies that make up the stimulus, and the luminance level to which the eye is adapted at a particular moment. In some embodiments, a perceptual quantization function maps linear input gray levels to output gray levels that better match the contrast sensitivity threshold in the human visual system. Exemplary PQ mapping functions are described in SMPTE ST2084:2014 "High Dynamic Range EOTF of Mastering Reference Displays" (hereinafter "SMPTE"), which is incorporated herein by reference in its entirety, and which indicate that, given a fixed stimulus size, for each luminance level (e.g., stimulus level, etc.), the minimum visible contrast step at that luminance level is selected (according to the HVS model) according to the adaptation level and the spatial frequency to which the luminance level is most sensitive.

[0009] As used herein, multiple-channel multiple regression (MMR) refers to a method that allows an encoder to approximate / predict a higher dynamic range image (such as HDR) given a low dynamic range image (such as SDR) and an MMR model. An example is provided in U.S. Patent No. 8,811,490, "Multiple Color Channel Multiple Regression Predictor," by Guan-Ming Su et al., which is incorporated herein by reference in its entirety.

[0010] As used herein, single-layer backward compatible (SLBC) refers to a single-layer coding system that supports both higher and lower dynamic range displays. This can be achieved using MMR coefficients provided in metadata either directly (e.g., coefficients as data) or indirectly (e.g., a pointer to one of several pre-generated MMR models). An example is provided in U.S. Pat. No. 11,277,627, "High-Fidelity Full Reference And High-Efficiency Reduced Reference Encoding In End-To-End Single-Layer Backward Compatible Encoding Pipeline," by Qing Song et al., which is incorporated herein by reference in its entirety.

[0011] As used herein, "d3DMT" refers to a dynamic 3D mapping table. A d3DMT is constructed from an HDR image and an SDR image to forward reshape chroma codewords in the HDR image to reshape chroma codewords in the SDR image, achieving relatively high (e.g., highest) fidelity of perceived color. An example is shown in the aforementioned U.S. Patent No. 11,277,627.

[0012] WO2021 / 076822A1 discloses a method for encoding forward-reshaped image data in a video signal. A backward-reshaping mapping table is first generated as the inverse of the forward-reshaping mapping table. The backward-reshaping mapping table is updated by replacing the content-mapped luma codeword with a forward-reshaped luma codeword generated by applying a luma forward mapping to a sampled luma codeword. The luma forward mapping is composed of the forward-reshaping mapping table. The backward-reshaping mapping table and the luma forward mapping are used to generate a backward-reshaping mapping for creating a reconstructed image from the forward-reshaped image. The forward-reshaped image is encoded in the video signal along with image metadata specifying the backward-reshaping mapping. A receiver of the video signal applies the backward-reshaping mapping to the forward-reshaped image to create a reconstructed image in a second dynamic range.

[0013] The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Thus, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section. Similarly, problems identified with one or more approaches should not be assumed to have been recognized in any prior art under this section unless otherwise indicated. Summary of the Invention

[0014] The present invention is defined in the independent claims. The dependent claims relate to optional features of some embodiments. An embodiment for achieving neutral color preservation comprises: constructing a dynamic 3D mapping table (d3DMT) from a first reference image and a second reference image, the first reference image having a higher dynamic range than the second reference image; calculating a content color matrix based on the d3DMT; calculating content saturation from the d3DMT; calculating at least one weighting factor from said content saturation; constructing a set of intermediate colors; calculating an intermediate color matrix based on the intermediate color set; combining the content color matrix, the at least one weighting coefficient, and the intermediate color matrix to solve for multi-channel multiple regression (MMR) coefficients; providing metadata to a decoder, the metadata including data related to said MMR coefficients, to enable backward compatibility of a single layer bitstream; The method includes:

[0015] The method can be programmed / embedded in an encoder such as a codec.

[0016] The systems and methods are not limited to the above embodiments, and further details and embodiments are provided in the description and figures provided herein. [Brief explanation of the drawings]

[0017] [Figure 1] 1 shows an example of a prior art SLBC codec without midtone preservation.

[0018] [Figure 2] Here is an example of an SLBC codec with midtone preservation:

[0019] [Figure 3] 1 shows an example chart of color distortion (average) vs. chroma saturation (d3DMT) for Cb and Cr channels.

[0020] [Figure 4] 10 shows an example of saturation versus minimum weighting factors for the Cb and Cr channels.

[0021] [Figure 5] An example of a clipping function derived from FIG.

[0022] [Figure 6] 10 shows an example graph of mid-tone area MAD using optimal weighting factors for Cb and Cr.

[0023] [Figure 7] 10 shows an example graph of the global image region MAD using optimal weighting factors for Cb and Cr. DETAILED DESCRIPTION OF THE INVENTION

[0024] As used herein, "neutral" refers to the center (0.5 on the normalized range from 0 to 1) of all color axes (e.g., Cb=0.5 and Cr=0.5 in the normalized YCbCr domain).

[0025] The SLBC algorithm includes two passes: a forward pass that maps an input high dynamic range image (e.g., HDR herein) to a low dynamic range image (e.g., SDR herein), and a backward pass that maps SDR to HDR. Figure 1 shows an example of SLBC based on using d3DMT to calculate MMR coefficients used in image prediction of the SLBC codec. A reference HDR image (110) and a corresponding reference SDR image (120) are used to construct a dynamic 3D mapping table (130). From this, a matrix (referred to herein as A) and a vector (referred to herein as b) are calculated (140), which solve for the MMR coefficients (referred to herein as m) (150).

[0026] Construction of d3DMT The i-th pixel of the reference HDR image frame is calculated for the three color channels (y, cb, cr) as follows: i y ,v i cb ,v i cr ) and the corresponding reference SDR image is denoted as (s i y ,s i cb ,si cr ) (here the frame index t is removed, but added back when solving the MMR coefficients). The pixel values ​​are normalized to 0001, and there are P pixels in an image. Here we use the forward pass (HDR to SDR) as an example, but the backward pass (SDR to HDR) can also be derived using the same process.

[0027] The number of bins for each component is Q y , Q Cb , Q Cr (Q y ×Q Cb ×Q Cr ) Calculate the 3D histogram. The bins of the HDR 3D histogram are given by:

number

number

[0028] For each channel, the minimum value of the HDR image signal (L v,ch ) and maximum value (H v,ch ) is calculated as follows:

number

[0029] Each channel is Q-scaled based on the minimum and maximum values. ch bins, where the range of bin j is:

number

[0030] For a given input HDR value, the bin index is determined by:

number

[0031] Calculate the HDR sum for each HDR3D bin as follows:

number

number

[0032] An example of this operation is:

[0033] STEP 1: Initializing the 3D SDR histogram and 3D mapped HDR chroma values:

number

[0034] STEP 2: Scan each pixel of HDR and SDR chroma from each of the P color patches (i).

number

[0035] Now, find the 3D HDR histogram bins that have a non-zero number of pixels, i.e., discard all bins that have no pixels.

number

number

[0036] For each bin, for each of the K bins (i), the SDR mapping value:

number

number

[0037] All of the following mapping pairs:

number

[0038] MMR Coefficient Configuration The forward pass is for converting from HDR to SDR (high dynamic range to low dynamic range), and the reverse pass is for converting from low dynamic range to high dynamic range. You can use different MMR coefficients for the forward and reverse passes, or you can specify the same MMR coefficients for both passes based on which pass is more important (e.g., if HDR to SDR conversion is prioritized, only the forward pass is calculated and used for both passes).

[0039] Forward Pass The i-th HDR entry of the d3DMT at frame t:

number

number

[0040] All non-zero K Ft Entries can be collected together as follows:

number

[0041] The observed chroma signal (e.g., ch can be the Cb channel or the Cr channel) can be expressed as:

number

[0042] The ch-th channel of the MMR coefficients is expressed as:

number

number

[0043] The optimal solution is obtained via a least squares solution:

number

number

[0044] To simplify the discussion, we also use the following notation:

number

number

[0045] Reverse Pass The MMR coefficients for the backward pass can be calculated using essentially the same process as for the forward pass. Using the predicted SDR image, construct the d3DMT at frame t as follows:

number

number

[0046] All K's except 0 t B The summary of the entries is as follows:

number

[0047] The observed chroma signal (e.g., ch can be the Cb channel or the Cr channel) can be expressed as:

number

[0048] The ch-th channel of the MMR coefficients is expressed as:

number

number

[0049] The optimal solution is obtained via a least squares solution:

number

[0050] This can be solved using a least squares solution algorithm.

number

[0051] To simplify the discussion, we also use the following notation:

number

[0052] The above process generates MMR coefficients that can bias the majority of colors in the reference image depending on the color distribution. This can cause mid-tone tinting, which means that mid-tones are not accurately preserved. In general, the more saturated an image is, the higher the likelihood of mid-tone shift. This can be corrected by introducing a set of virtual mid-tone patches into the MMR optimization procedure to provide an efficient solution for mitigating the mid-tone shift problem.

[0053] Preserving Neutral Tones Figure 2 shows an embodiment of an improved SLBC algorithm that addresses the issue of midtone shift. Similar to the previous process, an HDR image (210) and an SDR image (220) are used to create a d3DMT (230). However, two additional elements are added: 1) the saturation of the d3DMT is calculated (233) and a matrix of weighting values ​​w is calculated from the saturation (235). And 2) a midtone set is constructed from midtone patches added to the image (243) and additional A and b matrices are calculated from the patches (245). The additional matrices (w,A NC ,b NC ) into the normal content color matrix (A CC and b CC ) (250) to resolve MMR coefficients (260), which are biased to preserve neutral colors as a result of the above.

[0054] In other words, the left side of the flowchart (210, 220, 230, 233, 235, 240) shows the "natural image" path (content color, or "CC"). Given an HDR-SDR natural image pair (210, 220), we first construct a d3DMT (230). Based on the forward d3DMT, we calculate (240):

number

number

[0055] The right side of the flowchart (243, 245) shows the (added) neutral color (NC) pass, which we build (245) as follows:

number

number

[0056] In some embodiments, the system or method includes an intermediate color pass without weighting factors. In some embodiments, the system or method includes both a natural image pass and an intermediate color pass with weighting factors, as shown in FIG.

[0057] Next, we combine the natural image and the neutral color patches (250) using the following formula for the forward and backward passes separately:

number

[0058] The forward and reverse MMR coefficients can be solved as follows:

number

[0059] To bias the optimization of the MMR coefficients towards preserving midtones, modify the following matrix:

number

[0060] Forward Pass Bias For the forward pass, assign the ith color patch, i=0,...,N-1, as follows:

number

[0061] The MMR extension format of the ith HDR neutral color patch is:

number

[0062] All N color patches can be collected together as follows:

number

[0063] The observed chroma signal (e.g., ch can be the Cb channel or the Cr channel) can be expressed as:

number

[0064] Given the following matrix:

number

number

[0065] The MMR coefficients for the forward pass can be solved as follows:

number

[0066] Reverse Pass Bias For the backward pass, assign the ith color patch, i=0,...,N-1, as follows:

number

[0067] The reverse pass follows the same process. The corresponding reference SDR image is expressed as:

number

number

[0068] All N color patches can be collected together as follows:

number

[0069] The observed chroma signal (e.g., ch can be the Cb channel or the Cr channel) can be expressed as:

number

[0070] Given the following matrix:

number

number

[0071] The MMR coefficient for the reverse path can be solved as follows:

number

[0072] Determining the weighting factors As explained in the previous section, the weighting factor for the forward pass (w t (F) ) or the weighting factor for the reverse path (w t (B) ) must be determined. As shown here, empirical methods (experiments) can be used to find the optimal solution.

[0073] Content Saturation and Midtone Deviation The saturation of the content plays a role in determining the weighting coefficients. Note that in the YCbCr domain, a neutral color has a (Cb,Cr) value of (0.5, 0.5). Each color entry in the d3DMT:

number

number

[0074] This is the saturation of one entry. t F For a cube, the average saturation can be calculated as follows:

number

[0075] The average saturation (β) of a full grid of uniformly sampled RGB color points in a given color space (e.g., R.709 / P3 / R.2020) (F) ) can be determined within a container (e.g., R.2020). The wider the gamut, the higher the average saturation.

[0076] To see the effect of different values ​​of the weighting coefficients, we add N b A gray level bar containing pixels is placed. Since the location is known, we measure the deviation of the gray level bar from the neutral color between the final reconstructed HDR image and the neutral point 0.5. We use the mean of absolute difference (MAD) of the chroma channels of the neutral color patch.

[0077] To simplify the design, t =w t (F) =w t (B) The weighting coefficient is set as w t , the final reconstructed HDR pixel value is expressed as:

number

number

[0078] d t ch (w t The higher the value of , the greater the deviation from neutral and the more likely non-neutral colors will occur.

[0079] With the following applied:

number

number

number

[0080] Negative values ​​actually mean more chroma accuracy and less distortion, while positive values ​​mean less chroma accuracy due to the introduction of the weighting factor.

[0081] Data Analysis - Empirical Method for Determining Weighting Factors First, if you don't preserve intermediate colors, i.e. w t = 0. In Figure 3, the content saturation β t (F) and the gray level bars for both channels d t ch Here is an example of the correlation between the distortion of (0). The content saturation plays a role in influencing the deviation of mid-tones. The content saturation (β t (F) ) is higher, the saturation distortion (d t ch (0)), the more likely it is to show artifacts outside of the midtones. Note that in this example, the image is in 12-bit precision.

[0082] For each t-test image, find the minimum value of the weighting coefficient such that the deviation of the mid-tones is less than a threshold δ:

number

[0083] Figure 4 shows the results for δ=8 (12-bit precision) versus content saturation β t (F) necessary for t opt Note that the more saturated the content, the higher the weighting factor required to preserve mid-tones. In some situations, such as Figure 4 where the saturation is less than 0.1, there is no need to preserve mid-tones (i.e., w t opt =0). Other situations may require different weighting values. For example, as in Figure 4, saturation values ​​below 0.13 may use a weighting value of 5, while higher saturation values ​​may require a weighting value of 10 or 15.

[0084] While artifacts outside of neutral colors are rarely visible in images showing natural scene sets, they are more frequently detected in highly saturated synthetic datasets, especially in certain color spaces (e.g., R.2020).

[0085] How to select the adoption weighting factors A plot of saturation vs. minimum W (e.g., Figure 3) can be used to find a piecewise linear function that envelopes / upper bounds the data points. Content saturation β t (F) If you measure, the required weighting factor β t (F) can be determined. For example:

number

[0086] Or it can simply be expressed as a clipping function:

number

[0087] Figure 5 shows this function (510) as a line clipped below 0.05 and above 0.2. Different profiles can produce different curves / functions.

[0088] Figure 6 shows the piecewise linear model w t =f(β t (F) ) using MAD, d t ch (w t ) shows an example where the MAD is within tolerance (δ=8) with 12-bit precision. Subjective tests reveal that this small midtone shift is invisible.

[0089] Figure 7 shows the optimal w t =f(β t (F) ) using MAD, D t ch (w t ) is shown. As shown in Figure 7, most of the frames show negative values, which means that the chroma is actually improved by this neutral color preservation method. D t ch (w t ) is greater than 0 for very few frames, but still less than (δ=8). MAD does degrade performance, but it is not noticeable.

[0090] In some embodiments herein, a number of intermediate color patches (N) are multiplied by a weighting factor (W). In some embodiments, N can be fixed and W can be adjusted to provide intermediate color preservation. In other embodiments, W can be fixed and the number of intermediate color patches can be increased (e.g., to W*N). Typically, fixing N is less computationally intensive. Fixing W bypasses the step of determining W, and fixing W to 1 eliminates the need to multiply W by N in the calculation.

[0091] Hardware Embodiments One embodiment of the system is as part of a codec used in an encoder-decoder system. MMR coefficients can be resolved at the encoder (a processor / machine configured to receive input of images (discrete and / or video) and output a data stream containing the encoded images and metadata) using metadata sent to the decoder, which can contain either the coefficients themselves or a pointer to one of the predetermined coefficient sets stored in the decoder. For example, the decoder can contain a library of MMR coefficient sets covering various expected bias levels, and the metadata informs the decoder which set to use for compatibility.

[0092] As described herein, embodiments of the present invention may therefore relate to one or more of the following enumerated exemplary embodiments (EEE). Accordingly, the present invention may be embodied in any of the forms described herein, including, but not limited to, the following enumerated exemplary embodiments (EEE), which describe the structure, features, and functions of some portions of the present invention.

[0093] (EEE1) 1. A method for encoding image data while preserving neutral colors, the method comprising: constructing a dynamic 3D mapping table (d3DMT) from a first reference image and a second reference image, the first reference image having a different dynamic range than the second reference image; calculating a content color matrix based on the d3DMT; calculating content saturation from the d3DMT; calculating at least one weighting factor from said content saturation; constructing a set of intermediate colors; calculating an intermediate color matrix based on the intermediate color set; combining the content color matrix, the at least one weighting coefficient, and the intermediate color matrix to solve for multi-channel multiple regression (MMR) coefficients; providing metadata to a decoder, the metadata including data related to said MMR coefficients, to enable backward compatibility of a single layer bitstream; A method comprising:

[0094] (EEE2) 8. The method of claim 6, wherein the method is configured to provide MMR coefficients that include data related to both forward pass MMR coefficients and reverse pass MMR coefficients, the forward pass MMR coefficients being determined independently from the reverse pass MMR coefficients.

[0095] (EEE3) The method according to EEE1, wherein the method is configured to provide MMR coefficients that include data relating only to reverse path MMR coefficients.

[0096] (EEE4) The method of any of EEE1-3, wherein the content color matrix is ​​of the form A and b, where A is based on a vector of all non-zero bins and b is based on a vector of all non-zero bins and an observed chroma signal.

[0097] (EEE5) The method of any one of EEE1-4, wherein the MMR coefficients are evaluated using a least squares solution algorithm.

[0098] (EEE6) The MMR is A -1 The method described in EEE5, which is of the form b.

[0099] (EEE7) The method of any of EEE1-6, wherein the combining step includes the step of multiplying one of the intermediate color matrices by one of the at least one weighting factor and adding the product to a corresponding one of the content color matrices.

[0100] (EEE8) The method according to any one of EEE1 to 7, wherein constructing the set of intermediate colors comprises constructing a plurality of pseudo-intermediate color patches that are not present in either the first reference image or the second reference image.

[0101] (EEE9) The method according to any one of EEE1 to EEE8, wherein the at least one weighting factor comprises a forward path weighting factor and a reverse path weighting factor.

[0102] (EEE10) The method according to any one of EEE1 to 9, wherein the metadata includes a pointer to a set of MMR coefficients stored in the decoder.

[0103] (EEE11) The method according to any one of EEE1 to 10, wherein the step of calculating at least one weighting factor comprises comparing the content saturation to experimental data.

[0104] (EEE12) The method of EEE11, further comprising the step of establishing a linear function that envelopes the data points in the experimental data.

[0105] (EEE13) An encoder configured to perform a method according to any one of EEE1 to EEE12, the encoder comprising: a processor; a signal input configured to receive the image; a signal output configured to transmit the metadata; Encoder including.

[0106] (EEE14) A method for encoding image data while preserving neutral colors, said method comprising: constructing a dynamic 3D mapping table (d3DMT) from a first reference image and a second reference image, the first reference image having a different dynamic range than the second reference image; calculating a content color matrix based on the d3DMT; constructing a set of intermediate colors; calculating an intermediate color matrix based on the intermediate color set; combining the content color matrix and the intermediate color matrix to solve for multi-channel multiple regression (MMR) coefficients; providing metadata to a decoder, the metadata including data related to said MMR coefficients, to enable backward compatibility of a single layer bitstream; A method comprising:

[0107] <Equivalents, Extensions, Alternatives and Miscellaneous> Exemplary embodiments of color transformation under coding efficiency constraints for coding HDR video are thus described. In the foregoing specification, embodiments of the invention have been described with reference to numerous specific details that may vary from implementation to implementation. Accordingly, the sole and exclusive indication of what the invention is, and what Applicant intends to be the invention, is set forth in the claims as issued in particular form hereby, including any subsequent amendments. Any definitions expressly set forth herein for terms contained in such claims shall control the meaning of such terms as used in the claims. Accordingly, any limitation, element, feature, advantage, or attribute not expressly recited in the claims should not in any way limit the scope of the claims. The specification and drawings are, therefore, to be considered in an illustrative, rather than a restrictive, sense.

Claims

1. 1. A method for encoding image data while preserving intermediate colors, the intermediate colors being represented by centers of respective color axes in a normalized color domain, the method comprising: constructing a dynamic 3D mapping table (d3DMT) from a first reference image and a second reference image, the first reference image having a different dynamic range than the second reference image; Calculating a content color matrix based on the d3DMT, the content color matrix being A CC and b CC It is in the form of A CC is based on the vector of all non-zero bins, and b CC is a step based on the vector of all non-zero bins and the observed chroma signal; calculating content saturation from the d3DMT; calculating at least one weighting factor from said content saturation; constructing a neutral color set comprising a plurality of neutral color image patches; calculating an intermediate color matrix based on the intermediate color set, the intermediate color matrix being A NC and b NC It is in the form of A NC is based on the neutral color patch vector that collects multiple neutral color patches, and b NC is based on the neutral color patch vector and the neutral color observed chroma signal; multiplying one of the intermediate color matrices with one of the at least one weighting coefficients and adding the product to a corresponding one of the content color matrices to generate multi-channel multiple regression (MMR) coefficients; providing metadata to a decoder, the metadata including data related to said MMR coefficients, to enable backward compatibility of a single layer bitstream; A method comprising:

2. The method of claim 1 , wherein constructing a dynamic 3D mapping table (d3DMT) from the first reference image and the second reference image comprises determining 3D histogram bins that have a non-zero number of pixels.

3. 2. The method of claim 1, wherein constructing the intermediate color set includes generating the intermediate color patch vector, wherein in the intermediate color patches, luminance (Y) values ​​are in [0,1) and chroma values ​​(Cb, Cr) are fixed at 0.

5.

4. 2. The method of claim 1, wherein the method is configured to provide MMR coefficients that include data associated with both forward-pass MMR coefficients and backward-pass MMR coefficients, the forward-pass MMR coefficients being determined independently from the backward-pass MMR coefficients.

5. The method of claim 1 , wherein the method is configured to provide MMR coefficients that include data related only to reverse path MMR coefficients.

6. The method of claim 1 , wherein the MMR coefficients are determined using a least squares solution algorithm.

7. The MMR coefficient is A -１ 7. The method of claim 6, wherein the method is of the form b.

8. The method of claim 1 , wherein the at least one weighting factor comprises a forward path weighting factor and a reverse path weighting factor.

9. The method of claim 1 , wherein the metadata includes a pointer to a set of MMR coefficients stored in the decoder.

10. The method of claim 1 , wherein calculating the at least one weighting factor comprises comparing the content saturation to empirical data.

11. 11. The method of claim 10, wherein calculating the at least one weighting factor from the content saturation comprises finding a piecewise linear function to encompass data points in the experimental data, the piecewise linear function mapping the measured content saturation to the at least one weighting factor.

12. 10. An encoder configured to perform the method of claim 1, said encoder comprising: a processor; a signal input configured to receive the image data; a signal output configured to transmit the metadata; Encoder including.

Images