Method for converting an image and corresponding device

Abstract

The invention relates to a method for converting an input image comprising input luminance components constituted by elements into an output image comprising output luminance components constituted by elements, the output luminance component values having different range extensions than the corresponding ranges of input luminance component element values. The method comprises, for the input image, computing a general variable value representative of at least two input luminance component element values, transforming each input luminance component element value into a corresponding output luminance component element value as a function of the computed general variable value, and converting the input image using the determined output luminance component element values. The transforming step uses a set of predetermined output values organized into a 2D lookup table (2D LUT) comprising two input arrays indexing respectively a set of selected input luminance component values and a set of selected general variable values, each predetermined output value matching a pair of values constituted by an indexed input luminance component value and an indexed general variable value, the input luminance component element value being transformed into an output luminance component element value using at least one predetermined output value.

Description

Technical Field

[0001] This invention relates to image conversion in general, and more specifically, to the conversion of video images. Background Technology

[0002] Recent advances in display technology have begun to allow for an extended range of brightness in displays. The technology that allows for an extended range of brightness in image content is called "high dynamic range" imaging, often simply referred to as HDR.

[0003] To prepare standard content for HDR display devices, referred to here as "Standard Dynamic Range" (SDR), an inverse (or reverse) tone mapping operator (IMMO) can be employed. This method specifically processes luminance information of color regions within the image content to restore or recreate the appearance of the original scene. Typically, the IMMO takes an SDR image as input, expands the luminance range, and then locally processes highlight or bright areas to enhance the HDR appearance of the colors in the corresponding image. For example, patent application WO / 2017 / 190786 discloses a method for expanding the range of values ​​for the luminance component to create an HDR signal.

[0004] While minimizing changes to the visual experience, the ability to perform round trips—that is, the ability to convert SDR content to HDR and then back to SDR—may require devices similar to the display device. Similarly, round trips from HDR to SDR and back to HDR may also be necessary. Summary of the Invention

[0005] In the context of the first aspect of the invention, a method is proposed for converting an input image comprising an input luminance component composed of elements into an output luminance component comprising elements, wherein the output luminance component values ​​have a different range extension than the corresponding ranges of the input luminance component element values, the method comprising: for the input image,

[0006] Calculate the general variable (L) representing the element values ​​of at least two input luminance components. med The value of );

[0007] Based on the calculated general variable values, the element values ​​of each input luminance component are transformed into the corresponding element values ​​of the output luminance component; and

[0008] The input image is transformed using the determined output luminance component element values;

[0009] The transformation step uses a set of predetermined output values ​​organized into a 2D lookup table (2D LUT). The 2D lookup table includes two input arrays that index a set of selected input luminance component values ​​and a set of selected general variable values, respectively. Each predetermined output value is matched with a pair of values ​​consisting of an indexed input luminance component value and an indexed general variable value. The input luminance component element values ​​are transformed into output luminance component element values ​​using at least one predetermined output value.

[0010] Advantageously, mapping individual input luminance component element values ​​to output luminance component element values ​​allows bypassing the complex calculations involved in existing technical solutions to transform an image into an output image with high dynamic range. Furthermore, due to the general variables calculated image-by-image, the present invention allows the output luminance components to be dynamically adapted to each image.

[0011] In this implementation, the input image may be a video sequence comprising several input images; therefore, the transformation step is further based on an adaptable parameter (m) set by the operator for the entire video sequence according to a given context. a ,m b ,m c The adaptable parameters are used to update the values ​​of each indexed input luminance element to the context.

[0012] Preferably, the method further includes: processing the output image for display on a display device, wherein the value of the adaptable parameter can be adapted according to the characteristics of the display device constituting the context.

[0013] In this implementation, it is possible to increase the matched, indexed input luminance element values ​​to an exponential (f) that depends on the calculated general variable value. γ Each predetermined output value in the set of predetermined output values ​​is determined in advance.

[0014] For example, calculating general variable values ​​includes: calculating a central trend measurement of the values ​​representing the elemental values ​​of two or more input luminance components.

[0015] The central trend measurement can be the median value.

[0016] For example, each output value L in this set of output values out_pred It can be predetermined, such as:

[0017]

[0018] in,

[0019] L med_ind These are the values ​​of general variables that are indexed;

[0020] Llin_ind These are the indexed input luminance component element values;

[0021] f m It is a function based on adaptable parameters;

[0022] s is a control parameter;

[0023] f γ It is an index.

[0024] For example, it is possible to convert the function f m with f γ Defined as follows, such as:

[0025]

[0026] and

[0027]

[0028] in,

[0029] m a m b m c It is an adaptable parameter;

[0030] o is the predetermined offset; and

[0031] g is the predetermined gain value.

[0032] In an implementation, the method may further include:

[0033] The preliminary step involves transforming the element values ​​of each input luminance component into element values ​​of an intermediate luminance component using another set of predetermined output values, thereby determining the intermediate luminance component composed of elements based on adaptable parameters; and

[0034] In this process, the values ​​of general variables are calculated by using the values ​​of intermediate brightness components.

[0035] Alternatively, each output value L in this set of output values out_pred It can be predetermined, such as:

[0036]

[0037] in,

[0038] L med_ind These are the values ​​of general variables that are indexed;

[0039] L lin_ind These are the indexed input luminance component element values;

[0040] f mIt is a function based on the adaptable external parameters;

[0041] s is a control parameter;

[0042] f γ It is an index.

[0043] Therefore, it is possible to convert the function f m with f γ Defined as follows, such as:

[0044]

[0045] and

[0046] f γ (L med_ind ) = g * log 10 (L med_ind )+o

[0047] in,

[0048] m a m b m c It is an adaptable parameter;

[0049] o is the predetermined offset; and

[0050] g is the predetermined gain value.

[0051] According to a second aspect of the invention, an apparatus is provided for converting an input image comprising an input luminance component composed of elements into an output image comprising an output luminance component composed of elements, wherein the output luminance component values ​​have a different range extension than the corresponding ranges of the input luminance component element values. The apparatus is configured as follows:

[0052] Calculate the value of a general variable representing the elemental values ​​of at least two input luminance components;

[0053] Based on the calculated general variable values, the element values ​​of each input luminance component are transformed into the corresponding element values ​​of the output luminance component; and

[0054] The input image is transformed using the element values ​​of the output luminance component;

[0055] Specifically, each input luminance component element value is transformed into a corresponding output luminance component element value using a set of predetermined output values ​​organized into a 2D lookup table (2D LUT). The 2D lookup table includes two input arrays that index a set of selected input luminance component values ​​and a set of selected general variable values, respectively. Each predetermined output value is matched with a pair of values ​​consisting of an indexed input luminance component value and an indexed general variable value. The input luminance component element values ​​are transformed into output luminance component element values ​​using at least one predetermined output value.

[0056] In one implementation, the input image may be a video sequence comprising several input images, and wherein, each input luminance component element value is further transformed into a corresponding output luminance component element value based on adaptable parameters set by an operator for the entire video sequence according to a given context, the adaptable parameters being used to update each indexed input luminance element value to the context. Attached Figure Description

[0057] Further advantages of the present invention will become apparent to those skilled in the art upon viewing the accompanying drawings and detailed description. Embodiments of the invention will now be described only by way of example and with reference to the following drawings.

[0058] Figure 1 An embodiment of the method according to the first aspect of the present invention is shown;

[0059] Figure 2 express Figure 1 The first variation of the embodiment shown;

[0060] Figure 3 express Figure 1 A second variation of the embodiment shown;

[0061] Figure 4 The curves involved in the one-dimensional interpolation process are shown;

[0062] Figure 5 The curves involved in the two-dimensional interpolation process are shown;

[0063] Figure 6 An embodiment of the device according to the second aspect of the present invention is shown;

[0064] Figure 7 This indicates that it can be implemented using an FPGA. Figure 6 The equipment in;

[0065] Figure 8 An embodiment of the method according to the third aspect of the present invention is shown;

[0066] Figure 9An embodiment of the device according to the fourth aspect of the present invention is shown.

[0067] The invention will now be described with reference to specific, non-limiting exemplary embodiments and the figures. Detailed Implementation

[0068] Figure 1 A first embodiment of the present invention is shown for converting an input image composed of elements (pixels) into an output image.

[0069] For the purposes of this disclosure, "element" can refer to a single pixel (or picture element) of a component (e.g., a luminance component) of a still image or a single pixel of a frame or subframe of a video image. More generally, "element" refers to any structure associated with a specific value (here, a specific value of luminance) to locally define an image.

[0070] For example, the input image belongs to a video sequence consisting of several (at least two) consecutive input images. Here, the input image is an RGB color image composed of three linear color components: red (R)... lin Green G lin And blue B lin Each color component consists of elements corresponding to individual pixels in the input image. Similar to the YCbCr model, other color models can be used for the input image. In the following description, the RGB color model is considered.

[0071] In the first step S1, the input color component R is processed according to a known method. lin G lin And B lin Processing is performed to obtain the input luminance component L. lin For example, an RGB input image is transformed into another linear space, where the transformed image includes three components X, Y, and Z, where Y is the linear luminance component, hereinafter referred to as L. lin This corresponds to the input luminance component described below. Such a conversion process is described, for example, in standards ITU-R BT2020 and BT709.

[0072] In the optional second step S2, the input luminance component is preprocessed to provide the intermediate luminance component L. m .

[0073] In the third step S3, the general variable L is calculated. med The value of L. General variable L. med This indicates at least two input luminance component element values. See below for reference. Figure 2 and Figure 3 Give the value L of the general variable used to calculate it. med Examples.

[0074] In the fourth step S4, the given general variable value L calculated for the input image is... med Input component L lin Each input element value is mapped to the output luminance component L. out The element values. This step involves using a predefined set of output values ​​L. out_pred Preferably, the group is organized into a 2D lookup table (2D LUT), which comprises two input arrays indexing a selected set of input luminance component values ​​and a selected set of general variable values. Each predetermined output value is matched with a pair of values ​​consisting of an indexed input luminance component value and an indexed general variable value. The individual output luminance component element values ​​L... out By using one or more predetermined output values ​​L out_pred And that's determined. More information about the mapping process will be available in [reference]. Figure 2 , Figure 3 ,as well as Figure 5 supply.

[0075] In step S5, the output luminance component L is processed according to the same color model as the input color component. out Processing is performed to obtain the output linear color component R. out G out And B out For example, this step is described in "Color correction for tone reproduction" presented at the 2013 Color and Imaging Conference by Pouli, Tania, et al. As detailed below, the input linear color component R... lin G lin And B lin Perform the final step, S5.

[0076] Figure 1 The process described herein allows an input image having a first dynamic range to be converted into an output image having a second dynamic range. For example, the first dynamic range may be "standard dynamic range" or SDR, and the second dynamic range may be "high dynamic range" or HDR, which is higher than the first dynamic range. This conversion is often referred to as "tone expansion." Conversely, the first dynamic range may be higher than the second dynamic range. This other conversion is referred to as "tone compression." More specifically, implementations involve luminance expansion and compression.

[0077] Figure 2 It shows Figure 1This refers to the implementation of processes, such as when converting standard dynamic range (SDR) image or video content into high dynamic range (HDR) image or video content (e.g., ITU-R BT.2100). This conversion is often referred to as "tone extension".

[0078] In the first step S11, the input color component R is processed according to known methods described, for example, in standards ITU-R BT2020 and BT709. sdr G sdr And B sdr Processing is performed to obtain the input luminance component L. lin .

[0079] In the second optional step S21, the input luminance component L is... lin Preprocessing is performed to provide the intermediate luminance component L. m Advantageously, preprocessing allows the input luminance components to be updated to a context defined by adaptable external parameters. These parameters can define the display context of the entire video content, for example, the entire video content could be relevant to a commercial HDR television. Upon completion... Figure 2 Before the conversion, the operator adapts the parameter values ​​to specific predefined values.

[0080] This step may involve using another set of predetermined output values. Preferably, this other set is organized into a 1D lookup table (1D LUT), which includes an input array indexed with a selected set of input luminance component element values.

[0081] For example, we can conclude that:

[0082]

[0083] in,

[0084] L m_pred It is another set of predetermined values ​​for the intermediate brightness component elements.

[0085] L lin_ind It is one of the input luminance component element values ​​in this set of indexes, and

[0086] m a m b m c It can adapt to external parameters, whose values ​​have been set in advance.

[0087] Alternatively, if the input brightness L lin The element value is equal to the index L in the input array of the 1D LUT. lin_ind If a value is given, then the intermediate brightness L mThe corresponding element value is equal to another set of predetermined output values ​​L. m_pred The corresponding value. Otherwise, refer to the following: Figure 4 The intermediate brightness L is explained. m The element values ​​are determined by the other set of predetermined output values ​​L. m_pred The two values ​​are interpolated. In practice, to limit its size, only the values ​​of selected input luminance elements are indexed in the 1D LUT. This substitution is advantageous if element values ​​are represented in high-bit numbers to achieve a certain level of accuracy. In another alternative scheme, the intermediate luminance L... m There is no arbitrary interpolation in the element values. When values ​​are represented with a finite number of bits (e.g., 10 or 11 bits), 1DLUT indexes all possible values, thus allowing reasonable storage capacity to be saved (e.g., with regard to read-only memory ROM).

[0088] The third step S31 includes: from the intermediate brightness L m The general variable L is calculated by taking at least a portion of the element values. med The value of . In the implementation, this value refers to a measure of the central tendency of the intermediate input luminance component element values. To calculate the general variable value, the intermediate luminance element values ​​of all inputs can be considered, or only some of them (at least two) can be considered. For example, the measure of central tendency can be the median or the mean. In the following description, the central tendency is considered from the intermediate luminance L m The value of the general variable is obtained by calculating the median of all element values. If step S21 is not performed, the value is obtained from the input brightness L. lin The element values ​​are directly calculated for the general variable L. med The value of .

[0089] The general variable values ​​for each image are calculated, and when considering video sequences, the general variable values ​​are dynamically adapted image by image.

[0090] Alternatively, the intermediate brightness L can be calculated. m The maximum value of all element values ​​is used to obtain a general variable.

[0091] The fourth step S41 includes: calculating a given general variable value L for the input image. med Input component L lin Each input element value is mapped to the output luminance component L. out The element values. As previously explained, this step involves: using a predefined set of output values ​​L organized here into a 2D lookup table (2D LUT). out_pred The 2D lookup table includes an index of a selected set of input luminance component values ​​L. lin_ind and a set of selected general variable values ​​L med_indThe system has two input arrays. Each pair of values, consisting of the indexed input luminance element value and the indexed general variable value, is matched with one of a predetermined set of output values. According to one implementation, this is achieved by increasing the indexed input luminance element value to an exponent f representing the expansion exponent. γ And predetermine each output value L out_pred For example, the definition of an extended index is given in patent application WO2017 / 103399. Furthermore, according to a preferred embodiment, each output value L is predetermined by updating the input luminance element values ​​of each index to specific adaptable parameter values ​​predefined by the operator. out_pred .

[0092] Furthermore, step S41 allows control over the saturation of the output image by performing saturation adjustment on the input luminance component. In practice, performing tone expansion alters the initial saturation. According to one implementation, saturation is adjusted by increasing the indexed input luminance element value to an exponent s, where the value of the exponent s is chosen to correct for saturation variations.

[0093] For example, when performing hue expansion, the result is:

[0094]

[0095] in,

[0096] L med_ind It is one of the general variable values ​​in this set of indexes;

[0097] L lin_ind It is the input luminance component element value of one of the input luminance component element values ​​in this set of indices;

[0098] f m It is a function based on the adaptable external parameters used to update the input luminance component element values ​​of the index;

[0099] 's' is the control parameter used to perform saturation adjustment; and

[0100] f γ It is an index representing an expansion index.

[0101] Under the condition of implementing step S21, the same function f m Used to calculate each output value in this other set of predetermined output values.

[0102] Therefore, f m It can be represented by the following expression:

[0103]

[0104] fγ It can be represented by the following expression:

[0105]

[0106] in,

[0107] g is the predetermined gain; and

[0108] o is the predefined offset.

[0109] In this example, when L med_ind When f increases γ Decrease monotonously.

[0110] For example, when the operator wants to adapt the parameter m a m b and m c When adapting the values ​​to obtain a configuration file that is highly suitable for commercial HDR TVs, the following results were obtained:

[0111] m a =1.5284,

[0112] m b =0.5279, and

[0113] m c =0.7997.

[0114] For example, in the implementation method, it is found that:

[0115] s = 1, g = 0.06, and o = 1.

[0116] The expressions (3) and (4) above are given as examples, and f is not excluded. m and f γ Other implementation methods.

[0117] For example, alternatively, f m It can be in the form of an nth-order polynomial, where the polynomial parameters p can be determined by performing a polynomial regression of the desired form on the data. n ..., p0. For example, the alternative form can be manually defined by an artist-like operator, and potentially further modified to guide f. m Function behavior:

[0118]

[0119] When input brightness L lin element values ​​and general variable values ​​L med When matched with the index value, the output brightness L outThe element value corresponds to the output value L in the 2D LUT associated with the index value. out_pred Otherwise, please refer to the following: Figure 5 As explained, the output brightness L out The element value is determined by two output values ​​L from the predefined set of output values. out_pred It was inserted from within.

[0120] Advantageously, mapping individual input luminance component element values ​​to output luminance component element values ​​allows bypassing the complex calculations involved in prior art solutions to transform an image into an output image with a high dynamic range. These solutions involve applying an exponential function to each processed input luminance component element value. The complexity of this application is not well suited for implementations in some hardware, such as FPGAs (“Field Programmable Gate Arrays”) capable of performing a limited number of operations. Moreover, due to the general variables calculated on an image-by-image basis, the 2D LUT according to the invention allows the output luminance components to be dynamically adapted to individual images. As mentioned above, other settings related to operator selection can also be considered to provide the output luminance components by using specific adaptable parameters.

[0121] In the fifth step S51, taking into account the saturation correction already performed using parameter s in step S41 (see Equation 2), the output luminance component L is used. out and input color component R sdr G sdr B sdr To obtain the output color component R hdr G hdr B hdr For example, based on standard saturation correction schemes, such as the method described by R. Fattal, D. Lischinski, and M. Werman in "Gradient domain high dynamic range compression" Volume 21, Chapter 13, pp. 249-256, ACM Transactions on Graphics (TOG) in 2002, it is possible to correct the saturation-corrected output color component R. hdr G hdr and B hdr The calculation is as follows:

[0122]

[0123] Among them, X hdr It can be R hdr or G hdr Or B hdr ,and It can be or or

[0124] Because of the 2D LUT (see Equation 2) for L out_pred The calculations performed have already included through The normalization process described above scales the input color in a manner consistent with changes in brightness, while simultaneously correcting its saturation.

[0125] Figure 3 It shows Figure 1 Another way to convert high dynamic range (HDR) image or video content into standard dynamic range (SDR) image or video content is called "tone compression".

[0126] In the first step S12, the input color component R is processed according to the previously described known method. hdr G hdr And B hdr Processing is performed to obtain the input luminance component L. lin .

[0127] No preprocessing step is required to convert an HDR image to an SDR image. If step S2 is not optional, for example due to the choice of hardware implementation, then... Figure 3 The implementation shown may include additional steps for applying the identity function, because the output L lin Equal to input L lin .

[0128] In the second step S32, the general variable value L is directly calculated from the input luminance component element values. med For reference Figure 2 As described in step S31.

[0129] The third step S42 includes: calculating the given general variable value L for the input image. med Input component L lin Each input element value is mapped to the output luminance component L. out The element values, as previously referenced Figure 2 As described in step S41. Similarly, step S42 involves using a predetermined set of output values ​​L organized here into a 2D lookup table (2D LUT). out_pred The 2D lookup table includes an index of a selected set of input luminance component values ​​L. lin_ind and a set of selected general variable values ​​L med_indThe system has two input arrays. Each pair of values, consisting of the indexed input luminance element value and the indexed general variable value, is matched with one of a predetermined set of output values. Preferably, this is done by increasing the indexed input luminance element value to an exponent f representing the expansion exponent. γ And predetermine each output value L out_pred Furthermore, according to a preferred embodiment, each output value L is predetermined by updating the input brightness element values ​​of each index to specific adaptable parameter values ​​predefined by the operator. out_pred .

[0130] Furthermore, step S42 allows control over the saturation of the output image by performing saturation adjustment on the input luminance component. In practice, performing tone expansion alters the initial saturation. According to one implementation, saturation is adjusted by increasing the indexed input luminance element value to an index s whose value is selected to correct for saturation variations.

[0131] For example, when performing tone compression, the result is:

[0132]

[0133] in,

[0134] L med_ind It is a general variable value among a set of indexed general variable values;

[0135] L lin_ind It is an indexed input luminance component element value among a set of indexed input luminance component element values;

[0136] f m Based on allowing L lin_ind The function that updates the adaptable external parameters;

[0137] 's' is the control parameter used to perform saturation adjustment;

[0138] f γ It is an index.

[0139] For example, the function f m and f γ The following expressions can be used to represent them respectively:

[0140]

[0141] and

[0142]

[0143] in,

[0144] m a mb m c This refers to the previously mentioned feature that can adapt to external parameters;

[0145] o is the predetermined offset; and

[0146] g is the predetermined gain value.

[0147] about Figure 2 The embodiment shown in the figure, when the input brightness L lin The element values ​​and general variable values ​​L med When matched with the index value, the output brightness L out The element value corresponds to the output value L associated with the index value in the 2D LUT. out_pred Otherwise, please refer to the following: Figure 5 As explained, the output brightness L out The element value is determined by two output values ​​L from the predefined set of output values. out_pred It was inserted from within.

[0148] In the final step S52, the linear input color component R is used. hdr G hdr And B hdr The output luminance component L is calculated based on the same color model as the input color component. out Processing is performed to obtain the output color component R. sdr G sdr And B sdr ,

[0149] Figure 4 The interpolation process for performing step S21, which can be implemented in a 1D LUT, is shown in more detail. In fact, when the input brightness L... lin When the value of an element is not indexed, the intermediate brightness L m The element values ​​are interpolated from two values ​​in another set of predetermined output values. The curve represents the solid line of the actual function f, which modulates the intermediate luminance component L based on the input luminance element values. m The values ​​used to model the elements.

[0150] First, the interpolation process includes: determining the closest coordinates x0 and x1 based on x, where x is the input brightness L. lin The element values ​​of x are truncated to the resolution of the 1D LUT. Next, y0 and y1, as corresponding values ​​of x1 and x0, are obtained from the index values ​​of the 1D LUT. Third, given y = f(x), the local derivative dy / dx is determined. The local derivative is represented by dashed lines. Then, the offset dy is calculated, and finally, this offset is added to y0 to obtain the final value of y.

[0151] Figure 5The interpolation process implemented in the 2D LUT for performing step S4, or its implementation in steps S41 or S42, is shown in more detail. In this case, consider two dimensions x and y for the information array index (input luminance element values ​​and general variable values). In this case, the interpolation process of the 2D LUT can be viewed as two one-dimensional interpolation processes as described previously. First, the interpolation process includes: determining the closest index coordinates x0 and x1 based on x, where x corresponds to the value of the first index information. Truncate x to the resolution of the 2D LUT. Second, determining the closest index coordinates y0 and y1 based on y, where y corresponds to the value of the second index information. Truncate y to the resolution of the 2D LUT. Third, obtaining the index intersection points z corresponding to the inputs x0, y0 and x0, y1, respectively. 00 and z 01 It allows determining z0. Fourth, obtain the intersection points z corresponding to the inputs x1, y0 and x1, y1 respectively. 10 and z 11 This allows for the determination of z1. Finally, the final value z is interpolated from the y-axis and from the pre-obtained values ​​z0 and z1 from the 2D LUT, with the z value corresponding to the element-wise values ​​of the input luminance component.

[0152] Figure 6 An embodiment of the device DEV1 according to the present invention is illustrated schematically. The device can implement the above-described conversion method according to any different embodiments.

[0153] First, device DEV1 can receive data and provide data to external units via the communication device MCOM.

[0154] The device can be configured to perform a tone compression or tone expansion process, or both. In this case, the device DEV1 may include an optional configuration device MCONF1 for adapting the first computing device MCOMP1 and the mapping device MMAP1 according to the transformation being processed.

[0155] Device DEV1 includes a processing unit MPRO for processing input image data according to an adopted color model to obtain input luminance components. Optionally, the processing unit MPRO is paired with a first computing unit MCOMP1 capable of preprocessing the input luminance components. The computing unit MCOMP1 can configure the first computing unit MCOMP1 to preprocess the input luminance components using a 1D LUT, or apply an identity function to output the same input luminance components to the second computing unit MCOMPB. When device DEV1 only implements the tone compression process, it may not include the first computing unit MCOMP1, and the processing unit MPRO is directly paired with the second computing unit MCOMPB.

[0156] The second computing device MCOMPB can calculate general variable values ​​from the output data of the first computing device MCOMPA (or, if the first computing device MCOMPA is not included, calculate directly from the output of the processing device MPRO).

[0157] The device DEV1 further includes the aforementioned mapping device MMAP1, which is capable of receiving general variable values ​​and input luminance components element by element.

[0158] Here, the mapping device MMAP1 includes:

[0159] - A 2D lookup table (2D LUT) is used to index input luminance element values, general variable values, and corresponding predetermined output luminance values ​​stored using internal or external storage units (not shown); and

[0160] - Interpolation unit, which can interpolate the element values ​​of the output luminance component when needed.

[0161] Finally, the correction device MCOR is able to provide the output image components (here, R) from the output of the mapping device MMAP. hdr G hdr And B hdr Alternatively, the calibration device MCOR can be located outside the processor.

[0162] In one example, the device DEV1 may be a processor (e.g., a microprocessor) designed to continuously execute multiple instructions of a computer program stored in a memory cell (not shown). In another example, the device DEV1 may be a programmable logic device such as a field-programmable gate array (FPGA). The device is particularly suitable for implementing the present invention if it has limited power consumption and a small size that facilitates integration into a large system.

[0163] Figure 7 The method of implementing device DEV1 using an FPGA is shown. In this example, optional configuration devices are not included, and the FPGA allows for the conversion of SDR images into HDR images.

[0164] Device DEV1 includes a first conversion unit MCV1 for converting RGB packaged input components. Convert to linear color components R sdr G sdr B sdrThe first conversion device encapsulates the RGB encapsulated input components transmitted via the communication device into a transport stream to be transmitted, representing each color component in 10 bits. The conversion to linear color components involves a lookup table (LUT) and matrix multiplication by a multiplication device (MUL1) that provides linear components represented in 16 bits, enabling further operations on the linear components.

[0165] The processing unit MPRO is designed to calculate a weighted sum pixel-by-pixel using the linear components provided by the conversion unit MCV1. The weighted sum corresponds to the linear luminance value L associated with each pixel. lin .

[0166] The first computing device MCOMP1 is implemented using a 1D LUT, and the second computing device MCOMP2 is implemented using a median filter. (Reference) Figure 3 When performing the conversion from SDR to HDR format using the same equipment, the first computing device MCOMP1 is also able to implement the identity function detailed above.

[0167] The median calculated by the second computing device MCOMP2, along with the input luminance component composed of a 2D lookup table, is sent to the mapping device MMAP1. According to reference... Figure 3 In the detailed implementation, the mapping device MMAP1 provides an output brightness value.

[0168] The MCOR device, which includes three lookup tables, can receive linear components from the conversion device MCV1 to calculate the linear output color component R. hdr G hdr B hdr The correction device allows additional saturation correction to be performed by using three LUTs connected to three multipliers MUL to receive output luminance component values ​​from the mapping device, each multiplier allowing the implementation of Equation 6 mentioned above.

[0169] Finally, device DEV1 includes a second conversion unit MCV2 for converting linear color components R... hdr G hdr B hdr Convert back to RGB packaged input components compatible with transmission via communication devices. Regarding the first conversion device, the conversion performed by the second conversion device MCV2 involves matrix multiplication of the lookup table LUT and the multiplication device MUL2.

[0170] According to a third aspect of the invention, a method is provided for converting at least one input image from a video sequence into an output image comprising an output luminance component composed of elements, wherein the input image comprises an input luminance component composed of elements. The method includes:

[0171] Obtain one configuration from a set of predefined configurations, and define each configuration by a transformation function and at least one feature from a list of possible features;

[0172] The obtained configuration is used to configure a lookup table (LUT) comprising at least one input array indexed by a selected set of input luminance component values, wherein each predetermined output value from a predetermined set of output luminance element values ​​is matched with an indexed input luminance component value.

[0173] And for at least one input image,

[0174] The output luminance component element values ​​are determined by transforming each input luminance component element value into a corresponding output luminance component element value using at least one predetermined output value; and

[0175] The input image is converted into an output image using the provided output luminance component element values;

[0176] The list of possible features includes at least two of the following features:

[0177] a) One or more adaptable parameters set by the operator for the entire video sequence based on a given context, the adaptable parameters being used to update the input luminance element values ​​at each index to the context; and

[0178] b) Based on the exponent of at least two input luminance component element values, each of the predetermined output values ​​in the set is predetermined by increasing the input luminance element value of the corresponding index to the exponent.

[0179] In an implementation, at least one configuration is defined by combining at least two features from the list of possible features.

[0180] In the implementation, each configuration defines a type of conversion, and for at least one type of conversion, the corresponding ranges of the output luminance component values ​​and the input luminance component element values ​​have different range extensions.

[0181] In the implementation, each configuration defines a type of conversion, and at least one type of conversion implements a conversion from an input image using a first function that converts an electronic signal into an optical signal to an output image using a second function that converts an electronic signal into an optical signal.

[0182] According to a fourth aspect of the invention, a method is provided for converting at least one input image from a video sequence into an output image comprising an output luminance component composed of elements. The input image comprises an input luminance component composed of elements. The device is configured to:

[0183] Obtain one configuration from a set of predefined configurations, and define each configuration by a transformation function and at least one feature from a list of possible features;

[0184] The lookup table (LUT) is configured using the obtained configuration, the LUT comprising at least one input array indexing a selected set of input luminance component values, wherein each predetermined output value from a set of predetermined output luminance element values ​​is matched with an indexed input luminance component value;

[0185] And for at least one input image,

[0186] The output luminance component element values ​​are determined by transforming each input luminance component element value into a corresponding output luminance component element value using at least one predetermined output value; and

[0187] The input image is converted into an output image using the provided output luminance component element values;

[0188] The list of possible features includes at least two of the following features:

[0189] a) One or more adaptable parameters set by the operator for the entire video sequence based on a given context, the adaptable parameters being used to update the input luminance element values ​​at each index to the context; and

[0190] b) An index, such that each output value in the predetermined output values ​​of the group is predetermined by increasing the input brightness element value of the corresponding index to the index.

[0191] Figure 8 An example of a method according to a third aspect of the invention is shown. In a preferred embodiment, three types of transformations are considered:

[0192] - Convert the SDR input image to an HDR output image, CONV1;

[0193] - Convert the HDR input image to an SDR output image, CONV2; and

[0194] - Convert the input image to EOTF (“electro-optic conversion function”), CONV3.

[0195] The third aspect of the invention is not limited to these three types of transformations. For example, only two types of transformations may be implemented. In cases where some details of the image need to be enhanced, another type of transformation may be to convert the contrast of the input image from a first value to a second value.

[0196] refer to Figure 2 and Figure 3The first two types of conversions, CONV1 and CONV2, are described. The final type of conversion, CONV3, addresses the luminance transformation required to change the EOTF used for encoding HDR image content. For example, those skilled in the art may convert content encoded using a PQ EOTF to content encoded using an HLG EOTF. This transformation is known and described in standards reports such as the ITU-R BT.2390 Volume 0, 2016, "High dynamic range television for production and international programme exchange".

[0197] The first step S00 includes: obtaining one configuration from a set of predetermined configurations, defining each configuration by at least one feature from a list of possible features. In this instance, each configuration is associated with one of the three types of transformations listed above.

[0198] If the first configuration is selected, the color component R will be entered in the input field. lin G lin B lin The first conversion process CONV1 in step S11 is provided to obtain the output color component R of the HDR content. hdr G hdr B hdr In this case, the input color component R lin G lin B lin Includes SDR images. Already referenced. Figure 2 Steps S11, S21, S31, S41, and S51 are described. The selected configuration is defined by a function that allows the first type of transformation (e.g., according to Equation 2 mentioned above) and by a combination of the following two features:

[0199] - The adaptable parameter m, described above, is set to a selected value by the operator for the entire video sequence based on the given context. a m b and m c The adaptable parameters are used to update the input luminance element values ​​of each index to the already described context; and

[0200] - Exponent f γ Here, based on at least two input luminance component element values, each output value in the set of predetermined output values ​​is predetermined by increasing the input luminance component value of the corresponding index to the already described index.

[0201] If the second configuration is selected, the color component R will be entered in the input field. lin G lin B lin The output color component R of the SDR content is provided to the second conversion process CONV2 in step S12. sdr G sdr B sdr In this case, the input color component R lin G lin B lin Includes HDR images. Already referenced. Figure 3 Steps S12, S32, S42, and S52 are described. The selected configuration is defined by a function that allows a second type of transformation (e.g., according to Equation 3 mentioned above) and by a combination of the following two features:

[0202] - The adaptable parameter m, described above, is set to a selected value by the operator for the entire video sequence based on the given context. a m b and m c The adaptable parameters are used to update the input luminance element values ​​of each index to the already described context; and

[0203] - Exponent f γ Here, based on at least two input luminance component element values, each output value in the set of predetermined output values ​​is predetermined by increasing the input luminance component value of the corresponding index to the already described index.

[0204] If the third configuration is selected, the color component R will be entered in the input field. lin G lin B lin The output color component R of the HDR content is provided to the third conversion process CONV3 in step S13. hdr G hdr B hdr In this case, the input color component R lin G lin B lin Includes HDR images.

[0205] Similar to steps S11 and S12, step S13 allows the input color component R to be processed according to the known methods mentioned above. lin G lin And B lin Processing is performed to obtain the input luminance component L. lin .

[0206] In the case of HLG EOTF conversion, the input luminance component is transformed into the output luminance component according to the following function:

[0207]

[0208] The system index γ is defined as applied to the input luminance component L. lin Here, each L lin The additional division allows for adjustment of saturation. For example, in the case of HLG OOTF conversion, γ = 1.2.

[0209] Preferably, step S23 may involve using another set of predetermined output values ​​L out_pred This other set can be organized into a 1D lookup table (1D LUT), which includes an index of a selected set of input luminance component element values ​​L. lin_ind An input array, such as:

[0210]

[0211] When the input luminance component value is not indexed, the output luminance component L is... out The element value usage has been referenced. Figure 4 The described at least one predetermined output value L out_pred interpolated.

[0212] The selected configuration is defined by allowing a third type of conversion (e.g., according to Equation 9 above) and by the following features:

[0213] - The system index γ enables the predetermined output values ​​in the set of output values ​​to be determined by increasing the input brightness element values ​​of the corresponding index to the index.

[0214] As previously referenced Figure 1 As described in step S5, the final step S33 involves: outputting the luminance component L out Convert to include the output color components R of the converted HDR image. hdr G hdr B hdr .

[0215] Figure 9 An embodiment of the device DEV2 according to the fourth aspect of the present invention is illustrated schematically. The device is capable of implementing reference... Figure 8 The described conversion method.

[0216] First, device DEV2 can receive data and provide data to external units via communication device MCOM. It can receive control signals SEL provided to configuration device MCONF2, which selects a possible configuration. For example, the control signal SEL can be generated by the operator or based on analysis of the input image (e.g., regardless of whether the content is SDR or HDR).

[0217] Device DEV2 can be configured to perform the different types of transformations listed above. Configuration device MCONF2 can configure the first computing device MCOMPA, the second computing device MCOMPB, and the mapping device MMAP2 to perform the transformations as already referenced. Figure 6 This describes a type of conversion.

[0218] The configuration device MCONF2 can:

[0219] - To perform a third type of transformation by using the appropriate 1DLUT configuration mapping device MMAP2; and

[0220] - For example, when selecting the second CONV2 or the third CONV3 type of conversion, the first computing device MCOMPA and the second computing device MCOMPB are configured based on the selected type of conversion by enabling the first computing device MCOMPA to apply the identity function to output the same input luminance component to the second computing device MCOMPB.

[0221] Depending on the conversion type, the different values ​​involved in the 1D LUT and 2D LUT used by the mapping device MMAP2 are stored in internal or external storage units (not shown). The mapping device MMAP2 further includes an interpolation unit capable of interpolating the element values ​​of the output luminance component when needed.

[0222] Device DEV2 can be a processor (e.g., a microprocessor) designed to continuously execute multiple instructions of a computer program stored in a memory cell (not shown). In another example, device DEV2 can be a programmable logic device similar to a field-programmable gate array (FPGA). This device is particularly suitable for implementing the present invention, assuming it has limited power consumption and a small size that facilitates integration into large systems.

[0223] Advantageously, according to the fourth aspect of the invention:

[0224] • Due to its compact implementation, it achieves lower resource usage;

[0225] • Due to the limited number of devices, ease of maintenance is required;

[0226] • It can dynamically switch from one type of conversion to another (e.g., from luminance compression to expansion, or modify the luminance mapping profile used).

[0227] Because the same architecture can be applied to all types of transformations, it is sufficient to change the values ​​of optional 1D LUTs and 2D LUTs to alter the transformation performed. Therefore, due to the architecture's high flexibility, the device can adapt to future transformations.

Claims

1. A method for converting an input image comprising input luminance components made of elements into an output image comprising output luminance components made of elements, the respective ranges of output luminance component element values and input luminance component element values having different range extensions, the method comprising: For the input image, computing (S3) a general variable (L med ) value representing at least two input luminance component element values, Based on the calculated general variable values, the element values ​​of each input luminance component are transformed (S4) into the corresponding element values ​​of the output luminance component, and The input image is transformed using the determined output luminance component element values. The transformation step uses a set of predetermined output values ​​organized into a 2D lookup table (2D LUT), which includes two input arrays that respectively index a set of selected input luminance component values ​​and a set of selected general variable values. Each predetermined output value is matched with a pair of values ​​consisting of an indexed input luminance component value and an indexed general variable value. The input luminance component element values ​​are transformed into the output luminance component element values ​​using at least one predetermined output value.

2. The method of claim 1, wherein, The input image belongs to a video sequence comprising several input images, and the transformation step is further based on an adaptable parameter (m) set by the operator for the entire video sequence according to a given context. a , m b , m c The adaptable parameters are used to update the input luminance element value for each index to the context.

3. The method of claim 2, further comprising: The output image to be displayed on a display device is processed, and the values ​​of the adaptable parameters are adaptable to the characteristics of the display device constituting the context.

4. The method of claim 1, wherein, Each predetermined output value in the set of predetermined output values is predetermined by raising the matching indexed input luminance element value to an exponent dependent on the computed general variable value ) 5. The method of claim 1, wherein, Calculating the general variable value includes: calculating a measure representing the central tendency of the values ​​of two or more input luminance component elements.

6. The method of claim 5, wherein, The measure of the central tendency is the median.

7. The method of claim 2, wherein, each output value in a set of output values is predetermined to be: in, is the general variable value of the index, is the input luminance component element value of the index, is a function based on the adaptable parameters, is a control parameter, is an index.

8. The method of claim 7, wherein, The function is defined as: and in, is the adaptable parameter, is a predetermined offset, and is a predetermined gain value.

9. The method of claim 7, further comprising: The preliminary step involves determining the intermediate luminance component, composed of elements, by transforming the element values ​​of each input luminance component into element values ​​of the intermediate luminance component using another set of predetermined output values ​​based on the adaptable parameters. The general variable value is calculated using the intermediate luminance component element value.

10. The method of claim 2, wherein, each output value in a set of output values is predetermined to be: in, is the general variable value of the index, is the input luminance component element value of the index, is a function based on the adaptable parameters, is a control parameter, is an index.

11. The method of claim 10, wherein, The functions and are defined as: and in, is the adaptable parameter, is a predetermined offset, and is a predetermined gain value.

12. An apparatus for converting an input image comprising an input luminance component consisting of elements into an output image comprising an output luminance component consisting of elements, wherein the corresponding ranges of the output luminance component element values ​​and the input luminance component element values ​​have different range extensions, the apparatus being configured to: Calculate the value of a general variable representing the element values ​​of at least two input luminance components. Based on the calculated general variable values, transform the element values ​​of each input luminance component into the corresponding element values ​​of the output luminance component; and The input image is transformed using the output luminance component element values. wherein, Each input luminance component element value is transformed into a corresponding output luminance component element value using a set of predetermined output values ​​organized into a 2D lookup table (2D LUT). The 2D lookup table includes two input arrays that respectively index a set of selected input luminance component values ​​and a set of selected general variable values. Each predetermined output value is matched with a pair of values ​​consisting of an indexed input luminance component value and an indexed general variable value. The input luminance component element values ​​are transformed into the output luminance component element values ​​using at least one predetermined output value.

13. The apparatus of claim 12, wherein, The input image belongs to a video sequence comprising several input images, and the transformation of each input luminance component element value into a corresponding output luminance component element value is further based on an adaptable parameter (m) set by the operator for the entire video sequence according to a given context. a , m b , m c The adaptable parameters are used to update the input luminance element values ​​of each index to the context.

Images