METHOD FOR PROCESSING DATA BY ITERATIVE APPLICATION OF THE SAME LUT

Description

Technical Field

[0001] The present disclosure relates to the processing of data using lookup tables. It is mainly focuses on the processing of colors of images.Background Art

[0002] It is known to process data according to a processing function. For instance, the so-called Perceptual Quantizer Electro-Optical Transfer Function (PQ-EOTF) L = h(N) is, according to SMPTE ST-2084 standard, represented by the following function: L = N 1 m 2 − c 1 c 2 − c 3 N 1 m 2 1 m 1 m 1 = 2610 4096 × 1 4 = 0.1593017578125 m 2 = 2523 4096 × 128 = 78.84375 c 1 = 3424 4096 = 0.8359375 c 2 = 2413 4096 × 32 = 18.8515625 c 3 = 2392 4096 × 32 = 18.6875 where L is the Luminance and N is a non-linear color value .

[0003] Instead of using an algebraic function h(.) to process data, it is common and efficient to use a Lookup-table (LUT) that models such a function to process these data. Such a processing method is quite usual notably in the field of color and image processing. LUTs are typically used when evaluation of a processing function is difficult to implement (for instance because it involves expensive operations such as divisions, logarithms and the like). Look-up tables are commonly used to represent complex processing functions, or processing functions that would otherwise be difficult to implement in hardware. Replacing a processing function with a look-up table involves approximating the function with a number of bins. Each bin of such a LUT maps a sample input value to a sample output value. An input / output value is meant to be a scalar value, or a vector of several scalar values. If values are scalar, the LUT is said to be one-dimensional, if values are vectors, the LUT is said to be multi-dimensional.

[0004] An input value different from the sample input values of a LUT is generally mapped to an output value interpolated from the sample output values of this LUT and this means that a table-lookup is always an approximation of the function that it represents.

[0005] Globally, it means that the processing of input values by a function is replaced by two operations: 1 / application of a LUT - 2 / application of interpolation between output values of this LUT.

[0006] When the processing function represented by the LUT is nonlinear, the interpolation can be improved upon by means of non-linear interpolation. Another way to reduce the error made by this interpolation is to increase the number of bins of the LUT. Yet another method for reducing the error is to creates bins of varying size. For example, in intervals of input values in which the function has a high curvature, the bin density of the LUT may be higher. In summary, methods to make LUTs more accurate include increasing the number of elements (bins) in the LUT, applying linear or higher order interpolation, or having a variable spacing between the bins.

[0007] It should be noted that each of these well-known methods to decrease the error made by replacing a function by a LUT (+ interpolation) increases the computation time, as well as the complexity of the hardware. As a consequence, it is desirable to use LUTs with regularly sized bins and linear interpolation, even if a larger size of LUT, i.e. a high number of bins, is needed to achieve a given level of precision.

[0008] US2012188229A1 is one relevant background art disclosing alternative methods for approximating the inverse transfer function of a display device using two different small-size LUTs applied in series or in parallel.Summary of invention

[0009] The invention is defined by the appended claims.

[0010] An object of the invention is to find a more efficient way of modeling a processing function by the application of a LUT which is different from the well-known procedure where a single LUT is applied once to input values of color image data to process.

[0011] In the new processing method, a single, specific LUT is applied multiple times in succession, as illustrated in figure 1 for two times. The benefit is that for the same total number of bins, a repeated application of a single, specific, smaller LUT yields a combined error that is smaller than the error produced by a single application of a larger LUT according to the well-known procedure. Repeated application of a small LUT also has advantages in terms of hardware design over other techniques such as non-linear interpolation and uneven bin spacings, in that a very simple LUT design is repeated a number of times.Brief description of drawings

[0012] The invention will be more clearly understood on reading the description which follows, given by way of non-limiting examples and with reference to the appended figures in which: Figure 1 illustrates the representation of a processing function h(.) by an iterative application of a LUT over two iterations. Figure 2 illustrates the representation of a processing function h(x) by an iterative application of an iterative function f(x) over two iterations. Figure 3 illustrates a comparison between approximating a PQ EOTF processing function using a single application of a 5-bin LUT, or using a single application of a 10-bin LUT, or using a double application of a 5-bin LUT as in the method of figure 1. Figure 4 illustrates errors of the approximations of figure 3. Figure 5 illustrates that, by incorporating a small k-bin LUT" p"p" into an iterative function f p (x), the polynomial P f (x) approximating this iterative function f p (x) is not accurate. Figure 6 illustrates a comparison between approximating a Slog3 EOTF processing function using a single application of a 5-bin LUT, or using a single application of a 15-bin LUT, or using a triple application of a 5-bin LUT as in the iterative method disclosed herein. Figure 7 illustrates errors of the approximations of figure 6. Description of embodiments

[0013] It will be appreciated by those skilled in the art that flow charts presented herein represent conceptual views of illustrative circuitry embodying the invention. They may be substantially represented in computer readable media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.

[0014] The functions of the various elements shown in the figures may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. Explicit use of the term processor should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor ("DSP") hardware, Systems on Chip (SOCs) hardware or Field-Programmable Gate Arrays (FPGAs) hardware, read-only memory ("ROM") for storing software, random access memory ("RAM"), and non-volatile storage.

[0015] It is to be understood that the method of processing input values of data may be implemented in various forms of hardware, software, firmware, special purpose processors, or combinations thereof. The invention may be notably implemented as a combination of hardware and software. Moreover, the software may be implemented as an application program tangibly embodied on a program storage unit. The application program may be uploaded to, and executed by, a processing machine comprising any suitable architecture. Preferably, the processing machine is implemented on a platform having hardware such as one or more central processing units ("CPU"), a random access memory ("RAM"), and input / output ("I / O") interfaces. The platform may also include an operating system and microinstruction code. The various processes and functions described herein may be either part of the microinstruction code or part of the application program, or any combination thereof, which may be executed by a CPU. In addition, various other peripheral units may be connected to the platform such as an additional data storage unit and a display device. The platform may be notably a mobile device as a tablet, a communication device as a smartphone, a game device, a laptop, a camera, a chip as an encoding chip or a decoding chip, a server as a broadcast server or web server, a TV set or a Set-Top Box.

[0016] It is to be understood that input values of data to be processed by the method can be any kind of data, even if the below embodiment is described in the context of processing input values of color data.

[0017] In a first part of this embodiment, an approximation of a processing function h(x) by applying iteratively at least two times an iterative function f(x), i.e. according (f° ... °f° ... °f)(x) is explained. Then, the obtained result is extended to replace the iterative function f(x) with the application of a corresponding iterative LUT resulting to an approximative iterative function f'(x). Figure 1 illustrates the representation of a processing function h(.) by an iterative application of a LUT over two iterations. Similarly, figure 2 illustrates the representation of a processing function h(x) by an iterative application of an iterative function f(x) over two iterations ("repeated application f(f(x))"). Figure 2 shows that the iterative function f(x) has a lower curvature compared to that of the processing function h(x). The method of representing a processing function h(x) by an iterative application of an LUT or of a function f(x) can also be used similarly over more than two iterations.

[0018] A method to obtain a lookup table LUT to be applied iteratively over two iterations will be first described, in the context of processing color data x through a PQ electro-optical transfer function h PQ (x) as specified in ITU-R Recommendation BT2100. The variable x corresponds to the variable N in the PQ-EOTF quoted in the background art paragraph above.

[0019] First, a number k of bins is determined, based notably on memory size available on the processing platform. These k input values x 1 , ..., x j , ...., x k are chosen to be distributed uniformly over the range of input values. Therefore, the targeted iterative LUT will have k input values x 1 , ..., x j , ...., x k (and k corresponding output values y 1 , ..., y j , ...., y k ).

[0020] Then, an iterative function f PQ (x) is defined such that: h PQ x = f PQ f PQ x

[0021] Such an iterative function f PQ (x) can be for instance approximated in a manner known per se by a 6-degree polynomial P fPQ (x) as follows: P f PQ x = − 4.0793 e − 5 x 5 + 0.0116 x 4 + 0.2966 x 3 + 0.4146 x 2 + 0.1455 x + 0.1317

[0022] Note that, on a sampled grid of 1000 input values x, the mean absolute error of |h PQ (x)-f PQ (f PQ (x))| is 1.95e -5< , showing a high degree of correspondence between a double iteration f PQ (f PQ (x)) of the iterative function and the processing function itself h PQ (x). Similarly, the HLG opto-electrical transfer function (ITU-R Recommendation BT.2100) can be approximated with this procedure to within a mean absolute error of 9.01 e -4< .

[0023] Generally, for any processing function h (x), the iterative function f(x) cannot be obtained analytically. However, it is generally possible to approximate the iterative function f(x) using a polynomial P f (x) of a sufficiently high degree: P f x = ∑ i = 0 n m i x n − i where the polynomial P f (x) is defined by the degree n and the coefficients m i . The coefficients of the polynomial P f (x) can be determined by the equation: argmin m i h x − P f P f x 2 2

[0024] Here, an iterative polynomial P' f (x) is preferably evaluated based only on the k input values x 1 , ..., x j , ...., x k and their corresponding processing function values f(x 1 ), ... , f(x j ), ... , f(x k ). The k resulting values P' f (x 1 ), ... , P' f (x j ), ... , P' f (x k ) output by this iterative polynomial P' f (. ) then serve once more as input to this iterative polynomial P' f (x). The resulting coarsely k-discretized P' f (P' f (x j )) (with j in [1,k]) is then turned in a manner known per se into a LUT' p'p' representing P' f (P' f (x j )) and having the k input values x 1 , ..., x j , ...., x k as entries. This LUT' p'p' , which represents a function f p ′ p ′ ′ x , can be evaluated (by means of linear interpolation) at a large number i of sample points between x ∈ [0,1]. Thus, for a large number of points, say 1000, the following iterative function is evaluated: argmin m i h x − f pp ′ x 2 2

[0025] After minimizing the difference h x − f p ′ p ′ ′ x 2 2 for this large number i of sample points m i , another resulting polynomial P" p ' p '(.) with coefficients m i is once again evaluated as above at the same k sample points. This other resulting polynomial P" p'p' (x) is then turned into a final target iterative LUT" p"p" in a manner known per se.

[0026] As an example, this procedure is applied to the above PQ EOTF using a 6-degree polynomial, and a 5-bin target LUT, with results shown in figure 3. This figure illustrates a comparison between approximating the processing function ("function to approximate") using a single application of a 5-bin LUT, or using a single application of a 10-bin LUT, or using a double application of a 5-bin LUT as in the above method. This figure shows that the modeling of the processing function by a double application of a 5-bin LUT is as accurate as the modeling of the processing function by a single application of a 10-bin LUT.

[0027] The advantage of approximating a function of input values of data by applying iteratively a small size LUT to these input values will now be shown in comparison with approximating the function of input values of data by applying once a bigger LUT to these input values.

[0028] When comparing the above situation of a two-iterative 5-bin LUT" p"p" approximating the PQ EOTF with a situation of a single 10-bit LUT approximating the PQ EOTF, the mean absolute error induced by the double application of the small LUT" p"p" is 0.0046, whereas a single application of a 10-bin bigger LUT induces a higher mean absolute error of 0.0096. The double application of the small LUT" p"p" is therefore a better approximation of the PQ EOTF that a single application of the 10-bin bigger LUT.

[0029] The distribution shown in figure 4 of the absolute errors over the input range of input values shows that the error made by applying two times the smaller LUT" p"p" is the smallest over the full range of input values. It can also be seen that the error of this iterative method can be both positive and negative, so that the maximum error is small. Comparatively, as shown in figure 4, conventional single application of a LUT leads to errors that are always positive or always negative.

[0030] Finally, note that by incorporating a small k-bin LUT" p"p" into the iterative function f p (x), the approximating polynomial P f (x) is no longer accurate, as shown in figure 5. This demonstrates the interest of evaluating an iterative polynomial based not on a sampled grid of a large number of input values (for instance 1000 as described above), but based only on k input values x 1 , ..., x j , ...., x k distributed over the range of input values, where k is the number of bin of the targeted iterative LUT .

[0031] Still in the first part of this embodiment, a method to obtain a lookup table LUT to apply iteratively over more than two iterations will be described.

[0032] To approximate a function h(x) with n repeated applications of an iterative function f(x), the iterative function f p (x) is given by: argmin m i h x − f p ° f p ° … ° f p x 2 2

[0033] To approximate a function h(x) with an iterative LUT applied over n iterations, the iterative function f p (x) is given by: argmin m i h x − f pp … p ′ x 2 2 where f pp … p ′ x is a small LUT created by repeating the evaluation of polynomial f p (x) using a LUT of k bins for a total of n times.

[0034] As an example, using the above method, the so-called Slog3 opto-electrical transfer function (OETF) is approximated with an iterative 5-bin LUT applied over 3 iterations. The accuracy of this approximation is shown in figure 6 (see curve "3 LUTS, 5 elements" compared to "1 LUT, 5 elements" or compared to "1 LUT, 15 elements"). This figure illustrates that an iterative 5-bin LUT applied over 3 iterations is at least as accurate than a single application of a 15-bin LUT.

[0035] The error induced by this approximation is shown in figure 7. The mean absolute error of 3-iterated 5-bin LUT is 0.0038. For comparison, a conventional single-applied 15-bin LUT induces a higher mean absolute error of 0.0048.

[0036] Still in the first part of this embodiment, it may be noticed that, once a processing function h(x) is given, there is generally no uniquely defined solution to determine a iterative function such that: h x = f ° f x

[0037] However, under certain circumstances it is possible to approximate an iterative function to an arguably simpler problem, namely to find an approximation to the iterative function f such that when applied twice it approximates the processing function h(x): f ′ ° f ′ x ≈ h x where f' is an approximation to f.

[0038] In a second part of this embodiment, a method of processing input values of data x using a lookup-table (LUT) comprising applying iteratively this lookup-table over at least two iterations will be briefly described. This iterative lookup-table is notably defined as described in the first part of this embodiment, using usual computing means.

[0039] The method is notably implemented on a platform comprising at least one processor configured in a manner known per se to implement it. This platform can be for instance a mobile device as a tablet, a communication device as a smartphone, a game device, a laptop, a camera, a chip as an encoding chip, a server as a broadcast server or web server, a TV set or a Set-Top Box.

[0040] This processing method comprises at least two iterations : a first iteration in which the lookup-table is applied to the input values resulting in first output values and a second iteration in which the lookup-table is applied to the first output values resulting in second output values.

[0041] When the input value is different from an input value x 1 , ..., x j , ...., x k of the lookup table, an interpolation is performed between input values of the lookup table to output an output value that is interpolated between the corresponding output values of the lookup table. Such an interpolation is performed in an manner known per se in the field of lookup tables. Preferably, this interpolation is linear, saving then advantageously computing resources. Although interpolation is linear, the approximation of a processing function remains accurate due notably to the multiple iterations of the method.

[0042] Preferably, input values to process represents colors. For instance, input values to process are R,G,B values representing colors in a RGB color space. Preferably, as described in the first part of the embodiment above, the at least two iterations of the lookup table models a transfer function such as an Electro-Optical Transfer Function (EOTF), an Optical Electrical Transfer Function (OETF), a gamma function or a Slog / Slog2 / Slog3 function, or models a tone mapping or an inverse tone mapping function. By replacing these processing functions with multiple iterations of one LUT, a high accuracy approximation at a low computational cost may be obtained.

[0043] A processing function h(x) that can be modeled by iterative application of a given LUT preferably adheres to the following characteristics.

[0044] First, the processing function h(x) is preferably strictly monotonically increasing or strictly monotonically decreasing: dh x dx > 0 ∀ x ∈ 0 1 or dh x dx < 0 ∀ x ∈ 0 1

[0045] This means that the processing function h(x) is preferably bijective, although may have inflection points.

[0046] Further, a given non-linear processing function h(x) may have over its range of values a certain (varying) curvature. This curvature, along with the bin spacing and the assumed use of linear interpolation, has direct impact on the accuracy of a LUT. For example, for processing functions that have a smaller curvature (i.e., they are straighter), the accuracy of a corresponding LUT will be higher than for other functions.

[0047] Still further, when a highly non-linear target function h(x) is decomposed into a double application of a same f(x) such that h(x) = (f°f)(x), then the curvature of f(x) is less than that of h(x) in the range x ∈ [0,1]. Thus, a LUT with a given number of bins approximating f(x) will be of higher accuracy than a LUT with the same number of bins approximating directly h(x). In practice, it is found that applying two k-bin LUTs in succession to approximate (f°f)(x) is twice as accurate as approximating h(x) directly with a single 2k-bin LUT.

[0048] Note that when the processing function h(x) to be approximated is linear, finding a solution f(x) such that h(x) = (f°f)(x) is trivial. Assuming that h(x) = kx, then f(x) can be chosen to be f x = k x so that: f ° f x = k 2 x = kx = h x

[0049] When h(x) = x γ< (as for very usual so-called gamma functions) , the iterative function can be f x = x γ , so that: f ° f x = x γ γ = x γ = h x

[0050] Preferably, both input and output of a processing function are bounded to a given range. Such a range may be bounded to be between 0 and 1, although any number of other ranges are possible: x ∈ 0 1 h x ∈ 0 1

[0051] The method of processing input values of data is notably advantageous, because an approximation of a processing function h(x) by multiple iteration of a given function f(x) has been found generally more accurate than an approximation of this processing function h(x) by different intermediate functions f 1 (x),...,f r (x),..,f Q (x) such that h(x) = f 1 ( ... (f r ( ... (f Q (x)) ... )) ... ).

[0052] Although the illustrative embodiments of the invention have been described herein with reference to the accompanying drawings, it is to be understood that the present invention is not limited to those precise embodiments, and that various changes and modifications may be effected therein by one of ordinary skill in the pertinent art without departing from the invention. All such changes and modifications are intended to be included within the scope of the present invention as set forth in the appended claims.

Claims

1. A computer implemented method of processing data, comprising: inputting values of color image data; and applying an identical look-up table to said input values of color image data over at least two iterations thereby obtaining output values of color image data, wherein the at least two iterations of the identical look-up table models a non-linear processing function, and wherein the at least two iterations comprise a first iteration in which said lookup-table, LUT, is applied to said input values resulting in first output values and in which said LUT is applied to said first output values resulting in second output values.

2. The method of claim 1, further comprising interpolating for said iterative application of said LUT.

3. The method of claim 2, wherein said interpolating for an application of said LUT is performed between input values of said LUT and between output values of said LUT.

4. The method of claim 3, wherein said interpolating is linear.

5. The method of any one of the previous claims, wherein said LUT is multidimensional.

6. The method of any one of the previous claims, wherein the at least two iterations of said LUT model the processing function which is non-linear and bijective.

7. The method of claim 6, wherein said processing function is chosen in the group composed of a transfer function, a tone mapping, and an inverse tone mapping function.

8. A device comprising at least one processor configured to implement the method according to any one of claims 1 to 7.

9. The device of claim 8 chosen in the group composed of a mobile device, a communication device, a game device, a laptop, a camera, a chip, a server, a TV set and a Set-Top Box.

10. A computable readable storage medium comprising stored instructions that when executed by a processor performs the method of any one of claims 1 to 7.