System and method for ambient light compensation using PQ shift

Abstract

A novel method and system for compensating for ambient light around a display is disclosed. A shift of the PQ curve applied to an image can compensate for non-optimal ambient lighting conditions for display, where the PQ shift is either adding to a compensation value in PQ space followed by subtracting a compensation value in linear space, or adding to a compensation value in linear space and subtracting a compensation value in PQ space. Further adjustments to the PQ curve may be made to provide improved image quality in terms of image brightness.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 63 / 046,015, filed June 30, 2020, and European Patent Application No. 20183195.5, filed June 30, 2020, both of which are incorporated by reference in their entireties.

[0002] Technical Field The present disclosure relates to improvements for the processing of video signals, and in particular to processing video signals to improve display in different ambient lighting conditions. [Background technology]

[0003] A reference electro-optical transfer function (EOTF) for a given display characterizes the relationship between the color values ​​(e.g., luminance) of the input video signal and the output screen color values ​​(e.g., screen luminance) produced by the display. For example, “Color Space and Gamut Information,” IEEE Transactions on Color Transformation, vol. 13, No. 1, pp. 1171-1175, 2003, which is incorporated herein by reference in its entirety, defines a reference EOTF for flat panel displays based on the measured characteristics of a cathode ray tube (CRT). Given a video stream, information about its EOTF is typically embedded in the bitstream as metadata. As used herein, the term “metadata” refers to any auxiliary information that is transmitted as part of an encoded bitstream and assists a decoder to render a decoded image. Such metadata may include, but is not limited to, color space or gamut information, reference display parameters, and auxiliary signal parameters, as described herein. [Non-Patent Document 1] ITU Recommendation ITU-R BT. 1886, "Reference electro-optical transfer function for flat panel displays used in HDTV studio production", 03 / 2011 Summary of the Invention [Problem to be solved by the invention]

[0004] Currently, most consumer desktop displays have a resolution of 200-300cd / m 2 Display devices support a brightness of 1000 nits or 1000 nits. Most consumer HDTVs are in the 300-500 nits range, with newer models reaching 1000 nits. Commercially available smartphones typically range from 200-600 nits. These different display brightness levels pose a challenge when trying to display an image under different ambient lighting scenarios, as shown in FIG. 1. A viewer 110 is viewing an image (e.g., a video) on a screen 120. The image brightness 130 can be "washed out" by ambient light 140. The brightness level of the ambient light 140 can be measured by a sensor 150 in, on, or near the display. The brightness of the ambient light can vary, for example, from 5 nits in a dark room to 200 nits in a well-lit room with no sunlight, or 400 nits in a room with indirect sunlight, to over 600 nits outdoors. One solution has been to add a linear adjustment to the brightness control of the display, but this can result in an imbalance in the brightness of the display. [Means for solving the problem]

[0005] Various video processing systems and methods are disclosed herein. Some such systems and methods may include compensating an image to maintain image appearance with changes in surrounding environmental brightness levels. The methods may, in some embodiments, be computer-implemented. For example, the methods may be implemented, at least in part, via a control system having one or more processors and one or more non-transitory storage media.

[0006] In some examples, systems and methods are described for modifying an image to compensate for ambient lighting conditions around a display device, including determining a PQ curve for the image; determining a PQ shift for the PQ curve based on the ambient lighting conditions and a compensation value determined from the image, the PQ shift consisting of: adding to a compensation value in PQ space followed by subtracting the compensation value in linear space, or adding to the compensation value in linear space followed by subtracting the compensation value in PQ space; applying the PQ shift to the PQ curve to generate a shifted PQ curve; and modifying the image using the shifted PQ curve.

[0007] In some such examples, the method may include applying a tone map to the image before modifying the image. In some such examples, the method may be performed by software, firmware or hardware and may be part of a video decoder.

[0008] Some or all of the methods described herein may be performed by one or more devices according to instructions (e.g., software) stored in one or more non-transitory media. Such non-transitory media may include memory devices such as those described herein, including, but not limited to, random access memory (RAM) devices, read-only memory (ROM) devices, and the like. Thus, various innovative aspects of the subject matter described in this disclosure may be implemented in a non-transitory medium having software stored thereon. The software may be executable, for example, by one or more components of a control system as disclosed herein. The software may include, for example, instructions for performing one or more of the methods disclosed herein.

[0009] At least some aspects of the present disclosure may be implemented via an apparatus or apparatuses. For example, one or more devices may be configured to perform, at least in part, the methods disclosed herein. In some implementations, the apparatus may include an interface system and a control system. The interface system may include one or more network interfaces, one or more interfaces between the control system and a memory system, one or more interfaces between the control system and another device, and / or one or more external device interfaces. The control system may include at least one of a general-purpose single-chip or multi-chip processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic, or discrete hardware components. Thus, in some implementations, the control system may include one or more processors and one or more non-transitory storage media operatively coupled to the one or more processors.

[0010] Details of one or more implementations of the subject matter described herein are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages will become apparent from the specification, drawings, and claims. Please note that the relative dimensions of the following figures may not be drawn to scale. Like reference numbers and symbols in the various drawings generally indicate like elements, although different reference numbers do not necessarily indicate different elements between different drawings. [Brief description of the drawings]

[0011] [Figure 1] 4 shows an example of ambient light for a display.

[0012] [Diagram 2]1 shows an exemplary flow chart for a method of compensating for ambient light around a display.

[0013] [Diagram 3] 1 shows an exemplary graph of experimental data for the square root of image center PQ versus compensation value under different ambient lighting conditions.

[0014] [Figure 4] 1 is an exemplary graph of a fitted line for surround luminance PQ versus the slope of the experimental data.

[0015] [Diagram 5] 13 shows an exemplary graph of the fitted line for the y-intercept of the ambient luminance PQ versus the experimental data.

[0016] [Figure 6] 1 illustrates an exemplary PQ shift compensation curve.

[0017] [Figure 7] 1 shows an exemplary PQ shift compensation curve adjusted to reduce brightening.

[0018] [Figure 8] 1 shows an exemplary PQ shift compensation curve with relaxation added to avoid artifacts.

[0019] [Figure 9] 9A and 9B show exemplary PQ shift compensation curves with a clamp set below the visual threshold.

[0020] [Figure 10] 13 illustrates an exemplary PQ shift compensation curve with renormalization.

[0021] [Figure 11] 1 shows an exemplary PQ shift compensation curve adjusted for reflection. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] The term "PQ" as used herein refers to perceptual luminance amplitude quantization. The human visual system responds to increasing light levels in a highly nonlinear manner. The term "PQ space" as used herein refers to a nonlinear mapping of linear luminance amplitudes to nonlinear PQ luminance amplitudes as described in Rec.BT.2100. A human's ability to see a stimulus is affected by the luminance of the stimulus, the size of the stimulus, the spatial frequencies that make up the stimulus, and the luminance level to which the eye is adapted at the particular time the stimulus is viewed. In one example, a perceptual quantization function maps linear input gray levels to output gray levels that better match the contrast sensitivity threshold in the human visual system. An example of a PQ mapping function (or EOTF) is described in SMPTE ST2084:2014 "High Dynamic Range EOTF for Mastering Reference Displays," where, given a fixed stimulus size, for each luminance level (i.e., stimulus level), the minimum visible contrast increment at that luminance level is selected (according to the HVS model) according to the most sensitive adaptation level and the most sensitive spatial frequency. Compared to traditional gamma curves, which represent the response curve of a physical cathode ray tube (CRT) device and which happen to bear a very rough similarity to the way the human visual system responds, the PQ curve mimics the true visual response of the human visual system using a relatively simple functional model.

[0023] In this paper, a solution is described to the problem of adjusting the brightness of a display to match ambient lighting conditions by applying compensation to the image as a shift in PQ. Figure 2 shows an exemplary method for applying such compensation to an image on a display.

[0024] To generate data for the ambient light luminance measurement, sensor data 210 is taken from an area surrounding the display. The sensor data can be taken from one or more luminance sensors, which include light-sensitive elements such as photoresistors, photodiodes, and phototransistors. This sensor data is then used to calculate the ambient luminance PQ 220, which can be denoted as S. This calculation, as with all calculations described herein, can be performed locally to the display, such as on a processor or computer within or connected to the display, or can be performed on a remote device or server that delivers the image to the device.

[0025] Given an ambient luminance PQ S, two intermediate values ​​(here M and B) can be calculated as a function of S. In one example, M and B are calculated from the following equations: M=a*S+b formula 1 B=c*S 2 +d*S+e formula 2 where a, b, c, d, and e are constants. In this example, M is a linear function of S, and B is a quadratic function of S. The constants can be determined experimentally as shown herein. Image 240 can be analyzed for the range of luminance (e.g., luma values) it contains.

[0026] The images can be frames of a video. The images can be key frames of a video stream. From these luminance data, a mid PQ can be determined 250 from the complete image. The mid PQ can represent the average luminance of the image. An example of calculating the mid PQ is taking the average of the maximum values ​​of each component (e.g., R, G, and B) of the downsampled image. Another example of calculating the mid PQ is taking the average of the maximum values ​​of each component (e.g., R, G, and B) of the downsampled image. B C R The median PQ value is the average of the Y values ​​of the image in color space. This median PQ value can be denoted as X. The intermediate PQ value, the minimum and maximum can be calculated on the encoder side and provided in the metadata, or can be calculated on the decoder side.

[0027] From the calculated M and B values ​​230 and the calculated X value 250, a compensation value can be calculated 260. This compensation value can be denoted as C and can be calculated from the following formula: C=M(√X)+B Equation 3 The square root of X is used in this example because it allows for a linear relationship to the experimental data. It is possible to calculate C from X, but it results in a more complicated function. Keeping the function linear allows for easier calculations, especially when implemented in hardware rather than software.

[0028] The compensation value C can then be used in step 270 to correct the image with a PQ-shifted PQ curve. The PQ shift can be expressed as: PQ out =L2PQ(PQ2L(PQ in +C)-PQ2L(C)) Equation 4 Here, PQ out is the resulting PQ after the shift, and PQ in are the original PQ values, L2PQ() is the function that converts from linear space to PQ space, PQ2L() is the function that converts from PQ space to linear space, and C is the compensation value (for given values ​​of M and B for X of the image in question and the measured ambient light). Conversion between linear and PQ spaces is known in the art, for example as described in "Compensation for the Illumination of a 3D Image," by G. K., et al., "Compensation for the Illumination of a 3D Image," by G. K. K., et al., "Compensation for the Illumination of a 3D Image," by G. K. K., et al., "Compensation for the Illumination of a 3D Image," by G. K. K., and G. K ... T C P , Y.C. B C R Compensation can be done after tone mapping in chroma-separated spaces such as RGB, where the compensation is applied to each channel separately. Processing can be done on the luma (e.g., I) component, but chromatic adjustments can also be useful to preserve the intent of the content. Compensation can also be done after tone mapping in other color spaces such as RGB, where the compensation is applied to each channel separately. [Non-Patent Document 1] ITU-R BT.2100, "Image parameter values ​​for high dynamic range television for use in production and international program exchange"

[0029] This method provides compensation to images in high ambient surround luminance environments (e.g., outdoors in sunlight) to match how they would appear in an ideal ambient environment (e.g., a very dark room). An example of an ideal ambient target is 5 nits (cd / m 2 ) The dark detail contrast is increased to ensure that details remain visible. This method provides compensation to the image, such as when the surrounding ambient luminance environment is brighter than the reference value. The reference value may be a specific value or a range of values.

[0030] In another embodiment, the compensation is reversed to allow compensation for darker than ideal ambient lighting conditions. Such compensation is for cases where the ambient luminance environment is darker than the reference value. For example, if the image was originally intended to be viewed in a brightly lit room, the compensation can be set so that it has the correct appearance in a dark room. For this embodiment, the operation is reversed to have an addition in linear space and a subtraction in PQ space, as shown in the following equation: PQ out =L2PQ(PQ2L(PQ in )+PQ2L(C))-C Equation 5

[0031] In an embodiment, the compensation value C is determined experimentally by subjectively determining compensation values ​​for various image illumination values ​​under different ambient light conditions. An example would be to obtain data through psychovisual experiments where observers subjectively select the appropriate amount of compensation for various images at different ambient luminance levels. An example of this type of data is shown in FIG. 3. The graph shows data points 310 of the square root of the image median PQ value plotted against subjectively selected compensation values ​​for five different ambient light conditions (in this case 22, 42, 77, 139, and 245 nits; from a dark room to well-lit conditions). From these points 310, a trend line 320 can be fitted for the data points for each ambient light condition. Because the square root of the image median is used, it is easier to fit these points with a linear regression. Images with bright PQ medians in dark ambient conditions have data points 330 that bottom out at zero compensation. These points are not considered for the fit because they would erroneously distort the trend line.

[0032] From these lines 320, two useful values ​​can be determined: the slope of the line, ΔCompensation / Δsqrt(ImageMid), and the y-intercept, i.e., the value of Compensation where sqrt(ImageMid)=0, where sqrt(x) represents the square root of x, e.g., √x. These slopes and y-intercepts can then also be fitted to further functions, as shown in Figures 4 and 5.

[0033] FIG. 4 shows an example of fitting a line 410 (linear regression) to the slope of a line (e.g., that shown in FIG. 3) of Compensation versus sqrt(ImageMid) versus the peripheral (surrounding) luminance PQ. In some embodiments, extra data points 420 are added for the fit. This is so that the slope and the peripheral luminance PQ give zero compensation for the reference (ideal) peripheral luminance. From this fit, a function of M in terms of the peripheral luminance S can be found (see FIG. 2) for use in Equation 1. This allows the calculation of compensation values ​​a and b for Equation 1 (a is the slope of this fitted line and b is the y-intercept of this fitted line). These values ​​can then be put into Equation 1 along with the measured S peripheral luminance to determine the M value for that peripheral luminance (e.g., 5 nits).

[0034] 5 shows an example of fitting a curve 510 (a second order polynomial) to the y-intercept of a line (e.g., that shown in FIG. 3) of Compensation versus sqrt(ImageMid) versus the peripheral (surrounding) luminance PQ. In some embodiments, an extra data point 520 is added so that the y-intercept and the peripheral luminance PQ give zero compensation for some reference (ideal) peripheral luminance.

[0035] FIG. 6 shows an example PQ-shift (PQ Surround Adjustment) produced by Equation 4. The three filled circles represent the minimum 610, midpoint 620, and maximum 630 of the image after tone mapping has been performed. The solid line 640 is the adjustment using the PQ-shift method with a compensation value of 0.3 (calculated from Equation 4). The dashed line 650 represents the value without compensation. The minimum 610 of the image is located at approximately [0.01, 0.21]. Since the image does not contain any content below this level, the image may be over-brightened in this example.

[0036] In some embodiments, this over-brightening problem can be overcome by performing an additional shift in the PQ curve. This compensation can be achieved by shifting the PQ value based on the minimum pixel value of the tone mapped image, so that contrast enhancement is maintained only where that pixel is located, and over-brightening artifacts are minimized. An example of this is shown in FIG. 7, where the curve 640 from FIG. 6 is shifted to produce a new curve 740, where the minimum point 710 is shifted to zero compensation 650 (PQ in =PQ out ) and other values ​​including the midpoint 720 and maximum 730 are adjusted accordingly from that shift.

[0037] In some embodiments, an additional adjustment to the PQ compensation curve can be made to prevent banding artifacts caused by a sharp cutoff at the minimum value. Ease can be implemented by a cubic roll of the input points within some small value (e.g., 36 / 4096) of the minimum PQ (TminPQ) of the image. This value can be found by experimentally determining the minimum value that reduces the banding artifacts. The value can also be chosen arbitrarily, for example by visualizing the ease and determining what value provides a smooth transition to the zero compensation point.

[0038] Figure 8 shows an example of the use of relaxation to prevent banding. The original compensation curve 840 has a sharp transition 845 at the intersection with the zero compensation line 650. An in-out relaxation is performed from the minimum PQ of the image (which for this example is at the intersection point 845, as shown, for example, in Figure 7) to a point incremented by some small value above the minimum PQ (e.g., TminPQ+36 / 4096). The relaxation can be a cubic roll-off function that returns a value between 0 and 1, where 0 is returned near the minimum PQ and 1 is returned at incremented values. An example algorithm in (MATLAB®) is as follows, where, in one embodiment, without limitation, cubicEase() is a monotonically increasing sigmoid-like function for input PQ values ​​between TminPQ and TminPQ+36 / 4096 and output alpha in [0,1]: [Table 1]

[0039] As used herein, the term "ease" refers to a function that applies a nonlinear function to the data such that a Bezier or spline transformation / interpolation is applied (changing the curvature of the graphed data). "Ease-in" refers to a transformation near the beginning of the data (near zero) and "ease-out" refers to a transformation near the end of the data (near the maximum). "In-and-out" refers to a transformation near both the beginning and the end of the data. The specific algorithm for the transformation depends on the type of relaxation. There are several relaxation functions known in the art, such as cubic in-and-out, sinusoidal in-and-out, quartic in-and-out, quadratic in-and-out, etc. Relaxation is applied in and out of the curve to prevent sharp transitions.

[0040] In some embodiments, compensation can be clamped so that it is not applied below a threshold PQ value to prevent unnecessary stretching of dark details that would not be visible under ideal ambient lighting conditions (e.g., 5 nits of ambient light). The threshold PQ value can be determined experimentally by determining the point at which a human viewer is no longer able to determine detail under ideal conditions (e.g., 5 nits of ambient light, viewing at a distance of 3 picture heights). For these embodiments, (PQ in, below this threshold PQ no PQ shift (equation 4) is applied. An example of this is shown in Figures 9A and 9B. Figure 9A shows a graph of PQ compensation 910 (as shown in Figure 6) and PQ compensation with excess brightness adjustment 920 (as shown in Figure 7), along with a line 930 indicating the PQ threshold below which details would be indistinguishable under ideal conditions. Figure 9B shows the graph of Figure 9A zoomed in near the origin. This step is done after tone mapping and can be important for displays with low black levels, such as OLED displays.

[0041] In some embodiments, the compensation can be clamped to have a maximum value, for example 0.55. This can be done with or without the threshold PQ clamp described above. Maximum clamping can be useful for hardware implementation. Below is an example MATLAB® code to show an example algorithm for maximum clamping at 0.55. Here, ambient compensation is applied based on the target ambient luminance at PQ(Surr) and the source median (L1Mid) of the image. A, B, C, D, E are experimentally derived values ​​for a, b, c, d, e shown in Equations 1, 2 above: [Table 2]

[0042] In some embodiments, the PQ compensation curve is in This can be simplified to be linear over the points. For example, the compensation can be calculated to be linear over a PQ of 0.5 (out of the full range

[0001] ), providing the following exemplary algorithm: PQ in If <0.5 PQ out =L2PQ(PQ2L(PQ in +C)-PQ2L(C)) Equation 6 PQ in If ≧0.5 PQ out =PQ in +C formula 7 This simplification on that particular PQ point may be useful for hardware implementation of the method.

[0043] In some cases, ambient light compensation may push some pixels outside the range of the target display. In some embodiments, a roll-off curve can be further applied to compensate for this and re-normalize the image to the correct range. This can be done by using a tone mapping curve along with source metadata (e.g., metadata describing the minimum, average (or midpoint), and maximum luminance). Without limitation, exemplary tone mapping curves are described in U.S. Patent Nos. 10,600,166 and 8,593,480, both of which are incorporated herein by reference in their entirety. Take the resulting minimum, midpoint, and maximum values ​​of the tone mapped image (before applying ambient light compensation, e.g., Equation 4), apply ambient light compensation to those values, and then map the resulting image to the target display using a tone mapping technique. See, for example, U.S. Patent Application Publication No. 2019 / 0304379, which is incorporated herein by reference in its entirety. An example of a roll-off curve is shown in FIG. 10. The main feature of this roll-off is that the minimum 1010 and maximum 1020 points remain within the range of the target display. The result is that, due to the dynamic tone mapping nature of our tone curve, the brighter image 1030 will have less highlight roll-off (at the expense of dark / center contrast enhancement) and the darker image 1040 will have more dark detail enhancement (at the expense of highlight detail).

[0044] In some embodiments, further compensation can be performed to compensate for reflections from the display screen. In some embodiments, the amount of light reflected from the screen can be estimated from the sensor values ​​using the reflective properties of the screen, as in Equation 8: Reflected Light = Sensor Brightness * Screen Reflection Equation 8 The light reflected from the screen can be treated as a linear addition of light to the image, essentially raising the black level of the display. In these embodiments, tone mapping is done to a higher black level (e.g., to the level of reflected light), and at the end of the tone curve calculation, a subtraction is done in linear space to compensate for the additional luminosity due to reflections. See, for example, Equation 9. PQ out =L2PQ(PQ2L(PQ in )-ReflectedLight) Equation 9 An example of a tone curve with reflectance compensation is shown in Figure 11. The minimum 1110 and maximum 1120 levels remain the same as before reflectance compensation was applied, but the contrast at the low end 1130 has increased substantially on the curve 1140 applied to the pixel. Adding in the expected reflectance results in a perceived tone curve 1150 that is closer to the desired image quality.

[0045] Although several embodiments of the present disclosure have been described, it will be understood that various modifications can be made without departing from the spirit and scope of the disclosure. Accordingly, other embodiments are within the scope of the following claims.

[0046] Thus, as described herein, embodiments of the present invention may relate to one or more of the exemplary embodiments enumerated below. Thus, the present invention may be embodied in any of the forms described herein, including, but not limited to, the following enumerated exemplary embodiments (EEE) that describe the structure, features, and functionality of some portions of the present invention.

[0047] [EEE1] A method for modifying an image to compensate for ambient light conditions around a display device, the method comprising: determining perceptual luminance amplitude quantization (PQ) data for the image; determining a PQ shift for the PQ data based on the ambient light conditions and a compensation value determined from the image, the PQ shift consisting of: adding to the compensation value in PQ space followed by subtracting the compensation value in linear space, or adding to the compensation value in linear space followed by subtracting the compensation value in PQ space; and applying the PQ shift to the image to modify the PQ data of the image.

[0048] [EEE2] The method of bulleted exemplary embodiment 1, further comprising the step of applying a tone map to the image before applying the PQ shift.

[0049] [EEE3] The method of bulleted exemplary embodiment 1 or 2, wherein the compensation value is calculated from C=M√X+B, where C is the compensation value, M is a function of the surrounding luminance value, X is the median PQ value of the image, and B is a function of the surrounding luminance value.

[0050] [EEE4] The method of itemized exemplary embodiment 3, wherein functions M and B are derived from experimental data derived from subjective perceptual evaluation of image PQ compensation values ​​under different ambient lighting conditions.

[0051] [EEE5] The method of any one of bulleted exemplary embodiments 3 or 4, wherein M is a linear function of the surrounding luminance values ​​and B is a quadratic function of the surrounding luminance values.

[0052] [EEE6] The method of any one of itemized exemplary embodiments 1 to 5, further comprising applying an additional PQ shift to the image, the additional PQ shift adjusting the image so that a minimum pixel value has a compensation value of zero.

[0053] [EEE7] ​​The method of any one of itemized exemplary embodiments 1 to 6, further comprising applying a relaxation to the PQ shift.

[0054] [EEE8] The method of any one of bulleted exemplary embodiments 1-7, further comprising clamping the PQ shift so that it is not applied below a threshold.

[0055] [EEE9] The method of any one of itemized exemplary embodiments 1 to 8, wherein the PQ shift is calculated as a linear function over a predetermined PQ.

[0056] [EEE10] The method of any one of itemized exemplary embodiments 1 to 9, further comprising applying a roll-off curve to the image.

[0057] [EEE11] The method of any one of itemized exemplary embodiments 1 to 10, further comprising subtracting a reflection compensation value from the PQ data in linear space at the end of a tone curve calculation that provides compensation for expected screen reflection on the display device.

[0058] [EEE12] The method of claim 11, wherein the reflection compensation value is a function of an ambient luminance value of the device.

[0059] [EEE13] The method of any one of itemized exemplary embodiments 1 to 12, wherein applying the PQ shift is performed in hardware or firmware.

[0060] [EEE14] The method of any one of itemized exemplary embodiments 1 to 12, wherein applying the PQ shift is performed in software.

[0061] [EEE15] The method of any one of itemized exemplary embodiments 1 to 14, wherein the ambient light conditions are determined by a sensor within, on, or near the display device.

[0062] [EEE16] A video decoder having hardware and / or software configured to perform the method described in any one of itemized exemplary embodiments 1 to 12.

[0063] [EEE17] A non-transitory computer-readable medium having stored thereon software instructions which, when executed by a processor, cause the device to perform a method as recited in any one of itemized exemplary embodiments 1-12.

[0064] [EEE18] A system having at least one processor configured to execute the method described in any one of itemized exemplary embodiments 1 to 12.

[0065] The present disclosure is directed to certain implementations for purposes of describing some innovative aspects described herein and examples of contexts in which these innovative aspects may be implemented. However, the teachings herein may be applied in a variety of different ways. Moreover, the described embodiments may be implemented in a variety of hardware, software, firmware, and the like. For example, aspects of the present application may be embodied, at least in part, in an apparatus, a system including multiple devices, a method, a computer program product. Thus, aspects of the present application may take the form of hardware embodiments, software embodiments (including firmware, resident software, microcode, etc.), and / or embodiments combining both software and hardware aspects. Such embodiments may be referred to herein as "circuits," "modules," "devices," "apparatus," or "engines." Some aspects of the present application may take the form of a computer program product embodied in one or more non-transitory medium(s) having computer-readable program code embodied thereon. Such non-transitory media may include, for example, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. Thus, the teachings of the present disclosure are not intended to be limited to the implementations shown in the drawings and / or described herein, but instead have broad applicability. Several aspects will be described. [Aspect 1] 1. A method of modifying an image to compensate for ambient lighting conditions around a display device, the method comprising: determining perceptual luminance amplitude quantization (PQ) data for said image; determining a PQ shift for the PQ data based on the ambient light conditions and a compensation value determined from the image, the PQ shift consisting of: adding to the compensation value in PQ space followed by subtracting the compensation value in linear space, or adding to the compensation value in linear space followed by subtracting the compensation value in PQ space; applying the PQ shift to the image to modify the PQ data of the image. method. [Aspect 2] 1. A method of modifying an image to compensate for ambient lighting conditions around a display device, the method comprising: determining perceptual luminance amplitude quantization (PQ) data for said image; determining a PQ shift for the PQ data based on the ambient light conditions and a compensation value determined from the image; The compensation value is calculated from C=M(√X)+B, where C is the compensation value, M is a function of the surrounding luminance value S, X is the median PQ value of the image representing the average luminance of the image, and B is a function of the surrounding luminance values, where M=a*S+b and B=c*S 2 +d*S+e, where a, b, c, d and e are constants; the PQ shift consists of: adding to the compensation value in PQ space, calculated by PQout=L2PQ(PQ2L(PQin+C))-PQ2L(C), for when the surrounding luminance environment is brighter than the reference value, followed by subtracting the compensation value in linear space, or adding to the compensation value in linear space, calculated by PQout=L2PQ(PQ2L(PQin)+PQ2L(C))-C, for when the surrounding luminance environment is darker than the reference value, followed by subtracting the compensation value in PQ space, where PQout is the resulting PQ after the shift, PQin is the original PQ value, L2PQ() is a function that converts from linear space to PQ space, and PQ2L() is a function that converts from PQ space to linear space; applying the PQ shift to the image to modify the PQ data of the image. method. [Aspect 3] applying a tone map to the image before applying the PQ shift. 3. The method according to embodiment 1 or 2. Aspect 4 4. The method of claim 1 or 3, wherein the compensation value is calculated from C=M(√X)+B, where C is the compensation value, M is a function of surrounding luminance values, X is a median PQ value of the image, and B is a function of surrounding luminance values. Aspect 5 The method of embodiment 4, wherein functions M and B are derived from experimental data derived from subjective perceptual evaluation of image PQ compensation values ​​under different ambient lighting conditions. Aspect 6 6. The method of claim 4 or 5, wherein M is a linear function of the surrounding luminance values ​​and B is a quadratic function of the surrounding luminance values. Aspect 7 7. The method of any one of aspects 1 to 6, further comprising applying an additional PQ shift to the image, the additional PQ shift adjusting the image so that a minimum pixel value has a compensation value of zero. Aspect 8 8. The method of any one of the preceding embodiments, further comprising applying a relaxation to the PQ shift. Aspect 9 9. The method of any one of embodiments 1-8, further comprising clamping the PQ shift such that it is not applied below a threshold. Aspect 10 10. The method of any one of the preceding aspects, wherein the PQ shift is calculated as a linear function above a pre-determined PQ. Aspect 11 11. The method of any one of embodiments 1-10, further comprising applying a roll-off curve to the image. Aspect 12 12. The method of any one of aspects 1 to 11, further comprising subtracting a reflection compensation value from the PQ data in linear space at the end of a tone curve calculation that provides compensation for expected screen reflection on the display device. Aspect 13 13. The method of embodiment 12, wherein the reflectance compensation value is a function of an ambient luminance value of the device. Aspect 14 14. The method of any one of embodiments 1-13, wherein applying the PQ shift is performed in hardware or firmware. Aspect 15 14. The method of any one of embodiments 1-13, wherein applying the PQ shift is performed in software. Aspect 16 16. The method of any one of aspects 1-15, wherein the ambient light conditions are determined by a sensor within, on, or near the display device. Aspect 17 14. A video decoder having hardware and / or software configured to perform a method according to any one of aspects 1 to 13. Aspect 18 A non-transitory computer readable medium having stored thereon software instructions that, when executed by a processor, cause the method of any one of aspects 1 to 13 to be performed. Aspect 19 A system having at least one processor configured to perform the method of any one of aspects 1 to 13.

Claims

1. 1. A method of modifying an image to compensate for ambient lighting conditions around a display device, the method comprising: determining perceptual luminance amplitude quantization (PQ) data for said image; determining a PQ shift for the PQ data based on the ambient light conditions and a compensation value determined from the image, the PQ shift consisting of: adding to the compensation value in PQ space followed by subtracting the compensation value in linear space, or adding to the compensation value in linear space followed by subtracting the compensation value in PQ space; applying the PQ shift to the image to modify the PQ data of the image; the compensation value is calculated from C=M(√X)+B, where C is the compensation value, M is a first function of the surrounding luminance values ​​S, X is the median PQ value of the image, and B is a second function of the surrounding luminance values ​​S, M is a linear function of the surrounding luminance values ​​S, and B is a quadratic function of the surrounding luminance values ​​S; method.

2. 1. A method of modifying an image to compensate for ambient lighting conditions around a display device, the method comprising: determining perceptual luminance amplitude quantization (PQ) data for said image; determining a PQ shift for the PQ data based on the ambient light conditions and a compensation value determined from the image; The compensation value is calculated from C=M(√X)+B, where C is the compensation value, M is a linear function of the surrounding luminance value S, X is the median PQ value of the image representing the average luminance of the image, and B is a quadratic function of the surrounding luminance value S, where M=a*S+b and B=c*S 2 + d*S + e, where a, b, c, d and e are constants; the PQ shift consists of: adding to the compensation value in PQ space, calculated by PQout=L2PQ(PQ2L(PQin+C))-PQ2L(C), for when the surrounding ambient luminance environment is brighter than the reference value, followed by subtracting the compensation value in linear space, or adding to the compensation value in linear space, calculated by PQout=L2PQ(PQ2L(PQin)+PQ2L(C))-C, for when the surrounding ambient luminance environment is darker than the reference value, followed by subtracting the compensation value in PQ space, where PQout is the resulting PQ after the shift, PQin is the original PQ value, L2PQ() is a function that converts from linear space to PQ space, and PQ2L() is a function that converts from PQ space to linear space; applying the PQ shift to the image to modify the PQ data of the image. method.

3. applying a tone map to the image before applying the PQ shift. The method of claim 1.

4. 3. The method of claim 1 or 2, wherein the functions M and B are derived from experimental data derived from subjective perceptual evaluation of image PQ compensation values ​​under different ambient lighting conditions.

5. 2. The method of claim 1, further comprising applying an additional PQ shift to the image, the additional PQ shift adjusting the image such that a minimum pixel value of the tone mapped image has a compensation value of zero.

6. The method of claim 1 , further comprising applying a relaxation function to the PQ shift.

7. The method of claim 1 , further comprising clamping the PQ shift so that it is not applied if an original PQ value is below a threshold PQ value.

8. The method of claim 1 , wherein the PQ shift is calculated as a linear function above a given PQ.

9. The method of claim 1 , further comprising applying a roll-off curve to the image.

10. The method of claim 1 , further comprising subtracting a reflection compensation value from the PQ data in linear space at the end of a tone curve calculation that provides compensation for expected screen reflections on the display device.

11. The method of claim 10 , wherein the reflectance compensation value is a function of an ambient luminance value of the device.

12. The method of claim 1 , wherein applying the PQ shift is performed in hardware or firmware.

13. The method of claim 1 , wherein applying the PQ shift is performed in software.

14. The method of claim 1 , wherein the ambient light conditions are determined by a sensor in, on, or near the display device.

15. 13. A video decoder comprising hardware and / or software configured to perform the method of claim 1.

16. A non-transitory computer readable medium having stored software instructions that, when executed by a processor, perform the method of claim 1.

17. 13. A system having at least one processor configured to perform the method of claim 1.

Images