System and method for ambient light compensation using PQ offset

Abstract

A novel method and system for compensating for ambient light around a display is disclosed. An offset applied to a PQ curve of an image can compensate for suboptimal ambient light conditions of the display, wherein the PQ offset is either adding a compensation value in PQ space and then subtracting a compensation value in linear space, or adding a compensation value in linear space and then subtracting a compensation value in PQ space. Further adjustments to the PQ curve can also be made to provide improved image quality relative to image brightness.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 046,015 filed on June 30, 2020 and European Patent Application No. 20183195.5 filed on June 30, 2020, both of which are incorporated herein by reference in their entirety. Technical Field

[0003] The present disclosure relates to improvements in the processing of video signals. In particular, this disclosure relates to processing video signals to improve display under different ambient light conditions. Background Art

[0004] The reference electro-optical transfer function (EOTF) of a given display characterizes the relationship between the color values ​​of an input video signal (e.g., brightness) and the output screen color values ​​(e.g., screen brightness) produced by the display. For example, ITU Rec. ITU-R BT.1886, "Reference electro-optical transfer function for flat panel displays used in HDTV studio production" (03 / 2011), which is incorporated herein by reference in its entirety, defines a reference EOTF for a flat panel display based on measured characteristics of a cathode ray tube (CRT). Given a video stream, information about its EOTF is typically embedded in the bitstream as metadata. As used herein, the term "metadata" refers to any auxiliary information that is transmitted as part of an encoded bitstream and that assists a decoder in rendering a decoded image. Such metadata may include, but is not limited to, color space or gamut information, reference display parameters, and auxiliary signal parameters, such as those described herein.

[0005] Most consumer desktop monitors currently support 200 to 300 cd / m 2 Or nits of brightness. Most consumer HDTVs range from 300 to 500 nits, with newer models reaching 1000 nits. Commercial smartphones typically range from 200 to 600 nits. These varying display brightness levels present challenges when trying to display images in different ambient lighting scenarios, such as Figure 1. A viewer 110 is viewing an image (e.g., a video) on a screen 120. Image brightness 130 may be "washed out" by ambient light 140. The ambient light 140 brightness level may be measured by a sensor 150 in, on, or near the display. The brightness of the ambient light may vary, for example, from 5 nits in a dark room to 200 nits in a well-lit room with no daylight, or to 400 nits in a room with indirect sunlight, to 600+ nits outdoors. One solution is to make a linear adjustment to the brightness control of the display, but this may result in an unbalanced brightness of the display. Summary of the invention

[0006] Various video processing systems and methods are disclosed herein. Some such systems and methods may involve compensating an image to maintain its appearance as the ambient brightness level changes. In some embodiments, the method may be computer-implemented. For example, the method may be implemented at least in part via a control system including one or more processors and one or more non-transient storage media.

[0007] In some examples, a system and method for modifying an image to compensate for ambient light conditions around a display device is described, which includes determining a PQ curve for the image; determining a PQ offset of the PQ curve based on a compensation value determined from the ambient light conditions and the image, the PQ offset consisting of: adding a compensation value in a PQ space and then subtracting the compensation value in a linear space, or adding a compensation value in a linear space and then subtracting the compensation value in the PQ space; applying the PQ offset to the PQ curve to produce an offset PQ curve; and modifying the image using the offset PQ curve.

[0008] In some such examples, the method may involve applying tone mapping to the image before modifying the image. In some such examples, the method may be performed by software, firmware, or hardware, and may be part of a video decoder.

[0009] Some or all of the methods described herein can be performed by one or more devices according to instructions (e.g., software) stored on one or more non-transient media. Such non-transient media may include memory devices such as those described herein, including but not limited to random access memory (RAM) devices, read-only memory (ROM) devices, etc. Therefore, the various innovative aspects of the subject matter described in this disclosure can be implemented with non-transient media having software stored thereon. The software, for example, can be executable by one or more components of a control system such as those disclosed herein. The software, for example, may include instructions for executing one or more of the methods disclosed herein.

[0010] At least some aspects of the present disclosure may be implemented via one or more devices. For example, one or more devices may be configured to perform at least partially the method disclosed herein. In some implementations, the device may include an interface system and a control system. The interface system may include one or more network interfaces, one or more interfaces between a control system and a memory system, one or more interfaces between a control system and another device, and / or one or more external device interfaces. The control system may include at least one of a general-purpose single-chip or multi-chip processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic, or a discrete hardware component. Therefore, in some implementations, the control system may include one or more processors and one or more non-transient storage media operably coupled to the one or more processors.

[0011] Details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages will become apparent from the description, drawings, and claims. Note that the relative sizes of the following figures may not be drawn to scale. The same reference numerals and names in the various figures generally indicate the same elements, but different reference numerals do not necessarily specify different elements between different figures. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 An example of ambient light for a display is shown.

[0013] Figure 2 An example flow chart of a method of compensating for ambient light around a display is illustrated.

[0014] Figure 3 An example graph of experimental data illustrating compensation value vs. square root of PQ in the middle of the image under different ambient light conditions.

[0015] Figure 4 An example graph showing the slope of the experimental data vs. the fitted line of the ambient brightness PQ is shown.

[0016] Figure 5 An example graph showing the y-intercept of the experimental data vs. the fitted line of the ambient luminance PQ is shown.

[0017] Figure 6 An example PQ offset compensation curve is illustrated.

[0018] Figure 7 An example PQ offset compensation curve adjusted to reduce brightening is illustrated.

[0019] Figure 8An example PQ offset compensation curve is illustrated with relaxation added to avoid artifacts.

[0020] Fig.9A and 9B An example PQ offset compensation curve with a clamp set below a visual threshold is illustrated.

[0021] Fig.10 An example PQ offset compensation curve with renormalization is illustrated.

[0022] Fig.11 An example PQ offset compensation curve adjusted for reflections is illustrated. DETAILED DESCRIPTION

[0023] The term "PQ" as used herein refers to the quantization of the perceived brightness amplitude. The human visual system responds to gradually increasing light levels in a very nonlinear manner. The term "PQ space" as used herein refers to a nonlinear mapping of linear brightness amplitude to nonlinear PQ brightness amplitude, as described in Rec.BT.2100. The ability of a person to see a stimulus is affected by the brightness of the stimulus, the size of the stimulus, the spatial frequencies that constitute the stimulus, and the brightness level to which the eye has adapted at a particular moment when the stimulus is being viewed. In an example, the perceptual quantizer function maps a linear input grayscale level to an output grayscale level that better matches the contrast sensitivity threshold in the human visual system. An example of a PQ mapping function (or EOTF) is described in SMPTEST 2084:2014 "High Dynamic Range EOTF of Mastering Reference Displays", where a fixed stimulus size is given, for each brightness level (i.e., stimulus level), the minimum visible contrast step at the brightness level is selected according to the most sensitive adaptation level and the most sensitive spatial frequency (according to the HVS model). Compared to the traditional gamma curve, which represents the response curve of a physical cathode ray tube (CRT) device and, coincidentally, may have a very rough resemblance to the way the human visual system responds, the PQ curve simulates the true visual response of the human visual system using a relatively simple functional model.

[0024] This document describes a solution to the problem of adjusting the brightness of a display to ambient lighting conditions by applying compensation to the image as an offset in PQ. Figure 2 An example method for applying compensation to an image on a display is shown.

[0025] Sensor data 210 is obtained for the area surrounding the display to produce data for brightness measurements of the ambient light. The sensor data may be taken from one or more brightness sensors, which include light-sensitive elements such as photoresistors, photodiodes, and phototransistors. This sensor data is then used to calculate the ambient brightness PQ 220, which may be designated S. This calculation, like all calculations described herein, may be performed locally on the display, such as on a processor or computer in or connected to the display, or it may be performed on a remote device or server that delivers the image to the device.

[0026] Given the ambient brightness PQ S, two intermediate values ​​(herein M and B) can be calculated as a function of S. In the example, M and B are calculated from the following equations:

[0027] M = a*S + b Equation 1

[0028] B=c*S 2 +d*S+e Equation 2

[0029] Where a, b, c, d, and e are constants. In this example, M is a linear function of S, and B is a quadratic function of S. As shown in this article, the constants can be determined experimentally.

[0030] The image 240 may be analyzed for a range of brightness (e.g., luma values) that the image 240 contains. The image may be a video frame. The image may be a key frame of a video stream. From these brightness data, an intermediate PQ may be determined 250 from the complete image. The intermediate PQ may represent the average brightness of the image. An example of calculating the intermediate PQ is taking the average of the maximum values ​​of each component (e.g., R, G, and B) of the downsampled image. Another example of calculating the intermediate PQ is to perform a 250 averaging of the maximum values ​​of each component (e.g., R, G, and B) of the downsampled image. B C R The Y values ​​of the image in the color space are averaged. This intermediate PQ value can be designated as X. The intermediate PQ values, minimum and maximum values ​​can be calculated on the encoder side and provided in the metadata, or they can be calculated on the decoder side.

[0031] From the calculated M and B values ​​230 and the calculated X value 250, a compensation value may be calculated 260. This compensation value may be designated as C and is calculated from the following equation:

[0032]

[0033] The square root of X is used in this example because it allows a linear relationship for the experimental data. Calculating C from X could be done, but it would produce a more complicated function. Keeping the function linear allows the calculation to be easier, especially if it is implemented in hardware rather than software.

[0034] The compensation value C can then be used in step 270 to modify the image via the PQ curve of the PQ offset. The PQ offset can be expressed by the following equation:

[0035] PQ out =L2PQ(PQ2L(PQ in +C)-PQ2L(C)) Equation 4

[0036] Where PQ out is the PQ obtained after the offset, PQ in is the original PQ value, L2PQ() is the function converting from linear space to PQ space, PQ2L() is the function converting from PQ space to linear space, and C is the compensation value (for a given value of M and B for the measured ambient light and X for the image in question). Conversions between linear space and PQ space are known in the art, for example, as described in ITU-R BT.2100, "Image parameter values ​​for high dynamic range television for use in production and international programme exchange". Thus, Equation 4 represents addition in PQ space and subtraction in linear space. The compensated (modified) image 280 is then presented on the display. Compensation can occur in a display such as an IC T C P , YC B C R Processing can be done on the luma (e.g., I) component, but chroma adjustments may also be useful to maintain the intent of the image. Compensation can also occur after tone mapping in other color spaces such as RGB, where compensation is applied to each channel individually.

[0037] This method provides compensation for the image so that in a high ambient brightness environment (e.g., outside in sunlight) it matches what it would look like in an ideal ambient environment (e.g., a very dark room). An example of an ideal ambient target is 5 nits (cd / m 2 ). Increases the dark detail contrast to ensure that detail is still visible. For example, this method compensates for an image that is brighter than the reference value. The reference value can be a specific value or a range of values.

[0038] In another embodiment, the compensation is reversed to allow compensation for ambient lighting conditions that are darker than ideal. Such compensation is used for ambient lighting conditions that are darker than the reference value. For example, if the image was originally intended to be viewed in a brightly lit room, the compensation can be set so that it has the correct appearance in a dark room. For this embodiment, the operation is reversed, adding in linear space and subtracting in PQ space, as shown in the following equation.

[0039] PQ out =L2PQ(PQ2L(PQ in )+PQ2L(C))-C Equation 5

[0040] In an embodiment, the compensation value C is experimentally determined by subjectively determining compensation values ​​for various image illumination values ​​under different ambient light conditions. An example would be to obtain data through a psychovisual experiment in which an observer subjectively selects appropriate compensation amounts for various images at different ambient brightness levels. Figure 3 An example of this type of data is shown in . The figure shows data points 310 of the square root of the image mid-PQ value plotted against subjectively selected compensation values ​​for five different ambient light conditions (in this case 22, 42, 77, 139, and 245 nits; ranging from a dark room to a brightly lit condition). From these points 310, a trend line 320 can be fit to the data points for each ambient light condition. Since the square root of the image mid-value is used, it is easier to fit these points using linear regression. An image with a bright PQ mid-point in a dark ambient condition will have data points 330 that bottom out at zero compensation. Those points will incorrectly distort the trend line, so they are not considered for fitting.

[0041] From these lines 320, two useful values ​​can be determined: the slope of the line, ΔCompensation / Δsqrt(ImageMid), and the y-intercept, the compensation value at sqrt(ImageMid) = 0, where sqrt(x) represents the square root of x, e.g., ). These slopes and y-intercepts can then also be fitted to other functions, such as Figure 4 and Figure 5 as shown in .

[0042] Figure 4 The linear regression of line 410 to the compensation vs. sqrt(ImageMid) line (e.g., Figure 3 ) vs. ambient (environmental) brightness PQ. In some embodiments, additional data points 420 are added to the fit so that the slope and ambient brightness PQ result in a zero compensation for the reference (ideal) ambient brightness. From this fit, a function of M expressed in terms of ambient brightness S can be found for use in Equation 1 (see Figure 2 ). This allows the calculation of compensation values ​​a and b for Equation 1 (a is the slope of the fitted line and b is the y-intercept of the fitted line). These values ​​can then be put into Equation 1 along with the measured S ambient luminance to determine the M value for that ambient luminance (e.g., 5 nits).

[0043] Figure 5 The curve 510 is shown fitted (second order polynomial) to the compensation vs. sqrt(ImageMid) line (e.g., as Figure 3 5. In some embodiments, additional data points 520 are added to the fit so that the y-intercept and ambient luminance PQ result in zero compensation for the reference (ideal) ambient luminance.

[0044] Figure 6 An example PQ offset (PQ surrounding adjustment) as generated by Equation 4 is shown. The three black circles represent the minimum 610, midpoint 620, and maximum 630 of the image after tone mapping has occurred. The solid line 640 is the adjustment of the PQ offset method using a compensation value of 0.3 (calculated from Equation 4). The dashed line 650 represents the value without compensation. The minimum 610 of the image is located at approximately [0.01, 0, 21]. The image does not contain content below this level, so the image may be too bright in this example.

[0045] In some embodiments, this overbrightening problem can be overcome by performing an additional shift in the PQ curve. This compensation can be achieved by shifting the PQ value based on the minimum pixel value of the image after tone mapping, so that the contrast enhancement is maintained only where the pixel is located and the overbrightening artifacts are minimized. Figure 7 An example of this is shown in , where the Figure 6 The curve 640 has been shifted to produce a new curve 740, in which the minimum point 710 is adjusted to zero compensation 650 (PQ in =PQ out ), and the other values, including the midpoint 720 and the maximum value 730, are adjusted accordingly from this offset.

[0046] In some embodiments, additional adjustments to the PQ compensation curve may be made to prevent striping artifacts caused by a sharp cutoff at the minimum value. The easing may be achieved by three rolls of the input points within some small value (e.g., 36 / 4,096) of the minimum PQ (TminPQ) of the image. This value may be found by experimentally determining the minimum value that reduces striping artifacts. This value may also be chosen arbitrarily, for example by visualizing the easing and determining what value provides a smooth transition to the zero compensation point.

[0047] Figure 8An example of using easing to prevent banding is shown. The original compensation curve 840 has a sharp transition 845 at the intersection with the zero compensation line 650. From the minimum PQ of the image (which for this example is at the intersection 845, as shown in FIG. Figure 7 ) to a point where some small value (e.g., TminPQ+36 / 4096) is increased above the minimum PQ.

[0048] The easing may be a cubic roll-off function that returns values ​​between 0 and 1, where 0 is returned near the minimum PQ and 1 is returned at increasing values. An example algorithm in (MATLAB) is as follows, where, in an embodiment and without limitation, cubicEase() is a monotonically increasing sigmoid-type function for PQ values ​​of inputs between TminPQ and TminPQ+36 / 4096, and outputs alpha in [0,1]:

[0049]

[0050] As used herein, the term "easing" refers to a function that applies a nonlinear function to data so that a Bezier curve or spline transformation / interpolation (curvature change of the graphic data) is applied. "Easing in" refers to a transformation near the beginning of the data (near zero), and "easing out" refers to a transformation near the end of the data (near the maximum value). "In and out" refers to a transformation near both the beginning and the end of the data. The specific algorithm used for the transformation depends on the type of easing. There are many easing functions known in the art. For example, cubic in and out, sine in and out, quadratic in and out, and others. Easing is applied in both the in and out of the curve to prevent sharp transitions.

[0051] In some embodiments, compensation may be clamped to not be applied below a threshold PQ value in order to prevent unnecessary extension of dark details that would not otherwise be visible under ideal ambient lighting conditions (e.g., 5 nits ambient light). The threshold PQ value may be determined experimentally by determining the point at which a human observer cannot discern detail under ideal conditions (e.g., 5 nits ambient light, viewing from a distance of three picture heights). For these embodiments, below this threshold PQ (for PQ in ) does not apply PQ offset (Equation 4). Fig.9A and 9B An example of this is shown in . Fig.9A PQ compensation 910 is shown (eg Figure 6 ) and PQ compensation with over brightness adjustment 920 (as shown in Figure 7 ), where the line shows the PQ threshold 930 below which details would be indiscernible under ideal conditions. Fig. 9B shows a magnified image near the origin Fig.9A This process occurs after tone mapping and can be important for displays with low black levels, such as OLED displays.

[0052] In some embodiments, the compensation can be clamped to have a maximum value, such as 0.55. This can be done with or without the threshold PQ clamp described above. Maximum clamping can be useful for hardware implementation. The following is an example MATLAB code for an example algorithm showing maximum clamping at 0.55, where environmental compensation is to be applied based on the target ambient ambient brightness (Surr) in PQ and the source mid-value (L1Mid) of the image. A, B, C, D, and E are experimentally derived values ​​for a, b, c, d, e as shown in equations 1 and 2 above:

[0053]

[0054] In some embodiments, the PQ compensation curve is at a certain PQ in Points above can be simplified to be linear. For example, compensation above a PQ of 0.5 (in the total range of [0 1]) can be calculated to be linear, providing the following example algorithm:

[0055] For PQ in <0.5, PQ out =L2PQ(PQ2L(PQ in +C)-PQ2L(C)); and Equation 6

[0056] For PQ in ≥0.5, PQ out =PQ in +C Equation 7

[0057] This simplification above a certain PQ point is useful for hardware implementation of the method.

[0058] In some cases, ambient light compensation may push some pixels out of the range of the target display. In some embodiments, a roll-off curve may additionally be applied to compensate for this and renormalize the image to the correct range. This can be done by using a tone mapping curve along with source metadata (e.g., metadata describing minimum, average (or midpoint), and maximum brightness). Without limitation, example tone mapping curves are described in U.S. Patents 10,600,166 and 8,593,480, each of which is incorporated herein by reference in its entirety. Take the resulting minimum, midpoint, and maximum values ​​of the tone mapped image (before applying ambient light compensation (e.g., Equation 4)), apply ambient light compensation to these values, and then map the resulting image to the target display using tone mapping techniques. See, for example, U.S. Patent Application Publication No. 2019 / 0304379, which is incorporated herein by reference in its entirety. In Fig.10 An example of a roll-off curve is shown in . The main feature of this roll-off is that the minimum of 1010 points and the maximum of 1020 points are still within the range of the target display. The result is that the brighter image 1030 will have less highlight roll-off (at the expense of dark / mid contrast enhancement), and the darker image 1040 will have more dark detail enhancement (at the expense of highlight detail) due to the dynamic tone mapping nature of our tone curve.

[0059] In some embodiments, further compensation may be performed to compensate for reflections of the display screen. In some embodiments, the amount of light reflected from the screen may be estimated from the sensor values ​​using the reflective properties of the screen as in Equation 8 below.

[0060] ReflectedLight=SensorLuminance*ScreenReflection Equation 8

[0061] The light reflected from the screen can be considered as a linear addition to the light of the image, essentially raising the black level of the display. In these embodiments, a tone mapping is performed to a higher black level (e.g., to the level of reflected light), where, at the end of the tone curve calculation, a subtraction is performed in linear space to compensate for the added luminosity due to reflection. See, for example, Equation 9.

[0062] PQ out =L2PQ(PQ2L(PQ in )-ReflectedLight) Equation 9

[0063] exist Fig.11An example of a tone curve with reflection compensation is shown in . The minimum 1110 and maximum 1120 levels remain as they were before reflection compensation was applied, but the contrast at the bottom end 1130 is greatly increased on the curve 1140 to be applied to the pixel. Adding the expected reflected light produces a tone curve 1150 that is closer to the perception of the desired image quality.

[0064] Many embodiments of the present disclosure have been described. However, it will be appreciated that various modifications may be made without departing from the spirit and scope of the present disclosure. Therefore, other embodiments are within the scope of the following claims.

[0065] As described herein, embodiments of the present invention may therefore be directed to one or more of the following enumerated example embodiments. Therefore, the present invention may be implemented in any of the forms described herein, including but not limited to the following enumerated example embodiments (EEE) that describe the structure, features, and functions of some parts of the present invention:

[0066] EEE1. A method for modifying an image to compensate for ambient light conditions around a display device, the method comprising: determining perceived brightness amplitude quantization (PQ) data of the image; determining a PQ offset of the PQ data based on a compensation value determined from the ambient light conditions and the image, the PQ offset consisting of: adding the compensation value in the PQ space and then subtracting the compensation value in the linear space, or adding the compensation value in the linear space and then subtracting the compensation value in the PQ space; applying the PQ offset to the image to modify the PQ data of the image.

[0067] EEE2. The method as described in enumerated example embodiment 1 further includes: applying tone mapping to the image before applying the PQ offset.

[0068] EEE3. A method as described in enumerated example embodiments 1 or 2, wherein: the compensation value is calculated from C=M√X+B, where C is the compensation value, M is a function of the surrounding brightness value, X is the middle PQ value of the image, and B is a function of the surrounding brightness value.

[0069] EEE4. A method as described in enumerated example embodiment 3, wherein functions M and B are derived from experimental data derived from subjective perceptual evaluations of image PQ compensation values ​​under different ambient light conditions.

[0070] EEE5. A method as described in enumerated example embodiments 3 or 4, wherein M is a linear function of the ambient brightness values ​​and B is a quadratic function of the ambient brightness values.

[0071] EEE6. The method as described in any of the enumerated example embodiments 1-5 further includes applying an additional PQ offset to the image, wherein the additional PQ offset adjusts the image so that the minimum pixel value has a zero compensation value.

[0072] EEE7. The method as described in any of the enumerated example embodiments 1-6 further includes applying mitigation to the PQ offset.

[0073] EEE8. The method as described in any of the enumerated example embodiments 1-7 further includes clamping the PQ offset so that it is not applied below a threshold.

[0074] EEE9. A method as described in any of the enumerated example embodiments 1-8, wherein the PQ offset is calculated as a linear function above a predetermined PQ.

[0075] EEE10. The method of any one of Enumerated Example Embodiments 1-9, further comprising applying a roll-off curve to the image.

[0076] EEE11. A method as described in any of the enumerated example embodiments 1-10, further comprising subtracting a reflection compensation value from the PQ data in a linear space at the end of a tone curve calculation that provides compensation for expected screen reflections on the display device.

[0077] EEE12. The method of enumerated example embodiment 11, wherein the reflection compensation value is a function of an ambient brightness value of the device.

[0078] EEE13. A method as described in any of the enumerated example embodiments 1-12, wherein applying the PQ offset is performed in hardware or firmware.

[0079] EEE14. A method as described in any of the enumerated example embodiments 1-12, wherein applying the PQ offset is performed in software.

[0080] EEE15. A method as described in any of Enumerated Example Embodiments 1-14, wherein the ambient light condition is determined by a sensor in, on, or near the display device.

[0081] EEE16. A video decoder comprising hardware or software or both configured to perform a method as described in any one of the enumerated example embodiments 1-12.

[0082] EEE17. A non-transitory computer-readable medium comprising stored software instructions, which when executed by a processor cause a method as described in any of the enumerated example embodiments 1-12 to be performed.

[0083] EEE18. A system comprising at least one processor, wherein the at least one processor is configured to perform a method as described in any one of Enumerated Example Embodiments 1-12.

[0084] For the purpose of describing some innovative aspects described herein and examples of backgrounds in which these innovative aspects can be implemented, the present disclosure is directed to certain implementations. However, the teachings of this article can be applied in various different ways. In addition, the described embodiments can be implemented in various hardware, software, firmware, etc. For example, aspects of the present application can be implemented at least in part in a device, a system including more than one device, a method, a computer program product, etc. Therefore, aspects of the present application can take the form of hardware embodiments, software embodiments (including firmware, resident software, microcode, etc.) and / or embodiments of both combined software aspects and hardware aspects. Such embodiments may be referred to as "circuit", "module", "device", "device" or "engine" in this article. Some aspects of the present application may take the form of a computer program product, which is implemented in one or more non-transient media, and the one or more non-transient media have a computer-readable program code implemented thereon. Such non-transient media may, for example, include a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. Thus, the teachings of this disclosure are not intended to be limited to the implementations shown in the figures and / or described herein, but rather have broad applicability.

Claims

1. A method for modifying an image to compensate for ambient light conditions around a display device, the method comprising: Determining perceived brightness magnitude quantization (PQ) data of the image; determining a PQ offset for the PQ data based on a compensation value determined from the ambient light conditions and the image, The compensation value is Calculate, where C is the compensation value, M is a function of the surrounding brightness value S, X is the middle PQ value of the image representing the average brightness of the image, and B is a function of the surrounding brightness value, where M = a*S+b and B = c*S 2 +d*S+e, where a, b, c, d and e are constants; The PQ offset is composed of: for an ambient brightness environment that is brighter than a reference value, the compensation value is added in the PQ space and then subtracted in the linear space, which is calculated by PQout=L2PQ(PQ2L(PQin+C))-PQ2L(C)), or for an ambient brightness environment that is darker than the reference value, the compensation value is added in the linear space and then subtracted in the PQ space, which is calculated by PQout=L2PQ(PQ2L(PQin)+PQ2L(C))-C, where PQout is the PQ obtained after the offset, PQin is the original PQ value, L2PQ() is a function of converting from a linear space to a PQ space, and PQ2L() is a function of converting from a PQ space to a linear space; The PQ offset is applied to the image to modify the PQ data of the image.

2. The method according to claim 1, further comprising: Tone mapping is applied to the image before applying the PQ offset.

3. The method of claim 1, wherein functions M and B are derived from experimental data derived from subjective perceptual evaluations of image PQ compensation values ​​under different ambient light conditions.

4. The method of any one of claims 1-3, further comprising applying an additional PQ offset to the image, the additional PQ offset adjusting the image so that minimum pixel values ​​have a zero compensation value.

5. The method according to any one of claims 1-3, further comprising applying a relaxation to the PQ offset, wherein the relaxation refers to a function that applies a non-linear function to data.

6. The method according to any one of claims 1-3, further comprising clamping the PQ offset so that it is not applied below a threshold.

7. The method according to any one of claims 1-3, wherein the PQ offset is calculated as a linear function above a predetermined PQ.

8. The method of any one of claims 1-3, further comprising applying a roll-off curve to the image.

9. The method of any one of claims 1-3, further comprising subtracting a reflection compensation value from the PQ data in linear space at the end of a tone curve calculation that provides compensation for expected screen reflections on the display device.

10. The method of claim 9, wherein the reflection compensation value is a function of an ambient brightness value of the device.

11. The method of any of claims 1-3, wherein applying the PQ offset is performed in hardware or firmware.

12. The method of any of claims 1-3, wherein applying the PQ offset is performed in software.

13. The method of any one of claims 1-3, wherein the ambient light condition is determined by a sensor in, on, or near the display device.

14. A video decoder comprising hardware or software or both configured to perform the method according to any one of claims 1-13.

15. A non-transitory computer readable medium comprising stored software instructions which, when executed by a processor, cause the method of any one of claims 1-13 to be performed.

16. A system comprising at least one processor, the at least one processor being configured to perform the method according to any one of claims 1-13.

Images