Display management with ambient light compensation

Abstract

The invention relates to display management with ambient light compensation. A display device, a display management module, and a method for ambient light compensation are described. The display management module is configured to receive an input video signal including a sequence of video frames and determine whether a current video frame of the sequence of video frames follows a scene change. The display management module is further configured to adjust ambient light compensation applied to the input signal in dependence on a signal indicative of an intensity of ambient light only in response to determining that the current video frame of the sequence of video frames follows a scene change.

Description

[0001] Information about divisional applications

[0002] This application is a divisional application of the Chinese invention patent application with application number 202080013663.0, application date January 9, 2020, and invention name “Display Management with Ambient Light Compensation”.

[0003] CROSS-REFERENCE TO RELATED APPLICATIONS

[0004] This application claims the benefit of priority to U.S. Provisional Patent Application No. 62 / 790,058, filed on January 9, 2019, and European Patent Application No. 19150873.8, filed on January 9, 2019, the entire contents of each of which are incorporated herein by reference. Technical Field

[0005] The present disclosure relates generally to video technology. More particularly, embodiments of the present disclosure relate to video signal display in a viewing environment with variable ambient light. Background Art

[0006] In a typical content creation pipeline, video is color graded in a low-light environment, typically 5 nits for color-grading high dynamic range (HDR) video and 10 nits for color-grading standard dynamic range (SDR) video. In practice, viewers may display content in a variety of environments, for example, at 0 to 5 nits (e.g., watching a movie in a relatively dark home theater), at 100 to 150 nits (e.g., watching a movie in a relatively bright living room), or higher (e.g., watching a movie on a tablet in a very bright room or outdoors in daylight).

[0007] As appreciated by the inventors herein, improved techniques are desired for displaying video to compensate for ambient light conditions in the viewing environment. Summary of the invention

[0008] Embodiments of the present disclosure relate to a display management (DM) module and a corresponding method.

[0009] In an embodiment, a DM module is configured to receive an input video signal comprising a sequence of video frames and a signal indicating the intensity of ambient light. The DM module is further configured to determine whether a current video frame of the sequence of video frames follows a scene change. The DM module is further configured to transform the input video signal into an output video signal suitable for a target display device. Transforming the input video signal into the output signal comprises applying ambient light compensation to the input video signal in dependence on the signal indicating the intensity of the ambient light. The output video signal is output by the DM module for use by the target display device. The DM module is configured to adjust the ambient light compensation applied to the input signal in dependence on the signal indicating the intensity of the ambient light only in response to determining that the current video frame of the sequence of video frames follows a scene change.

[0010] Other embodiments of the present disclosure relate to a display device including the DM module, an ambient light sensor and a display, and a computer program for executing the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Embodiments of the invention are illustrated by way of example and not by way of limitation in the accompanying drawings in which like reference numerals refer to similar elements and in which:

[0012] Figure 1A to C showing schematic diagrams of exemplary display devices;

[0013] Figure 2 A flow chart showing an example of a method for applying ambient light compensation;

[0014] Figure 3A B shows a flowchart of an example of determining whether a video frame follows a scene change;

[0015] Figure 4 Demonstrates example functions for ambient light compensation;

[0016] Figure 5 A schematic diagram showing an exemplary ambient light sensor;

[0017] Fig. 6A a schematic diagram showing sensitivity as a function of wavelength for a first exemplary ambient light sensor;

[0018] Figure 6B showing a schematic diagram showing sensitivity as a function of wavelength for a second exemplary ambient light sensor; and

[0019] Figure 7 Adjustment of the ambient light signal for a measurement bias caused by detection of light emitted by the display device itself is schematically illustrated. DETAILED DESCRIPTION

[0020] Many specific details are described below to provide a thorough understanding of the present invention. However, the present invention can be practiced without these specific details. In addition, well-known parts may be described in less detail. The figures are schematic and include parts relevant to understanding the present disclosure, while other parts may be omitted or are merely suggested.

[0021] Figure 1A 4 to 5 C show examples of display devices 100, 200, 300 to which ambient light compensation is applied. In the context of the present disclosure, a display device may include, for example, a television, a laptop computer, a tablet computer, a smartphone, a projector, or any other electronic device for displaying video.

[0022] As used herein, the term "dynamic range" (DR) may relate to the ability of the human visual system (HVS) to perceive a range of intensities (e.g., illumination, brightness) in an image, e.g., from the darkest gray (dark colors or blacks) to the brightest white (highlights). In this sense, DR relates to "scene-referenced" intensities. DR may also relate to the ability of a display device to fully or approximately render a range of intensities of a particular width. In this sense, DR relates to "display-referenced" intensities. Unless a specific meaning is expressly stated to have a specific meaning at any point in the description herein, it should be inferred that this term can be used in either sense, e.g., interchangeably.

[0023] As used herein, the term "display management" refers to the process of adapting an image or picture (e.g., frame) of an input video signal to a target display device. Display management may include mapping an image or picture of an input video signal of a first dynamic range (e.g., 1000 nits) to a display device having a second dynamic range (e.g., 500 nits). Display management may include tone mapping and / or gamut mapping.

[0024] Display device 100( Figure 1A ) includes a display 102, an ambient light sensor 104, and a display management (DM) module 106. In this example, the ambient light sensor 104 communicates with the DM module 106, and the DM module 106 communicates with the display 102. The DM module 106 receives an input video signal 108 including a sequence of video frames. Optionally, the DM module 106 further receives metadata 110 related to the input video signal 108. For example, the DM module 106 can receive the input video signal 108 and the metadata 110 from a decoder internal or external to the device 100.

[0025] The DM module 106 receives a signal 112 indicative of ambient light from the sensor 104. The signal is indicative of at least the intensity of the ambient light recorded by the sensor 104. Optionally, the signal may further be indicative of the color of the ambient light recorded by the sensor.

[0026] Typically, sensor 104 is positioned on the front side of device 100 corresponding to the front side of display 102 to measure the illumination on the display screen, which is the ambient component that raises the black level of the display in terms of reflectivity. In such scenarios, sensor 104 measures the intensity and optionally the color of the light present at the front side of display 102. However, the present disclosure is not limited to a particular location of sensor 104, and instead, sensor 104 may be positioned at a different location, such as along an edge of the display device or on the back of the device.

[0027] In the example of the figures, a single sensor 104 is shown. However, the present disclosure is not limited to the use of a single sensor, and multiple sensors 104 may be used to generate the signal 112 indicative of ambient light for use by the DM module 106.

[0028] The DM module 106 is configured to transform the input video signal 108 into an output video signal 114 suitable for the display 102. The transformation may include tone mapping, e.g., for adjusting the dynamic range of the input signal 108 to a dynamic range supported by the display 102, and / or color gamut mapping, e.g., for adjusting the color of the input signal 108 to a color gamut supported by the display 102.

[0029] The optional metadata 110 may include parameters for tone mapping and / or gamut mapping operations. The DM module 106 may therefore adjust the tone mapping and / or gamut mapping operations to be applied to the input video signal 108 based on the metadata 108. The tone mapping and / or gamut mapping may further use parameters related to the characteristics of the display 102, which may be stored internally in the device 100 and thus accessible by the DM module 106.

[0030] An example of display management including tone mapping is described in PCT / US2014 / 016304, entitled "Display management for high dynamic range video," which is incorporated herein by reference in its entirety. PCT / US2014 / 016304 describes a tone mapping curve that maps input luminance values ​​to output luminance values ​​according to

[0031]

[0032] Where C1, C2 and C3 are constants, Y in is the input brightness value, Y outis the output brightness value, and n and m are parameters. C1, C2, and C3 are based on the brightness characteristics of the reference (or source) display and the brightness characteristics of the target display. The brightness characteristics of the reference (or source) display are usually extracted from the input metadata, such as the minimum brightness (S min ), average or median (S mid ) and maximum brightness (S max ). The brightness characteristics of the target display, such as minimum brightness (T min ), average or median value (T mid ) and maximum brightness (T max ), which is usually stored internally in the display device and can therefore be accessed by the DM module. Parameter S min , S mid , S max , T min , T mid , T max Define the three anchor points of the tone mapping curve, namely (S min , T min )、(S mid , T mid )、(S max , T max ), the three anchor points determine the values ​​of constants C1, C2 and C3 in turn.

[0033] Another example of display management including tone mapping is described in PCT / US2018 / 017830, entitled “Tone mapping for high dynamic range images,” which is incorporated herein by reference in its entirety. PCT / US2018 / 017830 describes a tone mapping curve that uses the same three anchor points but has a total of four segments: values ​​less than (S min , T min ) of the first linear section, from (S min , T min ) to (S mid , T mid ) of the first curve, from (S mid , T mid ) to (S max , T max ) of the second spline curve, and the value is greater than (S max , T max )'s second linear segment.

[0034] Other examples of display management including tone mapping are described in PCT / US2011 / 025366, entitled "Display Management Method and Apparatus," which is incorporated herein by reference in its entirety.

[0035] The transformation performed by the DM module 106 may further include applying ambient light compensation to the input video signal 108. The ambient light compensation is applied in response to a signal 112 received from the sensor 104 indicating at least the intensity of the ambient light. The DM module 106 applies ambient light compensation to the input signal 108, for example, in addition to applying a tone mapping operation, to thereby generate an output video signal 114 for presentation by the display 102. The ambient light compensation may be applied before or after the tone mapping and / or gamut mapping are applied. In another example, the ambient light compensation is combined with the tone mapping and / or gamut mapping into a single mapping operation. In such an example, the mapping applied by the DM module 106 to the input video signal 108 has parameters that depend on the signal 112, as well as parameters related to the tone mapping and / or gamut mapping, which parameters may be, for example, contained in or derived from the metadata 110, or derived from the input video signal 108 itself.

[0036] The DM module 106 is further configured to determine whether a current video frame of the input video signal 108 immediately follows a scene change (also referred to as a “scene cut”).

[0037] In other words, DM module 106 may be configured to determine whether the current video frame starts a new scene that follows a previous scene that includes the video frame immediately preceding the current video frame.

[0038] A scene includes a group of consecutive video frames having similar brightness characteristics and / or similar color characteristics. Scenes may be defined during the authoring of a video. For example, scene changes may be included in an edit decision list (EDL), which is used, for example, by a director and / or editor to create a movie from several different shots. Indications of these scene changes may be included in the metadata of the video signal.

[0039] Therefore, in an embodiment, metadata 110 includes metadata indicating a scene change. DM module 106 is configured to determine whether a current video frame of a sequence of video frames included in video signal 108 follows a scene change using metadata indicating a scene change. For example, metadata may include frame-by-frame metadata including a mark indicating whether the current frame follows a scene change, i.e., whether the current frame is the first frame of a new scene. In another example, metadata includes frame-by-frame metadata including an identifier identifying a scene associated with the corresponding frame, such as a scene number. In such scenarios, DM module 106 may determine whether the current video frame is in a different scene from the video frame immediately preceding the current video frame by comparing the scene identifier of the current frame with the scene identifier of the previous frame. If the scene identifiers of two consecutive video frames are different, it is determined that the current video frame follows a scene change.

[0040] In another embodiment, the scene change is determined by comparing characteristics of the current video frame with characteristics of one or more previous video frames by the DM module 106. For example, the characteristics may include brightness characteristics and / or color characteristics.

[0041] In an example, the DM module 106 determines whether the change in the characteristic between the current frame and the previous frame satisfies a predetermined threshold condition, for example, exceeds a predetermined threshold. If the change satisfies the threshold condition, the DM module 106 determines that a scene change occurs between the current video frame and the previous video frame, i.e., the current scene immediately follows a scene change.

[0042] In another example, the DM module 106 determines the average illumination value for each frame j j > and compare the average illumination value of the current frame j > Average illumination value from previous frame j-1 > the difference between them and the threshold T. If the difference | j >- j-1 >| exceeds the threshold T, that is, | j >- j-1 >|>T, then the DM module 106 determines that the current frame follows a scene change. Instead of an average illumination value, a different function of the brightness value of the frame can be used. For example, instead of an average illumination value, a median illumination value can be used, which can be calculated, for example, as the maximum illumination value (I max ) and minimum value (I min ), that is, the average value of (I max +I min ) / 2.

[0043] In another example, the characteristics of the video frame are derived from metadata indicating the characteristics. For example, referring to the example of the previous paragraph, the median illumination value or the average illumination value may be, for example, the metadata value S described above. mid The form of is available as frame-by-frame metadata contained in metadata 110 received by DM module 106 .

[0044] In another embodiment, the DM module 106 is capable of determining a scene change based on the metadata indicating a scene change and determining the scene change by the comparison of characteristics of the current video frame with characteristics of one or more previous video frames. In another example of this embodiment, the DM module 106 is configured to - in the presence of metadata indicating a scene change - determine the scene change based on the metadata rather than by the comparison. The advantage of metadata indicating a scene change overriding the automatic detection of the DM module 106 is that the display management process more closely follows the creator's intent.

[0045] exist Figure 1B and 1C In, with​​​​​​​ Figure 1A Elements similar to those of FIG. 1 are given the same reference numerals, increased by 100 or 200, respectively. These elements have the same Figure 1A The same features, configurations, and functionality described herein are the same unless otherwise described in the following description.

[0046] Display device 200( Figure 1B ) differs from the display device 100 in that it includes a display controller 216 that controls the operation of the display 202. The DM module 206 communicates with the display controller 216, and the display controller communicates with the display 202. The DM module 206 receives an input video signal 208 including a sequence of video frames. In addition, the DM module 206 optionally receives metadata 210, such as metadata including metadata for dynamic range mapping and / or metadata indicating scene changes. The DM module 206 receives information 218 about the characteristics of the display 202 from the display controller 216.

[0047] In an example, information 218 includes luminance characteristics of the target display for use by DM module 206 in tone mapping, such as minimum luminance (min), average or median value (mid), and maximum luminance (max). In this example, DM module 206 generates a tone mapping image based on source luminance characteristics (e.g., S min , S mid , S max ) and a target brightness characteristic (e.g., T min , T mid , T max ), applies tone mapping to the input video signal 208. In addition, the DM module 206 receives a signal 212 indicating the intensity (and optionally the color) of the ambient light from the ambient light sensor 204, and applies ambient light compensation in dependence on the signal 212. The DM module 206 transmits the tone mapped (and optionally gamut mapped) and ambient light compensated output video signal 214 to the display controller 216. The display controller 216 controls the display 202 to present the output signal 214.

[0048] Instead of using metadata 210 to determine source brightness characteristics, DM module 206 may determine these characteristics by analyzing video signal 208 (e.g., by determining minimum, median, and maximum values ​​for one or more images of video signal 208). Additionally, as described above with respect to display device 100, instead of using optional metadata 210, scene changes may be determined by DM module 206 of display device 200 by comparing characteristics of a current video frame to characteristics of one or more previous video frames.

[0049] In an example, the display 202 includes a backlight. For example, the display 202 may be an LCD type display. In this scenario, the display controller 216 is configured to control the backlight of the LCD display based on the video signal 214 received from the DM module 206. In other examples, the display 202 may not include a backlight. For example, the display 202 may be an OLED type display.

[0050] Display device 300( Figure 1C ) differs from the display device 200 in that the ambient light sensor 304 communicates with the display controller 316 instead of the DM module 306. The display controller 316 receives the signal 312 indicating the intensity of the ambient light from the sensor 304. The display controller 316 transmits information 320 about the intensity of the ambient light (and optionally about the color of the ambient light) to the DM module 306. In the example shown, the signal 312 of the sensor 304 is passed to the DM module 306. Alternatively, the display controller 316 may process the signal 312 of the sensor 304 to generate a parameter indicating the intensity of the ambient light and communicate the parameter to the DM module 306.

[0051] with Figure 1B Similar to Figure 1C 3, the display controller 316 communicates information about the characteristics of the display 302 to the DM module 306, for example, communicating brightness characteristics for tone mapping operations of the DM module 306. The DM module 306 applies a tone mapping operation and ambient light compensation to the input video signal 308, wherein the tone mapping operation is controlled via parameters of source characteristics included in metadata 310 (or determined directly from the video signal 308 by the DM module 306) and parameters of characteristics of the display 302 included in signal 318, and wherein the ambient light compensation is applied in dependence on a signal 320 indicating the intensity of ambient light received from the display controller 316.

[0052] The display devices 100, 200, and 300 include displays 102, 202, 302. In some embodiments, the display devices 100, 200, 300 may include a projector instead of a display. For example, the projector includes a laser projector.

[0053] In the above embodiments, the ambient light sensor 104, 204, 304 is depicted as part of the display device 100, 200, 300. Alternatively, the ambient light sensor may be located external to the display device 100, 200, 300, in which case the DM module 106, 206, 306 is configured to receive a signal indicating the intensity of the ambient light from the external ambient light sensor, e.g., via a wired or wireless connection 112, 212, 312.

[0054] Since ambient light is typically not constant over time, the signal 112, 212, 320 indicative of the intensity of ambient light will also typically vary over time. However, compensating for transient ambient light may result in "flickering": apparent rapid fluctuations in the brightness of the displayed image. Measurement noise inherent to an ambient light sensor may, for example, result in the brightness being adjusted in response to the measured ambient light intensity, even if the intensity of the ambient light has not actually changed. In another example, if the ambient light sensor is temporarily blocked, such as by a person standing in front of the ambient light sensor, the brightness may drop significantly, only to rise again immediately after the person leaves and no longer blocks the sensor.

[0055] exist Figure 1A In the embodiments of FIG. 4 , the DM modules 106, 206, 306 are configured to adjust the ambient light compensation only in response to determining that the current video frame of the sequence of video frames follows a scene change. In other words, the adjustment of the ambient light compensation is synchronized with the scene change of the input video signal. The inventors have found that when the adjustment is limited to scene changes, the adjustment of the ambient light compensation is not noticeable to the viewer. In a typical video, scene changes occur every 1 to 5 seconds. The inventors have found that even if the ambient light compensation is adjusted every 1 to 5 seconds, the change in the ambient light compensation is not noticeable if the adjustment is limited to scene changes. Therefore, by synchronizing the ambient light adjustment with the scene change, flicker is avoided or at least reduced. In addition, a quick response to changes in the ambient light is ensured because the next scene will have an ambient light adjustment suitable for the actual ambient light conditions.

[0056] The above description relates to sensor adaptive ambient light compensation, i.e., ambient light compensation that is automatically adjusted in response to a signal indicating the intensity of ambient light. Other types of adjusting the brightness and / or color of a displayed image may be available. For example, the DM module 106, 206, 306 and / or the display controller 216, 316 may allow a user to manually set the brightness level and / or color setting. In an embodiment, the DM module 106, 206, 306 is not limited to the other types of adjusting the brightness and / or color for scene changes, for example, the brightness setting may be adjusted by the user mid-scene.

[0057] Optionally, a form of temporal smoothing may be applied to the signals 112, 212, 312 of the sensors 104, 204, 304, while limiting the adjustment of ambient light compensation to scene changes, as described above. In an embodiment, a temporal smoothing filter is applied to the signal indicative of the intensity of ambient light, wherein the filter is reset in response to determining that the current video frame follows a scene change. For example, smoothing filters include moving average filters, weighted moving average filters, exponential smoothing filters. Resetting the filter may include clearing previously stored values ​​of the filter, such as setting the values ​​to zero or currently measured values.

[0058] For example, the cumulative average filter is implemented as:

[0059] y(n)=y(n-1)+I(n),

[0060] Where I(n) represents the ambient intensity measured by the sensor at discrete time instance n, and y(n) represents the cumulative average, where y(0) can be initialized to I(0). In this example, in response to determining that the current video frame follows a scene change, y(n) is reset to I(n).

[0061] Figure 1A 4 to 5 show exemplary display devices for ambient light compensation. However, the present disclosure is not limited to these examples. In particular, additional components of the display device that are not related to ambient light compensation have not been described.

[0062] Figure 2 A method for applying ambient light compensation is described. This method may be implemented by a display device. In an embodiment, the method is performed by a display management module of the display device. For example, the method may be performed by any of the DM modules 106, 206, 306 described above.

[0063] The method begins at step 402. A current video frame of a sequence of video frames is received in step 404. A determination is made as to whether a scene change has occurred 406. Specifically, in step 406, a determination is made as to whether the current frame immediately follows a scene change, which will be relevant to the Figure 3A and 3B Describe in more detail.

[0064] In response to determining that the current video frame follows a scene change, the method moves to step 408, where an ambient light signal is received. The ambient light signal includes at least an indication of the intensity of the ambient light. Optionally, the ambient light signal further includes an indication of the color of the ambient light. In step 410, the current settings of ambient light compensation are adjusted in dependence on the ambient light signal received in step 408. The method then moves to step 412, where the current settings of ambient light compensation—updated in step 410—are applied.

[0065] In response to determining that the current video frame does not immediately follow a scene change, the method moves from step 406 to step 412. In such a case, the setting of ambient light compensation is not adjusted, i.e., the previous setting of ambient light compensation is maintained. Thus, in this case, at least step 410 is omitted. At step 412, the current setting is applied, which in this scenario is the same as the ambient light setting of the previous frame.

[0066] After step 412, it is determined whether the current frame is the final video frame in step 414. If not, i.e., the next video frame is available, then the next video frame is set as the current video frame and the method returns to step 404. If the current frame is the final video frame, then the method ends at step 418.

[0067] exist Figure 2 In the example of , step 408 is performed immediately before step 410. However, step 408 may also be performed between steps 404 and 406. In another example, step 408 is performed before step 404, as the first step of the loop over the illustrated video frame, followed by step 404.

[0068] Figure 3A A first example of a sub-step of step 406 is shown, namely, a step of determining whether the current video frame immediately follows a scene change. Figure 3A In an example of , metadata is received in sub-step 406-1. The metadata includes metadata indicating a scene change. In an example, the metadata indicating a scene change includes a frame-by-frame marker indicating whether the current frame starts a new scene. In sub-step 406-2, the method determines whether the marker is set, i.e., whether the current frame starts a new scene. In response to determining that the marker is set, the method continues to step 408 as described above. In response to determining that the marker has not been set, the method continues to step 412 as described above. In another example, the metadata indicating a scene change includes a frame-by-frame identifier for the scene with which the current frame is associated. For example, the identifier may include a unique numeric, alphabetic, or alphanumeric identifier to identify different scenes. In such a scenario, step 406-2 compares the identifier of the current video frame with the identifier of the previous video frame (i.e., the video frame immediately preceding the current video frame). If it is determined that the current video frame has the same scene identifier as the previous video frame, i.e., they are associated with the same scene, then the metadata therefore indicates that no scene change has occurred, and the method continues to step 412. If it is determined that the current video frame has a different scene identifier than the previous video frame, ie, they are associated with different scenes, the method continues to step 408 .

[0069] Figure 3B A second example of sub-steps of step 406 is shown. In sub-step 406-A, characteristics of the current video frame are determined. For example, brightness characteristics of the current video frame are determined. For example, a median or average illumination value of the current video frame may be calculated or extracted from metadata, as described above.

[0070] In sub-step 406-B, a difference is calculated between one or more characteristics of the current video frame and the same one or more characteristics of at least one previous video frame. For example, a difference is calculated between the median illumination of the current video frame and the median illumination of the previous video frame. In another example, a difference is calculated between a characteristic of the current video frame and an average (or weighted average) of the same characteristic of N previous video frames, such as a difference between the median illumination of the current video frame and the average (or weighted average) of the median illumination of the N previous frames.

[0071] In step 406-C, it is determined whether the difference calculated in step 406-B satisfies a threshold condition. In the illustrated example, it is determined whether the difference exceeds a threshold. In response to determining that the difference exceeds the threshold, the method proceeds to step 408. In response to determining that the difference does not exceed the threshold, the method proceeds to step 412.

[0072] In another embodiment, the same functionality is implemented by determining whether the difference is less than a threshold. In response to determining that the difference is less than the threshold, the method proceeds to step 412. In response to determining that the difference is not less than the threshold, the method proceeds to step 408.

[0073] As described above, the DM module 106, 206, 306 applies ambient light compensation to the input video signal 108, 208, 308 depending on the ambient light intensity. For example, applying ambient light compensation includes adjusting brightness according to the ambient light intensity, where the brightness increases as the ambient light intensity increases. Let v ij Denote the luminance component of a video frame j with P pixels i=0…P–1, and let I denote the ambient light intensity. Then, the output video frame can be calculated as: Wherein f(I) represents a function of the ambient light intensity. Here, the ambient light intensity I is calculated based on the signal 112, 212, 312 received from the ambient light sensor 104, 204, 304. For example, the signal may be temporally smoothed as described above to obtain the ambient light intensity. The function f(I) may take both negative and positive values. For example, when the input video signal is written in a 5 nit reference environment, f(I) may take a negative value for a value of I corresponding to an ambient light intensity of less than 5 nits, f(I)=0 for a value of I corresponding to an ambient light intensity of 5 nits, and f(I)>0 for a value of I corresponding to an ambient light intensity of more than 5 nits. The function f(I) may be a linear or nonlinear function.

[0074] In an embodiment, applying ambient light compensation includes adjusting brightness based on both ambient light intensity and intensity values ​​of the input video frame (eg, pixel values ​​of a luminance component of the input video frame). ij represents the brightness component of a video frame j with P pixels i=0…P–1, and I represents the ambient light intensity. Then, the output video frame can be calculated as: where g(v ij , I) represents a function of both the ambient light intensity and the luminance value of pixel i in frame j.

[0075] In an embodiment, the function f(I) or g(v ij , I) is implemented as a look-up table (LUT).

[0076] Figure 4 An example of the function g(v ij , I) showing four different levels of ambient light intensity I is presented. In the Figure 4 graph, the abscissa shows the input illuminance expressed in candela per square meter, which corresponds one-to-one with the input luminance value v ij . The ordinate shows the output illuminance expressed in candela per square meter, which corresponds one-to-one with the output luminance value . The figure shows the relationship between the input illuminance and the output illuminance at 5 nits (405), 100 nits (410), 500 nits (415), and zero nits (420).

[0077] As Figure 4 depicted, when the viewing environment matches the reference environment (e.g., 5 nits), the function 505 represents a straight line with a slope = 1, i.e., no ambient compensation is applied. For darker (e.g., function 520) or brighter (e.g., function 510 or function 515) viewing environments, the ambient light compensation is decreased or increased as needed. As can be seen from Figure 4 , the ambient light compensation depends on both the ambient light intensity and the input illuminance of the input video frame.

[0078] The DM modules 106, 206, 306 can access a predetermined number of ambient light compensation maps (e.g., in the form of one or more LUTs or 3D LUTs) stored internally in the display devices 100, 200, 300 (e.g., stored in the memories of the DM modules 106, 206, 306). In this example, the DM modules 106, 206, 306 can be configured to select one of a set of ambient light compensation maps depending on a signal indicating the intensity of the ambient light. For ambient light intensities not represented in the stored LUTs, interpolation techniques can be applied to derive an appropriate LUT. For example, for two ambient light values I1 and I2, given the pre-computed curves g(v ij , I1) and g(v ij , I2), a new curve g(v ij , I) for I1 < I < I2 can be generated by interpolating between the g(v ij , I1) and g(v ij , I2) values.

[0079] As described above, the light sensor for ambient light compensation in the display device is usually positioned at the front side of the display device to obtain a measure of the light intensity of the black level of the displayed image. However, the positioning of the sensor near the front of the display device may cause the sensor to also record some light emitted by the display device itself. Therefore, the signal generated by the sensor may not be an accurate measure of ambient light. On the contrary, the signal of the sensor depends on both ambient light and the light emitted by the display device. Since the light emitted by the display device may change rapidly due to the change of the input video signal, this may also affect the output of the light sensor, and in traditional display devices, it may be another reason for the occurrence of flicker. The synchronization of the environment adjustment to the scene change as described above is also effective in reducing or eliminating the flicker caused by the sensor recording the light emitted by the display device itself. However, the signal generated by the sensor may be an inaccurate measure of ambient light intensity. Therefore, the ambient light compensation based on the sensor output application may increase or decrease the brightness of the displayed image unnecessarily in view of the actual ambient light.

[0080] LCD displays may blank the screen at regular intervals. Therefore, the above problem of reduced accuracy can be solved by only measuring from the ambient light sensor during the time when the screen is blanked. However, the disadvantage of this approach is that the amount of time the screen is blanked is relatively short, otherwise the average display brightness will be affected. If the screen is off 10% of the time, then the sensor will only measure ambient light 10% of the time and a gain of 10 times will need to be applied, resulting in more noise in the measurement. This is undesirable because noise in the output signal of the sensor is the cause of flicker.

[0081] The problem of display devices interfering with the measurements of the ambient light sensor increases with increasing brightness. Therefore, the interference is greater for HDR display devices, which typically have a higher maximum brightness. However, the problem also exists in SDR display devices, so the present disclosure is not limited to HDR, but covers both HDR and SDR implementations.

[0082] The inventors have recognized that by using an ambient light sensor that is less sensitive to wavelengths of light corresponding to the primary colors of the display device than to other wavelengths of light in the visible spectrum, the accuracy of ambient light compensation can be improved without increasing the occurrence of flicker.

[0083] To display a color image, a display device includes pixels of three or more different colors, referred to as primary colors of the display device. For example, a display device may include red (R), green (G), and blue (B) pixels. Ambient light sensors (e.g., sensors 104, 204, 304) are of a type that is not very sensitive to the primary colors of the display device (e.g., to the wavelengths corresponding to the RGB colors of the pixels of the display device).

[0084] Figure 5 The embodiments show examples of such ambient light sensors. Figure 5 The ambient light sensor 604 includes a photodetector 603a and an optical filter 603b. The photodetector 603a may include, for example, a photodiode, a phototransistor, a photoresistor, one or more reverse biased LEDs, or any other suitable type of photodetector. The optical filter 603b reduces or blocks one or more primary colors of the display device. For example, the optical filter 603b may be an optical band-stop filter (also referred to as an optical notch filter) that attenuates light of wavelengths corresponding to the primary colors of the display device. In other words, the stop band of the optical band-stop filter corresponds to the spectral power distribution of light emitted by the display device. Fig. 6A An example of such sensor characteristics is schematically shown in . The abscissa corresponds to wavelength and the ordinate corresponds to sensor sensitivity. The sensor has a generally flat response across the visible spectrum, except for three notches, each corresponding to one of the primary colors of an exemplary display device.

[0085] For example, an optical band-stop filter may be used as the optical filter 603b to implement Fig. 6A In another example, instead of Fig. 6A The illustrated sensor has a band-stop characteristic, and a sensor with a band-pass characteristic can be provided, wherein the passband of the sensor corresponds to a band that does not contain the primary colors of the display device. For example, an optical band-pass filter can be provided as the optical filter 603b. Figure 6B The sensitivity of a sensor having such a bandpass characteristic is schematically shown. The solid line illustrates a first passband, which corresponds to cyan light, such as light having a wavelength of 490 to 520 nm. The dashed line illustrates a second passband, which corresponds to yellow light, such as light having a wavelength of 570 to 590 nm. The response of the sensor may include only the first passband, only the second passband, or may include both the first passband and the second passband. For example, the bandpass characteristic of the sensor is achieved by using a yellow filter as the optical filter 603b, or by using a cyan filter as the optical filter 603b, or by using an optical filter 603b with dual passbands, or by using an optical filter 603b that includes a mosaic of yellow filters and cyan filters.

[0086] In another example, the optical filter 603b may include a color filter mosaic including a red filter (R), a green filter (G), a blue filter (B), a yellow filter (Y), and a cyan filter (C). This may be labeled as an RYGCB sensor. Optionally, a magenta filter (M) is also included. The R, G, and B filters of the mosaic correspond to the R, G, B primary colors of the display device. For ambient light compensation, only the Y and C pixels of the RYGCB sensor are considered to avoid adjusting the ambient light compensation to the light emitted by the display device itself. The R, G, and B pixels can be used to determine the color of the ambient light for adjusting the color mapping of the video signal to be displayed. Therefore, the present disclosure further relates to such a sensor itself: a light sensor including an optical filter in the form of a RYGCB mosaic (optionally a RYGCBM mosaic).

[0087] The optical filter 603b may thus block or attenuate other wavelengths, such as red light, green light, and blue light used in an RGB display device.

[0088] The use of an ambient light sensor that is not very sensitive to the primary colors of the display device is particularly effective for display devices that include one or more lasers (such as laser projectors), because the laser light source has a very narrow bandwidth. Therefore, the laser light illuminating the sensor can be filtered out very effectively.

[0089] An advantage of using a sensor that is sensitive only to yellow light is that in typical viewing environments the major contribution to the ambient light intensity is emitted by lamps that typically emit predominantly yellow light.

[0090] Some displays include blue and yellow phosphors, in which case it is advantageous to use a sensor that is sensitive only to cyan light.

[0091] Instead of using an optical filter 603b, an ambient light sensor that is inherently insensitive to the primary colors of the display device may be selected. For example, the ambient light sensor includes a photosensitive semiconductor device (e.g., a photodiode or a reverse biased LED) having a bandgap corresponding to a wavelength different from the wavelength of the primary colors of the display device. For example, a semiconductor device having a bandgap corresponding to a wavelength in the yellow portion of the spectrum (e.g., 570 to 590 nm). In another example, two or more photosensitive semiconductor devices with different bandgaps are used, each corresponding to a wavelength different from the wavelength of the primary colors of the display device. For example, the ambient light sensor includes a photodiode having a bandgap corresponding to a wavelength in the yellow portion of the spectrum (e.g., 570 to 590 nm) and another photodiode having a bandgap corresponding to a wavelength in the cyan portion of the spectrum (e.g., 490 to 520 nm).

[0092] The above combination describes a) synchronizing ambient light compensation with scene changes, and b) using a metric of an ambient light sensor with reduced sensitivity to light of wavelengths corresponding to the primary colors of the display device. However, the present disclosure is not limited to this combination, and these features can be implemented as independent metrics. It should be noted that the effect of reduced flicker can also be achieved when an ambient light sensor with reduced sensitivity to the primary colors of the display device is used as an independent metric, that is, when the ambient light adjustment is not synchronized with the scene changes, because flicker caused by detecting light emitted by the display device is prevented.

[0093] In an embodiment, a display device includes an ambient light sensor configured to generate a signal indicative of an intensity of ambient light, wherein the device is configured to receive an input video signal and to apply ambient light compensation to the input video signal in dependence on the signal indicative of the intensity of ambient light, wherein the ambient light sensor is less sensitive to light of wavelengths corresponding to primary colors of the display device than to light of other wavelengths of the visible spectrum.

[0094] For example, the display device includes a projector such as a laser projector or a display such as an LCD or OLED display.

[0095] For example, the display apparatus comprises a controller and / or a DM module configured to receive the input video signal and to apply ambient light compensation to the input signal in dependence on a signal indicative of the intensity of the ambient light.

[0096] As described above, by using an ambient light sensor with reduced sensitivity to the primary colors of the display device, interference of light emitted by the display device itself with the measurement performed by the ambient light sensor is avoided or at least reduced. Additionally or alternatively, the device (e.g., a DM module of the device) may be configured to determine an expected measurement bias based on the input video signal and take the expected measurement bias into account when applying ambient light compensation based on the sensor output.

[0097] Figure 7 An example of a process for accounting for expected measurement bias caused by a sensor recording light emitted by the display device itself is illustrated. An input video signal 702 includes a sequence of video frames 704-1 to 704-4. The current frame is depicted as "frame n", and the previous three video frames are depicted as "frame n-3", "frame n-2", and "frame n-1", respectively. The input video signal 702 may further include metadata (not shown), such as frame-based metadata and or scene-based metadata, as described in more detail above.

[0098] Function block 706 performs operation f on each video frame 704 to extract a measure of the overall intensity level of each video frame. Function block 706 is the same for each video frame 704, such that operation f is also the same for each video frame 704. Operation f can be performed on the image data of each video frame or the metadata of the video frame. For example, operation f can analyze the pixel intensity values of the image data to determine the overall intensity level of video frame 704. In another example, operation f uses frame metadata (such as the S min 、S mid or S max metadata) to determine the overall intensity level of video frame 704. The result of operation f is a scalar indicating the overall intensity level of the corresponding video frame.

[0099] For example, the overall intensity of video frame 704 can be calculated as the average pixel intensity value of the video frame. Let P represent the number of pixels in each frame, i represent the pixel index where 0 ≤ i < P, and L i represent the pixel intensity value. Then, in this example:

[0100]

[0101] Alternatively, the average intensity of the intensity values of the video frame may be available in the frame metadata, and operation f extracts the average intensity from the metadata of each frame.

[0102] In another example, the overall intensity of video frame 704 is calculated as the average or median of the pixel intensity values of the video frame. In yet another example, the overall intensity of video frame 704 is calculated as the average of the maximum pixel intensity value and the minimum pixel intensity value of the video frame:

[0103]

[0104] Alternatively, max(L i ) or min(L i ) may be available in the frame metadata, and operation f extracts those values from the metadata, calculates their sum, and divides by 2. In another example, instead of calculating the average of max and min, the operation directly extracts the S mid value from the frame metadata.

[0105] The term "pixel intensity value" refers to a per-pixel measure of the intensity of the pixel. For example, the pixel intensity value may correspond to the pixel value of the luminance component of the image data. In another example, the pixel intensity value is calculated from the pixel values of the image data. For example, in an RGB image, the intensity value of a pixel can be calculated as a linear combination of the R, G, and B pixel values.

[0106] In block 708, an expected measurement bias 712 is calculated, which includes the step of summing the intensity levels generated by block 706. In an embodiment, as illustrated by the Σ (sigma) symbol in block 708, the expected measurement bias 712 simply corresponds to the calculated sum. In another embodiment, the expected measurement bias 712 is calculated by dividing the sum by the number of video frames 704 used in the calculation to obtain an average intensity of the video frames 704. In another embodiment, the expected measurement bias 712 is calculated by computing a linear combination (e.g., weighted average) of the intensities of the video frames 704. For example, a weighted average where the intensity of the more recent video frames (e.g., 704-3) has a larger weighting factor than for the older video frames (e.g., 704-1).

[0107] The measurement configuration 712 calculated in block 708 is used in functional block 710 to adjust the ambient light intensity value 714 based on the measurements performed by the ambient light sensor. The measured value 714 can be obtained directly from the ambient light sensor, or preprocessing steps such as analog-to-digital conversion, scaling, or offset can be performed on the sensor's signal to convert it into a measured value of the ambient intensity 714. In block 710, the measured value 714 is adjusted by compensating for the expected measurement bias 712 determined based on the input video signal 702. The output of block 710 is a corrected value of the ambient light intensity, and subsequent ambient light compensation will be based on this corrected value.

[0108] For example, the compensation performed in block 710 can include subtracting the expected measurement bias 712 from the measured value 714: I corrected = I measured - I bias . In another example, the compensation performed in block 710 includes scaling the expected bias before subtracting: I corrected = I measured – a * I bias . For example, 0 < a ≤ 1. In another example, the expected bias determined in block 708 is transformed via a second-order polynomial before subtracting: I corrected = I measured – (α I bias 2 + β I bias + γ). In yet another example, the expected bias determined in block 708 is transformed via a LUT before subtracting from I measured . In yet another example, the compensation performed in block 710 uses a 2D look-up table (LUT) to transform the data pair (I measured , I bias ) into a corrected intensity value I corrected .

[0109] For example, a calibration procedure may be used to determine the scale factor a, the parameters α, β, γ, the LUT or the 2D LUT. In the first step of the calibration procedure, the display device is configured to display a test image with minimal light emission. For example, the display device is configured to display a black image. In an example of a display device having a display with a backlight, the backlight may be turned off for the first step of the calibration process. The response from the ambient light sensor is measured. In the second step of the calibration procedure, which may be performed before or after the first step, at least one test image is displayed and the response from the ambient light sensor is measured. For example, a series of test images are displayed sequentially, each of which has a different overall intensity. The first step and the second step are performed within a relatively small time frame, not exceeding 2 to 3 minutes, but typically less than 30 seconds. Therefore, the difference between the sensor output when displaying a black image and the sensor output when displaying a test image is mainly attributable to the sensor recording the light emitted by the display device itself. In an embodiment, the difference is calculated to obtain a calibration bias level for each test image. Each test image has a known overall intensity. Therefore, a LUT that associates the overall intensity of the image displayed on the display device with a calibrated sensor bias may be constructed. In an example, interpolation is used to obtain other entries of the LUT. Additionally or alternatively, a function may be fit to the measured pairs of overall image intensity and bias values, for example, to obtain the parameters a, α, β, or γ described above. In an embodiment, rather than measuring once with a black screen and then measuring a series of test images, measurements with a black screen are alternated with measuring one or more test images.

[0110] exist Figure 7 In the example of , the current video frame "Frame n" is also used to calculate the sum in block 708. However, as indicated by the dashed line, the inclusion of "Frame n" is optional, and in some embodiments, the calculation is based only on the video frames preceding the current video frame. In the example shown, the summation is performed over four frames, but the disclosure is not limited to summation over four frames. In one embodiment, the predetermined number of video frames used to calculate the measurement bias may be in the range of 2 to 120 video frames, such as in the range of 30 to 90 video frames. Additionally, in the illustrated example, the calculation is performed over consecutive video frames. However, the disclosure is not limited to using consecutive video frames for calculation of the measurement bias, for example, other video frames may be used.

[0111] The predetermined number of video frames used in the calculation corresponds to a certain duration. For example, in the case of 60 frames per second, 60 video frames correspond to a duration of 1 second. In an embodiment, the predetermined number of video frames to be used in the calculation (and optionally also during calibration) corresponds to a duration arranged in the order of the integration constant of the sensor. Typical values ​​for the duration are 0.01 to 2 seconds, for example 0.5 to 1.5 seconds.

[0112] The embodiments described herein may be implemented in hardware, software, firmware, and combinations thereof. For example, the embodiments may be implemented on a system including electronic circuit systems and components, such as a computer system. Examples of computer systems include desktop computer systems, portable computer systems (e.g., laptop computers), handheld devices (e.g., smartphones or tablet computers), and network devices. Systems for implementing the embodiments may, for example, include at least one of an integrated circuit (IC), a programmable logic device (PLD) (e.g., a field programmable gate array (FPGA)), a digital signal processor (DSP), an application specific IC (ASIC), a central processing unit (CPU), and a graphics processing unit (GPU).

[0113] Certain implementations of the embodiments described herein may include a computer program product including instructions that, when executed by a data processing system, cause the data processing system to perform the method of any of the embodiments described herein. The computer program product may include a non-transitory medium storing the instructions, such as physical media (e.g., magnetic data storage media including floppy disks and hard drives), optical data storage media including CD ROMs and DVDs, and electronic data storage media including ROMs, flash memories (e.g., flash RAMs or USB flash drives). In another example, the computer program product includes a data stream including the instructions, or a file including the instructions stored in a distributed computing system (e.g., in one or more data centers).

[0114] The present disclosure is not limited to the embodiments and examples described above. Many modifications and variations are possible without departing from the scope of the present disclosure as defined by the appended claims.

[0115] Various aspects of the present disclosure may be understood from the following enumerated example embodiments (EEE):

[0116] EEE 1. A display device comprising:

[0117] - an ambient light sensor configured to generate a signal indicative of the intensity of ambient light,

[0118] The display device is configured to:

[0119] - receiving an input video signal; and

[0120] - applying ambient light compensation to said input video signal in dependence on said signal indicative of the intensity of said ambient light,

[0121] The ambient light sensor is less sensitive to light of wavelengths corresponding to primary colors of the display device than to light of other wavelengths in the visible spectrum.

[0122] EEE 2. The display device according to EEE 1, wherein the ambient light sensor comprises an optical filter that attenuates light of wavelengths corresponding to primary colors of the display device. ...

[0123] EEE 3. The display device according to EEE 2, wherein the optical filter comprises an optical band-stop filter. EEE 3. The display device according to EEE 2, wherein the optical filter comprises an optical band-stop filter.

[0124] EEE 4. The display device according to EEE 1, wherein the ambient light sensor is sensitive only to one or more bands within the visible spectrum. ...

[0125] EEE 5. The display device according to EEE 4, wherein the ambient light sensor is sensitive only to yellow light. ...

[0126] EEE 6. The display device according to EEE 4, wherein the ambient light sensor is sensitive only to cyan light. EEE 6.

[0127] EEE 7. The display device according to EEE 4, wherein the ambient light sensor is sensitive only to yellow light and cyan light. EEE 7.

[0128] EEE 8. The display device according to any one of EEEs 4 to 7, wherein the ambient light sensor comprises an optical bandpass filter. EEE 8.

[0129] EEE 9. A display device according to any one of EEEs 1 to 8, further comprising a display, wherein the display device is configured to transform the input video signal into an output video signal suitable for the display, comprising applying ambient light compensation to the input video signal in dependence on the signal indicating the intensity of the ambient light, and wherein the display is configured to display the output video signal.

[0130] EEE 10. A display device according to any one of EEEs 1 to 8, further comprising a projector, wherein the display device is configured to transform the input video signal into an output video signal suitable for the projector, comprising applying ambient light compensation to the input video signal in dependence on the signal indicating the intensity of the ambient light.

[0131] EEE 11. The display device according to EEE 10, wherein the projector is a laser projector.

[0132] EEE 12. A display device comprising:

[0133] - an ambient light sensor configured to generate a signal indicative of the intensity of ambient light,

[0134] The display device is configured to:

[0135] - receiving an input video signal; and

[0136] - applying ambient light compensation to said input video signal in dependence on said signal indicative of the intensity of said ambient light,

[0137] wherein the display device is further configured to determine a measured bias attributable to the light emitted by the display device and recorded by the ambient light sensor based on at least one of the input video signal and metadata associated with the input video signal, and

[0138] Wherein applying ambient light compensation comprises applying ambient light compensation to the input video signal in dependence on the signal indicative of the intensity of the ambient light compensated by the determined measurement bias.

[0139] EEE 13. The display device according to EEE 12, wherein determining the measurement bias comprises:

[0140] determining an intensity level for each of a predetermined number of video frames preceding the current video frame; and

[0141] A sum of the intensity levels of the predetermined number of video frames is determined.

[0142] EEE 14. A display device according to EEE 12 or EEE 13, further comprising a display, wherein the display device is configured to transform the input video signal into an output video signal suitable for the display, comprising applying ambient light compensation to the input video signal in dependence on the signal indicating the intensity of the ambient light, and wherein the display is configured to display the output video signal.

[0143] EEE 15. A method for applying ambient light compensation, comprising:

[0144] - receiving an input video signal comprising a sequence of video frames;

[0145] - receiving a signal from an ambient light sensor indicating the intensity of ambient light;

[0146] - determining whether a current video frame of the sequence of video frames immediately follows a scene change; and

[0147] - transforming said input video signal into an output video signal suitable for a target display device, comprising applying ambient light compensation to said input video signal in dependence on said signal indicative of the intensity of said ambient light,

[0148] The method further comprises:

[0149] determining a measured bias attributable to the light emitted by the display device and recorded by the ambient light sensor based on at least one of the input video signal and metadata associated with the input video signal,

[0150] Wherein applying ambient light compensation comprises applying ambient light compensation to the input video signal in dependence on the signal indicative of the intensity of the ambient light compensated by the determined measurement bias.

[0151] EEE 16. The method according to EEE 15, wherein determining the measurement bias comprises:

[0152] determining an intensity level for each of a predetermined number of video frames preceding the current video frame; and

[0153] A sum of the intensity levels of the predetermined number of video frames is determined.

[0154] EEE 17. A display device comprising:

[0155] - an ambient light sensor configured to generate a signal indicative of the intensity of ambient light; and

[0156] - A display management module configured to:

[0157] o receiving an input video signal comprising a sequence of video frames;

[0158] o determining whether a current video frame of the sequence of video frames immediately follows a scene change; and

[0159] o applying ambient light compensation to said input video signal in dependence on said signal indicative of the intensity of said ambient light,

[0160] Wherein the display management module is configured to adjust the ambient light compensation applied to the input signal in dependence on the signal indicative of an intensity of ambient light only in response to determining that the current video frame of the sequence of video frames immediately follows a scene change.

[0161] EEE 18. A display device according to EEE 17, wherein the display management module is configured to receive metadata related to the sequence of video frames, the metadata including metadata indicating a scene change, and the display management module is configured to use the metadata indicating the scene change to determine whether the current video frame follows a scene change.

[0162] EEE 19. The display device according to EEE 17, wherein the display management module is configured to compare characteristics of the current video frame with characteristics of one or more previous video frames to determine whether the current video frame of the sequence of video frames follows a scene change. EEE 19.

[0163] EEE 20. The display device according to any one of EEE 17 to 19, wherein the ambient light sensor is less sensitive to light of wavelengths corresponding to primary colors of the display device than to light of other wavelengths of the visible spectrum. EEE 21. The display device according to any one of EEE 17 to 19, wherein the ambient light sensor is less sensitive to light of wavelengths corresponding to primary colors of the display device than to light of other wavelengths of the visible spectrum.

[0164] EEE 21. The display device according to EEE 20, wherein the ambient light sensor comprises an optical filter that attenuates light of wavelengths corresponding to primary colors of the display device. EEE 22. The display device according to EEE 22, wherein the ambient light sensor comprises an optical filter that attenuates light of wavelengths corresponding to primary colors of the display device.

[0165] EEE 22. The display device according to EEE 21, wherein the optical filter comprises an optical band-stop filter. EEE 23. The display device according to EEE 23, wherein the optical filter comprises an optical band-stop filter.

[0166] EEE 23. The display device according to EEE 22, wherein the ambient light sensor is sensitive only to one or more bands within the visible spectrum. ...

[0167] EEE 24. The display device according to EEE 23, wherein the ambient light sensor is sensitive only to yellow light, only to cyan light, or only to yellow light and cyan light. ...

[0168] EEE 25. A display device according to EEE 23 or 24, wherein the ambient light sensor comprises an optical bandpass filter.

[0169] EEE 26. A display device according to any one of EEEs 17 to 25, wherein applying ambient light compensation includes applying an ambient light compensation function, which maps input intensity values ​​of the video frames of the input video signal to output intensity values ​​of corresponding video frames of the output video signal, wherein the display management module is configured to select one of a set of ambient light compensation functions depending on the signal indicating the intensity of ambient light.

[0170] EEE 27. The display device according to any one of EEE 17 to 26, wherein the display management module is further configured to determine a measured bias due to the light emitted by the display device and recorded by the ambient light sensor based on at least one of the input video signal and metadata related to the input video signal,

[0171] Wherein applying, by the display management module, ambient light compensation comprises applying ambient light compensation to the input video signal in dependence upon the signal indicative of an intensity of the ambient light compensated by the determined measurement bias.

[0172] EEE 28. The display device according to EEE 27, wherein determining the measurement bias comprises:

[0173] determining an intensity level for each of a predetermined number of video frames preceding the current video frame; and

[0174] A sum of the intensity levels of the predetermined number of video frames is determined.

[0175] EEE 29. A display management module configured to:

[0176] o receiving an input video signal comprising a sequence of video frames;

[0177] o receiving a signal indicative of the intensity of ambient light;

[0178] o determining whether a current video frame of the sequence of video frames immediately follows a scene change;

[0179] o transforming the input video signal into an output video signal suitable for a target display device, comprising applying ambient light compensation to the input video signal in dependence on the signal indicative of the intensity of the ambient light; and

[0180] ○ outputting the output video signal for use by the target display device;

[0181] Wherein the display management module is configured to adjust the ambient light compensation applied to the input signal in dependence on the signal indicative of an intensity of ambient light only in response to determining that the current video frame of the sequence of video frames immediately follows a scene change.

[0182] EEE 30. A method for applying ambient light compensation, comprising:

[0183] - receiving an input video signal comprising a sequence of video frames;

[0184] - receiving a signal from an ambient light sensor indicating the intensity of ambient light;

[0185] - determining whether a current video frame of the sequence of video frames immediately follows a scene change; and

[0186] - transforming said input video signal into an output video signal suitable for a target display device, comprising applying ambient light compensation to said input video signal in dependence on said signal indicative of the intensity of said ambient light,

[0187] The method comprises adjusting the applied ambient light compensation in dependence upon the signal indication only in response to determining that the current video frame of the sequence of video frames immediately follows a scene change.

[0188] EEE 31. The method according to EEE 30 further includes receiving metadata related to the sequence of video frames, the metadata including metadata indicating a scene change, wherein the determining whether the current video frame of the sequence of video frames follows a scene change uses the metadata indicating the scene change.

[0189] EEE 32. The method according to EEE 30, wherein the determining whether the current video frame of the sequence of video frames corresponds to a scene change comprises automatically detecting the scene change. EEE 33. The method according to EEE 33, wherein the determining whether the current video frame of the sequence of video frames corresponds to a scene change comprises automatically detecting the scene change.

[0190] EEE 33. The method according to EEE 32, wherein automatically detecting the scene change comprises comparing characteristics of the current video frame with characteristics of one or more previous video frames. EEE 34. The method according to EEE 34, wherein automatically detecting the scene change comprises comparing characteristics of the current video frame with characteristics of one or more previous video frames.

[0191] EEE 34. A computer program product having instructions which, when executed by a computing device or system, cause the computing device or system to perform a method according to EEE 15 or any one of EEE 30 to 33. EEE 35.

[0192] EEE 35. A non-transitory computer-readable storage medium having computer-executable instructions stored thereon for performing the method according to any one of EEE 15 or 30 to 33. EEE 35.

[0193] EEE 36. A display device comprising:

[0194] -monitor;

[0195] - an ambient light sensor configured to generate a signal indicative of the intensity of ambient light; and

[0196] - A display management module configured to:

[0197] o receiving an input video signal comprising a sequence of video frames;

[0198] o determining whether a current video frame of the sequence of video frames immediately follows a scene change; and

[0199] o transforming the input video signal into an output video signal suitable for the display, comprising applying ambient light compensation to the input video signal in dependence on the signal indicative of the intensity of the ambient light,

[0200] wherein the display is configured to display the output video signal, and

[0201] Wherein the display management module is configured to adjust the ambient light compensation applied to the input signal in dependence on the signal indicative of an intensity of ambient light only in response to determining that the current video frame of the sequence of video frames immediately follows a scene change.

[0202] EEE 37. A display device according to EEE 36, wherein the display management module is configured to receive metadata related to the sequence of video frames, the metadata including metadata indicating a scene change, and the display management module is configured to use the metadata indicating the scene change to determine whether the current video frame follows a scene change.

[0203] EEE 38. The display device according to EEE 36, wherein the display management module is configured to compare characteristics of the current video frame with characteristics of one or more previous video frames to determine whether the current video frame of the sequence of video frames follows a scene change. EEE 38.

[0204] EEE 39. The display device according to any one of EEEs 36 to 38, wherein the ambient light sensor is less sensitive to light of wavelengths corresponding to primary colors of the display than to light of other wavelengths of the visible spectrum. ...

[0205] EEE 40. The display device according to EEE 39, wherein the ambient light sensor comprises an optical filter that attenuates light of wavelengths corresponding to primary colors of the display. EEE 40.

[0206] EEE 41. The display device according to EEE 40, wherein the optical filter comprises an optical band-stop filter. ...

[0207] EEE 42. A display device according to EEE 39, wherein the ambient light sensor is sensitive only to one or more bands within the visible spectrum. EEE 42.

[0208] EEE 43. The display device according to EEE 42, wherein the ambient light sensor is sensitive only to yellow light, only to cyan light, or only to yellow light and cyan light. ...

[0209] EEE 44. A display device according to EEE 42 or 43, wherein the ambient light sensor comprises an optical bandpass filter. EEE 44.

[0210] EEE 45. A display device according to any one of EEEs 36 to 44, wherein applying ambient light compensation includes applying an ambient light compensation function, which maps input intensity values ​​of the video frames of the input video signal to output intensity values ​​of corresponding video frames of the output video signal, and wherein the display management module is configured to select one of a set of ambient light compensation functions depending on the signal indicating the intensity of ambient light.

[0211] EEE 46. The display device of any one of EEE 36 to 45, wherein the display management module is further configured to determine a measured bias attributable to the light emitted by the display and recorded by the ambient light sensor based on at least one of the input video signal and metadata related to the input video signal,

[0212] Wherein applying, by the display management module, ambient light compensation comprises applying ambient light compensation to the input video signal in dependence upon the signal indicative of an intensity of the ambient light compensated by the determined measurement bias.

[0213] EEE 47. The display device according to EEE 46, wherein determining the measurement bias comprises:

[0214] determining an intensity level for each of a predetermined number of video frames preceding the current video frame; and

[0215] A sum of the intensity levels of the predetermined number of video frames is determined.

[0216] EEE 48. A display management module configured to:

[0217] o receiving an input video signal comprising a sequence of video frames;

[0218] o receiving a signal indicative of the intensity of ambient light;

[0219] o determining whether a current video frame of the sequence of video frames immediately follows a scene change;

[0220] o transforming the input video signal into an output video signal suitable for a target display, comprising applying ambient light compensation to the input video signal in dependence on the signal indicative of the intensity of the ambient light; and

[0221] ○ outputting the output video signal for use by the target display;

[0222] Wherein the display management module is configured to adjust the ambient light compensation applied to the input signal in dependence on the signal indicative of an intensity of ambient light only in response to determining that the current video frame of the sequence of video frames immediately follows a scene change.

[0223] EEE 49. A method for applying ambient light compensation, comprising:

[0224] - receiving an input video signal comprising a sequence of video frames;

[0225] - receiving a signal from an ambient light sensor indicating the intensity of ambient light;

[0226] - determining whether a current video frame of the sequence of video frames immediately follows a scene change; and

[0227] - transforming the input video signal into an output video signal suitable for a target display, comprising applying ambient light compensation to the input video signal in dependence on the signal indicative of the intensity of the ambient light,

[0228] The method comprises adjusting the applied ambient light compensation in dependence upon the signal indication only in response to determining that the current video frame of the sequence of video frames immediately follows a scene change.

[0229] EEE 50. A computer program product having instructions which, when executed by a computing device or system, cause the computing device or system to perform the method according to EEE 49.

Claims

1. A display device, comprising: a display configured to display an image in response to a first video signal, the display comprising pixels of three or more primary colors; as well as an ambient light sensor configured to generate a signal indicative of an intensity of ambient light, the ambient light sensor being positioned to register light emitted by the display in addition to the ambient light; The display device is configured to: receiving a second video signal; and generating the first video signal by applying ambient light compensation to the second video signal in dependence on the signal indicative of the intensity of the ambient light; wherein the ambient light sensor is less sensitive to light of wavelengths corresponding to the primary colors of the display than to light of other wavelengths of visible light; and wherein the display device is further configured to determine a measurement bias caused by the ambient light sensor recording the light emitted by the display, and to account for the measurement bias when applying the ambient light compensation by: computing a linear combination of overall frame intensity levels for a sequence of video frames; and The linear combination or a second order polynomial of the linear combination is subtracted from the signal indicative of the intensity of the ambient light.

2. The display device of claim 1, wherein the ambient light sensor comprises an optical filter that attenuates light of the wavelengths corresponding to the primary colors of the display. 3 . The display device of claim 1 , wherein the ambient light sensor comprises an optical band-stop filter having an optical stop-band at the wavelength corresponding to the primary color of the display. 4 . The display device of claim 1 , wherein the ambient light sensor comprises an optical bandpass filter having one or more passbands that pass only the other wavelengths.

5. The display device according to claim 4, wherein the primary colors are red, green and blue; and The ambient light sensor is only sensitive to yellow light.

6. The display device according to claim 4, wherein the primary colors are red, green and blue; and The ambient light sensor is only sensitive to cyan light.

7. The display device according to claim 4, wherein the primary colors are red, green and blue; and The ambient light sensor is only sensitive to yellow light and cyan light.

8. The display device according to claim 1, wherein the display device is configured to transform the second video signal into the first video signal suitable for the display; and Wherein the display is configured to display an image according to the first video signal.

9. The display apparatus of claim 1, wherein the display apparatus is further configured to synchronize changes in the ambient light compensation with scene changes of the second video signal.

10. A display device, comprising: a projector configured to project an image in response to a first video signal, the image comprising pixels of three or more primary colors; as well as an ambient light sensor configured to generate a signal indicative of an intensity of ambient light, the ambient light sensor being positioned to register light emitted by the projector in addition to the ambient light; The display device is configured to: receiving a second video signal; and generating the first video signal by applying ambient light compensation to the second video signal in dependence on the signal indicative of the intensity of the ambient light; wherein the ambient light sensor is less sensitive to light of wavelengths corresponding to the primary colors of the projector than to light of other wavelengths of visible light; and wherein the display device is further configured to determine a measurement bias caused by the ambient light sensor recording the light emitted by the projector, and to account for the measurement bias when applying the ambient light compensation by: computing a linear combination of overall frame intensity levels for a sequence of video frames; and The linear combination or a second order polynomial of the linear combination is subtracted from the signal indicative of the intensity of the ambient light. The display device according to claim 10 , wherein the projector is a laser projector.

12. The display apparatus of claim 10, wherein the display apparatus is further configured to synchronize changes in the ambient light compensation with scene changes of the second video signal.

Images