Energy-conscious TV SL-HDR

Abstract

The process includes obtaining reconstructed SDR data and metadata representing a first inverse tone mapping function that enables the conversion of the reconstructed SDR data into HDR data having a dynamic range with a first peak brightness (140); using the metadata to derive a second tone mapping function that enables the conversion of the SDR data into HDR data having a dynamic range with a second peak brightness lower than the first peak brightness (141); and reconstructing the HDR data intended to be displayed on a display panel using the second tone mapping function (143), wherein the second peak brightness is based on an average picture level representing the percentage of the display panel illuminated, or a comparison between the measured energy consumption of the display panel when displaying HDR data and an energy consumption target.

Description

[Technical Field]

[0001] At least one of the embodiments described herein generally relates to the field of high dynamic range (HDR) video displays, and more particularly to methods and devices for controlling the energy consumed to display HDR video. [Background technology]

[0002] Recent advances in display technology have made it possible to expand the dynamic range of color, brightness, and contrast of displayed images. In this specification, the term "image" refers to image content, which may be, for example, video, still images, or still images.

[0003] High Dynamic Range (HDR) video refers to video with a wider dynamic range than standard dynamic range video (SDR). HDR video-based applications involve capture, generation, content / encoding, and display. HDR capture and display devices enable brighter whites and deeper blacks. To accommodate this, HDR encoding standards use at least 10 bits of dynamic range (compared to the 8-bit (non-professional) and 10-bit (professional) dynamic range of SDR video) to enable higher maximum brightness and maintain accuracy across this extended range.

[0004] HDR technology offers a better viewing experience (or Quality of Experience, QoE) for video content, but consumes significantly more energy than SDR. In fact, HDR video displays consume up to twice as much energy as SDR video displays. The current trend in many areas is to reduce energy consumption, and overcoming the aforementioned drawbacks is desirable.

[0005] It is particularly desirable to propose solutions that allow for the control or reduction of energy consumed by HDR video displays, while maintaining, as much as possible, the quality of engineering (QoE) improvements provided by HDR technology and the artistic intent of content creators. [Overview of the project]

[0006] In the first aspect, one or more embodiments of this specification provide a method, and the method is Obtaining reconstructed SDR data and metadata representing a first inverse tone mapping function that enables the conversion of the reconstructed SDR data into HDR data with a dynamic range having a first peak brightness, Using metadata, we derive a second tone mapping function that enables the conversion of SDR data to HDR data with a dynamic range having a second peak brightness lower than the first peak brightness, This includes reconstructing HDR data intended to be displayed on a display panel using a second tone mapping function, The second peak brightness is based on the average picture level, which represents the percentage of the display panel that is illuminated, or a comparison between the measured energy consumption of the display panel when displaying HDR data and the energy consumption target.

[0007] In one embodiment, the second peak brightness is further based on a modulation coefficient that depends on a user-selected profile and the average picture level.

[0008] In one embodiment, the second peak brightness ensures the deactivation of an automatic brightness limiter process implemented by the display panel, which limits the illumination of the display panel in accordance with the maximum power consumption of the display panel.

[0009] In one embodiment, in response to the second peak brightness being based on a comparison between the measured energy consumption of the display panel when displaying HDR data and an energy consumption target, a recursive process is applied on an image basis until the energy consumption target is met when displaying HDR data, the recursive process includes applying an adaptation coefficient to the current image of the HDR data and the second peak brightness used for the previous image of the HDR data to obtain a second peak brightness used for the current image of the HDR data.

[0010] In one embodiment, the adaptation coefficient is either fixed or a function of the ratio between the energy consumption target and the measured energy consumption of the display panel.

[0011] In one embodiment, a recursive process is applied in response to the ratio between the energy consumption target and the measured energy consumption of the display panel satisfying a condition.

[0012] In one embodiment, the proportional-derivative controller calculates an adaptation coefficient using the error between the energy consumption target and the measured energy consumption of the display panel.

[0013] In a second aspect, one or more embodiments of this specification provide a device comprising an electronic circuit, Obtain the reconstructed SDR data and metadata representing a first inverse tone mapping function that enables the conversion of the reconstructed SDR data into HDR data with a dynamic range having a first peak brightness. Using metadata, we derive a second tone mapping function that enables the conversion of SDR data to HDR data with a dynamic range having a second peak brightness lower than the first peak brightness. It is configured to reconstruct HDR data intended to be displayed on the display panel using a second tone mapping function. The second peak brightness is based on the average picture level, which represents the percentage of the display panel that is illuminated, or a comparison between the measured energy consumption of the display panel when displaying HDR data and the energy consumption target.

[0014] In one embodiment, the second peak brightness is further based on a modulation coefficient that depends on a user-selected profile and the average picture level.

[0015] In one embodiment, the second peak brightness ensures the deactivation of an automatic brightness limiter process implemented by the display panel, which limits the illumination of the display panel in accordance with the maximum power consumption of the display panel.

[0016] In one embodiment, in response that the second peak brightness is based on a comparison between the measured energy consumption of the display panel when displaying HDR data and an energy consumption target, the electronics are further configured to apply a recursive process on an image basis until the energy consumption target is met when displaying HDR data, the recursive process includes applying an adaptation coefficient to the current image of the HDR data and the second peak brightness used for the previous image of the HDR data to obtain a second peak brightness used for the current image of the HDR data.

[0017] In one embodiment, the adaptation coefficient is either fixed or a function of the ratio between the energy consumption target and the measured energy consumption of the display panel.

[0018] In one embodiment, a recursive process is applied in response to the ratio between the energy consumption target and the measured energy consumption of the display panel satisfying a condition.

[0019] In one embodiment, the proportional-derivative controller calculates an adaptation coefficient using the error between the energy consumption target and the measured energy consumption of the display panel.

[0020] In a third aspect, one or more of the embodiments herein provides a non-transitory information storage medium that stores program code instructions for implementing the method according to the first aspect.

[0021] In a fourth aspect, one or more of the embodiments herein provides a computer program comprising program code instructions for implementing the method according to the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] [Figure 1] Schematically shows an example of a scenario in which various embodiments are implemented. [Figure 2A] Shows a tone mapping curve. [Figure 2B] Shows an inverse tone mapping curve without display adaptation. [Figure 2C] Shows an inverse tone mapping curve with display adaptation. [Figure 3] Shows several energy consumption values for different types of scenes, namely bright, medium, and dim scene luminance. [Figure 4A] Schematically shows an example of a hardware architecture of a processing module that can implement various aspects and embodiments. [Figure 4B] Shows a block diagram of an example of a first system in which various aspects and embodiments are implemented. [Figure 4C] Shows a block diagram of an example of a second system in which various aspects and embodiments are implemented. [Figure 5] Shows a post-processing process that enables control of energy consumed by a display device when displaying HDR video. [Figure 6] Shows four sigmoid functions. DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

[0023] Even though HDR-capable display devices have recently emerged, some of them have limited HDR capabilities. For example, an HDR-enabled display may have a brightness capability lower than the brightness of the HDR signal defined by the content creator that the display must display. For instance, if the reconstructed HDR signal has a peak brightness of "1000" nits, the display may only be able to render up to "500" nits.

[0024] Therefore, it is necessary to adapt the reconstructed signal to the capacity of the display device. The solution to this adaptation is to clip the reconstructed signal to a value within the range acceptable by the display device before displaying it. However, this solution falls far short of preserving the artistic intent of the content creator and the QoE acceptable by the HDR signal.

[0025] When discussing images, artistic intent often depends on the tones—that is, how shadows, midtones, and highlights are distributed within the scene. This is part of the color grading that artists (i.e., content creators) must do to convey a desired emotion and / or define the visual characteristics of their content.

[0026] Tone zones can be defined as follows: • Shadows: These correspond to the lowest part of the color distribution of the content being considered (e.g., represented by a histogram of the image's luminance values). • Midtone: Corresponds to the central part of the color distribution of the content being considered. • Highlights: Corresponds to the highest color distribution of the content being considered.

[0027] In addition to these three tone zones, it is common to define blacks as pixels with the lowest sample value found in shadows, while whites correspond to pixels with the brightest sample value found in highlights.

[0028] Clearly, defining these black and white points is crucial when color grading is performed by artists. An increase in black points results in scenes where areas darker than the black points are clipped. Similarly, a decrease in white points results in scenes where areas brighter than the white points are clipped. Clipping highlights can result in the loss of valuable highlight detail.

[0029] A solution more suited to "artistic intent" based on display adaptation has been proposed. Display adaptation enables achieving the best QoE while maintaining the artistic intent of the HDR signal and adapting the reconstructed HDR signal to the brightness capabilities of the display device. For example, display adaptation adjusts the tone to preserve highlights that cannot be clipped.

[0030] By definition, QoE (Quality of Energy) increases when display adaptation adapts a reconfigured HDR signal to the brightness capabilities of the display device. However, this still comes with high energy consumption. To enable a trade-off between energy consumption and QoE, the various embodiments described below propose considering energy consumption in display adaptation.

[0031] Figure 1 schematically shows an example of a situation in which various embodiments are implemented.

[0032] In Figure 1, the source device 10, such as a camera or streaming system that provides video content, generates the video content. The source device 10 is, for example, an SDR or HDR camera that generates SDR or HDR video content, respectively.

[0033] The video content is then provided to the preprocessing module 11. The preprocessing module 11, for example, conforms the content to the SL-HDRx standard, which is SL-HDR1. Therefore, if the video content is HDR video, the preprocessing module applies tone mapping (TM) to the HDR video to generate SDR video and generates SL-HDR1 metadata. If the video content is SDR video, the preprocessing module first estimates the HDR video and then applies TM to the estimated HDR video to generate SL-HDR1 metadata. HDR video generally has a peak in brightness called the master display peak brightness, which corresponds to the peak brightness defined by the content creator. The SL-HDR1 metadata includes information representing the inverse tone mapping function and color correction function that enable obtaining HDR video from SDR video. This metadata is dynamic and can be adapted to each image or group of images.

[0034] The SDR video and SL-HDR1 metadata are then provided to the encoding module 12. The SDR video and SL-HDR1 metadata are encoded within the bitstream (i.e., within the video data) by the encoding module 12 using a video compression format such as AVC (ISO / CEI 14496-10 / ITU-T H.264), HEVC (ISO / IEC 23008-2-MPEG-H Part 2, High Efficiency Video Coding / ITU-T H.265), VVC (ISO / IEC 23090-3-MPEG-I, Versatile Video Coding / ITU-T H.266), AV1, VP9, ​​EVC (ISO / CEI 23094-1 Essential Video Coding), or any other video compression format adapted to encode the SDR video and SL-HDR1 metadata. The output of the encoding module 12 is a bitstream (i.e., video data) representing the encoded SDR video and SL-HDR1 metadata.

[0035] The encoding module 12 then provides the video data to the decoding module 13, for example, via a network. The decoding module 13 decodes the bitstream to obtain the decoded (i.e., reconstructed) version of the SDR video and the SL-HDR1 metadata.

[0036] The reconstructed SDR video is provided directly to a display device 16 adapted to display SDR content.

[0037] The SDR video and SL-HDR1 metadata are also provided to the post-processing module 14. The post-processing module 14 applies an inverse tone mapping (ITM) step and a color correction step to the SDR video to obtain the HDR video.

[0038] Color correction involves calculating a lookup table (LUT) lutCC() from the SL-HDR1 metadata. The LUT lutCC() is then used to reconstruct the HDR chrominance signal of the HDR video.

[0039] For both constant brightness (CL) mode and non-constant brightness (NCL) mode, lutCC(Y) = f(Y).(1 / Y), and f(Y) = 1 / (R.sgf(1 / Y)), where Y is the value representing brightness. The function sgf(1 / Y) corresponds to the color correction function encoded in the SL-HDR1 metadata.

[0040] In NCL mode, f(Y) is a constant function, i.e., f(Y) = Ω, and therefore lutCC(Y) = Ω.(1 / Y).

[0041] In CL mode, f(Y) is not a constant function.

[0042] The ITM step involves deriving the LUT lutMapY() from the SL-HDR1 metadata. Then, using the LUT lutMapY(), inverse tone mapping is performed on the luminance signal of the SDR video to reconstruct the HDR luminance signal of the HDR video.

[0043] If display adaptation is required (i.e., in response to the HDR display 15 having a display peak brightness lower than the master display peak brightness (hereinafter referred to as the target display peak brightness)), the target display peak brightness is taken into consideration during inverse tone mapping. Three cases are possible. The target display peak brightness is the same as the master display peak brightness. In this case, the ITM curve is the inverse of the TM curve applied by the preprocessing module 11. Therefore, the LUT lutMapY() is derived directly from the SL-HDR1 metadata. • The target display peak brightness is "100" nits (i.e., HDR display 15 is an SDR display). In this case, the ITM curve is identical in the linear region. That is, when the target display peak brightness is "100" nits, the brightness of the reconstructed HDR signal is equal to the brightness of the reconstructed SDR signal. Note that this latter applies to NCL mode but not to CL mode. • The target display peak brightness is between "100" nits and the master display peak brightness. The ITM curve is between the identity curve and the inverse tone mapping curve specified in the SL-HDR1 metadata.

[0044] Figure 2A shows the TM curve used by the preprocessing module 11 to generate SDR video from the original HDR video.

[0045] Figure 2B shows the ITM curve resulting from the inversion of the TM curve in Figure 2A. The sequential application of the TM curve in Figure 2A and the ITM curve in Figure 2B makes it possible (theoretically) to re-acquire the original HDR video.

[0046] Figure 2C shows multiple ITM curves derived from the TM curve in Figure 2A when the display adaptation process is applied.

[0047] An example of the process for deriving the LUT lutMapY() when the target display peak brightness is between "100" nits and the master display peak brightness is described in Appendix E of the document ETSI TS 103 433-1 V1.2.1 (High-Performance Single Layer High Dynamic Range (HDR) System for use in Consumer Electronics devices; Part 1: Directly Standard Dynamic Range (SDR) Compatible HDR System (SL-HDR1)) (hereinafter simply referred to as SL-HDR1). Essentially, this process involves applying the process described in Figure 4 of Section 7.2.3.1.2 of the reference SL-HDR1 to compute a first LUT lutMapY'() representing the ITM curve without display adaptation (i.e., the first LUT lutMapY'() enables the conversion of the reconstructed SDR signal to an HDR signal with a luminance peak equal to the master display peak luminance), and then computing a second LUT lutMapY''() that enables mapping the HDR signal with a luminance peak equal to the master display peak luminance to an HDR signal with a peak luminance equal to the target display peak luminance. Thus, LUT lutMapY() is a combination of the first LUT lutMapY'() and the second LUT lutMapY''(). In Appendix E of the reference SL-HDR1, the target display peak luminance is referred to as the maximum luminance of the presentation display, and the variable L pdisp It is represented as follows.

[0048] One advantage of the display adaptation process described in Appendix E of document SL-HDR1 is that it preserves as much of the artistic intent defined by the content creator as possible.

[0049] Once reconfigured, the HDR video is delivered to the HDR display 15.

[0050] It should be noted that the post-processing module 14 can be integrated into the HDR display 15.

[0051] Figure 3 shows several energy consumption values ​​for different types of HDR scenes, namely bright, medium, and dim scene brightness. The energy consumption values ​​are a function of peak brightness, expressed in nits. In this example, the OLED (organic light-emitting diode) screen used in the test has a target display peak brightness of "1000" nits. Therefore, its energy consumption is highest for this value because the screen's full capacity is used. Reducing the peak brightness of the displayed content makes it possible to reduce energy consumption. Interestingly, the amount of reduction depends heavily on the scene brightness.

[0052] Figure 4A schematically shows an example of the hardware architecture of a processing module 40 used, for example, in a pre-processing module 11 or a post-processing module 14. The processing module 40 is connected by a communication bus 405 and includes, in non-limiting examples, one or more microprocessors, general-purpose computers, dedicated computers, and processors based on multi-core architectures, a processor or CPU (Central Processing Unit) 400, a random-access memory (RAM) 401, a read-only memory (ROM) 402, and a storage unit 403 which may include non-volatile memory and / or volatile memory, wherein the non-volatile memory and / or volatile memory may include electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random-access memory (RAM), dynamic random-access memory (DRAM), static random-access memory (SRAM), flash memory, magnetic disk drives and / or optical disk drives, or SD (Secure Digital) card readers and / or hard disk drives. The system comprises a storage unit 403, including but not limited to a storage medium reader such as a drive, HDD, and / or a network-accessible storage device, and at least one communication interface 404 for exchanging data with other modules, devices, systems, or equipment. The communication interface 404 may include, but is not limited to, a transceiver configured to transmit and receive data over a communication network 41. The communication interface 404 may include, but is not limited to, a modem or a network card.

[0053] For example, the communication interface 404 enables, for instance, the processing module 40 to receive HDR or SDR data and output HDR or SDR data along with SL-HDR1 metadata.

[0054] The processor 400 can execute instructions loaded into the RAM 401 from the ROM 402, from external memory (not shown), from a storage medium, or from a communication network. When the processing module 40 is powered on, the processor 400 can read instructions from the RAM 401 and execute those instructions. If the processing module 40 is included in the pre-processing module 11, these instructions form a computer program that results in the processor 400 implementing the TM process, for example (if the source module generates HDR video). If the processing module 40 is included in the post-processing module 14, these instructions form a computer program that results in the processor 400 implementing the ITM process, for example, including display adaptation according to the embodiments described below in this disclosure.

[0055] All or part of the algorithms and steps of the above process may be implemented in software form by executing a set of instructions by a programmable machine such as a DSP (Digital Signal Processor) or microcontroller, or in hardware form by a machine or dedicated component such as an FPGA (Field Programmable Gate Array) or ASIC (Application-Specific Integrated Circuit). Microprocessors, DSPs, FPGAs, and ASICs are considered electronic circuits.

[0056] Figure 4C shows a block diagram of an example of a system A that implements a post-processing module in which various forms and embodiments are carried out.

[0057] System A may be embodied as a device comprising various components or modules and configured to produce HDR-displayable video. Examples of such systems include, but are not limited to, various electronic systems such as personal computers, laptop computers, smartphones, tablets, TVs, or set-top boxes. The components of System A may be embodied individually or in combination as a single integrated circuit (IC), multiple ICs, and / or individual components. For example, in at least one embodiment, System A comprises a processing module 40 that implements a post-processing module 14. In various embodiments, System A may be communicably coupled to one or more other systems or other electronic devices, for example, via a communication bus or through dedicated input and / or output ports.

[0058] Inputs to the processing module 40 may be provided through various input modules, as shown in block 42. Such input modules include, but are not limited to, (i) a radio frequency (RF) module for receiving RF signals transmitted over the air by a broadcasting station, (ii) a component (COMP) input module (or a set of COMP input modules), (iii) a universal serial bus (USB) input module, and / or (iv) a high-definition multimedia interface (HDMI) input module. Other embodiments not shown in Figure 4C include composite video.

[0059] In various embodiments, the input module of block 42 has associated input processing elements as known in the art. For example, an RF module may be associated with elements suitable for (i) selecting a desired frequency (also called selecting a signal or band-limiting a signal to a certain frequency band), (ii) down-converting the selected signal, (iii) again band-limiting to a narrower frequency band so as to select a signal frequency band that may (for example) be called a channel in certain embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. RF modules of various embodiments include one or more elements that perform these functions, e.g., frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF portion may include wavelength tuners that perform various of these functions, e.g., down-converting the received signal to a lower frequency (e.g., an intermediate frequency or a frequency close to the baseband) or to the baseband. In various embodiments, the order of these (and other) elements may be rearranged, some of these elements may be removed, and / or other elements performing similar or different functions may be added. Adding elements may include inserting elements between existing elements, such as inserting an amplifier and an analog-to-digital converter. In various embodiments, the RF module includes an antenna.

[0060] Additionally, the USB and / or HDMI modules may include their respective interface processors for connecting System A to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing, such as Reed-Solomon error correction, may be implemented, for example, in a separate input processing IC or within the processing module 40, as needed. Similarly, aspects of USB or HDMI interface processing may be implemented in a separate interface IC or within the processing module 40, as needed. The demodulated, error-corrected, and demultiplexed streams are provided to the processing module 40.

[0061] Various elements of System A can be housed within an integrated housing. Within the integrated housing, the various elements are interconnected, and data can be transmitted between them using an internal bus known in the art, such as an Inter-IC (I2C) bus, wiring, and a printed circuit board, which is a suitable connection arrangement. For example, in System A, the processing module 40 is interconnected with the other elements of System A by bus 405.

[0062] The communication interface 404 of the processing module 40 enables system A to communicate over the communication network 41. The communication network 41 can be implemented, for example, within a wired and / or wireless medium.

[0063] In various embodiments, data is streamed to system A or otherwise provided using a wireless network such as a Wi-Fi network, e.g., IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). In these embodiments, the Wi-Fi signal is received via a communication network 41 and a communication interface 404 adapted for Wi-Fi communication. In these embodiments, the communication network 41 is typically connected to an access point or router that provides access to an external network, including the Internet, enabling streaming applications and other over-the-top communications. Still other embodiments provide the streamed data to system A using the RF connection of the input block 42. As described above, various embodiments provide data in a non-streaming manner, for example, if system A is a smartphone or tablet. Additionally, various embodiments use wireless networks other than Wi-Fi, e.g., a cellular network or a Bluetooth network.

[0064] System A can provide output signals to various output devices using the communication network 41 or bus 405. For example, System A can provide reconstructed HDR video.

[0065] System A can provide output signals to various output devices, including an HDR display 15, a speaker 46, and other peripheral devices 47. In various embodiments, the HDR display 15 includes, for example, one or more of a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The HDR display 15 may be for a television, tablet, laptop, mobile phone, or other device. The HDR display 15 may also be integrated with other components (for example, in a smartphone) or separate (for example, an external monitor for a laptop). In various examples of embodiments, the other peripheral devices 47 include one or more of a standalone digital video disc (or digital multi-purpose disc) (both terms DVR), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 47 that provide functions based on the output of System A. For example, a disc player performs the function of playing back the output of System A.

[0066] In various embodiments, control signals are communicated between System A and the HDR display 15, speaker 46, or other peripheral devices 47 using signal transmission such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable inter-device control with or without user intervention. Output devices may be communicably coupled to System A via dedicated connections through their respective interfaces 43, 44, and 45. Alternatively, output devices may be connected to System A using a communication network 41 via a communication interface 404. The HDR display 15 and speaker 46 may be integrated into a single unit with other components of System A in an electronic device such as a television. In various embodiments, the display interface 43 includes a display driver, such as a timing controller (TCon) chip.

[0067] The HDR display 15 and speaker 46 may, alternatively, be separate from one or more of the other components, for example, if the RF module of block 42 is part of a separate set-top box. In various embodiments where the HDR display 15 and speaker 46 are external components, the output signal may be provided, for example, via a dedicated output connection including an HDMI port, a USB port, or a COMP output unit.

[0068] Figure 4B shows a block diagram of an example of a system B adapted to implement a preprocessing module 11 in which various embodiments and models are carried out.

[0069] System B may be embodied as a device including the various components and modules described above, and is configured to perform one or more of the embodiments and models described in this document.

[0070] Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, cameras, smartphones, and servers. Elements or modules of System B can be embodied individually or in combination as a single integrated circuit (IC), multiple ICs, and / or individual components. For example, in at least one embodiment, System B comprises a processing module 40 that implements the preprocessing module 11. In various embodiments, System B is communicably coupled to one or more other systems or other electronic devices, for example, via a communication bus or through dedicated input and / or output ports.

[0071] The input to the processing module 40 can be provided through various input modules, as shown in block 42, which has already been described in relation to Figure 4C.

[0072] Various elements of System B can be housed within an integrated housing. Within the integrated housing, the various elements are interconnected, and data can be transmitted between them using a suitable connection arrangement, such as an internal bus known in the art, including an Inter-IC (I2C) bus, wiring, and a printed circuit board. For example, in System B, the processing module 40 is interconnected with the other elements of System B by bus 405.

[0073] The communication interface 404 of the processing module 40 enables system B to communicate over the communication network 41. The communication network 71 can be implemented, for example, within a wired and / or wireless medium.

[0074] In various embodiments, the data is streamed to System B or otherwise provided using a wireless network such as a Wi-Fi network, e.g., IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). In these embodiments, the Wi-Fi signal is received via a communication network 41 and a communication interface 404 adapted for Wi-Fi communication. In these embodiments, the communication network 41 is typically connected to an access point or router that provides access to an external network, including the Internet, enabling streaming applications and other over-the-top communications. Yet another embodiment provides the streamed data to System B using an RF connection in the input block 42. As described above, various embodiments provide data in a non-streaming manner.

[0075] When a diagram is presented as a flow chart, it should be understood that it also provides a block diagram of the corresponding device. Similarly, when a diagram is presented as a block diagram, it should be understood that it also provides a flow chart of the corresponding method / process.

[0076] The implementation forms and embodiments described herein can be implemented, for example, in methods or processes, apparatus, software programs, data streams, or signals. Even if considered only in the context of a single form of implementation (e.g., considered only as a method), the implementation forms of the considered features can also be implemented in other forms (e.g., apparatus or programs). Apparatus may be implemented, for example, in appropriate hardware, software, and firmware. These methods may be implemented, for example, in a processor, which refers to processing devices in general, including, for example, computers, microprocessors, integrated circuits, or programmable logic devices. Processors also include communication devices, such as computers, mobile phones, portable / personal digital assistants ("PDAs"), smartphones, tablets, and other devices that facilitate the communication of information between end users.

[0077] The terms "one embodiment" or "one embodiment," or "one implementation" or "one implementation," and any other variations thereof, mean that the specific features, structures, characteristics, etc., described in relation to the embodiments are included in at least one embodiment. Therefore, the appearance of the phrases "in one embodiment" or "in one embodiment," or "in one implementation" or "in one implementation," and any other variations, found in various places throughout this application, do not necessarily all refer to the same embodiment.

[0078] Additionally, this application may refer to "determining" various types of information. Determining information may include, for example, estimating information, calculating information, predicting information, retrieving information from memory, or obtaining information from, for example, another device, module, or user.

[0079] Furthermore, this application may also refer to “accessing” various types of information. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0080] Additionally, this application may refer to “receiving” various types of information. Receiving is intended to be a broad term, similar to “accessing.” Receiving information may include, for example, accessing information or retrieving information (for example, from memory). Furthermore, “receiving” typically accompanies, in some way, operations such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0081] For example, in the cases of "A / B", "A and / or B", "at least one of A and B", and "one or more of A and B", please understand that the use of any of the following " / ", "and / or", "at least one of", and "one or more of" is intended to cover the selection of only the first option (A), only the second option (B), or both options (A and B). As a further example, in the cases of "A, B, and / or C," "at least one of A, B, and C," and "one or more of A, B, and C," such phrasing is intended to cover the selection of only the first option (A), or only the second option (B), or only the third option (C), or only the first and second options (A and B), or only the first and third options (A and C), or only the second and third options (B and C), or the selection of all three options (A, B, and C). This may be extended to the number of items listed, as will be obvious to those skilled in the art.

[0082] As will be apparent to those skilled in the art, implementations or embodiments can generate various signals that are formatted to carry information that can be stored or transmitted. This information may include, for example, instructions for performing a method, or data generated by one of the implementations or embodiments described. For example, a signal may be formatted to carry an SDR image or video sequence and SL-HDRx metadata of the embodiment described. Such a signal may be formatted, for example, as an electromagnetic wave (using, for example, the radio frequency portion of the spectrum) or as a baseband signal. Formatting may include, for example, encoding an SDR image or video sequence using SL-HDR1 metadata in an encoded stream (i.e., in video data) and modulating a carrier using the encoded stream. The information carried by the signal may be, for example, analog or digital information. The signal may be transmitted over various different wired or wireless links, as is known. The signal may be stored on a processor-readable medium.

[0083] As described above, display adaptation allows for the display of HDR video that conforms to a target display peak brightness corresponding to the peak brightness supported by the display device displaying the HDR video, while preserving the artistic intent of the content creator. However, in this case, the energy consumed by the display device will be equivalent to the maximum energy that the display device can consume to display HDR video.

[0084] A common objective of the various embodiments described below is to reduce the energy consumption of the display device when displaying HDR video with respect to this maximum energy. To this end, the display adaptation process uses a peak brightness lower than the target display peak brightness, hereafter referred to as the energy consumption-based peak brightness, instead of the target display peak brightness. Using an energy consumption-based peak brightness lower than the target display peak brightness has a direct impact on reducing energy consumption.

[0085] Before delving into detailed descriptions of various embodiments, let's introduce some concepts and terminology.

[0086] The Auto Brightness Limiter (ABL) is an inherent limitation of OLED panels. The energy consumption of these panels depends on the displayed content. For a pure white image, all pixels must be lit, consuming maximum power depending on the brightness setting. Conversely, for a completely black image, each pixel is off and consumes no energy. However, for every pixel, the OLED panel has maximum power consumption. This means that a completely white image is limited by the OLED panel's maximum power consumption. This is why a completely white image on an OLED panel is always less bright than an isolated light spot. In fact, an isolated light spot is limited by the screen's maximum power consumption.

[0087] On an OLED panel, the proportion of the OLED panel that is illuminated compared to an all-white OLED panel is known as the Average Picture Level (APL). For example, an OLED panel displaying a small window of white pixels will have a low APL. This brightness level decreases as the size of the white pixel window increases. As can be seen from the figure, this phenomenon is related to ABL.

[0088] An OLED panel has a maximum brightness at which it can reach and maintain an image with perfect white (100% APL). When APL is reduced, power is reallocated to brighter areas, and therefore the peak brightness in these areas can be increased.

[0089] As can be seen from the diagram, ABL is uncontrolled and cannot guarantee the preservation of artistic intent.

[0090] Note that the terms peak brightness and peak luminance are used interchangeably in this specification and represent the same concept.

[0091] Figure 5 shows a post-processing process that allows control over the energy consumed by the display device when displaying HDR video.

[0092] The post-processing process shown in Figure 5 is executed by, for example, the processing module 40 of system A when it implements the post-processing module 14. Assume that system A receives encoded video data from system B. The decoding module 13 of system A then decodes the encoded video data and generates reconstructed SDR data and SL-HDR1 metadata. The post-processing module 14 then retrieves the reconstructed SDR data and SL-HDR1 metadata from the decoding module 13. The process in Figure 5 is executed before the ABL process performed by the processing module 40 of the HDR display 15.

[0093] Please note that the following description uses SL-HDR1 as an example. However, the various embodiments described below are applicable to any other HDR distribution technology that uses dynamic metadata, such as SL-HDR2, SL-HDR3, Dolby Vision, and HDR10+. Furthermore, the various embodiments use a modified version of the display adaptation process described in Appendix E of SL-HDR1. However, other display adaptation processes may be used, and the display adaptation process described in Appendix E of SL-HDR1 is merely one example of such a process.

[0094] In step 140, the processing module 40 of the post-processing module 14 obtains the reconstructed SDR data and metadata representing a first inverse tone mapping function that enables the conversion of the reconstructed SDR data into HDR data with a dynamic range having a first peak luminance. In the example in Figure 5, the metadata is the SL-HDR1 metadata generated by the pre-processing module 11. The first peak luminance is the master display peak luminance corresponding to the peak luminance defined by the content creator.

[0095] In step 141, the processing module 40 uses metadata to derive a second tone mapping function that enables the conversion of SDR data into HDR data with a dynamic range having a second peak brightness lower than the first peak brightness. The second peak brightness is typically the energy consumption-based peak brightness described above. To do this, the processing module 40 uses a variable L, for example, that represents the target display peak brightness (i.e., the maximum brightness of the presentation display). pdisp The variable L represents the peak brightness based on energy consumption. energyConso By replacing it, the display adaptation process described in Appendix E of reference SL-HDR1 is applied. The calculation of the first LUT lutMapY'() representing the ITM curve without display adaptation is not modified. Variable L pdisp Variable L energyConso Replacing it with the second LUT lutMapY'' energyConso This makes it possible to obtain () and map an HDR signal with a luminance peak equal to the master display peak luminance to an HDR signal with a peak luminance equal to the energy consumption-based peak luminance. LUT lutMapY() is a first LUT lutMapY'() and a second LUT lutMapY'' energyConso It is a combination with ().

[0096] In step 143, the processing module 40 reconstructs the HDR data using a second tone mapping function (i.e., LUT lutMapY()), and the HDR data is intended to be displayed on the HDR display 15.

[0097] Note that the energy-based peak brightness is lower than the target display peak brightness, which represents the peak brightness supported by the HDR display 15.

[0098] Optionally, if the SDR data includes a chroma component, the SL-HDR1 metadata includes information representing a color correction function, and the processing module 40 derives the color correction function in step 142, as described in the SL-HDR1 document. In step 143, the processing module 40 applies the color correction function to the SDR data to obtain the chrominance component of the HDR data.

[0099] In the first embodiment of step 141, display adaptation is driven by APL. More precisely, the energy consumption-based peak luminance value is determined based on APL. In one example, a LUT is used for this purpose. Table TAB1 discloses an example of a LUT in which the APL value Apl (in the first column from left to right) is mapped to the energy consumption-based peak luminance value E (in the second column from left to right). This LUT determines the value of the energy consumption-based peak luminance E from the APL value Apl. The first embodiment makes it possible to limit the activation of ABL, thereby making it possible to (i) reduce energy consumption and (ii) maintain artistic intent. In a modified example, the energy consumption-based peak luminance value in the LUT is defined to ensure that ABL is never activated.

[0100] [Table 1]

[0101] In one embodiment, the APL is estimated from displayed content. Here, it is assumed that the content is composed of YUV pictures, and each pixel of a YUV picture is represented by a luminance value Y and two chrominance values U and V. The APL value corresponding to an image is estimated based on the number of pixels of the image having a luminance component value Y exceeding a threshold TH. For example, when a YUV component is encoded with "8" bits, TH is equal to "240". The APL is considered to be equal to the proportion of pixels of the image having a luminance component value Y exceeding the threshold TH. When estimating the APL value Apl for each image, the process of FIG. 5 is applied to each image. In a variation, the APL is estimated for a group of consecutive images, and the APL value Apl is an average of proportions of pixels having a luminance component value Y exceeding the threshold TH determined for each image in the group of images. In this case, a single energy consumption-based peak luminance value E is determined for the group of images.

[0102] In a variation of the first embodiment, a user can select an energy consumption-based peak luminance and a user profile to further reduce energy consumption. Four user profiles are defined in the four rightmost columns of table TAB1. Each profile defines a modulation coefficient MOD for each APL value Apl. The modulation coefficient is applied to the energy consumption-based peak luminance E obtained from the APL value Apl to obtain a modulated energy consumption-based peak luminance E used as a second peak luminance in step 141 mod . E mod = E × MOD Equation 1

[0103] For example, when the selected user profile is low and the APL value is 50: E mod = 243 × 0.9

[0104] In a variation, the energy consumption-based peak luminance values E of the LUT and the modulation coefficients MOD are configured such that any obtained modulated energy consumption-based peak luminance value E modHowever, it is defined in a way that ensures ABL is never activated.

[0105] In a second embodiment, the energy consumption target is given by the user, for example, "75" watts per hour. The energy consumption-based peak brightness is then adjusted to satisfy the energy consumption target, based on a comparison of the measured energy consumption of the HDR display 15 when displaying HDR data with the energy consumption target. In this embodiment, it is conceivable that the HDR display 15 can monitor its energy consumption and record the average of this energy consumption. Instead of a simple average, the HDR display 15 can calculate a weighted average that more strongly emphasizes the most recent image.

[0106] To match the energy consumption target, the energy consumption-based peak brightness is recursively decreased over time. The process in Figure 5 is applied, for example, on an image basis. For each image, a simple decrease in energy consumption based on peak brightness is applied. When the energy consumption target is reached, the decrease stops. Obviously, if the gain is higher than the energy consumption target, a recursive increase can be performed.

[0107] In one example of the implementation, the energy consumption-based peak brightness E(t) of image t is calculated as follows, depending on the energy consumption-based peak brightness E(t-1) of image t-1.

[0108]

number

[0109] The function f is a function that allows control over variations in peak brightness based on energy consumption. In a simple embodiment, the function f is the identity function. In a second example, the function f is a sigmoid function centered at "1".

[0110]

number

[0111] The parameter 'a' allows for tuning the behavior of the sigmoid function around '1'. Figure 6 shows 'four' sigmoid functions with a = 5, 10, 15, and 20.

[0112] If the ratio is less than "1", the energy consumption-based peak brightness decreases. If the ratio is greater than "1", the energy consumption-based peak brightness increases (and is limited to the target display peak brightness (i.e., the maximum brightness of the HDR display 15)).

[0113] In a modified version of the second embodiment, the function

[0114]

number

[0115] In a modified version of the second embodiment, the process of reducing (or increasing) the energy consumption-based peak luminance value is activated only if the energy consumption target has not been reached. For example,

[0116]

number

[0117] Another variation of the second embodiment involves using a PID (Proportional-Integral-Derivative) controller. A PID controller is a control loop mechanism that uses feedback and is widely used in industrial control systems and various other applications requiring continuously modulated control. A PID controller continuously calculates an error value as the difference between a desired setpoint (SP) and a measured process variable (PV), and applies corrections based on proportional, integral, and derivative terms (denoted by P, I, and D, respectively).

[0118] In the context of the second embodiment, the desired setpoint SP is a user-defined energy consumption target. The measured process variable PV is the energy consumption measured by the HDR display 15. By measuring the energy consumption PV and subtracting the energy consumption from the energy consumption target SP, an error e is found, and from this error the controller calculates a scalar value within the range [0,1]. This scalar value is used, for example, to modulate the energy consumption-based peak luminance value in equation 4.

[0119] In a modification of the second embodiment, it is assumed that it is known when the ABL is activated in response to the APL. Furthermore, a LUT (such as the LUT in Table TAB1) allows a correlation between an energy consumption-based peak luminance value E(t) and the APL value. Thanks to the link between the energy consumption-based peak luminance value E(t) and the APL value, and the knowledge of the conditions for APL to activate the ABL, it is possible to know whether the determined energy consumption-based peak luminance value E(t) allows the ABL to be deactivated. In this modification of the second embodiment, if the determined energy consumption-based peak luminance value E(t) does not allow the ABL to be deactivated, the energy consumption-based peak luminance value E(t) is further reduced to a value that guarantees the deactivation of the ABL.

[0120] Up to this point, various embodiments have been described in the context of OLED panels. However, all embodiments also conform to other display technologies such as LCD (liquid crystal display) displays, QLED (quantum dot light-emitting diode) displays, or microLED displays, with or without local dimming. In that case, APL is the ratio of the illuminated display panel (LCD, QLED, microLED, etc.) compared to the all-white panel.

[0121] Several embodiments have been described above. Features of these embodiments can be provided individually or in any combination. Furthermore, embodiments may include, individually or in any combination, one or more of the following features, devices, or aspects across various categories and types of claims. A television, set-top box, mobile phone, tablet, personal computer, or other electronic device that performs at least one of the embodiments described and displays the resulting picture (for example, using a monitor, screen, or other type of display). A television, set-top box, mobile phone, tablet, personal computer, or other electronic device that tunes channels (e.g., using a tuner) to receive a signal containing encoded SDR video and metadata, and performs at least one of the embodiments described. A television, set-top box, mobile phone, tablet, or other electronic device that wirelessly receives a signal containing encoded SDR video and metadata (for example, using an antenna) and performs at least one of the embodiments described. A server, camera, mobile phone, tablet, personal computer, or other electronic device that tunes its channels (for example, using a tuner) to transmit a signal including SDR video and metadata, and performs at least one of the embodiments described. A server, camera, mobile phone, tablet, personal computer, or other electronic device that wirelessly transmits a signal including SDR video and metadata (for example, using an antenna) and performs at least one of the embodiments described.

Claims

1. It is a method, (140) Obtaining reconstructed SDR data and metadata representing a first inverse tone mapping function that enables the conversion of the reconstructed SDR data into HDR data having a dynamic range with a first peak brightness, Using the metadata, a second tone mapping function is derived that enables the conversion of the SDR data into HDR data having a dynamic range with a second peak brightness lower than the first peak brightness (141), The method includes (143) reconstructing the HDR data intended to be displayed on the display panel using the second tone mapping function, The method wherein the second peak brightness is based on an average picture level representing the proportion of the display panel that is illuminated, or on a comparison between the measured energy consumption of the display panel when displaying the HDR data and an energy consumption target.

2. The method according to claim 1, wherein the second peak brightness is further based on a modulation coefficient that depends on a profile selected by the user and the average picture level.

3. The method according to claim 1 or 2, wherein the second peak brightness ensures the deactivation of an automatic brightness limiter process performed by the display panel, which limits the illumination of the display panel in accordance with the maximum power consumption of the display panel.

4. The method according to claim 1, in response that the second peak brightness is based on a comparison between the measured energy consumption of the display panel when displaying the HDR data and an energy consumption target, a recursive process is applied on an image basis until the energy consumption target is met when displaying the HDR data, the recursive process comprising applying an adaptation coefficient to the current image of the HDR data to the second peak brightness used for the previous image of the HDR data to obtain the second peak brightness used for the current image of the HDR data.

5. The method according to claim 4, wherein the adaptation coefficient is fixed or is a function of the ratio between the energy consumption target and the measured energy consumption of the display panel.

6. The method according to claim 4 or 5, wherein the recursive process is applied in response to the ratio between the energy consumption target and the measured energy consumption of the display panel satisfying a condition.

7. The method according to claim 4, wherein the proportional-derivative controller calculates the adaptation coefficient using the error between the energy consumption target and the measured energy consumption of the display panel.

8. A device comprising an electronic circuit, wherein the electronic circuit is The reconstructed SDR data and metadata representing a first inverse tone mapping function that enables the conversion of the reconstructed SDR data into HDR data having a dynamic range with a first peak brightness are obtained (140), Using the metadata, a second tone mapping function is derived that enables the conversion of the SDR data into HDR data having a dynamic range with a second peak brightness lower than the first peak brightness (141). The HDR data intended to be displayed on the display panel is reconstructed using the second tone mapping function (143), The device wherein the second peak brightness is based on an average picture level representing the proportion of the display panel that is illuminated, or on a comparison between the measured energy consumption of the display panel when displaying the HDR data and an energy consumption target.

9. The device according to claim 8, wherein the second peak brightness is further based on a modulation coefficient that depends on a profile selected by the user and the average picture level.

10. The device according to claim 8 or 9, wherein the second peak brightness ensures the deactivation of an automatic brightness limiter process performed by the display panel, which limits the illumination of the display panel in accordance with the maximum power consumption of the display panel.

11. In response that the second peak brightness is based on a comparison between the measured energy consumption of the display panel when displaying the HDR data and an energy consumption target, the electronic circuit is configured to apply a recursive process on an image basis when displaying the HDR data until the energy consumption target is met, the recursive process includes applying an adaptation coefficient to the current image of the HDR data to the second peak brightness used for the previous image of the HDR data to obtain the second peak brightness used for the current image of the HDR data, according to claim 8.

12. The device according to claim 11, wherein the adaptation coefficient is fixed or is a function of the ratio between the energy consumption target and the measured energy consumption of the display panel.

13. The device according to claim 11 or 12, wherein the recursive process is applied in response to the ratio between the energy consumption target and the measured energy consumption of the display panel satisfying a condition.

14. The device according to claim 11, wherein the proportional-derivative controller calculates the adaptation coefficient using the error between the energy consumption target and the measured energy consumption of the display panel.

15. A non-temporary information storage medium for storing program code instructions for carrying out the method according to any one of claims 1 to 7.

16. A computer program comprising program code instructions for carrying out the method according to any one of claims 1 to 7.

Images