Inverse tone mapping with adaptive highlight decay

Abstract

The invention provides a method for inverse tone mapping, the method being performed by: obtaining (42) a gain function of a first inverse tone mapping function, referred to as an initial gain function; if (43) an analysis of a current image shows that at least one pixel of the current image having a luminance value at least equal to a predetermined percentage of luminance values (TH1) of the pixels of the current image has an expanded luminance value resulting from the application of the first ITM function to the current image which is higher than a target value, applying (45) to the current image a second ITM function, the second ITM function corresponding to the first inverse tone mapping function in which the gain function has been replaced by a modified gain function derived from the initial gain function in which the gain provided by the initial gain function is attenuated by an attenuation function, the attenuation function being weighted by a weighting factor controlling the intensity of the attenuation, the weighting factor depending on a statistical distribution of luminance values between the maximum luminance value in a histogram of the current image and the luminance value depending on the predetermined percentage.

Description

1. Technical Field

[0001] At least one embodiment of the present invention relates generally to the field of high dynamic range imaging, and more specifically to a method and apparatus for extending the dynamic range of a low dynamic range image or a standard dynamic range image, the method and apparatus being particularly concerned with how to automatically and adaptively attenuate bright areas in the resulting high dynamic range image to a specific target brightness. 2. Background Technology

[0002] Recent advances in display technology have begun to allow for an expanded dynamic range of color, brightness, and contrast in images to be displayed. The term "image" here refers to image content, such as video, still pictures, or images.

[0003] The technology that allows for an extended dynamic range of an image's brightness or luminance is called High Dynamic Range (HDR) imaging. Although many HDR display devices and imaging cameras capable of capturing images with an increased dynamic range have emerged, the amount of available HDR content remains very limited. Solutions are needed that allow for an extended dynamic range of existing content so that it can be efficiently displayed on HDR display devices.

[0004] To prepare conventional (referred to herein as Low Dynamic Range (LDR) or Standard Dynamic Range (SDR) content for HDR display devices, an inverse or inverse tone mapping operator (ITMO or ITM) can be employed. ITMO allows the generation of HDR images from conventional (LDR or SDR) images using algorithms that process the luminance information of pixels in an image, with the aim of recovering or recreating the appearance of the corresponding original scene. Typically, ITMO takes a conventional image as input, globally expands the luminance range of the image's colors, and then locally processes highlight areas or bright areas (i.e., light spots) to enhance the HDR appearance of the colors in the image.

[0005] To enhance bright areas in an image, it is known to create a luminance expansion map (or gain function), associating each pixel of the SDR image with an expansion value to apply to the brightness of that pixel, thereby obtaining the corresponding HDR image. However, even when applying the best possible luminance gain function (not to mention when applying a fixed luminance gain function that is not particularly suitable for the SDR image), undesirable gradation may occur in the resulting HDR image. Specifically, a wide highlight or bright area in the SDR image may cause overly bright areas in the corresponding HDR image. Therefore, some HDR display devices cannot display such HDR images correctly. In fact, since these HDR images exceed their display capacity, display devices apply more or less effective algorithms to locally or globally reduce the brightness of the HDR images. When not managed by the display device, large bright areas in the corresponding HDR image may dazzle the viewer or at least make their HDR viewing experience unpleasant. The brightness of these areas is then preferably attenuated in a controlled manner.

[0006] The aim is to overcome the above shortcomings.

[0007] Specifically, the goal is to define an inverse tone mapping method that allows bright areas to be automatically and adaptively attenuated to a specific extended target brightness. 3. Summary of the Invention

[0008] In a first aspect, one or more embodiments of the present invention provide a method for inverse tone mapping, wherein the method includes: obtaining an image, referred to as a current image; obtaining a gain function of a first inverse tone mapping function, referred to as an initial gain function; if analysis of the current image indicates that at least one pixel whose luminance value is at least equal to a luminance value depending on a predetermined percentage of the luminance value of that pixel in the current image has an extended luminance value higher than a target value due to the application of the first inverse tone mapping function to the current image, then applying a second inverse tone mapping function to the current image, the second inverse tone mapping function corresponding to the first inverse tone mapping function, wherein the gain function has been replaced by a modified gain function; the modified gain function is a function derived from the initial gain function, wherein the gain provided by the initial gain function is attenuated by an attenuation function, the attenuation function being an increasing function of luminance values ​​weighted by a weighting factor controlling the intensity of the attenuation, the weighting factor depending on the statistical distribution of luminance values ​​between the maximum luminance value and the luminance value depending on the predetermined percentage in a histogram of the current image.

[0009] In one embodiment, the method includes: browsing the histogram in descending order of brightness values, starting from the maximum brightness value; calculating an intermediate weighting factor for each browsed brightness value when the browsed brightness value is at least equal to the brightness value depending on the predetermined percentage and the result of applying the first inverse tone mapping function to the browsed brightness value is at least equal to the target extended brightness value; and setting the value of the weighting factor to a value corresponding to the maximum value of the calculated intermediate weighting factor.

[0010] In one implementation, the intermediate weighting factor depends on an intermediate extended target luminance value, which is a weighted sum between the extended target luminance value and the result of applying the first inverse tone mapping function to the browsed luminance value, each weight of the weighted sum depending on a value representing the proportion of pixels whose luminance value of the current image is higher than the browsed luminance value.

[0011] In one implementation, the value representing the proportion of pixels whose brightness value is higher than the brightness value being viewed also depends on the maximum authorized extended brightness value.

[0012] In one implementation, the method includes applying the first inverse tone mapping function to the current image when the number of pixels whose brightness values ​​are higher than the viewed brightness value is less than a predefined minimum number of pixels.

[0013] In one embodiment, the current image belongs to a video sequence and the method includes detecting scene cuts in the current video sequence, wherein the weighting factor controlling the intensity of the attenuation also depends on at least one other weighting factor controlling the intensity of the attenuation calculated for another image preceding the current image in the video sequence, and the scene cut has not yet been detected between the current image and the other image.

[0014] In one embodiment, the method includes verifying the monotonicity of a first inverse tone mapping curve obtained using the second inverse tone mapping function, and modifying at least one parameter of the second inverse tone mapping function to obtain a monotonic second inverse tone mapping curve when the first inverse tone mapping curve is not monotonic.

[0015] In a second aspect, one or more embodiments of the present invention provide an apparatus for inverse tone mapping, wherein the apparatus includes an electronic circuit system adapted to: acquire an image, referred to as a current image; acquire a gain function of a first inverse tone mapping function, referred to as an initial gain function; if analysis of the current image indicates that at least one pixel whose luminance value is at least equal to a luminance value depending on a predetermined percentage of the luminance value of that pixel in the current image has an extended luminance value higher than a target value due to the application of the first inverse tone mapping function to the current image, then apply a second inverse tone mapping function corresponding to the first inverse tone mapping function to the current image, wherein the gain function has been replaced by a modified gain function; the modified gain function is a function derived from the initial gain function, wherein the gain provided by the initial gain function is attenuated by an attenuation function, the attenuation function being an increasing function of luminance values ​​weighted by a weighting factor controlling the intensity of the attenuation, the weighting factor depending on the statistical distribution of luminance values ​​between the maximum luminance value and the luminance value depending on the predetermined percentage in the histogram of the current image.

[0016] In one embodiment, the electronic circuit system is further adapted to: browse the histogram in descending order of brightness values, starting from the maximum brightness value; calculate an intermediate weighting factor for each browsed brightness value when the browsed brightness value is at least equal to the brightness value depending on the predetermined percentage and the result of applying the first inverse tone mapping function to the browsed brightness value is at least equal to the target extended brightness value; and,

[0017] Set the value of the weighting factor to the maximum value corresponding to the calculated intermediate weighting factor.

[0018] In one implementation, the intermediate weighting factor depends on an intermediate extended target luminance value, which is a weighted sum between the extended target luminance value and the result of applying the first inverse tone mapping function to the browsed luminance value, each weight of the weighted sum depending on a value representing the proportion of pixels whose luminance value of the current image is higher than the browsed luminance value.

[0019] In one implementation, the value representing the proportion of pixels with a brightness value higher than the brightness value being viewed also depends on the maximum authorized extended brightness value.

[0020] In one embodiment, the electronic circuit system is further adapted to apply the first inverse tone mapping function to the current image when the number of pixels whose brightness value is higher than the brightness value being viewed is less than a predefined minimum number of pixels.

[0021] In one embodiment, the current image belongs to a video sequence and the electronic circuit system is also adapted to detect scene cuts in the current video sequence. The weighting factor controlling the intensity of the attenuation also depends on at least one other weighting factor controlling the intensity of the attenuation calculated for another image preceding the current image in the video sequence, where the scene cut has not yet been detected between the current image and the other image.

[0022] In one embodiment, the electronic circuit system is also adapted to verify the monotonicity of the first inverse tone mapping curve obtained using the second inverse tone mapping function, and to modify at least one parameter of the second inverse tone mapping function to obtain a monotonic second inverse tone mapping curve when the first inverse tone mapping curve is not monotonic.

[0023] In a third aspect, one or more embodiments of the present invention provide an apparatus comprising the device according to the second aspect.

[0024] In a fourth aspect, one or more embodiments of the present invention provide a signal generated by the method of the first aspect or by the device of the second aspect or the apparatus of the third aspect.

[0025] In a fifth aspect, one or more embodiments of the present invention provide a computer program comprising program code instructions for implementing the method according to the first aspect.

[0026] In a sixth aspect, one or more embodiments of the present invention provide an information storage device that stores program code instructions for implementing the method according to the first aspect. 4. Description of the attached drawings

[0027] Figure 1 An example of the context in which the following implementation scheme can be achieved is shown;

[0028] Figure 2 An example of a hardware architecture for a processing module capable of implementing various aspects and implementation schemes is illustrated schematically;

[0029] Figure 3 A block diagram of an example system in which various aspects and implementation schemes are implemented is shown;

[0030] Figure 4 A high-level representation of various implementations of methods for improving inverse tone mapping is schematically shown;

[0031] Figure 5A , Figure 5B and Figure 5C Three examples representing ITM curves;

[0032] Figure 6A A first embodiment of the step of determining whether bright spot attenuation is required for the current SDR image during the ITM process is illustrated;

[0033] Figure 6B A second embodiment of the step for determining whether bright spot attenuation is required for the current SDR image during the ITM process is illustrated schematically;

[0034] Figure 7 The method for obtaining a first threshold for determining whether the current SDR image requires bright spot attenuation is illustrated schematically.

[0035] Figure 8 The method for obtaining a second threshold for determining whether the current SDR image requires bright spot attenuation is illustrated schematically.

[0036] Figure 9A A first embodiment of the steps for applying the ITM function based on the modified gain function is illustrated schematically;

[0037] Figure 9B A second implementation of the steps for applying the ITM function based on the modified gain function is illustrated schematically;

[0038] Figure 10 A method for calculating the attenuation intensity factor, which is used to calculate the modified gain function, is illustrated schematically.

[0039] Figure 11A It is a graphical representation of the gain function G();

[0040] Figure 11B It is a graphical representation of the extended brightness value curve (i.e., the ITM curve);

[0041] Figure 12A Compare the graphical representation of the gain function G() with the modified gain function Gbs() that allows for spotlight attenuation;

[0042] Figure 12B The ITM curves obtained using the gain function G() and the modified gain function Gbs() were compared.

[0043] Figure 13A The non-monotonic ITM curve obtained by introducing bright spot decay into the ITM function is shown;

[0044] Figure 13B The correction of the ITM curve is shown, which allows for ensuring the monotonicity of the ITM curve; and,

[0045] Figure 14 This schematically illustrates a method that allows ensuring the monotonicity of the ITM curve. 5. Detailed Implementation

[0046] Different types of inverse tone mapping methods exist. For example, in the field of local tone mapping algorithms, patent application WO2015 / 096955 discloses a method that, for each pixel P of an SDR (or LDR) image, includes the step of obtaining the pixel extension value E(P) and then inverse tone mapping the brightness Y(P) of pixel P to the extended brightness value E(P) as follows. exp (P) steps:

[0047] Y exp (P)=Y(P) E(P) ×[Y enhance (P)] (Equation 1)

[0048] in:

[0049] ·Y exp (P) is the extended brightness value of pixel P.

[0050] Y(P) is the brightness value of pixel P in an SDR (or LDR) image.

[0051] ·Y enhance(P) It is the brightness enhancement value P of a pixel in an SDR (or LDR) image.

[0052] E(P) is the pixel extension value of pixel P.

[0053] A set of values ​​E(P) for all pixels in an SDR (or LDR) image forms the image's spread map, spread function, or gain function. This gain function can be generated using various methods. For example, one method involves low-pass filtering the luminance value Y(P) of each pixel P to obtain a low-pass filtered luminance value Y. low (P) and apply the quadratic function to the low-pass filtered brightness value, the quadratic function being defined by parameters a, b, and c according to the following equation:

[0054] E(P)=a[Y low (P)] 2 +b[Y low (P)]+c

[0055] Another approach to facilitating the concrete implementation of hardware, based on WO2015 / 096955, uses the following equation:

[0056]

[0057] The above equation can be expressed as follows:

[0058]

[0059] The parameter d can be set, for example, to d = 1.25. Y enhance(P) In this case, it is the image brightness value Y(P) and the low-pass version Y of the brightness value. low(P) The functions of both.

[0060] Document ITU-R BT.2446-0 discloses a method for converting SDR (or LDR) content to HDR content using formulas of the same kind as the following:

[0061] Y′ exp (P)=Y″(P) E(Y″(P))

[0062] in

[0063] Y′ is in the range [0……1]

[0064] ·Y″=255.0×Y′

[0065] When Y″≤T, E=a1Y″ 2 +b1Y″+c1,

[0066] When Y″>T, E=a2Y″ 2 +b2Y″+c2

[0067] ·T=70

[0068] ·a1=1.8712e-5, b1=-2.7334e-3, c1=1.3141

[0069] ·a2=2.8305e-6, b2=-7.4622e-4, c2=1.2528

[0070] As can be seen from the above, the gain function is based on a power function, and its exponent depends on the brightness value of the current pixel or on a filtered version of that brightness value.

[0071] More generally, all global expansion methods can be expressed as ITM functions of the following form for all input values ​​other than zero (for inputs that are zero, the output is logically zero):

[0072] Y exp =Y G(Y) (Equation 2)

[0073] Where G() is the gain function of brightness Y.

[0074] Similarly, all local expansion methods can be expressed for all input values ​​other than zero as follows:

[0075]

[0076] Where YF It is a filtered version of the brightness Y, where G() is the value of Y. F The gain function of Y, and Y enhance It is Y and its surrounding pixels Ys i The function.

[0077] In both cases (global or local), the gain function is monotonic in order to match the input SDR image.

[0078] Some inverse tone mapping (ITM) methods use a gain function G() based on predetermined extended parameters (such as those described in ITU-R BT.2446-0 document) without any adaptation to the image content. Patent application EP3249605 discloses a method for inverse tone mapping of images that automatically adapts to the image content. This method uses a set of profiles forming templates. These profiles are predetermined during a learning phase as offline processing. Each profile is defined by visual features associated with the gain function, such as a brightness histogram.

[0079] During the learning phase, a profile is determined from a large number of reference images that are manually graded by colorists who manually set the inverse tone mapping parameters and generate gain functions for these images. The reference images are then clustered based on these generated gain functions. Each cluster is processed to extract a representative brightness histogram and its associated representative gain function, thereby forming a profile emanating from the cluster.

[0080] When new SDR content is received, a histogram of the SDR image of the new SDR content is determined. Then, each calculated histogram is compared with each histogram in the template stored from the learning phase to find the best-matching histogram for the template. For example, the distance between the calculated histogram and each histogram in the template is calculated. Then, a gain function associated with the best-matching histogram for the template is selected and used to perform inverse tone mapping on the image (or multiple images) corresponding to the calculated histogram. In this way, the best gain function suitable for the template of the SDR image is applied to output the corresponding HDR image.

[0081] As mentioned above, even when applying the best possible gain function, inverse tone mapping operations may still produce HDR images that include overly bright areas (or highlights) that may exceed the capacity of the display device or be unpleasant to the user.

[0082] To address the aforementioned issues, patent application EP3503019A1 discloses a method that attenuates bright areas to a predefined target brightness value, provided that at least a predetermined percentage of pixels in the input image have brightness values ​​higher than the input brightness, whose extended value equals the target brightness value. However, this method relies on a fixed threshold: when applied to all incoming images, it can cause a flickering effect, or at least a sudden change in the overall brightness of the extended image. This might occur, for example, when a credits list with high brightness values ​​scrolls to the end of the content and triggers attenuation.

[0083] The various implementation schemes described below aim to improve this situation by adaptively and smoothly attenuating these bright areas to a predefined target brightness once the proportion of pixels included in the maximum bright area is greater than a predefined proportion.

[0084] Figure 1 An example of the context in which the following implementation scheme can be achieved is shown.

[0085] exist Figure 1 In the system 3, a device 1, which can be a camera, a storage device, a computer, or any device capable of delivering SDR content, uses communication channel 2 to transmit the SDR content to the system 3. Communication channel 2 can be a wired (e.g., Ethernet) or wireless (e.g., WiFi, 3G, 4G, or 5G) network link.

[0086] SDR content includes a fixed sequence of images or videos.

[0087] System 3 converts SDR content to HDR content; that is, it applies inverse tone mapping to SDR content to obtain HDR content.

[0088] The acquired HDR content is then transmitted to the display system 5 using communication channel 4, which can be a wired or wireless network. The display system then displays the HDR content.

[0089] In the implementation scheme, system 3 is included in display system 5.

[0090] In the implementation plan, device 1, system 3 and display system 5 are all included in the same system.

[0091] In the implementation scheme, display system 5 is replaced by a storage device that stores HDR content.

[0092] Figure 2An example of the hardware architecture of a processing module 30 included in system 3 and capable of implementing different aspects and implementations is illustrated schematically. As a non-limiting example, the processing module 30 includes the following items connected by a communication bus 305: a processor or CPU (Central Processing Unit) 300 containing one or more microprocessors, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture; random access memory (RAM) 301; read-only memory (ROM) 302; a storage unit 303, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives and / or optical disk drives, or storage media readers such as SD (Secure Digital) card readers and / or hard disk drives (HDDs) and / or network-accessible storage devices; and at least one communication interface 304 for exchanging data with other modules, devices, systems, or equipment. Communication interface 304 may include, but is not limited to, a transceiver configured to transmit and receive data through a communication channel. Communication interface 304 may include, but is not limited to, a modem or network interface card.

[0093] The communication interface 304 enables, for example, the processing module 30 to receive SDR content and provide HDR content.

[0094] Processor 300 is capable of executing instructions loaded into RAM 301 from ROM 302, external memory (not shown), storage media, or a communication network. When processing module 30 is powered on, processor 300 is capable of reading instructions from RAM 301 and executing these instructions. These instructions form a computer program that causes, for example, processor 300 to perform the following... Figure 4 The inverse tone mapping method is described.

[0095] All or part of the algorithm and steps of this inverse tone mapping method can be implemented in software by executing a set of instructions by a programmable machine such as a DSP (Digital Signal Processor) or a microcontroller, or in hardware by a machine or dedicated component such as an FPGA (Field Programmable Gate Array) or an ASIC (Application-Specific Integrated Circuit).

[0096] Figure 3A block diagram illustrating an example of System 3 in which various aspects and embodiments are implemented is shown. System 3 may be embodied as a device including the various components described below and configured to perform one or more aspects and embodiments described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Elements of System 3 may be embodied individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, System 3 includes a processing module 30 that implements an inverse tone mapping method. In various embodiments, System 3 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports.

[0097] Inputs to processing module 30 may be provided via various input modules as shown in box 32. Such input modules include, but are not limited to: (i) a radio frequency (RF) module that receives, for example, RF signals transmitted over the air by a broadcaster; (ii) a component (COMP) input module (or a set of COMP input modules); (iii) a universal serial bus (USB) input module; and / or (iv) a high-definition multimedia interface (HDMI) input module. Figure 3 Other examples not shown include composite video.

[0098] In various embodiments, the input module of block 32 has associated corresponding input processing elements as known in the art. For example, the RF module may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal, or limiting a signal band to a band), (ii) down-converting the selected signal, (iii) re-band-limiting the signal to a narrower band to select (e.g.,) a signal band that may be referred to as a channel in some embodiments), (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select the desired data packet stream. The RF module of various embodiments includes one or more elements for performing these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include tuners that perform various functions among these functions, including, for example, down-converting received signals to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or to baseband. In one set-top box implementation, the RF module and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to the desired frequency band. Various implementations rearrange the order of the aforementioned (and other) components, remove some of these components, and / or add other components that perform similar or different functions. Adding components may include inserting components between existing components, such as inserting amplifiers and analog-to-digital converters. In various implementations, the RF module includes an antenna.

[0099] Additionally, the USB and / or HDMI modules may include corresponding interface processors for connecting System 3 to other electronic devices across USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented as needed, for example, within a separate input processing IC or within processing module 30. Similarly, various aspects of USB or HDMI interface processing may be implemented as needed, either within a separate interface IC or within processing module 30. Demodulation, error correction, and demultiplexing streams are provided to processing module 30.

[0100] Various components of System 3 can be housed within an integrated housing. Within the integrated housing, the various components can be interconnected using suitable connection arrangements (e.g., internal buses known in the art, including inter-IC (I2C) buses, wiring, and printed circuit boards) and data can be transferred between these components. For example, in System 3, processing module 30 is interconnected with other components of System 3 via bus 305.

[0101] The communication interface 304 of the processing module 30 allows the system 3 to communicate on the communication channel 2. For example, the communication channel 2 can be implemented in a wired and / or wireless medium.

[0102] In various implementations, wireless networks such as Wi-Fi networks, such as IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers), are used to stream data or otherwise provide it to system 3. In these implementations, the Wi-Fi signal is received via a communication channel 2 and a communication interface 304 suitable for Wi-Fi communication. The communication channel 3 in these implementations is typically connected to an access point or router that provides access to external networks, including the Internet, for use in allowing streaming applications and other communications across carriers. Other implementations use a set-top box to provide streaming data to system 3, delivering data via an HDMI connection in input box 32. Other implementations use an RF connection in input box 32 to provide streaming data to system 3. As described above, various implementations provide data in a non-streaming manner. Additionally, various implementations use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0103] System 3 can provide output signals to various output devices, including a display 5, speakers 6, and other peripheral devices 7. The display 5 in various embodiments includes one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display 5 can be used in televisions, tablets, laptops, cellular phones (mobile phones), or other devices. The display 5 can also be integrated with other components (e.g., as in a smartphone) or with separate components (e.g., an external monitor for a laptop computer). The display device 5 is HDR content compatible. In various examples of embodiments, other peripheral devices 7 include one or more of a standalone digital video disc (or digital versatile disc) (DVR, which may represent both terms), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 7 that provide functionality based on the output of System 3. For example, a disc player performs the function of playing the output of System 3.

[0104] In various implementations, control signals are transmitted between system 3 and display 5, speaker 6, or other peripheral devices 7 using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable device-to-device control with or without user intervention. Output devices are communicatively coupled to system 3 via dedicated connections through corresponding interfaces 33, 34, and 35. Alternatively, output devices can be connected to system 3 via communication interface 304 using communication channel 2. Display 5 and speaker 6 may be integrated into a single unit with other components of system 3 in electronic devices such as televisions. In various implementations, display interface 5 includes a display driver, such as, for example, a timing controller (TCon) chip.

[0105] For example, if the RF section of input 32 is part of a separate set-top box, then display 5 and speaker 6 may optionally be separate from one or more other components. In various embodiments where display 5 and speaker 6 are external components, output signals may be provided via dedicated output connections, including, for example, HDMI ports, USB ports, or COMP outputs.

[0106] Various implementations involve the application of inverse tone mapping methods. As used in this application, inverse tone mapping may encompass all or part of a process performed, for example, on a received SDR image or video stream, to produce a final HDR output suitable for display. In various implementations, such a process includes one or more processes typically performed by an image or video decoder, such as a JPEG decoder or an H.264 / AVC (ISO / IEC 14496-10 – MPEG-4 Part 10, Advanced Video Coding), H.265 / HEVC (ISO / IEC 23008-2 – MPEG-H Part 2, High Efficiency Video Coding / ITU-T H.265) or H.266 / VVC (Various Video Coding) decoder.

[0107] When the accompanying drawings are presented as flowcharts, it should be understood that block diagrams of the corresponding devices are also provided. Similarly, when the accompanying drawings are presented as block diagrams, it should be understood that flowcharts of the corresponding methods / processes are also provided.

[0108] The specific embodiments and aspects described herein may be implemented, for example, in methods or processes, apparatus, software programs, data streams, or signals. Even if discussed only in the context of a single form of specific embodiment (e.g., discussed only as a method), specific embodiments of the discussed features may be implemented in other forms (e.g., apparatus or program). Apparatus may be implemented, for example, in suitable hardware, software, and firmware. Methods may be implemented, for example, in a processor, which generally refers to a processing device.

[0109] The processing device includes, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes communication devices, such as computers, mobile phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0110] The reference to "an implementation scheme" or "implementation scheme" or "a specific implementation" or "specific implementation," and other variations thereof, means that the specific features, structures, characteristics, etc., described in connection with the implementation scheme are included in at least one implementation scheme. Therefore, the appearance of the phrase "in an implementation scheme" or "in an implementation scheme" or "in a specific implementation" or "in a specific implementation," and any other variations appearing throughout this application, do not necessarily refer to the same implementation scheme.

[0111] Additionally, this application may involve "determining" various types of information. Determining information may include, for example, estimated information, calculated information, predicted information, information retrieved from memory, or information obtained, for example, from another device, module, or user, one or more of these.

[0112] Furthermore, this application may relate to "accessing" various types of information. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information, or more of these.

[0113] Furthermore, this application may relate to "receiving" various types of information. Like "access," "receiving" is intended to be a broad term. Receiving information may include, for example, accessing information or retrieving information (e.g., from memory) or more. Moreover, "receiving" typically involves one or more of the following during operations such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0114] It should be understood that, for example, in the cases of “A / B,” “A and / or B,” “at least one of A and B,” and “one or more of A and B,” the use of any of the following “ / ,” “and / or,” and “at least one,” “one or more” is intended to cover selecting only the first listed option (A), or only the second listed option (B), or selecting both options (A and B). As a further example, in the cases of “A, B, and / or C,” “at least one of A, B, and C,” and “one or more of A, B, and C,” such phrases are intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or selecting all three options (A, B, and C). As will be apparent to those skilled in the art and related fields, this can be extended to as many of the listed items as possible.

[0115] It will be apparent to those skilled in the art that specific embodiments or implementations can produce various signals formatted to carry, for example, information that can be stored or transmitted. The information may include, for example, instructions for performing a method or data generated by one of the embodiments or implementations. For example, a signal may be formatted to carry an HDR image or video sequence of the embodiment. Such signals may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or baseband signals. Formatting may include, for example, encoding the HDR image or video sequence as an encoded video stream and modulating a carrier using the encoded stream. The information carried by the signal may be, for example, analog or digital information. It is known that signals can be transmitted via various wired or wireless links. The signal may be stored on a processor-readable medium.

[0116] Figure 4 A high-level representation of various implementations of the inverse tone mapping method is schematically illustrated. In the various implementations described below, it is assumed that a gain function G() has been defined for the current SDR image. The definition of this gain function makes Y′ G(Y′) It is monotonic, and Y' is the gamma-rated version of the linear brightness value Y.

[0117] In various implementations, a histogram of the brightness value Y of the SDR image is used to detect the amount of information at high brightness levels.

[0118] As an example, in the rest of the document, ITMO’s target LMax (i.e., the target maximum brightness value) is “1000” nits, and the current input SDR image is assumed to be an “8” bit image, where the value “255” corresponds to 100 nits.

[0119] In this case, the ITM function can be written as follows:

[0120]

[0121] Where Y SDR ′ is the brightness value of the current input SDR image, and Y is... HDR Y' is the brightness value of the output HDR image. Brightness value Y SDR ′ (regardless of its bit depth) is normalized within the range [0; 255]. Similarly, if LMax is “1000” nits, the luminance value Y HDR ′ (regardless of its number of bits) is normalized within the range [0; 1000]. This means that if Y HDR The gain function of Y' makes Y' HDR ′= Above 1000 nits, that is, when linearized relative to 1000 nits, Y′ HDRIt is cropped to "1000" nits, thus removing any details in the corresponding area.

[0122] Y SDR ′ and Y HDR Both are γ-ized, and Y SDR and Y HDR Both are linear, for example:

[0123] Y SDR =(Y HDR (′ / 255) 2.4 ×100

[0124] Y HDR =(Y HDR (′ / 1000) 2.4 ×1000

[0125] In step 41, the processing module 30 obtains an SDR image, which is referred to as the current SDR image.

[0126] In step 42, processing module 30 obtains the gain function G() of the ITM function of Equation 4, which is called the initial gain function G(). Once obtained, the gain function G() allows the ITM curve to be obtained from the ITM function of Equation 4.

[0127] Figure 5A , Figure 5B and Figure 5C Three examples are shown, representing the ITM curve for a “1000” nit display as obtained using Equation 4. The SDR input is in the range [0…255] (meaning it must be normalized in the range [0; 255] if it is not an “8” bit image), and the output is normalized in the range [0; 1000]. Figure 5A The curve shows that the maximum input value "255" (corresponding to "100" nits SDR) produces an output of "1000" when linearized, which corresponds to "1000" nits. Figure 5B The curve shows a maximum value of approximately "700", which corresponds to "425" nits when linearized, and Figure 5C The curve shows a maximum value of approximately "1200", which corresponds to "1550" nits when linearized.

[0128] Figure 5A The curve does not produce a clipping value, but if the input image contains large bright areas (near-white areas), it may produce some dazzling effects. Regardless of the input, Figure 5B None of the curves produce any dazzling effect. If the input image contains large areas with high values ​​(and not just near-white areas), then Figure 5C The curve can potentially produce a higher value than... Figure 5AThe curves have a more dazzling effect.

[0129] In step 43, processing module 30 determines whether the current SDR image requires bright spot attenuation. Bright spot attenuation includes reducing the brightness of overexposed areas in the HDR image obtained from the SDR image to a target extended brightness value BSTarget. Bright spot attenuation is applied when at least a predetermined percentage of pixels in the current SDR image have brightness values ​​higher than a brightness value Y', such that the extended value obtained by applying the inverse tone mapping function of Equation 4 to the brightness value Y' is equal to the target extended brightness value BSTarget when linearized. Refer to Figure 6. Figure 7 and Figure 8 Step 43 is described in detail. In other words, if analysis of the current SDR image shows that at least a percentage (P) of the pixels in the HDR image obtained by applying the first inverse tone mapping function to the current SDR image have extended brightness values ​​that, once linearized, are higher than the target extended brightness value BSTarget.

[0130] If no bright spot attenuation is required for the current SDR image, step 44 is performed after step 43. During step 44, processing module 30 applies a first ITM function to the current SDR image to obtain an HDR image. The first ITM function is, for example, the ITM function based on equation 4 of the initial gain function G().

[0131] Otherwise, if bright spot attenuation is not required for the current SDR image, step 45 is performed after step 43. During step 45, processing module 30 applies a second ITM function to the current SDR image to obtain an HDR image. The second ITM function corresponds, for example, to the ITM function in Equation 4, where the initial gain function has been replaced by a modified gain function Gbs(). (See also...) Figure 6A , Figure 7 , Figure 8 and Figure 9A , Figure 6B , Figure 9B , Figure 10 , Figure 11A , Figure 11B , Figure 12A , Figure 12B , Figure 13A , Figure 13B and Figure 14 The implementation scheme for obtaining and applying the modified gain function Gbs() is described in detail.

[0132] Figure 6A A first embodiment of the steps for determining whether bright spot attenuation is required for the current SDR image during the ITM process is illustrated schematically.

[0133] Figure 6A The process corresponds to Figure 4The first implementation scheme of step 43 in the process.

[0134] In step 430, the processing module 30 calculates a first threshold TH1. The first threshold TH1 corresponds to the gamma-rated brightness value Y of pixels whose brightness values ​​are higher than or equal to the threshold brightness value TH1 in the current SDR image, representing a predetermined percentage P of pixels. HDR ′。 The following is for reference. Figure 7 The calculation of the first threshold TH1 is described in detail.

[0135] In step 431, processing module 30 calculates the second threshold TH2. The second threshold TH2 corresponds to Y. HDR The last gamma-rated brightness value in Y′ HDR When the value of ′ increases from zero to “255”, its extended brightness value Y during linearization HDR The value is less than or equal to the target extended brightness value BSTarget. In other words, the second threshold TH2 verifies Y. HDR '≤BSTarget' and (Y+1) HDR '>BSTarget', where 'BSTarget' is the gamma-rated version of BSTarget. See below for reference. Figure 8 The calculation of the second threshold TH2 is described in detail.

[0136] In step 432, processing module 30 compares the first threshold TH1 with the second threshold TH2. If TH1 > TH2, then processing module 30 determines in step 433 that the current SDR image requires bright spot attenuation during the ITM process. Otherwise, processing module 30 determines in step 434 that bright spot attenuation is not required during the ITM process of the current SDR image.

[0137] Figure 7 The method for obtaining the first threshold TH1 is illustrated schematically. Figure 7 The method corresponds to step 430 in Figure 6.

[0138] In step 4301, processing module 30 initializes the value Psum. Psum is the result of multiplying the percentage P (represented as a value between zero and one) as defined above by the total number of pixels in the current SDR image. The total number of pixels in the current SDR image is equal to the sum of all bins in the histogram Histo of the current SDR image:

[0139]

[0140] TH1 is Y SDR The first value of ′ (in descending order from “255” to zero), where:

[0141]

[0142] And the calculation is as follows:

[0143] In step 4302, the processing module 30 initializes the variable k to "255" and initializes the variable sum to zero.

[0144] In step 4303, the processing module 30 compares the variable sum with the value Psum. If sum < Psum, then step 4305 is performed after step 4303. During step 4305, the processing module 30 increases the value of the variable sum of the value Histo[k] at position k in the histogram Histo.

[0145] sum = sum + Histo[k]

[0146] In step 4306, the processing module 30 decrements the variable k by one unit. Step 4303 is performed after step 4306.

[0147] If sum ≥ Psum at step 4303, the processing module 30 sets the value of the first threshold TH1 to the value of the variable k.

[0148] Figure 8 Schematically shows a method for obtaining the second threshold TH2.

[0149] Figure 8 The method corresponds to step 431 in FIG. 6.

[0150] In step 4310, the processing module 30 initializes the variable k to zero.

[0151] In step 4312, the processing module 30 sets the variable Kexp to the value k G(k) :

[0152] Kexp = k G(k)

[0153] In step 4313, the processing module 30 compares the variable Kexp with the gamma target extended luminance value BSTarget'.

[0154] If Kexp ≤ BSTarget', the processing module 30 increments the variable k by one unit in step 4314. Step 4312 is performed after step 4314.

[0155] Otherwise, if Kexp > BSTarget', the processing module 30 sets the value of the second threshold TH2 to the value of the variable k - 1.

[0156] Figure 9A Schematically shows a first embodiment of steps for calculating and applying an ITM function based on a modified gain function.

[0157] Figure 9A The process corresponds to Figure 4 The first implementation scheme of step 45 in the process.

[0158] Document EP3503019A1 proposes the following modified gain function Gbs() to perform bright spot attenuation:

[0159] Gbs(Y')=G(Y')–hlCoef*(Y′ / 255) γ (Equation 5)

[0160] The preferred value for γ is "6". hlCoef is the attenuation intensity factor used to control the intensity of bright spot attenuation during the ITM process. It can be seen that the modified gain function Gbs() is a function derived from the initial gain function G(), where the gain G(Y') provided by the initial gain function G() is determined by the attenuation function hlCoef*(Y′ / 255). γ The attenuation function is an increasing function of the brightness value Y', which is weighted by the attenuation intensity factor hlCoef, which controls the intensity of the brightness attenuation.

[0161] In the first embodiment of step 45, the processing module 30 calculates the attenuation intensity factor hlCoef as follows:

[0162]

[0163] In step 453, the processing module applies the ITM function to the current SDR image based on the modified gain function Gbs(). The modified gain function Gbs() is obtained by using the attenuation intensity factor hlCoef obtained in step 451 in equation "5". The ITM function is, for example, the ITM function of equation "4", which becomes:

[0164]

[0165] Figure 11A It is a graphical representation of the gain function G(). Figure 11A The application of the gain function to input images with brightness values ​​between zero and "255" allows obtaining information about... Figure 11B The curve representing the extended brightness value (i.e., the ITM curve).

[0166] Figure 12A The graphical representation of the gain function G() is compared with the modified gain function Gbs() (represented by dashed lines) that allows for spotlight attenuation. Figure 12B The curves of the extended luminance values ​​obtained using the gain function G() and the modified gain function Gbs() (represented by dashed lines) were compared.

[0167] It can be seen that when using the initial gain function G(), an extended brightness value exceeding "1000" nits can be obtained. The application of the modified gain function Gbs() ensures that no extended brightness value exceeds the "1000" nit limit.

[0168] It can be noted that even if the attenuation intensity factor hlCoef is calculated for only one luminance value (i.e., for TH1), the attenuation intensity factor hlcoef is valid for all possible luminance values.

[0169] It can also be noted that if TH1 has a different value, the attenuation intensity factor hlCoef and the modified gain function Gbs() will both be different. This means that the modified gain function Gbs(), which allows for bright spot attenuation, is somewhat adaptive, but insufficient to prevent flickering or drastic changes in overall brightness.

[0170] It can also be noted that the closer the first threshold TH1 is to the second threshold TH2, the fewer bright areas are allowed to be attenuated by the modified gain function Gbs() compared to the initial gain function G(). This is because the attenuation intensity factor is calculated at the threshold TH1, not at the highest brightness value Y' in the Histo[] histogram. This may result in no bright spot attenuation being achieved, thus causing high brightness values ​​to be clipped.

[0171] As mentioned above, the attenuation intensity factor hlCoef is calculated at the threshold TH1. Therefore, if most pixels, expressed as a percentage P, have luminance values ​​near TH1, the calculated attenuation intensity factor hlCoef is valid. However, if most of these pixels correspond to higher luminance values ​​(i.e., >TH1 or even >>TH1), the calculated attenuation intensity factor hlCoef is insufficient to represent these pixels.

[0172] Furthermore, it would be interesting to introduce a smooth transition zone between the state "attenuated" and the state "not attenuated," which would prevent flickering when transferring from an SDR image that requires bright spot attenuation to an image that does not (this occurs when the proportion of pixels with a brightness value equal to "255" alternately goes above or below the percentage P, and there are no other brightness values ​​Y' in the histogram histo[] indicating enough pixels to trigger bright spot attenuation).

[0173] In the following text, combined with Figure 6B , Figure 10 and Figure 9B A second embodiment of steps 43 and 45 is described. This second embodiment allows for the reduction of flickering by introducing a smooth transition region between the state "decayed" and the state "undecayed".

[0174] Figure 6BA second embodiment of the steps for determining whether bright spot attenuation is required for the current SDR image during the ITM process is illustrated schematically.

[0175] Figure 6B The process corresponds to Figure 4 The second implementation scheme for step 43 in the above.

[0176] In step 430', the processing module 30 calculates the first threshold TH1, as described with respect to step 430.

[0177] In step 431', the processing module 30 calculates the attenuation intensity factor hlCoef and performs the following... Figure 10 The method described.

[0178] In step 432', the processing module 30 determines whether the attenuation intensity factor hlCoef is different from zero.

[0179] If hlCoef = 0, then processing module 30 considers that no bright spot attenuation is needed. In this case, processing module 30 applies the same step 433' as step 433.

[0180] If hlCoef ≠ 0, then processing module 30 considers that the current SDR image requires bright spot attenuation during the ITM process. In this case, processing module 30 applies the same step 434' as step 434.

[0181] Figure 9B A second implementation of the steps for applying the ITM function based on the modified gain function is illustrated schematically.

[0182] In step 453, the processing module applies the ITM function to the current SDR image based on the modified gain function Gbs(). The modified gain function Gbs() is obtained by using the attenuation intensity factor hlCoef obtained in step 431 in equation "5". The ITM function is, for example, the ITM function of equation "4", which becomes:

[0183]

[0184] Figure 10 The method for calculating the attenuation intensity factor hlCoef, which is used to calculate the modified gain function, is illustrated schematically.

[0185] Figure 10 The method corresponds to Figure 6B Step 431' in the middle.

[0186] In step 431A, processing module 30:

[0187] • Initialize variable k to "255". In the following text, k decreases from "255" to TH1;

[0188] • The variable SumOfHiBins decreases to zero;

[0189] • The attenuation intensity factor hlCoef decreases to zero; and,

[0190] • γ decreases to six.

[0191] In step 431B, the processing module compares variable k with the first threshold TH1.

[0192] If k ≥ TH1, then step 431C is performed after step 431B. During step 431C, the processing module 30 increments the variable SumOfHiBins, which represents the value of histo[k] in the histogram.

[0193] In step 431D, the variable α is set to sumOfHiBins / Psum:

[0194] α = sumOfHiBins / Psum

[0195] As a reminder:

[0196]

[0197] In step 431E, if variable α > 1, then in step 431F, α is set to "1". Otherwise, variable α is not modified. Step 431G is performed after steps 431E and 431F.

[0198] During step 431G, the extended value Kexp of k is calculated:

[0199] Kexp = k G(k)

[0200] In step 431H, the extended value Kexp is compared with the gamma-scaled target extended brightness value BSTarget'.

[0201] If Kexp≥BSTarget', then processing module 30 calculates the new target extended luminance value BSTarget'[k] during step 431I:

[0202] BSTarget'[k]=(1-α)*Kexp+α*BSTarget'

[0203] As can be seen, the target luminance is redefined for each value of k based on the extended value Kexp and the gamma-enhanced target luminance value BSTarget'. The smaller α is (i.e., the lower the proportion of pixels with luminance values ​​equal to or higher than k compared to the percentage P), the closer the new gamma-enhanced target luminance value BSTarget'[k] is to the extended value Kexp. The higher α is (i.e., the higher the proportion of pixels with luminance values ​​equal to or higher than k compared to the percentage P), the closer the new gamma-enhanced target luminance value BSTarget'[k] is to the gamma-enhanced target luminance value BSTarget'.

[0204] In step 431J, the processing module 30 calculates a new attenuation intensity factor hlCoef[k] for the luminance value k:

[0205]

[0206] It can be seen that the higher the value of log(BSTarget′[k]) / log(k), the lower the attenuation intensity factor hlCoef[k]. Therefore, when BSTarget'[k] equals BStarget', the maximum value of the attenuation intensity factor hlCoef[k] is obtained, while when BSTarget[k] equals the extended value Kexp, the minimum value of the attenuation intensity factor hlCoef[k] is obtained. In other words, the maximum value of the attenuation intensity factor hlCoef[k] is obtained when the proportion of pixels with values ​​equal to or higher than k is high compared to the percentage P, and the minimum value of the attenuation intensity factor hlCoef[k] is obtained when the proportion of pixels with values ​​equal to or higher than k is low compared to the percentage P.

[0207] In step 431K, processing module 30 determines whether variable k is equal to "255". If not, step 431L is performed after step 431K. If yes, step 431M is performed after step 431K.

[0208] During step 431M, the processing module 30 sets the value of the attenuation intensity factor hlCoef to the value hlCoef[k] and sets the value hlCoefPos to the value k.

[0209] Step 431N is performed after step 431M, during which k is decreased by one unit.

[0210] During step 431L, the processing module 30 compares the value hlCoef[k] with the current value of the attenuation intensity factor hlCoef.

[0211] If hlCoef[k] > hlCoef, then processing module 30 executes step 431M. Otherwise, processing module 30 executes step 431N.

[0212] As can be seen in steps 431K, 431L, and 431M, the attenuation intensity factor hlCoef[k] is compared with the last calculated value of the attenuation intensity factor hlCoef, and the highest value is saved and stored in hlCoef, and the corresponding value of k is stored in the variable hlCoefPos. If k = 255, this comparison cannot be performed. In this particular case, hlCoef = hlCoef

[255] , and hlCoefPos = 255. These iterations continue until TH1, but stop once kexp < BSTarget', which means the extended value of k is not high enough to attenuate.

[0213] Step 431B is performed after step 431N.

[0214] If k < TH1 in step 431B or if kexp < BSTarget' in step 431H, the processing module 30 performs step 431O.

[0215] During step 431O, the processing module 30 determines whether the attenuation intensity factor hlCoef is equal to zero. hlCoef = 0 means that all bins of the histogram histo above TH1 are empty (no pixel in the current SDR image has a luminance value higher than TH1). If so, the processing module 30 sets the value of hlCoef to TH1. Figure 10 The method ends at step 431Q. If the attenuation intensity factor hlCoef is different from zero, step 431Q is directly performed after step 431O.

[0216] The test of step 432' (hlCoef ≠ 0) is equivalent to checking whether the extended luminance value obtained by applying the inverse tone mapping function of Equation 4 to the luminance value Y' of at least one pixel of the current image is higher than the target extended luminance value BSTarget when linearized. In fact, after performing Figure 10 the method, hlCoef being equal to zero means that the processing module 30 cannot find at least one pixel in the current SDR image whose extended luminance value is higher than BSTarget'. This test is actually performed for each luminance value k during steps 431B and 431H when browsing the histogram in descending order from "255" to TH1. It can be noted that the test of step 431H is equivalent to testing k < TH2.

[0217] In Figure 10In this method, once a luminance value k is found in the histogram histo where the extended luminance value kexp is equal to or greater than the gamma-rendered target extended luminance value BSTarget', a new value hlcoef[k] is calculated. This new value hlcoef[k] being higher than hlCoef reveals that the attenuation intensity factor hlCoef calculated so far for the previous luminance values ​​in the histogram hlCoef is insufficient to effectively attenuate bright areas of the current SDR image during the ITM process. In fact, in this case, processing module 30 determines that the bright areas comprise more pixels than previously determined for the luminance k. A stronger attenuation is required, which prompts the replacement of the current value of hlCoef with hlCoef[k]. Figure 10 One beneficial effect of this method is that if a small bright area appears in an image and continues to grow in subsequent images, the bright area gradually decays from one image to the next until the proportion of pixels it contains is equal to (or slightly higher than) a percentage P (i.e., when sumOfHiBins is higher than Psum). This achieves a smooth transition between the "decayed" and "undecayed" states. When the proportion of pixels contained in the bright area equals a percentage P, its expansion value is set to 'BSTarget'. This is not the case with EP 3 503 019 A1, where the expansion value decays abruptly, which can produce a flickering effect in the video sequence.

[0218] In one example, assume BSTarget' = 862 and consider a bright area with growth at k = 250, while in the histogram, downwards until TH2, there are no other bins (i.e., the extension value of TH2 is BSTarget'). In this case, Figure 10 The following is an example of how the method is executed:

[0219] • Pixel ratio in bright area = 0 → α = 0 → BSTarget'[Y'] = Yexp' = 250 1.254 =1016→hlCoef=0;

[0220] • Pixel ratio in bright area = 0.25*Psum → α = 0.25 → BSTarget'[Y'] = 0.25*862 + 0.75*1016 = 977 → Gbs(Y') = log(977) / log(250) = 1.247 → hlCoef = 0.0079;

[0221] • Pixel ratio in bright area = 0.5 * Psum → α = 0.5 → BSTarget'[Y'] = 0.5 * 862 + 0.5 * 1016 = 939 → Gbs(Y') = log(939) / log(250) = 1.240 → hlCoef = 0.0158;

[0222] • Pixel ratio in bright area = 0.75 * Psum → α = 0.75 → BSTarget'[Y'] = 0.75 * 862 + 0.25 * 1016 = 900 → Gbs(Y') = log(900) / log(250) = 1.232 → hlCoef = 0.0248;

[0223] • Pixel ratio in bright area > Psum → α = 1 → BSTarget'[Y'] = 862 → Gbs(Y') = log(862) / log(250) = 1.224 → hlCoef = 0.0338;

[0224] As can be seen, the attenuation intensity factor hlCoef increases with the proportion of pixels in the bright area (reaching its value in EP3503019A1). Therefore, a smooth increase in bright spot attenuation is achieved.

[0225] Figure 10 Another effective effect of the method shown is that the attenuation strength factor hlCoef saved in the loop is the maximum attenuation strength factor calculated during the full iteration. This means that the final attenuation strength value hlCoef will be different if there is 75% Psum at "250" (G = 1,254 in this example) and 25% Psum at "240" (G = 1,239 in this example), or if there is 25% Psum at 250 and 75% Psum at 240. In the first case:

[0226] • At Y' = 250, the pixel ratio in the bright area = 0.75 * Psum → α = 0.75 → BSTarget'

[250] = 0.75 * 862 + 0.25 * 1016 = 900 → Gbs(250) = log(900) / log(250) = 1.232 → hlCoef

[250] = 0.0248;

[0227] • At Y' = 240, the pixel ratio in the bright area = 0.75 * Psum + 0.25 * Psum = Psum → BSTarget'

[240] = 862 → Gbs(240) = log(862) / log(240) = 1.233 → hlCoef

[240] = 0.0086;

[0228] For Y' = 250, select hlCoef: hlCoef = 0.0248;

[0229] In the second case:

[0230] • At Y' = 250, the pixel ratio in the bright area = 0.25 * Psum → α = 0.25 → BSTarget'[Y'] = 0.25 * 862 + 0.75 * 1016 = 977 → Gbs(Y') = log(977) / log(250) = 1.247 → hlCoef = 0.0079;

[0231] • At Y' = 240, the pixel ratio in the bright area = 0.25*Psum + 0.75*Psum = Psum → BSTarget'

[240] = 862 → Gbs(240) = log(862) / log(240) = 1.233 → hlCoef

[240] = 0.0086;

[0232] For Y' = 240, select hlCoef: hlCoef = 0.0086;

[0233] This means that the amount of bright spot decay (i.e., the value of the decay intensity factor hlCoef) depends on the statistical distribution of pixel brightness values ​​between TH1 and "255" in the histogram histo. At each iteration, the number of pixels corresponding to the current value of k is added to sumOfHiBins, and the method maintains a "memory" of what happened when the highest value of hlCoef[k] was stored.

[0234] In one implementation where the ITM curve becomes very high, exceeding the maximum authorized extended luminance value Lmax (e.g., Lmax = 1000 nits), α is replaced with α when kexp > Lmax. Lmax / kexp .

[0235] In an implementation suitable for allowing some highlights (e.g., stars in the sky, or sparks, or any kind of very small bright object), α is calculated as follows:

[0236] α=(SumOfHiBins-specularPassThrough) / (Psum–specularPassThrough)

[0237] The specularPassThrough is a fraction of Psum, for example equal to 20%. Then, if at value k, SumOfHiBins < specularPassThrough, α becomes negative and is set to zero. In this case, BSTarget[k] is equal to Kexp, and thus the attenuation intensity factor hlCoef is equal to zero. As long as sumOfHiBins is below specularPassThrough, highlight attenuation is not applied. specularPassThrough = 0 allows the retrieval Figure 10 of embodiments.

[0238] As described above, due to the method of calculating the attenuation intensity factor hlCoef, Figure 4 the method allows reducing the flickering effect in the video sequence to which the ITM process is applied. In one embodiment, additional improvement is obtained by introducing temporal filtering in the Figure 4 method. In this embodiment, the histogram histo is used to detect scene cuts in the video sequence by measuring the distance between the histograms of two consecutive images. As described above, the attenuation intensity factor hlCoef is used to calculate the modified gain function Gbs() to be applied to the current SDR image using Equation 5:

[0239] Gbs(Y') = G(Y') - hlCoef‘*(Y′ / 255) γ

[0240] The above equation can be written as follows:

[0241] G(Y') - hlCoef[hlCoefPos]*(Y′ / 255) γ

[0242] If a scene cut is detected in the current SDR image, the processing module 30 stores hlCoef and hlCoefPos in the variables hlCoefRec and the variable hlCoefPosRec respectively.

[0243] If no scene clipping is detected in the current SDR image, the processing module 30 stores hlCoef and hlCoefPos in the variables hlCoefCur and hlCoefPosCur, respectively. In this case, the processing module 30 mixes hlCoefCur (hlCoefPosCur) with hlCoefRec (hlCoefPosRec) calculated on SDR images before the current SDR image and after the SDR image on which the last scene clipping was detected to generate a new value for hlCoefRec (hlCoefPosRec). Depending on the relative values ​​of hlCoefCur and hlCoefRec, the mixing process can take two forms:

[0244] 1.hlCoefCur≥hlCoefRec: In this case, more bright spot attenuation must be applied to the current SDR image.

[0245] • If hlCoefRec = 0 (no bright spot attenuation is applied to the SDR images preceding the current SDR image until the last scene is clipped), then hlCoefPosRec = hlCoefPosCur.

[0246] • If hlCoefRec≠0 (some bright spot attenuation has been applied to the SDR images before the current SDR image until the last scene is clipped), then hlCoefPosRec=hlCoefPosCur*bsAttack+hlCoefPosRec*(1–bsAttack).

[0247] In both cases: hlCoefRec = hlCoefCur * bsAttack + hlCoefRec * (1 – bsAttack). bsAttack is a weighting factor between zero and one. The closer bsAttack is to one, the greater the impact of the new values ​​of hlCoef and hlCoefPos. In one example, bsAttack is set to 0.05 at 25 frames per second to achieve a smoothing effect in bright areas.

[0248] 2.hlCoefCur < hlCoefRec: In this case, less highlight attenuation must be applied to the current SDR image. hlCoefRec = hlCoefCur * bsRelease + hlCoefRec * (1 – bsRelease). If hlCoefCur ≠ 0: hlCoefPosRec = hlCoefPosCur * bsRelease + hlCoefPosRec * (1 – bsRelease). bsRelease is a value between zero and one. The closer bsRelease is to one, the greater the impact of the new values of hlCoef and hlCoefPos. In one example, bsRelease = 0.05 and 25 frames per second are used to obtain a smooth effect in the bright area.

[0249] In an embodiment with temporal filtering, hlCoefRec replaces hlCoef in Equation 5, thus allowing the calculation of Gbs().

[0250] The monotonicity of the ITM function is an important property. In some cases, the use of the modified gain function Gbs() can prevent the ITM function from being monotonic. Monotonicity loss may occur when the extended target brightness value BSTarget is very low, or when the original ITM curve becomes very high, for very high attenuation gain factor values of hlCoef and / or γ values. Two different situations may occur: the curve is not monotonic at the highest brightness value Y', or the curve loses monotonicity for some intermediate brightness value Y' and resumes monotonicity at higher brightness values Y'.

[0251] Figure 13A A typical case where the ITM curve (shown as a dashed line) is not monotonic is shown.

[0252] In Figure 13A , the first curve shown as a solid line represents the ITM curve without highlight attenuation, and the second curve shown as a dashed line represents the non - monotonic curve after attenuation of the first curve.

[0253] In this example, hlCoef and γ, especially γ, have very high values: hlCoef = 0.07 and γ = 12.

[0254] If hlCoefPos is equal to "255", which means hlCoefPos is calculated at the brightness value Y' = 255, the processing module 30 only needs to decrease γ to restore monotonicity (if Y' = 255, the value of Yexp' does not change when γ changes). For example, using γ = 9, the extended curve shown as a dashed line is monotonic again.

[0255] Figure 13BThe correction of the ITM curve is shown, which allows for ensuring the monotonicity of the ITM curve.

[0256] exist Figure 13B In the middle, the first curve and Figure 13A The first curve in the diagram is the same. The curve represented by the dashed line is the result of modifying the parameter γ in the ITM function, thus allowing the acquisition of... Figure 13A The second curve in the diagram.

[0257] If hlCoefPos ≠ 255, then processing module 30 needs to modify both hlCoef and γ to restore monotonicity, while maintaining the same Yexp' value at hlCoefPos. For example, if hlCoefPos = 243 and Yexp' = 748, then the processing module needs to decrease γ and hlCoef to restore monotonicity. This can be done recursively. For example, if hlCoef = 0.063 and γ = 9.8, the curve is monotonic again, while Yexp' remains the same value (Yexp' = 748).

[0258] In one implementation scheme Figure 9B The process is supplemented by optional step 452. During step 452, the monotonicity of the ITM curve obtained using bright spot decay is tested, and corrections are applied to the ITM curve if necessary to ensure the monotonicity of the ITM curve.

[0259] Figure 14 This schematically illustrates a method that allows ensuring the monotonicity of the ITM curve.

[0260] In step 4521, processing module 30 determines whether the ITM curve obtained by introducing the modified gain function Gbs() in equation 4 is monotonic. If Yexp(k) ≤ Yexp(k+1) for any k between zero and “254”, then the ITM curve is monotonic.

[0261] If the ITM curve is monotonic, no correction is applied to the ITM function in step 4525.

[0262] If the ITM function is not monotonic, then in step 4522, processing module 30 determines whether hlCoefPos = 255. If so, processing module 30 only modifies γ. For example, γ is recursively decreased until the ITM function is monotonic.

[0263] If hlCoefPos≠255, then processing module 30 modifies hlCoef and γ to ensure the monotonicity of the ITM curve, while maintaining the same extended luminance value Yexp' at the luminance value Y' = hlCoefPos. This can be accomplished by applying a recursive process that alternately decreases hlCoef and γ.

[0264] The foregoing describes several embodiments. Features of these embodiments may be provided individually or in any combination. Furthermore, embodiments may include one or more of the following features, devices, or aspects, individually or in any combination, across various claim classes and types:

[0265] • A television set, set-top box, mobile phone, tablet computer or other electronic device that performs at least one of the described implementation schemes.

[0266] • A television, set-top box, mobile phone, tablet computer, or other electronic device that performs at least one of the described implementation schemes and displays the resulting image (e.g., using a monitor, screen, or other type of display).

[0267] A television, set-top box, mobile phone, tablet computer, or other electronic device that tunes (e.g., uses a tuner) a channel to receive signals including encoded images or video streams and performs at least one of the described embodiments.

[0268] A television, set-top box, mobile phone, tablet computer, or other electronic device that receives signals including encoded images or video streams over the air (e.g., using an antenna) and performs at least one of the described embodiments.

Claims

1. A method for inverse tone mapping, wherein the method comprises: Get the current image; Obtain the initial gain function of the first inverse tone mapping function; Obtain the target brightness value; In response to at least one pixel in the current image whose luminance value is at least equal to the target luminance value having an extended value higher than the target extended luminance value due to the application of the first inverse tone mapping function to the current image, a second inverse tone mapping function is applied to the current image, the second inverse tone mapping function corresponding to the first inverse tone mapping function, wherein the gain function has been replaced by a modified gain function; the modified gain function is a function derived from the initial gain function, wherein the gain provided by the initial gain function is attenuated by an attenuation function, the attenuation function being an increasing function of luminance values ​​weighted by a weighting factor controlling the intensity of the attenuation, the weighting factor depending on the statistical distribution of luminance values ​​between the maximum luminance value and the target luminance value in the histogram of the current image.

2. The method according to claim 1, wherein the method comprises: Browse the bar chart in descending order of brightness value, starting from the maximum brightness value; An intermediate weighting factor is calculated for each viewed brightness value when the viewed brightness value is at least equal to the target brightness value and the result of applying the first inverse tone mapping function to the viewed brightness value is at least equal to the target extended brightness value. as well as, The value of the weighting factor is set to the value corresponding to the maximum value of the calculated intermediate weighting factor.

3. The method of claim 2, wherein the intermediate weighting factor depends on an intermediate extended target brightness value, the intermediate extended target brightness value being a weighted sum between the target extended brightness value and the result of applying the first inverse tone mapping function to the viewed brightness value, each weight of the weighted sum depending on a value representing the proportion of pixels whose brightness value of the current image is higher than the viewed brightness value.

4. The method of claim 3, wherein the value representing the proportion of pixels whose brightness value of the current image is higher than the brightness value viewed further depends on the maximum authorized extended brightness value.

5. The method according to claim 3, wherein the method includes applying the first inverse tone mapping function to the current image when the number of pixels whose brightness value is higher than the brightness value being viewed is less than a predefined minimum number of pixels.

6. The method according to any preceding claim, wherein the current image belongs to a video sequence and the method includes detecting scene cuts in the video sequence, wherein the weighting factor controlling the intensity of the attenuation further depends on at least one other weighting factor controlling the intensity of the attenuation calculated for another image preceding the current image in the video sequence, and the scene cut has not yet been detected between the current image and the other image.

7. The method according to any one of claims 1 to 5, the method comprising verifying the monotonicity of a first inverse tone mapping curve obtained using the second inverse tone mapping function, and modifying at least one parameter of the second inverse tone mapping function to obtain a monotonic second inverse tone mapping curve when the first inverse tone mapping curve is not monotonic.

8. The method according to any one of claims 1 to 5, wherein the target brightness value depends on a predetermined percentage of the pixels of the current image.

9. An apparatus for inverse tone mapping, wherein the apparatus includes an electronic circuit system adapted to: Get the current image; Obtain the initial gain function of the first inverse tone mapping function; Obtain the target brightness value; In response to at least one pixel in the current image whose luminance value is at least equal to the target luminance value having an extended luminance value higher than the target extended luminance value due to the application of the first inverse tone mapping function to the current image, a second inverse tone mapping function is applied to the current image, the second inverse tone mapping function corresponding to the first inverse tone mapping function, wherein the gain function has been replaced by a modified gain function; the modified gain function is a function derived from the initial gain function, wherein the gain provided by the initial gain function is attenuated by an attenuation function, the attenuation function being an increasing function of luminance values ​​weighted by a weighting factor controlling the intensity of the attenuation, the weighting factor depending on the statistical distribution of luminance values ​​between the maximum luminance value and the target luminance value in the histogram of the current image.

10. The device according to claim 9, wherein the electronic circuit system is further adapted to: Browse the bar chart in descending order of brightness value, starting from the maximum brightness value; An intermediate weighting factor is calculated for each viewed brightness value when the viewed brightness value is at least equal to the target brightness value and the result of applying the first inverse tone mapping function to the viewed brightness value is at least equal to the target extended brightness value; and, The value of the weighting factor is set to the value corresponding to the maximum value of the calculated intermediate weighting factor.

11. The device of claim 10, wherein the intermediate weighting factor depends on an intermediate extended target luminance value, the intermediate extended target luminance value being a weighted sum between the target extended luminance value and the result of applying the first inverse tone mapping function to the browsed luminance value, each weight of the weighted sum depending on a value representing the proportion of pixels whose luminance value of the current image is higher than the browsed luminance value.

12. The device of claim 11, wherein the value representing the proportion of pixels whose brightness value of the current image is higher than the brightness value viewed further depends on the maximum authorized extended brightness value.

13. The device of claim 11, wherein the electronic circuit system is further adapted to: apply the first inverse tone mapping function to the current image when the number of pixels whose brightness value is higher than the brightness value being viewed is less than a predefined minimum number of pixels.

14. The device according to any one of claims 9 to 13, wherein the current image belongs to a video sequence and the electronic circuit system is further adapted to detect scene cuts in the video sequence, the weighting factor controlling the intensity of the attenuation further depends on at least one other weighting factor controlling the intensity of the attenuation calculated for another image preceding the current image in the video sequence, where the scene cut has not yet been detected between the current image and the other image.

15. The device according to any one of claims 9 to 13, wherein the electronic circuit system is further adapted to: verify the monotonicity of a first inverse tone mapping curve obtained using the second inverse tone mapping function, and when the first inverse tone mapping curve is not monotonic, modify at least one parameter of the second inverse tone mapping function to obtain a monotonic second inverse tone mapping curve.

16. The device according to any one of claims 9 to 13, wherein the target brightness value depends on a predetermined percentage of the pixels of the current image.

17. An apparatus comprising the device according to any one of claims 9 to 13.

18. A computer program product comprising program code instructions for implementing the method according to any one of claims 1 to 5.

19. A computer-readable storage medium storing program code instructions for implementing the method according to any one of the preceding claims 1 to 5.

Images