Chroma upscaling for sl-hdrx system's sdr and hdr display conforming signals

Abstract

The invention provides a method comprising obtaining (90) a current RGB image; classifying (92) the colors of the pixels of the current RGB image into a plurality of classes; for each color class, determining (94) data representative of the color class comprising a dominant luminance value representative of a luminance for which colors in the class are dominant and determining (95) a value representative of a chroma gain from the data representative of the color class, the chroma gain being representative of a margin for increasing chroma components in the color class; and encoding (96) the dominant luminance value and the value representative of the gain corresponding to each class as metadata representative of a saturation gain function in a bitstream, the function defining a color correction to be applied to the pixels as a function of the luminance of the pixels.

Description

1. Technical Field

[0001] At least one embodiment of this implementation relates in general to the distribution of HDR video using the SL-HDRx system (x = 1, 2 or 3). 2. Background Technology

[0002] Recent advances in display technology have begun to allow for an expanded dynamic range of color, brightness, and contrast in images to be displayed. The term "image" here refers to image content, such as video, still pictures, or images.

[0003] High Dynamic Range (HDR) video describes video with a dynamic range greater than that of Standard Dynamic Range (SDR) video. HDR video involves capture, production, content / encoding, and display. HDR capture and display are capable of rendering brighter whites and deeper blacks. To accommodate this, the HDR encoding standard allows for higher maximum luminance and uses at least 10 bits of dynamic range (compared to 8 bits for non-professional video and 10 bits for professional SDR video) to maintain accuracy within this extended range.

[0004] While technically "HDR" strictly refers to the ratio between maximum and minimum brightness, the term "HDR video" is also often understood to mean a wide color gamut.

[0005] Despite the emergence of numerous HDR display devices and image cameras capable of capturing images with increased dynamic range, the amount of available HDR content remains very limited. A solution is needed that allows for extending the dynamic range of existing content so that it can be efficiently displayed on HDR display devices.

[0006] The standard SL-HDR1 (ETSI TS 103 433-1 series, latest version v1.3.1) provides direct backward compatibility through the use of metadata that allows the reconstruction of HDR signals from SDR video streams, which can be delivered using existing SDR distribution networks and services. SL-HDR 1 allows for HDR rendering on HDR devices and SDR rendering on SDR devices using a single-layer video stream.

[0007] The standard SL-HDR2 (ETSI TS 103 433-2 series, latest version v1.2.1) is suitable for HDR devices. Standard SL-HDR2 allows the transmission of ST-2084 (also known as PQ (Perceptual Quantizer) or HDR10) streams along with metadata. When a stream is received by a device that is only compatible with ST-2084 but not with the metadata, the latter ignores the metadata and displays the image without knowing all its technical details (depending on the device model and its processing power, color rendering and gradation details may differ from the original source). When a device that supports the ST-2084 format and metadata receives the stream, it displays an optimized image that best matches the content creator's intent.

[0008] The standard SL-HDR3 (ETSI TS 103 433-3 v1.1.1) allows the transmission of HLG (Hybrid Log-Gamma) streams as well as metadata. An SL-HDR3 system comprises an HDR / SDR reconstruction block based on the SL-HDR2 HDR / SDR reconstruction block; that is, it consists of a cascade of HLG to ST-2084 OETF (Photoelectric Transfer Function) converters and SL-HDR2 HDR / SDR reconstruction blocks. The OETF describes the sensor's actions, converting scene brightness into data.

[0009] In the SL-HDRx system, the chromaticity of the SDR and HDR displays can be tuned to fit the signal because color correction adjustment variables are included in the SL-HDRx metadata. These color correction adjustment variable metadata define a piecewise function called the SGF (Saturation Gain Function), which modifies the default color correction function in any SL-HDRx process. Color correction depends on luminance (the Y component of the image signal); that is, color correction modifies the color of a pixel (e.g., the U and V components) based on the luminance of that pixel.

[0010] Typically, SGF metadata defines up to six points using coordinates (sgf_x, sgf_y), where sgf_x represents luminance and sgf_y represents color correction at that luminance. The sgf_x and sgf_y coordinates are, for example, values ​​contained between "0" and "255".

[0011] By default, SGF provides the same default color correction for every luminance value. This default color correction, which is typically defined empirically, results in neutral SDR and HDR display adaptation signals.

[0012] The basic solution for increasing the chromaticity of SDR and HDR display adaptation signals is to globally increase chromaticity by performing different color corrections on each of the luminance values. This solution has some limitations:

[0013] ● Only global color control is available. Therefore, if some colors are already sufficiently saturated, adding color correction to these colors will make them appear oversaturated;

[0014] ● Adding uncontrolled color correction, i.e., excessively increasing the U and V values, may cause the U and V values ​​to go out of range, which may result in the U and V values ​​being clipped, and thus causing reconstruction errors.

[0015] The aim is to overcome the above shortcomings.

[0016] It is particularly desirable to define a method that allows for better control over color correction in the SL-HDR1, SL-HDR2, and SL-HDR3 systems (and also in any subsequent variants of the SL-HDRx system). 3. Summary of the Invention

[0017] In a first aspect, one or more embodiments of the present invention provide a method comprising: obtaining a current RGB image; analyzing the chromaticity components of the current RGB image, the analysis for each pixel of at least one subset of pixels of the current RGB image comprising: deriving a luminance component from the RGB components of the pixel; applying a tone mapping to the derived luminance component to obtain a tone-mapped luminance component; deriving the chromaticity component from the RGB components of the pixel; and applying joint normalization and color correction to the chromaticity component to obtain a corrected-normalized chromaticity component; classifying the colors of the pixels of the current RGB image into a plurality of categories using the tone-mapped luminance component and the corrected-normalized chromaticity component; for each color category, determining data representing the color category including a principal luminance value representing the luminance in which the color is dominant in the category and determining a value representing a chromaticity gain based on the data representing the color category, the chromaticity gain representing a margin for increasing the chromaticity component in the color category; and encoding the principal luminance value and the value representing the gain corresponding to each category into metadata representing a saturation gain function in a bit stream, the function defining a color correction to be applied to the pixel according to the luminance of the pixel.

[0018] In one implementation, the current RGB image is included in the video sequence, and a temporal filter is applied to the information representing the chroma gain based on the chroma gain calculated from the images in the video sequence preceding the current RGB image.

[0019] In one implementation, the temporal filter is reinitialized at the beginning of the video sequence or when a scene change is detected in the video sequence.

[0020] In one implementation, a color category is a color sector in the chromaticity plane surrounding pure primary colors and / or secondary colors.

[0021] In one implementation, the combination of sectors comprehensively covers the chromaticity plane.

[0022] In one implementation, determining the data representing the color category includes obtaining a histogram of pixels based on the brightness values ​​of the color category.

[0023] In one implementation, only pixels corresponding to brightness values ​​that fall within a predefined range are used to obtain the histogram.

[0024] In one implementation, the main luminance value corresponds to the luminance value of the pixel with the largest chromaticity value or the luminance value with the largest chromaticity energy in the histogram. The chromaticity energy of the histogram is calculated by multiplying the number of pixels corresponding to the bin by the luminance value of the largest chromaticity value or the luminance value with the largest average chromaticity energy found in the bin. The average chromaticity energy of the histogram is calculated by multiplying the number of pixels corresponding to the bin by the chromaticity value of the largest chromaticity value found in the bin.

[0025] In a second aspect, one or more embodiments of the present invention provide an apparatus comprising: means for obtaining a current RGB image; means for analyzing the chromaticity components of the current RGB image, the means for analysis for each pixel of at least one subset of pixels of the current RGB image comprising: means for deriving a luminance component from the RGB components of the pixel; means for applying a tone mapping to the derived luminance component to obtain a tone-mapped luminance component; means for deriving the chromaticity component from the RGB components of the pixel; and means for applying joint normalization and color correction to the chromaticity component to obtain a corrected-normalized chromaticity component. The apparatus includes: means for classifying the colors of pixels in a current RGB image into multiple categories using tone-mapped luminance components and corrected normalized chrominance components; means for determining, for each color category, data representing the color category including a principal luminance value indicating the luminance predominance of the color in the category and a value representing a chrominance gain based on the data representing the color category, the chrominance gain representing a margin for increasing the chrominance components in the color category; and means for encoding the principal luminance value and the value representing the gain corresponding to each category into metadata representing a saturation gain function in a bit stream, the function defining a color correction to be applied to the pixel according to the luminance of the pixel.

[0026] In one implementation, the current RGB image is included in a video sequence, and the device includes a temporal filtering device that is applied to information representing chroma gain based on chroma gain information calculated from images in the video sequence preceding the current RGB image.

[0027] In one implementation, the temporal filter is reinitialized at the beginning of the video sequence or when a scene change is detected in the video sequence.

[0028] In one implementation, a color category is a color sector in the chromaticity plane surrounding pure primary colors and / or secondary colors.

[0029] In one implementation, the combination of sectors comprehensively covers the chromaticity plane.

[0030] In one implementation, determining the data representing the color category includes obtaining a histogram of pixels based on the brightness values ​​of the color category.

[0031] In one implementation, only pixels corresponding to brightness values ​​that fall within a predefined range are used to obtain the histogram.

[0032] In one implementation, the main luminance value corresponds to the luminance value of the pixel with the largest chromaticity value or the luminance value with the largest chromaticity energy in the histogram. The chromaticity energy of the histogram is calculated by multiplying the number of pixels corresponding to the bin by the luminance value of the largest chromaticity value or the luminance value with the largest average chromaticity energy found in the bin. The average chromaticity energy of the histogram is calculated by multiplying the number of pixels corresponding to the bin by the chromaticity value of the largest chromaticity value found in the bin.

[0033] In a third aspect, one or more embodiments of the present invention provide a signal generated by the method of the first aspect or by the device of the second aspect.

[0034] In a fourth aspect, one or more embodiments of the present invention provide a computer program comprising program code instructions for implementing the method according to the first aspect.

[0035] In a fifth embodiment, one or more embodiments of the present invention provide an information storage device that stores program code instructions for implementing the method according to the first aspect. 4. Description of the attached drawings

[0036] Figure 1 An example of the SL-HDRx system is shown;

[0037] Figure 2 The post-processing module of the SL-HDRx system is shown schematically.

[0038] Figure 3 An example of a hardware architecture for a processing module capable of implementing various aspects and implementation schemes is illustrated schematically;

[0039] Figure 4A block diagram illustrating an example of a first system in which various aspects and implementation schemes are carried out;

[0040] Figure 5 A block diagram illustrating an example of a second system in which various aspects and implementation schemes are derived is shown;

[0041] Figure 6 The first example of a preprocessing procedure is illustrated schematically;

[0042] Figure 7 A second example of the preprocessing procedure is illustrated schematically;

[0043] Figure 8 An example of the reconstruction process in the post-processing stage is illustrated schematically;

[0044] Figure 9 The method of controlling color correction in the SL-HDRx system is illustrated schematically;

[0045] Figure 10 It represents the three primary colors (red, green, and blue) and the three secondary colors (magenta, yellow, and cyan) and the positions of the corresponding sectors;

[0046] Figure 11 An example of a time-stabilized method is illustrated schematically;

[0047] Figure 12 An example of the initialization phase of a time-stabilized method is illustrated schematically;

[0048] Figure 13 An example illustrating the filter parameter calculation process of the time-stabilized method is shown.

[0049] Figure 14 An implementation scheme for controlling color correction suitable for an SL-HDR2 system is schematically described;

[0050] Figure 15 First details of the method for controlling color correction suitable for the SL-HDR2 system are shown; and,

[0051] Figure 16 The second detail shows the method for controlling color correction suitable for the SL-HDR2 system. 5. Detailed Implementation

[0052] Figure 1An example of an SL-HDRx system is shown. SL-HDRx (x = 1, 2, or 3) includes a preprocessing module 10, an encoding module 12, and a post-processing module 14. The preprocessing module 10 is communicatively connected to the encoding module 12 via a communication link 11. In SL-HDR2 and SL-HDR3 systems, the preprocessing module 10 generates HDR content and dynamic metadata from the raw content, while in the SL-HDR1 system, the preprocessing module generates SDR content and dynamic metadata from the raw content. The preprocessing module 10 integrates a computational part that generates the output HDR or SDR content and an analytical part that analyzes the content and generates dynamic metadata. The raw content may have been generated by an acquisition device (such as a camera) via a computer graphics system or a combination of an acquisition device and a computer graphics system. The HDR or SDR content may include static metadata, such as acquisition device (i.e., camera) parameters representing the acquisition environment of the HDR or SDR content.

[0053] The preprocessing module 10 is fed with HDR content that has been created during the post-production process applied to the original HDR or SDR video to obtain the main video and still metadata. The post-production process includes, for example:

[0054] ● Color grading process, such as introducing artistic effects in the main video;

[0055] ●VFX compositing process, used to introduce visual effects into the main video;

[0056] ● The tone mapping process allows for the generation of SDR master video from HDR video;

[0057] ● The inverse tone mapping process allows the generation of HDR master video from SDR video.

[0058] Then, the preprocessing module 10 generates content and dynamic metadata suitable for SL-HDRx usage. In the SL-HDR1 system, the generated video is SDR video. In the SL-HDR2 system, the generated video is PQ HDR video. In the SL-HDR3 system, the generated video is HLG HDR video.

[0059] The following text is about Figure 6 An example of the preprocessing procedure implemented in the SL-HDR1 system is described.

[0060] The encoding module 12 receives the generated video and metadata from the preprocessing module 10, including both static metadata from post-production and dynamic metadata from SL-HDRx preprocessing, and is responsible for encoding the generated video and metadata.

[0061] Encoding module 12 generates an encoded video stream, for example, based on the generated video and metadata, which conform to the video compression standards HEVC (ISO / IEC 23008-2-MPEG-H Part 2, High Efficiency Video Coding / ITU-T H.265) or AVC (ISO / IEC 14496-10-MPEG-4 Part 10, Advanced Video Coding) or a standard under development called Universal Video Coding (VVC). Metadata is processed, for example, by SEI messages (such as HEVC Color Remapping Information (CRI) or Master Display Color Volume (MDCV) SEI messages).

[0062] In the following text, we will refer to the combination of preprocessing module 10 and encoding module 12 as the input module.

[0063] During encoding, the encoded video is transmitted to the post-processing module 14 via communication link 13.

[0064] Figure 2 The post-processing module 14 is shown schematically.

[0065] The post-processing module 14 includes a decoder 140 adapted to decode the encoded main video and associated metadata.

[0066] In an SL-HDR1 system, the encoded video represents SDR video, and metadata is used to generate HDR video from the SDR video. During decoding, in a post-processing device without SL-HDR1 integration, the SDR video is transmitted to an SDR display device 18 via communication link 17. The SDR display device 18 then displays the decoded SDR video. In a post-processing device with SL-HDR1 integration, the post-processing module 14 includes a reconstruction module 141. The reconstruction module 141 receives the decoded SDR video from the decoder 140 and reconstructs HDR video from the decoded SDR video using metadata. The reconstructed HDR video is then transmitted to an HDR display device 16 that displays the video. In some cases, the display device 16 is not an HDR display device, but an SDR display device or an MDR (Medium Dynamic Range) display device that acts as an intermediary between an SDR and HDR display device. In these cases, the reconstruction module obtains information representing the display capabilities of the MDR display device 16 and considers these capabilities during reconstruction to reconstruct video suitable for the MDR display device. Once reconstructed, the HDR (or SDR or MDR) video is transmitted to the HDR (or SDR or MDR) display device 16 via communication link 15. The HDR (or SDR or MDR) display device 16 then displays the reconstructed HDR (or SDR or MDR) video.

[0067] In an SL-HDR2 system, the encoded video represents PQ HDR video, and metadata is used to generate SDR (or MDR) video from the decoded PQ HDR video. During decoding, in a post-processing device without integrated SL-HDR2, the PQ HDR video is transmitted to an HDR display device 18 via communication link 17. The HDR display device 18 then displays the decoded PQ HDR video. In a post-processing device without integrated SL-HDR2, a reconstruction module 141 receives the decoded PQ HDR video, metadata, and, in some cases, the display capabilities of a display device 16 from a decoder 140. This display device can be an SDR, MDR, or HDR display device. Based on this data, the reconstruction module generates a video signal suitable for the capabilities of the display device 16 (SDR, MDR, or HDR video). Once reconstructed, the SDR (or MDR, or HDR) video is transmitted to the SDR (or MDR, or HDR) display device 16 via communication link 15. The SDR (or MDR, or HDR) display device 16 then displays the reconstructed SDR (or MDR, or HDR) video.

[0068] In the SL-HDR3 system, the encoded video represents HLG HDR video, and metadata is used to generate SDR (or MDR) video from the decoded HLG HDR video. The post-processing module 14 in the SL-HDR3 system functions very similarly to the post-processing module 14 in the SL-HDR2 system. The main difference lies in the reconstruction module 141. In practice, in this case, the reconstruction module 141 comprises a cascade of the HLG to ST-2084 OETF converter and the SL-HDR2 reconstruction module as described above.

[0069] Figure 3An example of the hardware architecture of the processing module 40 is schematically shown. This processing module is contained in the preprocessing module 10, the encoding module 12, the input module, or the postprocessing module 14, and is capable of implementing different aspects and implementation schemes. As a non-limiting example, processing module 40 includes the following items connected by communication bus 405: a processor or CPU (Central Processing Unit) 400 containing one or more microprocessors, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture; random access memory (RAM) 401; read-only memory (ROM) 402; storage unit 403, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drive and / or optical disk drive, or storage media reader, such as SD (Secure Digital) card reader and / or hard disk drive (HDD) and / or network accessible storage device; at least one communication interface 404 for exchanging data with other modules, devices, systems, or equipment. Communication interface 404 may include, but is not limited to, a transceiver configured to transmit and receive data via communication channel 5. Communication interface 404 may include, but is not limited to, a modem or network card.

[0070] Communication interface 404, for example, enables processing module 40 to:

[0071] ●When the processing module 40 is included in the preprocessing module 10, it receives SDR or HDR content and outputs the main video;

[0072] ● When the processing module 40 is included in the encoding module 12, it receives the main video and outputs the encoded main video including metadata;

[0073] ● When the processing module 40 is included in the input module, it receives SDR or HDR content and outputs an encoded main video including metadata;

[0074] ● When the processing module 40 is included in the post-processing module 40, it receives the encoded master video including metadata and outputs SDR, MDR and / or HDR video.

[0075] Processor 400 is capable of executing instructions loaded into RAM 301 from ROM 402, external memory (not shown), storage media, or a communication network. When processing module 40 is powered on, processor 400 is capable of reading instructions from RAM 401 and executing these instructions. These instructions form a computer program that causes, for example, preprocessing, encoding, decoding, or postprocessing procedures to be implemented by processor 400.

[0076] All or part of the algorithms and steps of the process can be implemented in software by executing a set of instructions by a programmable machine such as a DSP (Digital Signal Processor) or a microcontroller, or in hardware by a machine or dedicated component such as an FPGA (Field Programmable Gate Array) or an ASIC (Application-Specific Integrated Circuit).

[0077] Figure 4 A block diagram illustrating an example of System A is shown, which is adapted to implement a preprocessing module 10, an encoding module 12, or an input module, and wherein various aspects and embodiments are implemented. System A may be embodied as a device comprising the various components described below and configured to perform one or more aspects and embodiments described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, connected home appliances, servers, and cameras. The elements of System A may be embodied individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, System A includes a processing module 40 that implements a preprocessing process, an encoding process, or both. In various embodiments, System A is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports.

[0078] Inputs to processing module 40 may be provided by various input modules as shown in box 52. Such input modules include, but are not limited to: (i) a radio frequency (RF) module that receives, for example, RF signals transmitted over the air by a broadcaster; (ii) a component (COMP) input module (or a set of COMP input modules); (iii) a universal serial bus (USB) input module; and / or (iv) a high-definition multimedia interface (HDMI) input module. Figure 4 Other examples not shown include composite video.

[0079] In various embodiments, the input module of block 52 has associated corresponding input processing elements as known in the art. For example, the RF module may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal, or limiting a signal band to a band), (ii) down-converting the selected signal, (iii) re-band-limiting the signal to a narrower band to select (e.g.,) a signal band that may be referred to as a channel in some embodiments), (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select the desired data packet stream. The RF module of various embodiments includes one or more elements for performing these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include tuners that perform various functions among these functions, including, for example, down-converting received signals to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or to baseband. Various implementations rearrange the order of the above (and other) components, remove some of these components, and / or add other components that perform similar or different functions. Adding components may include inserting components between existing components, such as inserting amplifiers and analog-to-digital converters. In various implementations, the RF module includes an antenna.

[0080] Additionally, the USB and / or HDMI modules may include corresponding interface processors for connecting System 3 to other electronic devices across USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented as needed, for example, within a separate input processing IC or within processing module 40. Similarly, various aspects of USB or HDMI interface processing may be implemented as needed, either within a separate interface IC or within processing module 40. Demodulation, error correction, and demultiplexing streams are provided to processing module 40.

[0081] Various components of System A can be housed within an integrated housing. Within the integrated housing, the various components can be interconnected using suitable connection arrangements (e.g., internal buses known in the art, including inter-IC (I2C) buses, wiring, and printed circuit boards) and data can be transferred between these components. For example, in System A, processing module 40 is interconnected with other components of System A via bus 405.

[0082] The communication interface 404 of the processing module 40 allows system A to communicate on communication channel 5. For example, communication channel 5 can be implemented in wired and / or wireless media.

[0083] In various implementations, a wireless network such as Wi-Fi, such as IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers), is used to stream or otherwise provide data to System A. In these implementations, the Wi-Fi signal is received via a communication channel 5 and a communication interface 404 suitable for Wi-Fi communication. The communication channel 5 in these implementations is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other cloud-based communications. Other implementations use the RF connection of input box 52 to provide streaming data to System A. As mentioned above, for example, when System A is a camera, smartphone, or tablet, various implementations provide data in a non-streaming manner. Additionally, various implementations use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0084] System A can use communication channel 5 or bus 405 to provide output signals to various output devices. For example, when implementing preprocessing module 10, system A uses bus 405 or communication channel 5 to provide output signals to encoding module 12. When implementing encoding module 12 or input module, system A uses communication channel 5 to provide output signals to postprocessing module 14.

[0085] Various specific implementations involve application preprocessing and / or encoding processes. As used in this application, the preprocessing or encoding process may encompass all or part of the processes performed, for example, on received SDR or HDR images or video streams, to produce a master video or a master video encoded with metadata. In various implementations related to the encoding process, such processes include one or more processes typically performed by a video encoder (e.g., an H.264 / AVC (ISO / IEC 14496-10-MPEG-4 Part 10, Advanced Video Coding), H.265 / HEVC (ISO / IEC 23008-2-MPEG-H Part 2, High Efficiency Video Coding / ITU-T H.265), or H.266 / VVC (Universal Video Coding) encoder developed by a joint collaboration of ITU-T and ISO / IEC experts known as the Joint Video Experts Group (JVET).

[0086] Figure 5A block diagram of an example system B is shown, which is adapted to implement post-processing module 14 and therein implement various aspects and embodiments. System B may be embodied as a device including the various components described below and configured to perform one or more aspects and embodiments described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. The elements of system B may be embodied individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, system B includes a processing module 40 that implements the post-processing procedure. In various embodiments, system B is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports.

[0087] Inputs to processing module 40 may be provided by various input modules as shown in box 52. Such input modules include, but are not limited to: (i) a radio frequency (RF) module that receives, for example, RF signals transmitted over the air by a broadcaster; (ii) a component (COMP) input module (or a set of COMP input modules); (iii) a universal serial bus (USB) input module; and / or (iv) a high-definition multimedia interface (HDMI) input module. Figure 5 Other examples not shown include composite video.

[0088] In various embodiments, the input module of block 52 has associated corresponding input processing elements as known in the art. For example, the RF module may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal, or limiting a signal band to a band), (ii) down-converting the selected signal, (iii) re-band-limiting the signal to a narrower band to select (e.g.,) a signal band that may be referred to as a channel in some embodiments), (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select the desired data packet stream. The RF module of various embodiments includes one or more elements for performing these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include tuners that perform various functions among these functions, including, for example, down-converting received signals to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or to baseband. In one set-top box implementation, the RF module and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to the desired frequency band. Various implementations rearrange the order of the aforementioned (and other) components, remove some of these components, and / or add other components that perform similar or different functions. Adding components may include inserting components between existing components, such as inserting amplifiers and analog-to-digital converters. In various implementations, the RF module includes an antenna.

[0089] Additionally, the USB and / or HDMI modules may include corresponding interface processors for connecting System B to other electronic devices across USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented as needed, for example, within a separate input processing IC or within processing module 40. Similarly, various aspects of USB or HDMI interface processing may be implemented as needed, either within a separate interface IC or within processing module 40. Demodulation, error correction, and demultiplexing streams are provided to processing module 40.

[0090] Various components of System B may be housed within an integrated housing. Within the integrated housing, the various components can be interconnected using suitable connection arrangements (e.g., internal buses known in the art, including inter-IC (I2C) buses, wiring, and printed circuit boards) and data can be transferred between these components. For example, in System B, processing module 40 is interconnected with other components of System B via bus 405.

[0091] The communication interface 404 of the processing module 40 allows system B to communicate on communication channel 5. For example, communication channel 5 can be implemented in wired and / or wireless media.

[0092] In various implementations, a wireless network such as Wi-Fi, such as IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers), is used to stream or otherwise provide data to System B. In these implementations, the Wi-Fi signal is received via a communication channel 5 and a communication interface 404 suitable for Wi-Fi communication. The communication channel 5 in these implementations is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other cloud-based communications. Other implementations use the RF connection of input box 52 to provide streaming data to System B. As mentioned above, various implementations provide data in a non-streaming manner. Additionally, various implementations use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0093] System B can provide output signals to various output devices, including a display 5, a speaker 6, and other peripheral devices 7. The display 5 in various embodiments includes, for example, one or more of a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display 5 can be, for example... Figure 1 Display device 16 or 18. Display 5 can be used in a television, tablet, laptop, cellular phone (mobile phone), or other device. Display 5 can also be integrated with other components (e.g., in a smartphone) or standalone (e.g., an external monitor for a laptop). Display device 5 is compatible with SDR, MDR, or HDR content. In various examples of embodiments, other peripheral devices 7 include one or more of a standalone digital video disc (or digital universal disc, both terms being DVR), disc player, stereo system, and / or lighting system. Various embodiments use one or more peripheral devices 7 that provide functionality based on the output of system B. For example, a disc player performs the function of playing the output of system B.

[0094] In various implementations, control signals are transmitted between System B and display 5, speaker 6, or other peripheral devices 7 using signaling protocols such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable device-to-device control with or without user intervention. Output devices are communicatively coupled to System B via dedicated connections through corresponding interfaces 53, 54, and 55. Alternatively, output devices can be connected to System B via communication interface 404 using communication channel 5. Display 5 and speaker 6 can be integrated into a single unit with other components of System B in electronic devices such as televisions. In various implementations, display interface 5 includes a display driver, such as, for example, a timing controller (TCon) chip.

[0095] For example, if the RF portion of input 52 is part of a separate set-top box, then display 5 and speaker 6 may optionally be separate from one or more other components. In various embodiments where display 5 and speaker 6 are external components, the output signal may be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0096] Various specific implementations involve post-processing processes within the decoding process. As used in this application, post-processing may encompass all or part of the processes performed, for example, on the received encoded master video, to produce an SDR, MDR, or HDR output suitable for display. In various implementations, such processes include one or more processes typically performed by an image or video encoder (e.g., an H.264 / AVC (ISO / IEC 14496-10-MPEG-4 Part 10, Advanced Video Coding), H.265 / HEVC (ISO / IEC 23008-2-MPEG-H Part 2, High Efficiency Video Coding / ITU-T H.265) or / and H.266 / VVC (Universal Video Coding) decoder developed by a joint collaboration of experts from ITU-T and ISO / IEC known as the Joint Video Experts Group (JVET).

[0097] When the accompanying drawings are presented as flowcharts, it should be understood that block diagrams of the corresponding devices are also provided. Similarly, when the accompanying drawings are presented as block diagrams, it should be understood that flowcharts of the corresponding methods / processes are also provided.

[0098] The specific embodiments and aspects described herein may be implemented, for example, in methods or processes, apparatus, software programs, data streams, or signals. Even if discussed only in the context of a single form of specific embodiment (e.g., discussed only as a method), specific embodiments of the discussed features may also be implemented in other forms (e.g., apparatus or program). Apparatus may be implemented, for example, in suitable hardware, software, and firmware. Methods may be implemented, for example, in a processor that generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices, such as, for example, computers, mobile phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0099] The reference to "an implementation scheme" or "implementation scheme" or "a specific implementation" or "specific implementation," and other variations thereof, means that the specific features, structures, characteristics, etc., described in connection with the implementation scheme are included in at least one implementation scheme. Therefore, the appearance of the phrase "in an implementation scheme" or "in an implementation scheme" or "in a specific implementation" or "in a specific implementation," and any other variations appearing throughout this application, do not necessarily refer to the same implementation scheme.

[0100] Additionally, this application may involve "determining" various types of information. Determining information may include, for example, estimated information, calculated information, predicted information, information retrieved from memory, or information obtained, for example, from another device, module, or user, one or more of these.

[0101] Furthermore, this application may relate to "accessing" various types of information. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information, or more of these.

[0102] Furthermore, this application may relate to "receiving" various types of information. Like "access," "receiving" is intended to be a broad term. Receiving information may include, for example, accessing information or retrieving information (e.g., from memory) or more. Moreover, "receiving" typically involves one or more of the following during operations such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0103] It should be understood that, for example, in the cases of “A / B,” “A and / or B,” “at least one of A and B,” and “one or more of A and B,” the use of any of the following “ / ,” “and / or,” and “at least one,” “one or more” is intended to cover selecting only the first listed option (A), or only the second listed option (B), or selecting both options (A and B). As a further example, in the cases of “A, B, and / or C,” “at least one of A, B, and C,” and “one or more of A, B, and C,” such phrases are intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or selecting all three options (A, B, and C). As will be apparent to those skilled in the art and related fields, this can be extended to as many of the listed items as possible.

[0104] It will be apparent to those skilled in the art that specific embodiments or implementations can produce various signals formatted to carry, for example, information that can be stored or transmitted. The information may include, for example, instructions for performing a method or data generated by one of the embodiments or implementations. For example, a signal may be formatted to carry an SDR or HDR image or video sequence of the embodiment. Such signals may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or baseband signals. Formatting may include, for example, encoding the SDR or HDR image or video sequence using metadata from an encoded video stream and using a stream-modulated carrier. The information carried by the signal may be, for example, analog or digital information. It is known that signals can be transmitted via various wired or wireless links. The signal may be stored on a processor-readable medium.

[0105] Figure 6 An example of the computational portion of the preprocessing procedure is schematically illustrated. The first example of the preprocessing procedure is suitable for an SL-HDR1 system in NCL (Non-Constant Luminosity) mode. In this example, preprocessing module 10 receives HDR content and generates a master video representing the SDR content and metadata. The preprocessing procedure is performed by processing module 40 on each pixel of each image of the HDR content. Figure 6 In the example, the pixel includes three color components corresponding to the three primary colors of red (R), green (G) and blue (B), that is, the pixel is an RGB signal.

[0106] In step 601, the processing module 40 derives the luminance (luma) component L' from the RGB signal, as shown below:

[0107]

[0108] Where A1 is the transformation matrix and γ is the gamma factor, for example, equal to "2.4".

[0109] In step 602, the processing module 40 applies tone mapping to the luminance component L' to obtain the tone mapping value Y. pre0 As shown below:

[0110] Y pre0 =LUT TM (L′) (Equation 2)

[0111] Where Y pre0 ∈[0; 1023] and LUT TM () is a lookup table representing tone mapping functions.

[0112] In step 603, processing module 40 applies gamma correction to the RGB signal, as shown below:

[0113]

[0114] In step 604, processing module 40 derives the chroma component from the gamma-corrected RGB signal, as follows:

[0115]

[0116] Where A2 and A3 are transformation matrices. [A1A2A3] T This is, for example, a standard 3x3 RGB to YUV conversion matrix (e.g., as specified in ITU-R Rec.BT.2020 or ITU-R Rec.BT.709, depending on the color space).

[0117] In step 605, processing module 40 applies joint normalization and color correction to the chromaticity component U. pre0 and V pre0 To obtain the normalized corrected chromaticity component U pre1 and V pre1 As shown below:

[0118]

[0119] U pre1 and V pre1 It is clipped within [-512; 511]. It's a scaling function:

[0120]

[0121] And when Then:

[0122]

[0123] Where lutCC[] is a color correction lookup table, as defined, for example, in section 7.2.3.2 of the document ETSI TS 103 433-1 v1.3.1.

[0124] In step 606, the processing module 40 adjusts the hue mapping brightness value Y. pre0 To obtain the adjusted tone map brightness value Y pre1 As shown below:

[0125] Y pre1 =Y pre0 -max(0, aU) pre1 +bV pre1 (Equation 6)

[0126] In step 607, the processing module 40 converts the luminance and chromaticity values ​​Y... pre1U pre1 and V pre1 Convert to the given output format. Step 607 includes converting the chromaticity component U... pre1 and V pre1 The sub-step of adding a value of, for example, "512" for `midsample`, optionally a sub-step of downsampling the chroma components, which compresses the signal by reducing the number of chroma samples, and optionally a sub-step of converting from a full range of values ​​(ranging from "0" to "1023" for YUV components when encoded in 10 bits) to a finite range of values ​​(ranging from 64 to 940 for the Y component and from 64 to 960 for the UV component) to obtain the luminance and chroma components Y of the pixels representing the SDR signal. sdr U sar V sdr The purpose of step 607 is, for example, to convert a full-range YUV 444 signal into a limited-range YUV 420 signal.

[0127] Figure 7 A second example of the computational portion of the preprocessing procedure is schematically illustrated. This second example of the computational portion of the preprocessing procedure is suitable for an SL-HDR2 system. In this example, the preprocessing module 10 receives HDR content and generates a master video representing the HDR PQ signal and metadata. The preprocessing procedure is performed by the processing module 40 on each pixel of each image of the input HDR content. Figure 7 In the example, the pixels are also RGB signals.

[0128] In step 701, the processing module 40 obtains the luminance and chromaticity components Y of the PQ signal from the RGB signal. pre0 U pre0 V pre0 :

[0129]

[0130] Where A1 is the conversion function, and R″ (G″ and B″ respectively) are obtained from the RGB signal using an RGB to PQOETF converter.

[0131] In step 702, the processing module 40 processes the luminance and chromaticity values ​​Y. pre0 U pre0 and V pre0 Convert to the output format. Step 702 includes converting the chromaticity component U... pre0 and V pre0The sub-steps include adding a value of, for example, "512" for `midsample`, optionally a sub-step for downsampling the chroma components, which compresses the signal by reducing the number of chroma samples, and optionally a sub-step for converting from a full range of values ​​(ranging from "0" to "1023" for YUV components when encoded in 10 bits) to a finite range of values ​​(ranging from 64 to 940 for the Y component and from 64 to 960 for the UV component) to obtain the luminance and chroma components Y of the pixels representing the HDR signal. sdr U sdr V sdr The purpose of step 702 is, for example, to convert a full-range YUV 444 signal into a limited-range YUV 420 signal.

[0132] Figure 8 An example of the reconstruction process of the post-processing procedure is illustrated schematically. When the processing module 40 implements the post-processing module 14 and more specifically the reorganization module 141, it is executed by the processing module 40. Figure 8 The reconstruction process is applied to each pixel of the decoded master video generated by decoder 140. The following describes the adaptation of the reconstruction process to SL-HDR1 and SL-HDR2. Since the SL-HDR3 specification is based on the SL-HDR2 specification, all SL-HDR2 features are also applicable to SL-HDR3 as described below. Figure 8 The reconstruction process follows, for example Figure 6 or Figure 7 The preprocessing process is used to determine the input signal for the reconstruction process. Therefore, the reconstruction process receives a limited range of YUV 420 signals.

[0133] In step 801, the processing module 40 converts the received YUV 420 signal into a full-range YUV 444 signal (the reverse process of steps 607 and 702).

[0134] In the case of the SL-HDR1 system, the YUV 420 signal is SDR pixels. Once converted, the SDR pixels consist of luminance and chrominance components. y SDR cb SDR cr express.

[0135] In the case of the SL-HDR2 system, the YUV 420 signal is the HDR pixel. Once converted, the HDR pixel consists of luminance and chrominance components. y HDR cb HDR cr express.

[0136] After conversion, the processing module 40 centers the chroma components to obtain the centered chroma component U. post1 and Vpost1 In the case of the SL-HDR1 system, centering is performed as follows:

[0137]

[0138] In the case of the SL-HDR2 system, centering is performed as follows:

[0139]

[0140] Where midsample is, for example, equal to "512".

[0141] In step 802, processing module 40 readjusts the parameters applied to the luminance component. In the case of the SL-HDR1 system, the readjustment calculation is as follows:

[0142] Y post1 =SDR y +max(0;mu0×U post1 +mu1×V post1 )

[0143] The parameters mu0 and mu1 are defined in section 7.2.4 of the document ETSI TS 103 433-1 v1.3.1, and max(x, y) takes the maximum value of x and y.

[0144] In the case of the SL-HDR2 system, readjusting the calculations is even simpler:

[0145] Y post1 =HDR y

[0146] In SL-HDR1 and SL-HDR2, the luminance value y post1 Then it is clipped within [0; 1023] to obtain y. post2 .

[0147] In step 803, the processing module 40 constructs a color correction lookup table lutCC[Y].

[0148] In the case of SL-HDR1, the construction of the color correction lookup table is specified in section 7.2.3.2 of document ETSI TS 103 433-1 v1.3.1:

[0149]

[0150] Where, lutCC[0] = 0.125 and R sgf g(Y) n ), L(Y nThis is specified in the document ETSI TS 103 433-1 v1.2.1.

[0151] In the case of SL-HDR2, the construction of the color correction lookup table is specified in section 7.2.3.2 of document ETSI TS 103 433-2 v1.2.1:

[0152]

[0153] Where lutCC[0] = 0.125 and c(L) HDR L SDR L pdisp ), R sgf g(Y) n `maxsampleVal` is specified in document ETSI TS 103 433-2 v1.2.1.

[0154] In both SL-HDR1 and SL-HDR2 cases, g(Y) n ) is defined as:

[0155] g(Y n )=f sgf (Y n )×modFactor+(1-modFactor)÷R sgf

[0156] Where the saturation gain function It is derived from the piecewise linear pivot points defined by the metadata of the saturation gain function sgf_x and sgf_y, as detailed in section 7.3 of document ETSI TS 103 433-1 v1.3.1.

[0157] In step 804, the processing module 40 uses the constructed color correction lookup table lutCC[y] to check the centered chromaticity component U. post1 and V post1 Apply inverse color correction.

[0158] In the case of SL-HDR1, inverse color correction is described in section 7.2.4 of document ETSI TS 103 433-1 v1.3.1:

[0159]

[0160] In the case of SL-HDR2, inverse color correction is described in section 7.2.4 of document ETSI TS 103 433-2 v1.2.1:

[0161]

[0162] In step 805, the processing module calculates the intermediate values ​​s0 and u. 柱3 and v 柱3 In the case of SL-HDR1, the variable T is calculated as follows:

[0163] T = k o ×U post2 ×V post2 +k1×U post2 ×U post2 +k2×V post2 ×V post2

[0164] k1 and k2 are described in section 7.2.4 of document ETSI TS 103 433-1V1.2.1.

[0165] If T≤1, Then U post3 =U post2 And V post3 =V post2 .

[0166] Otherwise, if T > 1 and S0 = 0, then U post3 and V post3 The following derivation shows:

[0167]

[0168] This last equation applies only to SL-HDR1 "CL" mode. In SL-HDR1 "NCL" mode and SL-HDR2, k o =k1=k2=0, s0=1, and U post3 =U post2 and V post3 =V post2 .

[0169] It can be noted that Y post2 Corresponding to Y pre0 U post1 and V post1 Corresponding to U pred1 and V pred1 .

[0170] In step 806, the processing module calculates the intermediate RGB reconstructed values ​​R2, G2, and B2. This is done in two steps.

[0171] In the case of SL-HDR1, R1, G1, and B1 are described in section 7.2.4 of document ETSI TS 103 433-1V1.3.1:

[0172]

[0173] Where m i =matrixCoefficient[i] is part of the SL-HDRx metadata described in section 6.3.2.6 of document ETSI TS 103 433-1v1.3.1.

[0174] In the case of SL-HDR2, R1, G1, and B1 are described in section 7.2.4 of document ETSI TS 103 433-2 v1.2.1:

[0175]

[0176] Where m i =matrixCoefficient[i] is part of the SL-HDRx metadata described in section 6.3.2.6 of document ETSI TS 103 433-1 v1.3.1.

[0177] In the second step, the intermediate values ​​R2, G2, and B2 are calculated as described in Section 7.2.4 of the documents ETSI TS 103 433-1 v1.3.1 (for SL-HDR1) and ETSI TS 103 433-2 v1.2.1 (for SL-HDR2):

[0178]

[0179] LutMapY calculations are described in detail in section 7.2.3.1 of the documents ETSI TS 103 433-1 v1.3.1 (for SL-HDR1) and ETSI TS 103433-2 v1.2.1 (for SL-HDR2).

[0180] It should be noted that in the case of SL-HDR 1, LutMapY is used as an inverse tone mapping lookup table, which, in the case of adapting the display, will convert the SDR of the SL-HDR1 post-processor to SDR. y The input luminance signal is converted into an HDR output signal, or an SDR or MDR output signal.

[0181] In the case of SL-HDR2, LutMapY is used as a tone mapping lookup table, which, in accordance with display adaptation, will adjust the HDR of the SL-HDR2 post-processor. y The input luminance signal is converted into an HDR output signal, or an SDR or MDR output signal.

[0182] In step 807, the output HDR RGB reconstructed signal HDR is calculated. RHDR G and HDR B .

[0183] In the case of SL-HDR1, the calculations are described in section 7.2.4 of document ETSI TS 103 433-1 v1.3.1:

[0184]

[0185] In the case of SL-HDR2, the calculations are described in section 7.2.4 of document ETSI TS 103 433-2 v1.2.1:

[0186]

[0187] Therefore, the sgf_x and sgf_y metadata can be used to control the color correction of the SL-HDRx system. This metadata is linked to the color correction lookup table lutCC[Y] which controls the chroma component saturation. post2 [It has an impact. In the case of SL-HDR1, the sgf_x and sgf_y metadata will control U in step 605 of the HDR decomposition process.] pre1 and V pre1 The generation of U, and therefore will control U sdr V sdr SDR output. In the case of SL-HDR2, the sgf_x and sgf_y metadata will be controlled by U in step 804 of processing module 40. post2 and V post3 The generation of chroma will thus control the chroma output of the SL-HDR2 reconstruction process.

[0188] Figure 9 The diagram schematically illustrates a method for controlling color correction in an SL-HDRx system. When processing module 40 implements preprocessing module 10 or input module, it performs the following actions regarding... Figure 9 The method described. This method is applied to each image or video. Figure 9 The method is described in the context of the SL-HDR1 system in NCL mode. Processing module 40 receives HDR content.

[0189] In step 90, the processing module 40 obtains the current image of the HDR content.

[0190] In step 91, processing module 40 analyzes the chromaticity of the current image. To this end, processing module 40 applies... Figure 6 The process continues until step 605 to obtain the three color components Y of each pixel in the current image. pre0 U pre1 and Vpre1 .

[0191] In step 92, the processing module 40 classifies the colors of the pixels in the current image based on the three components Y in the color category. pre0 U pre1 and V pre1 This indicates that, in one embodiment of step 92, six categories are used:

[0192] ●The three categories correspond to the three primary colors: red, green, and blue;

[0193] ●The three categories correspond to three secondary colors: magenta, cyan, and yellow.

[0194] Each color can be represented using many different color spaces. In one implementation, RGB and its polar coordinates hue (H) and chromaticity (C) are used. Hue (H) and chromaticity (C) are calculated as follows:

[0195]

[0196]

[0197] In the case of SL-HDR1

[0198]

[0199]

[0200] An orientation is defined for each of the three primary and three secondary colors using the following formula in polar coordinates, where "c" in [0 ... 1] is the normalized value of each RGB value (c = 1 corresponds to a primary or secondary color, while c = 0 is the achromatic origin of the UV plane):

[0201] Regarding red (R=1, G=0, B=0):

[0202] ●U R =au×c

[0203] ●V R =av×c

[0204] ●

[0205] ●H R =arctan(av / au)

[0206] Where au and av are the coefficients of the transformation matrix A, allowing conversion from RGB to YUV:

[0207]

[0208] In the BT.2020 color gamut And in the BT.709 color gamut

[0209] Regarding green (R=0, G=1, B=0):

[0210] ●U G =bu×c

[0211] ●V G =bv×c

[0212] ●

[0213] ●H G =arctan(bv / bu)

[0214] Regarding blue (R=0, G=0, B=1):

[0215] ●U B =cu×c

[0216] ●V B =cv×c

[0217] ●

[0218] ●H B =arctan(cv / cu)

[0219] Regarding magenta (R=1, G=0, B=1):

[0220] ●U M = (au + cu) × c

[0221] ●V M = (av + cv) × c

[0222] ●

[0223] ●H M =arctan((av+cv) / (au+cu))

[0224] Regarding cyan (R=0, G=1, B=1):

[0225] ●U C = (bu + cu) × c

[0226] ●V C = (bv + cv) × c

[0227] ●

[0228] ●H C=arctan((bv+cv) / (bu+cu))

[0229] Regarding yellow (R=1, G=1, B=0):

[0230] ●U Y = (au + bu) × c

[0231] ●V Y = (av + bv) × c

[0232] ●

[0233] ●H Y =arctan((av+bv) / (au+bu))

[0234] Figure 10 This indicates the positions of the three primary colors (red, green, and blue) and secondary colors (magenta, yellow, and cyan) calculated using the above formula in the UV plane.

[0235] If its hue H value is equal to the hue value H calculated above G H R H B H M H C H Y One of the hue values ​​is determined by U. pre1 and V pre1 This indicates that a given pixel belongs to either the primary color system or the secondary color system.

[0236] However, in images, colors are rarely "pure" primary or secondary colors. Therefore, instead of defining six sectors (each centered on one of the six primary or secondary colors), we define six categories each corresponding to one of the primary or secondary colors. For each sector, we define a deviation angle Δ, and the corresponding sector is defined by four points:

[0237] ●Up=C max *cos(H+Δ)

[0238] ●Vp=C max *sin(H+Δ)

[0239] ●Um=C max *cos(H-Δ)

[0240] ●Vm=C max *sin(H-Δ)

[0241] in

[0242] ● and H=H R Used for red;

[0243] ● and H=H G For green purposes;

[0244] ● and H=H B Used for blue;

[0245] ● and H=H M Used for magenta;

[0246] ● and H=H C Used for cyan;

[0247] ● and H=H Y Used for yellow;

[0248] The inspection indicates U pre1 and V pre1 One way to determine whether a given pixel belongs to a sector is to compute a vector of the product of the normalized values ​​of that pixel using the two limits of the sector, as shown below:

[0249] U curr =U pre1 / 1023

[0250] V curr =V pre1 / 1023

[0251] PV p =U curr ×V P -V curr ×U P

[0252] PV m =U curr ×V m -V curr ×U m

[0253] If (PV) p ≥0) and (PV) m If the value is less than or equal to 0, then the pixel belongs to that sector.

[0254] In one implementation, the value Δ can be fixed and the same for all sectors. In other implementations, the value Δ is different for each sector.

[0255] In one implementation, all sectors are contiguous, meaning that any pixel in the frame will belong to one sector. In other implementations, at least some sectors are not contiguous, meaning that some pixels may not belong to any sector.

[0256] Figure 10 This represents six sectors corresponding to the three primary colors and three secondary colors in the UV plane. The sectors are defined by dashed lines 1000 to 1005. For example, the sector corresponding to red is defined by dashed lines 1000 and 1001.

[0257] In step 93, processing module 40 generates a statistical representation for each sector (i.e., each category). In one embodiment, the statistical representation includes:

[0258] ●The histogram histo represents the number of pixels found in a sector for each luminance value, indicating the V in SL-HDR1. pre1

[0259] ● For each luminance value lum in the histogram, the vector frame_chr_max[lum] represents the maximum chromaticity value of all pixels with luminance value lum in the sector;

[0260] ● For each luminance value lum in the histogram, the value frame_chr_av[lum] represents the average chromaticity value of all pixels with luminance value lum in the sector.

[0261] It can be noted that the chromaticity value chr_curr of the current pixel is calculated as follows:

[0262] Among them U cur and v cur These are the U and V components of the current pixel.

[0263] In one implementation, the number of bins for the histogram and the number of catalog entries for the two vectors are set to "64". In another implementation, the number of bins and the number of catalog entries for the two vectors are set to "256", corresponding to the range of sgf_x values ​​defined in the SL-HDRx standard for the saturation gain function metadata. In yet another implementation, the number of bins for the histogram and the number of catalog entries for the two vectors can be less than or greater than "64".

[0264] In step 94, processing module 40 determines data representing each sector (i.e., for each color category). In one embodiment, the data representing a sector is a dominant luminance value corresponding to the dominant luminance of the color in the current sector. The dominant luminance value of the sector is determined using at least one of the histogram histo corresponding to the sector and the vectors frame_chr_max[lum] and frame_chr_av[lum] corresponding to the sector.

[0265] In the first embodiment of step 94, the main brightness value of the sector is determined by scanning the histogram histo of the sector and by determining the bin with the highest number of pixels (i.e., the brightness value corresponding to the highest number of pixels).

[0266] Step 94 allows obtaining the vector `frame_idx_max_histo`, which includes a primary chromaticity value for each primary and secondary color (i.e., for each sector). For each sector, the chromaticity of the color will ultimately be modified at that primary chromaticity value using the SGF function.

[0267] In step 95, processing module 40 determines a chromaticity gain (i.e., scaling value or color correction) to be applied to the chromaticity value of each color sector. To this end, processing module 40 determines the maximum permissible chromaticity value for the color corresponding to the dominant luminance value of each sector. This chromaticity gain represents the margin for increasing the chromaticity of that color and indirectly represents the maximum permissible chromaticity value for each sector and each luminance value.

[0268] As mentioned above, matrix operations can be used to derive YUV values ​​from RGB values:

[0269]

[0270] For the three primary colors and three secondary colors, when processing the normalized RGB values ​​(values ​​within [0; 1]), this results in:

[0271] ● Regarding the primary color red (component R), processing module 40 calculates the following values:

[0272] ●R = s and G = B = 0;

[0273] ○Y=al×s and Y Rmax =al(Y in BT.2020 color gamut) Rmax =0.2627):

[0274] ○U=au×s=au / al×Y;

[0275] ○V=av×s=av / al×Y:

[0276] ○ and (In the BT.2020 color gamut C) Rmax =0.519);

[0277] Then, processing module 40 calculates the envelope of the permissible chromaticity values ​​for red. In one embodiment, this envelope consists of two straight lines:

[0278] ● In the YC (luminance / chrominance) space, from the first point AR at coordinates (Y=0; C=0) to the point at coordinates (Y=Y... Rmax C = C Rmax The second point BR is the increasing straight line.

[0279] ● A decreasing straight line from the second point to the third point CR in the YC (luminance / chrominance) space (Y=1; C=0).

[0280] ● Regarding the primary color green (component G), processing module 40 calculates the following values:

[0281] ●G = s and R = B = 0;

[0282] ○Y=bl×s and Y Gmax =bl(Y in BT.2020 color gamut) Gmax =0.678);

[0283] ○U=bu×s=bu / bl×Y;

[0284] ○V=bv×s=bv / bl×Y;

[0285] ○ and (In the BT.2020 color gamut C) smax =0.584);

[0286] Then, processing module 40 calculates the envelope of the permissible chromaticity values ​​for green. In one embodiment, this envelope consists of two straight lines:

[0287] ● In the YC (luminance / chrominance) space, from the first point AG at coordinates (Y=0; C=0) to the point at coordinates (Y=Y Gmax C = C Gmax The second point BG is the increasing straight line.

[0288] ●A decreasing straight line from the second point (Y=1; C=0) to the third point (CG) in the YC (luminance / chrominance) space.

[0289] ●Regarding the primary color blue (component B), processing module 40 calculates the following values:

[0290] ●B = s and R = G = 0;

[0291] ○Y=cl×s and Y Bmax =cl(Y in BT.2020 color gamut) Bmax =0.0593);

[0292] ○U=cu×s=cu / cl×Y;

[0293] ○V=cv×s=cv / cl×Y;

[0294] ○ and (In the BT.2020 color gamut C) Bmax =0.502);

[0295] Then, processing module 40 calculates the envelope of the permissible chromaticity values ​​for blue. In one embodiment, this envelope consists of two straight lines:

[0296] ● In the YC (luminance / chrominance) space, from the first point AB at coordinates (Y=0; C=0) to the point at coordinates (Y=Y... Bmax C = C Bmax The second point BB is an increasing straight line.

[0297] ●A decreasing straight line from the second point to the third point CB in the YC (luminance / chrominance) space (Y=1; C=0).

[0298] ●Regarding the secondary color magenta, processing module 40 calculates the following values:

[0299] ●R = B = s and G = 0;

[0300] ○Y=al×s+cl×C=(au+cu)×R and Y Mmax =al+cl(Y in BT.2020 color gamut) Mmax =0.322);

[0301] ○U=au×s+cu×C=(au+cu)×s and U=((au+cu)) / ((al+cl)×Y);

[0302] ○V=av×s+cv×C=(av+cv)×s and V=((av+cv)) / ((al+cl)×Y);

[0303] ○ and (In the BT.2020 color gamut C) Mmax =0.584);

[0304] Then, processing module 40 calculates the envelope of the permissible chromaticity values ​​for magenta. In one embodiment, this envelope consists of two straight lines:

[0305] ● In the YC (luminance / chrominance) space, from the first point AM at coordinates (Y=0; C=0) to the point at coordinates (Y=Y... Mmax C = C Mmax The increasing straight line of the second point BM.

[0306] ●A decreasing straight line from the second point to the third point CB in the YC (luminance / chrominance) space (Y=1; C=0).

[0307] ●Regarding secondary cyan, processing module 40 calculates the following values:

[0308] ●G = B = s and R = 0;

[0309] ○Y=bl×s+cl×C=(bl+cl)×s and Y Cmax =bl+cl(Y in BT.2020 color gamut) Cmax =0.7373);

[0310] ○U=bu×s+cu×C=(bu+cu)×s and U=(bu+cu) / (bl+cl)×Y;

[0311] ○V=bv×s+cv×C=(bv+cv)×s and V=(bv+cv) / (bl+cl)×Y;

[0312] ○ and (In the BT.2020 color gamut C) Cmax =0.519);

[0313] Then, processing module 40 calculates the envelope of the permissible chromaticity values ​​for cyan. In one embodiment, this envelope consists of two straight lines:

[0314] ● In the YC (luminance / chrominance) space, from the first point AC at coordinates (Y=0; C=0) to the point at coordinates (Y=Y... Cmax C = C Cmax The increasing straight line from the second point BC of the given line.

[0315] ●A decreasing straight line from the second point (Y=1; C=0) to the third point (CC) in the YC (luminance / chrominance) space.

[0316] ●Regarding the secondary color yellow, processing module 40 calculates the following values:

[0317] ○G=R=s and B=0;

[0318] ■Y=al×s+bl×C=(al+bl)×s and Y Ymax =al+bl(Y in BT.2020 color gamut) Ymax =0.9407);

[0319] ■U=au×s+bu×C=(au+bu)×s and U=((au+bu)) / ((al+bl)×Y);

[0320] ■V=av×s+bv×C=(av+bv)×s and V=(av+bv)(al+bl)×Y;

[0321] ■ and (In the BT.2020 color gamut C) Ymax =0.502);

[0322] Then, processing module 40 calculates the envelope of the permissible chromaticity values ​​for cyan. In one embodiment, this envelope consists of two straight lines:

[0323] ○ In the YC (luminance / chrominance) space, from the first point AY at coordinates (Y=0; C=0) to the point at coordinates (Y=Y Ymax C = C Ymax The second point BY is an increasing straight line.

[0324] ○ A decreasing straight line from the second point (Y=1; C=0) to the third point (CY) in the YC (luminance / chrominance) space.

[0325] In one implementation, the processing module 40 represents six computed envelopes using six vectors chr_envelop[S], one for each sector (i.e., one for each category), where S represents the index of the sector. For each sector S, the vector chr_envelop[S] includes information representing the maximum allowed chromaticity value for each bin of the histogram histo[S].

[0326] Within a sector, processing module 40 uses the vector chr_envelop[S] to determine the maximum permissible scaling value (i.e., the maximum permissible gain) scale_max[S], as follows:

[0327] cur_idx=frame_idx_max_histo[S];

[0328] scale_max[S]=chr_envelop[S][cur_idx] / frame_chr_max[S][cur_idx].

[0329] Within each sector S, the maximum permissible scaling value scale_max[S] provides a multiplication factor for the chromaticity at the selected brightness level. The maximum permissible scaling value scale_max[S] allows for the acquisition of images with more chromaticity for colors that require more chromaticity in a controlled and independent manner.

[0330] In step 96, the processing module encodes the information representing chroma gain into metadata in the bitstream. The metadata conforms to SL-HDR1. In one implementation, the information representing chroma gain is a vector of frame_idx_max_histo and a vector of the maximum allowed scaling value scale_max. This information is encoded in the SL-HDR1 metadata as an SGF function.

[0331] In one implementation where the number of bins in histo[S] is "256", the range of frame_idx_max_histo directly matches the range of sgf_x values ​​defined in the SL-HDRx standard for the saturation gain function metadata, and no adjustment is required; that is, the value of frame_idx_max_histo can be directly copied to one of the sgf_x values ​​in the SL-HDRx metadata. In an implementation where the number of bins is different from "256" (e.g., "64"), the processing module 40 rescales the vector frame_idx_max_histo to "256" before encoding it into the sgf_x that defines the SGF function.

[0332] Processing module 40 then assigns one of the six available sgf_x values ​​to the number of bins corresponding to the index cur_idx = frame_idx_max_histo[S], and modifies the corresponding default sgf_y value (typically equal to "118" in the case of SL-HDR1 NCL) using the maximum allowed scaling value scale_max[S]. In an implementation where the number of bins is equal to "256" (six sectors, i.e., three primary colors and three secondary colors), processing module 40 assigns one of the six available sgf_x values ​​to the number of bins corresponding to the index cur_idx = frame_idx_max_histo[S], and modifies the corresponding default sgf_y value using the maximum allowed scaling value scale_max[S].

[0333] For each sector S:

[0334] sgf_x[S]=frame_idx_max_histo[S] and sgf_y[S]=scale_max[S].

[0335] Then, processing module 40 reorders the sgf_x and sgf_y values ​​to ensure that the sgf_x[i] value monotonically increases as i increases. The reordered sgf_x and sgf_y values ​​allow the definition of an SGF function, which is transmitted to post-processing module 14 as metadata.

[0336] In one embodiment of step 91, analysis is performed on a subsampled version of the current image.

[0337] In other embodiments of step 92, any other category representing different hues or colors or different quantities of categories may be used.

[0338] In one embodiment of step 93, dark values ​​(low brightness values) are not considered during histogram construction because it is difficult to distinguish the colors of dark values. For example, brightness values ​​below a first brightness threshold are not considered.

[0339] In one embodiment of step 93, luminance values ​​(high luminance values) are not considered during histogram construction because it is difficult to distinguish the color of luminance values. For example, luminance values ​​above a first luminance threshold are not considered.

[0340] In one embodiment of step 93, chromaticity values ​​(chr_curr) below the chromaticity threshold are not considered in the construction of the histogram.

[0341] In the second embodiment of step 94, processing module 40 calculates a value max_energy_chroma representing the maximum chroma energy in each histogram histo (i.e., for each sector). To do this, for each bin of each histogram histo, processing module 40 calculates the value energy_chroma[lum] representing the chroma energy by multiplying the number of pixels at that bin histo[lum] by the maximum chroma value frame_chr_max[lum] found at that bin in the corresponding sector. Then, for each sector, processing module determines the maximum chroma energy max_energy_chroma by determining the maximum value of energy_chroma[lum]. The principal luminance value is the luminance value corresponding to the maximum chroma energy max_energy_chroma. One advantage of the second embodiment is that it correlates the maximum chroma value with the luminance value (or the number of bins) of a bin, thereby providing a better understanding of the most attractive areas in the image.

[0342] In the third embodiment of step 94, processing module 40 calculates a value max_av_energy_chroma representing the maximum average chromaticity energy in each histogram (i.e., for each sector). To this end, processing module 40 replaces the maximum chromaticity value frame_chr_max[lum] with the average chromaticity value frame_chrav[lum] during the second embodiment. The principal luminance value is the luminance value corresponding to the maximum average chromaticity energy max_av_energy_chroma.

[0343] In variations of the first, second, and third embodiments of step 94, only bins comprising at least the minimum number of pixels are considered during scanning.

[0344] In variations of the first, second, and third embodiments of step 94, only the bin corresponding to the envelope value of the allowed chromaticity value, which is higher than the minimum chromaticity value Chr_trigger, is considered during scanning. This embodiment avoids considering colors with extremely low saturation. In variations of the first, second, and third embodiments of step 94, each histogram histo is preprocessed before searching for the principal luminance value to smooth the final noise or remove excessively small peaks.

[0345] In one embodiment of step 95, the maximum permissible scaling value scale_max[S] can be limited to the maximum value absolute_scale_max[S] to avoid oversaturation of colors. In one embodiment, the maximum value absolute_scale_max[S] is the same for all color sectors, or it is different for each sector. In this case, the maximum permissible scaling value scale_max[S] is calculated as follows:

[0346] scale_max[S]=min(scale_max[S]; absolute_scale_max[S]).

[0347] In one embodiment of step 95, the maximum permissible scaling value scale_max[S] has a minimum value to avoid desaturating colors, even if analysis has shown that the envelope of the permissible chroma value is lower than the current maximum chroma chr_max[S].

[0348] In one embodiment of step 95, when determining the maximum permissible scaling value scale_max[S] for sector S, the same calculation can be performed for other color sectors at index cur_idx = frame_idx_max_histo[S]. If the maximum permissible scaling value scale_max[S′] of at least one sector S′ different from the current color sector S is lower than the maximum permissible scaling value scale_max[S], then the processing module 40 restricts the maximum permissible scaling value scale_max[S] to the lower maximum permissible scaling value scale_max[S′] found in another sector.

[0349] In one implementation of step 95, the maximum U and V value scaling can be further analyzed to avoid cropping the color sector at index cur_idx = frame_idx_max_histo[S], instead of limiting the maximum allowed scaling value scale_max[S] of the current sector S to the minimum of the maximum allowed scaling value scale_max[S'] found in other sectors S' at index cur_idx = frame_idx_max_histo[S]. This analysis provides the value scale_max_UV for limiting scale_max[S], as follows:

[0350] scale_max[S]=min(scale_max[S], scale_max_UV);

[0351] Without applying time stabilization (i.e., time filtering), the vectors frame_idx_max_histo[S] and scale_max[S] are at risk of fluctuating. Due to these fluctuating vectors, the preprocessor 10 risks generating unstable and unacceptable SDR image sequences.

[0352] In optional step 97, the processing module applies a time-stabilized method.

[0353] Figure 11 An example implementation of the time-stabilized optional step 97 is described in detail.

[0354] In step 971, processing module 40 determines whether the current image of the HDR content corresponds to a scene change. To do this, for example, processing module 40 compares the current image with images preceding the current image in the HDR content. For example, if the difference calculated as the sum of the absolute differences between the co-located pixels of the two images is higher than a threshold, processing module 40 determines that the current image corresponds to a scene change. In this case, step 971 proceeds to step 973. Otherwise, step 971 proceeds to step 972.

[0355] In step 973, processing module 40 initializes a set of parameters for the time stabilization method. In other words, time stabilization is reinitialized during step 973.

[0356] In step 972, the processing module 40 calculates the filter vectors frame_idx_max_histo and scale_max.

[0357] In one implementation, during step 971, the processing module determines whether the current image is the first image of the HDR content, rather than searching for a scene switch.

[0358] Figure 12 An example of an implementation scheme for step 973 is described in detail.

[0359] exist Figure 12 In the example, configurable circular buffers `frame_idx_max_histo_buf[S]` (and `scale_max_buf[S]` respectively) are used to compute a filtered version of the parameter for each parameter `frame_idx_max_histo[S]` (and `scale_max[S]` respectively). In one implementation, each buffer has the same size `n`, which represents the number of consecutive frames considered for computing the filtered version of the corresponding parameter. In one implementation, the buffer size `n = 10`. An invalid value `frame_idx_max_histo_invalid` (and `scale_max_invalid` respectively) is defined for each parameter `frame_idx_max_histo[S]` (and `scale_max[S]` respectively). When passed through Figure 9 When the method used to determine color correction generates this invalid value, it means that no valid histogram index and scale_max value have been calculated for the current color sector of the current frame; that is, the chromaticity of the current color sector does not need to be scaled for the current frame. For example, if a value of "64" is defined for luminance, then frame_idx_max_histo[S] is between "0" and "63". Scale_max[S] can also be defined, for example, no higher than "5". If for red (S = red), using Figure 9 The processing module 40, used to determine the color correction method, has determined that red can be saturated, so frame_idx_max_histo[red] is in the range [0; 63] and scale_max[red] is in the range [0; 5]. However, if using Figure 9 The method, where processing module 40 has determined that red must not be saturated, assigns invalid values ​​to frame_idx_max_histo[red] (e.g., "64") and scale_max[red] (e.g., "10"). Therefore, using... Figure 9 The method involves processing module 40 determining whether the values ​​of frame_idx_max_histo[red] and scale_max[red] are valid and therefore need to be stabilized.

[0360] Therefore, when an invalid value is detected, there is no need for time to stabilize the current parameter. Each value in each buffer is initialized as described below.

[0361] As mentioned above, in Figure 12 During the process, it is assumed that all buffers have the same size n:

[0362] In step 973A, the processing module 40 initializes the variable S representing the sector (i.e., the color) to zero.

[0363] In step 973B, processing module 40 determines whether variable S is less than the number of sectors NumOfSectors. For example, NumOfSectors = 6.

[0364] If S = NumOfSectors, then processing module 40 stops the initialization process 973.

[0365] Otherwise, in step 973D, the processing module 40 initializes variable i to zero.

[0366] In step 973E, the processing module 40 determines whether i is less than the buffer size n.

[0367] If i = n, then processing module 40 increments variable S by one unit in step 973F.

[0368] Otherwise, processing module 40 determines whether the parameter frame_idx_max_histo[S] is different from the invalid value frame_idx_max_histo_invalid. If frame_idx_max_histo[S] = frame_idx_max_histo_invalid, then processing module 40 sets the value of frame_idx_max_histo_buf[S][i] to frame_idx_max_invalid in step 973H. Otherwise, processing module 40 sets the value of frame_idx_max_histo_buf[S][i] to frame_idx_max_histo[S] in step 973I.

[0369] Following steps 973H and 973I is step 973J, during which processing module 40 compares the parameter scale_max[S] with the invalid value scale_max_invalid. If scale_max[S] = scale_max_invalid, then processing module 40 sets the value scale_max_buf[S][i] to scale_max_invalid. Otherwise, in step 973L, processing module 40 sets the value scale_max_buf[S][i] to scale_max[S].

[0370] In step 973M, processing module 40 sets the value frame_idx_max_histo_buf[S][i]×W i Add to the cumulative value cum_frame_idx_max_histo[S]. The cumulative value cum_frame_idx_max_histo[S] represents all values ​​in the corresponding buffer. W i It is a weighting factor. In one implementation, W i =1. In another implementation, W i The value of i is different for each value. In the last case, the cumulative value cum_frame_idx_max_histo[S] is a weighted sum of frame_idx_max_histo_buf[S][i], giving more weight to a specific position in the buffer.

[0371] In step 973N, processing module 40 sets the value scale_max_buf[S][i]×W i Add to the cumulative value cum_scale_max[S]. The cumulative value cum_scale_max[S] represents all values ​​in the corresponding buffer.

[0372] In step 973O, the processing module 40 initializes the index filterIndex, which represents the position of the current image in the buffer.

[0373] In one implementation, filterIndex = 0 when all buffers have the same size.

[0374] In another implementation, each buffer associated with the parameters of the vectors frame_idx_max_histo and scale_max has a different size. In this case, each buffer has an index filterIndex.

[0375] Figure 13 An example of an implementation scheme for step 972 is described in detail.

[0376] The example implementation of step 972 aims to filter the parameters of the vectors frame_idx_maxhisto and scale_max. Figure 13 The process is performed by processing module 40. The filtering of these parameters is described below:

[0377] ● For each parameter, the accumulated value is updated in the following way:

[0378] ○ Subtract the oldest parameter value corresponding to the parameter value found in the current index. Subtraction can be a simple subtraction or a weighted subtraction between the oldest parameter value and any of the following parameters;

[0379] Add the latest parameter value that was just received. Addition can be a simple addition or a weighted addition, which is a combination of the latest parameter value and any of the previous parameters.

[0380] ● Update the buffer at the current index using the latest parameters just received.

[0381] ● Calculate the filter value for each parameter. The filter value can be:

[0382] Simply divide the corresponding accumulated value by the size of the corresponding buffer;

[0383] ○ Divide the corresponding cumulative value by a number that corresponds to the weighted sum of the combinations of the latest parameter value considered when calculating the cumulative value and any of the previous parameter values.

[0384] In this step, processing module 40 checks whether the buffer has been initialized or not previously initialized during the current switch. If it has been initialized, and if the current value is valid, processing module 40 updates the current buffer value. If it has not been initialized, processing module 40 initializes the buffer and the accumulated value as described in step 973.

[0385] Figure 13 The example implementation applies when the buffer size n is the same for all parameters, the current index is i, the cumulative value is the simple sum of all parameters, and the filter value is simply divided by the buffer size n. In this case, all filter values ​​are calculated as follows:

[0386] In step 972A, the processing module 40 initializes the variable S representing the sector (i.e., the color).

[0387] In step 972B, the processing module 40 determines whether the variable S is less than the number of sectors NumOfSectors.

[0388] If the number is not less than 1, then processing module 40 stops in step 972C. Figure 13 The processing.

[0389] Otherwise, processing module 40 determines whether the parameter `frame_idx_max_histo[S]` is different from `frame_idx_max_histo_invalid`. If `frame_idx_max_histo[S]` = `frame_idx_max_histo_invalid`, then step 972D is followed by step 972P. During step 972P, processing module 40 reinitializes all buffer values ​​`frame_idx_max_histo_buf[S][x]` (x ranges from zero to the buffer size n) to `frame_idx_max_histo_invalid`. After this reinitialization, during step 972P, processing module 40 increments the variable S by one unit. Furthermore, during step 972P, step 972B is followed by step 972B.

[0390] If frame_idx_max_histo[S] ≠ frame_idx_max_histo_invalid, then step 972D is followed by step 972E. During step 972E, processing module 40 determines whether the value of the buffer frame_idx_max_histo_buf[S][i] is different from frame_idx_max_histo_invalid.

[0391] If frame_idx_max_histo_buf[S][i] = frame_idx_max_histo_invalid, step 972E is followed by step 972F, in which all buffer values ​​frame_idx_max_histo_buf[S][x] (x ranges from zero to the buffer size n) are initialized to frame_idx_max_histo[S]. Furthermore, during step 972F, processing module 40 assigns the value frame_idx_max_histo[S] to the filtered primary chroma value filtered_frame_idx_max_histo[S].

[0392] If frame_idx_max_histo_buf[S][i] ≠ frame_idx_max_histo_invalid, step 972E is followed by step 972G. During step 972G, processing module 40 updates the cumulative value cum_frame_idx_max_histo[S] as follows:

[0393] cum_frame_idx_max_histo[S]=cum_frame_idx_max_histo[S]-frame_idx_max_histo_buf[S][i]+frame_idx_max_histo[S].

[0394] In step 972H, processing module 40 updates the buffer value frame_idx_max_histo_buf[S][i] as follows:

[0395] frame_idx_max_histo_buf[S][i]=frame_idx_max_histo[S].

[0396] In step 972I, the processing module 40 obtains the filtered principal chromaticity value filtered_frame_idx_max_histo[S]:

[0397] filtered_frame_idx_max_histo[S]=cum_frame_idx_max_histo[S] / n.

[0398] Step 972I is followed by step 972J.

[0399] During step 972J, the processing module 40 determines whether the maximum allowable scaling value scale_max[S] is different from the value scale_max_invalid.

[0400] If scale_max[S] = scale_max_invalid, then processing module 40 reinitializes all buffer values ​​scale_max_buf[S][x] to scale_max_invalid. After this reinitialization, during step 972P, processing module 40 increments the variable S by one unit.

[0401] Otherwise, during step 972K, processing module 40 determines whether the buffer value scale_max_buf[S][i] is different from the value scale_max_invalid. If scale_max_buf[S][i] = scale_max_invalid, then in step 972L, the processing module sets the buffer value scale_max_buf[S][x] to scale_max[S] and sets the maximum allowed filtering scaling value filtered_scale_max[S] to the value scale_max[S]. Step 972L is followed by step 972P.

[0402] Otherwise, in step 972M, the processing module updates the cumulative value cum_scale_max[S] as follows:

[0403] cum_scale_max[S]=cum_scale_max[S]-scale_max_buf[S][i]+scale_max[S].

[0404] In step 972N, processing module 40 updates the buffer value scale_max_buf[S][i] as follows:

[0405] scale_max_buf[S][i]=scale_max[S].

[0406] In step 972O, processing module 40 obtains the maximum allowable scaling value for filtering as shown below:

[0407] filtered_scale_max[S]=cum_scale_max_histo[S] / n.

[0408] Step 972O is followed by step 972P.

[0409] Then, for each sector S, in the SGF function definition transmitted to the post-processing module in the form of metadata, the filtered value replaces the non-filtered value:

[0410] sgf_x[S]=filtered_frame_idx_max_histo[S] and sgf_y[S]=filtered_scale_max[S].

[0411] It should be noted that, in the case of SL-HDR1, all processes aimed at determining the SGF function are based on the Y value representing the output of the SL-HDR1 preprocessing module. pre0 U pre1 and V pre1 The intermediate signal is the SDR signal. In other words, all calculations are performed in the SDR domain.

[0412] Figure 14 An implementation scheme for controlling color correction suitable for an SL-HDR2 system is illustrated schematically.

[0413] Control and Figure 9 The relevant color correction methods resolved the issues of the SL-HDR1 system. In the SL-HDR2 preprocessor, no SDR signal is generated. [The last sentence appears to be incomplete and unrelated to the preceding text. It likely refers to a different system or approach.] Figure 14 In step 140 of the implementation scheme of the SL-HDR2 system, the chromaticity analysis of the current image is performed in the SL-HDR2 preprocessor by analyzing the chromaticity data from the relevant data. Figure 8 The reconstruction process described in steps 801 to 807 simulates the SL-HDR2 post-processor until the variable HDR is obtained. R HDR G and HDR B , The reconstruction process is performed (i.e., by processing module 40). It is completed by considering that the connected display is an SDR display. Therefore, the reconstruction process generates an HDR that is essentially an SDR signal. R HDR G and HDR B Signal.

[0414] The SL-HDR2 post-processing module (more accurately, the reconstruction module) captures HDR signals and generates SDR, MDR, or HDR signals. About Figure 9 The described method allows the SGF point coordinates sgf_x and sgf_y to be determined in the SDR domain. However, in a true SL-HDR2 post-processor, these points are applied to the input HDR signal. Therefore, in the SL-HDR2 case, all calculated SGF points are mapped to the HDR domain, which implies an estimate of the SDR-to-HDR transform.

[0415] In step 141, the processing module calculates the SDR to HDR conversion. This is done in two steps using two values, Lhisto_cur_sdr and Lhisto_cur_hdr, and three vectors: Lhisto_match_sdr_hdr_min, Lhisto_match_sdr_hdr_max, and Lhisto_match_sdr_hdr. In the first step, for each pixel of the current image containing HDR content, Lhisto_match_sdr_hdr_min and Lhisto_match_sdr_hdr_max are calculated as follows:

[0416] HDR R HDR G and HDR B In the SDR domain, and γ is a "2.4" gamma factor.

[0417] Lhisto_cur_sdr=CLAMP((Y post2 / 16+.5), 0, NumBins-1)

[0418] Lhisto_cur_hdr=CLAMP((Y post1 / 16+.5), 0, NumBins-1)

[0419] Where CLAMP(x, y, z) is min(max(x, y), z), and NumBins is the number of bins in the histogram (e.g., NumBins = 64).

[0420] Lhisto_match_sdr_hdr_min[Lhisto_cur]=min(Lhisto_match_sdr_hdr_min[Lhisto_cur_sdr], );

[0421] Lhisto_match_sdr_hdr_max[Lhisto_cur]=max(Lhisto_match_sdr_hdr_max[Lhisto_cur_sdr], );

[0422] In the second step, after analyzing all pixels, the estimated SDR to HDR transform is calculated, such as... Figure 15 As shown.

[0423] In step 1410, the processing module 40 initializes the variable Last_correct_value to zero.

[0424] In step 1411, the processing module 40 initializes the variable lum to zero.

[0425] In step 1412, the processing module 40 determines whether the variable lum is less than NumBins.

[0426] If lum = NumBins, the processing module stops. Figure 15 The process. Otherwise, in step 1404, processing module 40 calculates the value Lhisto_match_sdr_hdr[lum] as follows:

[0427] Lhisto_match_sdr_hdr[lum]=Lhisto_match_sdr_hdr_min[lum]+Lhisto_match_sdr_hdr_max[lum]) / 2.

[0428] In step 1415, the processing module 40 determines whether the value Lhisto_match_sdr_hdr[lum] is equal to NumBins.

[0429] If equal, then processing module 40 calculates the value Lhisto_match_sdr_hdr[lum] in step 1417 as follows:

[0430] Lhisto_match_sdr_hdr[lum]=last_correct_value.

[0431] Otherwise, in step 1416, the processing module calculates the value last_correct_value as follows:

[0432] last_correct_value=Lhisto_match_sdr_hdr[lum].

[0433] Steps 1416 and 1417 are followed by step 1418, during which the value lum is incremented by one unit.

[0434] In step 142, processing module 40 applies steps 90 to 96 to determine the vectors frame_idx_max_histo and scale_max.

[0435] In step 143, processing module 40 maps the parameters of vector frame_idx_max_histo to, for example... Figure 16 The HDR domain is shown.

[0436] In step 1430, the processing module 40 initializes the variable S to zero.

[0437] In step 1431, the processing module 40 determines whether variable S is less than NumOfSectors.

[0438] If S = NumOfSectors, then processing module 40 stops in step 1432. Figure 16 The process.

[0439] Otherwise, the processing module determines whether the parameter frame_idx_maxhisto[S] is less than NumBins.

[0440] If frame_idx_max_histo[S] = NumBins, then processing module 40 increments the variable S by one unit in step 1436. Step 1436 is followed by step 1431.

[0441] Otherwise, in step 1434, processing module 40 calculates the variable Lhisto_sdr as follows:

[0442] Lhisto_sdr=frame_idx_max_histo[S].

[0443] In step 1435, the processing module 40 calculates the parameter frame_idx_max_histo[S] as follows:

[0444] frame_idx_max_histo[S]=Lhisto_match_sdr_hdr[Lhisto_sdr].

[0445] Step 1435 is followed by step 1436.

[0446] return Figure 14 After step 143, processing module 40 executes step 144, during which processing module 40 calculates the SGF points representing the SGF function. For each sector S:

[0447] sgf_x[S]=frame_idx_max_histo[S] and sgf_y_tmp[S]=scale_max[S].

[0448] Then, processing module 40 reorders the sgf_x and sgf_y_tmp values ​​to ensure that the sgf_x[i] value monotonically increases as i increases. The reordered sgf_x and sgf_y_tmp values ​​allow the definition of an SGF function, which is transmitted to post-processing module 14 as metadata.

[0449] In step 142, all calculations are completed to generate the SGF function, which, in the case of SL-HDR1, enhances saturation in steps 90 through 96. Finally, because the SGF function works differently between SL-HDR1 and SL-HDR2, all sgf_y_tmp(Y) values ​​calculated in step 96 in the case of SL-HDR1 need to be adapted to the case of SL-HDR2.

[0450] In SL-HDR1, Figure 6 In step 605, SGF is applied on the preprocessor side, where

[0451]

[0452] in

[0453] Where the saturation gain function It is derived from the piecewise linear pivot points defined by the metadata of the saturation gain function sgf_x and sgf_y, as detailed in section 7.3 of document ETSI TS 103 433-1 v1.3.1.

[0454] Given a brightness Y, when sgf(Y) is increased by an increment value incr:

[0455]

[0456] Therefore, when sgf(Y) is increased by incr, Upre1 (respectively V) pre1 The following modifications have been made:

[0457]

[0458]

[0459] Any positive value of incr will increase U. pre1 (respectively V) pre1 This increases the saturation of pixels.

[0460] In SL-HDR2, Figure 8 In step 804, SGF is applied on the post-processor side, with lutCC[Y]. When sgf(Y) is increased by incr, U post2 (respectively V) post2 The following modifications have been made:

[0461]

[0462]

[0463] Any positive value of incr will decrease U. post2 (respectively V) post2 This will reduce the saturation of the corresponding pixel.

[0464] Therefore, in SL-HDR2, the increment incr_slhdr2 is calculated based on the corresponding SL-HDR1 increment incr_slhdr1, such that:

[0465]

[0466] Finally, this leads to the calculation of the final SL-HDR2 sgf_y value:

[0467]

Claims

1. A video processing method, the method comprising: obtaining a current multi-component image; classifying (92) colors of pixels of the current multi-component image into a plurality of color classes using a tone mapping luminance component and a correction normalized chrominance component derived from the current multi-component image; for each color class, determining data representative of the color class including a dominant luminance value representative of a luminance for which colors in the class are dominant and a value representative of a chrominance gain determined from the data representative of the color class, the chrominance gain being representative of a maximum allowed chrominance value for chrominance components in the color class; and, encoding the dominant luminance value and the value representative of the gain corresponding to each class as metadata representative of a saturation gain function in a bitstream, the function defining a color correction to be applied to pixels of the current multi-component image as a function of their luminance.

2. The method according to claim 1, wherein the tone mapping luminance component and the correction normalized chrominance component of pixels of the current multi-component image are obtained by analyzing chrominance components of the current multi-component image, the analysis comprising, for each pixel of at least a subset of pixels of the current multi-component image: • deriving a luminance component from the components of the pixel; • applying a tone mapping to the derived luminance component to obtain a tone mapping luminance component; • deriving chrominance components from the components of the pixel; and, • applying a joint normalization and color correction to the chrominance components to obtain correction normalized chrominance components.

3. The method according to claim 1, wherein the current multi-component image is included in a video sequence and a temporal filtering is applied to the information representative of the chrominance gain based on information representative of chrominance gains computed for images of the video sequence preceding the current multi-component image.

4. The method according to claim 3, wherein the temporal filtering is reinitialized at the beginning of the video sequence or when a scene cut is identified in the video sequence.

5. The method according to claim 1, wherein the color classes are color sectors in a chrominance plane around primary colors and / or secondary colors.

6. The method according to claim 5, wherein a combination of the sectors covers the chrominance plane as a whole.

7. The method according to claim 1, wherein determining data representative of the color class comprises obtaining a histogram of luminance values of pixels of the current multi-component image as a function of luminance values of the color class.

8. The method according to claim 7, wherein only pixels corresponding to luminance values included in a predefined range of luminance values are used for obtaining the histogram.

9. The method of claim 7, wherein the dominant luma value corresponds to a luma value for which there is a maximum number of pixels in the histogram or a luma value for which there is a maximum chroma energy, the chroma energy being calculated for a bin of the histogram by multiplying the number of pixels corresponding to the bin by the maximum chroma value found at the bin or the luma value for which there is a maximum average chroma energy, the average chroma energy being calculated for a bin of the histogram by multiplying the number of pixels corresponding to the bin by the maximum chroma value found at the bin.

10. A video processing device comprising electronic circuitry configured for: obtaining a current multi-component image; means for classifying the colors of pixels of the current multi-component image into a plurality of color categories using a tone mapping luma component and a correction normalized chroma component derived from the current multi-component image; for each color category, determining data representative of the color category including a dominant luma value representative of a luma for which colors in the category are dominant and determining (95) a value representative of a chroma gain from the data representative of the color category, the chroma gain being representative of a maximum allowed chroma value for chroma components in the color category; and, encoding the dominant luma value and the value representative of the gain corresponding to each category as metadata representing a saturation gain function in a bitstream, the function defining a color correction to be applied to pixels of the current multi-component image according to the luma of the pixels.

11. The device of claim 10, wherein the electronic circuitry is further configured to analyze the chroma components of the current multi-component image applied to each pixel of at least one subset of pixels of the current multi-component image to obtain the tone mapping luma component and the correction normalized chroma component of pixels of the current multi-component image, the analysis comprising: • deriving a luma component from the components of the pixel; • applying a tone mapping to the derived luma component to obtain a tone mapping luma component; • deriving a chroma component from the components of the pixel; and, • applying a joint normalization and color correction to the chroma component to obtain a correction normalized chroma component.

12. The device of claim 10, wherein the current multi-component image is included in a video sequence and the electronic circuitry is further configured for applying a temporal filtering to the information representative of the chroma gain based on information representative of chroma gains computed for images of the video sequence preceding the current multi-component image.

13. The device of claim 12, wherein the temporal filtering is reinitialized at the beginning of the video sequence or when a scene cut is identified in the video sequence.

14. The device of claim 10, wherein the color categories are color sectors in a chroma plane around primary colors and / or secondary colors.

15. The device of claim 14, wherein the sectors combine to cover the chroma plane as a whole.

16. The apparatus of claim 10, wherein determining data representative of the color class comprises obtaining a histogram of luminance values of pixels of the current multi-component image from luminance values of the color class.

17. The apparatus of claim 16, wherein only pixels corresponding to luminance values contained within a predefined range of luminance values are used to obtain the histogram.

18. The apparatus of claim 16, wherein the dominant luminance value corresponds to a luminance value where there is a maximum number of pixels in the histogram or a luminance value where there is a maximum chroma energy, the chroma energy is calculated for a bin of the histogram by multiplying the number of pixels corresponding to the bin by the maximum chroma value found at the bin or a luminance value where there is a maximum average chroma energy, the average chroma energy is calculated for a bin of the histogram by multiplying the number of pixels corresponding to the bin by the maximum chroma value found at the bin.

19. A computer program product, the computer program comprising program code instructions which, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 9.

20. An information storage device storing program code instructions which, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 9.

Images