Chroma boost of SDR and HDR display adaptive signals for SL-HDRx systems

Abstract

The invention provides a method comprising obtaining (90) a current RGB image; classifying (92) colors of pixels of the current RGB image into a plurality of categories; for each color category, determining (94) data representative of the color category comprising a dominant luminance value representative of a luminance in which a color in the category prevails and determining (95) from the data representative of the color category, a value representative of a chroma gain representative of a margin for increasing a chroma component in the color category; and encoding (96) the primary luminance value and a value representing a gain corresponding to each category as metadata representing a saturation gain function in the bitstream, the function defining a color correction to be applied to the pixel in accordance with the luminance of the pixel.

Description

[0001] This application is a divisional application of patent application No. 202180036592.0, filed on April 28, 2021, entitled "Chroma Enhancement of SDR and HDR Display Adaptation Signals for SL-HDRx Systems". 1. Technical Field At least one implementation of this embodiment relates in general to the distribution of HDR video using the SL-HDRx system (x=1, 2 or 3). 2. Background Technology Recent advances in display technology have begun to allow for an expanded dynamic range of color, brightness, and contrast in images to be displayed. The term "image" here refers to image content, such as video, still pictures, or images.

[0002] High Dynamic Range (HDR) video describes video with a dynamic range greater than that of Standard Dynamic Range (SDR) video. HDR video involves capture, production, content / encoding, and display. HDR capture and display are capable of rendering brighter whites and deeper blacks. To accommodate this, the HDR encoding standard allows for higher maximum luminance and uses at least 10 bits of dynamic range (compared to 8 bits for non-professional video and 10 bits for professional SDR video) to maintain accuracy within this extended range.

[0003] While technically "HDR" strictly refers to the ratio between maximum and minimum brightness, the term "HDR video" is also often understood to mean a wide color gamut.

[0004] Despite the emergence of numerous HDR display devices and image cameras capable of capturing images with increased dynamic range, the amount of available HDR content remains very limited. A solution is needed that allows for extending the dynamic range of existing content so that it can be efficiently displayed on HDR display devices.

[0005] The standard SL-HDR1 (ETSI TS 103 433-1 series, latest version v1.3.1) provides direct backward compatibility through the use of metadata that allows the reconstruction of HDR signals from SDR video streams, which can be delivered using existing SDR distribution networks and services. SL-HDR 1 allows for HDR rendering on HDR devices and SDR rendering on SDR devices using a single-layer video stream.

[0006] The standard SL-HDR2 (ETSI TS 103 433-2 series, latest version v1.2.1) is suitable for HDR devices. Standard SL-HDR2 allows the transmission of ST-2084 (also known as PQ (Perceptual Quantizer) or HDR10) streams along with metadata. When a stream is received by a device that is only compatible with ST-2084 but not with the metadata, the latter ignores the metadata and displays the image without knowing all its technical details (depending on the device model and its processing power, color rendering and gradation details may differ from the original source). When a device that supports the ST-2084 format and metadata receives the stream, it displays an optimized image that best matches the content creator's intent.

[0007] The standard SL-HDR3 (ETSI TS 103 433-3 v1.1.1) allows the transmission of HLG (Hybrid Log-Gamma) streams as well as metadata. An SL-HDR3 system comprises an HDR / SDR reconstruction block based on the SL-HDR2 HDR / SDR reconstruction block; that is, it consists of a cascade of HLG to ST-2084 OETF (Photoelectric Transfer Function) converters and SL-HDR2 HDR / SDR reconstruction blocks. The OETF describes the sensor's actions, converting scene brightness into data.

[0008] In the SL-HDRx system, the chromaticity of the SDR and HDR displays can be tuned to fit the signal because color correction adjustment variables are included in the SL-HDRx metadata. These color correction adjustment variable metadata define a piecewise function called the SGF (Saturation Gain Function), which modifies the default color correction function in any SL-HDRx process. Color correction depends on luminance (the Y component of the image signal), meaning that color correction modifies the color of a pixel (e.g., the U and V components) based on the luminance of that pixel.

[0009] Typically, SGF metadata uses coordinates ( sgf_x , sgf_y Define a maximum of six points. sgf_x Indicates brightness, and sgf_y This indicates the color correction at this brightness level. sgf_x and sgf_y The coordinates are, for example, values ​​contained between "0" and "255".

[0010] By default, SGF provides the same default color correction for every luminance value. This default color correction, which is typically defined empirically, results in neutral SDR and HDR display adaptation signals.

[0011] The basic solution for increasing the chromaticity of SDR and HDR display adaptation signals is to globally increase chromaticity by performing different color corrections on each of the luminance values. This solution has some limitations: • Only global color control is available. Therefore, if some colors are already sufficiently saturated, adding color correction to these colors will make them appear oversaturated; • Adding uncontrolled color correction, i.e., excessively increasing the U and V values, may cause the U and V values ​​to go out of range, which may result in the U and V values ​​being clipped, and thus causing reconstruction errors.

[0012] The aim is to overcome the above shortcomings.

[0013] It is particularly desirable to define a method that allows for better control over color correction in the SL-HDR1, SL-HDR2, and SL-HDR3 systems (and also in any subsequent variants of the SL-HDRx system). 3. Summary of the Invention In a first aspect, one or more embodiments of the present invention provide a method comprising: obtaining a current RGB image; analyzing the chromaticity components of the current RGB image, the analysis for each pixel of at least one subset of pixels of the current RGB image comprising: deriving a luminance component from the RGB components of the pixel; applying a tone mapping to the derived luminance component to obtain a tone-mapped luminance component; deriving the chromaticity component from the RGB components of the pixel; and applying joint normalization and color correction to the chromaticity component to obtain a corrected-normalized chromaticity component; classifying the colors of the pixels of the current RGB image into a plurality of categories using the tone-mapped luminance component and the corrected-normalized chromaticity component; for each color category, determining data representing the color category including a principal luminance value representing the luminance in which the color is dominant in the category and determining a value representing a chromaticity gain based on the data representing the color category, the chromaticity gain representing a margin for increasing the chromaticity component in the color category; and encoding the principal luminance value and the value representing the gain corresponding to each category into metadata representing a saturation gain function in a bit stream, the function defining a color correction to be applied to the pixel according to the luminance of the pixel.

[0014] In one implementation, the current RGB image is included in the video sequence, and a temporal filter is applied to the information representing the chroma gain based on the chroma gain calculated from the images in the video sequence preceding the current RGB image.

[0015] In one implementation, the temporal filter is reinitialized at the beginning of the video sequence or when a scene change is detected in the video sequence.

[0016] In one implementation, a color category is a color sector in the chromaticity plane surrounding pure primary colors and / or secondary colors.

[0017] In one implementation, the combination of sectors comprehensively covers the chromaticity plane.

[0018] In one implementation, determining the data representing the color category includes obtaining a histogram of pixels based on the brightness values ​​of the color category.

[0019] In one implementation, only pixels corresponding to brightness values ​​that fall within a predefined range are used to obtain the histogram.

[0020] In one implementation, the main luminance value corresponds to the luminance value of the pixel with the largest chromaticity value or the luminance value with the largest chromaticity energy in the histogram. The chromaticity energy of the histogram is calculated by multiplying the number of pixels corresponding to the bin by the luminance value of the largest chromaticity value or the luminance value with the largest average chromaticity energy found in the bin. The average chromaticity energy of the histogram is calculated by multiplying the number of pixels corresponding to the bin by the chromaticity value of the largest chromaticity value found in the bin.

[0021] In a second aspect, one or more embodiments of the present invention provide an apparatus comprising: means for obtaining a current RGB image; means for analyzing the chromaticity components of the current RGB image, the means for analysis for each pixel of at least one subset of pixels of the current RGB image comprising: means for deriving a luminance component from the RGB components of the pixel; means for applying a tone mapping to the derived luminance component to obtain a tone-mapped luminance component; means for deriving the chromaticity component from the RGB components of the pixel; and means for applying joint normalization and color correction to the chromaticity component to obtain a corrected-normalized chromaticity component. The apparatus includes: means for classifying the colors of pixels in a current RGB image into multiple categories using tone-mapped luminance components and corrected normalized chrominance components; means for determining, for each color category, data representing the color category including a principal luminance value indicating the luminance predominance of the color in the category and a value representing a chrominance gain based on the data representing the color category, the chrominance gain representing a margin for increasing the chrominance components in the color category; and means for encoding the principal luminance value and the value representing the gain corresponding to each category into metadata representing a saturation gain function in a bit stream, the function defining a color correction to be applied to the pixel according to the luminance of the pixel.

[0022] In one implementation, the current RGB image is included in a video sequence, and the device includes a temporal filtering device that is applied to information representing chroma gain based on chroma gain information calculated from images in the video sequence preceding the current RGB image.

[0023] In one implementation, the temporal filter is reinitialized at the beginning of the video sequence or when a scene change is detected in the video sequence.

[0024] In one implementation, a color category is a color sector in the chromaticity plane surrounding pure primary colors and / or secondary colors.

[0025] In one implementation, the combination of sectors comprehensively covers the chromaticity plane.

[0026] In one implementation, determining the data representing the color category includes obtaining a histogram of pixels based on the brightness values ​​of the color category.

[0027] In one implementation, only pixels corresponding to brightness values ​​that fall within a predefined range are used to obtain the histogram.

[0028] In one implementation, the main luminance value corresponds to the luminance value of the pixel with the largest chromaticity value or the luminance value with the largest chromaticity energy in the histogram. The chromaticity energy of the histogram is calculated by multiplying the number of pixels corresponding to the bin by the luminance value of the largest chromaticity value or the luminance value with the largest average chromaticity energy found in the bin. The average chromaticity energy of the histogram is calculated by multiplying the number of pixels corresponding to the bin by the chromaticity value of the largest chromaticity value found in the bin.

[0029] In a third aspect, one or more embodiments of the present invention provide a signal generated by the method of the first aspect or by the device of the second aspect.

[0030] In a fourth aspect, one or more embodiments of the present invention provide a computer program comprising program code instructions for implementing the method according to the first aspect.

[0031] In a fifth embodiment, one or more embodiments of the present invention provide an information storage device that stores program code instructions for implementing the method according to the first aspect. 4. Description of the attached drawings Figure 1 An example of the SL-HDRx system is shown; Figure 2 The post-processing module of the SL-HDRx system is shown schematically. Figure 3 An example of a hardware architecture for a processing module capable of implementing various aspects and implementation schemes is illustrated schematically; Figure 4 A block diagram illustrating an example of a first system in which various aspects and implementation schemes are carried out; Figure 5 A block diagram illustrating an example of a second system in which various aspects and implementation schemes are derived is shown; Figure 6The first example of a preprocessing procedure is illustrated schematically; Figure 7 A second example of the preprocessing procedure is illustrated schematically; Figure 8 An example of the reconstruction process in the post-processing stage is illustrated schematically; Figure 9 The method of controlling color correction in the SL-HDRx system is illustrated schematically; Figure 10 It represents the three primary colors (red, green, and blue) and the three secondary colors (magenta, yellow, and cyan) and the positions of their corresponding sectors; Figure 11 An example of a time-stabilized method is illustrated schematically; Figure 12 An example of the initialization phase of a time-stabilized method is illustrated schematically; Figure 13 An example illustrating the filter parameter calculation process of the time-stabilized method is shown. Figure 14 An implementation scheme for controlling color correction suitable for an SL-HDR2 system is schematically described; Figure 15 First details of the method for controlling color correction suitable for the SL-HDR2 system are shown; and, Figure 16 The second detail shows the method for controlling color correction suitable for the SL-HDR2 system. 5. Detailed Implementation Figure 1 An example of an SL-HDRx system is shown. SL-HDRx (x=1, 2, or 3) includes a preprocessing module 10, an encoding module 12, and a post-processing module 14. The preprocessing module 10 is communicatively connected to the encoding module 12 via a communication link 11. In SL-HDR2 and SL-HDR3 systems, the preprocessing module 10 generates HDR content and dynamic metadata from the raw content, while in the SL-HDR1 system, the preprocessing module generates SDR content and dynamic metadata from the raw content. The preprocessing module 10 integrates a computational part that generates the output HDR or SDR content and an analytical part that analyzes the content and generates dynamic metadata. The raw content may have been generated by an acquisition device (such as a camera) via a computer graphics system or a combination of an acquisition device and a computer graphics system. The HDR or SDR content may include static metadata, such as parameters of the acquisition device (i.e., camera) representing the acquisition environment of the HDR or SDR content.

[0032] The preprocessing module 10 is fed with HDR content that has been created during the post-production process applied to the original HDR or SDR video to obtain the main video and still metadata. The post-production process includes, for example: • Color grading process, such as introducing artistic effects in the main video; • VFX compositing process, used to introduce visual effects into the main video; • The tone mapping process allows for the generation of SDR master video from HDR video; • The inverse tone mapping process allows for the generation of HDR master video from SDR video.

[0033] Then, the preprocessing module 10 generates content and dynamic metadata suitable for SL-HDRx usage. In the SL-HDR1 system, the generated video is SDR video. In the SL-HDR2 system, the generated video is PQ HDR video. In the SL-HDR3 system, the generated video is HLG HDR video.

[0034] The following text is about Figure 6 An example of the preprocessing procedure implemented in the SL-HDR1 system is described.

[0035] The encoding module 12 receives the generated video and metadata from the preprocessing module 10, including both static metadata from post-production and dynamic metadata from SL-HDRx preprocessing, and is responsible for encoding the generated video and metadata.

[0036] Encoding module 12 generates, for example, an encoded video stream based on the generated video and metadata, which conform to the video compression standards HEVC (ISO / IEC 23008-2 – MPEG-H Part 2, High Efficiency Video Coding / ITU-T H.265) or AVC (ISO / IEC 14496-10 – MPEG-4 Part 10, Advanced Video Coding) or a standard under development called Universal Video Coding (VVC). Metadata is processed, for example, by SEI messages such as HEVC Color Remapping Information (CRI) or Master Display Color Volume (MDCV) SEI messages.

[0037] In the following text, we will refer to the combination of preprocessing module 10 and encoding module 12 as the input module.

[0038] During encoding, the encoded video is transmitted to the post-processing module 14 via communication link 13.

[0039] Figure 2 The post-processing module 14 is shown schematically.

[0040] The post-processing module 14 includes a decoder 140 adapted to decode the encoded main video and associated metadata.

[0041] In an SL-HDR1 system, the encoded video represents SDR video, and metadata is used to generate HDR video from the SDR video. During decoding, in a post-processing device without SL-HDR1 integration, the SDR video is transmitted to an SDR display device 18 via communication link 17. The SDR display device 18 then displays the decoded SDR video. In a post-processing device with SL-HDR1 integration, the post-processing module 14 includes a reconstruction module 141. The reconstruction module 141 receives the decoded SDR video from the decoder 140 and reconstructs HDR video from the decoded SDR video using metadata. The reconstructed HDR video is then transmitted to an HDR display device 16 that displays the video. In some cases, the display device 16 is not an HDR display device, but an SDR display device or an MDR (Medium Dynamic Range) display device that acts as an intermediary between an SDR and HDR display device. In these cases, the reconstruction module obtains information representing the display capabilities of the MDR display device 16 and considers these capabilities during reconstruction to reconstruct video suitable for the MDR display device. Once reconstructed, the HDR (or SDR or MDR) video is transmitted to the HDR (or SDR or MDR) display device 16 via communication link 15. The HDR (or SDR or MDR) display device 16 then displays the reconstructed HDR (or SDR or MDR) video.

[0042] In an SL-HDR2 system, the encoded video represents PQ HDR video, and metadata is used to generate SDR (or MDR) video from the decoded PQ HDR video. During decoding, in a post-processing device without integrated SL-HDR2, the PQ HDR video is transmitted to an HDR display device 18 via communication link 17. The HDR display device 18 then displays the decoded PQ HDR video. In a post-processing device without integrated SL-HDR2, a reconstruction module 141 receives the decoded PQ HDR video, metadata, and, in some cases, the display capabilities of a display device 16 from a decoder 140. This display device can be an SDR, MDR, or HDR display device. Based on this data, the reconstruction module generates a video signal suitable for the capabilities of the display device 16 (SDR, MDR, or HDR video). Once reconstructed, the SDR (or MDR, or HDR) video is transmitted to the SDR (or MDR, or HDR) display device 16 via communication link 15. The SDR (or MDR, or HDR) display device 16 then displays the reconstructed SDR (or MDR, or HDR) video.

[0043] In the SL-HDR3 system, the encoded video represents HLG HDR video, and metadata is used to generate SDR (or MDR) video from the decoded HLG HDR video. The post-processing module 14 in the SL-HDR3 system functions very similarly to the post-processing module 14 in the SL-HDR2 system. The main difference lies in the reconstruction module 141. In practice, in this case, the reconstruction module 141 comprises a cascade of the HLG to ST-2084 OETF converter and the SL-HDR2 reconstruction module as described above.

[0044] Figure 3 An example of the hardware architecture of the processing module 40 is schematically shown. This processing module is contained in the preprocessing module 10, the encoding module 12, the input module, or the postprocessing module 14, and is capable of implementing different aspects and implementation schemes. As a non-limiting example, processing module 40 includes the following items connected by communication bus 405: a processor or CPU (Central Processing Unit) 400 containing one or more microprocessors, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture; random access memory (RAM) 401; read-only memory (ROM) 402; storage unit 403, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drive and / or optical disk drive, or storage media reader, such as SD (Secure Digital) card reader and / or hard disk drive (HDD) and / or network accessible storage device; at least one communication interface 404 for exchanging data with other modules, devices, systems, or equipment. Communication interface 404 may include, but is not limited to, a transceiver configured to transmit and receive data via communication channel 5. Communication interface 404 may include, but is not limited to, a modem or network card.

[0045] Communication interface 404, for example, enables processing module 40 to: • When the processing module 40 is included in the preprocessing module 10, it receives SDR or HDR content and outputs the main video; • When the processing module 40 is included in the encoding module 12, it receives the main video and outputs the encoded main video including metadata; • When the processing module 40 is included in the input module, it receives SDR or HDR content and outputs an encoded main video including metadata; • When the processing module 40 is included in the post-processing module 40, it receives the encoded master video including metadata and outputs SDR, MDR and / or HDR video.

[0046] Processor 400 is capable of executing instructions loaded into RAM 301 from ROM 402, external memory (not shown), storage media, or a communication network. When processing module 40 is powered on, processor 400 is capable of reading instructions from RAM 401 and executing those instructions. These instructions form a computer program that causes, for example, preprocessing, encoding, decoding, or postprocessing procedures to be implemented by processor 400.

[0047] All or part of the algorithms and steps of the process can be implemented in software by executing a set of instructions by a programmable machine such as a DSP (Digital Signal Processor) or a microcontroller, or in hardware by a machine or dedicated component such as an FPGA (Field Programmable Gate Array) or an ASIC (Application-Specific Integrated Circuit).

[0048] Figure 4 A block diagram illustrating an example of System A is shown, which is adapted to implement a preprocessing module 10, an encoding module 12, or an input module, and wherein various aspects and embodiments are implemented. System A may be embodied as a device comprising the various components described below and configured to perform one or more aspects and embodiments described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, connected home appliances, servers, and cameras. Components of System A may be embodied individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, System A includes a processing module 40 that implements a preprocessing process, an encoding process, or both. In various embodiments, System A is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports.

[0049] Inputs to processing module 40 may be provided via various input modules as shown in box 52. Such input modules include, but are not limited to: (i) a radio frequency (RF) module that receives, for example, RF signals transmitted over the air by a broadcaster; (ii) a component (COMP) input module (or a set of COMP input modules); (iii) a universal serial bus (USB) input module; and / or (iv) a high-definition multimedia interface (HDMI) input module. Figure 4 Other examples not shown include composite video.

[0050] In various embodiments, the input module of block 52 has associated corresponding input processing elements as known in the art. For example, the RF module may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal, or limiting a signal band to a band), (ii) down-converting the selected signal, (iii) re-band-limiting the signal to a narrower band to select (e.g.,) a signal band that may be referred to as a channel in some embodiments), (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select the desired data packet stream. The RF module of various embodiments includes one or more elements for performing these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include tuners that perform various functions among these functions, including, for example, down-converting received signals to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or to baseband. Various implementations rearrange the order of the above (and other) components, remove some of these components, and / or add other components that perform similar or different functions. Adding components may include inserting components between existing components, such as inserting amplifiers and analog-to-digital converters. In various implementations, the RF module includes an antenna.

[0051] Additionally, the USB and / or HDMI modules may include corresponding interface processors for connecting System 3 to other electronic devices across USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented as needed, for example, within a separate input processing IC or within processing module 40. Similarly, various aspects of USB or HDMI interface processing may be implemented as needed, either within a separate interface IC or within processing module 40. Demodulation, error correction, and demultiplexing streams are provided to processing module 40.

[0052] Various components of System A can be housed within an integrated housing. Within the integrated housing, the various components can be interconnected using suitable connection arrangements (e.g., internal buses known in the art, including inter-IC (I2C) buses, wiring, and printed circuit boards) and data can be transferred between these components. For example, in System A, processing module 40 is interconnected with other components of System A via bus 405.

[0053] The communication interface 404 of the processing module 40 allows system A to communicate on communication channel 5. For example, communication channel 5 can be implemented in wired and / or wireless media.

[0054] In various implementations, a wireless network such as Wi-Fi, such as IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers), is used to stream or otherwise provide data to System A. In these implementations, the Wi-Fi signal is received via a communication channel 5 and a communication interface 404 suitable for Wi-Fi communication. The communication channel 5 in these implementations is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other cloud-based communications. Other implementations use the RF connection of input box 52 to provide streaming data to System A. As mentioned above, for example, when System A is a camera, smartphone, or tablet, various implementations provide data in a non-streaming manner. Additionally, various implementations use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0055] System A can use communication channel 5 or bus 405 to provide output signals to various output devices. For example, when implementing preprocessing module 10, system A uses bus 405 or communication channel 5 to provide output signals to encoding module 12. When implementing encoding module 12 or input module, system A uses communication channel 5 to provide output signals to postprocessing module 14.

[0056] Various specific implementations involve application preprocessing and / or encoding processes. As used in this application, the preprocessing or encoding process may encompass all or part of the processes performed, for example, on received SDR or HDR images or video streams, to produce the master video or master video encoded with metadata. In various implementations related to the encoding process, such processes include one or more processes typically performed by a video encoder, such as an H.264 / AVC (ISO / IEC 14496-10 – MPEG-4 Part 10, Advanced Video Coding), H.265 / HEVC (ISO / IEC 23008-2 – MPEG-H Part 2, High Efficiency Video Coding / ITU-T H.265), or H.266 / VVC (Universal Video Coding) encoder developed by a joint collaboration of ITU-T and ISO / IEC experts known as the Joint Video Experts Group (JVET).

[0057] Figure 5A block diagram of an example system B is shown, which is adapted to implement post-processing module 14 and therein implement various aspects and embodiments. System B may be embodied as a device including the various components described below and configured to perform one or more aspects and embodiments described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. The elements of system B may be embodied individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, system B includes a processing module 40 that implements the post-processing procedure. In various embodiments, system B is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports.

[0058] Inputs to processing module 40 may be provided via various input modules as shown in box 52. Such input modules include, but are not limited to: (i) a radio frequency (RF) module that receives, for example, RF signals transmitted over the air by a broadcaster; (ii) a component (COMP) input module (or a set of COMP input modules); (iii) a universal serial bus (USB) input module; and / or (iv) a high-definition multimedia interface (HDMI) input module. Figure 5 Other examples not shown include composite video.

[0059] In various embodiments, the input module of block 52 has associated corresponding input processing elements as known in the art. For example, the RF module may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal, or limiting a signal band to a band), (ii) down-converting the selected signal, (iii) re-band-limiting the signal to a narrower band to select (e.g.,) a signal band that may be referred to as a channel in some embodiments), (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select the desired data packet stream. The RF module of various embodiments includes one or more elements for performing these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include tuners that perform various functions among these functions, including, for example, down-converting received signals to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or to baseband. In one set-top box implementation, the RF module and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to the desired frequency band. Various implementations rearrange the order of the aforementioned (and other) components, remove some of these components, and / or add other components that perform similar or different functions. Adding components may include inserting components between existing components, such as inserting amplifiers and analog-to-digital converters. In various implementations, the RF module includes an antenna.

[0060] Additionally, the USB and / or HDMI modules may include corresponding interface processors for connecting System B to other electronic devices across USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented as needed, for example, within a separate input processing IC or within processing module 40. Similarly, various aspects of USB or HDMI interface processing may be implemented as needed, either within a separate interface IC or within processing module 40. Demodulation, error correction, and demultiplexing streams are provided to processing module 40.

[0061] Various components of System B may be housed within an integrated housing. Within the integrated housing, the various components can be interconnected using suitable connection arrangements (e.g., internal buses known in the art, including inter-IC (I2C) buses, wiring, and printed circuit boards) and data can be transferred between these components. For example, in System B, processing module 40 is interconnected with other components of System B via bus 405.

[0062] The communication interface 404 of the processing module 40 allows system B to communicate on communication channel 5. For example, communication channel 5 can be implemented in wired and / or wireless media.

[0063] In various implementations, a wireless network such as Wi-Fi, such as IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers), is used to stream or otherwise provide data to System B. In these implementations, the Wi-Fi signal is received via a communication channel 5 and communication interface 404 suitable for Wi-Fi communication. The communication channel 5 in these implementations is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other cloud-based communications. Other implementations use the RF connection of input box 52 to provide streaming data to System B. As mentioned above, various implementations provide data in a non-streaming manner. Additionally, various implementations use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0064] System B can provide output signals to various output devices, including a display 5, a speaker 6, and other peripheral devices 7. The display 5 in various embodiments includes one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display 5 can be, for example... Figure 1 Display device 16 or 18. Display 5 can be used in a television, tablet, laptop, cellular phone (mobile phone), or other device. Display 5 can also be integrated with other components (e.g., in a smartphone) or standalone (e.g., an external monitor for a laptop). Display device 5 is compatible with SDR, MDR, or HDR content. In various examples of embodiments, other peripheral devices 7 include one or more of a standalone digital video disc (or digital universal disc, both terms being DVR), disc player, stereo system, and / or lighting system. Various embodiments use one or more peripheral devices 7 that provide functionality based on the output of system B. For example, a disc player performs the function of playing the output of system B.

[0065] In various implementations, control signals are transmitted between System B and display 5, speaker 6, or other peripheral devices 7 using signaling protocols such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable device-to-device control with or without user intervention. Output devices are communicatively coupled to System B via dedicated connections through corresponding interfaces 53, 54, and 55. Alternatively, output devices can be connected to System B via communication interface 404 using communication channel 5. Display 5 and speaker 6 can be integrated into a single unit with other components of System B in electronic devices such as televisions. In various implementations, display interface 5 includes a display driver, such as, for example, a timing controller (TCon) chip.

[0066] For example, if the RF portion of input 52 is part of a separate set-top box, then display 5 and speaker 6 may optionally be separate from one or more other components. In various embodiments where display 5 and speaker 6 are external components, the output signal may be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0067] Various specific implementations involve post-processing processes within the decoding process. As used in this application, post-processing may encompass all or part of the processes performed, for example, on the received encoded master video, to produce an SDR, MDR, or HDR output suitable for display. In various implementations, such processes include one or more processes typically performed by an image or video encoder, such as an H.264 / AVC (ISO / IEC 14496-10 – MPEG-4 Part 10, Advanced Video Coding), H.265 / HEVC (ISO / IEC 23008-2 – MPEG-H Part 2, High Efficiency Video Coding / ITU-T H.265) or / and H.266 / VVC (Universal Video Coding) decoder developed by a joint collaboration of ITU-T and ISO / IEC experts known as the Joint Video Experts Group (JVET).

[0068] When the accompanying drawings are presented as flowcharts, it should be understood that block diagrams of the corresponding devices are also provided. Similarly, when the accompanying drawings are presented as block diagrams, it should be understood that flowcharts of the corresponding methods / processes are also provided.

[0069] The specific embodiments and aspects described herein may be implemented, for example, in methods or processes, apparatus, software programs, data streams, or signals. Even if discussed only in the context of a single form of specific embodiment (e.g., discussed only as a method), specific embodiments of the discussed features may be implemented in other forms (e.g., apparatus or program). Apparatus may be implemented, for example, in suitable hardware, software, and firmware. Methods may be implemented, for example, in a processor that generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices, such as, for example, computers, mobile phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0070] The reference to "an implementation scheme" or "implementation scheme" or "a specific implementation" or "specific implementation," and other variations thereof, means that the specific features, structures, characteristics, etc., described in connection with the implementation scheme are included in at least one implementation scheme. Therefore, the appearance of the phrase "in an implementation scheme" or "in an implementation scheme" or "in a specific implementation" or "in a specific implementation," and any other variations appearing throughout this application, do not necessarily refer to the same implementation scheme.

[0071] Additionally, this application may involve "determining" various types of information. Determining information may include, for example, estimated information, calculated information, predicted information, information retrieved from memory, or information obtained, for example, from another device, module, or user, one or more of these.

[0072] Furthermore, this application may relate to "accessing" various types of information. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information, or more of these.

[0073] Furthermore, this application may relate to "receiving" various types of information. Like "access," "receiving" is intended to be a broad term. Receiving information may include, for example, accessing information or retrieving information (e.g., from memory) or more. Moreover, "receiving" typically involves one or more of the following during operations such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0074] It should be understood that, for example, in the cases of “A / B,” “A and / or B,” “at least one of A and B,” and “one or more of A and B,” the use of any of the following “ / ,” “and / or,” and “at least one,” “one or more” is intended to cover selecting only the first listed option (A), or only the second listed option (B), or selecting both options (A and B). As a further example, in the cases of “A, B, and / or C,” “at least one of A, B, and C,” and “one or more of A, B, and C,” such phrases are intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or selecting all three options (A, B, and C). As will be apparent to those skilled in the art and related fields, this can be extended to as many of the listed items as possible.

[0075] It will be apparent to those skilled in the art that specific embodiments or implementations can produce various signals formatted to carry, for example, information that can be stored or transmitted. The information may include, for example, instructions for performing a method or data generated by one of the embodiments or implementations. For example, a signal may be formatted to carry an SDR or HDR image or video sequence of the embodiment. Such signals may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or baseband signals. Formatting may include, for example, encoding the SDR or HDR image or video sequence using metadata from an encoded video stream and using a stream-modulated carrier. The information carried by the signal may be, for example, analog or digital information. It is known that signals can be transmitted via various wired or wireless links. The signal may be stored on a processor-readable medium.

[0076] Figure 6 An example of the computational portion of the preprocessing procedure is schematically illustrated. The first example of the preprocessing procedure is suitable for an SL-HDR1 system in NCL (Non-Constant Luminosity) mode. In this example, preprocessing module 10 receives HDR content and generates a master video representing the SDR content and metadata. The preprocessing procedure is performed by processing module 40 on each pixel of each image of the HDR content. Figure 6 In the example, the pixel includes three color components corresponding to the three primary colors of red (R), green (G) and blue (B), that is, the pixel is an RGB signal.

[0077] In step 601, the processing module 40 derives the luminance (luma) component from the RGB signal. L’ As shown below: (Equation 1) in, It is a transformation matrix, and γ It is the gamma factor, for example, equal to "2.4".

[0078] In step 602, the processing module 40 applies tone mapping to the luminance component. L’ To obtain tone mapping values As shown below: (Equation 2) in and It is a lookup table representing tone mapping functions.

[0079] In step 603, processing module 40 applies gamma correction to the RGB signal, as shown below: (Equation 3) In step 604, processing module 40 derives the chroma component from the gamma-corrected RGB signal, as follows: (Equation 4) in and It is a transformation matrix. This is, for example, a standard 3x3 RGB to YUV conversion matrix (e.g., as specified in ITU-R Rec.BT.2020 or ITU-R Rec.BT.709, depending on the color space).

[0080] In step 605, processing module 40 applies joint normalization and color correction to the chromaticity components. and To obtain normalized corrected chromaticity components and As shown below: (Equation 5) and It is clipped within [-512; 511]. It's a scaling function: 1024 And when Then: 1024 in It is a color correction lookup table, as defined, for example, in section 7.2.3.2 of the document ETSI TS 103 433-1 v1.3.1.

[0081] In step 606, the processing module 40 adjusts the brightness value applied to the tone mapping. To obtain the adjusted tone map brightness value As shown below: (Equation 6) In step 607, the processing module 40 processes the luminance and chrominance values. , and Convert to the given output format. Step 607 includes converting the chromaticity components... and Add a value, for example, equal to "512". midsampleThe sub-steps optionally include downsampling the chroma components, which compresses the signal by reducing the number of chroma samples, and optional sub-steps converting from a full range of values ​​(ranging from "0" to "1023" for YUV components when encoded with 10 bits) to a finite range of values ​​(ranging from 64 to 940 for the Y component and from 64 to 960 for the UV component) to obtain the luminance and chroma components of the pixels representing the SDR signal. The purpose of step 607 is, for example, to convert a full-range YUV 444 signal into a limited-range YUV 420 signal.

[0082] Figure 7 A second example of the computational portion of the preprocessing procedure is schematically illustrated. This second example of the computational portion of the preprocessing procedure is suitable for an SL-HDR2 system. In this example, the preprocessing module 10 receives HDR content and generates a master video representing the HDR PQ signal and metadata. The preprocessing procedure is performed by the processing module 40 on each pixel of each image of the input HDR content. Figure 7 In the example, the pixels are also RGB signals.

[0083] In step 701, the processing module 40 obtains the luminance and chrominance components of the PQ signal from the RGB signal. , , :

[0084] in It is a transformation function, and R'' (They are respectively) G'' and B'' The RGB signal is obtained from the RGB signal using an RGB to PQOETF converter.

[0085] In step 702, the processing module 40 processes the luminance and chrominance values. , and Convert to the output format. Step 702 includes converting the chroma components... and Add a value, for example, equal to "512". midsample The sub-steps optionally include downsampling the chroma components, which compresses the signal by reducing the number of chroma samples, and optional sub-steps converting from a full range of values ​​(ranging from "0" to "1023" for YUV components when encoded in 10 bits) to a finite range of values ​​(ranging from 64 to 940 for the Y component and from 64 to 960 for the UV component) to obtain the luminance and chroma components of the pixels representing the HDR signal. The purpose of step 702 is, for example, to convert a full-range YUV 444 signal into a limited-range YUV 420 signal.

[0086] Figure 8 An example of the reconstruction process of the post-processing procedure is illustrated schematically. When the processing module 40 implements the post-processing module 14 and more specifically the reorganization module 141, it is executed by the processing module 40. Figure 8 The reconstruction process is applied to each pixel of the decoded master video generated by decoder 140. The following describes the adaptation of the reconstruction process to SL-HDR1 and SL-HDR2. Since the SL-HDR3 specification is based on the SL-HDR2 specification, all SL-HDR2 features are also applicable to SL-HDR3 as described below. Figure 8 The reconstruction process follows, for example Figure 6 or Figure 7 The preprocessing process is used to determine the input signal for the reconstruction process. Therefore, the reconstruction process receives a limited range of YUV 420 signals.

[0087] In step 801, the processing module 40 converts the received YUV 420 signal into a full-range YUV 444 signal (the reverse process of steps 607 and 702).

[0088] In the case of the SL-HDR1 system, the YUV 420 signal is SDR pixels. Once converted, the SDR pixels consist of luminance and chrominance components. SDR y , SDR cb , SDR cr express.

[0089] In the case of the SL-HDR2 system, the YUV 420 signal is the HDR pixel. Once converted, the HDR pixel consists of luminance and chrominance components. HDR y , HDR cb , HDR cr express.

[0090] After conversion, the processing module 40 centers the chroma components to obtain centered chroma components. and In the case of the SL-HDR1 system, centering is performed as follows:

[0091] In the case of the SL-HDR2 system, centering is performed as follows:

[0092] in midsample For example, it equals "512".

[0093] In step 802, processing module 40 readjusts the parameters applied to the luminance component. In the case of the SL-HDR1 system, the readjustment calculation is as follows:

[0094] Where parameters and As defined in section 7.2.4 of the document ETSI TS 103 433-1 v1.3.1, max(x,y) takes the maximum value of x and y.

[0095] In the case of the SL-HDR2 system, readjusting the calculations is even simpler:

[0096] In SL-HDR1 and SL-HDR2, the brightness value Then it is clipped within [0; 1023] to obtain .

[0097] In step 803, processing module 40 constructs a color correction lookup table. lutCC[Y] .

[0098] In the case of SL-HDR1, the construction of the color correction lookup table is specified in section 7.2.3.2 of document ETSI TS 103 433-1 v1.3.1:

[0099] in, lutCC[0] =0.125 and , , As specified in document ETSI TS 103 433-1v1.2.1.

[0100] In the case of SL-HDR2, the construction of the color correction lookup table is specified in section 7.2.3.2 of document ETSI TS 103 433-2 v1.2.1:

[0101] in lutCC[0] =0.125 and , , , As specified in document ETSI TS 103 433-2 v1.2.1.

[0102] In the case of both SL-HDR1 and SL-HDR2 Defined as:

[0103] Where the saturation gain function It is derived from the piecewise linear pivot points defined by the metadata of the saturation gain function sgf_x and sgf_y, as detailed in section 7.3 of document ETSI TS 103 433-1 v1.3.1.

[0104] In step 804, the processing module 40 uses the constructed color correction lookup table. For the centered chromaticity component and Apply inverse color correction.

[0105] In the case of SL-HDR1, inverse color correction is described in section 7.2.4 of document ETSI TS 103 433-1 v1.3.1:

[0106] In the case of SL-HDR2, inverse color correction is described in section 7.2.4 of document ETSI TS 103 433-2 v1.2.1:

[0107] In step 805, the processing module calculates the intermediate value. , and In the case of SL-HDR1, the variables T The calculations are as follows:

[0108] in and It is described in section 7.2.4 of the document ETSI TS 103 433-1V1.2.1.

[0109] if , ,but and .

[0110] Otherwise, if T>1, =0, then and The following derivation shows:

[0111] This last equation applies only to SL-HDR1 "CL" mode. In SL-HDR1 "NCL" mode and SL-HDR2, , =1, and as well as .

[0112] It can be noted that, Corresponding to , and Corresponding to and .

[0113] In step 806, the processing module calculates the intermediate RGB reconstructed values. , and This is done in two steps.

[0114] In the case of SL-HDR1 , and This is described in section 7.2.4 of document ETSI TS 103 433-1V1.3.1:

[0115] in =matrixCoefficient[i] is part of the SL-HDRx metadata described in section 6.3.2.6 of document ETSI TS 103 433-1 v1.3.1.

[0116] In the case of SL-HDR2 , and This is described in section 7.2.4 of document ETSI TS 103 433-2 v1.2.1:

[0117] in =matrixCoefficient[i] is part of the SL-HDRx metadata described in section 6.3.2.6 of document ETSI TS 103 433-1 v1.3.1.

[0118] In the second step, the intermediate value is calculated. , and As described in Section 7.2.4 of the documents ETSI TS 103 433-1 v1.3.1 (for SL-HDR1) and ETSI TS 103 433-2 v1.2.1 (for SL-HDR2):

[0119] LutMapY The calculations are detailed in section 7.2.3.1 of the documents ETSI TS 103 433-1 v1.3.1 (for SL-HDR1) and ETSI TS 103433-2 v1.2.1 (for SL-HDR2).

[0120] It should be noted that in the case of SL-HDR 1, LutMapY Used as an inverse tone mapping lookup table, it adapts the SDR of the SL-HDR1 post-processor to the display. The input luminance signal is converted into an HDR output signal, or an SDR or MDR output signal.

[0121] In the case of SL-HDR2 LutMapY Used as a tone mapping lookup table, it adapts the HDR of the SL-HDR2 post-processor to the display. The input luminance signal is converted into an HDR output signal, or an SDR or MDR output signal.

[0122] In step 807, the output HDR RGB reconstructed signal is calculated. , and .

[0123] In the case of SL-HDR1, the calculations are described in section 7.2.4 of document ETSI TS 103 433-1 v1.3.1:

[0124] In the case of SL-HDR2, the calculations are described in section 7.2.4 of document ETSI TS 103 433-2 v1.2.1:

[0125] Therefore, the sgf_x and sgf_y metadata can be used to control the color correction of the SL-HDRx system. These metadata are linked to a color correction lookup table that controls the chroma component saturation. Yes, it has an impact. In the case of SL-HDR1, the sgf_x and sgf_y metadata will be controlled in step 605 of the HDR decomposition process. and The generation of, and therefore control SDR output. In the case of SL-HDR2, the sgf_x and sgf_y metadata will be controlled in step 804 of processing module 40. and The generation of chroma will thus control the chroma output of the SL-HDR2 reconstruction process.

[0126] Figure 9 The diagram schematically illustrates a method for controlling color correction in an SL-HDRx system. When processing module 40 implements preprocessing module 10 or input module, it performs the following actions regarding... Figure 9 The method described. This method is applied to each image or video. Figure 9 The method is described in the context of the SL-HDR1 system in NCL mode. Processing module 40 receives HDR content.

[0127] In step 90, the processing module 40 obtains the current image of the HDR content.

[0128] In step 91, processing module 40 analyzes the chromaticity of the current image. To this end, processing module 40 applies... Figure 6 The process continues until step 605 to obtain the three color components of each pixel in the current image. , and .

[0129] In step 92, the processing module 40 classifies the colors of the pixels in the current image by three components in the color category. , and This indicates that, in one embodiment of step 92, six categories are used: • The three categories correspond to the three primary colors: red, green, and blue; • The three categories correspond to three secondary colors: magenta, cyan, and yellow.

[0130] Each color can be represented using many different color spaces. In one implementation, RGB and its polar coordinates hue (H) and chromaticity (C) are used. Hue (H) and chromaticity (C) are calculated as follows:

[0131]

[0132] In the case of SL-HDR1

[0133]

[0134] An orientation is defined for each of the three primary and three secondary colors using the following formula in polar coordinates, where “c” in [0 … 1] is the normalized value of each RGB value (c=1 corresponds to a primary or secondary color, while c=0 is the achromatic origin of the UV plane): Regarding red (R=1, G=0, B=0): •

[0135] •

[0136] •

[0137] •

[0138] in au and av It is a transformation matrix A The coefficients allow for conversion from RGB to YUV:

[0139] In the BT.2020 color gamut, A= And in the BT.709 color gamut, A= .

[0140] Regarding green (R=0, G=1, B=0): •

[0141] •

[0142] •

[0143] •

[0144] Regarding blue (R=0, G=0, B=1): •

[0145] •

[0146] •

[0147] •

[0148] Regarding magenta (R=1, G=0, B=1): •

[0149] •

[0150] •

[0151] •

[0152] Regarding cyan (R=0, G=1, B=1): •

[0153] •

[0154] •

[0155] •

[0156] Regarding yellow (R=1, G=1, B=0): •

[0157] •

[0158] •

[0159] •

[0160] Figure 10 This indicates the positions of the three primary colors (red, green, and blue) and secondary colors (magenta, yellow, and cyan) calculated using the above formula in the UV plane.

[0161] If its hue H value is equal to the hue value calculated above. , , , , , One of the hue values ​​is determined by... and This indicates that a given pixel belongs to either the primary color system or the secondary color system.

[0162] However, in images, colors are rarely "pure" primary or secondary colors. Therefore, instead of defining six sectors (each centered on one of the six primary or secondary colors), we define six categories each corresponding to one of the primary or secondary colors. For each sector, we define a bias angle. Δ The corresponding sector is defined by four points: •Up = C max cos (H + Δ) •Vp = C max sin (H + Δ) •Um = C max cos (H - Δ) •Vm = C max sin (H - Δ) in • as well as H = Used for red; • as well as H = For green purposes; • as well as H = Used for blue; • as well as H = Used for magenta; • as well as H = Used for cyan; • as well as H = Used for yellow; Inspection Indication and One way to determine whether a given pixel belongs to a sector is to compute a vector of the product of the normalized values ​​of that pixel using the two limits of the sector, as shown below:

[0163]

[0164]

[0165]

[0166] if( )and( If a pixel is a sector, then that pixel belongs to that sector.

[0167] In one implementation, the value Δ It can be fixed and the same for all sectors. In other implementations, the value for each sector... Δ They are all different.

[0168] In one implementation, all sectors are contiguous, meaning that any pixel in the frame will belong to one sector. In other implementations, at least some sectors are not contiguous, meaning that some pixels may not belong to any sector.

[0169] Figure 10 This represents six sectors corresponding to the three primary colors and three secondary colors in the UV plane. The sectors are defined by dashed lines 1000 to 1005. For example, the sector corresponding to red is defined by dashed lines 1000 and 1001.

[0170] In step 93, processing module 40 generates a statistical representation for each sector (i.e., each category). In one embodiment, the statistical representation includes: • Histogram histo This indicates the number of pixels found in the sector for each brightness value, and represents the number of pixels in the SL-HDR1 case.

[0171] • For each brightness value in the histogram lum ,vector frame_chr_max[lum] This represents all sectors with brightness values. lum The maximum chromaticity value of the pixel; • for Each brightness value in the histogram lum ,value frame_chr_av[lum] This represents all sectors with brightness values. lum The average chromaticity value of the pixels.

[0172] It can be noted that the chromaticity value of the current pixel chr_curr The calculations are as follows: ,in and These are the U and V components of the current pixel.

[0173] In one implementation, the histogram histo The number of bins and the number of directory entries for the two vectors are set to "64". In another implementation, the number of bins and the number of directory entries for the two vectors are set to "256", corresponding to the metadata definition for the saturation gain function in the SL-HDRx standard. sgf_x Value range. In another implementation, the number of bins in the histogram and the number of catalog entries in the two vectors can be less than "64" or greater than "64".

[0174] In step 94, processing module 40 determines data representing each sector (i.e., for each color category). In one embodiment, the data representing a sector is the dominant luminance value corresponding to the dominant color luminance in the current sector. A histogram corresponding to the sector is used. histo and the vector corresponding to the sector frame_chr_max [lum] and frame_chr_av[lum] At least one of them is used to determine the main brightness value of the sector.

[0175] In the first embodiment of step 94, the main brightness value of the sector is obtained by scanning the histogram of the sector. histo This was determined by identifying the bin with the highest number of pixels (i.e., the brightness value corresponding to the highest number of pixels).

[0176] Step 94 allows obtaining the vector frame_idx_max_histo This vector includes a primary chromaticity value for each primary and secondary color (i.e., for each sector). For each sector, the SGF function will ultimately be used to modify the chromaticity of the color at that primary chromaticity value.

[0177] In step 95, processing module 40 determines a chromaticity gain (i.e., scaling value or color correction) to be applied to the chromaticity value of each color sector. To this end, processing module 40 determines the maximum permissible chromaticity value for the color corresponding to the dominant luminance value of each sector. This chromaticity gain represents the margin for increasing the chromaticity of that color and indirectly represents the maximum permissible chromaticity value for each sector and each luminance value.

[0178] As mentioned above, matrix operations can be used to derive YUV values ​​from RGB values:

[0179] For the three primary colors and three secondary colors, when processing the normalized RGB values ​​(values ​​within [0; 1]), this results in: • Regarding the primary color red (component R), processing module 40 calculates the following values: •R=s and G=B=0; ○ and (in the BT.2020 color gamut) ); ○ ; ○ ; ○ and (in the BT.2020 color gamut) ); Then, processing module 40 calculates the envelope of the permissible chromaticity values ​​for red. In one embodiment, this envelope consists of two straight lines: • In the YC (luminance / chrominance) space, from coordinates ( Y =0; C The first point of =0) AR To coordinates ( Y = ; C = The second point BR An increasing straight line.

[0180] • In the YC (luminance / chrominance) space, from coordinates ( Y =1; C The second to third point of (=0) CR A decreasing straight line.

[0181] • Regarding the primary color green (component G), processing module 40 calculates the following values: •G=s and R=B=0; ○ and (in the BT.2020 color gamut) ); ○ ; ○ ; ○ and (in the BT.2020 color gamut) ); Then, processing module 40 calculates the envelope of the permissible chromaticity values ​​for green. In one embodiment, this envelope consists of two straight lines: • In the YC (luminance / chrominance) space, from coordinates ( Y =0; C The first point of =0) AG To coordinates ( Y = ; C = The second point BG An increasing straight line.

[0182] • In the YC (luminance / chrominance) space, from coordinates ( Y =1; C The second to third point of (=0) CG A decreasing straight line.

[0183] • Regarding the primary color blue (component B), processing module 40 calculates the following values: •B=s and R=G=0; ○ and (in the BT.2020 color gamut) ); ○ ; ○ ; ○ and (in the BT.2020 color gamut) ); Then, processing module 40 calculates the envelope of the permissible chromaticity values ​​for blue. In one embodiment, this envelope consists of two straight lines: • In the YC (luminance / chrominance) space, from coordinates ( Y =0; C The first point of =0) AB To coordinates ( Y = ; C = The second point BB An increasing straight line.

[0184] • In the YC (luminance / chrominance) space, from coordinates ( Y =1; C The second to third point of (=0) CB A decreasing straight line.

[0185] • Regarding the secondary color magenta, processing module 40 calculates the following values: •R=B=s and G=0; ○ and (in the BT.2020 color gamut) ); ○ and ; ○ and ; ○ and (in the BT.2020 color gamut) ); Then, processing module 40 calculates the envelope of the permissible chromaticity values ​​for magenta. In one embodiment, this envelope consists of two straight lines: • In the YC (luminance / chrominance) space, from coordinates ( Y =0; C The first point of =0) AM To coordinates ( Y = ; C = The second point BM An increasing straight line.

[0186] • In the YC (luminance / chrominance) space, from coordinates ( Y =1; C The second to third point of (=0) CB A decreasing straight line.

[0187] • Regarding secondary cyan, processing module 40 calculates the following values: •G=B=s and R=0; ○ and (in the BT.2020 color gamut) ); ○ and ; ○ and ; ○ and (in the BT.2020 color gamut) ); Then, processing module 40 calculates the envelope of the permissible chromaticity values ​​for cyan. In one embodiment, this envelope consists of two straight lines: • In the YC (luminance / chrominance) space, from coordinates ( Y =0; C The first point of =0) AC To coordinates ( Y = ; C = The second point BC An increasing straight line.

[0188] • In the YC (luminance / chrominance) space, from coordinates ( Y =1; C The second to third point of (=0) CC A decreasing straight line.

[0189] • Regarding the secondary color yellow, processing module 40 calculates the following values: ○G=R=s and B=0; and (in the BT.2020 color gamut) ); and ; and ; and (in the BT.2020 color gamut) ); Then, processing module 40 calculates the envelope of the permissible chromaticity values ​​for cyan. In one embodiment, this envelope consists of two straight lines: ○ In the YC (luminance / chrominance) space, from coordinates (Y =0; C The first point of =0) AY To coordinates ( Y = ; C = The second point BY An increasing straight line.

[0190] ○ In the YC (luminance / chrominance) space, from coordinates ( Y =1; C The second to third point of (=0) CY A decreasing straight line.

[0191] In one implementation, the processing module 40 uses six vectors chr_envelop[S] Represents the envelope of six computations, one for each sector (i.e., one for each category), where S Indicates the index of the sector. For each sector... S ,vector chr_ envelop[S] Including histogram representation histo [S] Information on the maximum allowed chromaticity values ​​for each bin.

[0192] Within a sector, processing module 40 uses vectors. chr_envelop[S] To determine the maximum allowable scaling value (i.e., the maximum allowable gain). scale_max[S] As shown below: cur_idx=frame_idx_max_histo[S] ; scale_max[S]=chr_envelop[S][cur_idx] / frame_chr_max[S][cur_idx] .

[0193] In each sector S In the middle, the maximum allowed scaling value scale_max[S] Provides a scaling factor for chromaticity at the selected brightness level. Maximum allowed scaling value. scale_max[S] It allows for the creation of images with more chroma in a controlled and independent manner, enabling the processing of colors that require greater chroma.

[0194] In step 96, the processing module encodes the information representing chroma gain into metadata in the bitstream. The metadata conforms to SL-HDR1. In one implementation, the information representing chroma gain is a vector. frame_idx_max_histo and maximum allowed scaling value scale_max The vector. This information is encoded in the SL-HDR1 metadata as an SGF function.

[0195] In histogram histo [S] In one implementation where the number of bins is "256", frame_idx_max_ histo The range directly matches the metadata definition for the saturation gain function in the SL-HDRx standard. sgf_x The value range, and no adjustment is required, i.e. frame_idx_max_histo The value can be directly copied to the SL-HDRx metadata. sgf_x In one implementation where the number of bins is different from "256" (e.g., "64"), the processing module 40 is encoded into the definition of the SGF function. sgf_x Before that, the vector frame_idx_max_histo Rescale to "256".

[0196] Processing module 40 will then process six available sgf_x One of the values ​​is assigned to the index corresponding to the value. cur_idx = frame_idx_max_histo[S] The number of bins, and using the maximum allowed scaling value. scale_max[S] Modify the corresponding default sgf_y The value (typically equal to "118" in the case of SL-HDR1 NCL). In one implementation where the bin count is equal to "256" (six sectors, i.e., three primary colors and three secondary colors), the processing module 40 will have six available... sgf_x One of the values ​​is assigned to the index corresponding to the value. cur_idx = frame_idx_max_histo[S] The number of bins in one of the indices, and using the maximum allowed scaling value. scale_max[S] To modify the corresponding default sgf_y value: For each sector S: sgf_x[S] = frame_idx_max_histo[S] and sgf_y[S] = scale_max[S] .

[0197] Then process module 40 pairs sgf_x and sgf_y Values ​​are reordered to ensure that when i When increasing, sgf_x[i] The value monotonically increases. Reordered sgf_x and sgf_y The value allows the definition of an SGF function, which is transmitted to post-processing module 14 in the form of metadata.

[0198] In one embodiment of step 91, analysis is performed on a subsampled version of the current image.

[0199] In other embodiments of step 92, any other category representing different hues or colors or different quantities of categories may be used.

[0200] In one embodiment of step 93, dark values ​​(low brightness values) are not considered during histogram construction because it is difficult to distinguish the colors of dark values. For example, brightness values ​​below a first brightness threshold are not considered.

[0201] In one embodiment of step 93, luminance values ​​(high luminance values) are not considered during histogram construction because it is difficult to distinguish the color of luminance values. For example, luminance values ​​above a first luminance threshold are not considered.

[0202] In one embodiment of step 93, chromaticity values ​​below a chromaticity threshold are not considered in the construction of the histogram. chr_curr ).

[0203] In the second embodiment of step 94, the processing module 40 processes each histogram. histo (That is, for each sector) calculate the value representing the maximum chromaticity energy in the sector. max_energy_chroma Therefore, for each histogram histo For each bin, the processing module 40 counts the number of pixels at that bin. histo[lum] Multiply by the maximum chromaticity value found at that bin in the corresponding sector. frame_chr_max[lum] To calculate the value representing chromaticity energy. energy_chroma[lum] Then, for each sector, the processing module determines... energy_chroma[lum] The maximum value is used to determine the maximum chromaticity energy. max_ energy_chroma The principal luminance value corresponds to the maximum chromaticity energy. max_energy_chroma The second implementation has the advantage of associating the maximum chromaticity value with the luminance value (or number of bins) of the bin, thus providing a better understanding of the most attractive areas in the image.

[0204] In the third embodiment of step 94, the processing module 40 processes each histogram. histo (That is, for each sector) calculate the value representing the maximum average chromaticity energy in the sector. max _ av_energy_chroma Therefore, the processing module 40 will process the maximum chromaticity value during the second implementation process. frame_chr_max[lum] Replace with average chromaticity value frame_chr_av [lum] The principal luminance value corresponds to the maximum average chromaticity energy. max_av_energy_chroma The brightness value.

[0205] In variations of the first, second, and third embodiments of step 94, only bins comprising at least the minimum number of pixels are considered during scanning.

[0206] In variations of the first, second, and third embodiments of step 94, only the bin corresponding to the envelope value of the permissible chromaticity value, which is higher than the minimum chromaticity value, is considered during scanning. Chr_trigger This implementation avoids considering colors with extremely low saturation. In variations of the first, second, and third implementations in step 94, each histogram is processed before searching for the principal brightness value. histoPreprocessing is performed to smooth out the final noise or remove excessively small peaks.

[0207] In one embodiment of step 95, the maximum allowable scaling value scale_max[S] It can be limited to the maximum value. absolute_scale_max[S] To avoid oversaturation of colors. In one implementation, the maximum value... absolute_scale_ max[S] For all color sectors to be the same, or for each sector to be different. In this case, the maximum allowed scaling value is... scale_ max[S] The calculations are as follows: scale_max[S]=min(scale_max[S] ; absolute_scale_max[S]) .

[0208] In one embodiment of step 95, the maximum allowable scaling value scale_max[S] It has a minimum value to avoid desaturating colors, even if analysis has shown that the envelope of the chromaticity value is allowed to be lower than the current maximum chromaticity. chr_max[S] That's also true.

[0209] In one embodiment of step 95, when it is a sector S Determine the maximum allowable scaling value scale_max[S] At that time, you can be in the index cur_idx = frame_idx_max_histo[S] The same calculation is performed for other color sectors. If it is the same as the current color sector... S At least one different sector S' Maximum allowed scaling value scale_max[S'] Below the maximum allowed scaling value scale_max[S] Then the processing module 40 will set the maximum allowed scaling value. scale_max[S] The limit is the lower maximum allowed scaling value found in another sector. scale_max[S'] .

[0210] In one embodiment of step 95, the maximum U and V value scaling can be further analyzed to avoid indexing. cur_idx = frame_idx_max_histo[S] Clip the colored sector instead of the current sector. S Maximum allowed scaling value scale_max[S] Restricted to index cur_idx = frame_idx_max_histo[S] Other sectors S' The maximum allowed scaling value found scale_max[S'] The minimum value. This analysis provides a limit for... scale_max[S] value scale_max_UV As shown below: scale_max[S] = min(scale_max[S], scale_max_UV) ; If time-stabilized (i.e., time filtering) is not applied, then the vector frame_idx_max_histo[S] and scale_ max[S]There is a risk of fluctuation. Due to these fluctuating vectors, the preprocessor 10 risks generating unstable and unacceptable SDR image sequences.

[0211] In optional step 97, the processing module applies a time-stabilized method.

[0212] Figure 11 An example implementation of the time-stabilized optional step 97 is described in detail.

[0213] In step 971, processing module 40 determines whether the current image of the HDR content corresponds to a scene change. To do this, for example, processing module 40 compares the current image with images preceding the current image in the HDR content. For example, if the difference calculated as the sum of the absolute differences between the co-located pixels of the two images is higher than a threshold, processing module 40 determines that the current image corresponds to a scene change. In this case, step 971 proceeds to step 973. Otherwise, step 971 proceeds to step 972.

[0214] In step 973, processing module 40 initializes a set of parameters for the time stabilization method. In other words, time stabilization is reinitialized during step 973.

[0215] In step 972, processing module 40 calculates the filter vector. frame_idx_max_histo and scale_max .

[0216] In one implementation, during step 971, the processing module determines whether the current image is the first image of the HDR content, rather than searching for a scene switch.

[0217] Figure 12 An example of an implementation scheme for step 973 is described in detail.

[0218] exist Figure 12 In the example, a configurable circular buffer frame_idx_max_histo_buf[S] (respectively) scale_max_buf[S] ) is used for each parameter frame_idx_max_histo[S] (respectively) scale_max[S] The filtered version of the parameters is calculated. In one implementation, each buffer has the same size. n This represents the number of consecutive frames considered for calculating the filtered version of the corresponding parameters. In one implementation, the buffer size... n =10. For each parameter frame_idx_max_histo[S] (respectively) scale_max[S] Define invalid values frame_idx_max_histo_ invalid (respectively) scale_max_invalid When through Figure 9When the method used to determine color correction generates this invalid value, it means that no valid histogram index and index were calculated for the current color sector of the current frame. scale_max The value indicates the chromaticity of the current color sector that does not need to be scaled for the current frame. For example, if a value of "64" is defined for luminance, then... frame_ idx_max_histo[S] Between "0" and "63". Scale_max[S] It can also be defined, for example, as not exceeding "5". If for red (S=red), use... Figure 9 The processing module 40, which determines the method for color correction, has determined that red can be saturated. frame_idx_max_histo[red] Within the range [0; 63] and scale_max[red] Within the range of [0; 5]. However, if using Figure 9 In this method, if processing module 40 has determined that red must not be saturated, then invalid values ​​will be assigned to it. frame_idx_max_histo[red] (e.g., "64") and scale_max[red] (For example, "10"). Therefore, using Figure 9 The method, processing module 40 according to frame_idx_max_histo[red] and scale_max[red] The values ​​are known to be valid and therefore need to be stabilized.

[0219] Therefore, when an invalid value is detected, there is no need for time to stabilize the current parameter. Each value in each buffer is initialized as described below.

[0220] As mentioned above, in Figure 12 During the process, it is assumed that all buffers have the same size. n : In step 973A, the processing module 40 will process the variable representing the sector (i.e., the color). S Initialize to zero.

[0221] In step 973B, processing module 40 determines the variables. S Is it less than the number of sectors? NumOfSectors .For example, NumOfSectors= 6.

[0222] if S = NumOfSectors If so, the processing module 40 stops the initialization process 973.

[0223] Otherwise, in step 973D, processing module 40 will change the variable i Initialize to zero.

[0224] In step 973E, processing module 40 determines i Is it smaller than the buffer size? n .

[0225] if i=n Then, in step 973F, the processing module 40 will process the variable... S Increment by one unit.

[0226] Otherwise, processing module 40 determines the parameters. frame_idx_max_histo[S] Is it different from an invalid value? frame_ idx_max_histo_invalid .if frame_idx_max_histo[S] = frame_idx_max_histo_ invalid Then, in step 973H, the processing module 40 will... frame_idx_max_histo_buf[S][i] The value is set to frame_idx_max_invalid Otherwise, processing module 40 will in step 973I... frame_idx_max_histo_buf[S] [i] The value is set to frame_idx_max_histo[S] .

[0227] Steps 973H and 973I are followed by step 973J, during which processing module 40 will process the parameters. scale_max [S] Invalid values scale_max_invalid Compare. If scale_max[S] = scale_max_invalid Then the processing module 40 will value scale_max_buf[S][i] Set as scale_max_invalid Otherwise, processing module 40 will change the value in step 973L. scale_max_buf[S][i] Set as scale_max[S] .

[0228] In step 973M, the processing module 40 will assign the value frame_idx_max_histo_buf[S][i] Add to cumulative value cum_frame_idx_max_histo[S] Cumulative value cum_frame_idx_max_histo[S] This represents all values ​​in the corresponding buffer. It is a weighting factor. In one implementation scheme, In another implementation scheme, for i Each value is different. In the last case, the cumulative value... cum_frame_idx_max_histo[S] yes frame_ The weighted sum assigns more weight to specific locations in the buffer.

[0229] In step 973N, the processing module 40 will assign the value idx_max_histo_buf[S][i] Add to cumulative value scale_max_buf[S][i] Cumulative value This represents all values ​​in the corresponding buffer.

[0230] In step 9730, the processing module 40 initializes the index representing the current image's position in the buffer.cum_ .

[0231] In one implementation, when all buffers have the same size, scale_max[S] =0.

[0232] In another implementation, with vector cum_scale_max[S] and filterIndex Each buffer associated with the parameter has a different size. In this case, each buffer has an index. filterIndex .

[0233] frame_idx_max_histo An example of an implementation scheme for step 972 is described in detail.

[0234] The purpose of the example implementation of step 972 is to analyze vectors. scale_max and filterIndex Parameter filtering. Figure 13 The process is performed by processing module 40. The filtering of these parameters is described below: • For each parameter, the accumulated value is updated in the following way: ○ Subtract the oldest parameter value corresponding to the parameter value found in the current index. Subtraction can be a simple subtraction or a weighted subtraction between the oldest parameter value and any of the following parameters; Add the latest parameter value that was just received. Addition can be a simple addition or a weighted addition, which is a combination of the latest parameter value and any of the previous parameters.

[0235] • Update the buffer at the current index using the latest parameters just received.

[0236] • Calculate the filter value for each parameter. The filter value can be: Simply divide the corresponding accumulated value by the size of the corresponding buffer; ○ Divide the corresponding cumulative value by a number that corresponds to the weighted sum of the combinations of the latest parameter value considered when calculating the cumulative value and any of the previous parameter values.

[0237] In this step, processing module 40 checks whether the buffer has been initialized or not previously initialized during the current switch. If it has been initialized, and if the current value is valid, processing module 40 updates the current buffer value. If it has not been initialized, processing module 40 initializes the buffer and the accumulated value as described in step 973.

[0238] frame_idx_max_histo An example of an implementation scheme with buffer size for all parameters. n It applies when the same index is used; the current index is... iThe cumulative value is the simple sum of all parameters, and the filter value is simply divided by the buffer size. n In this case, all filter values ​​are calculated as follows: In step 972A, processing module 40 initializes the variables representing sectors (i.e., colors). S .

[0239] In step 972B, processing module 40 determines the variables. S Is it less than the number of sectors? scale_max .

[0240] If the number is not less than 1, then processing module 40 stops in step 972C. Figure 13 The processing.

[0241] Otherwise, processing module 40 determines the parameters. Figure 13 Is it different? NumOfSectors Figure 13 .if frame_idx_max_histo[S] = frame_idx_max_ Then, step 972D is followed by step 972P. During step 972P, processing module 40 processes all buffer values. histo_invalid frame_idx_max_histo[S] ( x From zero to buffer size n Reinitialize to frame_idx_max_histo_invalid Following this reinitialization, during step 972P, processing module 40 will change the variables. S Increment by one unit. Furthermore, during step 972P, step 972B follows step 972P.

[0242] if frame_idx_max_ ≠ histo_buf[S][x] Then, step 972D is followed by step 972E. During step 972E, processing module 40 determines the buffer. frame_idx_max_histo_invalid Is the value different from frame_idx_max_histo[S] .

[0243] if frame_idx_max_histo_invalid = frame_idx_max_histo_buf[S][i] Step 972E is followed by step 972F, in which all buffer values ​​are... frame_idx_max_histo_invalid ( x From zero to buffer size n Initialize to frame_idx_max_histo_buf[S][i] Furthermore, during step 972F, the processing module 40 will assign a value... frame_idx_max_histo_invalid Assign primary chromaticity value to filter frame_idx_max_histo_buf[S][x] frame_idx_max_histo[S] frame_idx_max_histo[S] filtered_frame_idx_max_histo[S] .

[0244] if frame_idx_max_histo_buf[S][i] ≠ frame_idx_max_histo_invalidStep 972E is followed by step 972G. During step 972G, processing module 40 updates the accumulated value as shown below. cum_frame_idx_ max_histo[S] : cum_frame_idx_max_histo[S] = cum_frame_idx_max_histo[S] - frame_idx_max_ histo_buf[S][i] + frame_idx_max_histo[S] .

[0245] In step 972H, processing module 40 updates the buffer value as shown below. frame_idx_max_histo_buf[S] [i] : frame_idx_max_histo_buf[S][i] = frame_idx_max_histo[S] .

[0246] In step 972I, the processing module 40 obtains the filtered primary chromaticity value. filtered_frame_idx_max_histo [S] : filtered_frame_idx_max_histo[S] = cum_frame_idx_max_histo[S] / n .

[0247] Step 972I is followed by step 972J.

[0248] During step 972J, processing module 40 determines the maximum allowable scaling value. scale_max[S] Is it different from the value? scale_max_invalid .

[0249] if scale_max[S] = scale_max_invalid Then the processing module 40 will process all buffer values scale_ max_buf[S][x] Reinitialize to scale_max_invalid Following this reinitialization, during step 972P, processing module 40 will change the variables. S Increment by one unit.

[0250] Otherwise, processing module 40 determines the buffer value during step 972K. scale_max_buf[S][i] Is it different from the value? scale_max_invalid .if ​ = ​ In step 972L, the processing module will set the buffer value... ​ Set as ​ And filter the maximum allowed scaling value ​ Set as value ​ Step 972L is followed by step 972P.

[0251] Otherwise, in step 972M, the processing module updates the accumulated value as follows. ​ : ​= ​ – ​ + ​ .

[0252] In step 972N, processing module 40 updates the buffer value as shown below. ​ : ​ = ​ .

[0253] In step 972O, processing module 40 obtains the maximum allowable scaling value for filtering as shown below: ​ = ​ / n .

[0254] Step 972O is followed by step 972P.

[0255] Then, for each sector S In the SGF function definition, which is transmitted to the post-processing module in the form of metadata, the filtered value replaces the non-filtered value: ​ = ​ and ​ = ​ ​ .

[0256] It should be noted that, in the case of SL-HDR1, all processing aimed at determining the SGF function is based on the representation of the output of the SL-HDR1 preprocessing module. , and The intermediate signal is the SDR signal. In other words, all calculations are performed in the SDR domain.

[0257] ​ An implementation scheme for controlling color correction suitable for an SL-HDR2 system is illustrated schematically.

[0258] Control and ​ The relevant color correction methods resolved the issues of the SL-HDR1 system. In the SL-HDR2 preprocessor, no SDR signal is generated. [The last sentence appears to be incomplete and unrelated to the preceding text. It likely refers to a different system or approach.] ​ In step 140 of the implementation scheme of the SL-HDR2 system, the chromaticity analysis of the current image is performed in the SL-HDR2 preprocessor by analyzing the chromaticity data from the relevant data. ​ The reconstruction process described in steps 801 to 807 simulates the SL-HDR2 post-processor until the variables are obtained. , and , This is performed (i.e., by processing module 40). The reconstruction process is completed by considering that the connected display is an SDR display. Therefore, the reconstruction process generates what is actually an SDR signal. , and Signal.

[0259] The SL-HDR2 post-processing module (more accurately, the reconstruction module) captures HDR signals and generates SDR, MDR, or HDR signals. About ​ The described method allows for the determination of SGF point coordinates in the SDR domain. ​ and ​ However, in a true SL-HDR2 post-processor, these points are applied to the input HDR signal. Therefore, in the case of SL-HDR2, all calculated SGF points are mapped into the HDR domain, which implies an estimate of the SDR to HDR transform.

[0260] In step 141, the processing module calculates the SDR to HDR conversion. This is done using two values. ​ and ​ And three vectors: ​ , ​ ​ and ​ This is done in two steps. In the first step, for each pixel of the current image of the HDR content, ​ and ​ The calculations are as follows: ,in , and In the SDR domain, and ​ The gamma factor is "2.4".

[0261] Lhisto_cur_sdr = CLAMP((Y post2 / 16 +.5), 0, NumBins - 1) Lhisto_cur_hdr = CLAMP((Y post1 / 16 +.5), 0, NumBins - 1) in CLAMP(x,y,z) Pick min(max(x,y),z ),and NumBins It is the number of bins in the histogram (e.g., NumBins = 64).

[0262] Lhisto_match_sdr_hdr_min[Lhisto_cur] = min(Lhisto_match_sdr_hdr_min [Lhisto_cur_sdr], ) ; Lhisto_match_sdr_hdr_max[Lhisto_cur] = max(Lhisto_match_sdr_hdr_max [Lhisto_cur_sdr], ) ; In the second step, after analyzing all pixels, the estimated SDR to HDR transform is calculated, such as... Figure 15 As shown.

[0263] In step 1410, the processing module 40 will change the variables Last_correct_value Initialize to zero.

[0264] In step 1411, the processing module 40 will process the variables lum Initialize to zero.

[0265] In step 1412, processing module 40 determines the variables. lum Is it less than NumBins .

[0266] If lum = NumBins, the processing module stops. Figure 15 The process. Otherwise, in step 1404, processing module 40 calculates the following value. Lhisto_match_sdr_hdr[lum] : Lhisto_match_sdr_hdr[lum] =Lhisto_match_sdr_hdr_min[lum]+Lhisto_match_sdr_hdr_max[lum]) / 2 .

[0267] In step 1415, the processing module 40 determines the value. Lhisto_match_sdr_hdr[lum] Is it equal to NumBins .

[0268] If equal, then processing module 40 calculates the value as shown in step 1417. Lhisto_match_sdr_hdr [lum] : Lhisto_match_sdr_hdr[lum] = last_correct_value .

[0269] Otherwise, in step 1416, the processing module calculates the value as shown below. last_correct_value : last_correct_value = Lhisto_match_sdr_hdr[lum] .

[0270] Steps 1416 and 1417 are followed by step 1418, during which the value... lum Increment by one unit.

[0271] In step 142, processing module 40 applies steps 90 to 96 to determine the vector. frame_idx_max_histo and scale_max .

[0272] In step 143, the processing module 40 will process the vector frame_idx_max_histo The parameters are mapped to, for example Figure 16 The HDR domain is shown.

[0273] In step 1430, the processing module 40 will change the variables S Initialize to zero.

[0274] In step 1431, processing module 40 determines the variables. S Is it less than NumOfSectors .

[0275] if S = NumOfSectors Then the processing module 40 stops in step 1432. Figure 16 The process.

[0276] Otherwise, the processing module determines the parameters. frame_idx_max_histo[S] Is it less than NumBins .

[0277] if frame_idx_max_histo[S] = NumBins Then, in step 1436, the processing module 40 will process the variable... S Increment by one unit. Step 1436 is followed by step 1431.

[0278] Otherwise, in step 1434, processing module 40 calculates the variables as shown below. Lhisto_sdr : Lhisto_sdr = frame_idx_max_histo[S] .

[0279] In step 1435, the processing module 40 calculates the parameters as follows: frame_idx_max_histo[S] : frame_idx_max_histo[S] = Lhisto_match_sdr_hdr[Lhisto_sdr] .

[0280] Step 1435 is followed by step 1436.

[0281] return Figure 14 After step 143, processing module 40 executes step 144, during which processing module 40 calculates the SGF points representing the SGF function. For each sector S: sgf_x[S] = frame_idx_max_histo[S] and sgf_y_tmp[S] = scale_max[S] .

[0282] Then process module 40 pairs sgf_x and sgf_y_tmp Values ​​are reordered to ensure that when i When increasing, sgf_x[i] The value monotonically increases. Reordered sgf_x and sgf_y_tmp The value allows the definition of an SGF function, which is transmitted to post-processing module 14 in the form of metadata.

[0283] In step 142, all calculations are completed to generate the SGF function, which, in the case of SL-HDR1, enhances saturation in steps 90 through 96. Finally, because the SGF function works differently between SL-HDR1 and SL-HDR2, all calculations performed in step 96 in the case of SL-HDR1 are performed differently. sgf_y_tmp(Y) The values ​​all need to be suitable for SL-HDR2.

[0284] In SL-HDR1, Figure 6 In step 605, SGF is applied on the preprocessor side, where

[0285] in , where the saturation gain function From the metadata of the saturation gain function sgf_x and sgf_y The defined piecewise linear pivot point is derived as detailed in section 7.3 of document ETSI TS 103 433-1 v1.3.1.

[0286] Given a brightness Y, when sgf(Y) increases by an increment value... incr hour:

[0287] Therefore, sgf(Y) Increase incr At that time, U pre1 (V respectively) pre1 The following modifications have been made:

[0288]

[0289] incr Any positive value will increase U pre1 (V respectively) pre1 This will increase the saturation of the pixels.

[0290] In SL-HDR2, Figure 8 In step 804, SGF is applied on the post-processor side, having .Will sgf (Y) Increase incr At that time, U post2 (V respectively) post2 The following modifications have been made:

[0291]

[0292] incr Any positive value will decrease U post2 (V respectively) post2 Therefore, it will reduce the saturation of the corresponding pixel.

[0293] Therefore, in SL-HDR2, incremental incr_slhdr2 It is based on the corresponding SL-HDR1 increment incr_slhdr1 The calculation results in:

[0294] Finally, this leads to the calculation of the final SL-HDR2. sgf_y value:

Claims

1. A method comprising: Receive high dynamic range (HDR) image data associated with the current frame; The HDR image data is converted into standard dynamic range SDR image data to obtain multiple reference pixel values; For multiple color categories, determine multiple chroma saturation gain values ​​to be applied to the multiple reference pixel values; Based on the multiple chroma saturation gain values, a saturation gain function is generated that maps brightness to chroma gain for the multiple color categories; as well as The saturation gain function is encoded as metadata associated with the SDR image data.

2. The method of claim 1, further comprising: Obtain the chromaticity plane representing the color gamut; as well as The chromaticity plane is divided into multiple chromaticity sectors, each of which corresponds to a specific color category.

3. The method of claim 1, further comprising, for pixels in the pixel set of the current frame: The luminance component is derived from the HDR image data; Apply hue mapping to the luminance component to obtain hue-mapped luminance components; The chromaticity components are derived from the HDR image data; Joint normalization and color correction are applied to the chromaticity components to obtain corrected normalized chromaticity components; as well as The pixel is classified into one of the plurality of color categories using the hue-mapped luminance component and the corrected normalized chrominance component.

4. The method of claim 1, further comprising, for each of the plurality of color categories, determining a maximum permissible chromaticity value, wherein the maximum permissible chromaticity value is a chromaticity value that prevents cropping in the representation of the SDR image data.

5. The method of claim 4, further comprising, as part of the metadata, encoding a principal luminance value for each of the plurality of color categories and the maximum permissible chromaticity value determined for the respective color category.

6. The method of claim 1, wherein the SDR image data and the metadata are included in the video data, such that an SDR display renders the SDR image data and an HDR display reconstructs HDR content by applying the metadata to the SDR image data.

7. The method of claim 1, wherein the HDR image data is retained as a base layer, and the metadata is included in video data along with the HDR image data, the video data being configured to convert the HDR base layer into SDR for SDR display by a decoder.

8. The method of claim 1, wherein the current frame is included in a video sequence, and a temporal filter is applied to the information representing the chroma saturation gain value based on information representing a chroma saturation gain value calculated in at least one frame preceding the current frame in the video sequence.

9. The method of claim 5, wherein a temporal filter is applied to the information representing the chroma saturation gain value, and the temporal filter is reinitialized at the beginning of the video sequence or when a scene change is detected in the video sequence.

10. The method of claim 1, further comprising, for each of the plurality of color categories: Obtain a histogram of the luminance values ​​of the pixels associated with the color category in the current frame; and Choose one of the following as the primary brightness value: The brightness value corresponding to the maximum number of pixels in the brightness value histogram; The luminance value corresponding to the maximum chromaticity energy for a histogram bar, wherein the chromaticity energy for the bar is the product of the number of pixels in the bar and the maximum chromaticity value found at the bar; and The luminance value corresponding to the maximum average chromaticity energy for a histogram bar, wherein the average chromaticity energy for the bar is the product of the number of pixels in the bar and the maximum chromaticity value found at the bar.

11. An apparatus for video encoding, the apparatus comprising: The processor is configured as follows: Receive high dynamic range (HDR) image data associated with the current frame; The HDR image data is converted into standard dynamic range SDR image data to obtain multiple reference pixel values; For multiple color categories, determine multiple chroma saturation gain values ​​to be applied to the multiple reference pixel values; Based on the multiple chroma saturation gain values, a saturation gain function is generated that maps brightness to chroma gain for the multiple color categories; as well as The saturation gain function is encoded as metadata associated with the SDR image data.

12. The device of claim 11, wherein the processor is further configured to: Obtain the chromaticity plane representing the color gamut; and The chromaticity plane is divided into multiple chromaticity sectors, each of which corresponds to a specific color category.

13. The device of claim 11, wherein the processor is further configured to, for pixels in the pixel set of the current frame: The luminance component is derived from the HDR image data; Apply hue mapping to the luminance component to obtain hue-mapped luminance components; The chromaticity components are derived from the HDR image data; Joint normalization and color correction are applied to the chromaticity components to obtain corrected normalized chromaticity components; as well as The pixel is classified into one of the color categories using the hue-mapped luminance component and the corrected normalized chrominance component.

14. The device of claim 11, wherein the processor is further configured to determine, for each of the plurality of color categories, a maximum permissible chromaticity value, wherein the maximum permissible chromaticity value is a chromaticity value that prevents cropping in the representation of the SDR image data.

15. The device of claim 14, wherein the processor is further configured to, as part of the metadata, encode the principal luminance value of each of the plurality of color categories and the maximum permissible chromaticity value determined for the respective color category.

16. The device of claim 11, wherein the SDR image data and the metadata are included in video data, such that an SDR display renders the SDR image data and an HDR display reconstructs HDR content by applying the metadata to the SDR image data.

17. The device of claim 11, wherein the HDR image data is retained as a base layer, and the metadata is included in video data along with the HDR image data, the video data being configured to convert the HDR base layer to SDR for SDR display by a decoder.

18. The device of claim 11, wherein the current frame is included in a video sequence, and temporal filtering is applied to the information representing the chroma saturation gain value based on information representing a chroma saturation gain value calculated in at least one frame preceding the current frame in the video sequence.

19. The device of claim 15, wherein a temporal filter is applied to the information representing the chroma saturation gain value, and the temporal filter is reinitialized at the beginning of the video sequence or when a scene change is detected in the video sequence.

20. The device of claim 11, wherein the processor is further configured to, for each of the plurality of color categories: Obtain a histogram of the luminance values ​​of the pixels associated with the color category in the current frame; and Choose one of the following as the primary brightness value: The brightness value corresponding to the maximum number of pixels in the brightness value histogram; The luminance value corresponding to the maximum chromaticity energy of a histogram bar, wherein the chromaticity energy of the bar is the product of the number of pixels in the bar and the maximum chromaticity value found at the bar; and The luminance value corresponding to the maximum average chromaticity energy for a histogram bar, wherein the average chromaticity energy for the bar is the product of the number of pixels in the bar and the maximum chromaticity value found at the bar.

Images