Resolving input color space inference ambiguities in sl-hdr1 standard

Abstract

Methods and apparatus are provided for color space inference in high dynamic range video, such as in the SL-HDR1 video standard. In one embodiment, an encoder sends information including standard dynamic range video and metadata contained in a supplemental enhancement information message to indicate the color space included in the video. In another embodiment, a decoder parses the bitstream and infers values ​​from the data in the supplemental enhancement information message to inform the decoder of the color space included in the video. The high dynamic range signal is reconstructed based on the standard dynamic range data and the values ​​inferred from the metadata.

Description

Cross-references to related applications

[0001] This application claims the benefit of European serial number 23306977.2, filed on 15 November 2023, which is incorporated herein by reference in its entirety. Technical Field

[0002] At least one of these embodiments generally relates to a method or apparatus for video encoding or decoding, compression or decompression. Background Technology

[0003] To achieve high compression efficiency, image and video coding schemes typically employ prediction (including motion vector prediction) and transform to utilize spatial and temporal redundancy in the video content. Generally, intra-frame or inter-frame prediction is used to leverage intra-frame or inter-frame correlations, followed by transforming, quantizing, and entropy coding of the differences between the original and predicted images (typically represented as prediction error or prediction residual). To reconstruct the video, the compressed data is decoded through the inverse processes corresponding to entropy coding, quantization, transform, and prediction. Summary of the Invention

[0004] At least one of these embodiments generally relates to a method or apparatus for video encoding or decoding, and more specifically, to a method or apparatus for color space inference to convey standard dynamic range (SDR) video with metadata so as to be able to reconstruct high dynamic range (HDR) video.

[0005] According to a first aspect, a method is provided. The method includes the steps of: conveying video information, the video information including standard dynamic range data and metadata, wherein, when the metadata does not include information for a transmitted original image, the metadata conveys a first value indicating the color space primary color information of a high dynamic range image by using a second value in the metadata indicating color space primary color information of a high dynamic range display and a third value in the metadata indicating color space primary color information of a standard dynamic range image; and when the metadata includes information about the transmitted original image, and when a fourth value in the metadata indicating color space primary color information of a master display of the original image has a value of one, setting the first value to zero, or when the metadata includes information about the transmitted original image, and when the fourth value has a value of nine, setting the first value to one; and encoding the standard dynamic range data and the metadata in a bitstream.

[0006] According to the second aspect, another method is provided. This method includes the following steps: parsing a bitstream of video information, the video information including standard dynamic range data and metadata, wherein, when the metadata does not include information for the original image being transmitted, the metadata conveys a first value indicating the color space primary color information of the high dynamic range image by using a second value in the metadata indicating color space primary color information of a high dynamic range display and a third value in the metadata indicating color space primary color information of a standard dynamic range image; and when the metadata includes information about the original image being transmitted, and when a fourth value in the metadata indicating color space primary color information of a master display of the original image is one, inferring a first value of zero, or when the metadata includes information about the original image being transmitted, and when a fourth value is nine, inferring a first value of one; and decoding the standard dynamic range data and the metadata from the bitstream to reconstruct the high dynamic range image using the inferred value.

[0007] According to another aspect, an apparatus is provided. The apparatus includes a processor and a memory. The processor can be configured to operate on digital video data according to the method described above.

[0008] According to another aspect, an apparatus is provided. The apparatus includes a processor and a memory. The processor can be configured to encode blocks of video or decode video data by performing any of the methods described above.

[0009] According to another general aspect of at least one embodiment, an apparatus is provided that includes means according to any embodiment of the decoding embodiment; and at least one of: (i) an antenna configured to receive a signal including a video block, (ii) a band limiter configured to limit the received signal to a band including the video block, or (iii) a display configured to display an output representing the video block.

[0010] According to another general aspect of at least one embodiment, a non-transitory computer-readable medium is provided that contains data content generated according to any of the described encoding embodiments or variations.

[0011] According to another general aspect of at least one embodiment, a signal is provided that includes video data generated according to any of the described encoding embodiments or variations.

[0012] According to another general aspect of at least one embodiment, video data or bitstreams are formatted to include data content generated according to any of the described encoding embodiments or variations.

[0013] According to another general aspect of at least one embodiment, a computer program product including instructions is provided that, when executed by a computer, cause the computer to perform any of the described decoding embodiments or variations.

[0014] These and other aspects, features, and advantages of the general aspects will become apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Attached Figure Description

[0015] Figure 1 An overview of the SL-HDR1 system is shown.

[0016] Figure 2 An embodiment of the first method according to the described aspects is shown.

[0017] Figure 3 An embodiment of the second method according to the described aspects is shown.

[0018] Figure 4 An embodiment of the apparatus according to the described aspects is shown.

[0019] Figure 5 This demonstrates a standard, general-purpose video compression scheme.

[0020] Figure 6 This demonstrates a standard, general-purpose video decompression scheme.

[0021] Figure 7 A processor-based system for encoding / decoding is shown, based on aspects described in general.

[0022] Figure 8 A diagram illustrating a use case managed according to one embodiment of the aspects described in general is shown.

[0023] Figure 9 A diagram illustrating a use case for management based on another embodiment of the aspects described in general is shown. Detailed Implementation

[0024] The embodiments described herein belong to the field of video compression and generally relate to video compression as well as video encoding and decoding, and more specifically to a method or apparatus for color space inference in the SL-HDR1 standard.

[0025] To achieve high compression efficiency, image and video coding schemes typically employ block-based prediction (including motion vector prediction) and transform to utilize spatial and temporal redundancy in the video content. Generally, intra-frame or inter-frame prediction is used to leverage intra-frame or inter-frame correlations, followed by transforming, quantizing, and entropy coding of the differences between the original and predicted blocks (typically represented as prediction error or prediction residuals). To reconstruct the video, the compressed data is decoded through the inverse process corresponding to entropy coding, quantization, transform, and prediction.

[0026] In the HEVC (High Efficiency Video Coding) video compression standard, motion-compensated time prediction is used to take advantage of the redundancy between consecutive frames in a video.

[0027] To this end, motion vectors are associated with each prediction unit (PU). Each CTU (coding tree unit) is represented by a coding tree in the compressed domain. This is a quadtree partition of the CTU, where each leaf is called a coding unit (CU).

[0028] Then, each CU is given some intra-frame or inter-frame prediction parameters (prediction information). For this purpose, it is spatially divided into one or more prediction units (PUs), and each PU is assigned some prediction information. Intra-frame or inter-frame coding modes are assigned at the CU level.

[0029] The SL-HDR1 standard describes how to reconstruct HDR video using metadata carried by SDR video.

[0030] During this HDR reconstruction process, inferences about the input color space may result in incorrect colors.

[0031] This input color space inference can be solved by relying on the semantics of the metadata carried by the SDR video.

[0032] ETSI TS 103433-1 allows the distribution of standard dynamic range (SDR) video with associated metadata to reconstruct HDR video.

[0033] Traditional devices that receive video simply ignore the metadata and display the SDR video.

[0034] SL-HDR1 compatible devices use the metadata carried by the SDR video to reconstruct and display HDR video.

[0035] exist Figure 1 An overview of the SL-HDR1 system is shown in the image.

[0036] HDR inputs are primarily characterized by the hdrPicColourSpace and hdrDisplayColourSpace variables, and SDR outputs are characterized by sdrPicColourSpace, all of which are defined in ETSI TS 103433-1.

[0037] The `hdrPicColourSpace` variable characterizes the image color space (BT.709 or BT.2020) by indicating the white point and primary color chromaticity coordinates of the HDR image color space according to the CIE 1931 definition of x and y as specified in ISO 11664-1. This corresponds to the so-called video container (for HDR video).

[0038]

[0039] The hdrDisplayColourSpace variable characterizes the color space (BT.709, BT.2020, or DCI-P3) of the master display used for mastering HDR images by indicating the white point and chromaticity coordinates of the primary colors of the HDR master display according to the CIE1931 definition of x and y as specified in ISO 11664-1.

[0040]

[0041] The `sdrPicColourSpace` variable characterizes the picture color space of an SDR video by indicating the white point and primary color chromaticity coordinates of the SDR picture color space according to the CIE 1931 definition of x and y as specified in ISO 11664-1. This corresponds to the so-called video container (for SDR video).

[0042]

[0043] The SL-HDR1 preprocessor generates SL-HDR metadata. This metadata is carried using Supplemental Enhancement Information (SEI) messages as defined in recommendations such as ITU-T H.266, ITU-T H.264, or ITU-T H.265.

[0044] The SL-HDR1 postprocessor uses SL-HDR metadata, as described in ETSI TS 103433-1, to reconstruct HDR video.

[0045] Instead of explicitly providing hdrPicColourSpace, hdrDisplayColourSpace, or sdrPicColourSpace variables, these SL-HDR metadata provide other metadata variables that allow the SL-HDR1 post-processor to retrieve these variables as follows.

[0046] hdrDisplayColourSpace is directly mapped to Metadata, as detailed in Section A.2.3.3.3 of ETSI TS 103 433-1.

[0047] sdrPicColourSpace maps directly to the target_picture_primaries metadata, as detailed in section A.2.3.4.2 of ETSI TS103 433-1.

[0048] The hdrPicColourSpace is inferred from hdrDisplayColourSpace and sdrPicColourSpace, as detailed in Table A.3 of Section A.2.3.3.2 of ETSI TS 103 433-1.

[0049]

[0050] Inference using hdrPicColourSpace Figure 8 The managed use cases are shown.

[0051] However, the case with the characteristics hdrDisplayColourSpace = 0, SdrPicColourSpace = 0, and hdrPicColourSpace = 1 is not described and is easily misunderstood. This case describes the source HDR input in the BT.2020 color space and is characterized by the color space of the master display used for mastering HDR images in BT.709 and the SDR-distributed video in the BT.709 color space. In reality, on the SL-HDR1 post-processor side, the color space of the reconstructed HDR video can be either the BT.709 color space (hdrPicColourSpace = 0) or the BT.2020 color space (hdrPicColourSpace = 1), but it should be the BT.2020 color space.

[0052] This corresponds to Figure 9 The example shown.

[0053] If the BT.709 color space is selected for the output reconstructed HDR video, incorrect colors will be generated.

[0054] Question: How can I prevent the use of an incorrect color space in the output reconstructed HDR video? To correctly retrieve the hdrPicColourSpace variable in the SL-HDR1 postprocessor for all use cases during reconstruction, the original_picture_info_present_flag and original_picture_primaries from the SL-HDR SEI message syntax can be used.

[0055] original_picture_primaries has the same semantics as the colour_primaries syntax element specified in clause E.3.1 of the HEVC specification.

[0056] `original_picture_primaries` indicates the primary colors used for the master display of the original image, and it only exists when `original_picture_info_present_flag` is set to 1.

[0057] When original_picture_info_present_flag equals 0, the value of hdrPicColourSpace should be inferred from the values ​​of hdrDisplayColourSpace and sdrPicColourSpace.

[0058] Otherwise, when original_picture_info_present_flag equals 1, the following will be applied.

[0059] When original_picture_primaries equals 1, hdrPicColourSpace should equal 0.

[0060] When original_picture_primaries equals 9, hdrPicColourSpace should equal 1.

[0061] These signals are sent by an encoder or processor that implements the encoding process, and received by a decoder or processor that implements the decoding process. The names of the signals may differ, but the example used above is only one embodiment of the proposed solution.

[0062] exist Figure 2An embodiment of method 200 according to the general aspects described herein is illustrated. The method begins at start box 201, and control proceeds to box 210 for conveying video information including standard dynamic range data and metadata, wherein, when the metadata does not include information for the original image being transmitted, the metadata conveys a first value indicating the color space primary color information of the high dynamic range image by using a second value in the metadata indicating color space primary color information of a high dynamic range display and a third value in the metadata indicating color space primary color information of a standard dynamic range image. Control proceeds from box 210 to box 220 for setting a value such that when the metadata includes information about the original image being transmitted, and when a fourth value in the metadata indicating color space primary color information of a master display of the original image is one, the first value is set to zero, or when the metadata includes information about the original image being transmitted, and when the fourth value is nine, the first value is set to one. Control proceeds from box 220 to box 230 for encoding the standard dynamic range data and the metadata in a bitstream.

[0063] exist Figure 3 An embodiment of method 300 according to the general aspects described herein is illustrated. The method begins at start box 301 and control proceeds to box 310 for parsing a bitstream of video information, which includes standard dynamic range data and metadata, wherein, when the metadata does not include information for the original image being transmitted, the metadata conveys a first value indicating the color space primary color information of the high dynamic range image by using a second value in the metadata indicating color space primary color information of a high dynamic range display and a third value in the metadata indicating color space primary color information of a standard dynamic range image. Control proceeds from box 310 to box 320 for inferring a value such that when the metadata includes information about the original image being transmitted, and when the value of a fourth value in the metadata indicating color space primary color information of a master display of the original image is one, the first value is inferred to be zero; or when the metadata includes information about the original image being transmitted, and when the value of the fourth value is nine, the first value is inferred to be one. Control proceeds from box 320 to box 330 for decoding the standard dynamic range data and the metadata from the bitstream to reconstruct the high dynamic range image using the inferred values.

[0064] Figure 4 An embodiment of an apparatus 400 for encoding, decoding, compressing, decompressing, or filtering video data using the methods described above is shown. The apparatus includes a processor 410 and can be interconnected with a memory 420 via at least one port. Both the processor 410 and the memory 420 may also have one or more additional interconnects for external connection.

[0065] The processor 410 is also configured to insert or receive information in the bit stream and to compress, encode, or decode using any of the described aspects.

[0066] The embodiments described herein encompass various aspects, including tools, features, embodiments, models, methods, etc. Many of these aspects are described in a specific manner and, at least for the purpose of illustrating the individual features, are generally described in a way that may sound limiting. However, this is for the purpose of clarity of description and does not limit the application or scope of these aspects. In fact, all the different aspects can be combined and interchanged to provide other aspects. Furthermore, these aspects can also be combined and interchanged with those described in previous documents.

[0067] The aspects described and envisioned in this application can be realized in many different forms. Figure 5 , Figure 6 and Figure 7 Some embodiments have been provided, but other embodiments are contemplated, and... Figure 5 , Figure 6 and Figure 7 The discussion does not limit the breadth of implementations. At least one aspect generally relates to video encoding and decoding, and at least one other aspect generally relates to transmitting the generated or encoded bitstream. These and other aspects can be implemented as methods, apparatus, computer-readable storage media having instructions thereon stored thereon for encoding or decoding video data according to any of the methods, and / or computer-readable storage media having bitstreams generated according to any of the methods stored thereon.

[0068] In this application, the terms "reconstruction" and "decoding" are used interchangeably, as are the terms "pixel" and "sample," and the terms "image," "picture," and "frame." Generally, but not necessarily, the term "reconstruction" is used on the encoder side, while "decoding" is used on the decoder side.

[0069] This document describes various methods, and each of these methods includes one or more steps or actions for implementing the method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions may be modified or combined.

[0070] The various methods and other aspects described in this application can be used to modify, for example... Figure 5 and Figure 6The modules of the video encoder 100 and decoder 200 shown include, for example, intra-frame prediction, entropy coding, and / or decoding modules (160, 260, 145, 230). Furthermore, this aspect is not limited to VVC or HEVC and can be applied to, for example, other standards and recommendations (whether pre-existing or future developments), and any extensions to such standards and recommendations (including VVC and HEVC). Unless otherwise stated or technically excluded, these aspects described in this application may be used alone or in combination.

[0071] Various numerical values ​​are used in this application. Specific values ​​are for illustrative purposes only, and the aspects described are not limited to these specific values.

[0072] Figure 5 Encoder 100 is shown. Variations of this encoder 100 are envisioned, but for clarity, encoder 100 is described below without describing all anticipated variations.

[0073] Before being encoded, the video sequence may undergo pre-coding (101), for example, applying color transformations to the input color image (e.g., a conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing remapping of the input image components to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization with one of the color components). Metadata may be associated with the pre-processing and appended to the bitstream.

[0074] In encoder 100, the image is encoded by encoder elements as described below. The image to be encoded is partitioned (102) and processed in units such as CUs. Each unit is encoded using, for example, an intra-frame or inter-frame mode. When a unit is encoded in intra-frame mode, intra-frame prediction (160) is performed. In inter-frame mode, motion estimation (175) and compensation (170) are performed. The encoder determines (105) which mode, intra-frame or inter-frame, to use to encode the unit and indicates the intra-frame / inter-frame decision by, for example, a prediction mode flag. For example, the prediction residual is calculated by subtracting (110) the prediction block from the original image block.

[0075] The predicted residual is then transformed (125) and quantized (130). The quantized transform coefficients, motion vectors, and other syntax elements are entropy encoded (145) to output a bitstream. The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can bypass both the transform and quantization, i.e., directly encode the residual without applying either the transform or quantization process.

[0076] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (155) to reconstruct the image block. An in-loop filter (165) is applied to the reconstructed image to perform, for example, deblocking / SAO (Sample Adaptive Shift) filtering, thereby reducing coding artifacts. The filtered image is stored in a reference image buffer (180).

[0077] Figure 6 A block diagram of a video decoder 200 is shown. In decoder 200, bitstreams are decoded by decoder elements, as described below. Video decoder 200 typically performs functions similar to... Figure 5 The encoding process shown is the reverse of the decoding process. Encoder 100 typically also performs video decoding as part of the encoded video data.

[0078] Specifically, the decoder's input includes a video bitstream, which can be generated by the video encoder 100. First, entropy decoding (230) is performed on the bitstream to obtain transform coefficients, motion vectors, and other encoded information. Image partitioning information indicates how the image is partitioned. Therefore, the decoder can partition (235) the image based on the decoded image partitioning information. The transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residuals. The decoded prediction residuals and prediction blocks are combined (255) to reconstruct the image blocks. Prediction blocks (270) can be obtained from intra-frame prediction (260) or motion-compensated prediction (i.e., inter-frame prediction) (275). An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference image buffer (280).

[0079] The decoded image can also undergo post-decoding processing (285), such as inverse color transformation (e.g., conversion from YcbCr 4:2:0 to RGB 4:4:4) or inverse remapping, which is the inverse of the remapping process performed in the pre-encoding process (101). Post-decoding processing can utilize metadata derived in the pre-encoding process and signaled in the bitstream.

[0080] Figure 7A block diagram illustrating an example system implementing various aspects and embodiments is shown. System 1000 may be embodied as a device including the various components described below and configured to perform one or more aspects described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Elements of system 1000 may be embodied individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing elements and encoder / decoder elements of system 1000 are distributed across multiple ICs and / or discrete components. In various embodiments, system 1000 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various embodiments, system 1000 is configured to implement one or more aspects described in this document.

[0081] System 1000 includes at least one processor 1010 configured to execute instructions loaded thereon to implement various aspects described herein, such as those described. Processor 1010 may include embedded memory, input / output interfaces, and various other circuitry known in the art. System 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). System 1000 includes a storage device 1040 that may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives, and / or optical disk drives. As a non-limiting example, storage device 1040 may include internal storage devices, attached storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices.

[0082] System 1000 includes an encoder / decoder module 1030 configured to, for example, process data to provide encoded or decoded video, and the encoder / decoder module 1030 may include its own processor and memory. The encoder / decoder module 1030 represents a module that can be included in a device to perform encoding and / or decoding functions. It is well known that a device may include one or both encoding and decoding modules. Alternatively, the encoder / decoder module 1030 may be implemented as a separate element of system 1000, or it may be incorporated within processor 1010 as a combination of hardware and software known to those skilled in the art.

[0083] Program code to be loaded onto processor 1010 or encoder / decoder 1030 to execute the various aspects described herein may be stored in storage device 1040 and subsequently loaded onto memory 1020 for execution by processor 1010. According to various embodiments, one or more of processor 1010, memory 1020, storage device 1040, and encoder / decoder module 1030 may store one or more of various items during the execution of the processes described herein. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from equations, formulas, operations, and operational logic processing.

[0084] In some embodiments, the memory within the processor 1010 and / or encoder / decoder module 1030 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, external memory (e.g., the processing device may be the processor 1010 or the encoder / decoder module 1030) is used for one or more of these functions. External memory may be memory 1020 and / or storage device 1040, such as volatile memory and / or non-volatile flash memory. In several embodiments, external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one embodiment, a fast external dynamic volatile memory, such as RAM, is used as working memory for video encoding and decoding operations, such as for MPEG-2 (MPEG stands for Moving Picture Experts Group; MPEG-2 is also known as ISO / IEC 13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC stands for High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Various Video Coding, i.e., a new standard developed by JVET (Joint Video Experts Group)).

[0085] Inputs to the components of system 1000 can be provided through various input devices, as indicated in box 1130. Such input devices include, but are not limited to, (i) a radio frequency (RF) section that receives, for example, RF signals transmitted over the air by a broadcaster, (ii) a component (COMP) input terminal (or a set of COMP input terminals), (iii) a universal serial bus (USB) input terminal, and / or (iv) a high-definition multimedia interface (HDMI) input terminal. Figure 7 Other examples not shown include composite video.

[0086] In various embodiments, the input device of block 1130 has associated corresponding input processing elements known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal, or limiting the signal band to a band), (ii) down-converting the selected signal, (iii) further band-limiting to a narrower band to select, for example, a signal band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select the desired data packet stream. The RF section in various embodiments includes one or more elements performing these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include tuners performing various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or baseband. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and filtering again to the desired frequency band. Various embodiments rearrange the order of the above (and other) components, remove some of these components, and / or add other components that perform similar or different functions. Adding components may include inserting components between existing components, such as insert amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0087] Additionally, USB and / or HDMI terminals may include their respective interface processors for connecting system 1000 to other electronic devices via USB and / or HDMI connections. It should be understood that aspects of input processing (e.g., Reed-Solomon error correction) may be implemented, for example, within a separate input processing IC or within processor 1010 as needed. Similarly, aspects of USB or HDMI interface processing may be implemented, as needed, within a separate interface IC or within processor 1010. The demodulated, error-corrected, and demultiplexed stream is provided to various processing elements, including, for example, processor 1010 and an encoder / decoder 1030 operating in conjunction with memory and storage elements, to process the data stream as needed for presentation on an output device.

[0088] Various components of the system 1000 can be housed within an integrated housing. Within the integrated housing, the various components can be interconnected and transmit data between them using suitable connection means (e.g., internal buses known in the art, including inter-IC (I2C) buses, wiring, and printed circuit boards).

[0089] System 1000 includes a communication interface 1050 that enables communication with other devices via a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to send and receive data via the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 1060 may be implemented, for example, within a wired and / or wireless medium.

[0090] In various embodiments, data is streamed to or otherwise provided to system 1000 using a wireless network, such as a Wi-Fi network, for example, IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). In these embodiments, the Wi-Fi signal is received via a communication channel 1060 and a communication interface 1050 adapted for Wi-Fi communication. The communication channel 1060 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top communications. Other embodiments use a set-top box to provide streaming data to system 1000, delivering data via an HDMI connection to input block 1130. Still other embodiments use an RF connection to input block 1130 to provide streaming data to system 1000. As indicated above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0091] System 1000 can provide output signals to various output devices, including display 1100, speaker 1110, and other peripheral devices 1120. Display 1100 in various embodiments includes one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. Display 1100 can be used in a television, tablet computer, laptop computer, mobile phone, or another device. Display 1100 can also be integrated with other components (e.g., as in a smartphone) or standalone (e.g., an external monitor for a laptop computer). In various examples of embodiments, other peripheral devices 1120 include one or more of a standalone digital video disc (or digital versatile disc) (DVR, for both terms), an optical disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 1120 to provide functionality based on the output of system 1000. For example, an optical disc player performs the function of playing the output of system 1000.

[0092] In various embodiments, signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable device-to-device control with or without user intervention are used to transmit control signals between system 1000 and display 1100, speaker 1110, or other peripheral devices 1120. Output devices may be communicatively coupled to system 1000 via dedicated connections through corresponding interfaces 1070, 1080, and 1090. Alternatively, output devices may be connected to system 1000 via communication interface 1050 using communication channel 1060. Display 1100 and speaker 1110 may be integrated into a single unit along with other components of system 1000 in an electronic device, such as a television. In various embodiments, display interface 1070 includes a display driver, such as a timing controller (TCon) chip.

[0093] For example, if the RF input section 1130 is part of a separate set-top box, the display 1100 and speaker 1110 can alternatively be separate from one or more of the other components. In various embodiments where the display 1100 and speaker 1110 are external components, the output signal can be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0094] The embodiments can be executed by processor 1010 or by computer software implemented by hardware or a combination of hardware and software. As a non-limiting example, the embodiments can be implemented by one or more integrated circuits. As a non-limiting example, memory 1020 can be of any type suitable for the technical environment and can be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. As a non-limiting example, processor 1010 can be of any type suitable for the technical environment and can encompass one or more microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures.

[0095] Various implementations involve decoding. As used herein, "decoding" can encompass all or part of a process performed, for example, on a received encoded sequence to produce a final output suitable for display. In various embodiments, such a process includes one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such a process also includes, or alternatively includes, processes performed by the decoder of the various implementations described herein.

[0096] As another example, in one embodiment, "decoding" refers only to entropy decoding; in another embodiment, "decoding" refers only to differential decoding; and in yet another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Based on the specific context of the description, it will be clear whether the phrase "decoding process" is intended to specifically refer to a subset of operations or to refer to a broader decoding process, and it is believed that those skilled in the art will readily understand this.

[0097] Various implementations involve encoding. Similar to the discussion of "decoding" above, "encoding" as used herein can include, for example, all or part of the processing performed on an input video sequence to produce an encoded bitstream. In various embodiments, such processes include one or more processes typically performed by an encoder, such as partitioning, differential coding, transform, quantization, and entropy coding. In various embodiments, such processes also include, or alternatively include, processes performed by an encoder of the various implementations described herein.

[0098] As another example, in one embodiment, "encoding" refers only to entropy encoding; in another embodiment, "encoding" refers only to differential encoding; and in yet another embodiment, "encoding" refers to a combination of entropy encoding and differential encoding. Based on the specific context of the description, it will be clear whether the phrase "encoding process" is intended to specifically refer to a subset of operations or to refer to a broader encoding process, and it is believed that those skilled in the art will readily understand this.

[0099] It should be noted that the grammatical elements used in this article are descriptive terms. Therefore, the use of other grammatical element names is not excluded.

[0100] When a diagram is presented as a flowchart, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when a diagram is presented as a block diagram, it should be understood that it also provides a flowchart of the corresponding method / process.

[0101] Various implementations may involve parametric models and rate-distortion optimization. In particular, during the encoding process, a balance or trade-off between rate and distortion is typically considered, often due to constraints on computational complexity. This can be measured by rate-distortion optimization (RDO) or by least mean square (LMS), mean absolute error (MAE), or other such measures. Rate-distortion optimization is generally expressed as minimizing a rate-distortion function that is a weighted sum of rate and distortion. Different approaches exist to address the rate-distortion optimization problem. For example, these approaches may be based on extensive testing of all encoding options, including all considered modes or encoding parameter values, where a comprehensive evaluation of the encoding cost and the associated distortion of the reconstructed signal is performed after encoding and decoding. Faster methods can also be used to save encoding complexity, particularly by calculating approximate distortion based on prediction or prediction of the residual signal rather than the reconstructed signal. A hybrid of these two approaches can also be used, such as by using approximate distortion only for some possible encoding options and full distortion for others. Other methods evaluate only a subset of possible encoding options. More generally, many approaches employ any of a variety of techniques to perform optimization, but optimization is not necessarily a comprehensive assessment of coding costs and associated distortions.

[0102] The implementations and aspects described herein can be implemented, for example, in methods or processes, apparatuses, software programs, data streams, or signals. Even if discussed only in the context of a single implementation (e.g., discussed only as a method), the implementation of the features in question can also be implemented in other forms (e.g., apparatuses or programs). Apparatuses can be implemented, for example, with appropriate hardware, software, and firmware. Methods can be implemented, for example, in a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices, such as computers, cellular phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0103] The references to "an embodiment" or "an embodiment" or "an implementation" or "an implementation," and their variations, mean that a particular feature, structure, characteristic, etc., described in connection with that embodiment is included in at least one embodiment. Therefore, the phrases "in an embodiment" or "in an embodiment" or "in an implementation" or "in an implementation," and any other variations appearing throughout this application, do not necessarily refer to the same embodiment.

[0104] Additionally, this application may involve "determining" various types of information. Determining information may include one or more of the following: for example, estimated information, calculated information, predicted information, or information retrieved from memory.

[0105] Furthermore, this application may relate to "accessing" various types of information. Accessing information may include one or more of the following: for example, receiving information, retrieving information (e.g., retrieving from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0106] Additionally, this application may relate to "receiving" various types of information. Like "access," the intent to receive is a broad term. Receiving information may include one or more of the following: for example, accessing information or retrieving information (e.g., retrieving from memory). Furthermore, "receiving" is generally referred to in one or more ways during operation, such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0107] It should be understood that, for example, in the cases of “A / B,” “A and / or B,” and “at least one of A and B,” the use of any of the following “ / ,” “and / or,” and “at least one” is intended to cover selecting only the first listed option (A), or only the second listed option (B), or selecting both options (A and B). As yet another example, in the cases of “A, B, and / or C” and “at least one of A, B, and C,” this wording is intended to include selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or selecting all three options (A, B, and C). As will be apparent to those skilled in the art and related fields, this can be extended to a large number of listed items.

[0108] Furthermore, as used herein, the word “signal” specifically refers to instructing the corresponding decoder to do something. For example, in some embodiments, the encoder signals a particular one of multiple transforms, encoding modes, or flags. In this way, in embodiments, the same transform, parameter, or mode is used on both the encoder and decoder sides. Thus, for example, the encoder can send (explicit signaling) a specific parameter to the decoder so that the decoder can use the same specific parameter. Conversely, if the decoder already has the specific parameter as well as other parameters, signaling can be used without sending (implicit signaling) to simply allow the decoder to know and select the specific parameter. Bit savings are achieved in various embodiments by avoiding the transmission of any actual functionality. It should be understood that signaling can be implemented in various ways. For example, in various embodiments, one or more syntax elements, flags, etc., are used to send information to the corresponding decoder. Although the verb form of the word “signal” has been referred to above, the word “signal” can also be used as a noun herein.

[0109] It will be apparent to those skilled in the art that implementations can generate various signals that are formatted to carry, for example, information that can be stored or transmitted. The information may include, for example, instructions for performing a method, or data generated by one of the described implementations. For example, the signal may be formatted to carry a bitstream of the described embodiments. Such a signal may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or baseband signals. Formatting may include, for example, encoding the data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. It is well known that signals can be transmitted via a variety of different wired or wireless links. The signal may be stored on a processor-readable medium.

[0110] The preceding sections describe numerous embodiments spanning various claim classes and types. Features of these embodiments may be provided individually or in any combination. Furthermore, embodiments may include one or more of the following features, devices, or aspects, individually or in any combination, spanning various claim classes and types: At least one embodiment includes using syntax in supplemental enhancement information messages to convey information about the color space of a high dynamic range image.

[0111] At least one embodiment includes the embodiments described above, wherein the syntax is used for other purposes.

[0112] At least one embodiment includes sending standard dynamic range information, along with metadata including the syntax described above, from the encoder to the decoder.

[0113] At least one embodiment includes any of the above embodiments, wherein the encoder signals information to the decoder for color space adjustment in an SDR or HDR environment.

[0114] At least one embodiment includes any encoding or decoding operation based on the above operations.

[0115] At least one embodiment includes encoding or decoding a sub-block using the method described above.

[0116] At least one embodiment includes a bitstream or signal that includes one or more of the described syntax elements or variations thereof.

[0117] At least one embodiment includes a bitstream or signal that includes a syntax for conveying information generated according to any embodiment described herein.

[0118] At least one embodiment includes creating and / or transmitting and / or receiving and / or decoding according to any of the described embodiments.

[0119] At least one embodiment includes a method, process, apparatus, medium for storing instructions, medium for storing data, or signal according to any of the described embodiments.

[0120] At least one embodiment includes inserting a syntax element into the signaling that enables the decoder to determine the decoded information in a manner corresponding to that used by the encoder.

[0121] At least one embodiment includes creating and / or transmitting and / or receiving and / or decoding a bitstream or signal, which includes one or more of the described syntax elements or variations thereof.

[0122] At least one embodiment includes a TV, set-top box, mobile phone, tablet computer or other electronic device that performs the transformation method according to any of the described embodiments.

[0123] At least one embodiment includes a TV, set-top box, mobile phone, tablet computer or other electronic device that performs a transformation method according to any of the described embodiments to determine and display (e.g., using a monitor, screen or other type of display) the resulting image.

[0124] At least one embodiment includes a TV, set-top box, mobile phone, tablet computer or other electronic device that selects, band-limits or tunes (e.g., using a tuner) a channel to receive a signal including an encoded image and performs a transformation method according to any of the described embodiments.

[0125] At least one embodiment includes a TV, set-top box, mobile phone, tablet computer or other electronic device that receives signals including encoded images over the air (e.g., using an antenna) and performs a transformation method.

Claims

1. A method, the method comprising: The video information is conveyed, including standard dynamic range data and metadata, wherein when the metadata does not include information for the original image being transmitted, the metadata conveys a first value indicating the color space primary color information of the high dynamic range image by using a second value in the metadata indicating the color space primary color information of the high dynamic range display and a third value in the metadata indicating the color space primary color information of the standard dynamic range image; and When the metadata includes information about the original image being sent, and when the fourth value in the metadata indicating the primary color information of the color space of the master display of the original image is one, the first value is set to zero, or When the metadata includes information about the original image being sent, and when the fourth value is nine, the first value is set to one; and The standard dynamic range data and the metadata are encoded in a bitstream.

2. An apparatus, the apparatus comprising: Memory, and Processor, the processor being configured to: The video information is conveyed, including standard dynamic range data and metadata, wherein when the metadata does not include information for the original image being transmitted, the metadata conveys a first value indicating the color space primary color information of the high dynamic range image by using a second value in the metadata indicating the color space primary color information of the high dynamic range display and a third value in the metadata indicating the color space primary color information of the standard dynamic range image; and When the metadata includes information about the original image being sent, and when the fourth value in the metadata indicating the primary color information of the color space of the master display of the original image is one, the first value is set to zero, or When the metadata includes information about the original image being sent, and when the fourth value is nine, the first value is set to one; and The standard dynamic range data and the metadata are encoded in a bitstream.

3. A method, the method comprising: Parse the bitstream of video information, which includes standard dynamic range data and metadata. When the metadata does not include information for the original image being transmitted, the metadata conveys a first value indicating the color space primary color information of the high dynamic range image by using a second value in the metadata indicating the color space primary color information of the high dynamic range display and a third value in the metadata indicating the color space primary color information of the standard dynamic range image; and When the metadata includes information about the original image being sent, and when the fourth value in the metadata indicating the primary color information of the color space of the master display of the original image is one, it is inferred that the first value is zero, or When the metadata includes information about the original image being sent, and when the fourth value is nine, the first value is inferred to be one; and The standard dynamic range data and the metadata are decoded from the bitstream to reconstruct a high dynamic range image using the inferred values.

4. An apparatus comprising: Memory, and Processor, the processor being configured to: Parse the bitstream of video information, which includes standard dynamic range data and metadata. When the metadata does not include information for the original image being transmitted, the metadata conveys a first value indicating the color space primary color information of the high dynamic range image by using a second value in the metadata indicating the color space primary color information of the high dynamic range display and a third value in the metadata indicating the color space primary color information of the standard dynamic range image; and When the metadata includes information about the original image being sent, and when the fourth value in the metadata indicating the primary color information of the color space of the master display of the original image is one, it is inferred that the first value is zero, or When the metadata includes information about the original image being sent, and when the fourth value is nine, the first value is inferred to be one; and The standard dynamic range data and the metadata are decoded from the bitstream to reconstruct a high dynamic range image using the inferred values.

5. The method of claim 3 or the apparatus of claim 4, wherein the method or the apparatus further comprises reconstructing high dynamic range video from the standard dynamic range data and the metadata.

6. The method according to any one of claims 1, 3, and 5, or the apparatus according to any one of claims 2, 4, and 5, wherein, The first value corresponds to hdrPicColourSpace, the second value corresponds to hdrDisplayColourSpace, the third value corresponds to sdrPicColourSpace, and the fourth value corresponds to original_picture_primaries in the ETSISL-HDR1 standard recommendation, and also includes original_picture_info_present_flag.

7. The method or apparatus according to claim 6, wherein, original_picture_info_present_flag and original_picture_primaries are included in the SL-HDR supplemental enhancement information message.

8. An apparatus, said apparatus comprising: The apparatus according to claim 4; as well as At least one of the following: (i) an antenna configured to receive a signal including a video block; (ii) a band limiter configured to limit the received signal to a band including the video block; and (iii) a display configured to display an output representing the video block.

9. A non-transitory computer-readable medium comprising data content generated by the method of any one of claims 1 to 7 or by the apparatus of any one of claims 2 to 7 for playback using a processor.

10. A signal comprising video data generated by the method of any one of claims 1 to 7 or by the apparatus of any one of claims 2 to 7, for playback using a processor.

11. A computer program product comprising instructions that, when executed by a computer, cause the computer to perform the method according to any one of claims 1, 3, 5 to 7.

Images