Encoding high dynamic range video data

Abstract

An example apparatus for encoding high dynamic range (HDR) video data includes: a memory configured to store video data; and one or more processors implemented in circuitry and configured to: compute a histogram for an image of the video data, the image being represented in a linear light format; encoding a value of the histogram for the image represented in the linear light format; and encoding the image. Data for the histogram may be represented in an array of variables having a size of 210x18 bits. The apparatus may encode a codeword representing a value for a bin of the histogram, where the codeword may be selected from a set of codewords in a PQ10 format for an HDR image. The bins of the histogram may represent a range of unequal widths.

Description

Technical Field

[0001] The present disclosure relates to video coding, including video encoding and video decoding. Background Art

[0002] Digital video capabilities may be incorporated into a wide variety of devices, including digital televisions, digital live broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e-book readers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, cellular or satellite radio telephones (so-called "smart phones"), video teleconferencing devices, video streaming devices, etc. Digital video devices implement video coding techniques, such as those described in standards defined by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4 Part 10, Advanced Video Coding (AVC), ITU-T H.265 / High Efficiency Video Coding (HEVC), ITU-T H.266 / Versatile Video Coding (VVC), and extensions of such standards, as well as proprietary video codecs / formats such as AOMedia Video 1 (AV1) developed by the Alliance for Open Media. By implementing such video coding techniques, video devices may more efficiently send, receive, encode, decode, and / or store digital video information.

[0003] Video coding techniques include spatial (intra-picture) prediction and / or temporal (inter-picture) prediction to reduce or remove redundancy inherent in video sequences. For block-based video coding, a video slice (e.g., a video picture or a portion of a video picture) may be divided into video blocks, which may also be referred to as coding tree units (CTUs), coding units (CUs), and / or coding nodes. Video blocks in an intra-coded (I) slice of a picture are encoded using spatial prediction relative to reference samples in neighboring blocks in the same picture. Video blocks in an inter-coded (P or B) slice of a picture may use spatial prediction relative to reference samples in neighboring blocks in the same picture or temporal prediction relative to reference samples in other reference pictures. Pictures may be referred to as frames, and reference pictures may be referred to as reference frames. Summary of the invention

[0004] In general, the present disclosure describes techniques for transcoding (encoding and decoding) high dynamic range (HDR) video data. In order to encode HDR video data, statistical information, such as one or more histograms, is calculated for one or more pictures of the HDR video data. Typically, a histogram includes a plurality of bins, each bin representing a range of possible luminance and / or chrominance values, and having a value for indicating the number of samples within the range of possible luminance and / or chrominance values. Pictures of video data may be represented in a variety of different formats, such as a red-green-blue (RGB) format or a luminance and chrominance (e.g., YUV or YCbCr, which represents luminance, blue hue chrominance, and red hue chrominance) format. Thus, a histogram may be calculated in one of a variety of formats for a picture, which is not necessarily the same format as that used when encoding the picture.

[0005] The present disclosure recognizes that computing a histogram for a picture of HDR video data represented in a linear light (LL) RGB format may be more efficient in terms of memory storage for the histogram than computing a histogram for a picture using the PQ10 format. For example, the PQ10 format may require a size of 2 16 *16-bit lookup table, the lookup table for LL RGB format can have 2 10 *18-bit size. Therefore, according to the techniques of the present disclosure, a video encoding system for encoding HDR video data may calculate a histogram for one or more pictures in LL RGB format.

[0006] In one example, a method of encoding high dynamic range (HDR) video data includes: calculating a histogram for an image of the video data, the image being represented in a linear light format; extracting one or more parameters of the histogram; encoding the one or more parameters of the histogram of the image represented in the linear light format; and encoding the image.

[0007] In another example, a device for encoding high dynamic range (HDR) video data includes: a memory configured to store histogram data; and one or more processors implemented in a circuit and configured to: calculate a histogram for an image of the video data, the image being represented in a linear light format; extract one or more parameters of the histogram; encode the one or more parameters of the histogram of the image represented in the linear light format; and encode the image.

[0008] In another example, an apparatus for encoding high dynamic range (HDR) video data includes: a unit for calculating a histogram for an image of the video data, the image being represented in a linear light format; a unit for extracting one or more parameters of the histogram; a unit for encoding the one or more parameters of the histogram of the image represented in the linear light format; and a unit for encoding the image.

[0009] In another example, a computer-readable storage medium is encoded with instructions that, when executed, cause a programmable processor to: calculate a histogram for an image of video data, the image being represented in a linear light format; extract one or more parameters of the histogram; encode the one or more parameters of the histogram of the image represented in the linear light format; and encode the image.

[0010] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a block diagram illustrating an example video encoding and decoding system that may perform the techniques of this disclosure.

[0012] Figure 2 is a block diagram illustrating an example high dynamic range (HDR) video encoding system in accordance with techniques of this disclosure.

[0013] Figure 3 is a schematic diagram showing an example set of bins for a histogram to be calculated according to the techniques of this disclosure.

[0014] Figure 4 is a block diagram illustrating an example video encoder that may perform video encoding according to techniques of this disclosure.

[0015] Figure 5 is a block diagram illustrating an example video decoder that may perform video decoding in accordance with techniques of this disclosure.

[0016] Figure 6 is a flow chart illustrating an example method for encoding picture and histogram data according to techniques of this disclosure.

[0017] Fig. 7A and 7B is a conceptual diagram showing an example of HDR and WCG representation conversion.

[0018] Figure 8 are schematic diagrams showing various examples of transfer functions.

[0019] Fig. 9 is a schematic diagram illustrating an example PQ OETF.

[0020] Fig.10 Here is an example graph of deriving DistributionY99 and DistributionY100 based on the percentiles of the histogram.

[0021] Fig.11 is a block diagram illustrating an example HDR10+ capable live encoder framework.

[0022] Fig.12 is a conceptual diagram illustrating an example process for deriving HDR10+ statistical characteristics of video data, in accordance with techniques of this disclosure. DETAILED DESCRIPTION

[0023] The technology of the present disclosure relates to decoding (encoding and decoding) high dynamic range (HDR) video data. When encoding HDR data, in addition to encoding the image, the video encoder can also calculate statistical information of one or more images, such as a histogram of the image. Therefore, the video encoder can encode both the image itself and the statistical data. In some cases, the image can be represented in various formats. For example, the image can be captured in a red-green-blue (RGB) format and then converted to brightness and chrominance (e.g., YUV or YCbCr, which represents brightness, blue hue chrominance, and red hue chrominance).

[0024] Certain video applications may operate with video data representing captured scenery with HDR and wide color gamut (WCG). The parameters of the utilized dynamic range and color gamut are two independent properties of video content, and their specifications for the purposes of digital television and multimedia services are defined by several international standards. For example, ITU-R Rec.709 defines parameters for HDTV, such as standard dynamic range and standard color gamut, and ITU-R Rec.2020 specifies ultra-high-definition television (UHDTV) parameters, such as high dynamic range and wide color gamut. There are other Standards Development Organization (SDO) documents that specify these properties in other systems, such as the P3 color gamut is defined in SMPTE-231-2, and STMPTE-2084 defines certain parameters for HDR.

[0025] Dynamic range is usually defined as the ratio between the minimum and maximum brightness of a video signal. Dynamic range is also measured in "f-stops", where the difference between one f-stop and another corresponds to a doubling of the signal's dynamic range. The Moving Picture Experts Group (MPEG) defines high dynamic range content as content with a brightness variation of more than 16 f-stops. In some cases, levels between 10 and 16 f-stops are considered intermediate dynamic range, but may be considered HDR in other definitions. At the same time, the human visual system is capable of perceiving a much larger dynamic range. However, the human visual system includes an adaptation mechanism to reduce the so-called simultaneous range.

[0026] ITU-R Rec.709 describes a standard definition range (SDR) that typically supports a brightness (or luminance) range of about 0.1 to 100 candelas (cd) per square meter (commonly referred to as "nits"), resulting in less than 10 stops of dynamic range. Next-generation video services are expected to offer up to 16 stops of dynamic range. Although detailed specifications are currently being developed, some initial parameters have been specified in SMPTE-2084 and ITU-R Rec.2020.

[0027] In addition to HDR, another aspect aimed at a more realistic video experience is the color dimension, which is usually defined by the color gamut. The SDR color gamut can be visualized as a triangle based on the BT.709 color red, green, and blue primaries. WCG is based on the red, green, and blue primaries of BT.2020 and includes the so-called spectral locus, delimited by a tongue-shaped area representing the limits of natural colors. The move from BT.709 to BT.2020 color primaries is intended to provide approximately 70% more colors for UHDTV services. White can also be specified. Table 1 below provides examples of various color gamut specifications for colorimetric parameters of an example color space:

[0028] Table 1

[0029]

[0030] Images of video data are usually encoded in YUV format. For traditional HDR encoding, it has been expected to calculate the histogram for images in R'G'B'P3 / Perceptual Quantizer (PQ) format. The image is first converted from linear light RGB format to PQ format, such as PQ10, based on the PQ Optoelectronic Transfer Function (OETF). Calculating the histogram for an image in PQ10 format requires using a 2-bit histogram. 16 *16-bit Look-up Table (LUT).

[0031] The present disclosure recognizes that if the histogram is calculated from an image in a linear light format, a smaller LUT may be used, for example, a size of 210 *18-bit LUT. Similarly, histograms can be calculated from linear-light format images without performing a conversion from linear-light RGB to PQ10. Therefore, calculating histograms using linear-light RGB format images can reduce processing operations, reduce latency, and reduce memory consumption for storing LUTs.

[0032] Therefore, according to the technology of the present disclosure, a histogram calculation unit can be incorporated into a video encoder or can be separated from a video encoder, and a histogram for one or more images of HDR video data represented in a linear light format can be calculated. When calculating the histogram, the histogram calculation unit can divide the possible range of sample values ​​(brightness and / or chrominance values) into sub-ranges, which correspond to the boxes of the histogram. The sub-ranges for each box in the box are not necessarily, but can be the same width. That is, some sub-ranges can have the same width, while other sub-ranges can have different widths. In some examples, all sub-ranges can have the same width. In some examples, all sub-ranges can have different corresponding widths. The histogram calculation unit can initialize each bin value in the bin value to zero. Then, the histogram calculation unit can traverse the samples of one or more images in the image. The histogram calculation unit can determine the sub-range containing the current sample value, and then increment the bin value corresponding to the determined sub-range.

[0033] After calculating the bin values ​​of the histogram, the histogram calculation unit (or video encoder) may encode the bin values, for example, using a codeword. In some examples, the histogram calculation unit may determine the codeword for the bin value from a subset of PQ10 codewords of conventional HDR video coding techniques.

[0034] Figure 1 1 is a block diagram illustrating an example video encoding and decoding system 100 that may perform techniques of the present disclosure. The techniques of the present disclosure generally relate to decoding (encoding and / or decoding) video data. Generally, video data includes any data used to process video. Thus, video data may include original, undecoded video, encoded video, decoded (e.g., reconstructed) video, and video metadata, such as signaling data.

[0035] like Figure 1, in this example, system 100 includes a source device 102 that provides encoded video data to be decoded and displayed by a destination device 116. In particular, source device 102 provides the video data to destination device 116 via computer-readable medium 110. Source device 102 and destination device 116 may include any of a wide range of devices, including desktop computers, notebook (i.e., laptop) computers, mobile devices, tablet computers, set-top boxes, telephone handsets such as smartphones, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, broadcast receiver devices, etc. In some cases, source device 102 and destination device 116 may be equipped for wireless communication and may therefore be referred to as wireless communication devices.

[0036] exist Figure 1 In the example of , source device 102 includes video source 104, memory 106, video encoder 200, and output interface 108. Destination device 116 includes input interface 122, video decoder 300, memory 120, and display device 118. Therefore, source device 102 represents an example of a video encoding device, and destination device 116 represents an example of a video decoding device. In other examples, source device and destination device may include other components or arrangements. For example, source device 102 may receive video data from an external video source such as an external camera. Similarly, destination device 116 may interface with an external display device rather than including an integrated display device.

[0037] like Figure 1 The system 100 shown in is only an example. The source device 102 and the destination device 116 are only examples of such decoding devices, wherein the source device 102 generates decoded video data for transmission to the destination device 116. The present disclosure refers to a "decoding" device as a device that performs decoding (e.g., encoding and / or decoding) of data. Therefore, the video encoder 200 and the video decoder 300 represent examples of decoding devices, in particular, respectively represent a video encoder and a video decoder. In some examples, the source device 102 and the destination device 116 can operate in a substantially symmetrical manner, so that each of the source device 102 and the destination device 116 includes a video encoding and decoding component. Therefore, the system 100 can support one-way or two-way video transmission between the source device 102 and the destination device 116, for example, for video streaming, video playback, video broadcasting, or video telephony.

[0038] Typically, the video source 104 represents a source of video data (i.e., original, undecoded video data), and provides a sequence of pictures (also referred to as "frames") of the video data to the video encoder 200, which encodes the data for the pictures. The video source 104 of the source device 102 may include a video capture device, such as a camera, a video archive unit containing previously captured original video, and / or a video feed interface for receiving video from a video content provider. As another alternative, the video source 104 may generate computer graphics-based data as the source video, or a combination of real-time video, archived video, and computer-generated video. In each case, the video encoder 200 encodes the captured, pre-captured, or computer-generated video data. The video encoder 200 may rearrange the pictures from the received order (sometimes referred to as "display order") to a decoding order for decoding. The video encoder 200 may generate a bitstream comprising encoded video data. Source device 102 may then output the encoded video data onto computer-readable medium 110 via output interface 108 to be received and / or retrieved by, for example, input interface 122 of destination device 116 .

[0039] The memory 106 of the source device 102 and the memory 120 of the destination device 116 represent general purpose memory. In some examples, the memories 106, 120 can store original video data, for example, the original video from the video source 104 and the original decoded video data from the video decoder 300. In addition or alternatively, the memories 106, 120 can store software instructions executable by, for example, the video encoder 200 and the video decoder 300, respectively. Although the memory 106 and the memory 120 are shown separately from the video encoder 200 and the video decoder 300 in this example, it should be understood that the video encoder 200 and the video decoder 300 can also include internal memory for functionally similar or equivalent purposes. In addition, the memories 106, 120 can store encoded video data, for example, the output from the video encoder 200 and the input to the video decoder 300. In some examples, portions of the memories 106, 120 can be allocated as one or more video buffers, for example, to store original, decoded and / or encoded video data.

[0040] The computer-readable medium 110 may represent any type of medium or device capable of transmitting the encoded video data from the source device 102 to the destination device 116. In one example, the computer-readable medium 110 represents a communication medium that enables the source device 102 to directly send the encoded video data to the destination device 116 in real time, for example, via a radio frequency network or a computer-based network. According to a communication standard such as a wireless communication protocol, the output interface 108 may modulate the transmission signal including the encoded video data, and the input interface 122 may demodulate the received transmission signal. The communication medium may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form a portion of a packet-based network such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include a router, a switch, a base station, or any other device that may be useful for facilitating communication from the source device 102 to the destination device 116.

[0041] In some examples, source device 102 may output the encoded data from output interface 108 to storage device 112. Similarly, destination device 116 may access the encoded data from storage device 112 via input interface 122. Storage device 112 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, Blu-ray disc, DVD, CD-ROM, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.

[0042] In some examples, source device 102 may output the encoded video data to file server 114 or another intermediate storage device that may store the encoded video data generated by source device 102. Destination device 116 may access the stored video data from file server 114 via streaming or downloading.

[0043] The file server 114 may be any type of server device capable of storing encoded video data and sending the encoded video data to the destination device 116. The file server 114 may represent a network server (e.g., for a website), a server configured to provide a file transfer protocol service (e.g., File Transfer Protocol (FTP) or File Delivery over Unidirectional Transport (FLUTE) protocol), a content delivery network (CDN) device, a hypertext transfer protocol (HTTP) server, a Multimedia Broadcast Multicast Service (MBMS) or enhanced MBMS (eMBMS) server, and / or a network attached storage (NAS) device. The file server 114 may additionally or alternatively implement one or more HTTP streaming protocols, such as Dynamic Adaptive Streaming over HTTP (DASH), HTTP Live Streaming (HLS), Real Time Streaming Protocol (RTSP), HTTP Dynamic Streaming, etc.

[0044] Destination device 116 may access the encoded video data from file server 114 through any standard data connection, including an Internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both, suitable for accessing the encoded video data stored on file server 114. Input interface 122 may be configured to operate according to any one or more of the various protocols discussed above for retrieving or receiving media data from file server 114 or other such protocols for retrieving media data.

[0045] Output interface 108 and input interface 122 may represent wireless transmitters / receivers, modems, wired networking components (e.g., Ethernet cards), wireless communication components that operate according to any of the various IEEE 802.11 standards, or other physical components. In examples where output interface 108 and input interface 122 include wireless components, output interface 108 and input interface 122 may be configured to transmit data (such as encoded video data) according to a cellular communication standard (such as 4G, 4G-LTE (Long Term Evolution), LTE Advanced, 5G, etc.). In some examples where output interface 108 includes a wireless transmitter, output interface 108 and input interface 122 may be configured to transmit data (such as encoded video data) according to other wireless standards (such as IEEE 802.11 specifications, IEEE 802.15 specifications (e.g., ZigBee TM ), Bluetooth TMStandards, etc.) to transmit data (such as encoded video data). In some examples, source device 102 and / or destination device 116 may include corresponding system-on-chip (SoC) devices. For example, source device 102 may include a SoC device for performing functions belonging to video encoder 200 and / or output interface 108, and destination device 116 may include a SoC device for performing functions belonging to video decoder 300 and / or input interface 122.

[0046] The techniques of the present disclosure may be applied to video decoding to support any of a variety of multimedia applications, such as over-the-air television broadcasts, cable television transmissions, satellite television transmissions, Internet streaming video transmissions (such as Dynamic Adaptive Streaming over HTTP (DASH)), digital video encoded onto a data storage medium, decoding of digital video stored on a data storage medium, or other applications.

[0047] The input interface 122 of the destination device 116 receives the encoded video bitstream from the computer-readable medium 110 (e.g., a communication medium, a storage device 112, a file server 114, etc.). The encoded video bitstream may include signaling information such as the following syntax elements defined by the video encoder 200 (which are also used by the video decoder 300): the syntax elements have values ​​that describe the characteristics and / or processing of video blocks or other decoding units (e.g., slices, pictures, groups of pictures, sequences, etc.). The display device 118 displays the decoded pictures of the decoded video data to the user. The display device 118 may represent any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0048] Despite Figure 1 Not shown, but in some examples, the video encoder 200 and the video decoder 300 can both be integrated with an audio encoder and / or an audio decoder, and can include appropriate MUX-DEMUX units or other hardware and / or software to process multiplexed streams including both audio and video in a common data stream.

[0049] The video encoder 200 and the video decoder 300 can be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the technology is partially implemented in software, the device can store instructions for the software in a suitable non-transitory computer-readable medium, and use one or more processors to execute the instructions in hardware to perform the technology of the present disclosure. Each of the video encoder 200 and the video decoder 300 can be included in one or more encoders or decoders, and any of the encoders or decoders can be integrated as part of a combined encoder / decoder (CODEC) in the corresponding device. The device including the video encoder 200 and / or the video decoder 300 may include an integrated circuit, a microprocessor, and / or a wireless communication device (such as a cellular phone).

[0050] The video encoder 200 and the video decoder 300 may operate according to a video coding standard such as ITU-T H.265 (also known as High Efficiency Video Coding (HEVC)) or an extension thereof such as multi-view and / or scalable video coding extensions. Alternatively, the video encoder 200 and the video decoder 300 may operate according to other proprietary or industry standards such as ITU-T H.266, also known as Versatile Video Coding (VVC). In other examples, the video encoder 200 and the video decoder 300 may operate according to a proprietary video codec / format such as AOMedia Video 1 (AV1), an extension of AV1, and / or a subsequent version of AV1 (e.g., AV2). In other examples, the video encoder 200 and the video decoder 300 may operate according to other proprietary formats or industry standards. However, the techniques of the present disclosure are not limited to any particular coding standard or format.

[0051] In general, the video encoder 200 and the video decoder 300 may perform block-based decoding of a picture. The term "block" generally refers to a structure that includes data to be processed (e.g., to be encoded, decoded, or otherwise used in an encoding and / or decoding process). For example, a block may include a two-dimensional matrix of samples of luminance and / or chrominance data. Typically, the video encoder 200 and the video decoder 300 may decode video data represented in a YUV (e.g., Y, Cb, Cr) format. That is, instead of decoding red, green, and blue (RGB) data for samples of a picture, the video encoder 200 and the video decoder 300 may decode luminance and chrominance components, wherein the chrominance components may include both red hue and blue hue chrominance components. In some examples, the video encoder 200 converts the received RGB formatted data to a YUV representation before encoding, and the video decoder 300 converts the YUV representation to an RGB format. Alternatively, a pre-processing and post-processing unit (not shown) may perform these conversions.

[0052] The present disclosure may generally relate to the decoding (e.g., encoding and decoding) of a picture to include the process of encoding or decoding data for the picture. Similarly, the present disclosure may relate to the decoding of a block of a picture to include the process of encoding or decoding data for the block (e.g., prediction and / or residual decoding). The encoded video bitstream generally includes a series of values ​​of syntax elements for representing decoding decisions (e.g., decoding modes) and partitioning of pictures into blocks. Therefore, references to the decoding of a picture or a block should generally be understood as the decoded values ​​of the syntax elements that form the picture or block.

[0053] HEVC defines various blocks, including coding units (CUs), prediction units (PUs), and transform units (TUs). According to HEVC, a video decoder (such as the video encoder 200) divides a coding tree unit (CTU) into CUs according to a quadtree structure. That is, the video decoder divides the CTU and the CU into four equal, non-overlapping squares, and each node of the quadtree has zero or four child nodes. A node without child nodes may be referred to as a "leaf node", and the CU of such a leaf node may include one or more PUs and / or one or more TUs. The video decoder may further divide the PU and TU. For example, in HEVC, a residual quadtree (RQT) represents the division of a TU. In HEVC, a PU represents inter-frame prediction data, and a TU represents residual data. An intra-predicted CU includes intra-frame prediction information, such as an intra-frame mode indication.

[0054] As another example, the video encoder 200 and the video decoder 300 may be configured to operate according to VVC. According to VVC, a video decoder (such as the video encoder 200) divides a picture into a plurality of coding tree units (CTUs). The video encoder 200 may divide the CTUs according to a tree structure (such as a quadtree-binary tree (QTBT) structure or a multi-type tree (MTT) structure). The QTBT structure removes the concept of multiple partition types, such as the separation between CU, PU, ​​and TU of HEVC. The QTBT structure includes two levels: a first level divided according to quadtree partitioning, and a second level divided according to binary tree partitioning. The root node of the QTBT structure corresponds to the CTU. The leaf nodes of the binary tree correspond to coding units (CUs).

[0055] In the MTT partition structure, blocks can be partitioned using quadtree (QT) partitioning, binary tree (BT) partitioning, and one or more types of ternary tree (TT) (also referred to as ternary tree (TT)) partitioning. A ternary tree or ternary tree partitioning is a partitioning in which a block is divided into three sub-blocks. In some examples, the ternary tree or ternary tree partitioning divides the block into three sub-blocks without dividing the original block through the center. The partitioning types in MTT (e.g., QT, BT, and TT) can be symmetric or asymmetric.

[0056] When operating according to the AV1 codec, the video encoder 200 and the video decoder 300 can be configured to decode the video data in blocks. In AV1, the largest decoding block that can be processed is called a super block. In AV1, a super block can be 128x128 luminance samples or 64x64 luminance samples. However, in subsequent video decoding formats (e.g., AV2), super blocks can be defined by different (e.g., larger) luminance sample sizes. In some examples, the super block is the top layer of the block quadtree. The video encoder 200 can further divide the super block into smaller decoding blocks. The video encoder 200 can use square or non-square division to divide the super block and other decoding blocks into smaller blocks. Non-square blocks can include N / 2xN, NxN / 2, N / 4xN and NxN / 4 blocks. The video encoder 200 and the video decoder 300 can perform separate prediction and transformation processes on each decoding block in the decoding block.

[0057] AV1 also defines tiles of video data. A tile is a rectangular array of super blocks that can be decoded independently of other tiles. That is, the video encoder 200 and the video decoder 300 can encode and decode coded blocks within a tile, respectively, without using video data from other tiles. However, the video encoder 200 and the video decoder 300 can perform filtering across tile boundaries. The size of the tile can be uniform or non-uniform. Tile-based decoding can achieve parallel processing and / or multi-threading for encoder and decoder implementations.

[0058] In some examples, the video encoder 200 and the video decoder 300 may use a single QTBT or MTT structure to represent each of the luma component and the chroma components, while in other examples, the video encoder 200 and the video decoder 300 may use two or more QTBT or MTT structures, such as one QTBT / MTT structure for the luma component and another QTBT / MTT structure for the two chroma components (or two QTBT / MTT structures for the respective chroma components).

[0059] The video encoder 200 and the video decoder 300 may be configured to use quadtree partitioning, QTBT partitioning, MTT partitioning, super block partitioning, or other partitioning structures.

[0060] In some examples, a CTU includes a coding tree block (CTB) of luma samples for a picture with three sample arrays, two corresponding CTBs of chroma samples, or a CTB of samples for a monochrome picture or a picture coded using three separate color planes, and a syntax structure for coding the samples. A CTB can be an NxN block of samples for some value of N, such that partitioning a component into a CTB is a partition. A component can be a single sample from one of the three arrays (luminance and two chroma) for a picture in 4:2:0, 4:2:2, or 4:4:4 color format, or an array or a single sample of the array for a picture in monochrome format. In some examples, a coding block is an MxN block of samples for some values ​​of M and N, such that partitioning a CTB into coding blocks is a partition.

[0061] Blocks (e.g., CTUs or CUs) may be grouped in a picture in various ways. As an example, a brick may refer to a rectangular area of ​​a CTU row within a particular tile in a picture. A slice may be a rectangular area of ​​a CTU within a particular tile column and a particular tile row in a picture. A tile column refers to a rectangular area of ​​a CTU having a height equal to the height of the picture and a width specified by a syntax element (e.g., such as in a picture parameter set). A tile row refers to a rectangular area of ​​a CTU having a height specified by a syntax element (e.g., such as in a picture parameter set) and a width equal to the width of the picture.

[0062] In some examples, a slice may be divided into multiple bricks, each brick may include one or more CTU rows within the slice. A slice that is not divided into multiple bricks may also be referred to as a brick. However, a brick that is a true subset of a slice may not be referred to as a slice. Bricks in a picture may also be arranged in slices. A slice may be an integer number of bricks of a picture that may be uniquely contained in a single network abstraction layer (NAL) unit. In some examples, a slice includes a continuous sequence of multiple complete slices or only complete bricks of one slice.

[0063] This disclosure may use "NxN" and "N by N" interchangeably to refer to the sample size of a block (such as a CU or other video block) in terms of vertical and horizontal dimensions, for example, 16x16 samples or 16 by 16 samples. Typically, a 16x16 CU will have 16 samples in the vertical direction (y=16) and 16 samples in the horizontal direction (x=16). Similarly, an NxNCU typically has N samples in the vertical direction and N samples in the horizontal direction, where N represents a non-negative integer value. The samples in a CU may be arranged in rows and columns. In addition, a CU need not necessarily have the same number of samples in the horizontal direction as in the vertical direction. For example, a CU may include NxM samples, where M is not necessarily equal to N.

[0064] The video encoder 200 encodes video data representing prediction and / or residual information and other information for a CU. The prediction information indicates how to predict the CU in order to form a prediction block for the CU. The residual information generally represents the sample-by-sample difference between the samples of the CU before encoding and the prediction block.

[0065] In order to predict a CU, the video encoder 200 may generally form a prediction block of the CU by inter-frame prediction or intra-frame prediction. Inter-frame prediction generally refers to predicting a CU from data of a previously decoded picture, while intra-frame prediction generally refers to predicting a CU from previously decoded data of the same picture. In order to perform inter-frame prediction, the video encoder 200 may use one or more motion vectors to generate a prediction block. The video encoder 200 may generally perform a motion search to identify a reference block that closely matches the CU, for example, based on the difference between the CU and the reference block. The video encoder 200 may calculate a difference metric using the sum of absolute differences (SAD), the sum of squared differences (SSD), the mean absolute difference (MAD), the mean square difference (MSD), or other such difference calculations to determine whether the reference block closely matches the current CU. In some examples, the video encoder 200 may use unidirectional prediction or bidirectional prediction to predict the current CU.

[0066] Some examples of VVC also provide an affine motion compensation mode, which can be considered an inter-frame prediction mode. In the affine motion compensation mode, the video encoder 200 can determine two or more motion vectors representing non-translational motion (such as zooming in or out, rotation, perspective motion, or other irregular motion types).

[0067] To perform intra prediction, the video encoder 200 may select an intra prediction mode to generate a prediction block. Some examples of VVC provide sixty-seven intra prediction modes, including various directional modes, as well as planar modes and DC modes. In general, the video encoder 200 selects an intra prediction mode that describes neighboring samples of a current block (e.g., a block of a CU) and predicts samples of the current block from the neighboring samples. Assuming that the video encoder 200 decodes CTUs and CUs in raster scan order (from left to right, from top to bottom), such samples may typically be above, above left, or to the left of the current block in the same picture as the current block.

[0068] The video encoder 200 encodes data representing the prediction mode of the current block. For example, for inter-frame prediction mode, the video encoder 200 may encode data indicating which of various available inter-frame prediction modes is used and motion information of the corresponding mode. For example, for unidirectional or bidirectional inter-frame prediction, the video encoder 200 may encode motion vectors using advanced motion vector prediction (AMVP) or merge mode. The video encoder 200 may use a similar mode to encode motion vectors for affine motion compensation mode.

[0069] AV1 includes two general techniques for encoding and decoding coding blocks of video data. The two general techniques are intra prediction (e.g., intra frame prediction or spatial prediction) and inter prediction (e.g., inter frame prediction or temporal prediction). In the context of AV1, when an intra prediction mode is used to predict a current frame of video data, the video encoder 200 and the video decoder 300 do not use video data from other frames of the video data. For most intra prediction modes, the video encoder 200 encodes a block of the current frame based on the difference between the sample values ​​in the current block and the prediction values ​​generated from the reference samples in the same frame. The video encoder 200 determines the prediction values ​​generated from the reference samples based on the intra prediction mode.

[0070] After prediction (such as intra-frame prediction or inter-frame prediction for a block), the video encoder 200 can calculate residual data for the block. The residual data (such as a residual block) represents the sample-by-sample difference between the block and a prediction block for the block, which is formed using a corresponding prediction mode. The video encoder 200 can apply one or more transforms to the residual block to produce transformed data in a transform domain rather than in a sample domain. For example, the video encoder 200 can apply a discrete cosine transform (DCT), an integer transform, a wavelet transform, or a conceptually similar transform to the residual video data. In addition, the video encoder 200 can apply a secondary transform after the first transform, such as a mode-dependent non-separable secondary transform (MDNSST), a signal-dependent transform, a Karhunen-Loeve transform (KLT), etc. The video encoder 200 generates transform coefficients after applying one or more transforms.

[0071] As noted above, after any transforms that produce transform coefficients, the video encoder 200 may perform quantization of the transform coefficients. Quantization generally refers to a process in which transform coefficients are quantized to possibly reduce the amount of data used to represent the transform coefficients, thereby providing further compression. By performing the quantization process, the video encoder 200 may reduce the bit depth associated with some or all of the transform coefficients. For example, the video encoder 200 may round down an n-bit value to an m-bit value during quantization, where n is greater than m. In some examples, to perform quantization, the video encoder 200 may perform a bitwise right shift on the value to be quantized.

[0072] After quantization, the video encoder 200 may scan the transform coefficients to generate a one-dimensional vector from a two-dimensional matrix including the quantized transform coefficients. The scan may be designed to place the transform coefficients of higher energy (and therefore lower frequency) in front of the vector and the transform coefficients of lower energy (and therefore higher frequency) in the back of the vector. In some examples, the video encoder 200 may scan the quantized transform coefficients using a predefined scan order to generate a serialized vector, and then entropy encode the quantized transform coefficients of the vector. In other examples, the video encoder 200 may perform adaptive scanning. After scanning the quantized transform coefficients to form a one-dimensional vector, the video encoder 200 may entropy encode the one-dimensional vector, for example, according to context adaptive binary arithmetic coding (CABAC). The video encoder 200 may also entropy encode the values ​​of the syntax elements describing the metadata associated with the encoded video data for use by the video decoder 300 when decoding the video data.

[0073] To perform CABAC, the video encoder 200 may assign context within a context model to a symbol to be transmitted. The context may relate to, for example, whether neighboring values ​​of the symbol are zero values. The probability determination may be based on the context assigned to the symbol.

[0074] The video encoder 200 may further generate syntax data (such as block-based syntax data, picture-based syntax data, and sequence-based syntax data), or other syntax data (such as a sequence parameter set (SPS), a picture parameter set (PPS), or a video parameter set (VPS)), for example, in a picture header, a block header, a slice header, for the video decoder 300. The video decoder 300 may also decode such syntax data to determine how to decode the corresponding video data.

[0075] In this way, the video encoder 200 can generate a bitstream including the encoded video data, for example, a syntax element describing the partition of the picture into blocks (e.g., CU) and prediction and / or residual information for the blocks. Finally, the video decoder 300 can receive the bitstream and decode the encoded video data.

[0076] In general, the video decoder 300 performs a process that is the reverse of the process performed by the video encoder 200 to decode the encoded video data of the bitstream. For example, the video decoder 300 may use CABAC to decode the values ​​of the syntax elements for the bitstream in a manner substantially similar to, but reverse to, the CABAC encoding process of the video encoder 200. The syntax elements may define partition information for partitioning a picture into CTUs and partitioning each CTU according to a corresponding partition structure such as a QTBT structure to define CUs of the CTUs. The syntax elements may further define prediction and residual information for a block of video data (e.g., a CU).

[0077] The residual information may be represented by, for example, quantized transform coefficients. The video decoder 300 may inverse quantize and inverse transform the quantized transform coefficients of the block to reproduce a residual block for the block. The video decoder 300 uses the signaled prediction mode (intra-frame prediction or inter-frame prediction) and related prediction information (e.g., motion information for inter-frame prediction) to form a prediction block for the block. The video decoder 300 may then combine the prediction block and the residual block (on a sample-by-sample basis) to reproduce the original block. The video decoder 300 may perform additional processing, such as performing a deblocking process to reduce visual artifacts along the boundaries of the blocks.

[0078] The present disclosure may generally involve "signaling" certain information (such as syntax elements). The term "signaling" may generally refer to the transmission of values ​​for syntax elements and / or other data used to decode encoded video data. That is, the video encoder 200 may signal the values ​​of syntax elements in a bitstream. Generally, signaling refers to generating a value in a bitstream. As noted above, the source device 102 may transmit the bitstream to the destination device 116 in substantially real time or in non-real time (such as may occur when storing syntax elements to the storage device 112 for later retrieval by the destination device 116).

[0079] Figure 2 1 is a block diagram illustrating an example high dynamic range (HDR) video encoding system 140 according to the disclosed techniques. The HDR video encoding system 140 may be incorporated into Figure 1 In the source device 102.

[0080] In this example, the HDR video encoding system 140 includes a video source 104, a video format converter 130, a video encoder 200, an HDR statistics engine 132, a memory 133, a lookup table (LUT) 134, and a packaging unit 136. Figure 1As discussed, video source 104 may generally capture and / or generate video data, ie, raw, undecoded video data. The raw, undecoded video data may be in a red-green-blue (RGB) format.

[0081] Video format converter 130 is generally configured to convert raw, uncoded video data from video source 104 into one or more of a variety of different formats. For example, video format converter 130 may convert raw, uncoded video data into a luma and chroma (YUV or YCbCr) format for encoding by video encoder 200. In accordance with techniques of the present disclosure, video format converter 130 may convert raw, uncoded video data from video source 104 into a linear light (LL) format, such as RGB LL16.

[0082] The HDR statistics engine 132 may calculate a histogram based on one or more images in the LL format. The HDR statistics engine 132 may store the values ​​for the histogram in the memory 133. The HDR statistics engine 132 may use the LUT 134 to calculate the histogram. In general, the LUT 134 may map sample values ​​of an image to an index value, where the index value corresponds to a bin of the histogram. The LUT 134 may be a 2-bit LUT. 10 x 16-bit constant array. In one example, the HDR statistics engine 132 may calculate the index value as follows:

[0083] maxRGB=max(R,G,B),16bits P3 LL

[0084] if(maxRGB<2)

[0085] index=maxRGB

[0086] else{

[0087] index1=LUT1(log2(maxRGB))

[0088] index2=isEqual(maxRGB,within LUT2(log2(maxRGB))

[0089] index=index1+index2

[0090] }

[0091] Where "index" represents the index value corresponding to the bin of the histogram, and max(R,G,B) returns the maximum value (e.g., maxRGB) among the values ​​of the samples at the same position in each of the red, green, and blue color component images of the current image. The HDR statistics engine 132 can calculate a size of 210 A histogram of the variables in an array of x18.

[0092] The HDR statistics engine 132 may increment the value of the bin of the histogram corresponding to the determined index value. For example, the HDR statistics engine 132 may perform a "hist[index]++" operation, where "hist[index]" represents a bin of the histogram, and the "++" operator increments the value of the corresponding bin by 1. The search complexity involved in obtaining the bin index may be O(10) because there may be 10 comparators for each calculation of maxRGB. The search may be a binary search, such as a binary search tree.

[0093] After calculating the values ​​of the bins of the histogram, the HDR statistics engine 132 may determine a codeword for the bin value. The codeword may be selected from the same set of codewords as the histogram calculated using the PQ10 format image. For example, the HDR statistics engine 132 may select a codeword from a set of codewords in the PQ10 format for HDR images. As another example, the HDR statistics engine 132 may initially form a strict subset of possible codewords from the set of codewords in the PQ10 format for HDR images, and then select a codeword from the formed subset. The histogram bins do not necessarily represent a uniform width of values ​​in the LL format.

[0094] The present disclosure recognizes that PQ / Linear Light defines the pixel brightness of a sample in the range of 0 to 10,000 nits. HDR10+ specifies up to 0.1 candela per square meter (cd / m 2 PQ10 has 1024 codewords to cover a range of 10,000 possible values ​​with a granularity of 0.1 cd / m 2 . Therefore, the histogram calculated in the PQ10 domain is suboptimal for HDR10+. Instead, according to the techniques of the present disclosure, the HDR statistics engine 132 may instead calculate the histogram in the RGB P3 linear domain. The quality of the HDR10 histogram in the PQ10 domain can be achieved with fewer than 1023 bins in the linear domain. The histogram index can be calculated by downscaling and right shifting with respect to the values. For example, the HDR statistics engine 132 may perform the following operations:

[0095] Calculate N ranges: TS[N]

[0096] For example, N may be 1023 or less.

[0097] In some examples, N may be as low as 650.

[0098] Process Flow:

[0099] Compare the linear maxRGB value to TS[i].

[0100] The range index is derived by binary search or an analytical process.

[0101] Range index used as histogram index hi = function (maxRGB, TS[i])

[0102] histLinRGB[hi]++

[0103] Hardware implementation:

[0104] TS LUT: Nx18 bits

[0105] Histogram: 1024x18 bits (i.e. 2 10 x 18 bits)

[0106] Binary search of LUT segments (up to 40 LUT entries)

[0107] The video encoder 200 may encode each of the images used to calculate the histogram. The encapsulation unit 136 may encapsulate the encoded images (e.g., encoded slices of the images) and the encoded histograms (e.g., the determined codewords or encoded versions of the codewords) from the HDR statistics engine 132. Ultimately, the encapsulation unit 136 may form a bitstream 138 including data for the encoded images and the encoded histograms.

[0108] Each of the video format converter 130, the HDR statistics engine 132, and the packaging unit 136 may be implemented in one or more processing units, i.e., one or more hardware-based processors are implemented in a circuit. For example, a processing unit may include one or more circuits or logic elements as part of a hardware circuit, or as part of a processor, ASIC, or FPGA. The memory 133 and the LUT 134 may correspond to various storage devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), solid-state storage devices, or other types of storage devices.

[0109] In this manner, the video encoding system 140 represents an example of a device for encoding high dynamic range (HDR) video data, comprising: a memory configured to store histogram data; and one or more processors implemented in circuitry and configured to: calculate a histogram for an image of the video data, the image being represented in a linear light format; encode values ​​of the histogram for the image represented in the linear light format; and encode the image.

[0110] Figure 31 is a diagram 150 showing an example set of bins for a histogram calculated according to the techniques of the present disclosure. X-axis 152 represents possible index values ​​for a bin, while Y-axis 154 represents possible maxRGB values. Stepwise function 156 approximates curve 158. Curve 158 represents codewords for PQ10, while stepwise function 156 represents codewords calculated from maxRGB using a linear light format according to the techniques of the present disclosure.

[0111] It can be seen that the step function 156 has different widths for different synchronizations. Figure 3 As shown, the range for the steps of the stepwise function 156 near the left side of the schematic diagram 150 is narrower than the steps of the stepwise function 156 near the right side of the schematic diagram 150. Thus, a first range of possible maxRGB values ​​can be mapped to a single bin of the histogram, and a second range of possible maxRGB values ​​can be mapped to a different bin of the histogram, where the second range can be wider than the first range.

[0112] Figure 4 is a block diagram illustrating an example video encoder 200 that may perform video encoding in accordance with techniques of this disclosure. Figure 4 The present disclosure generally illustrates and describes the techniques of the present invention. For the purpose of explanation, the present disclosure describes a video encoder 200 according to VVC (ITU-T H.266, under development) and HEVC (ITU-T H.265) techniques. However, the techniques of the present disclosure may be performed by video encoding devices configured for other video coding standards and video coding formats, such as AV1 and successors to the AV1 video coding format.

[0113] exist Figure 4In the example of , the video encoder 200 includes a video data memory 230, a mode selection unit 202, a residual generation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a filter unit 216, a decoded picture buffer (DPB) 218, and an entropy coding unit 220. Any or all of the video data memory 230, the mode selection unit 202, the residual generation unit 204, the transform processing unit 206, the quantization unit 208, the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the filter unit 216, the DPB 218, and the entropy coding unit 220 can be implemented in one or more processors or in processing circuits. For example, the units of the video encoder 200 can be implemented as one or more circuits or logic elements as part of a hardware circuit, or as part of a processor, ASIC, or FPGA. In addition, the video encoder 200 may include additional or alternative processors or processing circuits to perform these and other functions.

[0114] Video data memory 230 may store video data to be encoded by components of video encoder 200. Video encoder 200 may receive video data from, for example, video source 104 ( Figure 1 ) receives video data stored in video data memory 230. DPB 218 can act as a reference picture memory, which stores reference video data for use when subsequent video data is predicted by video encoder 200. Video data memory 230 and DPB 218 can be formed by any memory device in a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), solid-state storage device, or other types of memory devices. Video data memory 230 and DPB 218 can be provided by the same memory device or a separate memory device. In various examples, video data memory 230 can be on chip with other components of video encoder 200 (as shown), or off chip relative to those components.

[0115] In the present disclosure, references to the video data memory 230 should not be interpreted as limited to memory internal to the video encoder 200 (unless specifically described as such) or memory external to the video encoder 200 (unless specifically described as such). Rather, references to the video data memory 230 should be understood as a reference memory that stores video data received by the video encoder 200 for encoding (e.g., video data for a current block to be encoded). Figure 1 The memory 106 may also provide temporary storage for outputs from the various units of the video encoder 200 .

[0116] Show Figure 4 The various units of the video encoder 200 are described to help understand the operations performed by the video encoder 200. The units may be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Fixed-function circuits refer to circuits that provide specific functions and pre-set the operations that can be performed. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functions in the operations that can be performed. For example, a programmable circuit can execute software or firmware that causes the programmable circuit to operate in a manner defined by the instructions of the software or firmware. Fixed-function circuits can execute software instructions (e.g., receive parameters or output parameters), but the types of operations performed by fixed-function circuits are generally immutable. In some examples, one or more of the units may be different circuit blocks (fixed-function or programmable), and in some examples, one or more of the units may be integrated circuits.

[0117] The video encoder 200 may include an arithmetic logic unit (ALU), an elementary function unit (EFU), a digital circuit, an analog circuit, and / or a programmable core formed by a programmable circuit. In an example in which the operation of the video encoder 200 is performed using software executed by a programmable circuit, the memory 106 ( Figure 1 ) may store instructions (eg, object code) for software received and executed by the video encoder 200, or another memory (not shown) within the video encoder 200 may store such instructions.

[0118] The video data memory 230 is configured to store received video data. The video encoder 200 may retrieve pictures of video data from the video data memory 230 and provide the video data to the residual generation unit 204 and the mode selection unit 202. The video data in the video data memory 230 may be original video data to be encoded.

[0119] The mode selection unit 202 includes a motion estimation unit 222, a motion compensation unit 224, and an intra prediction unit 226. The mode selection unit 202 may include additional functional units to perform video prediction according to other prediction modes. As an example, the mode selection unit 202 may include a palette unit, an intra block copy unit (which may be part of the motion estimation unit 222 and / or the motion compensation unit 224), an affine unit, a linear model (LM) unit, etc.

[0120] The mode selection unit 202 typically coordinates multiple encoding passes to test combinations of encoding parameters and resulting rate-distortion values ​​for such combinations. The encoding parameters may include the partitioning of CTUs into CUs, the prediction mode for a CU, the transform type for residual data of a CU, the quantization parameter for residual data of a CU, etc. The mode selection unit 202 may ultimately select a coding parameter combination that has a better rate-distortion value than other tested combinations.

[0121] The video encoder 200 may divide the picture retrieved from the video data memory 230 into a series of CTUs, and encapsulate one or more CTUs in a slice. The mode selection unit 202 may divide the CTU of the picture according to a tree structure (such as the above-mentioned MTT structure, QTBT structure, super block structure, or quadtree structure). As described above, the video encoder 200 may form one or more CUs by dividing the CTU according to the tree structure. Such a CU may also be generally referred to as a "video block" or "block".

[0122] In general, the mode selection unit 202 also controls its components (e.g., the motion estimation unit 222, the motion compensation unit 224, and the intra prediction unit 226) to generate a prediction block for a current block (e.g., a current CU, or in HEVC, the overlapping portion of a PU and a TU). In order to inter-predict the current block, the motion estimation unit 222 may perform a motion search to identify one or more closely matching reference blocks in one or more reference pictures (e.g., one or more previously decoded pictures stored in the DPB 218). In particular, the motion estimation unit 222 may calculate values ​​representing how similar potential reference blocks will be to the current block, for example, based on the sum of absolute differences (SAD), the sum of squared differences (SSD), the mean absolute difference (MAD), the mean squared difference (MSD), etc. The motion estimation unit 222 may generally perform these calculations using the sample-by-sample differences between the current block and the reference block under consideration. The motion estimation unit 222 may identify the reference block having the lowest value resulting from these calculations, which indicates the reference block that best matches the current block.

[0123] The motion estimation unit 222 may form one or more motion vectors (MVs) that define the position of the reference block in the reference picture relative to the position of the current block in the current picture. The motion estimation unit 222 may then provide the motion vectors to the motion compensation unit 224. For example, for unidirectional inter prediction, the motion estimation unit 222 may provide a single motion vector, while for bidirectional inter prediction, the motion estimation unit 222 may provide two motion vectors. The motion compensation unit 224 may then use the motion vectors to generate a prediction block. For example, the motion compensation unit 224 may use the motion vectors to retrieve data for the reference block. As another example, if the motion vector has fractional sample precision, the motion compensation unit 224 may interpolate the values ​​for the prediction block according to one or more interpolation filters. In addition, for bidirectional inter prediction, the motion compensation unit 224 may retrieve data for two reference blocks identified by the corresponding motion vectors and combine the retrieved data, for example, by sample-by-sample averaging or weighted averaging.

[0124] When operating according to the AV1 video coding format, the motion estimation unit 222 and the motion compensation unit 224 may be configured to encode coding blocks of the video data (e.g., both luma coding blocks and chroma coding blocks) using translational motion compensation, affine motion compensation, overlapped block motion compensation (OBMC), and / or composite inter-intra prediction.

[0125] As another example, for intra prediction or intra prediction decoding, the intra prediction unit 226 may generate a prediction block from samples adjacent to the current block. For example, for directional mode, the intra prediction unit 226 may generally mathematically combine the values ​​of adjacent samples and fill these calculated values ​​across the current block in a defined direction to produce a prediction block. As another example, for DC mode, the intra prediction unit 226 may calculate an average value of adjacent samples of the current block and generate a prediction block to include the resulting average value for each sample of the prediction block.

[0126] When operating according to the AV1 video coding format, the intra prediction unit 226 may be configured to encode coding blocks of video data (e.g., both luma coding blocks and chroma coding blocks) using directional intra prediction, non-directional intra prediction, recursive filter intra prediction, chroma from luma (CFL) prediction, intra block copy (IBC), and / or palette mode. The mode selection unit 202 may include additional functional units for performing video prediction according to other prediction modes.

[0127] The mode selection unit 202 provides the prediction block to the residual generation unit 204. The residual generation unit 204 receives the original undecoded version of the current block from the video data memory 230 and receives the prediction block from the mode selection unit 202. The residual generation unit 204 calculates the sample-by-sample difference between the current block and the prediction block. The resulting sample-by-sample difference defines a residual block for the current block. In some examples, the residual generation unit 204 may also determine the difference between the sample values ​​in the residual block to generate the residual block using residual differential pulse coding modulation (RDPCM). In some examples, the residual generation unit 204 may be formed using one or more subtractor circuits that perform binary subtraction.

[0128] In an example where the mode selection unit 202 divides the CU into PUs, each PU may be associated with a luma prediction unit and a corresponding chroma prediction unit. The video encoder 200 and the video decoder 300 may support PUs of various sizes. As noted above, the size of a CU may refer to the size of a luma coding block of the CU, and the size of a PU may refer to the size of a luma prediction unit of the PU. Assuming that the size of a particular CU is 2Nx2N, the video encoder 200 may support PU sizes of 2Nx2N or NxN for intra-prediction, and 2Nx2N, 2NxN, Nx2N, NxN or similar symmetric PU sizes for inter-prediction. The video encoder 200 and the video decoder 300 may also support asymmetric partitioning of PU sizes of 2NxnU, 2NxnD, nLx2N, and nRx2N for inter-prediction.

[0129] In an example where mode selection unit 202 does not further divide a CU into PUs, each CU may be associated with a luma coding block and a corresponding chroma coding block. As above, the size of a CU may refer to the size of the luma coding block of the CU. Video encoder 200 and video decoder 300 may support CU sizes of 2Nx2N, 2NxN, or Nx2N.

[0130] For other video coding techniques (such as intra block copy mode coding, affine mode coding, and linear model (LM) mode coding, to name a few examples), mode selection unit 202 generates a prediction block for the current block being encoded via a corresponding unit associated with the coding technique. In some examples (such as palette mode coding), mode selection unit 202 may not generate a prediction block, but instead generate syntax elements indicating a manner for reconstructing the block based on the selected palette. In such modes, mode selection unit 202 may provide these syntax elements to entropy encoding unit 220 for encoding.

[0131] As described above, the residual generation unit 204 receives video data for a current block and a corresponding prediction block. Then, the residual generation unit 204 generates a residual block for the current block. To generate the residual block, the residual generation unit 204 calculates the sample-by-sample difference between the prediction block and the current block.

[0132] The transform processing unit 206 applies one or more transforms to the residual block to generate a block of transform coefficients (referred to herein as a "transform coefficient block"). The transform processing unit 206 may apply various transforms to the residual block to form a transform coefficient block. For example, the transform processing unit 206 may apply a discrete cosine transform (DCT), a directional transform, a Karhunen-Loeve transform (KLT), or a conceptually similar transform to the residual block. In some examples, the transform processing unit 206 may perform a variety of transforms on the residual block, for example, a primary transform and a secondary transform (such as a rotation transform). In some examples, the transform processing unit 206 does not apply a transform to the residual block.

[0133] When operating in accordance with AV1, the transform processing unit 206 may apply one or more transforms to the residual block to generate a block of transform coefficients (referred to herein as a "transform coefficient block"). The transform processing unit 206 may apply various transforms to the residual block to form a transform coefficient block. For example, the transform processing unit 206 may apply a horizontal / vertical transform combination, which may include a discrete cosine transform (DCT), an asymmetric discrete sine transform (ADST), a flipped ADST (e.g., an ADST in reverse order), and an identity transform (IDTX). When an identity transform is used, a transform is skipped in one of the vertical or horizontal directions. In some examples, the transform process may be skipped.

[0134] Quantization unit 208 may quantize transform coefficients in a transform coefficient block to produce a quantized transform coefficient block. Quantization unit 208 may quantize the transform coefficients of the transform coefficient block according to a QP value associated with the current block. Video encoder 200 (e.g., via mode selection unit 202) may adjust the degree of quantization applied to the transform coefficient block associated with the current block by adjusting the QP value associated with the CU. Quantization may cause a loss of information, and therefore, the quantized transform coefficients may have a lower precision than the original transform coefficients produced by transform processing unit 206.

[0135] The inverse quantization unit 210 and the inverse transform processing unit 212 may apply inverse quantization and inverse transform, respectively, to the quantized transform coefficient block to reconstruct a residual block from the transform coefficient block. The reconstruction unit 214 may generate a reconstructed block corresponding to the current block (although potentially with some degree of distortion) based on the reconstructed residual block and the prediction block generated by the mode selection unit 202. For example, the reconstruction unit 214 may add samples of the reconstructed residual block to corresponding samples of the prediction block generated by the mode selection unit 202 to generate a reconstructed block.

[0136] Filter unit 216 may perform one or more filter operations on the reconstructed block. For example, filter unit 216 may perform a deblocking operation to reduce blockiness artifacts along the edges of the CU. In some examples, the operations of filter unit 216 may be skipped.

[0137] When operating according to AV1, filter unit 216 may perform one or more filtering operations on the reconstructed block. For example, filter unit 216 may perform a deblocking operation to reduce blocking artifacts along the edges of the CU. In other examples, filter unit 216 may apply a constrained directional enhancement filter (CDEF), which may be applied after deblocking and may include the application of an inseparable, nonlinear, low-pass directional filter based on an estimated edge direction. Filter unit 216 may also include a loop restoration filter, which is applied after CDEF and may include a separable symmetric normalized Wiener filter or a dual self-guided filter.

[0138] The video encoder 200 stores the reconstructed blocks in the DPB 218. For example, in an example where the operation of the filter unit 216 is not performed, the reconstruction unit 214 can store the reconstructed blocks in the DPB 218. In an example where the operation of the filter unit 216 is performed, the filter unit 216 can store the filtered reconstructed blocks in the DPB 218. The motion estimation unit 222 and the motion compensation unit 224 can retrieve the reference picture formed by the reconstructed (and potentially filtered) blocks from the DPB 218 to perform inter-frame prediction on the blocks of the subsequently encoded pictures. In addition, the intra-frame prediction unit 226 can use the reconstructed blocks of the current picture in the DPB 218 to perform intra-frame prediction on other blocks in the current picture.

[0139] In general, the entropy coding unit 220 may entropy encode syntax elements received from other functional components of the video encoder 200. For example, the entropy coding unit 220 may entropy encode a quantized transform coefficient block from the quantization unit 208. As another example, the entropy coding unit 220 may entropy encode a prediction syntax element (e.g., motion information for inter-frame prediction or intra-frame mode information for intra-frame prediction) from the mode selection unit 202. The entropy coding unit 220 may perform one or more entropy coding operations on the syntax element (which is another example of video data) to generate entropy-encoded data. For example, the entropy coding unit 220 may perform a context-adaptive variable length coding (CAVLC) operation, a CABAC operation, a variable-to-variable (V2V) length coding operation, a syntax-based context-adaptive binary arithmetic coding (SBAC) operation, a probability interval partitioning entropy (PIPE) coding operation, an exponential Golomb coding operation, or another type of entropy coding operation on the data. In some examples, the entropy coding unit 220 may operate in a bypass mode where the syntax element is not entropy encoded.

[0140] The video encoder 200 may output a bitstream including entropy-encoded syntax elements required to reconstruct blocks of a slice or picture. In particular, the entropy encoding unit 220 may output a bitstream.

[0141] According to AV1, the entropy coding unit 220 may be configured as a symbol-to-symbol adaptive multi-symbol arithmetic decoder. The syntax elements in AV1 include an alphabet of N elements, and the context (e.g., a probability model) includes a set of N probabilities. The entropy coding unit 220 may store the probabilities as an n-bit (e.g., 15-bit) cumulative distribution function (CDF). The entropy coding unit 220 may perform recursive scaling (with an update factor based on the alphabet size) to update the context.

[0142] The operations described above are described with respect to blocks. Such descriptions should be understood as operations for luma coding blocks and / or chroma coding blocks. As described above, in some examples, the luma coding blocks and chroma coding blocks are the luma components and chroma components of a CU. In some examples, the luma coding blocks and chroma coding blocks are the luma components and chroma components of a PU.

[0143] In some examples, operations performed with respect to luma coding blocks need not be repeated for chroma coding blocks. As an example, operations to identify motion vectors (MVs) and reference pictures for luma coding blocks need not be repeated to identify MVs and reference pictures for chroma blocks. Instead, the MVs for luma coding blocks may be scaled to determine MVs for chroma blocks, and the reference pictures may be the same. As another example, the intra prediction process may be the same for luma coding blocks and chroma coding blocks.

[0144] Figure 5 is a block diagram illustrating an example video decoder 300 that may perform video decoding in accordance with techniques of this disclosure. Figure 5 It is provided for the purpose of explanation and is not a limitation of the techniques extensively illustrated and described in this disclosure. For the purpose of explanation, this disclosure describes the video decoder 300 according to the techniques of VVC (ITU-T H.266, under development) and HEVC (ITU-T H.265). However, the techniques of this disclosure may be performed by video decoding devices configured for other video decoding standards.

[0145] exist Figure 5 In the example of , the video decoder 300 includes a coded picture buffer (CPB) memory 320, an entropy decoding unit 302, a prediction processing unit 304, an inverse quantization unit 306, an inverse transform processing unit 308, a reconstruction unit 310, a filter unit 312, and a decoded picture buffer (DPB) 134. Any or all of the CPB memory 320, the entropy decoding unit 302, the prediction processing unit 304, the inverse quantization unit 306, the inverse transform processing unit 308, the reconstruction unit 310, the filter unit 312, and the DPB 134 may be implemented in one or more processors or in processing circuits. For example, the units of the video decoder 300 may be implemented as one or more circuits or logic elements as part of a hardware circuit, or as part of a processor, ASIC, or FPGA. In addition, the video decoder 300 may include additional or alternative processors or processing circuits to perform these and other functions.

[0146] The prediction processing unit 304 includes a motion compensation unit 316 and an intra prediction unit 318. The prediction processing unit 304 may include additional units to perform predictions according to other prediction modes. As an example, the prediction processing unit 304 may include a palette unit, an intra block copy unit (which may form part of the motion compensation unit 316), an affine unit, a linear model (LM) unit, etc. In other examples, the video decoder 300 may include more, fewer, or different functional components.

[0147] When operating according to AV1, the compensation unit 316 may be configured to decode coding blocks of video data (e.g., both luma coding blocks and chroma coding blocks) using translational motion compensation, affine motion compensation, OBMC, and / or composite inter-intra prediction, as described above. The intra prediction unit 318 may be configured to decode coding blocks of video data (e.g., both luma coding blocks and chroma coding blocks) using directional intra prediction, non-directional intra prediction, recursive filter intra prediction, CFL, intra block copy (IBC), and / or palette mode, as described above.

[0148] CPB memory 320 may store video data, such as an encoded video bitstream, to be decoded by components of video decoder 300. For example, the video bitstream may be obtained from computer readable medium 110 ( Figure 1 ) obtains video data stored in CPB memory 320. CPB memory 320 may include a CPB that stores encoded video data (e.g., syntax elements) from an encoded video bitstream. In addition, CPB memory 320 may store video data other than syntax elements of decoded pictures, such as temporary data representing outputs from various units of video decoder 300. DPB 314 typically stores decoded pictures, and video decoder 300 may output decoded pictures and / or use decoded pictures as reference video data when decoding subsequent data or pictures of the encoded video bitstream. CPB data memory 320 and DPB 314 may be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), solid-state storage devices, or other types of memory devices. CPB memory 320 and DPB 314 may be provided by the same memory device or separate memory devices. In various examples, CPB memory 320 can be on-chip with other components of video decoder 300 , or off-chip relative to those components.

[0149] Additionally or alternatively, in some examples, video decoder 300 may retrieve the video from memory 120 ( Figure 1 ) to retrieve the decoded video data. That is, memory 120 may utilize CPB memory 320 to store data as discussed above. Likewise, when some or all of the functionality of video decoder 300 is implemented in software to be executed by processing circuitry of video decoder 300, memory 120 may store instructions to be executed by video decoder 300.

[0150] Shown in Figure 5 The various units shown in FIG. 3 are provided to help understand the operations performed by the video decoder 300. The units may be implemented as fixed function circuits, programmable circuits, or a combination thereof. Figure 4, fixed-function circuits refer to circuits that provide specific functions and are pre-set with respect to the operations that can be performed. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functions in the operations that can be performed. For example, a programmable circuit can execute software or firmware that causes the programmable circuit to operate in a manner defined by the instructions of the software or firmware. Fixed-function circuits can execute software instructions (e.g., receive parameters or output parameters), but the types of operations performed by fixed-function circuits are generally immutable. In some examples, one or more of the units may be different circuit blocks (fixed-function or programmable), and in some examples, one or more of the units may be integrated circuits.

[0151] The video decoder 300 may include an ALU, an EFU, a digital circuit, an analog circuit, and / or a programmable core formed by a programmable circuit. In an example where the operation of the video decoder 300 is performed by software executed on the programmable circuit, an on-chip or off-chip memory may store instructions (e.g., object code) of the software received and executed by the video decoder 300.

[0152] The entropy decoding unit 302 may receive the encoded video data from the CPB and entropy decode the video data to reproduce the syntax elements. The prediction processing unit 304, the inverse quantization unit 306, the inverse transform processing unit 308, the reconstruction unit 310, and the filter unit 312 may generate decoded video data based on the syntax elements extracted from the bitstream.

[0153] Generally, the video decoder 300 reconstructs a picture block by block. The video decoder 300 may perform a reconstruction operation on each block individually (wherein a block currently being reconstructed (ie, decoded) may be referred to as a "current block").

[0154] The entropy decoding unit 302 may entropy decode syntax elements defining the quantized transform coefficients of the quantized transform coefficient block and transform information, such as a quantization parameter (QP) and / or a transform mode indication. The inverse quantization unit 306 may use the QP associated with the quantized transform coefficient block to determine a degree of quantization and, likewise, a degree of inverse quantization to be applied by the inverse quantization unit 306. The inverse quantization unit 306 may, for example, perform a bitwise left shift operation to inverse quantize the quantized transform coefficients. The inverse quantization unit 306 may thereby form a transform coefficient block including the transform coefficients.

[0155] After the inverse quantization unit 306 forms the transform coefficient block, the inverse transform processing unit 308 may apply one or more inverse transforms to the transform coefficient block to generate a residual block associated with the current block. For example, the inverse transform processing unit 308 may apply an inverse DCT, an inverse integer transform, an inverse Karhunen-Loeve transform (KLT), an inverse rotational transform, an inverse directional transform, or another inverse transform to the transform coefficient block.

[0156] In addition, prediction processing unit 304 generates a prediction block based on the prediction information syntax elements entropy decoded by entropy decoding unit 302. For example, if the prediction information syntax elements indicate that the current block is inter-predicted, motion compensation unit 316 can generate the prediction block. In this case, the prediction information syntax elements can indicate a reference picture in DPB 314 from which to retrieve the reference block, and a motion vector that identifies the position of the reference block in the reference picture relative to the position of the current block in the current picture. Motion compensation unit 316 can generally generate a prediction block in a manner substantially similar to that described with respect to motion compensation unit 224 ( Figure 4 ) in the manner described above to perform the inter-frame prediction process.

[0157] As another example, if the prediction information syntax element indicates that the current block is intra-predicted, the intra-prediction unit 318 may generate a prediction block according to the intra-prediction mode indicated by the prediction information syntax element. Similarly, the intra-prediction unit 318 may generally generate a prediction block in the same manner as described with respect to the intra-prediction unit 226 ( Figure 4 The intra prediction process is performed in a manner substantially similar to that described in the foregoing description. The intra prediction unit 318 may retrieve data of neighboring samples of the current block from the DPB 314.

[0158] The reconstruction unit 310 may reconstruct the current block using the prediction block and the residual block. For example, the reconstruction unit 310 may add samples of the residual block to corresponding samples of the prediction block to reconstruct the current block.

[0159] The filter unit 312 may perform one or more filter operations on the reconstructed block. For example, the filter unit 312 may perform a deblocking operation to reduce blocking artifacts along the edges of the reconstructed block. The operations of the filter unit 312 may not necessarily be performed in all examples.

[0160] The video decoder 300 may store the reconstructed block in the DPB 314. For example, in examples where the operation of the filter unit 312 is not performed, the reconstruction unit 310 may store the reconstructed block in the DPB 314. In examples where the operation of the filter unit 312 is performed, the filter unit 312 may store the filtered reconstructed block in the DPB 314. As discussed above, the DPB 314 may provide reference information (such as samples of the current picture for intra-frame prediction and previously decoded pictures for subsequent motion compensation) to the prediction processing unit 304. In addition, the video decoder 300 may output a decoded picture (e.g., a decoded video) from the DPB 314 for use in, for example, Figure 1 Subsequent presentation on a display device such as display device 118.

[0161] Figure 6 is a flow chart illustrating an example method for encoding picture and histogram data according to techniques of this disclosure. Figure 6 The method is explained with respect to the video encoding system 140, including Figure 2 HDR statistics engine 132. However, other devices or systems may be configured to perform this method or similar methods.

[0162] Initially, the video format converter 130 may convert one or more images to a linear light format (350). The HDR statistics engine 132 may then determine a range of values ​​for a bin of a histogram to be calculated for the one or more images (352). For example, the range may correspond to Figure 4 The stepwise function has 156 steps.

[0163] The HDR statistics engine 132 may calculate a histogram, including incrementing a bin for each sample of the image in the linear light format (354). For example, as described above, the HDR statistics engine 132 may calculate the maximum value of the samples of the red, green, and blue components of the image and use the maximum value to determine a corresponding bin of the histogram, and then increment the value of the bin by 1. The HDR statistics engine 132 may perform this operation for each sample of each image.

[0164] The HDR statistics engine 132 may also determine a set of possible codewords for the bin values ​​of the histogram (356). As described above, the set of possible codewords for the histogram calculated from the LL format image may be selected from the PQ10 format codewords, e.g., as a subset of the PQ10 format codewords. After calculating the histogram and determining the set of possible codewords, the HDR statistics engine 132 may determine a codeword for each bin value of the histogram (358).

[0165] Video encoder 200 may also encode the image (360), for example, as described above with respect to Figure 1 and Figure 4 As explained. For example, the video encoder 200 may divide the image into blocks and predictively encode each block using, for example, intra-frame or inter-frame prediction. After predicting the block, the video encoder 200 may calculate a residual block representing the difference between the original block and the predicted block. The video encoder 200 may further encode the residual block, including transforming the residual block into a transform domain block and quantizing the transform coefficients. The video encoder 200 may then entropy encode the syntax element values ​​for the quantized transform coefficients and prediction information.

[0166] HDR statistics engine 132 may also encode codewords for the bin values ​​of the histogram (362). For example, HDR statistics engine 132 may encode codewords similar to Figure 4 The HDR statistics engine 132 may entropy encode the codeword in a manner of the entropy encoding unit 220. Optionally, the HDR statistics engine 132 may pass the codeword to the video encoder 200 to be entropy encoded by the entropy encoding unit 220. Finally, the encapsulation unit 136 may form a bitstream 138 including the encoded image and the codeword (364).

[0167] In this way, Figure 6 The method represents an example of a method for encoding high dynamic range (HDR) video data, including: calculating a histogram for an image of the video data, the image being represented in a linear light format; encoding a value of the histogram for the image represented in the linear light format; and encoding the image.

[0168] Fig. 7A and 7B is a conceptual diagram showing an example of HDR and WCG representation conversion. In particular, Fig. 7A An example encoder-side representation conversion process 400 is depicted, and Figure 7B An example decoder-side representation conversion process is depicted. HDR / WCG is typically acquired in linear light and stored with high precision per component (e.g. RGB 16-bit fixed or even floating point representation) and in a very wide color space (BT.2020). This representation targets high precision and is (almost) mathematically lossless. However, this format has many redundancies and is not optimal for compression purposes. Lower precision formats with HVS-based assumptions are typically used for state-of-the-art video applications.

[0169] Typically, HDR video data format conversion for compression purposes includes three main elements: a non-linear transfer function (TF) for dynamic range compression (e.g., BT.2100PQ or BT.2100HLG), color conversion to a more compact or robust color space (e.g., P3), and floating point to integer representation conversion / quantization (e.g., 10-bit fixed point representation).

[0170] exist Fig. 7A In the example of , the video data is initially presented in a linear light RGB format 402. As described above, the HDR / WCG conversion process typically includes three stages. Fig. 7A In an example of , the three stages include an optoelectronic transfer function (OETF) stage 404, a color conversion stage 406, and a quantization stage 408. The three stages process video data 402 in a linear light RGB format to produce data 410 represented by 10 bits. In an example, the video data 402 in a linear light RGB format can have a high dynamic range. This high dynamic range of the input RGB data can be represented using linear and floating point representations. The nonlinear OETF stage 404 can compress the high dynamic range, for example, using a QP TF defined in BT.2100 or SMPTE-2084. The color conversion stage 406 can convert the RGB representation to a target color space, such as YCbCr. The quantization stage 408 can quantize the sample values ​​to achieve an integer representation.

[0171] exist Figure 7B In the example of , video data 422 represented in 10 bits is initially received. The three stages in this example include an inverse quantization stage 424, an inverse color conversion stage 426, and an inverse OETF stage 428. These three stages process the video data 422 represented in 10 bits to reproduce video data 430 in a linear light RGB format.

[0172] The order of these elements is given as an example and may differ in actual applications. For example, color conversion may precede the TF module. In some examples, additional processing may be performed, such as spatial subsampling applied to color components.

[0173] Figure 8 is a schematic diagram showing various examples of transfer functions. In particular, Figure 8An example SDR transfer function 442 and an example HDR transfer function 444 are described. A transfer function (TF) is applied to video data to compress the dynamic range of the video data and make it possible to represent the video data with a limited number of bits. The function is typically a one-dimensional (1D) nonlinear function that reflects the inverse of the electro-optical transfer function (EOTF) of the end-user display, as specified in Rec.709 for SDR. For HDR signals, one of the most popular formats is defined by an optoelectronic transfer function (OETF), which approximates the perception of the human visual system (HVS) to brightness changes, such as the PQ TF specified for HDR in BT.2100 (also known as SMPTE-2084). The inverse of the OETF is the EOTF (electro-optical transfer function), which maps the code level back to brightness. This nonlinearity is the basis for PQ10, HDR10, Dolby Vision, and HDR10+ video formats.

[0174] The specification of the PQ transfer function is defined in ST 2084 as EOTF. Its inverse, OETF, is applied to the normalized linear R, G, B values, which results in a nonlinear representation of R'G'B'. PQ defines normalization by NORM=10000, which is associated with a peak brightness of 10000 nits (cd / m2). For the purposes of this disclosure, this nonlinearity is referred to as PQ_OETF. The nonlinearity can be defined as follows:

[0175] ●R'=PQ_OETF(max(0,min(R / NORM,1)))

[0176] ●G'=PQ_OETF(max(0,min(G / NORM,1)))

[0177] ●B'=PQ_OETF(max(0,min(B / NORM,1)))

[0178] ●Among them

[0179] ○

[0180] ○

[0181] ○

[0182] ○

[0183] ○

[0184] Fig. 9 is a schematic diagram 450 illustrating an example PQ OETF. Fig. 9450 shows a visualization of a PQ OETF using input values ​​(linear color values) normalized to a range of 0..1 and normalized output values ​​(non-linear color values). As can be seen from the diagram 450, a relatively small number of low luminance values ​​of the dynamic range of the input signal are converted to 50% of the dynamic range of the output signal.

[0185] The PQ EOTF specified in ST2084 can be defined as follows:

[0186] R = 10000 * PQ_EOTF (R')

[0187] ●G=10000*PQ_EOTF(G')

[0188] B = 10000 * PQ_EOTF (B')

[0189] ●Among them

[0190] ○

[0191] ○

[0192] ○

[0193] ○

[0194] ○

[0195] With floating point precision, the sequential application of EOTF and OETF provides perfect reconstruction without errors. However, this representation is not optimal for streaming or broadcast services. A more compact representation with fixed point bit precision of non-linear R'G'B' data is described below.

[0196] RGB data is often used as input because it is generated from image capture sensors. However, the RGB color space has high redundancy between its components and is not optimal for compact representation. In order to achieve a more compact and robust representation, the RGB components are often converted to a more uncorrelated color space that is more suitable for compression, such as YCbCr. This color space separates brightness in the form of luminance and color information in different uncorrelated chrominance components, such as red and blue hue components.

[0197] For modern video decoding systems, the color space commonly used is ITU-R BT.709 or YCbCr specified in ITU-R BT.709. The YCbCr color space in the BT.709 standard specifies the following conversion process from R'G'B' to Y'CbCr (non-constant brightness representation):

[0198] ●Y'=0.2126*R'+0.7152*G'+0.0722*B'

[0199] ●

[0200] ●

[0201] The above method can also be implemented using the following approximate conversion to avoid division for the Cb and Cr components:

[0202] ●Y'=0.212600*R'+0.715200*G'+0.072200*B'

[0203] ●Cb=-0.114572*R'-0.385428*G'+0.500000*B' (4)

[0204] ●Cr=0.500000*R'-0.454153*G'-0.045847*B'

[0205] The ITU-R BT.2020 standard specifies the following conversion process from R'G'B' to Y'CbCr (non-constant luminance representation):

[0206] ●Y'=0.2627*R'+0.6780*G'+0.0593*B'

[0207] ●

[0208] ●

[0209] The above method can also be implemented using the following approximate conversion to avoid division for the Cb and Cr components:

[0210] ●Y'=0.262700*R'+0.678000*G'+0.059300*B'

[0211] ●Cb=-0.139630*R'-0.360370*G'+0.500000*B' (6)

[0212] ●Cr=0.500000*R'-0.459786*G'-0.040214*B'

[0213] Both color spaces can be kept normalized. So for input values ​​normalized in the range 0...1, the resulting value will be mapped to the range 0..1. Usually, color transforms implemented in floating point precision provide perfect reconstruction, so the process may be lossless.

[0214] The color transformation and nonlinear transform function processing stages described above can be implemented using floating point precision representation. Therefore, these processes can be implemented losslessly. However, for most consumer electronics applications, this type of precision may be considered redundant and expensive. For such services, the input data in the target color space can be converted to the target bit depth fixed point precision. Some studies have shown that 10 to 12 bits of precision combined with PQTF are sufficient to provide 16 stops of HDR data with distortion below just perceptible differences. Data represented with 10 bits of precision can be further decoded using most state-of-the-art video decoding solutions. This conversion process includes signal quantization and is an element of lossy decoding and a source of inaccuracies introduced to the converted data.

[0215] An example of such quantization applied to codewords in the target color space is described below. In this example, YCbCr is shown. The input value YCbCr represented in floating point precision can be converted to a fixed bit depth signal BitDepthY for Y values, and BitDepthC for chrominance values ​​(Cb, Cr):

[0216] ●D Y′ =Clip1 Y (Round((1<<(BitDepth Y -8))*(219*Y′+16)))

[0217] ●D Cb =Clip1 C (Round((1<<(BitDepth C -8))*(224*Cb+128)))(7)

[0218] ●D Cr =Clip1 C (Round((1<<(BitDepth C -8))*(224*Cr+128)))

[0219] ●Among which:

[0220] ○Round(x)=Sign(x)*Floor(Abs(x)+0.5)

[0221] ○ If x < 0, Sign(x) = -1; if x = 0, Sign(x) = 0; if x > 0, Sign(x) = 1

[0222] ○ Floor(x) is the largest integer less than or equal to x

[0223] ○ If x >= 0, Abs(x) = x; if x < 0, Abs(x) = -x

[0224] ○ Clip1Y(x) = Clip3(0, (1 << BitDepthY) - 1, x)

[0225] ○ Clip1C(x) = Clip3(0, (1 << BitDepthC) - 1, x)

[0226] ○ If z < x, Clip3(x, y, z) = x; if z > y, Clip3(x, y, z) = y; otherwise Clip3(x, y, z) = z

[0227] There are several systems that use dynamic metadata based on HDR10 (PQ). Most of the HDR dynamic metadata currently in use is specified in the SMPTE ST 2094 document suite. These documents specify four different color volume transformations (mapping functions). Each color volume transformation (CVT) is published as a separate ST 2094-xx document, where xx equals -10 for CVT 1, xx equals -20 for CVT 2, xx equals -30 for CVT 3, and xx equals -40 for CVT 4. Details of the HDR parameters for CVT 4 are provided below, which are defined in ST 2094-40 (SMPTE: “Dynamic Metadata for Color Volume Transformation – Application #4,”, Doc. ST 2094-40 (2016), Society of Motion Picture and Television Engineer, White Plains, NY).

[0228] The metadata in SMPTE ST 2094-40 can be divided into three groups of syntax elements. The first group is for identification purposes, the second group describes the statistical characteristics of the video signal, and the third group describes the base OOTF (Optical-optical Transfer Function) for the producer-specified target peak luminance. This base OOTF can be used to construct a guided OOTF, which defines the adaptation to the presentation display.

[0229] The metadata elements associated with the identifier are not used for image processing. These elements include, as defined in SMPTE ST 2094-40:

[0230] ● itu_t_t35_country_code: itu_t_t35_country_code shall be a byte with the value as the country code as specified by Rec. ITU-T T.35 Annex A

[59] . The value shall be 0xB5.

[0231] ●itu_t_t35_terminal_provider_code:itu_t_t35_terminal_provider_code shall be a fixed 16-bit field. The value shall be 0x003C.

[0232] ●itu_t_t35_terminal_provider_oriented_code: itu_t_t35_terminal_provider_oriented_code shall be a 16-bit code. The value shall be as specified in Table 149 in SMPTE ST 2094-40.

[0233] ●CVT_identifier (also called "application_identifier"): application_identifier identifies the application (CVT) in the ST-2094 suite and its definition document. application_identifier can be set to 4.

[0234] Metadata elements associated with statistical characteristics include:

[0235] maxscl[w][i]: Specifies the maximum value of the i-th color component of the linearized RGB values ​​in the w-th processing window in the scene. The value of maxscl[w][i] should be in the range of 0 to 1 (inclusive) and is a multiple of 0.00001. maxscl[w][0], maxscl[w][1], and maxscl[w][2] correspond to the R, G, and B color components, respectively.

[0236] ●average_maxrgb[w]: specifies the average of the linearized maxRGB values ​​in the wth processing window in the scene. The value of average_maxrgb[w] should be in the range of 0 to 1 (inclusive) and be a multiple of 0.00001.

[0237] num_distributions[w]

[0238] num_distributions_maxrgb_percentiles[w]: indicates the number of linearized maxRGB values ​​at a given percentile in the wth processing window in the scene. The maximum value of num_distribution_maxrgb_percentiles[w] should be 15.

[0239] distribution_index[w][i]

[0240] distribution_values[w][i]

[0241] distribution_maxrgb_percentages[w][i]: Specifies the integer percentage value corresponding to the i-th percentile of linearized RGB values ​​in the w-th processing window in the scene. The value of distribution_maxrgb_percentages[w][i] should be in the range of 0 to 100 (inclusive).

[0242] distribution_maxrgb_percentiles[w][i]: specifies the linearized maxRGB value at the i-th percentile in the w-th processing window in the scene. The value of distribution_maxrgb_percentiles[w][i] should be in the range of 0 to 1 (inclusive) and be a multiple of 0.00001.

[0243] fraction_bright_pixels[w]: specifies the fraction of the selected pixels in the image that contains the brightest pixel in the scene. The value of fraction_bright_pixels[w] should be in the range of 0 to 1 (inclusive) and be a multiple of 0.001.

[0244] The metadata elements associated with the base OOTF include:

[0245] ● targeted_system_display_maximum_luminance: Specifies the nominal maximum display luminance of the targeted system display in units of 1 candela per square meter. The value of targeted_system_display_maximum_luminance should be in the range of 0 to 10000 (inclusive).

[0246] tone_mapping_flag[w] equal to 1 indicates the presence of metadata for the tone mapping function in the wth processing window.

[0247] knee_point_x[w] specifies the x-coordinate of the separation point between the linear and curve parts of the tone mapping function. The value of knee_point_x[w][i] should be in the range of 0 to 1, excluding 0, and be a multiple of 1 / 4095.

[0248] knee_point_y[w] specifies the y coordinate of the separation point between the linear and curve parts of the tone mapping function. The value of knee_point_y[w][i] should be in the range of 0 to 1, excluding 0, and be a multiple of 1 / 4095.

[0249] num_bezier_curve_anchors[w] indicates the number of intermediate anchor point parameters of the tone mapping function in the wth processing window. The maximum value of num_bezier_curve_anchors[w] should be 15.

[0250] bezier_curve_anchors[w][i] specifies the i-th intermediate anchor point parameters of the tone mapping function in the w-th processing window in the scene. The value of bezier_curve_anchors[w][i] should be in the range of 0 to 1 (inclusive) and be a multiple of 1 / 1023.

[0251] According to the techniques of the present disclosure, these various metadata elements may be referred to as parameters of the histogram, any or all of which may be extracted and encoded by the HDR statistics engine 132 ( Figure 2 ).

[0252] The HDR10+ system specification is based on ST 2094-40 and further constrains the values ​​of syntax elements (HDR10+ TECHNICAL SPECIFICATION Part 1: Metadata and Devices, version 1.6, HDR10+ Technologies LLC, Dec. 2021). Metadata and Devices, version 1.6, HDR10+ Technologies LLC, December 2021). The following constraints are imposed on syntax elements that describe video statistical features:

[0253] ●distribution_index[w][i] should be set as shown in Table 2.

[0254] ● distribution_values[w][i] shall be as shown in Table 2. distribution_values[w][i] shall be in the range of 0–100 or 255 (i=2). Otherwise (i=0,1,3...8), distribution_values[w][i] shall be in the range of 0x00000–0x186A0. distribution_values[w][i] for (i=0-1,3-8) shall be multiplied by 10. When processing HDR10+ metadata, the value of distribution_values[w][i] for (i=0-1,3-8) shall be divided by 10.

[0255] Table 2

[0256]

[0257]

[0258] Fig.10 is an example diagram 454 of deriving DistributionY99 and DistributionY100 based on percentiles of a histogram. Further parameters include DistributionY99 and DistributionY100nits, which can be calculated based on Fig.10 The DistributionY99 value is the maximum of the 99YF values ​​within the video scene. The 99YF value is derived as the value below which 99.99% of the linearized luminance values ​​for each frame fall. The DistributionY100nits value is the average of the DPY100F values ​​within the scene. DistributionY100F is derived as the percentage, between 0 and 100, of the cumulative number of pixels with luminance less than or equal to 100 nits for each frame.

[0259] Fig.11 is a block diagram showing an example HDR10+ capable live encoder framework. In this example, Fig.11A video source 460 and an HDR encoder 470 are depicted. The HDR encoder 470 includes a video encoder 472, a percentile estimator 474, a tone mapping curve estimator 476, and an HDR10+ metadata injector 478. In general, for HDR10+, statistical characteristics of video data generated or captured by the video source 460 can be determined during content production and / or by the HDR encoder 470. For example, the HDR encoder 470 can generate statistical measurements of video data samples through a mathematical process and represent the resulting statistical measurements with linearized RGB values. The tone mapping curve estimator 476 can then generate a tone mapping curve for representing the calculated statistical measurements, and both the statistical measurements and the tone mapping curve can be provided to the HDR10+ metadata injector 478. The HDR10+ metadata injector 478 can then generate metadata parameters representing the statistical measurements and the tone mapping curve, and provide the metadata parameters to the video encoder 472, which can encode the parameters along with the video data. Ultimately, the video encoder 472 may generate a bitstream 480 including encoded video data and encoded parameters.

[0260] In the case of real-time encoding, the percentile estimator 474 can generate HDR10+ statistical features of the video data received from the video source 460. Such statistical features can be represented by PQ10, including Y'CbCr 10-bit 4:2:0, BT.2100 PQEOTF and / or BT.2020 color containers. SMPTE: "Dynamic Metadata for Color Volume Transformation–Application #4," Doc. ST 2094-40 (2016), Society of Motion Picture and Television Engineer, White Plains, NY, static information such as maximum and / or average, and histogram percentiles calculated in non-linear Y'CbCr, allowing implementation with low complexity and relatively low memory requirements in fixed-point arithmetic. The memory required to perform histogram derivation for 8k UHD content may be limited to 1024 entries with 25-bit data. The conversion of the derived statistical parameters (to a linearized RGB representation) can be done according to formula (2) above, with the accuracy required for the HDR10+ syntax (accuracy up to 0.1 cd / m2).

[0261] However, in some applications, there may be a requirement to estimate HDR10+ statistical parameters in a color space different from the source color space. An example of such a requirement could be to calculate statistical properties in the P3-D65 color space, while the source is in BT.2100 (BT.2020 color space).

[0262] The present disclosure recognizes that, as an alternative, the processing may additionally include the application of a PQ EOTF, RGB BT.2020 to RGB P3 color space conversion. The latter may be implemented in a linear RGB representation and may therefore require >16-bit or even floating point representation. After this, the video data statistics may be estimated in a linear RGB domain (alternative implementation 1) or a non-linear RGB representation (alternative implementation 2), which would require the application of a PQ OETF.

[0263] Each of these alternative implementations may introduce complexities compared to the techniques of the present disclosure, as described below.

[0264] For alternative implementation 1, in order to achieve the accuracy specified by HDR10+ (0.1cd / m2), the calculation of the histogram in the linear RGB domain requires a memory size allocation of at least 2^17 entries out of 25-bit entries. This is a significant memory increase compared to the derivation of the histogram in the non-linear R'G'B' domain (2^10 entries out of 25-bit entries). However, the present disclosure recognizes that an advantage of this approach is that the bins of the histogram in LL can be used directly to generate HDR10+ syntax elements without the need for EOTF conversion.

[0265] For alternative implementation 2, the application of PQ OETF on linear RGB requires a LUT size of 2^16 entries, where the following statistical derivation is used in the linear domain and the PRE EOTF is applied to generate the HDR10+ parameters. In addition, implementing the color space conversion using 16-bit fixed-point arithmetic and supporting a dynamic range of 0...2^10cd / m2 will result in a quantization error of up to 0.17cd / m2, which is higher than the accuracy constraint of HDR10+. In addition, due to the application of PQ OETF, this quantization error in the linear light representation may cause the quantization error in the PQ domain to increase to up to 30% of the value. In order to reduce this quantization error, the bit depth needs to be increased to >18 bits per sample. Another disadvantage of this approach is that the histogram parameters (percentiles) derived from the histogram need to be converted to LL representation by applying a LUT.

[0266] The case of a real-time encoder framework is particularly relevant to user-generated or user-captured content applications, which are typically deployed on computing platforms that are constrained by cost, memory, computational complexity, and power consumption, such as mobile devices. Therefore, the techniques of the present disclosure may be particularly useful on mobile devices for providing HDR / WCG video data, as these techniques can be implemented in low-complexity, low-memory environments, such as mobile devices.

[0267] Fig.12is a conceptual diagram illustrating an example process 500 for deriving HDR10+ statistical features of video data according to techniques of the present disclosure. In this example, process 500 begins by receiving video data 502 in Y'CbCr 10b PQ, BT2020 format. A color conversion process 504 converts the Y'CbCr format to an R'G'B' format. A PQ EOTF stage 506 then applies a transform function to the R'G'B' format data. A quantization stage 508 then converts the RGB BT.2020 format data to RGB P3 quantized values. A histogram calculation stage 510 can then derive a histogram index using a static array lookup table (LUT) 512. A linear light histogram (histogramLL) can then be represented in a 1024-entry LUT 514.

[0268] The techniques of this disclosure can perform low-complexity, low-memory estimation of statistical properties (histograms) of video data represented in the linear light RGB domain. The histogram calculation accuracy may be sufficient for HDR10+ parameter derivation (up to .1 cd / m2) without significantly increasing the memory requirements for computational complexity.

[0269] Process 500 may be performed by, for example Figure 2 The HDR video encoding system 140 is executed. The HDR statistics engine 132 can perform histogram estimation in the linear light (LL) RGB domain, so that histogramLL[idx]++,idx=function(R,G,B). In some examples, a single histogramLL for RGB data is constructed on a single statistical feature, for example, on the maximum value of 3 samples, where maxRGB=max(R,G,B); histogramLL[idx]++,idx=function(maxRGB).

[0270] The bins of the histogram LL may be aligned / associated with linear light values ​​represented by codewords in a non-linear RGB representation (R'G'B') to a certain precision. In some examples, the non-linearity may be BT.2100 / PQ. In some examples, the precision is 0.1 cd / m2 precision, which is aligned with the HDR10+ specification. Thus, histogramLL[idx], idx = PQ_EOTF[maxRGB], where maxRGB is the maximum value of the R', G', and B' codewords in the PQ domain.

[0271] The derivation of the histogramLL index can be performed by mapping the LL RGB sample values ​​to the histogram index through a non-linear or piecewise linear mapping process. In one example, the index derivation is performed by comparing the LL RGB values ​​with a threshold array (ArrayTs), which is represented by LL intensity values ​​and sorted in ascending order, where the maximum exceeded threshold provides the index of the histogram bin. In some examples, the index derivation is performed by searching the threshold array, where the threshold provides the minimum distance (e.g., SAD or MSE) from the LL RGB value.

[0272] In some examples, where the nonlinear R'G'B' samples are represented by 10-bit codewords, the number of thresholds in ArrayTs used for comparison may be less than or equal to 1024. The PQ codeword space in the LL representation may be redundant for the target accuracy (e.g., 0.1 cd / m2), and the thresholds in ArrayTs may be quantized to a number of unique levels less than 1024.

[0273] In some examples, the search for or comparison to a threshold with a minimum distance can be performed using a hybrid approach, where a portion of the index (indx1) or initial offset within ArrayTs is derived analytically from the RGB sample values, e.g., as indx1=log2(maxRGB), and another portion of the index (indx2) is searched within a subrange specified by indx1. This search can be implemented using a reduced number of comparators (rather than 10 for a LUT size of 1024), at the expense of an increase in the storage of the static LUT by 15 entries, marking the ArrayTs locations where the threshold is equal to a power of 2.

[0274] In some examples, the histogram index may be analytically derived from the RGB values.

[0275] In some examples, the derivation of parameters aligned to HDR10+ (i.e. histogram percentiles in the LL representation) can be done by accessing the provided ArrayTs LUT instead of the PQ EOTF application.

[0276] Various example implementations consistent with the various techniques of this disclosure are described below.

[0277] In one example implementation, the source video data is represented in Y'CbCr 4:2:10 10-bit, PQ, BT.2020. The target color space used for the calculation is HDR10+ with statistical properties RGB P3-D65. The output is HDR10+ percentiles as shown in Table 2 above.

[0278] A static threshold LUT ArrayTs of size 1024 is predetermined and provided as side information. The derivation can be performed as follows:

[0279] ●Variable maxLL set equal to 10000cd / m2

[0280] The space of 10-bit codewords allowed in the PQ RGB domain is cwPQ, where cwPQ is in the range of 0 to 1023 (inclusive):

[0281] ○cw10=0:1023

[0282] The space of allowed normalized PQ codewords (cw10n) is normalized to the range 0 to 1, consisting of dividing each element by 1024:

[0283] ○cw10n=cw10. / 2 10

[0284] The space of allowed normalized LL values ​​(CWn) and LL (in cd / m2(CW)) covered by the PQ codeword is derived as follows, where the function is applied to each entry of the array separately:

[0285] ○CWn=PQ_EOTF(cw10n)

[0286] ○CW=WCn*maxLL

[0287] The spatial derivation of the allowed LL values ​​(which are quantized to the HDR10+ required accuracy of 0.1 cd / m2) is as follows, where CW2 is the LL value in cd / m2 and CW2n is a normalized value in the range 0..1:

[0288] ○CW2=round(CW*10.0) / 10 / 0

[0289] ○CW2n=WC2 / maxRGB

[0290] The static LUT (e.g. 16-bit) of the thresholds in the target fixed-point arithmetic implementation is derived as follows:

[0291] ○numUpBits=16

[0292] ○qScaler=2^numUpBits

[0293] ○ArrayTs=round(CW2n*qScaler)

[0294] Each entry of ArrayT provides a LL value (threshold) associated with an allowed PQ codeword, represented in 10 bits with an accuracy up to 0.1 cd / m2.

[0295] To derive the histogram index from ArrayTs, the HDR video encoding system 140 may be configured to perform the following operations in this example:

[0296] Input:

[0297] ○Input samples are represented in linear light R, G, B, with 16-bit fixed-point representation.

[0298] ● A variable maxRGB for each pixel is determined as the maximum of the three components:

[0299] ○maxRGB=max(R,G,B)

[0300] ● In some examples, a lower bin boundary threshold may be used. The histogram index may be determined as the maxRGB value that is less than or equal to the threshold ArraysTs. The smallest ArrayTs index for which the threshold entry is greater than or equal to the maxRGB value provides the index for the histogram update:

[0301] ○idxHist=min(find(maxRGB<=ArrayTs))

[0302] ○histogramLL(idxHist)++

[0303] ● In some examples, an upper bin boundary threshold may be used. The histogram index bin may be determined as the maxRGB value greater than or equal to the threshold ArraysTs. The last found ArrayTs entry greater than or equal to the maxRGB value provides the index for the histogram update.

[0304] ○idxHist=max(find(maxRGB>=ArrayTs))

[0305] ○histogramLL(idxHist)++

[0306] ●In some examples, a linear interpolation process may be used to determine histogram index positions between a lower threshold and an upper threshold.

[0307] ○idxHist1=min(find(value<=pqThresholdsLL))

[0308] ○idxHist2=max(find(value>=pqThresholdsLL))

[0309] ○if(idxHist1!=idxHist2)

[0310] ■idxHist=round((idxHist1+idxHist2) / 2)

[0311] ○else

[0312] idxHist=idxHist1

[0313] In some examples, a binary search on a sorted array may be used. The worst case complexity of such a search for a LUT size of 1024 is 10 consecutive comparisons.

[0314] In some examples, a hybrid search approach on a sorted array may be used. The initial offset for the search may be derived from an additional LUT (OffsetLUT) used to specify the offset of a location to ArrayTs. The access index to the OffsetLUT is derived as a log2(maxRGB) value. For a LUT of size 1024, this mechanism will allow reducing the number of comparisons for a binary search from 10 to 7.

[0315] In some examples, static LUTArrayTs can be provided as side information. The calculation of the LL RGB thresholds can be expressed in 16-bit fixed point representation. In one example:

[0316] ArrayTs={0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,2,2,2,2,2,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,4,4,4,4,4,4,4,4,5,5,5,5,5,5,5,5,5,5,5,5,5,6,6,6,6,6,7,7,7,7,7,7,7,7,7,7,8,8,8,8,8,9,9,9,9,9,9,9,9,10,10,10,10,10,10,10,10,11,11,11,12,12,12,12,12,12,12,13,13,13,14,14,14,14,14,15,15,15,16,16,16,16,16,17,17,17,18,18,18,18,18,19,19,20,20,20,20,21,21,22,22,22,22,23,23,24,24,24,24,25,25,26,26,26,27,27,28,28,28,29,29,29,29,30,31,31,31,32,32,33,33,34,34,35,35,35,36,37,37,37,38,39,39,40,40,41,41,42,43,43,44,44,45,45,46,47,47,48,48,49,50,50,51,52,52,53,54,54,55,56,56,57,58,58,60,60,61,62,62,63,64,65,66,66,68,68,69,70,71,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,96,96,98,99,100,101,102,104,105,106,107,108,109,111,112,113,115,116,117,119,120,122,123,125,126,127,128,130,132,133,135,136,138,140,141,142,144,145,147,149,151,152,154,156,157,159,161,163,164,166,168,170,172,174,176,178,180,182,184,185,187,189,192,194,196,198,201,203,204,207,209,211,214,216,218,220,223,225,227,230,233,235,237,240,242,245,248,250,253,256,258,261,264,267,269,273,275,278,281,284,287,290,293,296,299,302,305,309,312,315,319,322,325,328,332,335,338,342,345,349,353,357,360,364,367,371,375,379,383,387,391,395,398,402,406,410,415,419,423,427,432,436,440,445,450,454,458,463,468,473,477,482,487,492,496,501,507,511,516,522,527,532,537,543,548,553,559,564,570,575,581,587,593,598,604,611,617,623,628,635,641,647,653,660,667,673,680,686,693,699,706,713,720,727,734,741,748,756,763,771,778,786,793,801,809,817,824,832,840,848,857,864,873,881,890,898,907,916,924,933,942,951,960,969,978,988,997,1007,1016,1026,1035,1045,1056,1066,1075,1086,1096,1107,1117,1128,1138,1149,1160,1171,1182,1193,1205,1216,1227,1239,1250,1262,1274,1286,1298,1310,1323,1335,1347,1361,1373,1386,1399,1412,1425,1439,1452,1465,1479,1493,1507,1521,1535,1549,1564,1579,1593,1608,1623,1638,1653,1669,1684,1700,1716,1731,1748,1764,1780,1796,1813,1830,1847,1864,1882,1899,1916,1934,1952,1970,1988,2007,2025,2043,2062,2081,2100,2120,2140,2159,2179,2199,2219,2240,2260,2281,2302,2323,2344,2366,2387,2410,2432,2454,2477,2499,2522,2545,2568,2592,2616,2640,2664,2688,2713,2737,2762,2788,2813,2839,2865,2891,2917,2944,2971,2998,3025,3053,3080,3108,3137,3165,3194,3223,3253,3282,3312,3342,3372,3403,3434,3465,3496,3528,3561,3593,3625,3658,3692,3725,3758,3793,3827,3861,3897,3932,3968,4004,4040,4076,4113,4150,4188,4226,4264,4302,4342,4381,4420,4460,4501,4542,4582,4624,4666,4707,4750,4793,4837,4880,4924,4968,5014,5059,5104,5150,5197,5244,5291,5339,5386,5435,5484,5534,5584,5634,5685,5736,5787,5839,5892,5945,5999,6053,6107,6162,6218,6274,6330,6387,6445,6502,6561,6620,6680,6740,6801,6862,6923,6985,7048,7112,7176,7240,7305,7371,7438,7505,7572,7640,7709,7778,7848,7919,7989,8062,8134,8207,8281,8355,8431,8507,8583,8660,8738,8817,8896,8976,9056,9138,9220,9303,9386,9471,9556,9642,9728,9816,9904,9994,10083,10174,10266,10357,10451,10545,10640,10735,10832,10929,11028,11127,11227,11328,11429,11532,11637,11741,11847,11953,12061,12169,12279,12390,12501,12614,12727,12842,12957,13074,13192,13310,13430,13552,13673,13797,13921,14046,14173,14301,14430,14559,14691,14824,14957,15092,15228,15366,15505,15644,15785,15928,16071,16216,16363,16510,16660,16810,16962,17115,17270,17426,17583,17743,17902,18064,18228,18393,18559,18727,18896,19067,19240,19414,19589,19767,19946,20127,20309,20493,20679,20866,21055,21246,21439,21633,21829,22027,22227,22429,22632,22838,23045,23255,23466,23679,23894,24111,24330,24552,24775,25000,25227,25457,25689,25923,26159,26397,26638,26880,27125,27372,27622,27874,28128,28384,28644,28905,29169,29435,29704,29976,30249,30525,30805,31086,31371,31658,31947,32240,32535,32833,33133,33437,33743,34053,34364,34680,34998,35319,35643,35971,36301,36634,36971,37310,37654,38000,38349,38702,39058,39418,39781,40147,40517,40891,41267,41648,42032,42420,42811,43207,43606,44009,44415,44826,45240,45658,46081,46508,46938,47373,47811,48254,48702,49153,49609,50070,50534,51003,51477,51955,52438,52926,534 18,53915,54417,54923,55434,55951,56472,56999,57530,58067,58609,59155,59708,60266,60829,61397,61971,62551,63137,63728,64325,64927}. ,

[0317] In some examples, static LUT ArrayTs can be provided as side information. The calculation of the LL RGB thresholds can be expressed in 17-bit fixed-point representation. For example:

[0318] ArrayTs={0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,4,4,4,4,4,4,4,4,4,4,4,4,4,5,5,5,5,5,5,5,5,5,5,7,7,7,7,7,7,7,7,7,8,8,8,8,8,8,8,8,9,9,9,9,9,9,9,10,10,10,10,10,10,12,12,12,12,12,13,13,13,13,13,13,14,14,14,14,16,16,16,16,16,17,17,17,17,18,18,18,18,20,20,20,20,21,21,21,21,22,22,22,24,24,24,24,25,25,25,26,26,26,28,28,28,29,29,30,30,30,31,31,31,33,33,34,34,34,35,35,37,37,37,38,38,39,39,41,41,42,42,43,43,45,45,46,46,47,47,48,48,50,50,51,51,52,54,54,55,55,56,58,58,59,59,60,62,62,63,64,64,66,67,68,68,69,71,71,72,73,75,75,76,77,79,80,80,81,83,84,85,87,88,88,89,90,92,93,94,96,97,98,100,101,102,104,105,106,107,109,110,111,113,114,115,117,119,121,122,123,125,126,128,130,131,132,135,136,138,140,142,143,144,147,148,151,152,153,156,157,160,161,164,165,168,169,172,173,176,178,180,182,184,186,189,191,193,195,198,201,202,204,207,210,212,215,216,219,222,224,227,229,232,235,237,240,244,246,249,252,254,257,261,263,266,270,273,275,279,282,284,288,291,295,298,301,304,308,312,315,319,322,325,329,333,337,341,343,347,351,355,359,363,367,371,375,379,384,388,392,396,401,405,409,414,418,422,427,431,436,440,446,451,455,460,465,471,474,480,485,490,495,501,506,511,516,523,528,533,539,545,551,556,562,568,574,579,586,592,598,604,611,617,624,630,637,644,650,657,663,670,676,684,691,699,705,713,720,727,734,742,750,758,765,773,781,789,797,805,813,821,830,838,847,855,864,872,881,890,899,908,916,925,936,945,954,963,974,983,992,1003,1013,1022,1033,1043,1054,1064,1075,1085,1096,1106,1118,1129,1140,1151,1163,1174,1186,1197,1208,1222,1233,1245,1257,1270,1282,1295,1307,1320,1333,1346,1359,1372,1385,1399,1413,1426,1440,1455,1468,1482,1497,1511,1526,1541,1556,1572,1586,1602,1617,1633,1649,1665,1680,1696,1713,1729,1746,1763,1780,1797,1814,1831,1848,1866,1884,1902,1920,1939,1957,1975,1995,2013,2033,2053,2071,2091,2112,2131,2151,2172,2193,2214,2233,2256,2277,2298,2320,2342,2365,2387,2409,2431,2455,2477,2501,2524,2548,2572,2597,2620,2645,2670,2695,2721,2746,2772,2797,2823,2851,2877,2903,2931,2958,2986,3013,3042,3070,3099,3127,3158,3186,3217,3245,3277,3307,3337,3369,3400,3431,3463,3496,3527,3560,3593,3627,3661,3694,3728,3763,3797,3833,3868,3903,3940,3977,4013,4050,4087,4125,4163,4201,4240,4280,4319,4358,4399,4438,4480,4521,4563,4605,4647,4688,4732,4775,4820,4864,4909,4953,4998,5044,5091,5137,5184,5231,5280,5328,5377,5425,5475,5525,5576,5627,5678,5729,5782,5834,5888,5941,5995,6050,6105,6160,6217,6273,6331,6388,6446,6505,6564,6624,6683,6745,6807,6868,6930,6993,7057,7121,7185,7251,7316,7383,7450,7517,7585,7655,7723,7794,7864,7935,8007,8079,8153,8226,8301,8376,8452,8529,8605,8684,8762,8841,8921,9002,9083,9165,9247,9331,9415,9500,9587,9673,9760,9849,9937,10027,10117,10208,10301,10394,10487,10581,10677,10773,10870,10968,11068,11167,11268,11369,11471,11575,11679,11785,11891,11997,12106,12215,12325,12436,12548,12660,12774,12890,13005,13123,13241,13360,13479,13601,13723,13846,13971,14097,14224,14352,14481,14611,14743,14875,15009,15144,15280,15418,15556,15696,15837,15979,16123,16269,16414,16562,16710,16861,17013,17165,17320,17476,17633,17792,17952,18113,18275,18439,18606,18772,18941,19112,19283,19456,19632,19809,19987,20167,20348,20531,20715,20902,21089,21280,21471,21664,21859,22055,22253,22454,22656,22859,23065,23273,23482,23694,23906,24121,24339,24558,24779,25002,25227,25454,25684,25914,26148,26383,26621,26861,27103,27347,27593,27842,28093,28346,28601,28859,29119,29382,29647,29915,30185,30456,30731,31009,31288,31570,31856,32143,32432,32726,33021,33320,33620,33924,34231,34540,34852,35167,35485,35805,36129,36455,36785,37118,37454,37792,38134,38480,38827,39179,39534,39892,40254,40618,40986,41357,41732,42111,42492,42878,43266,43659,44055,44454,44858,45264,45676,46090,46510,46932,47358,47789,48223,48660,49104,49550,50000,50455,50915,51378,51846,52317,52794,53276,53760,54251,54745,55244,55748,56256,56769,57288,57809,58338,58870,59408,59951,60499,61051,61609,62173,62742,63316,63895,64480,65069,65666,66266,66874,67486,68105,68729,69359,69996,70639,71286,71941,72602,73268,73942, 74621,75307,75999,76698,77405,78116,78836,79562,80295,81034,81781,82535,83296,84064,84840,85623,86413,87211,88017,88830,89652,90480, 91317,92162,93015,93876,94745,95622,96508,97404,98307,99219,100139,101068,102007,102954,103910,104876,105851,106835,107829,108833,10 9846,110869,111901,112945,113997,115060,116134,117218,118311,119416,120531,121658,122795,123943,125103,126273,127456,128650,129854}. ,

[0319] The following pseudocode represents an example implementation of static LUT ArrayTs construction:

[0320]

[0321] The following pseudocode represents an example implementation for constructing a histogram in linear light:

[0322]

[0323] The techniques of this disclosure may allow for a low complexity and low storage implementation of histogram computation in the linear light RGB domain, with a granularity comparable to that of a non-linear representation in the PQ domain, while still meeting HDR10+ requirements for accuracy.

[0324] Various examples of the disclosed technology are represented in the following terms:

[0325] Item 1: A method of encoding high dynamic range (HDR) video data, the method comprising: calculating a histogram for an image of the video data, the image being represented in a linear light format; encoding values ​​of the histogram for the image represented in the linear light format; and encoding the image.

[0326] Clause 2: A method according to clause 1, wherein calculating the histogram comprises: forming a plurality of bins for the histogram, each of the bins corresponding to a corresponding range in the range of linear light values ​​of the image, wherein a first bin of the plurality of bins corresponds to a first range in the range of linear light values ​​having a first width, and a second bin of the plurality of bins corresponds to a second range in the range of linear light values, the first width being different from the second width; and for each bin of the plurality of bins, calculating the number of samples of the image within the corresponding range in the range of linear light values.

[0327] Clause 3: A method according to clause 2, wherein encoding the value for the histogram includes: for each bin in the plurality of bins, determining a corresponding codeword for representing the number of samples of the image within the corresponding range in the range of linear light values.

[0328] Clause 4: The method of clause 3, wherein determining the corresponding codeword comprises selecting the corresponding codeword from a set of codewords in a PQ10 format for HDR images.

[0329] Clause 5: A method according to clause 3, wherein determining the corresponding codeword comprises: forming a set of linear light codewords as a subset of a set of codewords in a PQ10 format for HDR images, the set of linear light codewords being smaller than the set of codewords in the PQ10 format for HDR images; and selecting the corresponding codeword from the set of linear light codewords.

[0330] Clause 6: The method of clause 1, wherein calculating the histogram comprises forming a variable array having a size of 210x18 bits for representing data for the histogram.

[0331] Clause 7: The method of clause 1, wherein encoding the image comprises: converting the image into a luminance, blue hue chroma, and red hue chroma (YUV) format image; and encoding the YUV format image.

[0332] Clause 8: The method of clause 1, further comprising: forming a bitstream comprising data for the encoded image and data for the encoded values ​​of the histogram.

[0333] Item 9: A device for encoding high dynamic range (HDR) video data, the device comprising: a memory configured to store video data; and one or more processors implemented in circuits and configured to: calculate a histogram for an image of the video data, the image being represented in a linear light format; encode values ​​of the histogram for the image represented in the linear light format; and encode the image.

[0334] Clause 10: A device according to clause 9, wherein, in order to calculate the histogram, the one or more processors are configured to: form a plurality of bins for the histogram, each of the bins corresponding to a corresponding range in the range of linear light values ​​of the image, wherein a first bin of the plurality of bins corresponds to a first range in the range of linear light values ​​having a first width, and a second bin of the plurality of bins corresponds to a second range in the range of linear light values, the first width being different from the second width; and for each bin of the plurality of bins, calculate the number of samples of the image within the corresponding range in the range of linear light values.

[0335] Clause 11: A device according to clause 10, wherein, in order to encode the value for the histogram, the one or more processors are configured to: for each bin in the plurality of bins, determine a corresponding codeword for representing the number of samples of the image within the corresponding range in the range of linear light values.

[0336] Clause 12: The apparatus of clause 11, wherein, to determine the corresponding codeword, the one or more processors are configured to: select the corresponding codeword from a set of codewords in a PQ10 format for HDR images.

[0337] Clause 13: An apparatus according to clause 11, wherein, to determine the corresponding codeword, the one or more processors are configured to: form a set of linear light codewords as a subset of a set of codewords in a PQ10 format for HDR images, the set of linear light codewords being smaller than the set of codewords in the PQ10 format for HDR images; and select the corresponding codeword from the set of linear light codewords.

[0338] Clause 14: The apparatus of clause 9, wherein computing the histogram comprises forming an array of variables having a size of 210x18 bits for representing data for the histogram.

[0339] Clause 15: The apparatus of clause 9, wherein encoding the image comprises: converting the image into a luminance, blue hue chroma, and red hue chroma (YUV) format image; and encoding the YUV format image.

[0340] Clause 16: The apparatus of clause 9, further comprising: forming a bitstream comprising data for the encoded image and data for the encoded values ​​of the histogram.

[0341] Clause 17: The device of clause 9, wherein the device comprises one or more of: a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

[0342] Item 18: A device for encoding high dynamic range (HDR) video data, the device comprising: a unit for calculating a histogram for an image of the video data, the image being represented in a linear light format; a unit for encoding values ​​of the histogram for the image represented in the linear light format; and a unit for encoding the image.

[0343] Clause 19: An apparatus according to clause 18, wherein the unit for calculating the histogram includes: a unit for forming a plurality of bins for the histogram, each of the bins corresponding to a corresponding range in the range of linear light values ​​of the image, wherein a first bin of the plurality of bins corresponds to a first range in the range of linear light values ​​having a first width, and a second bin of the plurality of bins corresponds to a second range in the range of linear light values, the first width being different from the second width; and a unit for calculating, for each bin of the plurality of bins, the number of samples of the image within the corresponding range in the range of linear light values.

[0344] Clause 20: An apparatus according to clause 19, wherein the unit for encoding the value for the histogram includes: a unit for determining, for each bin in the plurality of bins, a corresponding codeword for representing the number of samples of the image within the corresponding range in the range of linear light values.

[0345] Clause 21: The apparatus of clause 20, wherein means for determining the corresponding codeword comprises means for selecting the corresponding codeword from a set of codewords in a PQ10 format for HDR images.

[0346] Clause 22: An apparatus according to clause 20, wherein the means for determining the corresponding codeword includes: means for forming a set of linear light codewords as a subset of a set of codewords in a PQ10 format for HDR images, the set of linear light codewords being smaller than the set of codewords in the PQ10 format for HDR images; and means for selecting the corresponding codeword from the set of linear light codewords.

[0347] Clause 23: The apparatus of clause 18, wherein means for calculating the histogram comprises means for forming an array of variables having a size of 210x18 bits for representing data for the histogram.

[0348] Clause 24: The apparatus of clause 18, wherein the unit for encoding the image comprises: a unit for converting the image into a luminance, blue hue chroma, and red hue chroma (YUV) format image; and a unit for encoding the YUV format image.

[0349] Clause 25: The apparatus of clause 18, further comprising means for forming a bitstream, the bitstream comprising data for the encoded image and data for the encoded values ​​of the histogram.

[0350] Clause 26: A method of encoding high dynamic range (HDR) video data, the method comprising: calculating a histogram for an image of the video data, the image being represented in a linear light format; encoding values ​​of the histogram for the image represented in the linear light format; and encoding the image.

[0351] Clause 27: A method according to clause 26, wherein calculating the histogram includes: forming a plurality of bins for the histogram, each of the bins corresponding to a corresponding range in the range of linear light values ​​of the image, wherein a first bin of the plurality of bins corresponds to a first range in the range of linear light values ​​having a first width, and a second bin of the plurality of bins corresponds to a second range in the range of linear light values, the first width being different from the second width; and for each bin of the plurality of bins, calculating the number of samples of the image within the corresponding range in the range of linear light values.

[0352] Clause 28: The method of clause 27, wherein encoding the values ​​for the histogram comprises, for each bin in the plurality of bins, determining a corresponding codeword representing the number of samples of the image within the corresponding range in the range of linear light values.

[0353] Clause 29: The method of clause 28, wherein determining the corresponding codeword comprises selecting the corresponding codeword from a set of codewords in a PQ10 format for HDR images.

[0354] Clause 30: A method according to clause 28, wherein determining the corresponding codeword comprises: forming a set of linear light codewords as a subset of a set of codewords in a PQ10 format for HDR images, the set of linear light codewords being smaller than the set of codewords in the PQ10 format for HDR images; and selecting the corresponding codeword from the set of linear light codewords.

[0355] Clause 31: The method of any of clauses 26-30, wherein computing the histogram comprises forming an array of variables having a size of 210x18 bits for representing data for the histogram.

[0356] Clause 32: A method according to any of clauses 26-31, wherein encoding the image comprises: converting the image into a luminance, blue hue chroma and red hue chroma (YUV) format image; and encoding the YUV format image.

[0357] Clause 33: A method as described in any of clauses 26-32, further comprising: forming a bitstream, the bitstream comprising data for the encoded image and data for the encoded values ​​of the histogram.

[0358] Item 34: A device for encoding high dynamic range (HDR) video data, the device comprising: a memory configured to store video data; and one or more processors implemented in circuits and configured to: calculate a histogram for an image of the video data, the image being represented in a linear light format; encode values ​​of the histogram for the image represented in the linear light format; and encode the image.

[0359] Clause 35: A device according to clause 34, wherein, in order to calculate the histogram, the one or more processors are configured to: form a plurality of bins for the histogram, each of the bins corresponding to a corresponding range in the range of linear light values ​​of the image, wherein a first bin of the plurality of bins corresponds to a first range in the range of linear light values ​​having a first width, and a second bin of the plurality of bins corresponds to a second range in the range of linear light values, the first width being different from the second width; and for each bin of the plurality of bins, calculate the number of samples of the image within the corresponding range in the range of linear light values.

[0360] Clause 36: A device according to clause 35, wherein, in order to encode the value for the histogram, the one or more processors are configured to: for each bin in the plurality of bins, determine a corresponding codeword for representing the number of samples of the image within the corresponding range in the range of linear light values.

[0361] Clause 37: The apparatus of clause 36, wherein, to determine the corresponding codeword, the one or more processors are configured to: select the corresponding codeword from a set of codewords in a PQ10 format for HDR images.

[0362] Clause 38: An apparatus according to clause 36, wherein, to determine the corresponding codeword, the one or more processors are configured to: form a set of linear light codewords as a subset of a set of codewords in a PQ10 format for HDR images, the set of linear light codewords being smaller than the set of codewords in the PQ10 format for HDR images; and select the corresponding codeword from the set of linear light codewords.

[0363] Clause 39: Apparatus according to any of clauses 34-38, wherein computing the histogram comprises forming an array of variables having a size of 210x18 bits for representing data for the histogram.

[0364] Clause 40: A device according to any of clauses 34-39, wherein encoding the image comprises: converting the image into a luminance, blue hue chroma and red hue chroma (YUV) format image; and encoding the YUV format image.

[0365] Clause 41: Apparatus according to any of clauses 34-40, further comprising: forming a bitstream comprising data for the encoded image and data for the encoded values ​​of the histogram.

[0366] Clause 42: The device of clauses 34-41, wherein the device comprises one or more of: a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

[0367] Item 43: A device for encoding high dynamic range (HDR) video data, the device comprising: a unit for calculating a histogram for an image of the video data, the image being represented in a linear light format; a unit for encoding values ​​of the histogram for the image represented in the linear light format; and a unit for encoding the image.

[0368] Clause 44: An apparatus according to clause 43, wherein the unit for calculating the histogram includes: a unit for forming a plurality of bins for the histogram, each of the bins corresponding to a corresponding range in the range of linear light values ​​of the image, wherein a first bin of the plurality of bins corresponds to a first range in the range of linear light values ​​having a first width, and a second bin of the plurality of bins corresponds to a second range in the range of linear light values, the first width being different from the second width; and a unit for calculating, for each bin of the plurality of bins, the number of samples of the image within the corresponding range in the range of linear light values.

[0369] Clause 45: An apparatus according to clause 44, wherein the unit for encoding the value for the histogram includes: a unit for determining, for each bin in the plurality of bins, a corresponding codeword for representing the number of samples of the image within the corresponding range in the range of linear light values.

[0370] Clause 46: The apparatus of clause 45, wherein means for determining the corresponding codeword comprises means for selecting the corresponding codeword from a set of codewords in a PQ10 format for HDR images.

[0371] Clause 47: An apparatus according to clause 45, wherein the unit for determining the corresponding codeword includes: a unit for forming a set of linear light codewords as a subset of a set of codewords in a PQ10 format for HDR images, the set of linear light codewords being smaller than the set of codewords in the PQ10 format for HDR images; and a unit for selecting the corresponding codeword from the set of linear light codewords.

[0372] Clause 48: Apparatus according to any of clauses 43-47, wherein the means for calculating the histogram comprises means for forming a variable array having a size of 210x18 bits for representing data for the histogram.

[0373] Clause 49: An apparatus according to any of clauses 43-48, wherein the unit for encoding the image includes: a unit for converting the image into a luminance, blue hue chroma and red hue chroma (YUV) format image; and a unit for encoding the YUV format image.

[0374] Clause 50: Apparatus according to any of clauses 43-49, further comprising means for forming a bitstream comprising data for the encoded image and data for the encoded values ​​of the histogram.

[0375] It is to be appreciated that, according to examples, certain actions or events of any technique described herein may be performed in a different order, may be added, combined, or omitted entirely (e.g., not all described actions or events are necessary to implement the technique). In addition, in some examples, actions or events may be performed concurrently (e.g., through multithreading, interrupt handling, or multiple processors) rather than sequentially.

[0376] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on a computer-readable medium or transmitted via a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium (which corresponds to a tangible medium such as a data storage medium) or a communication medium, the communication medium including, for example, any medium that facilitates the transfer of a computer program from one place to another according to a communication protocol. In this manner, a computer-readable medium may generally correspond to (1) a non-temporary tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to obtain instructions, codes, and / or data structures for implementing the techniques described in the present disclosure. A computer program product may include a computer-readable medium.

[0377] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store the desired program code in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately referred to as a computer-readable medium. For example, if the instruction is transmitted from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology (e.g., infrared, radio and microwave), the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology (e.g., infrared, radio and microwave) is included in the definition of the medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals or other temporary media, but are instead directed to non-temporary tangible storage media. As used herein, disks and optical disks include compact disks (CDs), laser optical disks, optical disks, digital versatile disks (DVDs), floppy disks and blue-ray disks, wherein disks usually copy data magnetically, while optical disks use lasers to optically copy data. Combinations of the above should also be included within the scope of computer-readable media.

[0378] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the terms "processor" and "processing circuitry" as used herein may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein. In addition, in some aspects, the functions described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. In addition, the techniques may be implemented entirely in one or more circuits or logic elements.

[0379] The techniques of the present disclosure may be implemented in a variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units are described in the present disclosure to emphasize the functional aspects of a device configured to perform the disclosed techniques, but do not necessarily require implementation by different hardware units. Instead, as described above, the various units may be combined in a codec hardware unit, or provided by a collection of interoperable hardware units (including one or more processors as described above) in combination with appropriate software and / or firmware.

[0380] Various examples have been described. These and other examples are within the scope of the following claims.

Claims

1. A method for encoding high dynamic range (HDR) video data, the method comprising: calculating a histogram for an image of the video data, the image being represented in a linear light format; extracting one or more parameters of the histogram; encoding the one or more parameters of the histogram of the image represented in the linear light format; as well as The image is encoded.

2. The method according to claim 1, wherein: Calculating the histogram includes: forming a plurality of bins for the histogram, each of the bins corresponding to a respective range in the range of linear light values ​​for the image, wherein a first bin in the plurality of bins corresponds to a first range in the range of linear light values ​​having a first width, and a second bin in the plurality of bins corresponds to a second range in the range of linear light values, the first width being different than the second width; and For each bin of the plurality of bins, a number of samples of the image within a corresponding range of the range of linear light values ​​is calculated.

3. The method according to claim 2, wherein: One or more of the bins correspond to different ranges of the range of linear light values ​​having the same width.

4. The method according to claim 2, wherein: Encoding the one or more parameters of the histogram includes determining, for one or more bins of the plurality of bins, a corresponding codeword representing the number of samples of the image within the corresponding range in the range of linear light values.

5. The method according to claim 4, wherein: Determining the corresponding codeword includes selecting the corresponding codeword from a set of codewords for nonlinear functions.

6. The method according to claim 5, wherein: The nonlinear function includes an ST.2084PQ nonlinear function, a PQ10 nonlinear function, or a PQ12 nonlinear function.

7. The method according to claim 4, wherein: Determining the corresponding codeword includes: forming a set of linear light codewords as a subset of a set of codewords in a PQ10 format for HDR images, the set of linear light codewords being smaller than the set of codewords in the PQ10 format for HDR images and being represented using a fixed number of bits; and The corresponding codeword is selected from the set of linear light codewords.

8. The method according to claim 7, wherein: The fixed number of bits includes 7 bits.

9. The method according to claim 1, wherein: Calculating the histogram includes: forming a matrix having a value of 2 for representing data for the histogram. 10 x18-bit size variable array.

10. The method according to claim 1, wherein: Encoding the image comprises: Converting the image into a luminance, blue hue chroma, and red hue chroma (YUV) format image; and Encode the YUV format image.

11. The method according to claim 1, further comprising: A bitstream is formed, the bitstream comprising data for the encoded image and data for the encoded one or more parameters of the histogram.

12. An apparatus for encoding high dynamic range (HDR) video data, the apparatus comprising: a memory configured to store histogram data; as well as One or more processors implemented in circuitry and configured to: calculating a histogram for an image of the video data, the image being represented in a linear light format; extracting one or more parameters of the histogram; encoding the one or more parameters of the histogram of the image represented in the linear light format; as well as The image is encoded.

13. The device according to claim 12, wherein: To calculate the histogram, the one or more processors are configured to: forming a plurality of bins for the histogram, each of the bins corresponding to a respective range in the range of linear light values ​​for the image, wherein a first bin in the plurality of bins corresponds to a first range in the range of linear light values ​​having a first width, and a second bin in the plurality of bins corresponds to a second range in the range of linear light values, the first width being different than the second width; and For each bin of the plurality of bins, a number of samples of the image within a corresponding range of the range of linear light values ​​is calculated.

14. The device according to claim 13, wherein: One or more of the bins correspond to different ranges of the range of linear light values ​​having the same width.

15. The apparatus according to claim 13, wherein: To encode the one or more parameters of the histogram, the one or more processors are configured to determine, for one or more bins of the plurality of bins, a corresponding codeword representing the number of samples of the image within the corresponding range in the range of linear light values.

16. The device according to claim 15, wherein: To determine the corresponding codeword, the one or more processors are configured to select the corresponding codeword from a set of codewords for non-linear functions.

17. The apparatus according to claim 16, wherein: The nonlinear function includes an ST.2084PQ nonlinear function, a PQ10 nonlinear function, or a PQ12 nonlinear function.

18. The apparatus according to claim 15, wherein: To determine the corresponding codeword, the one or more processors are configured to: forming a set of linear light codewords as a subset of a set of codewords in a PQ10 format for HDR images, the set of linear light codewords being smaller than the set of codewords in the PQ10 format for HDR images and being represented using a fixed number of bits; and The corresponding codeword is selected from the set of linear light codewords.

19. The apparatus according to claim 18, wherein: The fixed number of bits includes 7 bits.

20. The apparatus of claim 12, wherein: Calculating the histogram includes: forming a matrix having a value of 2 for representing data for the histogram. 10 x18-bit size variable array.

21. The apparatus of claim 12, wherein: Encoding the image comprises: Converting the image into a luminance, blue hue chroma, and red hue chroma (YUV) format image; and Encode the YUV format image.

22. The apparatus of claim 12, further comprising: A bitstream is formed, the bitstream comprising data for the encoded image and data for the encoded one or more parameters of the histogram.

23. The apparatus of claim 12, wherein: The device includes one or more of the following: a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

24. An apparatus for encoding high dynamic range (HDR) video data, the apparatus comprising: means for calculating a histogram for an image of video data, the image being represented in a linear light format; means for extracting one or more parameters of said histogram; means for encoding said one or more parameters of said histogram of said image represented in said linear light format; as well as Unit for encoding said image.

Images