Method and apparatus for encoding high dynamic range video data

Abstract

An example device for encoding high dynamic range (HDR) video data includes a memory configured to store video data, and one or more processors implemented in circuitry and configured to compute a histogram for an image of the video data, the image represented in a linear light format, encode values of the histogram for the image represented in the linear light format, and encode the image. Data for the histogram can be represented in a variable array of size 2 10 x18 bits. The device can encode codewords for representing values of bins for the histogram, where the codewords can be selected from a set of codewords for a PQ10 format of an HDR image. The bins of the histogram can represent ranges of unequal widths.

Description

Technical Field

[0001] This disclosure relates to video decoding, including video encoding and video decoding. Background Technology

[0002] Digital video capabilities can be incorporated into a wide variety of devices, including digital televisions, digital live broadcast systems, wireless broadcasting systems, personal digital assistants (PDAs), laptops or desktop computers, tablets, e-book readers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, cellular or satellite radio phones (so-called "smartphones"), video conferencing equipment, video streaming devices, and more. Digital video devices implement video decoding technologies, such as those defined in standards like MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4 Part 10, Advanced Video Decoding (AVC), ITU-T H.265 / High-Efficiency Video Decoding (HEVC), and ITU-T H.266 / Various Video Decoding (VVC), as well as extensions to these standards, and proprietary video codecs / formats such as AOMedia Video1 (AV1) developed by the Open Media Consortium. By implementing such video decoding technologies, video devices can more efficiently send, receive, encode, decode, and / or store digital video information.

[0003] Video decoding techniques include spatial (intra-picture) prediction and / or temporal (inter-picture) prediction to reduce or remove redundancy inherent in video sequences. For block-based video decoding, video slices (e.g., video pictures or portions of video pictures) can be divided into video blocks, which may also be referred to as decoding tree units (CTUs), decoding units (CUs), and / or decoding nodes. Video blocks in a slice of a picture that has been intra-decoded (I) are encoded using spatial prediction relative to reference samples in adjacent blocks within the same picture. Video blocks in a slice of a picture that has been inter-decoded (P or B) can use spatial prediction relative to reference samples in adjacent blocks within the same picture or temporal prediction relative to reference samples in other reference pictures. A picture may be called a frame, and a reference picture may be called a reference frame. Summary of the Invention

[0004] Generally, this disclosure describes techniques for decoding (encoding and decoding) high dynamic range (HDR) video data. To encode HDR video data, statistics, such as one or more histograms, are calculated for one or more images of the HDR video data. Typically, a histogram comprises multiple bins, each representing a range of possible luminance and / or chrominance values, and has a value indicating the number of samples within that range of possible luminance and / or chrominance values. Images of video data can be represented in various formats, such as red-green-blue (RGB) format or luminance and chrominance (e.g., YUV or YCbCr, representing luminance, blue hue chrominance, and red hue chrominance) formats. Therefore, a histogram can be calculated using one of the various formats for the image, which is not necessarily the same format used when encoding the image.

[0005] This disclosure recognizes that, in terms of memory storage for histograms, calculating a histogram for an image of HDR video data represented in Linear Light (LL) RGB format may be more efficient than calculating a histogram for an image using PQ10 format. For example, PQ10 format might require a size of 2 16 A 16-bit lookup table, the lookup table for LL RGB format can have 2... 10 The size is 18 bits. Therefore, according to the techniques of this disclosure, a video coding system for encoding HDR video data can calculate histograms for one or more images in LL RGB format.

[0006] In one example, a method for encoding high dynamic range (HDR) video data includes: calculating a histogram of an image of the video data, the image being represented in a linear light format; extracting one or more parameters of the histogram; encoding the one or more parameters of the histogram of the image represented in the linear light format; and encoding the image.

[0007] In another example, an apparatus for encoding high dynamic range (HDR) video data includes: a memory configured to store histogram data; and one or more processors implemented in a circuit and configured to: calculate a histogram of an image of the video data, the image being represented in a linear light format; extract one or more parameters of the histogram; encode the one or more parameters of the histogram of the image represented in the linear light format; and encode the image.

[0008] In another example, an apparatus for encoding high dynamic range (HDR) video data includes: a unit for calculating a histogram of an image for video data, the image being represented in a linear light format; a unit for extracting one or more parameters of the histogram; a unit for encoding the one or more parameters of the histogram of the image represented in the linear light format; and a unit for encoding the image.

[0009] In another example, a computer-readable storage medium is encoded with instructions that, when executed, cause a programmable processor to: calculate a histogram of an image for video data, the image being represented in a linear light format; extract one or more parameters of the histogram; encode the one or more parameters of the histogram of the image represented in the linear light format; and encode the image.

[0010] Details of one or more examples are set forth in the accompanying drawings and the following description. Other features, objects, and advantages will be apparent from the description, drawings, and claims. Attached Figure Description

[0011] Figure 1 This is a block diagram illustrating an example video encoding and decoding system that can perform the techniques described in this disclosure.

[0012] Figure 2 This is a block diagram illustrating an example high dynamic range (HDR) video coding system based on the technology of this disclosure.

[0013] Figure 3 This is a schematic diagram illustrating an example set of bins for a histogram to be calculated, based on the technology of this disclosure.

[0014] Figure 4 This is a block diagram illustrating an example video encoder capable of performing video encoding according to the techniques disclosed herein.

[0015] Figure 5 This is a block diagram illustrating an example video decoder capable of performing video decoding according to the techniques disclosed herein.

[0016] Figure 6 This is a flowchart illustrating an example method for encoding image and histogram data using techniques based on this disclosure.

[0017] Figure 7A and 7B This is a conceptual diagram illustrating an example of HDR and WCG representation conversion.

[0018] Figure 8 This is a schematic diagram illustrating various examples of transfer functions.

[0019] Figure 9 This is a schematic diagram illustrating an example PQ OETF.

[0020] Figure 10 This is an example diagram for deriving DistributionY99 and DistributionY100 based on histogram percentiles.

[0021] Figure 11 This is a block diagram illustrating an example live encoder framework with HDR10+ capability.

[0022] Figure 12 This is a conceptual diagram illustrating an example process for deriving HDR10+ statistical characteristics of video data using techniques according to this disclosure. Detailed Implementation

[0023] The technology disclosed herein relates to decoding (encoding and decoding) high dynamic range (HDR) video data. When encoding HDR data, in addition to encoding the images, the video encoder can also calculate statistical information of one or more images, such as the image histogram. Therefore, the video encoder can encode both the image itself and statistical data. In some cases, images can be represented in various formats. For example, an image can be captured in red-green-blue (RGB) format and then converted to luminance and chrominance (e.g., YUV or YCbCr, which represent luminance, blue hue chrominance, and red hue chrominance).

[0024] Some video applications can operate using video data representing captured landscapes with HDR and wide color gamut (WCG). Dynamic range and color gamut are two separate properties of video content, and their specifications for digital television and multimedia service purposes are defined by several international standards. For example, ITU-R Rec.709 defines parameters for HDTV, such as standard dynamic range and standard color gamut, and ITU-R Rec.2020 specifies parameters for Ultra High Definition Television (UHDTV), such as high dynamic range and wide color gamut. Other Standards Development Organization (SDO) documents specify these properties in other systems; for example, the P3 color gamut is defined in SMPTE-231-2, and SMPTE-2084 defines certain parameters for HDR.

[0025] Dynamic range is typically defined as the ratio between the minimum and maximum brightness of a video signal. Dynamic range is also measured in "f-stops," where the difference between one f-stop and another corresponds to doubling the signal's dynamic range. The Moving Picture Experts Group (MPEG) defines high dynamic range content as content with a brightness variation of more than 16 f-stops. In some cases, levels between 10 and 16 f-stops are considered intermediate dynamic range, but in other definitions, they may be considered HDR. Meanwhile, the human visual system is capable of perceiving an even greater dynamic range. However, the human visual system includes an adaptive mechanism to narrow the range known as simultaneous range.

[0026] ITU-R Rec. 709 describes standard definition range (SDR), which typically supports a brightness (or luminance) range (often referred to as "nits") of approximately 0.1 to 100 candela (cd) per square meter, resulting in fewer than 10 stops. Next-generation video services are expected to offer dynamic ranges of up to 16 stops. Although detailed specifications are currently being developed, some initial parameters have been specified in SMPTE-2084 and ITU-R Rec. 2020.

[0027] Besides HDR, another aspect for a more realistic video experience is the color dimension, typically defined by the color gamut. The SDR color gamut can be visualized as a triangle based on the BT.709 primary colors of red, green, and blue. WCG is based on the BT.2020 primary colors of red, green, and blue and includes what is called the spectral locus, defined by a tongue-shaped region representing the limits of natural colors. Moving from BT.709 to BT.2020 primary colors aims to provide approximately 70% more color for UHDTV services. White can also be specified. Table 1 below provides examples of various color gamut specifications for colorimetric parameters for example color spaces:

[0028] Table 1

[0029]

[0030] Video data images are typically encoded in YUV format. For traditional HDR encoding, it has always been expected that a histogram for the image be calculated in RGB / Perceptual Quantizer (PQ) format. The image is first converted from linear light RGB format to PQ format, such as PQ10, according to the PQ photoelectric transfer function (OETF). Calculating the histogram for an image in PQ10 format requires a 2x2x2 pixel array. 16 16-bit lookup table (LUT).

[0031] This disclosure recognizes that if the histogram is calculated from an image in a linear light format, a smaller LUT, for example, of size 2, can be used. 10 An 18-bit LUT. Similarly, histograms can be computed from linear light format images without performing a conversion from linear light RGB to PQ10. Therefore, using linear light RGB format images to compute histograms reduces processing operations, latency, and memory consumption for storing the LUT.

[0032] Therefore, according to the technology of this disclosure, the histogram calculation unit can be incorporated into the video encoder or can be separate from the video encoder, and can calculate histograms for one or more images of HDR video data represented in linear light format. When calculating the histogram, the histogram calculation unit can divide the possible range of sample values ​​(luminance and / or chrominance values) into subranges, which correspond to bins of the histogram. The subranges for each bin in a bin are not necessarily, but can be, of the same width. That is, some subranges can have the same width, while other subranges may have different widths. In some examples, all subranges can have the same width. In some examples, all subranges can have different corresponding widths. The histogram calculation unit can initialize each bin value in the bin values ​​to zero. The histogram calculation unit can then iterate through samples of one or more images in the image. The histogram calculation unit can determine the subrange containing the current sample value and then increment the bin value corresponding to the determined subrange.

[0033] After calculating the bin values ​​of the histogram, the histogram calculation unit (or video encoder) can encode the bin values, for example, using codewords. In some examples, the histogram calculation unit can determine the codewords for the bin values ​​from a subset of the PQ10 codewords of conventional HDR video decoding techniques.

[0034] Figure 1 This is a block diagram illustrating an example video encoding and decoding system 100 capable of performing the techniques described in this disclosure. The techniques described in this disclosure generally involve decoding (encoding and / or decoding) video data. Typically, video data includes any data used for processing video. Therefore, video data can include raw, undecoded video, encoded video, decoded (e.g., reconstructed) video, and video metadata, such as signaling data.

[0035] like Figure 1As shown in the example, system 100 includes source device 102, which provides encoded video data to be decoded and displayed by destination device 116. Specifically, source device 102 provides the video data to destination device 116 via computer-readable medium 110. Source device 102 and destination device 116 can include any of a wide range of devices, including desktop computers, laptop computers, mobile devices, tablet computers, set-top boxes, handsets such as smartphones, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, broadcast receiver devices, etc. In some cases, source device 102 and destination device 116 may be configured for wireless communication and therefore may be referred to as wireless communication devices.

[0036] exist Figure 1 In this example, source device 102 includes a video source 104, memory 106, a video encoder 200, and an output interface 108. Destination device 116 includes an input interface 122, a video decoder 300, memory 120, and a display device 118. Therefore, source device 102 represents an example of a video encoding device, while destination device 116 represents an example of a video decoding device. In other examples, the source and destination devices may include other components or arrangements. For example, source device 102 may receive video data from an external video source such as an external camera. Similarly, destination device 116 may interface with an external display device, rather than including an integrated display device.

[0037] like Figure 1 The system 100 shown is merely an example. The source device 102 and destination device 116 are merely examples of such decoding devices, wherein the source device 102 generates decoded video data for transmission to the destination device 116. In this disclosure, a “decoding” device is referred to as a device that performs the decoding (e.g., encoding and / or decoding) of data. Thus, video encoder 200 and video decoder 300 represent examples of decoding devices, specifically, video encoder and video decoder, respectively. In some examples, the source device 102 and destination device 116 may operate in a substantially symmetrical manner, such that each of the source device 102 and destination device 116 includes video encoding and decoding components. Therefore, system 100 can support one-way or two-way video transmission between the source device 102 and destination device 116, for example, for video streaming, video playback, video broadcasting, or video telephony.

[0038] Typically, video source 104 represents a source of video data (i.e., raw, undecoded video data) and provides a sequential series of pictures (also referred to as "frames") of the video data to video encoder 200, which encodes the data for the pictures. Video source 104 of source device 102 may include video capture devices such as a camera, a video archive unit containing previously captured raw video, and / or a video feed interface for receiving video from a video content provider. Alternatively, video source 104 may generate computer graphics-based data as source video, or a combination of live video, archived video, and computer-generated video. In each case, video encoder 200 encodes the captured, pre-captured, or computer-generated video data. Video encoder 200 may rearrange the pictures from their received order (sometimes referred to as "display order") to a decoding order for decoding. Video encoder 200 may generate a bitstream including the encoded video data. Then, source device 102 can output encoded video data to computer-readable medium 110 via output interface 108 for reception and / or retrieval by, for example, input interface 122 of destination device 116.

[0039] The memory 106 of source device 102 and the memory 120 of destination device 116 represent general-purpose memory. In some examples, memories 106 and 120 may store raw video data, such as raw video from video source 104 and raw decoded video data from video decoder 300. Alternatively, memories 106 and 120 may store software instructions executable by, for example, video encoder 200 and video decoder 300. Although memories 106 and 120 are shown separately from video encoder 200 and video decoder 300 in this example, it should be understood that video encoder 200 and video decoder 300 may also include internal memory for functionally similar or equivalent purposes. Furthermore, memories 106 and 120 may store encoded video data, such as output from video encoder 200 and input to video decoder 300. In some examples, portions of memories 106 and 120 may be allocated as one or more video buffers, for example, to store raw, decoded, and / or encoded video data.

[0040] Computer-readable medium 110 can represent any type of medium or device capable of transmitting encoded video data from source device 102 to destination device 116. In one example, computer-readable medium 110 represents a communication medium that enables source device 102 to directly transmit encoded video data to destination device 116 in real time, for example, via a radio frequency network or a computer-based network. According to a communication standard such as a wireless communication protocol, output interface 108 can modulate the transmitted signal including the encoded video data, and input interface 122 can demodulate the received transmitted signal. The communication medium can include any wireless or wired communication medium, such as radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium can form part of a packet-based network such as a local area network, a wide area network, or a global network such as the Internet. The communication medium can include a router, switch, base station, or any other device that may be useful for facilitating communication from source device 102 to destination device 116.

[0041] In some examples, source device 102 can output encoded data from output interface 108 to storage device 112. Similarly, destination device 116 can access encoded data from storage device 112 via input interface 122. Storage device 112 may include any data storage medium of various distributed or locally accessed data storage media, such as hard disk drives, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.

[0042] In some examples, source device 102 can output encoded video data to file server 114 or another intermediate storage device that can store the encoded video data generated by source device 102. Destination device 116 can access the stored video data from file server 114 via streaming or downloading.

[0043] File server 114 can be any type of server device capable of storing encoded video data and sending such encoded video data to destination device 116. File server 114 can represent a web server (e.g., for a website), a server configured to provide file transfer protocol services (e.g., File Transfer Protocol (FTP) or File Transfer over One-Way Transmission (FLUTE) protocol), a Content Delivery Network (CDN) device, a Hypertext Transfer Protocol (HTTP) server, a Multimedia Broadcast Multicast Service (MBMS) or Enhanced MBMS (eMBMS) server, and / or a Network Attached Storage (NAS) device. File server 114 may additionally or alternatively implement one or more HTTP streaming protocols, such as Dynamic Adaptive Streaming over HTTP (DASH), HTTP Live Streaming (HLS), Real-Time Streaming Protocol (RTSP), HTTP Dynamic Streaming, etc.

[0044] Destination device 116 can access encoded video data from file server 114 via any standard data connection, including an internet connection. This can include wireless channels (e.g., Wi-Fi connections), wired connections (e.g., digital subscriber line (DSL), cable modems, etc.), or combinations thereof, suitable for accessing encoded video data stored on file server 114. Input interface 122 can be configured to operate according to any one or more of the various protocols discussed above for retrieving or receiving media data from file server 114, or other such protocols for retrieving media data.

[0045] Output interface 108 and input interface 122 may represent a wireless transmitter / receiver, a modem, a wired networking component (e.g., an Ethernet card), a wireless communication component operating according to any of the various IEEE 802.11 standards, or other physical components. In examples where output interface 108 and input interface 122 include wireless components, output interface 108 and input interface 122 may be configured to transmit data (such as encoded video data) according to cellular communication standards (such as 4G, 4G-LTE (Long Term Evolution), LTE Advanced, 5G, etc.). In some examples where output interface 108 includes a wireless transmitter, output interface 108 and input interface 122 may be configured to transmit data (such as encoded video data) according to other wireless standards (such as the IEEE 802.11 specification, the IEEE 802.15 specification (e.g., ZigBee™), the Bluetooth™ standard, etc.). In some examples, source device 102 and / or destination device 116 may include corresponding system-on-chip (SoC) devices. For example, source device 102 may include a SoC device for performing functions belonging to video encoder 200 and / or output interface 108, and destination device 116 may include a SoC device for performing functions belonging to video decoder 300 and / or input interface 122.

[0046] The technology disclosed herein can be applied to video decoding to support any multimedia application in a variety of multimedia applications, such as over-the-air television broadcasting, cable television transmission, satellite television transmission, internet streaming video transmission (such as Dynamic Adaptive Streaming (DASH) via HTTP), digital video encoded onto data storage media, decoding digital video stored on data storage media, or other applications.

[0047] The input interface 122 of the destination device 116 receives an encoded video bitstream from a computer-readable medium 110 (e.g., a communication medium, storage device 112, file server 114, etc.). The encoded video bitstream may include signaling information defined by the video encoder 200, such as syntax elements (which are also used by the video decoder 300), which have values ​​describing the characteristics and / or processing of video blocks or other decoding units (e.g., slices, pictures, picture groups, sequences, etc.). The display device 118 displays decoded images of the decoded video data to the user. The display device 118 may represent any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display device.

[0048] Despite Figure 1Not shown, but in some examples, both the video encoder 200 and the video decoder 300 may be integrated with the audio encoder and / or audio decoder, and may include appropriate MUX-DEMUX units or other hardware and / or software to process multiplexed streams that include both audio and video in a common data stream.

[0049] Both the video encoder 200 and the video decoder 300 can be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the technology is implemented in part in software, the device may store instructions for the software in a suitable non-transitory computer-readable medium and use one or more processors to execute the instructions in hardware to perform the technology of this disclosure. Each of the video encoder 200 and the video decoder 300 may be included in one or more encoders or decoders, and either encoder or decoder may be integrated as part of a combined encoder / decoder (CODEC) in the respective device. Devices including the video encoder 200 and / or the video decoder 300 may include integrated circuits, microprocessors, and / or wireless communication devices (such as cellular phones).

[0050] The video encoder 200 and video decoder 300 may operate according to video decoding standards such as ITU-T H.265 (also known as High Efficiency Video Decoding (HEVC)) or extensions thereof such as MultiView and / or Extensible Video Decoding Extensions. Alternatively, the video encoder 200 and video decoder 300 may operate according to other proprietary or industry standards such as ITU-T H.266 (also known as Universal Video Decoding (VVC)). In other examples, the video encoder 200 and video decoder 300 may operate according to proprietary video codecs / formats such as AOMedia Video 1 (AV1), extensions to AV1, and / or subsequent versions of AV1 (e.g., AV2). In other examples, the video encoder 200 and video decoder 300 may operate according to other proprietary formats or industry standards. However, the technology of this disclosure is not limited to any particular decoding standard or format.

[0051] Generally, video encoder 200 and video decoder 300 perform block-based decoding of images. The term "block" typically refers to a structure that includes data to be processed (e.g., encoded, decoded, or otherwise used during encoding and / or decoding). For example, a block may include a two-dimensional matrix of samples of luminance and / or chrominance data. Typically, video encoder 200 and video decoder 300 can decode video data represented in YUV (e.g., Y, Cb, Cr) format. That is, instead of decoding the red, green, and blue (RGB) data used for images, video encoder 200 and video decoder 300 can decode both luminance and chrominance components, where chrominance components may include both red hue and blue hue chrominance components. In some examples, video encoder 200 converts received RGB formatted data to YUV representation before encoding, and video decoder 300 converts the YUV representation to RGB format. Alternatively, preprocessing and post-processing units (not shown) may perform these conversions.

[0052] This disclosure can generally relate to the decoding (e.g., encoding and decoding) of images to include the process of encoding or decoding the data of an image. Similarly, this disclosure can relate to the decoding of blocks of images to include the process of encoding or decoding the data used for the blocks (e.g., prediction and / or residual decoding). Encoded video bitstreams typically include a series of values ​​for syntax elements that represent decoding decisions (e.g., decoding modes) and the partitioning of images into blocks. Therefore, references to the decoding of an image or block should generally be understood as the decoded values ​​of the syntax elements that form the image or block.

[0053] HEVC defines various blocks, including decoding units (CUs), prediction units (PUs), and transform units (TUs). According to HEVC, a video decoder (such as a video encoder 200) divides the decoding tree unit (CTU) into CUs based on a quadtree structure. That is, the video decoder divides the CTU and CU into four equal, non-overlapping squares, and each node of the quadtree has zero or four child nodes. Nodes without child nodes can be called "leaf nodes," and the CU of such leaf nodes can include one or more PUs and / or one or more TUs. The video decoder can further divide the PUs and TUs. For example, in HEVC, a residual quadtree (RQT) represents the partitioning of TUs. In HEVC, PUs represent inter-frame prediction data, while TUs represent residual data. CUs with intra-frame prediction include intra-frame prediction information, such as intra-frame mode indication.

[0054] As another example, video encoder 200 and video decoder 300 can be configured to operate according to VVC. According to VVC, the video decoder (such as video encoder 200) divides the image into multiple decoding tree units (CTUs). Video encoder 200 can divide the CTUs according to a tree structure (such as a quadtree-binary tree (QTBT) structure or a multi-type tree (MTT) structure). The QTBT structure removes the concept of multiple partitioning types, such as the separation between CUs, PUs, and TUs in HEVC. The QTBT structure includes two levels: a first level based on quadtree partitioning and a second level based on binary tree partitioning. The root node of the QTBT structure corresponds to a CTU. The leaf nodes of the binary tree correspond to decoding units (CUs).

[0055] In the MTT partitioning structure, blocks can be partitioned using quadtree (QT), binary tree (BT), and one or more types of ternary tree (TT) partitioning (also known as triplet tree (TT)). A ternary tree or triplet tree partition is a partition in which a block is divided into three sub-blocks. In some examples, a ternary tree or triplet tree partition divides a block into three sub-blocks without partitioning the original block through a center. The partitioning types in MTT (e.g., QT, BT, and TT) can be symmetric or asymmetric.

[0056] When operating according to the AV1 codec, the video encoder 200 and video decoder 300 can be configured to decode video data block by block. In AV1, the largest decoded block that can be processed is called a superblock. In AV1, a superblock can be 128x128 luma samples or 64x64 luma samples. However, in subsequent video decoding formats (e.g., AV2), superblocks can be defined by different (e.g., larger) luma sample sizes. In some examples, the superblock is the top level of a block quadtree. The video encoder 200 can further divide the superblock into smaller decoded blocks. The video encoder 200 can use square or non-square partitioning to divide the superblock and other decoded blocks into smaller blocks. Non-square blocks can include N / 2xN, NxN / 2, N / 4xN, and NxN / 4 blocks. The video encoder 200 and video decoder 300 can perform separate prediction and transform processes on each decoded block within the decoded block.

[0057] AV1 also defines video data tiles. A tile is a rectangular array of superblocks that can be decoded independently of other tiles. That is, the video encoder 200 and video decoder 300 can encode and decode the decoded blocks within a tile separately, without using video data from other tiles. However, the video encoder 200 and video decoder 300 can perform filtering across tile boundaries. The size of the tiles can be uniform or non-uniform. Tile-based decoding enables parallel processing and / or multithreading for both the encoder and decoder.

[0058] In some examples, the video encoder 200 and the video decoder 300 may use a single QTBT or MTT structure to represent each of the luma and chroma components, while in other examples, the video encoder 200 and the video decoder 300 may use two or more QTBT or MTT structures, such as one QTBT / MTT structure for the luma component and another QTBT / MTT structure for the two chroma components (or two QTBT / MTT structures for the respective chroma components).

[0059] The video encoder 200 and the video decoder 300 can be configured to use quadtree partitioning, QTBT partitioning, MTT partitioning, superblock partitioning, or other partitioning structures.

[0060] In some examples, a CTU includes: a decoded tree block (CTB) of luminance samples for an image with three sample arrays, two corresponding CTBs of chrominance samples, or a CTB of samples for a monochrome image or an image decoded using three separate color planes, and a syntax structure for decoding the samples. A CTB can be an NxN block of samples for some value of N, such that partitioning a component into a CTB is a partition. A component can be an array (luminance and two chrominance) from one of three arrays (luminance and two chrominance) for an image in a 4:2:0, 4:2:2, or 4:4:4 color format, or a single sample from said array, or an array for an image in monochrome format, or a single sample from said array. In some examples, a decoded block is an MxN block of samples for some values ​​of M and N, such that partitioning a CTB into a decoded block is a partition.

[0061] Blocks (e.g., CTUs or CUs) can be grouped in various ways within an image. As an example, a brick can refer to a rectangular area of ​​a row of CTUs within a specific tile in an image. A tile can be a rectangular area of ​​CTUs within a specific tile column or row in an image. A tile column refers to a rectangular area of ​​CTUs with a height equal to the height of the image and a width specified by syntax elements (e.g., in an image parameter set). A tile row refers to a rectangular area of ​​CTUs with a height specified by syntax elements (e.g., in an image parameter set) and a width equal to the width of the image.

[0062] In some examples, a slice can be divided into multiple bricks, each brick potentially including one or more CTU rows within the slice. A slice that is not divided into multiple bricks can also be called a brick. However, bricks that are a true subset of a slice cannot be called a slice. Bricks in an image can also be arranged in slices. A slice can be an integer number of bricks in an image, which can be uniquely contained within a single Network Abstraction Layer (NAL) unit. In some examples, a slice comprises multiple complete slices or a continuous sequence of complete bricks from a single slice.

[0063] This disclosure uses "NxN" and "N by N" interchangeably to refer to the sample size of a block (such as a CU or other video block) in the vertical and horizontal dimensions, for example, 16x16 samples or 16 by 16 samples. Typically, a 16x16 CU will have 16 samples in the vertical direction (y = 16) and 16 samples in the horizontal direction (x = 16). Similarly, an NxN CU typically has N samples in the vertical direction and N samples in the horizontal direction, where N represents a non-negative integer value. Samples in a CU can be arranged in rows and columns. Furthermore, a CU does not necessarily need to have the same number of samples in the horizontal direction as it does in the vertical direction. For example, a CU can include NxM samples, where M is not necessarily equal to N.

[0064] The video encoder 200 encodes video data for use with representation prediction and / or residual information, as well as other information, for the control unit (CU). Prediction information indicates how the CU should be predicted to form a prediction block of the CU. Residual information typically represents the sample-by-sample difference between a sample of the CU before encoding and the prediction block.

[0065] To predict the Cubic Frame (CU), the video encoder 200 typically forms a predicted block of the CU through inter-frame prediction or intra-frame prediction. Inter-frame prediction generally refers to predicting the CU from data of a previously decoded image, while intra-frame prediction generally refers to predicting the CU from data of a previously decoded image within the same frame. To perform inter-frame prediction, the video encoder 200 can use one or more motion vectors to generate the predicted block. The video encoder 200 can typically perform a motion search to identify a reference block that closely matches the CU, for example, based on the differences between the CU and a reference block. The video encoder 200 can use the sum of absolute differences (SAD), sum of squared differences (SSD), mean absolute difference (MAD), mean squared difference (MSD), or other such difference calculations to compute a difference metric to determine whether the reference block closely matches the current CU. In some examples, the video encoder 200 can use unidirectional or bidirectional prediction to predict the current CU.

[0066] Some examples of VVC also provide an affine motion compensation mode, which can be considered an inter-frame prediction mode. In affine motion compensation mode, the video encoder 200 can determine two or more motion vectors representing non-translational motion (such as zooming in or out, rotation, perspective motion, or other irregular types of motion).

[0067] To perform intra-frame prediction, the video encoder 200 can select an intra-frame prediction mode to generate prediction blocks. Some examples of VVC provide sixty-seven intra-frame prediction modes, including various directional modes, as well as planar and DC modes. Generally, the video encoder 200 selects an intra-frame prediction mode that describes the neighboring samples of the current block (e.g., a block of a CU), predicting samples of the current block from the neighboring samples. Assuming the video encoder 200 decodes the CTU and CU in raster scan order (from left to right, from top to bottom), such samples can typically be located above, to the upper left, or to the left of the current block within the same image.

[0068] The video encoder 200 encodes data representing the prediction mode of the current block. For example, for inter-frame prediction modes, the video encoder 200 may encode data indicating which of the various available inter-frame prediction modes is used, along with the motion information for the corresponding mode. For instance, for unidirectional or bidirectional inter-frame prediction, the video encoder 200 may encode motion vectors using Advanced Motion Vector Prediction (AMVP) or merging modes. The video encoder 200 may use similar modes to encode motion vectors for affine motion compensation modes.

[0069] AV1 includes two common techniques for encoding and decoding video data blocks. These two common techniques are intra-prediction (e.g., intra-frame prediction or spatial prediction) and inter-prediction (e.g., inter-frame prediction or temporal prediction). In the context of AV1, when predicting the current frame of video data using intra-prediction mode, the video encoder 200 and video decoder 300 do not use video data from other frames of the video data. For most intra-prediction modes, the video encoder 200 encodes the block of the current frame based on the difference between the sample values ​​in the current block and the predicted values ​​generated from reference samples in the same frame. The video encoder 200 determines the predicted values ​​generated from the reference samples based on the intra-prediction mode.

[0070] After prediction (such as intra-frame or inter-frame prediction for a block), the video encoder 200 can compute residual data for the block. The residual data (such as a residual block) represents the sample-by-sample difference between the block and the prediction block used to form the block, which is formed using the corresponding prediction mode. The video encoder 200 can apply one or more transforms to the residual block to produce transformed data in the transform domain rather than the sample domain. For example, the video encoder 200 can apply a Discrete Cosine Transform (DCT), an integer transform, a wavelet transform, or a conceptually similar transform to the residual video data. Additionally, the video encoder 200 can apply a secondary transform after the first transform, such as a Mode-dependent Inseparable Quadratic Transform (MDNSST), a Signal-dependent Transform, a Karhunen-Loeve Transform (KLT), etc. The video encoder 200 produces transform coefficients after applying one or more transforms.

[0071] As noted above, after any transformation that produces the transform coefficients, the video encoder 200 can perform quantization on the transform coefficients. Quantization generally refers to the process where the transform coefficients are quantized to reduce the amount of data used to represent them, thereby providing further compression. By performing the quantization process, the video encoder 200 can reduce the bit depth associated with some or all of the transform coefficients. For example, the video encoder 200 can reduce the bit depth associated with some or all of the transform coefficients during quantization. n The position value is rounded down to 1 m Bit value, where n Greater than m In some examples, in order to perform quantization, the video encoder 200 may perform a bitwise right shift on the value to be quantized.

[0072] After quantization, the video encoder 200 can scan the transform coefficients to produce a one-dimensional vector from a two-dimensional matrix including the quantized transform coefficients. The scan can be designed to place higher-energy (and therefore lower-frequency) transform coefficients before the vector and lower-energy (and therefore higher-frequency) transform coefficients after the vector. In some examples, the video encoder 200 can utilize a predefined scan order to scan the quantized transform coefficients to produce a serialized vector, and then entropy-encode the quantized transform coefficients of that vector. In other examples, the video encoder 200 can perform an adaptive scan. After scanning the quantized transform coefficients to form a one-dimensional vector, the video encoder 200 can entropy-encode the one-dimensional vector, for example, according to context-adaptive binary arithmetic decoding (CABAC). The video encoder 200 can also entropy-encode the values ​​of syntax elements describing metadata associated with the encoded video data for use by the video decoder 300 when decoding the video data.

[0073] To perform CABAC, the video encoder 200 can assign context within a context model to the symbols to be transmitted. Context may involve, for example, whether the neighboring values ​​of a symbol are zero. Probability determination can be based on the context assigned to the symbols.

[0074] The video encoder 200 can further generate syntax data (such as block-based syntax data, image-based syntax data, and sequence-based syntax data) or other syntax data (such as sequence parameter sets (SPS), image parameter sets (PPS), or video parameter sets (VPS)) for the video decoder 300, for example, in image headers, block headers, slice headers. The video decoder 300 can also decode such syntax data to determine how to decode the corresponding video data.

[0075] In this way, the video encoder 200 can generate a bitstream that includes encoded video data, such as syntax elements describing the partitioning of images into blocks (e.g., CUs) and prediction and / or residual information for the blocks. Finally, the video decoder 300 can receive the bitstream and decode the encoded video data.

[0076] Generally, the video decoder 300 performs the reverse process of the video encoder 200 to decode the encoded video data of the bitstream. For example, the video decoder 300 can use CABAC to decode the values ​​of the syntax elements used for the bitstream in a manner substantially similar to, but reversed, the CABAC encoding process of the video encoder 200. The syntax elements can define partitioning information for dividing the image into CTUs and for partitioning each CTU according to a corresponding partitioning structure such as the QTBT structure to define the CUs of the CTU. The syntax elements can further define prediction and residual information for video data blocks (e.g., CUs).

[0077] The residual information can be represented, for example, by quantized transform coefficients. The video decoder 300 can inversely quantize and inverse transform the quantized transform coefficients of the block to reconstruct the residual block for the block. The video decoder 300 uses a prediction mode (intra-frame prediction or inter-frame prediction) notified by signaling and associated prediction information (e.g., motion information for inter-frame prediction) to form a prediction block for the block. The video decoder 300 can then combine the prediction block and the residual block (on a sample-by-sample basis) to reconstruct the original block. The video decoder 300 can perform additional processing, such as performing a deblocking process to reduce visual artifacts along the block boundaries.

[0078] This disclosure may generally involve "signaling" certain information (such as syntax elements). The term "signaling" can generally refer to the transmission of values ​​for syntax elements and / or other data for decoding encoded video data. That is, video encoder 200 may signal the values ​​of syntax elements in the bitstream. Typically, signaling refers to generating a value in the bitstream. As noted above, source device 102 may transmit the bitstream to destination device 116 substantially in real time or not in real time (such as when syntax elements are stored in storage device 112 for later retrieval by destination device 116).

[0079] Figure 2 This is a block diagram illustrating an example high dynamic range (HDR) video coding system 140 according to the present disclosure. The HDR video coding system 140 can be incorporated into... Figure 1 In the source device 102.

[0080] In this example, the HDR video encoding system 140 includes a video source 104, a video format converter 130, a video encoder 200, an HDR statistics engine 132, a memory 133, a lookup table (LUT) 134, and a packaging unit 136. (See also: Regarding...) Figure 1The video source 104 discussed here can typically capture and / or generate video data, i.e., raw, undecoded video data. Raw, undecoded video data can be in red-green-blue (RGB) format.

[0081] Video format converter 130 is typically configured to convert raw, undecoded video data from video source 104 into one or more of a variety of different formats. For example, video format converter 130 can convert raw, undecoded video data into luminance and chrominance (YUV or YCbCr) formats for encoding by video encoder 200. According to the technology of this disclosure, video format converter 130 can convert raw, undecoded video data from video source 104 into linear light (LL) formats, such as RGB LL16.

[0082] The HDR statistics engine 132 can calculate a histogram based on one or more images in LL format. The HDR statistics engine 132 can store the histogram values ​​in memory 133. The HDR statistics engine 132 can use a LUT 134 to calculate the histogram. Generally, the LUT 134 maps sample values ​​of an image to index values, where the index values ​​correspond to bins of the histogram. The LUT 134 can be of size 2. 10 A 16-bit constant array. In one example, the HDR statistics engine 132 can calculate the index value as follows:

[0083] maxRGB = max (R, G, B), 16 bits P3 LL

[0084] if (maxRGB < 2)

[0085] index = maxRGB

[0086] else {

[0087] index1 = LUT1(log2(maxRGB))

[0088] index2 = isEqual(maxRGB, within LUT2(log2(maxRGB))

[0089] index = index1 + index2

[0090] }

[0091] Here, "index" represents the index value corresponding to the bin in the histogram, and max(R, G, B) returns the maximum value among samples at the same location for each of the red, green, and blue color components in the current image (e.g., maxRGB). The HDR statistics engine 132 can calculate a size of 2 10 Histogram in an array of 18 × 18 variables.

[0092] The HDR statistics engine 132 can increment the value of the bin corresponding to the determined index value in the histogram. For example, the HDR statistics engine 132 can perform a "hist[index]++" operation, where "hist[index]" represents the bin in the histogram, and the "++" operator increments the value of the corresponding bin by 1. The search complexity involved in obtaining the bin index can be O(10), since there may be 10 comparators for each calculation of maxRGB. The search can be a binary search, for example, a binary search tree.

[0093] After calculating the bin values ​​of the histogram, the HDR statistics engine 132 can determine the codewords for those bin values. The codewords can be selected from the same set of codewords used in the histogram calculated with the PQ10 format image. For example, the HDR statistics engine 132 can select codewords from the set of PQ10 format codewords used for the HDR image. As another example, the HDR statistics engine 132 can initially form a strictly possible subset of codewords from the set of PQ10 format codewords used for the HDR image, and then select codewords from the formed subset. The histogram bins do not necessarily represent a uniform width for the LL format values.

[0094] This disclosure recognizes that PQ / linear light defines the pixel luminance of a sample in the range of 0 to 10,000 nits. HDR10+ specifies up to 0.1 candela per square meter (cd / m²). 2 The PQ10 has 1024 codewords to cover a range of 10,000 possible values ​​with a granularity of 0.1 cd / m. 2 Therefore, the histogram calculated in the PQ10 domain is suboptimal for HDR10+. Instead, according to the techniques of this disclosure, the HDR statistics engine 132 can alternatively calculate the histogram in the RGB P3 linear light domain. The quality of the HDR10 histogram in the PQ10 domain can be achieved with fewer than 1023 bins in the linear light domain. The histogram index can be calculated by downscaling and right shifting related to the values. For example, the HDR statistics engine 132 can perform the following operations:

[0095] Calculate N ranges: TS[N]

[0096] For example, N can be 1023 or less.

[0097] In some examples, N may be as low as 650.

[0098] Process flow:

[0099] Compare the linear maxRGB value with TS[i].

[0100] The range index is derived through a binary search or analytic process.

[0101] The range index is used as the histogram index hi = function(maxRGB, TS[i])

[0102] histLinRGB[hi]++

[0103] Hardware implementation:

[0104] TS LUT: Nx18 bits

[0105] Histogram: 1024x18 bits (i.e., 2) 10 (x 18 bits)

[0106] Perform a binary search on the LUT fragment (up to 40 LUT entries).

[0107] The video encoder 200 can encode each image in the image used to calculate the histogram. The encapsulation unit 136 can encapsulate the encoded image (e.g., an encoded slice of an image) and the encoded histogram from the HDR statistics engine 132 (e.g., determined codewords or encoded versions of codewords). Finally, the encapsulation unit 136 can form a bitstream 138 including data for the encoded image and the encoded histogram.

[0108] Each of the video format converter 130, the HDR statistics engine 132, and the packaging unit 136 can be implemented in one or more processing units, i.e., one or more hardware-based processors implemented in circuitry. For example, the processing unit may include one or more circuit or logic elements as part of hardware circuitry, or as part of a processor, ASIC, or FPGA. The memory 133 and LUT 134 may correspond to various storage devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), solid-state storage devices, or other types of storage devices.

[0109] In this way, video coding system 140 represents an example of a device for encoding high dynamic range (HDR) video data, comprising: a memory configured to store histogram data; and one or more processors implemented in circuitry and configured to: calculate a histogram of an image for video data, the image being represented in a linear light format; encode the values ​​of the histogram for the image represented in the linear light format; and encode the image.

[0110] Figure 3 This is a schematic diagram 150 illustrating an example set of bins for calculating a histogram according to the techniques of this disclosure. The X-axis 152 represents possible index values ​​for the bins, while the Y-axis 154 represents possible maxRGB values. A stepwise function 156 approximates to curve 158. Curve 158 represents the codeword for PQ10, while the stepwise function 156 represents the codeword calculated according to maxRGB using a linear light format according to the techniques of this disclosure.

[0111] It can be seen that the different steps of the step function 156 have different widths. For example, as Figure 3 As shown, the range of the step function 156 closer to the left of schematic diagram 150 is narrower than that of the step function 156 closer to the right of schematic diagram 150. Therefore, a first range of maxRGB values ​​may be mapped to a single bin of the histogram, and a second range of maxRGB values ​​may be mapped to different bins of the histogram, wherein the second range may be wider than the first range.

[0112] Figure 4 This is a block diagram illustrating an example video encoder 200 capable of performing video encoding according to the technology described in this disclosure. Figure 4 This disclosure is provided for illustrative purposes and should not be construed as limiting the techniques illustrated and described in this disclosure in a general manner. For illustrative purposes, this disclosure describes a video encoder 200 based on VVC (ITU-T H.266, under development) and HEVC (ITU-T H.265) technologies. However, the techniques of this disclosure can be implemented by video encoding devices configured for other video decoding standards and video decoding formats, such as AV1 and subsequent AV1 video decoding formats.

[0113] exist Figure 4In the example, the video encoder 200 includes a video data memory 230, a mode selection unit 202, a residual generation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a filter unit 216, a decoded picture buffer (DPB) 218, and an entropy coding unit 220. Any or all of the units in the video data memory 230, mode selection unit 202, residual generation unit 204, transform processing unit 206, quantization unit 208, inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, filter unit 216, DPB 218, and entropy coding unit 220 can be implemented in one or more processors or in processing circuitry. For example, the units of the video encoder 200 can be implemented as one or more circuit or logic elements as part of hardware circuitry, or as part of a processor, ASIC, or FPGA. Furthermore, the video encoder 200 may include additional or alternative processors or processing circuitry to perform these and other functions.

[0114] The video data storage device 230 can store video data to be encoded by the components of the video encoder 200. The video encoder 200 can obtain data from, for example, a video source 104 (…). Figure 1 The video data memory 230 receives video data stored in the video data memory 230. The DPB 218 can act as a reference picture memory, storing reference video data for use when the video encoder 200 predicts subsequent video data. The video data memory 230 and DPB 218 can be formed from any of a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), solid-state storage devices, or other types of memory devices. The video data memory 230 and DPB 218 can be provided by the same memory device or separate memory devices. In various examples, the video data memory 230 can be on-chip (as shown) with other components of the video encoder 200, or off-chip relative to those components.

[0115] In this disclosure, references to video data memory 230 should not be construed as limited to memory inside video encoder 200 (unless specifically described as such) or memory outside video encoder 200 (unless specifically described as such). Rather, references to video data memory 230 should be understood as reference memory storing video data received by video encoder 200 for encoding (e.g., video data for the current block to be encoded). Figure 1 The memory 106 can also provide temporary storage for the outputs from various units of the video encoder 200.

[0116] Show Figure 4 The various units help to understand the operations performed by the video encoder 200. The units can be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Fixed-function circuits are circuits that provide a specific function and are pre-defined in terms of the operations they can perform. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functionality in the operations they can perform. For example, a programmable circuit can execute software or firmware that causes the programmable circuit to operate in a manner defined by the instructions of the software or firmware. Fixed-function circuits can execute software instructions (e.g., receiving or outputting parameters), but the type of operation performed by a fixed-function circuit is generally immutable. In some examples, one or more units in the unit may be different circuit blocks (fixed-function or programmable), and in some examples, one or more units in the unit may be integrated circuits.

[0117] The video encoder 200 may include an arithmetic logic unit (ALU), an essential function unit (EFU), digital circuitry, analog circuitry, and / or a programmable core, all formed by programmable circuitry. In an example where the operation of the video encoder 200 is performed using software executed by programmable circuitry, memory 106 ( Figure 1 The video encoder 200 may store instructions (e.g., object code) of the software received and executed by the video encoder 200, or another memory (not shown) within the video encoder 200 may store such instructions.

[0118] The video data storage unit 230 is configured to store received video data. The video encoder 200 can retrieve images of the video data from the video data storage unit 230 and provide the video data to the residual generation unit 204 and the mode selection unit 202. The video data in the video data storage unit 230 can be the raw video data to be encoded.

[0119] The mode selection unit 202 includes a motion estimation unit 222, a motion compensation unit 224, and an intra-frame prediction unit 226. The mode selection unit 202 may include additional functional units for performing video prediction based on other prediction modes. As an example, the mode selection unit 202 may include a palette unit, an intra-frame block copying unit (which may be part of the motion estimation unit 222 and / or the motion compensation unit 224), an affine unit, a linear model (LM) unit, etc.

[0120] The mode selection unit 202 typically coordinates multiple encoding passes to test combinations of encoding parameters and the resulting rate-distortion values ​​for such combinations. Encoding parameters may include the CTU-CU partitioning, the prediction mode used for the CU, the transformation type of the residual data used for the CU, and the quantization parameters of the residual data used for the CU. The mode selection unit 202 can ultimately select a combination of encoding parameters that has a better rate-distortion value compared to other tested combinations.

[0121] The video encoder 200 can divide the image retrieved from the video data storage 230 into a series of CTUs and encapsulate one or more CTUs within a slice. The mode selection unit 202 can divide the image's CTUs according to a tree structure (such as the MTT structure, QTBT structure, superblock structure, or quadtree structure described above). As described above, the video encoder 200 can form one or more CUs by dividing the CTUs according to a tree structure. Such CUs can also be referred to as "video blocks" or "blocks".

[0122] Generally, mode selection unit 202 also controls its components (e.g., motion estimation unit 222, motion compensation unit 224, and intra-prediction unit 226) to generate predicted blocks for the current block (e.g., the current CU, or, in HEVC, the overlapping portion of PU and TU). To perform inter-frame prediction for the current block, motion estimation unit 222 can perform a motion search to identify one or more closely matching reference blocks in one or more reference pictures (e.g., one or more previously decoded pictures stored in DPB 218). Specifically, motion estimation unit 222 can calculate values ​​representing the similarity between a potential reference block and the current block, for example, based on sum of absolute differences (SAD), sum of squared differences (SSD), mean absolute difference (MAD), mean squared difference (MSD), etc. Motion estimation unit 222 can typically perform these calculations using sample-by-sample differences between the current block and the considered reference blocks. Motion estimation unit 222 can identify reference blocks with the lowest values ​​produced by these calculations, which indicate the reference block that best matches the current block.

[0123] Motion estimation unit 222 can generate one or more motion vectors (MVs) that define the position of a reference block in a reference image relative to the position of the current block in the current image. Motion estimation unit 222 can then provide the motion vectors to motion compensation unit 224. For example, for unidirectional inter-frame prediction, motion estimation unit 222 can provide a single motion vector, while for bidirectional inter-frame prediction, motion estimation unit 222 can provide two motion vectors. Motion compensation unit 224 can then use the motion vectors to generate a prediction block. For example, motion compensation unit 224 can use the motion vectors to retrieve data for the reference block. As another example, if the motion vectors have fractional-sample precision, motion compensation unit 224 can interpolate values ​​for the prediction block according to one or more interpolation filters. Furthermore, for bidirectional inter-frame prediction, motion compensation unit 224 can retrieve data for two reference blocks identified by corresponding motion vectors and combine the retrieved data, for example, by per-sample averaging or weighted averaging.

[0124] When operating according to the AV1 video decoding format, the motion estimation unit 222 and the motion compensation unit 224 can be configured to encode the decoded blocks of video data (e.g., both luma and chroma decoded blocks) using translational motion compensation, affine motion compensation, overlap block motion compensation (OBMC), and / or composite inter-intra-frame prediction.

[0125] As another example, for intra-prediction or intra-prediction decoding, intra-prediction unit 226 may generate a prediction block from samples adjacent to the current block. For example, for directional mode, intra-prediction unit 226 may typically mathematically combine the values ​​of adjacent samples and fill these calculated values ​​across the current block in a defined direction to generate a prediction block. As another example, for DC mode, intra-prediction unit 226 may calculate the average of the adjacent samples of the current block and generate a prediction block to include the obtained average for each sample of the prediction block.

[0126] When operating according to the AV1 video decoding format, the intra-prediction unit 226 can be configured to encode decoded blocks of video data (e.g., both luma and chroma decoded blocks) using directional intra-prediction, non-directional intra-prediction, recursive filter intra-prediction, chroma-based (CFL) prediction, intra-block copying (IBC), and / or palette modes. The mode selection unit 202 may include additional functional units for performing video prediction based on other prediction modes.

[0127] Mode selection unit 202 provides the prediction block to residual generation unit 204. Residual generation unit 204 receives the original, undecoded version of the current block from video data memory 230 and the prediction block from mode selection unit 202. Residual generation unit 204 calculates the sample-by-sample difference between the current block and the prediction block. The resulting sample-by-sample difference defines the residual block for the current block. In some examples, residual generation unit 204 can also determine the differences between sample values ​​in the residual block to generate the residual block using residual differential pulse decode-modulation (RDPCM). In some examples, residual generation unit 204 can be formed using one or more subtractor circuits that perform binary subtraction.

[0128] In the example where mode selection unit 202 divides a CU into PUs, each PU can be associated with a luma prediction unit and a corresponding chroma prediction unit. Video encoder 200 and video decoder 300 can support PUs of various sizes. As noted above, the size of a CU can refer to the size of the luma decoding block of the CU, while the size of a PU can refer to the size of the luma prediction unit of the PU. Assuming a particular CU size is 2Nx2N, video encoder 200 can support PU sizes of 2Nx2N or NxN for intra-frame prediction, and symmetric PU sizes of 2Nx2N, 2NxN, Nx2N, NxN, or similar for inter-frame prediction. Video encoder 200 and video decoder 300 can also support asymmetric partitioning of PU sizes of 2NxnU, 2NxnD, nLx2N, and nRx2N for inter-frame prediction.

[0129] In an example where mode selection unit 202 does not further divide the CU into PUs, each CU can be associated with a luminance decoding block and a corresponding chrominance decoding block. As mentioned above, the size of the CU can refer to the size of the luminance decoding block of the CU. The video encoder 200 and the video decoder 300 can support CU sizes of 2Nx2N, 2NxN, or Nx2N.

[0130] For other video decoding techniques (such as intra-block copy mode decoding, affine mode decoding, and linear model (LM) mode decoding, mode selection unit 202 generates a prediction block for the current block being encoded via a corresponding unit associated with the decoding technique. In some examples (such as palette mode decoding), mode selection unit 202 may not generate a prediction block, but instead generate syntax elements indicating how the block is reconstructed based on the selected palette. In such modes, mode selection unit 202 may provide these syntax elements to entropy coding unit 220 for encoding.

[0131] As described above, the residual generation unit 204 receives video data for the current block and the corresponding prediction block. Then, the residual generation unit 204 generates a residual block for the current block. To generate the residual block, the residual generation unit 204 calculates the sample-by-sample difference between the prediction block and the current block.

[0132] Transform processing unit 206 applies one or more transformations to the residual block to generate a block of transform coefficients (referred to herein as a "transform coefficient block"). Transform processing unit 206 may apply various transformations to the residual block to form the transform coefficient block. For example, transform processing unit 206 may apply a discrete cosine transform (DCT), direction transform, Karhunen-Loeve transform (KLT), or conceptually similar transformations to the residual block. In some examples, transform processing unit 206 may perform multiple transformations on the residual block, such as primary and secondary transformations (e.g., rotation transformations). In some examples, transform processing unit 206 does not apply any transformations to the residual block.

[0133] When operating according to AV1, the transform processing unit 206 can apply one or more transforms to the residual block to generate a block of transform coefficients (referred to herein as a "transform coefficient block"). The transform processing unit 206 can apply various transforms to the residual block to form the transform coefficient block. For example, the transform processing unit 206 can apply a combination of horizontal / vertical transforms, which may include the Discrete Cosine Transform (DCT), the Asymmetric Discrete Sine Transform (ADST), the Reversed ADST (e.g., the ADST in reverse order), and the Identity Transform (IDTX). When using the Identity Transform, the transform is skipped in either the vertical or horizontal direction. In some examples, transform processing may be skipped.

[0134] Quantization unit 208 can quantize the transform coefficients in a transform coefficient block to produce a quantized transform coefficient block. Quantization unit 208 can quantize the transform coefficients of the transform coefficient block based on the QP value associated with the current block. Video encoder 200 (e.g., via mode selection unit 202) can adjust the degree of quantization applied to the transform coefficient block associated with the current block by adjusting the QP value associated with the CU. Quantization may cause information loss, and therefore, the quantized transform coefficients may have lower accuracy compared to the original transform coefficients produced by transform processing unit 206.

[0135] The inverse quantization unit 210 and the inverse transform processing unit 212 can apply inverse quantization and inverse transform to the quantized transform coefficient block, respectively, to reconstruct the residual block based on the transform coefficient block. The reconstruction unit 214 can generate a reconstructed block corresponding to the current block (although potentially with some degree of distortion) based on the reconstructed residual block and the prediction block generated by the mode selection unit 202. For example, the reconstruction unit 214 can add samples from the reconstructed residual block to corresponding samples from the prediction block generated by the mode selection unit 202 to generate the reconstructed block.

[0136] Filter unit 216 can perform one or more filtering operations on the reconstructed block. For example, filter unit 216 can perform deblocking operations to reduce blockiness artifacts along the edges of the CU. In some examples, the operations of filter unit 216 can be skipped.

[0137] When operating according to AV1, filter unit 216 may perform one or more filtering operations on the reconstructed block. For example, filter unit 216 may perform a deblocking operation to reduce block artifacts along the edges of the CU. In other examples, filter unit 216 may apply a constrained directional enhancement filter (CDEF), which may be applied after deblocking, and may include the application of an inseparable, nonlinear, low-pass directional filter based on the estimated edge direction. Filter unit 216 may also include a loop recovery filter, which is applied after the CDEF, and may include a separable symmetric normalized Wiener filter or a dual-guided filter.

[0138] The video encoder 200 stores reconstructed blocks in the DPB 218. For example, in an example where the filter unit 216 is not operated, the reconstruction unit 214 may store reconstructed blocks in the DPB 218. In an example where the filter unit 216 is operated, the filter unit 216 may store filtered reconstructed blocks in the DPB 218. The motion estimation unit 222 and the motion compensation unit 224 may retrieve a reference image formed by the reconstructed (and potentially filtered) blocks from the DPB 218 to perform inter-frame prediction of blocks in subsequently encoded images. Additionally, the intra-frame prediction unit 226 may use the reconstructed blocks of the current image in the DPB 218 to perform intra-frame prediction of other blocks in the current image.

[0139] Typically, entropy coding unit 220 can entropy-encode syntax elements received from other functional components of video encoder 200. For example, entropy coding unit 220 can entropy-encode quantized transform coefficient blocks from quantization unit 208. As another example, entropy coding unit 220 can entropy-encode predictive syntax elements (e.g., motion information for inter-frame prediction or intra-frame mode information for intra-frame prediction) from mode selection unit 202. Entropy coding unit 220 can perform one or more entropy coding operations on syntax elements (which is another example of video data) to generate entropy-encoded data. For example, entropy coding unit 220 can perform context-adaptive variable-length decoding (CAVLC), CABAC, variable-to-variable (V2V) length decoding, syntax-based context-adaptive binary arithmetic decoding (SBAC), probability interval partitioning entropy (PIPE) decoding, exponential Golomb coding, or another type of entropy coding operation on the data. In some examples, entropy coding unit 220 can operate in a bypass mode where syntax elements are not entropy-encoded.

[0140] The video encoder 200 can output a bitstream containing entropy-encoded syntax elements required to reconstruct slices or blocks of images. Specifically, the entropy coding unit 220 can output a bitstream.

[0141] According to AV1, entropy coding unit 220 can be configured as a symbol-to-symbol adaptive multi-symbol arithmetic decoder. The syntax elements in AV1 consist of an N-element alphabet, and the context (e.g., a probability model) consists of a set of N probabilities. Entropy coding unit 220 can store the probabilities as an n-bit (e.g., 15-bit) cumulative distribution function (CDF). Entropy coding unit 220 can perform recursive scaling (with an update factor based on letter size) to update the context.

[0142] The operations described above are relative to blocks. Such descriptions should be understood as operations applied to luma decoding blocks and / or chroma decoding blocks. As described above, in some examples, the luma decoding block and chroma decoding block are the luma and chroma components of the CU. In some examples, the luma decoding block and chroma decoding block are the luma and chroma components of the PU.

[0143] In some examples, it is not necessary to repeat the operations performed for the luma-coded block for the chroma-decoded block. As one example, it is not necessary to repeat the operations of identifying the motion vector (MV) and reference image of the luma-decoded block to identify the MV and reference image of the chroma block. Instead, the MV used for the luma-decoded block can be scaled to determine the MV used for the chroma block, and the reference image can be the same. As another example, the intra-frame prediction process can be the same for both the luma-decoded block and the chroma-decoded block.

[0144] Figure 5 This is a block diagram illustrating an example video decoder 300 capable of performing video decoding according to the technology described in this disclosure. Figure 5 This disclosure is provided for illustrative purposes and is not intended to limit the techniques extensively illustrated and described herein. For illustrative purposes, this disclosure describes a video decoder 300 based on VVC (ITU-T H.266, under development) and HEVC (ITU-TH.265) technologies. However, the technologies of this disclosure can be implemented by video decoding devices configured for other video decoding standards.

[0145] exist Figure 5 In the example, the video decoder 300 includes a decoded picture buffer (CPB) memory 320, an entropy decoding unit 302, a prediction processing unit 304, an inverse quantization unit 306, an inverse transform processing unit 308, a reconstruction unit 310, a filter unit 312, and a decoded picture buffer (DPB) 134. Any or all of the units of the CPB memory 320, entropy decoding unit 302, prediction processing unit 304, inverse quantization unit 306, inverse transform processing unit 308, reconstruction unit 310, filter unit 312, and DPB 134 can be implemented in one or more processors or in processing circuitry. For example, the units of the video decoder 300 can be implemented as one or more circuit or logic elements as part of hardware circuitry, or as part of a processor, ASIC, or FPGA. Furthermore, the video decoder 300 may include additional or alternative processors or processing circuitry to perform these and other functions.

[0146] The prediction processing unit 304 includes a motion compensation unit 316 and an intra-prediction unit 318. The prediction processing unit 304 may include additional units that perform predictions based on other prediction modes. As an example, the prediction processing unit 304 may include a palette unit, an intra-block copying unit (which may form part of the motion compensation unit 316), an affine unit, a linear model (LM) unit, etc. In other examples, the video decoder 300 may include more, fewer, or different functional components.

[0147] When operating according to AV1, compensation unit 316 can be configured to decode decoded blocks of video data (e.g., both luma and chroma decoded blocks) using translational motion compensation, affine motion compensation, OBMC, and / or composite inter-intra-frame prediction, as described above. Intra-frame prediction unit 318 can be configured to decode decoded blocks of video data (e.g., both luma and chroma decoded blocks) using directional intra-frame prediction, non-directional intra-frame prediction, recursive filter intra-frame prediction, CFL, intra block copying (IBC), and / or palette mode, as described above.

[0148] CPB memory 320 can store video data to be decoded by components of video decoder 300, such as encoded video bitstreams. For example, it can be stored from computer-readable medium 110 ( Figure 1 The video data stored in the CPB memory 320 is obtained. The CPB memory 320 may include a CPB that stores encoded video data (e.g., syntax elements) from the encoded video bitstream. Additionally, the CPB memory 320 may store video data other than the syntax elements of the decoded picture, such as temporary data representing the output of various units from the video decoder 300. The DPB 314 typically stores decoded pictures, which the video decoder 300 may output and / or use as reference video data when decoding subsequent data or pictures from the encoded video bitstream. The CPB data memory 320 and the DPB 314 may be formed from any of a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), solid-state storage devices, or other types of memory devices. The CPB memory 320 and the DPB 314 may be provided by the same memory device or separate memory devices. In various examples, the CPB memory 320 may be on-chip along with other components of the video decoder 300, or off-chip relative to those components.

[0149] Additionally or alternatively, in some examples, the video decoder 300 can be generated from the memory 120 ( Figure 1 The decoded video data is retrieved from the memory. In other words, memory 120 can utilize CPB memory 320 to store data, as discussed above. Similarly, when some or all of the functions of video decoder 300 are implemented using software to be executed by the processing circuitry of video decoder 300, memory 120 can store instructions to be executed by video decoder 300.

[0150] Shown in Figure 5 The various units shown below help to understand the operations performed by the video decoder 300. The units can be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Similar to... Figure 4Fixed-function circuits refer to circuits that provide a specific function and are pre-configured regarding the operations that can be performed. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functionality in the operations that can be performed. For example, a programmable circuit can execute software or firmware that causes the programmable circuit to operate in a manner defined by the instructions of the software or firmware. Fixed-function circuits can execute software instructions (e.g., receive or output parameters), but the type of operation performed by a fixed-function circuit is typically immutable. In some examples, one or more units in the unit may be different circuit blocks (fixed-function or programmable), and in some examples, one or more units in the unit may be integrated circuits.

[0151] The video decoder 300 may include an ALU, EFU, digital circuitry, analog circuitry, and / or a programmable core formed by programmable circuitry. In an example where the operation of the video decoder 300 is performed by software executed on programmable circuitry, on-chip or off-chip memory may store instructions (e.g., object code) of the software received and executed by the video decoder 300.

[0152] Entropy decoding unit 302 can receive encoded video data from the CPB and perform entropy decoding on the video data to reproduce the syntax elements. Prediction processing unit 304, inverse quantization unit 306, inverse transform processing unit 308, reconstruction unit 310, and filter unit 312 can generate decoded video data based on the syntax elements extracted from the bitstream.

[0153] Typically, the video decoder 300 reconstructs the image block by block. The video decoder 300 can perform the reconstruction operation on each block individually (where the block currently being reconstructed (i.e., decoded) can be referred to as the "current block").

[0154] Entropy decoding unit 302 can entropy decode the syntax elements of the quantized transform coefficients defining the quantized transform coefficient block, as well as transform information such as quantization parameters (QP) and / or transform mode indications. Inverse quantization unit 306 can use the QP associated with the quantized transform coefficient block to determine the degree of quantization, and similarly, determine the degree of inverse quantization to be applied by inverse quantization unit 306. Inverse quantization unit 306 can, for example, perform a bitwise left shift operation to inverse quantize the quantized transform coefficients. Inverse quantization unit 306 can thereby form a transform coefficient block including the transform coefficients.

[0155] After the inverse quantization unit 306 forms the transform coefficient block, the inverse transform processing unit 308 can apply one or more inverse transforms to the transform coefficient block to generate a residual block associated with the current block. For example, the inverse transform processing unit 308 can apply an inverse DCT, an inverse integer transform, an inverse Karhunen-Loeve transform (KLT), an inverse rotation transform, an inverse direction transform, or another inverse transform to the transform coefficient block.

[0156] Furthermore, prediction processing unit 304 generates prediction blocks based on prediction information syntax elements entropy-decoded by entropy decoding unit 302. For example, if the prediction information syntax elements indicate that the current block is inter-frame predicted, motion compensation unit 316 can generate prediction blocks. In this case, the prediction information syntax elements may indicate a reference picture in DPB 314 from which the reference block is to be retrieved, and a motion vector identifying the position of the reference block in the reference picture relative to the position of the current block in the current picture. Motion compensation unit 316 can generally be configured in a manner substantially similar to that of motion compensation unit 224 ( Figure 4 The method described herein is used to perform the inter-frame prediction process.

[0157] As another example, if the prediction information syntax element indicates that the current block is intra-predictive, then intra-predictive unit 318 can generate a prediction block according to the intra-predictive mode indicated by the prediction information syntax element. Similarly, intra-predictive unit 318 can typically be configured with respect to intra-predictive unit 226 ( Figure 4 The intra-prediction process is performed in a substantially similar manner to that described above. The intra-prediction unit 318 can retrieve data from neighboring samples of the current block from the DPB 314.

[0158] Reconstruction unit 310 can reconstruct the current block using the prediction block and the residual block. For example, reconstruction unit 310 can add the samples of the residual block to the corresponding samples of the prediction block to reconstruct the current block.

[0159] Filter unit 312 can perform one or more filter operations on the reconstructed block. For example, filter unit 312 can perform a deblocking operation to reduce block artifacts along the edges of the reconstructed block. The operation of filter unit 312 is not necessarily performed in all examples.

[0160] The video decoder 300 can store reconstructed blocks in the DPB 314. For example, in an example where the operation of the filter unit 312 is not performed, the reconstruction unit 310 can store reconstructed blocks in the DPB 314. In an example where the operation of the filter unit 312 is performed, the filter unit 312 can store filtered reconstructed blocks in the DPB 314. As discussed above, the DPB 314 can provide reference information (such as samples of the current image for intra-frame prediction and samples of previously decoded images for subsequent motion compensation) to the prediction processing unit 304. Furthermore, the video decoder 300 can output decoded images (e.g., decoded video) from the DPB 314 for use in applications such as... Figure 1 The subsequent presentation on display devices such as display device 118.

[0161] Figure 6 This is a flowchart illustrating an example method for encoding image and histogram data using techniques based on this disclosure. Figure 6 The method is explained in relation to video coding system 140, including... Figure 2 The HDR statistics engine 132. However, other devices or systems can be configured to perform this method or a similar method.

[0162] Initially, the video format converter 130 can convert one or more images to a linear light format (350). The HDR statistics engine 132 can then determine the range of bin values ​​for the histogram to be calculated for the one or more images (352). For example, the range may correspond to... Figure 4 The stepwise function has 156 steps.

[0163] The HDR statistics engine 132 can compute a histogram, including incrementing bins (354) for each sample of an image in linear light format. For example, as described above, the HDR statistics engine 132 can compute the maximum value of a sample of the red, green, and blue components of the image, use the maximum value to determine the corresponding bin of the histogram, and then increment the value of that bin by 1. The HDR statistics engine 132 can do this for each sample of each image.

[0164] The HDR statistics engine 132 can also determine the set of possible codewords for the bin values ​​of the histogram (356). As described above, the set of possible codewords for the histogram calculated from the LL format image can be selected from the PQ10 format codewords, for example, as a subset of the PQ10 format codewords. After calculating the histogram and determining the set of possible codewords, the HDR statistics engine 132 can determine the codeword for each bin value of the histogram (358).

[0165] The video encoder 200 can also encode images (360), for example, as mentioned above.Figure 1 and Figure 4 As explained, for example, video encoder 200 can divide an image into blocks and predictively encode each block using, for example, intra-frame or inter-frame prediction. After predicting the blocks, video encoder 200 can compute residual blocks to represent the differences between the original blocks and the predicted blocks. Video encoder 200 can further encode the residual blocks, including transforming the residual blocks into transform-domain blocks and quantizing the transform coefficients. Video encoder 200 can then entropy encode the syntax element values ​​used for the quantized transform coefficients and the prediction information.

[0166] The HDR statistics engine 132 can also encode codewords for the bin values ​​of the histogram (362). For example, the HDR statistics engine 132 can encode codewords similar to... Figure 4 The codewords are entropy encoded by the entropy encoding unit 220. Optionally, the HDR statistics engine 132 can pass the codewords to the video encoder 200 for entropy encoding by the entropy encoding unit 220. Finally, the encapsulation unit 136 can form a bitstream 138 (364) including the encoded image and codewords.

[0167] In this way, Figure 6 The method represents an example of a method for encoding high dynamic range (HDR) video data, including: calculating a histogram of an image of the video data, represented in a linear light format; encoding the values ​​of the histogram of the image represented in a linear light format; and encoding the image.

[0168] Figure 7A and 7B This is a conceptual diagram illustrating an example of HDR and WCG representation conversion. Specifically, Figure 7A The example encoder side represents the conversion process 400, while Figure 7B The example decoder-side representation conversion process is depicted. HDR / WCG is typically captured under linear light and stored with high precision per component (e.g., RGB 16-bit fixed or even floating-point representation) and a very wide color space (BT.2020). This representation aims for high precision and is (almost) mathematically lossless. However, this format has many redundancies and is not the optimal choice for compression purposes. Lower-precision formats with HVS assumptions are typically used for state-of-the-art video applications.

[0169] Typically, HDR video data format conversion for compression purposes involves three main elements: a non-linear transfer function (TF) for dynamic range compression (e.g., BT.2100 PQ or BT.2100 HLG), color conversion to a more compact or robust color space (e.g., P3), and floating-point to integer representation conversion / quantization (e.g., 10-bit fixed-point representation).

[0170] exist Figure 7A In the example, the video data was initially presented in linear light RGB format 402. As mentioned above, the HDR / WCG conversion process typically involves three stages. Figure 7A In this example, the three stages include a photoelectric transfer function (OETF) stage 404, a color conversion stage 406, and a quantization stage 408. These three stages process video data 402 in linear light RGB format to produce data 410 in a 10-bit representation. In one example, the video data 402 in linear light RGB format can have a high dynamic range. This high dynamic range of the input RGB data can be represented using both linear and floating-point representations. The non-linear OETF stage 404 can compress the high dynamic range, for example, using QP TF as defined in BT.2100 or SMPTE-2084. The color conversion stage 406 can convert the RGB representation to a target color space, such as YCbCr. The quantization stage 408 can quantize sample values ​​to achieve an integer representation.

[0171] exist Figure 7B In the example, video data 422, initially represented in 10 bits, is received. The three stages in this example include an inverse quantization stage 424, an inverse color conversion stage 426, and an inverse OETF stage 428. These three stages process the 10-bit video data 422 to reproduce the video data 430 in linear light RGB format.

[0172] The order of these elements is given as an example and may differ in real-world applications. For instance, color conversion may precede the TF module. In some examples, additional processing, such as spatial subsampling applied to color components, may be performed.

[0173] Figure 8 This is a schematic diagram illustrating various examples of transfer functions. In particular, Figure 8Example SDR transfer function 442 and example HDR transfer function 444 are described. A transfer function (TF) is applied to video data to compress its dynamic range and make it possible to represent the video data with a finite number of bits. This function is typically a one-dimensional (1D) non-linear function that reflects the inverse of the electro-optical transfer function (EOTF) of the end-user display, as specified for SDR in Rec.709. For HDR signals, one of the most popular formats is defined by the photoelectric transfer function (OETF), which approximates human visual system (HVS) perception of changes in brightness, such as the PQ TF specified for HDR in BT.2100 (also known as SMPTE-2084). The inverse process of the OETF is the EOTF (electro-optical transfer function), which maps the code level back to luminance. This non-linearity is the basis for PQ10, HDR10, Dolby Vision, and HDR10+ video formats.

[0174] The specification for the PQ transfer function is defined as EOTF in ST 2084. Its inverse, OETF, is applied to normalized linear R, G, B values, resulting in a nonlinear representation of R'G'B'. PQ is normalized by NORM=10000, which is associated with a peak brightness of 10000 nits (cd / m²). For the purposes of this disclosure, this nonlinearity is referred to as PQ_OETF. This nonlinearity can be defined as follows:

[0175] R' = PQ_OETF(max(0, min(R / NORM,1)) )

[0176] G' = PQ_OETF(max(0, min(G / NORM,1)) )

[0177] B' = PQ_OETF(max(0, min(B / NORM,1)) )

[0178] in (1)

[0179]

[0180]

[0181]

[0182]

[0183]

[0184] Figure 9 This is a schematic diagram 450 showing an example PQ OETF. Figure 9The example illustrates a visualization of the PQ OETF using input values ​​(linear color values) normalized to the 0..1 range and output values ​​(non-linear color values) normalized to the same range. As can be seen from diagram 450, a relatively small number of low-brightness values ​​from the dynamic range of the input signal are converted to 50% of the dynamic range of the output signal.

[0185] The PQ EOTF specified in ST2084 can be defined as follows:

[0186] R = 10000 PQ_EOTF(R')

[0187] G = 10000 PQ_EOTF(G')

[0188] B = 10000 PQ_EOTF(B')

[0189] in (2)

[0190]

[0191]

[0192]

[0193]

[0194]

[0195] With their floating-point precision, sequential applications of EOTF and OETF provide perfect reconstruction in the absence of errors. However, this representation is not optimal for streaming or broadcast services. A more compact representation with fixed-point bit precision for nonlinear R'G'B' data is described below.

[0196] RGB data is commonly used as input because it is generated from image capture sensors. However, the RGB color space has high redundancy among its components and is not optimal for compact representation. To achieve a more compact and robust representation, RGB components are often converted to a less correlated color space, such as YCbCr, which is more suitable for compression. This color space separates brightness into different uncorrelated chromaticity components, such as red and blue hue components, in the form of luminance and color information.

[0197] For modern video decoding systems, the commonly used color space is ITU-R BT.709 or YCbCr as specified in ITU-R BT.709. The YCbCr color space in the BT.709 standard specifies the following conversion process from R'G'B' to Y'CbCr (non-constant luminance representation):

[0198] Y' = 0.2126 R' + 0.7152 G' + 0.0722 B'

[0199] (3)

[0200]

[0201] The above method can also be implemented using the following approximate transformation to avoid division for Cb and Cr components:

[0202] Y' = 0.212600 R' + 0.715200 G' + 0.072200 B'

[0203] Cb = -0.114572 R' - 0.385428 G' + 0.500000 B' (4)

[0204] Cr = 0.500000 R' - 0.454153 G' - 0.045847 B'

[0205] The ITU-R BT.2020 standard specifies the following conversion process from R'G'B' to Y'CbCr (a non-constant luminance representation):

[0206] Y' = 0.2627 R' + 0.6780 G' + 0.0593 B'

[0207] (5)

[0208]

[0209] The above method can also be implemented using the following approximate transformation to avoid division for Cb and Cr components:

[0210] Y' = 0.262700 R' + 0.678000 G' + 0.059300 B'

[0211] Cb = -0.139630 R' - 0.360370 G' + 0.500000 B' (6)

[0212] Cr = 0.500000 R' - 0.459786 G' - 0.040214 B'

[0213] Both color spaces can be normalized. Therefore, for an input value normalized to the range 0...1, the resulting value will be mapped to the range 0..1. Typically, color transformations implemented with floating-point precision can provide perfect reconstruction, so the process can be lossless.

[0214] The aforementioned color transformation and nonlinear transformation function processing stages can be implemented using floating-point precision. Therefore, these processes can be implemented losslessly. However, for most consumer electronics applications, this type of precision may be considered redundant and expensive. For such services, the input data in the target color space can be converted to target bit-depth fixed-point precision. Some studies suggest that 10 to 12-bit precision combined with PQTF is sufficient to provide HDR data across 16 apertures, where distortion is less than just perceptible differences. Data represented with 10-bit precision can be further decoded using most state-of-the-art video decoding solutions. This conversion process includes signal quantization and is an element of lossy decoding, and a source of inaccuracies introduced into the converted data.

[0215] The following describes an example of this quantization applied to codewords in a target color space. In this example, YCbCr is shown. The input value YCbCr, expressed in floating-point precision, can be converted to a signal BitDepthY with a fixed bit depth for the Y value, and BitDepthC with respect to the chromaticity values ​​(Cb, Cr):

[0216]

[0217] (7)

[0218]

[0219] Where:

[0220] Round( x ) = Sign( x ) Floor( Abs( x ) + 0.5 )

[0221] If x < 0, Sign ( x ) = -1, if x = 0, Sign ( x ) = 0, if x > 0, Sign ( x) = 1

[0222] Floor( x ) is the largest integer less than or equal to x

[0223] If x >= 0, Abs( x ) = x, if x < 0, Abs( x ) = -x

[0224] Clip1Y( x ) = Clip3( 0, ( 1 << BitDepthY ) - 1, x )

[0225] Clip1C( x ) = Clip3( 0, ( 1 << BitDepthC ) - 1, x )

[0226] If z < x, Clip3( x,y,z ) = x, if z > y, Clip3( x,y,z ) = y, otherwise Clip3( x,y,z ) = z

[0227] There are several systems that use dynamic metadata based on HDR10 (PQ). Most of the HDR dynamic metadata currently in use is specified in the SMPTE ST 2094 document suite. These documents specify four different color volume transformations (mapping functions). Each color volume transformation (CVT) is published as a separate ST 2094-xx document, where xx equals -10 for CVT 1, xx equals -20 for CVT 2, xx equals -30 for CVT 3, and xx equals -40 for CVT 4. Details of the HDR parameters for CVT 4 are provided below, which are defined in ST 2094-40 (SMPTE: “Dynamic Metadata for Color Volume Transformation – Application #4,”, Doc. ST 2094-40 (2016), Society of Motion Picture and Television Engineer, White Plains, NY).

[0228] The metadata of SMPTE ST 2094-40 can be divided into three groups of syntax elements. The first group is used for identification purposes, the second group describes the statistical characteristics of the video signal, and the third group describes the basic OOTF (Optical-Optical Transfer Function) for the target peak brightness specified by the producer. This basic OOTF can be used to build a guided OOTF, which defines the adaptation for the presentation display.

[0229] Metadata elements associated with the identifier are not used for image processing. These elements include those defined in SMPTE ST 2094-40:

[0230] itu_t_t35_country_code: itu_t_t35_country_code should be a byte having a value as specified as the country code by Rec. ITU-T T.35 Annex A

[59] . This value should be 0xB5.

[0231] itu_t_t35_terminal_provider_code: itu_t_t35_terminal_provider_code should be a fixed 16-bit field. Its value should be 0x003C.

[0232] itu_t_t35_terminal_provider_oriented_code: itu_t_t35_terminal_provider_oriented_code should be a 16-bit code. This value should be as specified in Table 149 of SMPTE ST 2094-40.

[0233] CVT_identifier (also known as "application_identifier"): The application_identifier identifies the application (CVT) and its definition document in the ST-2094 suite. The application_identifier can be set to 4.

[0234] Metadata elements associated with statistical characteristics include:

[0235] `maxscl[w][i]`: Specifies the maximum value of the `i`-th color component of the linearized RGB values ​​in the `w`-th processing window of the scene. The value of `maxscl[w][i]` should be in the range of 0 to 1 (inclusive) and a multiple of 0.00001. `maxscl[w][0]`, `maxscl[w][1]`, and `maxscl[w][2]` correspond to the `R`, `G`, and `B` color components, respectively.

[0236] average_maxrgb[w]: Specifies the average of the linearized maxRGB values ​​in the w-th processing window of the scene. The value of average_maxrgb[w] should be in the range of 0 to 1 (inclusive) and be a multiple of 0.00001.

[0237] num_distributions [w]

[0238] num_distributions_maxrgb_percentiles [w]: Indicates the number of linearized maxRGB values ​​at a given percentile in the w-th processing window of the scene. The maximum value of num_distribution_maxrgb_percentiles [w] should be 15.

[0239] distribution_index[ w ][ i ]

[0240] distribution_values[w][i]

[0241] `distribution_maxrgb_percentages[w][i]`: Specifies the integer percentage value corresponding to the linearized RGB value of the i-th percentile in the w-th processing window of the scene. The value of `distribution_maxrgb_percentages[w][i]` should be in the range of 0 to 100 (inclusive).

[0242] `distribution_maxrgb_percentiles[ w ][ i ]`: Specifies the linearized maxRGB value at the i-th percentile of the w-th processing window in the scene. The value of `distribution_maxrgb_percentiles[ w ][ i ]` should be in the range of 0 to 1 (inclusive) and be a multiple of 0.00001.

[0243] fraction_bright_pixels[w]: Specifies the fraction of selected pixels in the image that includes the brightest pixels in the scene. The value of fraction_bright_pixels[w] should be in the range of 0 to 1 (inclusive) and a multiple of 0.001.

[0244] Metadata elements associated with the base OOTF include:

[0245] targeted_system_display_maximum_luminance: Specifies the nominal maximum display brightness of the target system display, in units of 1 candela per square meter. The value of targeted_system_display_maximum_luminance should be in the range of 0 to 10000 (inclusive).

[0246] The value of tone_mapping_flag[w] equals 1, indicating that metadata for the tone mapping function exists in the w-th processing window.

[0247] `knee_point_x[ w ]` specifies the x-coordinate of the point where the linear and curved parts of the tone mapping function separate. The value of `knee_point_x[ w ][ i ]` should be in the range of 0 to 1, excluding 0, and a multiple of 1 / 4095.

[0248] `knee_point_y[ w ]` specifies the y-coordinate of the point where the linear and curved parts of the tone mapping function separate. The value of `knee_point_y[ w ][ i ]` should be in the range of 0 to 1, excluding 0, and a multiple of 1 / 4095.

[0249] `num_bezier_curve_anchors[w]` indicates the number of intermediate anchor point parameters in the tone mapping function within the `w`th processing window. The maximum value of `num_bezier_curve_anchors[w]` should be 15.

[0250] `bezier_curve_anchors[w][i]` specifies the i-th intermediate anchor parameter of the tone mapping function in the w-th processing window of the scene. The value of `bezier_curve_anchors[w][i]` should be in the range of 0 to 1 (inclusive) and be a multiple of 1 / 1023.

[0251] According to the technology disclosed herein, these different metadata elements can be referred to as parameters of a histogram, and any or all of these items can be extracted and encoded by the HDR statistics engine 132. Figure 2 ).

[0252] The HDR10+ system specification is based on ST 2094-40 and further constrains the values ​​of syntax elements (HDR10+TECHNICAL SPECIFICATION Part 1: Metadata and Devices, version 1.6, HDR10+ Technologies LLC, Dec. 2021). The following constraints are imposed on syntax elements describing video statistical features:

[0253] The distribution_index[w][i] should be set as shown in Table 2.

[0254] The distribution_values[w][i] should be as shown in Table 2. Distribution_values[w][i] should be in the range of 0–100 or 255 (i=2). Otherwise (i=0,1,3...8), distribution_values[w][i] should be in the range of 0x00000–0x186A0. The distribution_values[w][i] for (i=0-1, 3-8) should be multiplied by 10. When processing HDR10+ metadata, the value of distribution_values[w][i] for (i=0-1, 3-8) should be divided by 10.

[0255] Table 2

[0256]

[0257] Figure 10 This is an example diagram 454 illustrating the derivation of DistributionY99 and DistributionY100 based on histogram percentiles. Further parameters include DistributionY99 and DistributionY100 nits, which can be determined according to... Figure 10 The derivation process is illustrated intuitively. The DistributionY99 value is the maximum value of the 99YF value within the video scene. The 99YF value is derived as a value for 99.99% of the linearized luminance values ​​per frame that fall below this value. The DistributionY100nits value is the average value of the DPY100F values ​​within the scene. DistributionY100F is derived as the percentage of the cumulative number of pixels per frame with a luminance of 100 nits or less, between 0 and 100.

[0258] Figure 11 This is a block diagram illustrating an example live encoder framework with HDR10+ capability. In this example, Figure 11 A video source 460 and an HDR encoder 470 are depicted. The HDR encoder 470 includes a video encoder 472, a percentile estimator 474, a tone mapping curve estimator 476, and an HDR10+ metadata injector 478. Generally, for HDR10+, the statistical characteristics of the video data generated or captured by the video source 460 can be determined during content production and / or by the HDR encoder 470. For example, the HDR encoder 470 can generate statistical measurements of a sample of video data through a mathematical process and represent the resulting statistical measurements with linearized RGB values. The tone mapping curve estimator 476 can then generate a tone mapping curve to represent the calculated statistical measurements, and both the statistical measurements and the tone mapping curve can be provided to the HDR10+ metadata injector 478. The HDR10+ metadata injector 478 can then generate metadata parameters representing the statistical measurements and the tone mapping curve, and provide the metadata parameters to the video encoder 472, which can encode the parameters along with the video data. Ultimately, the video encoder 472 can generate a bitstream 480 that includes encoded video data and encoded parameters.

[0259] In real-time encoding, percentile estimator 474 can generate HDR10+ statistical features of the video data received from video source 460. These statistical features can be represented using PQ10, including Y'CbCr 10-bit 4:2:0, BT.2100 PQEOTF, and / or BT.2020 color containers. Static information, such as maximum and / or average values, and histogram percentiles calculated non-linearly using Y'CbCr, as described in SMPTE: “Dynamic Metadata for Color Volume Transformation – Application #4,” Doc. ST 2094-40 (2016), Society of Motion Picture and Television Engineer, White Plains, NY, allows for low complexity and relatively low memory requirements in fixed-point arithmetic. The memory required to perform histogram derivation on 8k UHD content might be limited to 1024 entries with 25 bits of data. The conversion of the derived statistical parameters (to the linearized RGB representation) can be accomplished according to the above formula (2), which has the precision required for the HDR10+ syntax (up to 0.1 cd / m2).

[0260] However, in some applications, it may be necessary to estimate HDR10+ statistical parameters in a color space different from the source color space. An example of such a requirement could be calculating statistical properties in the P3-D65 color space, while the source is in the BT.2100 (BT.2020 color space).

[0261] This disclosure recognizes that, alternatively, the processing may additionally include the application of PQ EOTF, RGB BT.2020 to RGB P3 color space conversion. The latter can be implemented in a linear RGB representation and therefore may require >16-bit or even floating-point representation. Following this, video data statistics can be estimated in either the linear RGB domain (alternative implementation 1) or the non-linear RGB representation (alternative implementation 2), which will require the application of PQ OETF.

[0262] Each of these alternative implementations may introduce complexity compared to the technology disclosed herein, as described below.

[0263] For Alternative Implementation 1, to achieve the precision specified for HDR10+ (0.1 cd / m²), the computation of the histogram in the linear RGB domain requires a memory allocation of at least 2^17 entries out of 25 entries. This is a significant memory increase (2^10 entries out of 25 entries) compared to the derivation of the histogram in the nonlinear R'G'B' domain. However, this disclosure recognizes the advantage of this method: the bins of the histogram in the LL domain can be directly used to generate HDR10+ syntax elements without requiring an EOTF transformation.

[0264] For alternative implementation 2, the application of PQ OETF to linear RGB requires a LUT size of 2^16 entries, where the following statistical derivation is used in the linear domain, and PRE EOTF is applied to generate HDR10+ parameters. Furthermore, using 16-bit fixed-point arithmetic for color space conversion and supporting a dynamic range of 0...2^10 cd / m² results in a quantization error as high as 0.17 cd / m², exceeding the precision constraints of HDR10+. Moreover, this quantization error in the linear light representation due to the application of PQ OETF can increase the quantization error in the PQ domain to as high as 30% of the value. To reduce this quantization error, the bit depth needs to be increased to >18 bits per sample. Another drawback of this method is that the histogram parameters (percentiles) derived from the histogram need to be converted to LL representation by applying a LUT.

[0265] The situation regarding real-time encoder frameworks is particularly relevant to user-generated or user-captured content applications, which are typically deployed on computing platforms constrained by cost, memory, computational complexity, and power consumption, such as mobile devices. Therefore, the techniques of this disclosure may be especially useful on mobile devices used to deliver HDR / WCG video data, as these techniques can be implemented in low-complexity, low-memory environments, such as mobile devices.

[0266] Figure 12 This is a conceptual diagram illustrating an example process 500 for deriving HDR10+ statistical characteristics of video data according to the techniques described herein. In this example, process 500 begins by receiving video data 502 in Y'CbCr 10b PQ, BT.2020 format. A color conversion process 504 converts the Y'CbCr format to R'G'B' format. A PQ EOTF stage 506 then applies a transform function to the R'G'B format data. A quantization stage 508 then converts the RGB BT.2020 format data to RGB P3 quantization values. A histogram calculation stage 510 then derives the histogram index using a static array lookup table (LUT) 512. The linear light histogram (histogramLL) can then be represented in a 1024-entry LUT 514.

[0267] The techniques disclosed herein can perform low-complexity, low-memory estimation of statistical properties (histograms) of video data represented in the linear light RGB domain. The histogram calculation accuracy may be sufficient for HDR10+ parameter derivation (up to .1 cd / m2) without significantly increasing memory requirements for computational complexity.

[0268] Process 500 can be achieved through, for example Figure 2 The HDR video coding system 140 performs this operation. The HDR statistics engine 132 can perform histogram estimation in the linear light (LL) RGB domain, such that histogramLL[idx]++, idx = function(R,G,B). In some examples, a single histogramLL for RGB data is constructed on a single statistical feature, for example, on the maximum value among 3 samples, where maxRGB = max(R,G,B); histogramLL[idx]++, idx = function(maxRGB).

[0269] The bins of the histogram LL can be aligned / associated with linear light values ​​represented by codewords in a nonlinear RGB representation (R'G'B') to a certain degree of precision. In some examples, the nonlinearity can be BT.2100 / PQ. In some examples, the precision is 0.1 cd / m², which is aligned with the HDR10+ specification. Therefore, histogramLL[idx], idx = PQ_EOTF[maxRGB], where maxRGB is the maximum value of the R', G', and B' codewords in the PQ domain.

[0270] The derivation of the histogram LL index can be performed through a non-linear or piecewise linear mapping process, mapping LL RGB sample values ​​to the histogram index. In one example, index derivation is performed by comparing LL RGB values ​​to threshold arrays (ArrayTs) represented by LL intensity values ​​and sorted in ascending order, where the maximum value exceeding the threshold provides the index of the histogram bin. In some examples, index derivation is performed by searching the threshold arrays, where the thresholds provide the minimum distance (e.g., SAD or MSE) from the LL RGB values.

[0271] In some examples, where nonlinear R'G'B' samples are represented using 10-bit codewords, the number of thresholds in the ArrayTs used for comparison can be less than or equal to 1024. The PQ codeword space in the LL representation may be redundant for the target precision (e.g., 0.1 cd / m²), and the thresholds in the ArrayTs can be quantized into several unique levels less than 1024.

[0272] In some examples, a hybrid approach can be used for searching for or comparing against a threshold with the minimum distance, where a portion of the index (indx1) or the initial offset within the ArrayTs is derived analytically from the RGB sample values, e.g., indx1 = log2(maxRGB), and another portion of the index (indx2) is searched within a subrange specified by indx1. This search can be achieved by reducing the number of comparators (instead of the 10 used for a 1024-size LUT), at the cost of increasing the storage of the static LUT by 15 entries marking the ArrayTs positions where the threshold is a power of 2.

[0273] In some examples, histogram indices can be derived analytically based on RGB values.

[0274] In some examples, the derivation of parameters aligned to HDR10+ (i.e., histogram percentiles in LL representation) can be done by accessing the provided ArrayTs LUT instead of applying the PQ EOTF.

[0275] The following describes various example implementations consistent with the technologies disclosed herein.

[0276] In one example implementation, the source video data is represented as Y'CbCr 4:2:10 10-bit, PQ, BT.2020. The target color space used for calculation is HDR10+ with statistical properties RGB P3-D65. The output is the HDR10+ percentile, as shown in Table 2 above.

[0277] A static threshold LUT ArrayTs of size 1024 is predetermined and provided as edge information. The derivation can be performed as follows:

[0278] The variable maxLL is set to equal 10000 cd / m2

[0279] The allowed 10-bit codeword space in the PQ RGB field is cwPQ, where cwPQ is in the range of 0 to 1023 (inclusive):

[0280] cw10 = 0:1023

[0281] The allowed normalized PQ codeword (cw10n) space is normalized to the range of 0 to 1, including dividing each element by 1024:

[0282] cw10n = cw10 / 2 10

[0283] The spatial derivation of the allowed normalized LL values ​​(CWn) and LL (in cd / m2 (CW)) covered by the PQ codeword is as follows, where the function is applied to each entry of the array respectively:

[0284] CWn = PQ_EOTF(cw10n)

[0285] CW = WCn maxLL

[0286] The spatial derivation of the permissible LL values ​​(quantized to the precision of 0.1 cd / m² required by HDR10+) is as follows, where CW2 is the LL value in cd / m², and CW2n is the normalized value in the range of 0..1:

[0287] CW2=round(CW 10.0) / 10 / 0

[0288] CW2n = WC2 / maxRGB

[0289] The derivation of the static LUT (e.g., 16-bit) for the threshold in the target fixed-point arithmetic implementation is as follows:

[0290] numUpBits = 16

[0291] qScaler = 2^numUpBits

[0292] ArrayTs = round(CW2n qScaler)

[0293] Each entry in ArrayT provides an LL value (threshold) associated with an allowed PQ codeword, represented in 10 bits, with a precision of up to 0.1 cd / m².

[0294] To derive histogram indices from ArrayTs, the HDR video encoding system 140 can be configured to perform the following in this example:

[0295] enter:

[0296] The input samples are represented by linear light R, G, B, with a 16-bit fixed-point representation.

[0297] The variable maxRGB for each pixel is determined to be the maximum value of the three components:

[0298] maxRGB = max(R,G,B)

[0299] In some examples, a lower bin boundary threshold can be used. The histogram index can be determined as the maxRGB value of the threshold ArraysTs that is less than or equal to the maxRGB value. The smallest ArrayTs index of the threshold entry that is greater than or equal to the maxRGB value provides the index for histogram updates:

[0300] idxHist = min(find(maxRGB<= ArrayTs))

[0301] histogramLL(idxHist)++

[0302] In some examples, an upper bin boundary threshold can be used. The histogram index bin can be determined as the maxRGB value of ArraysTs that is greater than or equal to the threshold. The last ArrayTs entry found that is greater than or equal to the maxRGB value provides an index for histogram updates.

[0303] idxHist = max(find(maxRGB>= ArrayTs))

[0304] histogramLL(idxHist)++

[0305] In some examples, linear interpolation can be used to determine the histogram index position between the lower and upper thresholds.

[0306] idxHist1 = min(find(value<= pqThresholdsLL))

[0307] idxHist2 = max(find(value>= pqThresholdsLL))

[0308] if (idxHist1 != idxHist2)

[0309] idxHist = round((idxHist1 + idxHist2) / 2)

[0310] else

[0311] idxHist = idxHist1

[0312] In some examples, a binary search on the sorted array can be used. The worst-case complexity of such a search for a LUT of size 1024 is 10 consecutive comparisons.

[0313] In some examples, a hybrid search approach can be used on sorted arrays. The initial offset for the search can be derived from an additional LUT (OffsetLUT) used to offset the specified position from the ArrayTs. The access index of the OffsetLUT is derived as a log2(maxRGB) value. For a LUT of size 1024, this mechanism will allow the number of comparisons for a binary search to be reduced from 10 to 7.

[0314] In some examples, static LUTArrayTs can be provided as edge information. The calculation of the LL RGB threshold can be represented using a 16-bit fixed-point representation. In one example:

[0315] ArrayTs = { 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,2,2,2,2,2,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,4,4,4,4,4,4,4,4,5,5,5,5,5,5,5,5,5,5,5,5,5,6,6,6,6,6,7,7,7,7,7,7,7,7,7,7,8,8,8,8,8,9,9,9,9,9,9,9,9,10,10,10,10,10,10,10,10,11,11,11,12,12,12,12,12,12,12,13,13,13,14,14,14,14,14,15,15,15,16,16,16,16,16,17,17,17,18,18,18,18,18,19,19,20,20,20,20,21,21,22,22,22,22,23,23,24,24,24,24,25,25,26,26,26,27,27,28,28,28,29,29,29,29,30,31,31,31,32,32,33,33,34,34,35,35,35,36,37,37,37,38,39,39,40,40,41,41,42,43,43,44,44,45,45,46,47,47,48,48,49,50,50,51,52,52,53,54,54,55,56,56,57,58,58,60,60,61,62,62,63,64,65,66,66,68,68,69,70,71,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,96,96,98,99,100,101,102,104,105,106,107,108,109,111,112,113,115,116,117,119,120,122,123,125,126,127,128,130,132,133,135,136,138,140,141,142,144,145,147,149,151,152,154,156,157,159,161,163,164,166,168,170,172,174,176,178,180,182,184,185,187,189,192,194,196,198,201,203,204,207,209,211,214,216,218,220,223,225,227,230,233,235,237,240,242,245,248,250,253,256,258,261,264,267,269,273,275,278,281,284,287,290,293,296,299,302,305,309,312,315,319,322,325,328,332,335,338,342,345,349,353,357,360,364,367,371,375,379,383,387,391,395,398,402,406,410,415,419,423,427,432,436,440,445,450,454,458,463,468,473,477,482,487,492,496,501,507,511,516,522,527,532,537,543,548,553,559,564,570,575,581,587,593,598,604,611,617,623,628,635,641,647,653,660,667,673,680,686,693,699,706,713,720,727,734,741,748,756,763,771,778,786,793,801,809,817,824,832,840,848,857,864,873,881,890,898,907,916,924,933,942,951,960,969,978,988,997,1007,1016,1026,1035,1045,1056,1066,1075,1086,1096,1107,1117,1128,1138,1149,1160,1171,1182,1193,1205,1216,1227,1239,1250,1262,1274,1286,1298,1310,1323,1335,1347,1361,1373,1386,1399,1412,1425,1439,1452,1465,1479,1493,1507,1521,1535,1549,1564,1579,1593,1608,1623,1638,1653,1669,1684,1700,1716,1731,1748,1764,1780,1796,1813,1830,1847,1864,1882,1899,1916,1934,1952,1970,1988,2007,2025,2043,2062,2081,2100,2120,2140,2159,2179,2199,2219,2240,2260,2281,2302,2323,2344,2366,2387,2410,2432,2454,2477,2499,2522,2545,2568,2592,2616,2640,2664,2688,2713,2737,2762,2788,2813,2839,2865,2891,2917,2944,2971,2998,3025,3053,3080,3108,3137,3165,3194,3223,3253,3282,3312,3342,3372,3403,3434,3465,3496,3528,3561,3593,3625,3658,3692,3725,3758,3793,3827,3861,3897,3932,3968,4004,4040,4076,4113,4150,4188,4226,4264,4302,4342,4381,4420,4460,4501,4542,4582,4624,4666,4707,4750,4793,4837,4880,4924,4968,5014,5059,5104,5150,5197,5244,5291,5339,5386,5435,5484,5534,5584,5634,5685,5736,5787,5839,5892,5945,5999,6053,6107,6162,6218,6274,6330,6387,6445,6502,6561,6620,6680,6740,6801,6862,6923,6985,7048,7112,7176,7240,7305,7371,7438,7505,7572,7640,7709,7778,7848,7919,7989,8062,8134,8207,8281,8355,8431,8507,8583,8660,8738,8817,8896,8976,9056,9138,9220,9303,9386,9471,9556,9642,9728,9816,9904,9994,10083,10174,10266,10357,10451,10545,10640,10735,10832,10929,11028,11127,11227,11328,11429,11532,11637,11741,11847,11953,12061,12169,12279,12390,12501,12614,12727,12842,12957,13074,13192,13310,13430,13552,13673,13797,13921,14046,14173,14301,14430,14559,14691,14824,14957,15092,15228,15366,15505,15644,15785,15928,16071,16216,16363,16510,16660,16810,16962,17115,17270,17426,17583,17743,17902,18064,18228,18393,18559,18727,18896,19067,19240,19414,19589,19767,19946,20127,20309,20493,20679,20866,21055,21246,21439,21633,21829,22027,22227,22429,22632,22838,23045,23255,23466,23679,23894,24111,24330,24552,24775,25000,25227,25457,25689,25923,26159,26397,26638,26880,27125,27372,27622,27874,28128,28384,28644,28905,29169,29435,29704,29976,30249,30525,30805,31086,31371,31658,31947,32240,32535,32833,33133,33437,33743,34053,34364,34680,34998,35319,35643,35971,36301,36634,36971,37310,37654,38000,38349,38702,39058,39418,39781,40147,40517,40891,41267,41648,42032,42420,42811,43207,43606,44009,44415,44826,45240,45658,46081,46508,46938,47373,47811,48254,48702,49153,49609,50070,50534,51003,51477,51955,52438,52926,534 18,53915,54417,54923,55434,55951,56472,56999,57530,58067,58609,59155,59708,60266,60829,61397,61971,62551,63137,63728,64325,64927}. ,

[0316] In some examples, static LUT ArrayTs can be provided as edge information. The calculation of the LL RGB threshold can be represented using a 17-bit fixed-point representation. For example:

[0317] ArrayTs = { 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,4,4,4,4,4,4,4,4,4,4,4,4,4,5,5,5,5,5,5,5,5,5,5,7,7,7,7,7,7,7,7,7,8,8,8,8,8,8,8,8,9,9,9,9,9,9,9,10,10,10,10,10,10,12,12,12,12,12,13,13,13,13,13,13,14,14,14,14,16,16,16,16,16,17,17,17,17,18,18,18,18,20,20,20,20,21,21,21,21,22,22,22,24,24,24,24,25,25,25,26,26,26,28,28,28,29,29,30,30,30,31,31,31,33,33,34,34,34,35,35,37,37,37,38,38,39,39,41,41,42,42,43,43,45,45,46,46,47,47,48,48,50,50,51,51,52,54,54,55,55,56,58,58,59,59,60,62,62,63,64,64,66,67,68,68,69,71,71,72,73,75,75,76,77,79,80,80,81,83,84,85,87,88,88,89,90,92,93,94,96,97,98,100,101,102,104,105,106,107,109,110,111,113,114,115,117,119,121,122,123,125,126,128,130,131,132,135,136,138,140,142,143,144,147,148,151,152,153,156,157,160,161,164,165,168,169,172,173,176,178,180,182,184,186,189,191,193,195,198,201,202,204,207,210,212,215,216,219,222,224,227,229,232,235,237,240,244,246,249,252,254,257,261,263,266,270,273,275,279,282,284,288,291,295,298,301,304,308,312,315,319,322,325,329,333,337,341,343,347,351,355,359,363,367,371,375,379,384,388,392,396,401,405,409,414,418,422,427,431,436,440,446,451,455,460,465,471,474,480,485,490,495,501,506,511,516,523,528,533,539,545,551,556,562,568,574,579,586,592,598,604,611,617,624,630,637,644,650,657,663,670,676,684,691,699,705,713,720,727,734,742,750,758,765,773,781,789,797,805,813,821,830,838,847,855,864,872,881,890,899,908,916,925,936,945,954,963,974,983,992,1003,1013,1022,1033,1043,1054,1064,1075,1085,1096,1106,1118,1129,1140,1151,1163,1174,1186,1197,1208,1222,1233,1245,1257,1270,1282,1295,1307,1320,1333,1346,1359,1372,1385,1399,1413,1426,1440,1455,1468,1482,1497,1511,1526,1541,1556,1572,1586,1602,1617,1633,1649,1665,1680,1696,1713,1729,1746,1763,1780,1797,1814,1831,1848,1866,1884,1902,1920,1939,1957,1975,1995,2013,2033,2053,2071,2091,2112,2131,2151,2172,2193,2214,2233,2256,2277,2298,2320,2342,2365,2387,2409,2431,2455,2477,2501,2524,2548,2572,2597,2620,2645,2670,2695,2721,2746,2772,2797,2823,2851,2877,2903,2931,2958,2986,3013,3042,3070,3099,3127,3158,3186,3217,3245,3277,3307,3337,3369,3400,3431,3463,3496,3527,3560,3593,3627,3661,3694,3728,3763,3797,3833,3868,3903,3940,3977,4013,4050,4087,4125,4163,4201,4240,4280,4319,4358,4399,4438,4480,4521,4563,4605,4647,4688,4732,4775,4820,4864,4909,4953,4998,5044,5091,5137,5184,5231,5280,5328,5377,5425,5475,5525,5576,5627,5678,5729,5782,5834,5888,5941,5995,6050,6105,6160,6217,6273,6331,6388,6446,6505,6564,6624,6683,6745,6807,6868,6930,6993,7057,7121,7185,7251,7316,7383,7450,7517,7585,7655,7723,7794,7864,7935,8007,8079,8153,8226,8301,8376,8452,8529,8605,8684,8762,8841,8921,9002,9083,9165,9247,9331,9415,9500,9587,9673,9760,9849,9937,10027,10117,10208,10301,10394,10487,10581,10677,10773,10870,10968,11068,11167,11268,11369,11471,11575,11679,11785,11891,11997,12106,12215,12325,12436,12548,12660,12774,12890,13005,13123,13241,13360,13479,13601,13723,13846,13971,14097,14224,14352,14481,14611,14743,14875,15009,15144,15280,15418,15556,15696,15837,15979,16123,16269,16414,16562,16710,16861,17013,17165,17320,17476,17633,17792,17952,18113,18275,18439,18606,18772,18941,19112,19283,19456,19632,19809,19987,20167,20348,20531,20715,20902,21089,21280,21471,21664,21859,22055,22253,22454,22656,22859,23065,23273,23482,23694,23906,24121,24339,24558,24779,25002,25227,25454,25684,25914,26148,26383,26621,26861,27103,27347,27593,27842,28093,28346,28601,28859,29119,29382,29647,29915,30185,30456,30731,31009,31288,31570,31856,32143,32432,32726,33021,33320,33620,33924,34231,34540,34852,35167,35485,35805,36129,36455,36785,37118,37454,37792,38134,38480,38827,39179,39534,39892,40254,40618,40986,41357,41732,42111,42492,42878,43266,43659,44055,44454,44858,45264,45676,46090,46510,46932,47358,47789,48223,48660,49104,49550,50000,50455,50915,51378,51846,52317,52794,53276,53760,54251,54745,55244,55748,56256,56769,57288,57809,58338,58870,59408,59951,60499,61051,61609,62173,62742,63316,63895,64480,65069,65666,66266,66874,67486,68105,68729,69359,69996,70639,71286,71941,72602,73268,7 3942,74621,75307,75999,76698,77405,78116,78836,79562,80295,81034,81781,82535,83296,84064,84840,85623,86413,87211,88017,88830,89652,904 80,91317,92162,93015,93876,94745,95622,96508,97404,98307,99219,100139,101068,102007,102954,103910,104876,105851,106835,107829,108833, 109846,110869,111901,112945,113997,115060,116134,117218,118311,119416,120531,121658,122795,123943,125103,126273,127456,128650,129854}.

[0318] The following pseudocode illustrates an example implementation of static LUT ArrayTs construction:

[0319] int constructPQThresholdsLL(int input_bitdepth, int system_bitdepth)

[0320] {

[0321] int maxValueLL = 10000; / / 10k cd / m2

[0322] int maxValueInput = (1< <input_bitdepth);

[0323] int maxValueSystem = (1< <system_bitdepth);

[0324] int lengthArray = 1< <input_bitdepth;

[0325] int thresholdsArrayLL = (int malloc(lengthArray) sizeof(int));

[0326] for (int i = 0; i <lengthArray; i++)

[0327] {

[0328] double value = (double)i / maxValueInput; / / Normalize pq to 0.1

[0329] value = PQ_EOTF(value); / / Normalized LL 0.1

[0330] value = value maxValueLL; / / LL 0..10000

[0331] value = (double)((int)(10.0 value + 0.5)) / (double)10; / / Quantize LL to 0.1cd / m2

[0332] value = value / maxValueLL; / / Normalized LL 0.1

[0333] value = (int)(value maxValueSystem + 0.5); / / Normalize LL values ​​to a fixed-point representation in the system.

[0334] value = value>maxValueSystem ? maxValueSystem : value;

[0335] value = value<0 ? 0 : value;

[0336] thresholdsArrayLL[i] = (int) value;

[0337] }

[0338] return thresholdsArrayLL;

[0339] }

[0340] The following pseudocode illustrates an example implementation for constructing a histogram of linear light:

[0341] int derivePQHistogramLL(int input_data, int numInputPoints, int thresholdsArrayLL, int numThresholds)

[0342] {

[0343] int output_histogram = (int malloc(numInputPoints) sizeof(int)); / /

[0344] memset(output_histogram, 0, numInputPoints sizeof(int));

[0345] for (int i=0; i <numInputPoints; i++)

[0346] {

[0347] int value = input_data[i];

[0348] int j = 0;

[0349] while ( value>thresholdsArrayLL[j]&&j <numThresholds)

[0350] {

[0351] j++;

[0352] }

[0353] output_histogram[j]++;

[0354] }

[0355] return output_histogram;

[0356] }

[0357] The technology disclosed herein allows for a low-complexity and low-memory implementation of histogram computation in the linear light RGB domain, with a granularity comparable to that of nonlinear representations in the PQ domain, while still meeting the accuracy requirements of HDR10+.

[0358] Various examples of the technology disclosed herein are shown in the following clauses:

[0359] Clause 1: A method for encoding high dynamic range (HDR) video data, the method comprising: calculating a histogram of an image of the video data, the image being represented in a linear light format; encoding the values ​​of the histogram of the image represented in the linear light format; and encoding the image.

[0360] Clause 2: The method according to Clause 1, wherein calculating the histogram comprises: forming a plurality of bins for the histogram, each bin corresponding to a corresponding range within the range of linear light values ​​of the image, wherein a first bin of the plurality of bins corresponds to a first range within the range of linear light values ​​having a first width, and a second bin of the plurality of bins corresponds to a second range within the range of linear light values, the first width being different from the second width; and for each bin of the plurality of bins, calculating the number of samples of the image within the corresponding range of the linear light values.

[0361] Clause 3: The method according to Clause 2, wherein encoding the value for the histogram comprises: for each of the plurality of bins, determining a corresponding codeword for representing the number of samples in the corresponding range of the linear light values ​​of the image.

[0362] Clause 4: The method according to Clause 3, wherein determining the corresponding codeword comprises: selecting the corresponding codeword from a set of codewords in PQ10 format for HDR images.

[0363] Clause 5: The method according to Clause 3, wherein determining the corresponding codeword comprises: forming a set of linear optical codewords as a subset of a set of codewords in PQ10 format for an HDR image, the set of linear optical codewords being smaller than the set of codewords in PQ10 format for an HDR image; and selecting the corresponding codeword from the set of linear optical codewords.

[0364] Clause 6: The method according to Clause 1, wherein calculating the histogram comprises: forming a variable array having a size of 210x18 bits for representing data for the histogram.

[0365] Clause 7: The method according to Clause 1, wherein encoding the image comprises: converting the image into a lightness, blue hue, and red hue (YUV) format image; and encoding the YUV format image.

[0366] Clause 8: The method according to Clause 1 further comprises: forming a bitstream, the bitstream including data for the encoded image and data for the encoded values ​​of the histogram.

[0367] Clause 9: An apparatus for encoding high dynamic range (HDR) video data, the apparatus comprising: a memory configured to store the video data; and one or more processors implemented in a circuit and configured to: calculate a histogram of an image of the video data, the image being represented in a linear light format; encode values ​​of the histogram for the image represented in the linear light format; and encode the image.

[0368] Clause 10: The apparatus according to Clause 9, wherein, in order to calculate the histogram, the one or more processors are configured to: form a plurality of bins for the histogram, each bin corresponding to a corresponding range in the range of linear light values ​​of the image, wherein a first bin of the plurality of bins corresponds to a first range in the range of linear light values ​​having a first width, and a second bin of the plurality of bins corresponds to a second range in the range of linear light values, the first width being different from the second width; and for each of the plurality of bins, calculate the number of samples of the image in the corresponding range in the range of linear light values.

[0369] Clause 11: The apparatus according to Clause 10, wherein, in order to encode the values ​​for the histogram, the one or more processors are configured to: for each of the plurality of bins, determine a corresponding codeword for representing the number of samples in the corresponding range of the linear light values ​​of the image.

[0370] Clause 12: The device according to Clause 11, wherein, in order to determine the corresponding codeword, the one or more processors are configured to: select the corresponding codeword from a set of codewords in PQ10 format for HDR images.

[0371] Clause 13: The apparatus according to Clause 11, wherein, in order to determine the corresponding codeword, the one or more processors are configured to: form a set of linear optical codewords as a subset of a set of codewords in PQ10 format for an HDR image, the set of linear optical codewords being smaller than the set of codewords in PQ10 format for an HDR image; and select the corresponding codeword from the set of linear optical codewords.

[0372] Clause 14: The apparatus according to Clause 9, wherein calculating the histogram comprises: forming a variable array having a size of 210x18 bits for representing data for the histogram.

[0373] Clause 15: The device according to Clause 9, wherein encoding the image comprises: converting the image into a luminance, blue hue, and red hue (YUV) format image; and encoding the YUV format image.

[0374] Clause 16: The apparatus according to Clause 9 further includes: forming a bitstream, the bitstream comprising data for the encoded image and data for the encoded values ​​of the histogram.

[0375] Clause 17: The device as described in Clause 9, wherein the device includes one or more of the following: camera, computer, mobile device, broadcast receiver device or set-top box.

[0376] Clause 18: An apparatus for encoding high dynamic range (HDR) video data, the apparatus comprising: a unit for calculating a histogram of an image of the video data, the image being represented in a linear light format; a unit for encoding values ​​of the histogram of the image represented in the linear light format; and a unit for encoding the image.

[0377] Clause 19: The apparatus according to Clause 18, wherein the unit for calculating the histogram comprises: a unit for forming a plurality of bins for the histogram, each bin corresponding to a corresponding range in the range of linear light values ​​of the image, wherein a first bin of the plurality of bins corresponds to a first range in the range of linear light values ​​having a first width, and a second bin of the plurality of bins corresponds to a second range in the range of linear light values, the first width being different from the second width; and a unit for calculating, for each bin of the plurality of bins, the number of samples of the image in the corresponding range in the range of linear light values.

[0378] Clause 20: The apparatus according to Clause 19, wherein the unit for encoding the values ​​for the histogram comprises: a unit for determining, for each of the plurality of bins, a corresponding codeword representing the number of samples of the image within the corresponding range of the linear light values.

[0379] Clause 21: The device according to Clause 20, wherein the unit for determining the corresponding codeword comprises: a unit for selecting the corresponding codeword from a set of codewords in PQ10 format for HDR images.

[0380] Clause 22: The apparatus according to Clause 20, wherein the unit for determining the corresponding codeword comprises: a unit for forming a linear optical codeword set as a subset of a PQ10 format codeword set for an HDR image, the linear optical codeword set being smaller than the PQ10 format codeword set for the HDR image; and a unit for selecting the corresponding codeword from the linear optical codeword set.

[0381] Clause 23: The apparatus according to Clause 18, wherein the unit for calculating the histogram comprises: a unit for forming a variable array having a size of 210x18 bits for representing data for the histogram.

[0382] Clause 24: The apparatus according to Clause 18, wherein the unit for encoding the image comprises: a unit for converting the image into a luminance, blue hue, and red hue (YUV) format image; and a unit for encoding the YUV format image.

[0383] Clause 25: The apparatus according to Clause 18 further includes: a unit for forming a bitstream, the bitstream including data for the encoded image and data for the encoded values ​​of the histogram.

[0384] Clause 26: A method for encoding high dynamic range (HDR) video data, the method comprising: calculating a histogram of an image of the video data, the image being represented in a linear light format; encoding values ​​of the histogram of the image represented in the linear light format; and encoding the image.

[0385] Clause 27: The method according to Clause 26, wherein calculating the histogram comprises: forming a plurality of bins for the histogram, each bin corresponding to a corresponding range within a range of linear light values ​​of the image, wherein a first bin of the plurality of bins corresponds to a first range within the range of linear light values ​​having a first width, and a second bin of the plurality of bins corresponds to a second range within the range of linear light values, the first width being different from the second width; and for each bin of the plurality of bins, calculating the number of samples of the image within the corresponding range of linear light values.

[0386] Clause 28: The method according to Clause 27, wherein encoding the values ​​for the histogram comprises: for each of the plurality of bins, determining a corresponding codeword for representing the number of samples in the corresponding range of the linear light values ​​of the image.

[0387] Clause 29: The method according to Clause 28, wherein determining the corresponding codeword comprises: selecting the corresponding codeword from a set of codewords in PQ10 format for HDR images.

[0388] Clause 30: The method according to Clause 28, wherein determining the corresponding codeword comprises: forming a set of linear optical codewords as a subset of a set of codewords in PQ10 format for an HDR image, the set of linear optical codewords being smaller than the set of codewords in PQ10 format for an HDR image; and selecting the corresponding codeword from the set of linear optical codewords.

[0389] Clause 31: The method according to any of Clauses 26-30, wherein calculating the histogram comprises: forming a variable array having a size of 210x18 bits for representing data for the histogram.

[0390] Clause 32: The method according to any of Clauses 26-31, wherein encoding the image comprises: converting the image into a luminance, blue hue, and red hue (YUV) format image; and encoding the YUV format image.

[0391] Clause 33: The method described under any of Clauses 26-32 further comprises: forming a bitstream, the bitstream comprising data for the encoded image and data for the encoded values ​​of the histogram.

[0392] Clause 34: An apparatus for encoding high dynamic range (HDR) video data, the apparatus comprising: a memory configured to store the video data; and one or more processors implemented in a circuit and configured to: calculate a histogram of an image of the video data, the image being represented in a linear light format; encode values ​​of the histogram of the image represented in the linear light format; and encode the image.

[0393] Clause 35: The apparatus according to Clause 34, wherein, in order to calculate the histogram, the one or more processors are configured to: form a plurality of bins for the histogram, each bin corresponding to a corresponding range in the range of linear light values ​​of the image, wherein a first bin of the plurality of bins corresponds to a first range in the range of linear light values ​​having a first width, and a second bin of the plurality of bins corresponds to a second range in the range of linear light values, the first width being different from the second width; and for each of the plurality of bins, calculate the number of samples of the image in the corresponding range in the range of linear light values.

[0394] Clause 36: The apparatus according to Clause 35, wherein, in order to encode the values ​​for the histogram, the one or more processors are configured to: for each of the plurality of bins, determine a corresponding codeword for representing the number of samples in the corresponding range of the linear light values ​​of the image.

[0395] Clause 37: The device according to Clause 36, wherein, in order to determine the corresponding codeword, the one or more processors are configured to: select the corresponding codeword from a set of codewords in PQ10 format for HDR images.

[0396] Clause 38: The apparatus according to Clause 36, wherein, in order to determine the corresponding codeword, the one or more processors are configured to: form a set of linear optical codewords as a subset of a set of codewords in PQ10 format for an HDR image, the set of linear optical codewords being smaller than the set of codewords in PQ10 format for an HDR image; and select the corresponding codeword from the set of linear optical codewords.

[0397] Clause 39: The device according to any of Clauses 34-38, wherein calculating the histogram comprises: forming a variable array having a size of 210x18 bits for representing data for the histogram.

[0398] Clause 40: The device according to any of Clauses 34-39, wherein encoding the image comprises: converting the image into a luminance, blue hue, and red hue (YUV) format image; and encoding the YUV format image.

[0399] Clause 41: The apparatus pursuant to any of Clauses 34-40 further includes: forming a bitstream, the bitstream comprising data for the encoded image and data for the encoded values ​​of the histogram.

[0400] Clause 42: The device as described in Clauses 34-41, wherein the device includes one or more of the following: camera, computer, mobile device, broadcast receiver device or set-top box.

[0401] Clause 43: An apparatus for encoding high dynamic range (HDR) video data, the apparatus comprising: a unit for calculating a histogram of an image of the video data, the image being represented in a linear light format; a unit for encoding values ​​of the histogram of the image represented in the linear light format; and a unit for encoding the image.

[0402] Clause 44: The apparatus according to Clause 43, wherein the unit for calculating the histogram comprises: a unit for forming a plurality of bins for the histogram, each bin corresponding to a corresponding range in the range of linear light values ​​of the image, wherein a first bin of the plurality of bins corresponds to a first range in the range of linear light values ​​having a first width, and a second bin of the plurality of bins corresponds to a second range in the range of linear light values, the first width being different from the second width; and a unit for calculating, for each bin of the plurality of bins, the number of samples of the image in the corresponding range in the range of linear light values.

[0403] Clause 45: The device according to Clause 44, wherein the unit for encoding the values ​​for the histogram comprises: a unit for determining, for each of the plurality of bins, a corresponding codeword representing the number of samples of the image within the corresponding range of the linear light values.

[0404] Clause 46: The device according to Clause 45, wherein the unit for determining the corresponding codeword comprises: a unit for selecting the corresponding codeword from a set of codewords in PQ10 format for HDR images.

[0405] Clause 47: The apparatus according to Clause 45, wherein the unit for determining the corresponding codeword comprises: a unit for forming a linear optical codeword set as a subset of a PQ10 format codeword set for an HDR image, the linear optical codeword set being smaller than the PQ10 format codeword set for the HDR image; and a unit for selecting the corresponding codeword from the linear optical codeword set.

[0406] Clause 48: The device according to any of Clauses 43-47, wherein the unit for calculating the histogram comprises: a unit for forming a variable array of size 210x18 bits for representing data for the histogram.

[0407] Clause 49: The device according to any of Clauses 43-48, wherein the unit for encoding the image comprises: a unit for converting the image into a luminance, blue hue, and red hue (YUV) format image; and a unit for encoding the YUV format image.

[0408] Clause 50: The apparatus pursuant to any of Clauses 43-49 further includes: a unit for forming a bitstream, the bitstream comprising data for an encoded image and data for encoded values ​​of the histogram.

[0409] It should be recognized that, depending on the examples, certain actions or events of any technique described herein may be performed in a different order, and may be added, combined, or omitted entirely (e.g., not all described actions or events are necessary for implementing the technique). Furthermore, in some examples, actions or events may be performed concurrently (e.g., through multithreading, interrupt handling, or multiple processors) rather than sequentially.

[0410] In one or more examples, the described functionality can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality can be stored as one or more instructions or code on a computer-readable medium or transmitted through a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium can include a computer-readable storage medium (which corresponds to a tangible medium such as a data storage medium) or a communication medium, including, for example, any medium that facilitates the transfer of a computer program from one place to another according to a communication protocol. In this way, a computer-readable medium can generally correspond to (1) a non-transitory tangible computer-readable storage medium or (2) a communication medium such as a signal or carrier wave. A data storage medium can be any available medium that can be accessed by one or more computers or one or more processors to obtain instructions, code, and / or data structures for implementing the techniques described in this disclosure. Computer program products can include computer-readable media.

[0411] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium capable of storing desired program code in the form of instructions or data structures and accessible by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technology (e.g., infrared, radio, and microwave), then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technology (e.g., infrared, radio, and microwave) is included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but instead refer to non-transient tangible storage media. As used herein, disks and optical discs include compact optical discs (CDs), laser optical discs, optical discs, digital versatile optical discs (DVDs), floppy disks, and Blu-ray discs, wherein disks typically magnetically copy data, while optical discs optically copy data using lasers. The above combinations should also be included within the scope of computer-readable media.

[0412] Instructions can be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the terms "processor" and "processing circuitry" as used herein can refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein can be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into combined codecs. Furthermore, the techniques can be implemented entirely within one or more circuit or logic elements.

[0413] The techniques disclosed herein can be implemented in a wide variety of devices or apparatuses, including wireless mobile phones, integrated circuits (ICs), or a set of ICs (e.g., chipsets). Various components, modules, or units are described in this disclosure to emphasize functional aspects of a device configured to perform the disclosed techniques, but implementation by different hardware units is not necessarily required. Rather, as described above, various units can be combined in a codec hardware unit, or provided by a collection of interoperable hardware units (including one or more processors as described above) combined with appropriate software and / or firmware.

[0414] Examples have been described. These and other examples are within the scope of the following claims.

Claims

1. A method for encoding high dynamic range (HDR) video data, the method comprising: Calculating a histogram for an image of video data, the image being represented in a linear light format, wherein calculating the histogram includes: Forming a plurality of bins for the histogram, each bin corresponding to a corresponding range within the range of linear light values ​​of the image, wherein a first bin corresponds to a first range within the range of linear light values ​​having a first width, and a second bin corresponds to a second range within the range of linear light values ​​having a second width, the first width being different from the second width; and For each of the plurality of boxes, calculate the number of samples of the image within the corresponding range of the linear light values; Extract one or more parameters from the histogram; Encoding one or more parameters of the histogram of the image represented in the linear light format, wherein encoding the one or more parameters of the histogram includes: for one or more bins of the plurality of bins, determining a corresponding codeword for representing the number of samples of the image within the corresponding range of the linear light values. Determining the corresponding codeword includes: A linear optical codeword set is formed as a subset of the codeword set in the perceptual quantizer PQ format for HDR images, the linear optical codeword set being smaller than the codeword set in the PQ format for HDR images, and represented using a fixed number of bits; and Select the corresponding codeword from the linear optical codeword set; and The image is encoded.

2. The method according to claim 1, wherein, One or more of the boxes correspond to different ranges with the same width within the range of linear light values.

3. The method according to claim 1, wherein, Determining the corresponding codeword includes: selecting the corresponding codeword from a set of codewords for nonlinear functions.

4. The method according to claim 3, wherein, The nonlinear function includes the ST.2084 PQ nonlinear function or the PQ10 nonlinear function.

5. The method according to claim 1, wherein, The fixed number of bits includes 7 bits.

6. The method according to claim 1, wherein, Calculating the histogram includes: forming a representation of the data for the histogram, having a value of 2. 10 An array of variables of size x18 bits.

7. The method according to claim 1, wherein, Encoding the image includes: Convert the image to a YUV format image with luminance, blue hue chromaticity, and red hue chromaticity; and The YUV format image is encoded.

8. The method according to claim 1, further comprising: A bitstream is generated by encoding the image and encoding one or more parameters of the histogram.

9. An apparatus for encoding high dynamic range (HDR) video data, the apparatus comprising: A memory configured to store histogram data; as well as One or more processors implemented in the circuit are configured to perform the following operations: Calculate a histogram for an image of video data, the image being represented in a linear light format, wherein, in order to calculate the histogram, the one or more processors are configured to: Forming a plurality of bins for the histogram, each bin corresponding to a corresponding range within the range of linear light values ​​of the image, wherein a first bin corresponds to a first range within the range of linear light values ​​having a first width, and a second bin corresponds to a second range within the range of linear light values ​​having a second width, the first width being different from the second width; and For each of the plurality of boxes, calculate the number of samples of the image within the corresponding range of the linear light values; Extract one or more parameters from the histogram; Encoding one or more parameters of the histogram of the image represented in the linear light format, wherein, in order to encode the one or more parameters of the histogram, the one or more processors are configured to: For one or more of the plurality of bins, a corresponding codeword is determined to represent the number of samples in the corresponding range of the linear light values ​​of the image, and wherein, in order to determine the corresponding codeword, the one or more processors are configured to: A linear optical codeword set is formed as a subset of the codeword set in the perceptual quantizer PQ format for HDR images, the linear optical codeword set being smaller than the codeword set in the PQ format for HDR images, and represented using a fixed number of bits; and Select the corresponding codeword from the linear optical codeword set; and The image is encoded.

10. The device according to claim 9, wherein, One or more of the boxes correspond to different ranges with the same width within the range of linear light values.

11. The device according to claim 9, wherein, In order to determine the corresponding codeword, the one or more processors are configured to select the corresponding codeword from a set of codewords for nonlinear functions.

12. The device according to claim 11, wherein, The nonlinear function includes the ST.2084 PQ nonlinear function or the PQ10 nonlinear function.

13. The device according to claim 9, wherein, The fixed number of bits includes 7 bits.

14. The device according to claim 9, wherein, Calculating the histogram includes: forming a representation of the data for the histogram, having a value of 2. 10 An array of variables of size x18 bits.

15. The device according to claim 9, wherein, Encoding the image includes: Convert the image to a YUV format image with luminance, blue hue chromaticity, and red hue chromaticity; and The YUV format image is encoded.

16. The device according to claim 9, wherein, The one or more processors are further configured to generate a bitstream by encoding the image and encoding the one or more parameters of the histogram.

17. The device according to claim 9, wherein, The device includes one or more of the following: camera, computer, mobile device, broadcast receiver device or set-top box.

18. An apparatus for encoding high dynamic range (HDR) video data, the apparatus comprising: A unit for calculating a histogram of an image for video data, the image being represented in a linear light format, wherein the unit for calculating the histogram includes: Units for forming a plurality of bins for the histogram, each bin corresponding to a corresponding range of linear light values ​​of the image, wherein a first bin of the plurality of bins corresponds to a first range of linear light values ​​having a first width, and a second bin of the plurality of bins corresponds to a second range of linear light values ​​having a second width, the first width being different from the second width; and A unit for calculating, for each of the plurality of boxes, the number of samples of the image within a corresponding range of the linear light values; A unit for extracting one or more parameters of the histogram; A unit for encoding one or more parameters of the histogram of the image represented in the linear light format, wherein the unit for encoding the one or more parameters of the histogram includes: a unit for determining, for one or more bins of the plurality of bins, a corresponding codeword representing the number of samples of the image within the corresponding range of the linear light values. The unit used to determine the corresponding codeword includes: Units for forming a linear optical codeword set as a subset of the codeword set in the perceptual quantizer PQ format for an HDR image, wherein the linear optical codeword set is smaller than the codeword set in the PQ format for the HDR image and is represented using a fixed number of bits; and A unit for selecting the corresponding codeword from the set of linear optical codewords; and Units used for encoding the image.

Images