Transform unit design for video encoding and decoding

Abstract

An example device includes a memory and one or more processors. The one or more processors are configured to determine whether to use single-tree coding for a first codec tree unit (CTU), determine a first transform unit (TU) based on a first luma sample transform block (TB) and first two corresponding chroma sample TBs based on the first CTU using the single-tree coding. The one or more processors are further configured to determine whether to use dual-tree coding for a second CTU, determine a second TU based on (I) a second luma sample TB or (II) a second set of two chroma sample TBs based on the second CTU using the dual-tree coding. The first TU includes a syntax structure for transforming the first TB samples, and the second TU includes a syntax structure for transforming the second TB samples.

Description

[0001] This application claims priority to U.S. Application No. 17 / 024,422, filed on September 17, 2020, which claims the benefit of U.S. Provisional Application No. 62 / 902,234, filed on September 18, 2019, and claims priority to U.S. Provisional Application No. 62 / 905,721, filed on September 25, 2019, the entire contents of each of which are incorporated herein by reference. Technical Field

[0002] The present disclosure relates to video encoding and video decoding. Background Art

[0003] Digital video capabilities can be incorporated into a wide range of devices, including digital televisions, digital direct broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e-book readers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, cellular or satellite radio telephones, so-called "smart phones", video teleconferencing devices, video streaming devices, etc. Digital video devices implement video codec technologies, such as those described in the standards defined by the Moving Picture Experts Group (MPEG)-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4, Part 10, Advanced Video Codec (AVC), ITU-T H.265 / High Efficiency Video Codec (HEVC), and extensions to these standards. By implementing such video codec technologies, video devices can more efficiently transmit, receive, encode, decode, and / or store digital video information.

[0004] Video coding techniques include spatial (intra-image) prediction and / or temporal (inter-image) prediction to reduce or eliminate redundancy inherent in video sequences. For block-based video coding, a video slice (e.g., a video image or a portion of a video image) can be divided into video blocks, which can also be referred to as codec tree units (CTUs), codec units (CUs), and / or codec nodes. Video blocks in an intra-coded (I) slice of an image are encoded using spatial prediction with respect to reference samples in neighboring blocks in the same image. Video blocks in an inter-coded (P or B) slice of an image can use spatial prediction with respect to reference samples in neighboring blocks in the same image or temporal prediction with respect to reference samples in other reference images. An image can be referred to as a frame, and a reference image can be referred to as a reference frame. Summary of the Invention

[0005] In general, this disclosure describes techniques for transform unit design and quantization parameter (QP) inference for luma and chroma components of video codecs. Because a video decoder cannot determine what is included in a transform unit or whether to use the actual values ​​of some syntax elements or infer the values ​​of the syntax elements, it may be configured in a manner that does not correctly decode video data. According to the techniques of this disclosure, a video decoder can be configured in a manner consistent with the configuration of a video encoder, enabling the video decoder to determine what is included in a transform unit and whether to use the actual values ​​of syntax elements or infer the values ​​of syntax elements.

[0006] In one example, a method includes determining, by one or more processors, whether to use a single tree to encode and decode a first codec tree unit (CTU); determining, by the one or more processors, a first transform unit (TU) based on a first set of first transform block (TB) samples based on encoding and decoding the first CTU using a single tree, the first set of first transform block samples including a first luma sample TB and a first two corresponding chroma samples TB; encoding and decoding a codec unit (CU) of the first CTU based on the first TU; determining, by the one or more processors, whether to use a dual tree to encode and decode a second CTU; determining, by the one or more processors, a second TU based on a second set of TB samples based on encoding and decoding the second CTU using a dual tree, the second set of TB samples including (I) a second luma sample TB or (III) a second two chroma sample TB; and encoding and decoding the CU of the second CTU based on the second TU, wherein the first TU includes a syntax structure for transforming the first set of TB samples and the second TU includes a syntax structure for transforming the second set of TB samples.

[0007] In another example, a device includes: a memory configured to store video data, and one or more processors executed in circuitry and communicatively coupled to the memory, the one or more processors configured to: determine whether to use a single tree to encode and decode a first CTU; based on using a single tree to encode and decode the first CTU, determine a first TU based on a first group of TB samples, the first group of TB samples including a first luma sample TB and a first two corresponding chroma samples TB; encode and decode a CU of the first CTU based on the first TU; determine whether to use a dual tree to encode and decode a second CTU; based on using a dual tree to encode and decode the second CTU, determine a second TU based on a second group of TB samples, the second group of TB samples including (I) a second luma sample TB or (II) a second two chroma sample TB; and encode and decode the CU of the second CTU based on the second TU, wherein the first TU includes a syntax structure for transforming the first group of TB samples, and the second TU includes a syntax structure for transforming the second group of TB samples.

[0008] In another example, a non-transitory computer-readable storage medium stores thereon instructions that, when executed by one or more processors, cause the one or more processors to: determine whether to use a single tree to encode and decode a first CTU; based on using a single tree to encode and decode the first CTU, determine a first TU based on a first group of TB samples, the first group of TB samples including a first luma sample TB and a first two corresponding chroma samples TB; encode and decode a CU of the first CTU based on the first TU; determine whether to use a dual tree to encode and decode a second CTU; based on using a dual tree to encode and decode the second CTU, determine a second TU based on a second group of TB samples, the second group of TB samples including (I) a second luma sample TB or (II) a second two chroma sample TB; and encode and decode the CU of the second CTU based on the second TU, wherein the first TU includes a syntax structure for transforming the first group of TB samples, and the second TU includes a syntax structure for transforming the second group of TB samples.

[0009] In another example, a device includes: a component for determining whether to use a single tree to encode and decode a first CTU; a component for determining a first TU based on a first set of TB samples based on encoding and decoding the first CTU using the single tree, the first set of TB samples including a first luma sample TB and a first two corresponding chroma samples TB; a component for encoding and decoding a CU of the first CTU based on the first TU; a component for determining whether to use a dual tree to encode and decode a second CTU; a component for determining a second TU based on a second set of TB samples based on encoding and decoding the second CTU using the dual tree, the second set of TB samples including (I) a second luma sample TB or (II) a second two chroma samples TB; and a component for encoding and decoding the CU of the second CTU based on the second TU, wherein the first TU includes a syntax structure for transforming the first set of TB samples and the second TU includes a syntax structure for transforming the second set of TB samples.

[0010] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a block diagram illustrating an example video encoding and decoding system that may perform the techniques of this disclosure;

[0012] Figure 2A and 2B is a conceptual diagram illustrating an example quad-tree binary tree (QTBT) structure and corresponding codec tree unit (CTU);

[0013] Figure 3 is a block diagram illustrating an example video encoder that may perform the techniques of this disclosure;

[0014] Figure 4 is a block diagram illustrating an example video decoder that may perform the techniques of this disclosure;

[0015] Figure 5 is a flow chart illustrating a technique for determining a transform unit (TU) according to the techniques of this disclosure;

[0016] Figure 6 is a flowchart illustrating a method of encoding video data according to the techniques of this disclosure; and

[0017] Figure 7 is a flowchart illustrating a method of decoding video data according to the techniques of this disclosure. DETAILED DESCRIPTION

[0018] In some video decoder implementations, the video decoder may be configured in a manner that prevents it from correctly decoding video data in certain situations. For example, the video decoder may not be able to determine what is contained in a transform unit, or whether to use actual values ​​of transform unit syntax elements (such as tu_cbf_cb and tu_cbf_cr (which may not yet be available)) or inferred values ​​when parsing the luma tree. As a result, the video decoder may decode the video data in a manner that the video encoder did not intend, and the video decoder may incorrectly decode the video data, which may result in defects in the visual presentation of the video data.

[0019] According to the technology of the present disclosure, a video decoder can be configured in a manner consistent with the configuration of a video encoder to facilitate correct decoding of video data. For example, the video decoder can correctly determine what is included in a transform unit and determine whether to use the actual values ​​of transform unit syntax elements, such as tu_cbf_cb and tu_cbf_cr (which may not yet be available), or use inferred values ​​when parsing the luma tree. In this way, the video decoder can more accurately decode the encoded video data, resulting in better visual presentation and fewer defects.

[0020] Figure 1 is a block diagram illustrating an example video encoding and decoding system 100 that can implement the techniques of the present disclosure. The techniques of the present disclosure are generally directed to encoding and decoding (encoding and / or decoding) video data. Generally, video data includes any data used to process video. Thus, video data can include original, unencoded video, encoded video, decoded (e.g., reconstructed) video, and video metadata, such as signaling data.

[0021] like Figure 1As shown, in this example, a video encoding and decoding system 100 includes a source device 102 that provides encoded video data to be decoded and displayed by a destination device 116. Specifically, the source device 102 provides the video data to the destination device 116 via a computer-readable medium 110. The source device 102 and the destination device 116 can include any of a variety of devices, including desktop computers, notebook (i.e., laptop) computers, tablet computers, set-top boxes, and telephone handsets such as smartphones, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some cases, the source device 102 and the destination device 116 can be equipped for wireless communication and, therefore, can be referred to as wireless communication devices.

[0022] exist Figure 1 In the example of , source device 102 includes a video source 104, a memory 106, a video encoder 200, and an output interface 108. Destination device 116 includes an input interface 122, a video decoder 300, the memory 120, and a display device 118. According to the present disclosure, the video encoder 200 of source device 102 and the video decoder 300 of destination device 116 can be configured to apply techniques for transform unit design and quantization parameter (QP) inference for luma and chroma. Thus, source device 102 represents an example of a video encoding device, while destination device 116 represents an example of a video decoding device. In other examples, the source device and destination device can include other components or arrangements. For example, source device 102 can receive video data from an external video source (e.g., an external camera). Similarly, destination device 116 can interface with an external display device rather than including an integrated display device.

[0023] Figure 1 The illustrated video encoding and decoding system 100 is merely an example. Generally, any digital video encoding and / or decoding device can implement techniques for transform unit design and QP inference for luma and chroma. Source device 102 and destination device 116 are merely examples of such codec devices, where source device 102 generates encoded video data for transmission to destination device 116. This disclosure refers to a "codec" device as a device that performs encoding and / or decoding of data. Thus, video encoder 200 and video decoder 300 represent examples of codec devices, specifically, a video encoder and a video decoder, respectively. In some examples, source device 102 and destination device 116 can operate in a substantially symmetrical manner, such that each of source device 102 and destination device 116 includes video encoding and decoding components. Thus, video encoding and decoding system 100 can support one-way or two-way video transmission between source device 102 and destination device 116, for example, for video streaming, video playback, video broadcasting, or video telephony.

[0024] Generally speaking, video source 104 represents a source of video data (i.e., raw, unencoded video data) and provides a continuous series of images (also referred to as "frames") of video data to video encoder 200, which encodes the image data. Video source 104 of source device 102 may include a video capture device, such as a camera, a video archive containing previously captured raw video, and / or a video feed interface that receives video from a video content provider. As a further alternative, video source 104 may generate graphics computer-based data as the source video, or a combination of live video, archived video, and computer-generated video. In each case, video encoder 200 encodes captured, pre-captured, or computer-generated video data. Video encoder 200 may rearrange the images from the order in which they were received (sometimes referred to as "display order") into a codec order for encoding and decoding. Video encoder 200 may generate a bitstream comprising the encoded video data. Source device 102 may then output the encoded video data onto computer-readable medium 110 via output interface 108 for receipt and / or retrieval by, for example, input interface 122 of destination device 116 .

[0025] Memory 106 of source device 102 and memory 120 of destination device 116 represent general purpose memory. In some examples, memory 106 and memory 120 can store raw video data, such as raw video from video source 104 and raw decoded video data from video decoder 300. Additionally or alternatively, memory 106 and memory 120 can each store software instructions that are executable by, for example, video encoder 200 and video decoder 300, respectively. Although in this example, memory 106 and memory 120 are shown as separate from video encoder 200 and video decoder 300, it should be understood that video encoder 200 and video decoder 300 can also include internal memory for functionally similar or equivalent purposes. Furthermore, memory 106 and memory 120 can store encoded video data, such as data output from video encoder 200 and input to video decoder 300. In some examples, portions of memory 106 and memory 120 can be allocated as one or more video buffers, for example, to store raw, decoded, and / or encoded video data.

[0026] The computer-readable medium 110 can represent any type of medium or device capable of transmitting encoded video data from the source device 102 to the destination device 116. In one example, the computer-readable medium 110 represents a communication medium that enables the source device 102 to transmit the encoded video data directly to the destination device 116 in real time, for example, via a radio frequency network or a computer-based network. The output interface 108 can modulate a transmission signal including the encoded video data, and the input interface 122 can demodulate the received transmission signal according to a communication standard such as a wireless communication protocol. The communication medium can include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium can form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium can include routers, switches, base stations, or any other facilities that facilitate communication from the source device 102 to the destination device 116.

[0027] In some examples, source device 102 may output the encoded data from output interface 108 to storage device 112. Similarly, destination device 116 may access the encoded data from storage device 112 via input interface 122. Storage device 112 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, a Blu-ray disc, a DVD, a CD-ROM, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.

[0028] In some examples, source device 102 may output encoded video data to file server 114 or another intermediate storage device that may store the encoded video generated by source device 102. Destination device 116 may access the stored video data from file server 114 via streaming or downloading. File server 114 may be any type of server device capable of storing and transmitting encoded video data to destination device 116. File server 114 may represent a network server (e.g., for a website), a file transfer protocol (FTP) server, a content delivery network device, or a network attached storage (NAS) device. Destination device 116 may access the encoded video data from file server 114 via any standard data connection, including an internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both, suitable for accessing encoded video data stored on file server 114. File server 114 and input interface 122 may be configured to operate according to a streaming protocol, a download transfer protocol, or a combination thereof.

[0029] Output interface 108 and input interface 122 may represent wireless transmitters / receivers, modems, wired network components (e.g., Ethernet cards), wireless communication components operating according to any of a variety of IEEE 802.11 standards, or other physical components. In examples where output interface 108 and input interface 122 include wireless components, output interface 108 and input interface 122 may be configured to transmit data, such as encoded video data, according to a cellular communication standard (e.g., 4G, 4G-LTE (Long Term Evolution), LTE Advanced, 5G, etc.). In some examples where output interface 108 includes a wireless transmitter, output interface 108 and input interface 122 may be configured to transmit data, such as encoded video data, according to other wireless standards (e.g., IEEE 802.11 specifications, IEEE 802.15 specifications (e.g., ZigBee 5G), etc.). TM ),Bluetooth TM Standards, etc.) for transmitting data, such as encoded video data. In some examples, source device 102 and / or destination device 116 may include respective system-on-chip (SoC) devices. For example, source device 102 may include an SoC device for performing functions belonging to video encoder 200 and / or output interface 108, and destination device 116 may include an SoC device for performing functions belonging to video decoder 300 and / or input interface 122.

[0030] The techniques disclosed herein can be applied to video encoding and decoding to support a variety of multimedia applications, such as over-the-air television broadcasting, cable television transmission, satellite television transmission, Internet streaming video transmission, such as Dynamic Adaptive Streaming over HTTP (DASH), digital video encoded onto a data storage medium, decoding of digital video stored on a data storage medium, or any other application.

[0031] The input interface 122 of the destination device 116 receives an encoded video bitstream from the computer-readable medium 110 (e.g., a communication medium, a storage device 112, a file server 114, etc.). The encoded video bitstream may include signaling information defined by the video encoder 200 and used by the video decoder 300, such as syntax elements with values ​​describing characteristics and / or processing of video blocks or other codec units (e.g., slices, pictures, groups of pictures, sequences, etc.). The display device 118 displays decoded images of the decoded video data to a user. The display device 118 may represent any of a variety of display devices, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0032] although Figure 1Although not shown, in some examples, the video encoder 200 and the video decoder 300 may each be integrated with an audio encoder and / or an audio decoder and may include appropriate MUX-DEMUX units or other hardware and / or software to process a multiplexed stream including audio and video in a common data stream. If applicable, the MUX-DEMUX units may conform to the ITU H.223 multiplexer protocol, or other protocols such as the User Datagram Protocol (UDP).

[0033] The video encoder 200 and the video decoder 300 can each be implemented as any of various suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When these techniques are partially implemented in software, the device can store the instructions of the software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the techniques of the present disclosure. Each of the video encoder 200 and the video decoder 300 can be included in one or more encoders or decoders, any of which can be integrated as part of a combined encoder / decoder (CODEC) in the respective device. The device including the video encoder 200 and / or the video decoder 300 can include an integrated circuit, a microprocessor, and / or a wireless communication device, such as a cellular phone.

[0034] The video encoder 200 and the video decoder 300 can operate in accordance with a video coding standard, such as ITU-T H.265, also known as High Efficiency Video Coding (HEVC), or an extension thereof, such as the multi-view and / or scalable video coding extension. Alternatively, the video encoder 200 and the video decoder 300 can operate in accordance with other proprietary or industry standards, such as ITU-T H.266, also known as Versatile Video Coding (VVC). A draft of the VVC standard is described in Bross et al., “Versatile Video Coding (Draft 6)”, Joint Video Experts Team (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 15th Meeting: July 3-12, 2019, Gothenburg, Sweden, JVET-O2001-vE (hereinafter referred to as “VVC Draft 6”). The latest draft of the VVC standard is described in Bross et al., “Versatile Video Coding (Draft 10)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 19th Meeting: by teleconference, June 22–July 1, 2020, JVET-S2001-vA (hereinafter referred to as “VVC Draft 10”). However, the techniques of this disclosure are not limited to any particular coding standard.

[0035] Typically, the video encoder 200 and the video decoder 300 can perform block-based encoding and decoding of images. The term "block" generally refers to a structure that includes data to be processed (e.g., data that has been encoded, decoded, or used in the encoding and / or decoding process). For example, a block can include a two-dimensional matrix of samples of luminance and / or chrominance data. Typically, the video encoder 200 and the video decoder 300 can encode and decode video data represented in a YUV (e.g., Y, Cb, Cr) format. That is, rather than encoding and decoding red, green, and blue (RGB) data of samples of an image, where the chrominance components may include red and blue hue chrominance components, the video encoder 200 and the video decoder 300 can encode and decode luminance and chrominance components. In some examples, the video encoder 200 converts received RGB formatted data into a YUV representation before encoding, and the video decoder 300 converts the YUV representation into an RGB format. Alternatively, pre-processing and post-processing units (not shown) can perform these conversions.

[0036] The present disclosure may generally refer to the encoding and decoding of an image (e.g., encoding and decoding) to include the process of encoding or decoding the data of the image. Similarly, the present disclosure may refer to the encoding and decoding of a block of an image to include the process of encoding or decoding the data of the block, such as prediction and / or residual encoding and decoding. The encoded video bitstream generally includes a series of values ​​of syntax elements that represent codec decisions (e.g., codec mode) and the division of the image into blocks. Therefore, references to a codec image or block should generally be understood to be the codec values ​​of the syntax elements that form the image or block.

[0037] HEVC defines various blocks, including codec units (CUs), prediction units (PUs), and transform units (TUs). According to HEVC, a video codec (e.g., video encoder 200) divides a codec tree unit (CTU) into CUs according to a quadtree structure. That is, the video encoder divides the CTU and CU into four equal, non-overlapping squares, and each node of the quadtree has zero or four child nodes. A node without child nodes may be referred to as a "leaf node," and the CU of such a leaf node may include one or more PUs and / or one or more TUs. The video codec may further divide the PU and TU. For example, in HEVC, the residual quadtree (RQT) represents the division of the TU. In HEVC, PU represents inter-prediction data, and TU represents residual data. An intra-predicted CU includes intra-prediction information, such as an intra-mode indication.

[0038] As another example, the video encoder 200 and the video decoder 300 can be configured to operate according to VVC. According to VVC, a video encoder (such as the video encoder 200) divides an image into multiple codec tree units (CTUs). The video encoder 200 can divide the CTU according to a tree structure such as a quadtree-binary tree (QTBT) structure or a multi-type tree (MTT) structure. The QTBT structure deletes the concept of multiple partition types, such as the separation between CU, PU and TU of HEVC. The QTBT structure includes two levels: a first level divided according to quadtree partitioning, and a second level divided according to binary tree partitioning. The root node of the QTBT structure corresponds to the CTU. The leaf nodes of the binary tree correspond to the codec units (CUs).

[0039] In the MTT partitioning structure, blocks can be partitioned using quadtree (QT) partitioning, binary tree (BT) partitioning, and one or more types of ternary tree (TT) partitioning. Ternary tree partitioning is the partitioning of a block into three sub-blocks. In some examples, ternary tree partitioning divides a block into three sub-blocks without dividing the original block through the center. Partition types in MTT (such as QT, BT, and TT) can be symmetric or asymmetric.

[0040] In some examples, the video encoder 200 and the video decoder 300 may use a single QTBT or MTT structure to represent each of the luma component and the chroma components, while in other examples, the video encoder 200 and the video decoder 300 may use two or more QTBT or MTT structures, such as one QTBT / MTT structure for the luma component and another QTBT / MTT structure for the two chroma components (or two QTBT / MTT structures for corresponding chroma components).

[0041] The video encoder 200 and the video decoder 300 can be configured to use per-HEVC quadtree partitioning, QTBT partitioning, MTT partitioning, or other partitioning structures. For the purpose of explanation, the description of the technology of the present disclosure is presented with respect to QTBT partitioning. However, it should be understood that the technology of the present disclosure can also be applied to video codecs configured to use quadtree partitioning or other types of partitioning.

[0042] These blocks (e.g., CTUs or CUs) can be grouped in a picture in various ways. As an example, a brick can refer to the rectangular area of ​​a CTU row within a particular slice in a picture. A slice can be the rectangular area of ​​a CTU within a particular slice column and a particular slice row in a picture. A slice column refers to a rectangular area of ​​a CTU with a height equal to the height of the picture and a width specified by a syntax element (e.g., in a picture parameter set). A slice row refers to a rectangular area of ​​a CTU with a height specified by a syntax element (e.g., in a picture parameter set) and a width equal to the width of the picture.

[0043] In some examples, a slice can be divided into multiple bricks, each of which can include one or more CTU rows within the slice. A slice that is not divided into multiple bricks can also be referred to as a brick. However, a brick that is a true subset of a slice cannot be referred to as a slice.

[0044] The tiles in an image can also be arranged in slices. A slice is an integer number of tiles of an image that can be individually contained in a single Network Abstraction Layer (NAL) unit. In some examples, a slice includes multiple complete slices or only a continuous sequence of complete tiles of a slice.

[0045] This disclosure may use "N×N" and "N by N" interchangeably to refer to the sample dimensions of a block (e.g., a CU or other video block) in terms of vertical and horizontal dimensions (e.g., 16×16 samples or 16 by 16 samples). In general, a 16×16 CU has 16 samples in the vertical direction (y=16) and 16 samples in the horizontal direction (x=16). Similarly, an N×N CU typically has N samples in the vertical direction and N samples in the horizontal direction, where N represents a non-negative integer value. The samples in a CU can be arranged in rows and columns. In addition, a CU does not necessarily have the same number of samples in the horizontal direction as in the vertical direction. For example, a CU may include N×M samples, where M is not necessarily equal to N.

[0046] The video encoder 200 encodes the video data of the CU representing the prediction and / or residual information and other information. The prediction information indicates how to predict the CU in order to form the prediction block of the CU. The residual information generally represents the sample-by-sample difference between the CU samples before encoding and the prediction block.

[0047] In order to predict a CU, the video encoder 200 may typically form a prediction block for the CU through inter-frame prediction or intra-frame prediction. Inter-frame prediction typically refers to predicting a CU from data of a previously coded image, while intra-frame prediction typically refers to predicting a CU from previously coded data of the same image. In order to perform inter-frame prediction, the video encoder 200 may use one or more motion vectors to generate the prediction block. The video encoder 200 may typically perform a motion search to identify a reference block that closely matches the CU, for example, based on the difference between the CU and the reference block. The video encoder 200 may calculate a difference metric using the sum of absolute differences (SAD), the sum of squared differences (SSD), the mean absolute difference (MAD), the mean squared difference (MSD), or other such difference calculation results to determine whether the reference block closely matches the current CU. In some examples, the video encoder 200 may use unidirectional prediction or bidirectional prediction to predict the current CU.

[0048] Some examples of VVC also provide an affine motion compensation mode, which can be considered an inter-frame prediction mode. In affine motion compensation mode, the video encoder 200 can determine two or more motion vectors representing non-translational motion, such as zooming in or out, rotation, perspective motion, or other irregular motion types.

[0049] To perform intra prediction, the video encoder 200 can select an intra prediction mode to generate the prediction block. Some examples of VVC provide sixty-seven intra prediction modes, including various directional modes, as well as a planar mode and a DC mode. In general, the video encoder 200 selects an intra prediction mode that describes the neighboring samples of a current block (e.g., a block of a CU) from which the samples of the current block are predicted. Assuming that the video encoder 200 encodes and decodes CTUs and CUs in raster scan order (from left to right, from top to bottom), such samples can typically be above, above and to the left of, or to the left of the current block in the same image as the current block.

[0050] The video encoder 200 encodes data indicating the prediction mode for the current block. For example, for inter-frame prediction mode, the video encoder 200 may encode data indicating which of various available inter-frame prediction modes to use, as well as motion information for the corresponding mode. For example, for unidirectional or bidirectional inter-frame prediction, the video encoder 200 may use Advanced Motion Vector Prediction (AMVP) or Merge Mode to encode motion vectors. The video encoder 200 may use a similar mode to encode motion vectors for affine motion compensation mode.

[0051] After prediction, such as intra-frame prediction or inter-frame prediction of a block, the video encoder 200 can calculate residual data for the block. The residual data, such as a residual block, represents the sample-by-sample difference between the block and a prediction block for the block, which is formed using a corresponding prediction mode. The video encoder 200 can apply one or more transforms to the residual block to produce transform data in a transform domain rather than a sample domain. For example, the video encoder 200 can apply a discrete cosine transform (DCT), an integer transform, a wavelet transform, or a conceptually similar transform to the residual video data. In addition, the video encoder 200 can apply a second transform after the first transform, such as a mode-dependent non-separable second transform (MDNSST), a signal-dependent transform, a Karhunen-Loeve transform (KLT), etc. The video encoder 200 generates transform coefficients after applying the one or more transforms.

[0052] As described above, after any transforms used to produce transform coefficients, the video encoder 200 can perform quantization of the transform coefficients. Quantization generally refers to the process of quantizing the transform coefficients to minimize the amount of data used to represent the transform coefficients, thereby providing further compression. By performing this quantization process, the video encoder 200 can reduce the bit depth associated with some or all transform coefficients. For example, the video encoder 200 can round down an n-bit value to an m-bit value during quantization, where n is greater than m. In some examples, to perform quantization, the video encoder 200 can perform a bitwise right shift on the value to be quantized.

[0053] After quantization, the video encoder 200 can scan the transform coefficients to produce a one-dimensional vector from the two-dimensional matrix including the quantized transform coefficients. The scan can be designed to place transform coefficients with higher energy (and therefore lower frequency) at the front of the vector and transform coefficients with lower energy (and therefore higher frequency) at the back of the vector. In some examples, the video encoder 200 can scan the quantized transform coefficients using a predefined scan order to produce a serialized vector, and then entropy encode the quantized transform coefficients of the vector. In other examples, the video encoder 200 can perform adaptive scanning. After scanning the quantized transform coefficients to form a one-dimensional vector, the video encoder 200 can entropy encode the one-dimensional vector, for example, according to context-adaptive binary arithmetic coding (CABAC). The video encoder 200 can also entropy encode the values ​​of syntax elements that describe metadata associated with the encoded video data for use by the video decoder 300 when decoding the video data.

[0054] To perform CABAC, the video encoder 200 may assign a context from a context model to a symbol to be transmitted. The context may relate to, for example, whether the adjacent values ​​of the symbol are zero. The probability determination may be based on the context assigned to the symbol.

[0055] The video encoder 200 may also generate syntax data, such as block-based syntax data, picture-based syntax data, and sequence-based syntax data, to the video decoder 300, such as in a picture header, block header, slice header, or other syntax data, such as a sequence parameter set (SPS), a picture parameter set (PPS), or a video parameter set (VPS). The video decoder 300 may also decode such syntax data to determine how to decode the corresponding video data.

[0056] In this way, the video encoder 200 can generate a bitstream that includes the encoded video data, such as syntax elements describing the division of an image into blocks (e.g., CUs) and prediction and / or residual information for the blocks. Ultimately, the video decoder 300 can receive the bitstream and decode the encoded video data.

[0057] Generally, the video decoder 300 performs a process that is inverse to the process performed by the video encoder 200 to decode the encoded video data of the bitstream. For example, the video decoder 300 can use CABAC to decode the value of a syntax element of the bitstream in a manner substantially similar to, but inverse to, the CABAC encoding process of the video encoder 200. The syntax element can define the partitioning information of the image into CTUs and divide each CTU according to a corresponding partitioning structure (e.g., a QTBT structure) to define CUs of the CTU. The syntax element can further define prediction and residual information for a block (e.g., a CU) of video data.

[0058] The residual information may be represented by, for example, quantized transform coefficients. The video decoder 300 may inverse quantize and inverse transform the quantized transform coefficients of the block to reproduce a residual block for the block. The video decoder 300 uses the signaled prediction mode (intra-frame or inter-frame prediction) and related prediction information (e.g., motion information for inter-frame prediction) to form a prediction block for the block. The video decoder 300 may then combine the prediction block and the residual block (on a sample-by-sample basis) to reproduce the original block. The video decoder 300 may perform additional processing, such as deblocking to reduce visual artifacts along block boundaries.

[0059] According to the technology of the present disclosure, as will be explained in more detail below, a method includes determining, by one or more processors, whether to use a single tree to encode and decode a first CTU; determining, by the one or more processors, a first TU based on a first set of TB samples based on encoding and decoding the first CTU using the single tree, the first set of TB samples including a first luma sample TB and a first two corresponding chroma samples TB; encoding and decoding a CU of the first CTU based on the first TU; determining, by the one or more processors, whether to use a dual tree to encode and decode a second CTU; determining, by the one or more processors, a second TU based on a second set of TB samples based on encoding and decoding the second CTU using the dual tree, the second set of TB samples including (I) a second luma sample TB or (II) a second two chroma sample TB; and encoding and decoding a CU of the second CTU based on the second TU, wherein the first TU includes a syntax structure for transforming the first set of TB samples, and the second TU includes a syntax structure for transforming the second set of TB samples.

[0060] According to other example techniques of the present disclosure, a device includes a memory configured to store the video data, and one or more processors implemented in circuitry and communicatively coupled to the memory, the one or more processors configured to: determine whether to use a single tree to encode and decode a first CTU; based on using a single tree to encode and decode the first CTU, determine a first TU based on a first group of TB samples, the first group of TB samples including a first luma sample TB and a first two corresponding chroma samples TB; encode and decode a CU of the first CTU based on the first TU; determine whether to use a dual tree to encode and decode a second CTU; based on using a dual tree to encode and decode the second CTU, determine a second TU based on a second group of TB samples, the second group of TB samples including (I) a second luma sample TB or (II) a second two chroma sample TB; and encode and decode the CU of the second CTU based on the second TU, wherein the first TU includes a syntax structure for transforming the first group of TB samples, and the second TU includes a syntax structure for transforming the second group of TB samples.

[0061] According to other example techniques of the present disclosure, a non-transitory computer-readable storage medium has instructions stored thereon, which, when executed by one or more processors, cause the one or more processors to: determine whether to use a single tree to encode and decode a first CTU; based on using a single tree to encode and decode the first CTU, determine a first TU based on a first group of TB samples, the first group of TB samples including a first luma sample TB and a first two corresponding chroma samples TB; encode and decode a CU of the first CTU based on the first TU; determine whether to use a dual tree to encode and decode a second CTU; based on using a dual tree to encode and decode the second CTU, determine a second TU based on a second group of TB samples, the second group of TB samples including (I) a second luma sample TB or (II) a second two chroma sample TB; and encode and decode the CU of the second CTU based on the second TU, wherein the first TU includes a syntax structure for transforming the first group of TB samples, and the second TU includes a syntax structure for transforming the second group of TB samples.

[0062] According to other example techniques of the present disclosure, a device includes means for determining whether to use a single tree to encode a first CTU; means for determining a first TU based on a first set of TB samples based on encoding and decoding the first CTU using the single tree, the first set of TB samples including a first luma sample TB and a first two corresponding chroma samples TB; means for encoding and decoding a CU of the first CTU based on the first TU; means for determining whether to use a dual tree to encode and decode a second CTU; means for determining a second TU based on a second set of TB samples based on encoding and decoding the second CTU using the dual tree, the second set of TB samples including (I) a second luma sample TB or (II) a second two chroma sample TB; and

[0063] Means for encoding and decoding the CU of the second CTU based on the second TU, wherein the first TU includes a syntax structure for transforming the first set of TB samples, and the second TU includes a syntax structure for transforming the second set of TB samples.

[0064] This disclosure may generally refer to "signaling" certain information, such as syntax elements. The term "signaling" may generally refer to the communication of values ​​of syntax elements and / or other data used to decode encoded video data. That is, the video encoder 200 may signal the values ​​of syntax elements in the bitstream. Generally, signaling refers to generating values ​​in the bitstream. As described above, the source device 102 may transmit the bitstream to the destination device 116 in substantially real time or in non-real time, such as may occur when storing syntax elements to the storage device 112 for later retrieval by the destination device 116.

[0065] Figure 2A and 2Bis a conceptual diagram illustrating an example quadtree binary tree (QTBT) structure 130 and a corresponding codec tree unit (CTU) 132. Solid lines represent quadtree splits and dashed lines represent binary tree splits. In each split (i.e., non-leaf) node of the binary tree, a flag is signaled to indicate which split type (i.e., horizontal or vertical) is used, where in this example, 0 indicates horizontal splitting and 1 indicates vertical splitting. For quadtree splitting, there is no need to specify the split type because the quadtree node divides a block horizontally and vertically into 4 sub-blocks of equal size. Therefore, the video encoder 200 can encode and the video decoder 300 can decode the syntax elements (e.g., split information) of the region tree level of the QTBT structure 130 (i.e., solid lines) and the syntax elements (e.g., split information) of the prediction tree level of the QTBT structure 130 (i.e., dashed lines). The video encoder 200 may encode and the video decoder 300 may decode video data, eg, prediction and transformation data, which is video data of a CU represented by a terminal leaf node of the QTBT structure 130 .

[0066] generally, Figure 2B The CTU 132 may be associated with parameters defining the size of a block corresponding to the nodes of the first and second level QTBT structures 130. These parameters may include the CTU size (indicating the size of the CTU 132 in samples), the minimum quadtree size (MinQTSize, indicating the minimum allowed quadtree leaf node size), the maximum binary tree size (MaxBTSize, indicating the maximum allowed binary tree root node size), the maximum binary tree depth (MaxBTDepth, indicating the maximum allowed binary tree depth), and the minimum binary tree size (MinBTSize, indicating the minimum allowed binary tree leaf node size).

[0067] The root node of the QTBT structure corresponding to the CTU can have four child nodes at the first level of the QTBT structure, and each child node can be divided according to the quadtree partitioning. That is, the nodes at the first level are either leaf nodes (no child nodes) or have four child nodes. The example of the QTBT structure 130 represents such nodes as child nodes including a parent node and a solid line with branches. If the nodes at the first level are not larger than the maximum allowed binary tree root node size (MaxBTSize), these nodes can be further divided by their respective binary trees. The binary tree splitting of a node can be iterated until the node resulting from the split reaches the minimum allowed binary tree leaf node size (MinBTSize) or the maximum allowed binary tree depth (MaxBTDepth). The example of the QTBT structure 130 represents such nodes as a dotted line with branches. The binary tree leaf node is called a coding unit (CU), which is used for prediction (e.g., intra-image or inter-image prediction) and transformation without any further partitioning. As described above, a CU can also be referred to as a "video block" or "block."

[0068] In one example of the QTBT partitioning structure, the CTU size is set to 128×128 (luminance sample and two corresponding 64×64 chrominance samples), the MinQTSize is set to 16×16, the MaxBTSize is set to 64×64, the MinBTSize (width and height) is set to 4, and the MaxBTDepth is set to 4. Quadtree partitioning is first applied to the CTU to generate a quadtree leaf node. The size of the quadtree leaf node can be from 16×16 (i.e., MinQTSize) to 128×128 (i.e., CTU size). If the leaf quadtree node is 128×128, it will not be further split by the binary tree because its size exceeds MaxBTSize (i.e., 64×64 in this case). Otherwise, the leaf quadtree node will be further divided by the binary tree. Therefore, the quadtree leaf node is also the root node of the binary tree, and the binary tree depth is 0. When the depth of the binary tree reaches MaxBTDepth (4 in this example), no further splitting is allowed. When the width of the binary tree node is equal to MinBTSize (4 in this example), it means that no further horizontal splitting is allowed. Similarly, a binary tree node with a height equal to MinBTSize means that no further vertical splitting of the binary tree node is allowed. As described above, the leaf nodes of the binary tree are called CUs and are further processed according to prediction and transformation without further division.

[0069] Figure 3 is a block diagram illustrating an example video encoder 200 that may perform the techniques of this disclosure. Figure 3This is provided for purposes of explanation and should not be considered limiting of the techniques broadly exemplified and described in this disclosure. For purposes of explanation, this disclosure describes the video encoder 200 in the context of video codec standards, such as the HEVC video codec standard and the developing H.266 video codec standard. However, the techniques of this disclosure are not limited to these video codec standards and are generally applicable to video encoding and decoding.

[0070] exist Figure 3 In the example of FIG, the video encoder 200 includes a video data memory 230, a mode selection unit 202, a residual generation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a filter unit 216, a decoded picture buffer (DPB) 218, and an entropy coding unit 220. Any or all of the video data memory 230, the mode selection unit 202, the residual generation unit 204, the transform processing unit 206, the quantization unit 208, the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the filter unit 216, the DPB 218, and the entropy coding unit 220 may be implemented in one or more processors or processing circuits. Moreover, the video encoder 200 may include additional or alternative processors or processing circuits to perform these and other functions.

[0071] The video data memory 230 may store video data to be encoded by the components of the video encoder 200. The video encoder 200 may receive video data from, for example, the video source 104 ( Figure 1 ) receives video data stored in the video data memory 230. The DPB 218 can act as a reference picture memory that stores reference video data for the video encoder 200 to predict subsequent video data. The video data memory 230 and the DPB 218 can be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. The video data memory 230 and the DPB 218 can be provided by the same memory device or separate memory devices. In various examples, the video data memory 230 can be on-chip with the other components of the video encoder 200, as shown, or off-chip relative to these components.

[0072] In this disclosure, references to the video data memory 230 should not be construed as limited to memory internal to the video encoder 200, unless specifically described as such, or to memory external to the video encoder 200, unless specifically described as such. Rather, references to the video data memory 230 should be understood as reference memory that stores video data received by the video encoder 200 for encoding (e.g., video data of a current block to be encoded). Figure 1 The memory 106 may also provide temporary storage of outputs from the various units of the video encoder 200 .

[0073] Shown Figure 3 The various units of the video encoder 200 are described to help understand the operations performed by the video encoder 200. These units can be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Fixed-function circuits refer to circuits that provide specific functions and are pre-set on executable operations. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functions in executable operations. For example, a programmable circuit can execute software or firmware so that the programmable circuit operates in a manner defined by the instructions of the software or firmware. Fixed-function circuits can execute software instructions (for example, receive parameters or output parameters), but the type of operation performed by the fixed-function circuit is generally immutable. In some examples, one or more units can be different circuit blocks (fixed-function or programmable), and in some examples, one or more units can be integrated circuits.

[0074] The video encoder 200 may include an arithmetic logic unit (ALU), an elementary function unit (EFU), a digital circuit, an analog circuit, and / or a programmable core formed by a programmable circuit. In an example where the operation of the video encoder 200 is performed using software executed by the programmable circuit, the memory 106 ( Figure 1 ) may store instructions (eg, object code) for software received and executed by the video encoder 200, or another memory (not shown) within the video encoder 200 may store such instructions.

[0075] The video data memory 230 is configured to store received video data. The video encoder 200 can retrieve images of the video data from the video data memory 230 and provide the video data to the residual generation unit 204 and the mode selection unit 202. The video data in the video data memory 230 can be original video data to be encoded.

[0076] The mode selection unit 202 includes a motion estimation unit 222, a motion compensation unit 224, and an intra prediction unit 226. The mode selection unit 202 may include additional functional units to perform video prediction according to other prediction modes. For example, the mode selection unit 202 may include a palette unit, an intra-block reproduction unit (which may be part of the motion estimation unit 222 and / or the motion compensation unit 224), an affine unit, a linear model (LM) unit, etc.

[0077] The mode selection unit 202 typically coordinates multiple encoding processes to test combinations of encoding parameters and the resulting rate-distortion values ​​of such combinations. The encoding parameters may include the division of a CTU into CUs, the prediction mode of the CU, the transform type of the CU's residual data, the quantization parameter of the CU's residual data, etc. The mode selection unit 202 may ultimately select a combination of encoding parameters that has a better rate-distortion value than other tested combinations.

[0078] The video encoder 200 may divide the image retrieved from the video data memory 230 into a series of CTUs and encapsulate one or more CTUs into a slice. The mode selection unit 202 may divide the CTUs of the image according to a tree structure, such as the QTBT structure or quadtree structure of HEVC described above. As described above, the video encoder 200 may form one or more CUs by dividing the CTUs according to the tree structure. Such CUs may also be generally referred to as "video blocks" or "blocks."

[0079] Generally speaking, the mode select unit 202 also controls its components (e.g., the motion estimation unit 222, the motion compensation unit 224, and the intra prediction unit 226) to generate a prediction block for the current block (e.g., the current CU, or, in HEVC, the overlapping portion of the PU and TU). For inter prediction of the current block, the motion estimation unit 222 may perform a motion search to identify one or more closely matching reference blocks in one or more reference pictures (e.g., one or more previously coded pictures stored in the DPB 218). Specifically, the motion estimation unit 222 may calculate a value indicating how similar the potential reference blocks are to the current block, for example, based on the sum of absolute differences (SAD), sum of squared differences (SSD), mean absolute difference (MAD), mean squared difference (MSD), etc. The motion estimation unit 222 may generally perform these calculations using the sample-by-sample differences between the current block and the reference block under consideration. The motion estimation unit 222 may identify the reference block with the lowest value obtained from these calculations, indicating the reference block that most closely matches the current block.

[0080] Motion estimation unit 222 may form one or more motion vectors (MVs) that define the position of a reference block in a reference image relative to the position of a current block in a current image. Motion estimation unit 222 may then provide the motion vectors to motion compensation unit 224. For example, for unidirectional inter prediction, motion estimation unit 222 may provide a single motion vector, while for bidirectional inter prediction, motion estimation unit 222 may provide two motion vectors. Motion compensation unit 224 may then use the motion vectors to generate a prediction block. For example, motion compensation unit 224 may use the motion vectors to retrieve data for the reference block. As another example, if the motion vectors have fractional sample precision, motion compensation unit 224 may interpolate the values ​​of the prediction block based on one or more interpolation filters. Furthermore, for bidirectional inter prediction, motion compensation unit 224 may retrieve data for two reference blocks identified by corresponding motion vectors and combine the retrieved data, for example, by sample-by-sample averaging or weighted averaging.

[0081] As another example, for intra prediction or intra prediction codecs, the intra prediction unit 226 can generate the prediction block from samples adjacent to the current block. For example, for directional mode, the intra prediction unit 226 can generally mathematically combine the values ​​of the adjacent samples and pad these calculated values ​​in the defined direction of the current block to produce the prediction block. As another example, for DC mode, the intra prediction unit 226 can calculate the average of the adjacent samples of the current block and generate the prediction block to include the final average value for each sample of the prediction block.

[0082] Mode selection unit 202 provides the prediction block to residual generation unit 204. Residual generation unit 204 receives the original uncoded version of the current block from video data memory 230 and the prediction block from mode selection unit 202. Residual generation unit 204 calculates the sample-by-sample difference between the current block and the prediction block. The resulting sample-by-sample difference defines a residual block for the current block. In some examples, residual generation unit 204 may also determine the difference between sample values ​​in the residual block to generate the residual block using residual differential pulse code modulation (RDPCM). In some examples, residual generation unit 204 may be formed using one or more subtractor circuits that perform binary subtraction.

[0083] In the example where the mode selection unit 202 divides the CU into PUs, each PU may be associated with a luma prediction unit and a corresponding chroma prediction unit. The video encoder 200 and the video decoder 300 may support PUs of various sizes. As described above, the size of a CU may refer to the size of the luma codec block of the CU, and the size of a PU may refer to the size of the luma prediction unit of the PU. Assuming that the size of a particular CU is 2N×2N, the video encoder 200 may support PU sizes of 2N×2N or N×N for intra prediction, and symmetric PU sizes of 2N×2N, 2N×N, N×2N, N×N, or similar for inter prediction. The video encoder 200 and the video decoder 300 may also support asymmetric partitioning of PU sizes of 2N×nU, 2N×nD, nL×2N, and nR×2N for inter prediction.

[0084] In the example where the mode selection unit 202 does not further split the CU into multiple PUs, each CU can be associated with a luma codec block and a corresponding chroma codec block. As described above, the size of a CU can refer to the size of the luma codec block of the CU. The video encoder 200 and the video decoder 300 can support CU sizes of 2N×2N, 2N×N, or N×2N.

[0085] For other video codecs, such as intra-block reproduction mode codecs, affine mode codecs, and linear model (LM) mode codecs, as a few examples, the mode selection unit 202 generates a prediction block for the current block being encoded via the corresponding unit associated with the codec. In some examples, such as palette mode codecs, the mode selection unit 202 may not generate a prediction block, but instead generate syntax elements that indicate how to reconstruct the block based on the selected palette. In this mode, the mode selection unit 202 may provide these syntax elements to the entropy coding unit 220 for encoding.

[0086] As described above, the residual generation unit 204 receives video data of a current block and a corresponding prediction block. The residual generation unit 204 then generates a residual block for the current block. To generate the residual block, the residual generation unit 204 calculates the sample-by-sample difference between the prediction block and the current block.

[0087] Transform processing unit 206 applies one or more transforms to the residual block to generate a block of transform coefficients (referred to herein as a "transform coefficient block"). Transform processing unit 206 may apply various transforms to the residual block to form the transform coefficient block. For example, transform processing unit 206 may apply a discrete cosine transform (DCT), a directional transform, a Karhunen-Lössl transform (KLT), or a conceptually similar transform to the residual block. In some examples, transform processing unit 206 may perform multiple transforms on the residual block, such as a primary transform and a secondary transform, such as a rotation transform. In some examples, transform processing unit 206 does not apply a transform to the residual block.

[0088] For example, the transform processing unit 206 may determine whether to use a single tree to encode the first CTU. The transform processing unit 206 may determine the first TU based on the first luma sample TB and the first two corresponding chroma sample TBs based on the use of the single tree to encode the first CTU. Additionally or alternatively, the transform processing unit 206 may determine whether to use a dual tree to encode the second CTU. The transform processing unit 206 may determine the second TU based on (I) the second luma sample TB or (II) the second two chroma sample TBs based on the use of the dual tree to encode the second CTU.

[0089] Quantization unit 208 may quantize the transform coefficients in the transform coefficient block to produce a quantized transform coefficient block. Quantization unit 208 may quantize the transform coefficients of the transform coefficient block according to a QP value associated with the current block. Video encoder 200 (e.g., via mode select unit 202) may adjust the degree of quantization applied to the transform coefficient block associated with the current block by adjusting the QP value associated with the CU. Quantization may introduce information loss, and thus, the quantized transform coefficients may have lower precision than the original transform coefficients produced by transform processing unit 206.

[0090] The inverse quantization unit 210 and the inverse transform processing unit 212 may apply inverse quantization and inverse transform, respectively, to the quantized transform coefficient block to reconstruct a residual block from the transform coefficient block. The reconstruction unit 214 may generate a reconstructed block corresponding to the current block (although possibly with some degree of distortion) based on the reconstructed residual block and the prediction block generated by the mode selection unit 202. For example, the reconstruction unit 214 may add samples of the reconstructed residual block to corresponding samples of the prediction block generated by the mode selection unit 202 to generate the reconstructed block.

[0091] Filter unit 216 may perform one or more filtering operations on the reconstructed block. For example, filter unit 216 may perform a deblocking operation to reduce blocking artifacts along the edges of the CU. In some examples, the operations of filter unit 216 may be skipped.

[0092] The video encoder 200 stores the reconstructed blocks in the DPB 218. For example, in examples where the operation of the filter unit 216 is not required, the reconstruction unit 214 can store the reconstructed blocks in the DPB 218. In examples where the operation of the filter unit 216 is required, the filter unit 216 can store the filtered reconstructed blocks in the DPB 218. The motion estimation unit 222 and the motion compensation unit 224 can retrieve a reference image from the DPB 218, which is formed by the reconstructed (and possibly filtered) blocks, to perform inter-frame prediction on blocks of subsequently encoded images. In addition, the intra-frame prediction unit 226 can use the reconstructed blocks in the DPB 218 of the current image to perform intra-frame prediction on other blocks in the current image.

[0093] In general, the entropy coding unit 220 may entropy encode syntax elements received from other functional components of the video encoder 200. For example, the entropy coding unit 220 may entropy encode quantized transform coefficient blocks from the quantization unit 208. As another example, the entropy coding unit 220 may entropy encode prediction syntax elements (e.g., motion information for inter-frame prediction or intra-frame mode information for intra-frame prediction) from the mode selection unit 202. The entropy coding unit 220 may perform one or more entropy coding operations on the syntax elements, another example of video data, to generate entropy-encoded data. For example, the entropy coding unit 220 may perform a context-adaptive variable length coding (CAVLC) operation, a CABAC operation, a variable-to-variable (V2V) length coding operation, a syntax-based context-adaptive binary arithmetic coding (SBAC) operation, a probability interval partitioning entropy (PIPE) coding operation, an exponential-Golomb coding operation, or another type of entropy coding operation on the data. In some examples, entropy encoding unit 220 may operate in a bypass mode, where syntax elements are not entropy encoded.

[0094] The video encoder 200 may output a bitstream including entropy-encoded syntax elements required to reconstruct a block of a slice or picture. Specifically, the entropy encoding unit 220 may output the bitstream.

[0095] The operations described above are described for blocks. This description should be understood as operations on luma codec blocks and / or chroma codec blocks. As described above, in some examples, the luma codec blocks and chroma codec blocks are the luma components and chroma components of a CU. In some examples, the luma codec blocks and chroma codec blocks are the luma components and chroma components of a PU.

[0096] In some examples, the operations performed for the luma codec block do not need to be repeated for the chroma codec block. As an example, the operations for identifying the motion vector (MV) and reference image for the luma codec block do not need to be repeated to identify the MV and reference image for the chroma block. Instead, the MV of the luma codec block can be scaled to determine the MV of the chroma block, and the reference image can be the same. As another example, the intra prediction process can be the same for the luma codec block and the chroma codec block.

[0097] The video encoder 200 represents an example of a device configured to encode video data, the device including a memory configured to store video data, and one or more processing units implemented in circuitry and configured to determine whether to use a single tree to encode and decode a first CTU; based on using a single tree to encode and decode the first CTU, determine a first TU based on a first set of TB samples, the first set of TB samples including a first luma sample TB and a first two corresponding chroma samples TB; encode a CU of the first CTU based on the first TU; determine whether to use a dual tree to encode and decode a second CTU; based on using a dual tree to encode and decode the second CTU, determine a second TU based on a second set of TB samples, the second set of TB samples including (I) a second luma sample TB or (II) a second two chroma samples TB; and encode a CU of the second CTU based on the second TU, wherein the first TU includes a syntax structure for transforming the first set of TB samples and the second TU includes a syntax structure for transforming the second set of TB samples.

[0098] Figure 4 is a block diagram illustrating an example video decoder 300 that may perform the techniques of this disclosure. Figure 4 This is provided for the purpose of explanation and does not limit the techniques broadly exemplified and described in this disclosure. For the purpose of explanation, this disclosure describes a video decoder 300 based on techniques of VVC and HEVC. However, the techniques of this disclosure can be performed by video codec devices configured for other video codec standards.

[0099] exist Figure 4In the example of FIG, the video decoder 300 includes a coded picture buffer (CPB) memory 320, an entropy decoding unit 302, a prediction processing unit 304, an inverse quantization unit 306, an inverse transform processing unit 308, a reconstruction unit 310, a filter unit 312, and a decoded picture buffer (DPB) 314. Any or all of the CPB memory 320, the entropy decoding unit 302, the prediction processing unit 304, the inverse quantization unit 306, the inverse transform processing unit 308, the reconstruction unit 310, the filter unit 312, and the DPB 314 may be implemented in one or more processors or processing circuits. Moreover, the video decoder 300 may include additional or alternative processors or processing circuits to perform these and other functions.

[0100] The prediction processing unit 304 includes a motion compensation unit 316 and an intra prediction unit 318. The prediction processing unit 304 may include additional units to perform prediction according to other prediction modes. As an example, the prediction processing unit 304 may include a palette unit, an intra-block rendering unit (which may form part of the motion compensation unit 316), an affine unit, a linear model (LM) unit, etc. In other examples, the video decoder 300 may include more, fewer, or different functional components.

[0101] The CPB memory 320 may store video data, such as an encoded video bitstream, to be decoded by components of the video decoder 300. The video data stored in the CPB memory 320 may be, for example, read from the computer readable medium 110 ( Figure 1 ). The CPB memory 320 may include a CPB that stores coded video data (e.g., syntax elements) from a coded video bitstream. In addition, the CPB memory 320 may store video data other than syntax elements of coded images, such as temporary data representing outputs from various units of the video decoder 300. The DPB 314 typically stores decoded images that the video decoder 300 may output and / or use as reference video data when decoding subsequent data or images of the coded video bitstream. The CPB memory 320 and the DPB memory 314 may be formed by any of a variety of memory devices, such as DRAM including SDRAM, MRAM, RRAM, or other types of memory devices. The CPB memory 320 and the DPB memory 314 may be provided by the same memory device or by separate memory devices. In various examples, the CPB memory 320 may be on-chip with the other components of the video decoder 300, or off-chip relative to these components.

[0102] Additionally or alternatively, in some examples, video decoder 300 may retrieve the video from memory 120 ( Figure 1) to retrieve the encoded video data. That is, the memory 120 may utilize the CPB memory 320 to store data as described above. Similarly, when some or all of the functions of the video decoder 300 are implemented in software to be executed by the processing circuitry of the video decoder 300, the memory 120 may store instructions to be executed by the video decoder 300.

[0103] Figure 4 The various units shown in FIG are shown to aid in understanding the operations performed by the video decoder 300. These units may be implemented as fixed function circuits, programmable circuits, or a combination thereof. Figure 3 , fixed-function circuits refer to circuits that provide specific functions and are preset on the operations that can be performed. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functions in the operations that can be performed. For example, a programmable circuit can execute software or firmware so that the programmable circuit operates in a manner defined by the instructions of the software or firmware. Fixed-function circuits can execute software instructions (for example, receive parameters or output parameters), but the type of operation performed by the fixed-function circuit is generally immutable. In some examples, one or more units may be different circuit blocks (fixed-function or programmable), and in some examples, one or more units may be integrated circuits.

[0104] The video decoder 300 may include an ALU, an EFU, digital circuits, analog circuits, and / or a programmable core formed of programmable circuits. In examples where the operation of the video decoder 300 is performed by software executed on the programmable circuits, on-chip or off-chip memory may store instructions (e.g., object code) of the software received and executed by the video decoder 300.

[0105] The entropy decoding unit 302 may receive the encoded video data from the CPB and perform entropy decoding on the video data to reproduce the syntax elements. The prediction processing unit 304, the inverse quantization unit 306, the inverse transform processing unit 308, the reconstruction unit 310, and the filter unit 312 may generate decoded video data based on the syntax elements extracted from the bitstream.

[0106] Typically, the video decoder 300 reconstructs an image on a block-by-block basis. The video decoder 300 may perform a reconstruction operation on each block separately (where the block currently being reconstructed, ie, the decoded block, may be referred to as the "current block").

[0107] The entropy decoding unit 302 may entropy decode syntax elements defining the quantized transform coefficients of the quantized transform coefficient block, as well as transform information, such as a QP and / or a transform mode indication. The inverse quantization unit 306 may use the QP associated with the quantized transform coefficient block to determine the degree of quantization and, similarly, the degree of inverse quantization to be applied by the inverse quantization unit 306. The inverse quantization unit 306 may, for example, perform a bitwise left shift operation to inverse quantize the quantized transform coefficients. The inverse quantization unit 306 may thereby form a transform coefficient block comprising the transform coefficients.

[0108] After the inverse quantization unit 306 forms the transform coefficient block, the inverse transform processing unit 308 may apply one or more inverse transforms to the transform coefficient block to generate a residual block associated with the current block. For example, the inverse transform processing unit 308 may apply an inverse DCT, an inverse integer transform, an inverse Karhunen-Loewenstein transform (KLT), an inverse rotational transform, an inverse directional transform, or another inverse transform to the transform coefficient block.

[0109] For example, the inverse transform processing unit 308 may determine whether to use a single tree to encode the first CTU. The inverse transform processing unit 308 may determine the first TU based on the first luma sample TB and the first two corresponding chroma sample TBs based on the use of a single tree to encode the first CTU. Additionally or alternatively, the inverse transform processing unit 308 may determine whether to use a dual tree to encode the second CTU. The inverse transform processing unit 308 may determine the second TU based on (I) the second luma sample TB or (II) the second two chroma sample TBs based on the use of a dual tree to encode the second CTU.

[0110] In addition, prediction processing unit 304 generates a prediction block based on the prediction information syntax element entropy decoded by entropy decoding unit 302. For example, if the prediction information syntax element indicates that the current block is inter-predicted, motion compensation unit 316 can generate the prediction block. In this case, the prediction information syntax element can indicate a reference picture in DPB 314 (from which the reference block is retrieved) and a motion vector that identifies the position of the reference block in the reference picture relative to the current block in the current picture. Motion compensation unit 316 can generally generate the prediction block in a manner substantially similar to motion compensation unit 224 ( Figure 3 ) to perform the inter-frame prediction process in the manner described.

[0111] As another example, if the prediction information syntax element indicates that the current block is intra-predicted, the intra-prediction unit 318 may generate the prediction block according to the intra-prediction mode indicated by the prediction information syntax element. Likewise, the intra-prediction unit 318 may generally generate the prediction block in a manner substantially similar to that of the intra-prediction unit 226 ( Figure 3The intra prediction process is performed in the manner described in ). The intra prediction unit 318 can retrieve the data of the neighboring samples of the current block from the DPB 314.

[0112] The reconstruction unit 310 may use the prediction block and the residual block to reconstruct the current block. For example, the reconstruction unit 310 may add samples of the residual block to corresponding samples of the prediction block to reconstruct the current block.

[0113] The filter unit 312 may perform one or more filtering operations on the reconstructed block. For example, the filter unit 312 may perform a deblocking operation to reduce blocking artifacts along the edges of the reconstructed block. The operations of the filter unit 312 need not be performed in all examples.

[0114] The video decoder 300 may store the reconstructed block in the DPB 314. For example, in an example where the operation of the filter unit 312 is not performed, the reconstruction unit 310 may store the reconstructed block in the DPB 314. In an example where the operation of the filter unit 312 is performed, the filter unit 312 may store the filtered reconstructed block in the DPB 314. As described above, the DPB 314 may provide reference information to the prediction processing unit 304, such as samples of the current image for intra-frame prediction and previously decoded images for subsequent motion compensation. In addition, the video decoder 300 may output a decoded image (e.g., a decoded video) from the DPB 314 for subsequent presentation on a display device, such as Figure 1 display device 118.

[0115] In this manner, the video decoder 300 represents an example of a video decoding device that includes a memory configured to store video data, and one or more processors implemented in circuitry and configured to determine whether to use a single tree to encode and decode a first CTU; based on using a single tree to encode and decode the first CTU, determine a first TU based on a first set of TB samples, the first set of TB samples including a first luma sample TB and a first two corresponding chroma samples TB; decode a CU of the first CTU based on the first TU; determine whether to use a dual tree to encode and decode a second CTU; based on using a dual tree to encode and decode the second CTU, determine a second TU based on a second set of TB samples, the second set of TB samples including (I) a second luma sample TB or (II) a second two chroma sample TB; and decode a CU of the second CTU based on the second TU, wherein the first TU includes a syntax structure for transforming the first set of TB samples and the second TU includes a syntax structure for transforming the second set of TB samples.

[0116] Video codec standards include ITU-T H.261, ISO / IEC MPEG-1 Visual, ITU-T H.262 or ISO / IEC MPEG-2 Visual, ITU-T H.263, ISO / IEC MPEG-4 Visual, and ITU-T H.264 (also known as ISO / IEC MPEG-4 AVC), including its Scalable Video Codec (SVC) and Multi-view Video Codec (MVC) extensions. In April 2013, HEVC was finalized by the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC MPEG Joint Collaboration Team on Video Coding (JCT-VC).

[0117] JVET is a collaborative team consisting of MPEG and VCEG of ITU-T Study Group 16 that is working on a new video codec standard called VVC. The main goal of VVC is to achieve significant improvements in compression performance over the existing HEVC standard, helping to deploy higher-quality video services and emerging applications such as 360° immersive multimedia and high dynamic range (HDR) video.

[0118] A video codec (e.g., video encoder 200 or video decoder 300) uses QP values ​​to determine the step size used to quantize / inverse quantize transform coefficients. In some examples, QP values ​​are specified in the range of -QpBdOffset to 63 (inclusive), with 63 being the maximum QP value. QpBdOffset is specified as a fixed value for a specific bit depth, inferred to be 6*(bit depth 8). A QP prime, calculated by adding QpBdOffset to the specified QP value, is used to infer the actual step size. For ease of description, the QP and QP prime are used interchangeably in the remainder of this disclosure, with the understanding that only the QP value is used in most of the QP inference process, while the QP prime is only used in the final stage before step size determination. A change of 1 in QP value corresponds to a roughly 12% change in step size; a change of 6 in QP value corresponds to a 2x change in step size. Higher quantization parameter values ​​result in larger quantization step sizes and a coarser representation of the quantized coefficients.

[0119] A quantization group (QG) is a region of a picture where a QP offset parameter can be signaled. When signaled, the offset parameter is used to infer the QP value for one or more codec blocks in the quantization group. This offset parameter is often referred to as a delta QP value.

[0120] The video encoder 200 may signal the QG by, for example, a maximum depth from a codec tree block (CTB) level or using a variable indicating the number of samples in a group relative to the CTB size.

[0121] Note that in some examples, the QP value of a QG is signaled only for the first block with non-zero coefficients in that QG. For blocks in the QG that precede the first block with non-zero coefficients, the QP value can be inferred as a predicted QP value, as described later in this disclosure. Blocks with zero coefficients do not require any QP value because there are no coefficients to quantize / inverse quantize. Instead, the video decoder 300 can use the predicted QP value in the deblocking process to determine some parameters of the deblocking filter to be applied. Typically, a QG refers to a luma QG, where the luma QG is the area where the codec blocks have the same predicted QP value.

[0122] In VVC draft 6, the following are syntax elements related to the QP parameters for luma and chroma inferred by the video encoder 200 and the video decoder 300 .

[0123] Sequence parameter set

[0124] min_qp_prime_ts_minus4 ue(v)

[0125] Table 1

[0126] The syntax element min_qp_prime_ts_minus4 in Table 1 is used to infer the minimum QP value for transform skip mode.

[0127]

[0128] Table 2

[0129] The syntax elements in Table 2 are used to specify a mapping table to infer chrominance QP values ​​from luma QP values ​​of Cb, Cr, and joint Cb-Cr components.

[0130]

[0131] Table 3

[0132] The syntax elements related to QP values ​​in Table 3 apply to the luma adaptive QP offset, which will be applied to the luma deblocking process.

[0133] Image parameter set

[0134]

[0135] Table 4

[0136] The syntax elements in Table 4 are used to infer the initial QP of the slice of the reference PPS; and specify the QG of luma and chroma and the offset value table indicating the CU chroma offset.

[0137] Strip Head

[0138]

[0139]

[0140] Table 5

[0141] The syntax elements in Table 5 further specify the increments of the initial QP of the slice for the luma and chroma components.

[0142] Transformation Unit Syntax

[0143]

[0144] Table 6

[0145] The syntax elements of Table 6 specify the CU-level QP offsets for the luma and chroma components used to derive the QP value of the QG.

[0146] In VVC draft 6, the video encoder 200 or video decoder 300 can infer the QP values ​​for luma and chroma as follows. For the luma component, the slice QP can be inferred from the initial QP (signaled in the PPS) and the QP offset signaled in the slice. A QG is specified, and a predicted QP is derived for each QP, i.e., QP Y_PRED Based on the relative position of QG in the QG brick and QG's CTB row, QP Y_PRED The value of QP may vary. Y The value is obtained by QP Y_PRED and the incremental QP signaled to the QG is inferred.

[0147] It should be noted that QP can be inferred for all codec blocks in the QG Y_PRED However, delta QP is signaled only for blocks under certain conditions. Therefore, QP values ​​can be inferred only in certain circumstances.

[0148] The chroma QP is inferred by mapping the luma QP of the sample collocated with the center of the chroma block and any QP offset that may be applicable to the codec block. Similar to the luma QP offset, the chroma QP offset is signaled for the first chroma codec block with non-zero coefficients in the chroma quantization group.

[0149] The video encoder 200 and the video decoder 300 can follow the inference process of the QP value in VVC draft 6, as described below. The input to the inference process is: 1) the luma position (xCb, yCb), which specifies the position of the upper left luma sample of the current codec block relative to the upper left luma sample of the current image; 2) the variable cbWidth, which specifies the width of the current codec block in luma samples; 3) the variable cbHeight, which specifies the height of the current coding block in luma samples; and 4) the variable treeType, which specifies whether a single tree (SINGLE_TREE) or a dual tree is used to partition the CTU, and when a dual tree is used, whether the luma component (DUAL_TREE_LUMA) or the chroma component (DUAL_TREE_CHROMA) is currently being processed.

[0150] During the inference process, the luma quantization parameter Qp′Y and the chroma quantization parameters Qp′Cb and Qp′Cr are inferred.

[0151] The luma position (xQg, yQg) specifies the top-left luma sample of the current quantization group relative to the top-left luma sample of the current image. The horizontal and vertical positions xQg and yQg are set equal to CuQgTopLeftX and CuQgTopLeftY, respectively.

[0152] It should be noted that the current QG is the same qP shared by the CTB internal Y_PRED The width and height of the current QG are equal to the width and height of the codec tree node, and the brightness sample position of the upper left corner of the codec tree node is assigned to the variables CuQgTopLeftX and CuQgTopLeftY.

[0153] When treeType is equal to SINGLE_TREE or DUAL_TREE_LUMA, the video decoder 300 can infer the predicted luma quantization parameter qP by the following ordered steps: Y_PRED Step 1) Variable qP Y_PREV The inference is as follows: If one or more of the following conditions are true, then qP Y_PREV is set equal to SliceQp Y :a) The current QG is the first QG in the stripe; b) The current QG is the first QG in the brick; c) The current QG is the first QG in the CTB row of the brick, and entropy_coding_sync_enabled_flag is equal to 1. Otherwise, qP Y_PREV Set equal to the luma quantization parameter Qp of the last luma codec unit in the previous QG in decoding order Y .

[0154] Step 2) Invoke the inference procedure for neighbor block availability specified in VVC draft 6, clause 6.4.4, with position (xCurr, yCurr) set equal to (xCb, yCb), neighbor position (xNbY, yNbY) set equal to (xQg–1, yQg), checkPredModeY set to FALSE, cIdx set to 0 as input, and assign the output to availableA. Variable qP Y_A The inference is as follows: If one or more of the following conditions are true, then qP Y_A Set equal to qP Y_PREV : availableA is equal to false (FALSE); or the CTB containing the luma codec block covering the luma position (xQg–1, yQg) is not equal to the CTB containing the current luma codec block at (xCb, yCb), for example, all of the following conditions are true: (xQg–1)>>CtbLog2SizeY is not equal to (xCb)>>CtbLog2SizeY and (yQg)>>CtbLog2SizeY is not equal to (yCb)>>CtbLog2SizeY. Otherwise, qP Y_A Set equal to the luma quantization parameter Qp of the codec unit containing the luma codec block cover (xQg–1,yQg) Y .

[0155] Step 3) Invoke the inference process for neighbor block availability specified in VVC draft 6, clause 6.4.4, with position (xCurr, yCurr) set equal to (xCb, yCb), neighbor position (xNbY, yNbY) set equal to (xQg, yQg–1), checkPredModeY set to FALSE, cIdx set to 0 as input, and assign the output to availableB. Variable qP Y_B The inference is as follows: If one or more of the following conditions are true, then qP Y_B Set equal to qP Y_PREV : availableB is equal to FALSE; or the CTB of the luma codec block covering the luma position (xQg, yQg–1) is not equal to the CTB of the current luma codec block contained at (xCb, yCb), for example, all of the following conditions are true: (xQg)>>CtbLog2SizeY is not equal to (xCb)>>CtbLog2SizeY and (yQg–1)>>CtbLog2SizeY is not equal to (yCb)>>CtbLog2SizeY. Otherwise, qP Y_B Set equal to the luma quantization parameter Qp of the codec unit containing the luma codec block cover (xQg, yQg–1) Y .

[0156] Step 4) Predicted luminance quantization parameter qP Y_PRED The inference is as follows: If all of the following conditions are true, then qP Y_PRED Set equal to the luma quantization parameter Qp of the codec unit containing the luma codec block cover (xQg, yQg–1) Y : availableB is equal to TRUE; and the current quantization group is the first quantization group in the CTB row within the brick, and entropy_coding_sync_enabled_flag is equal to 0. Otherwise, qP Y_PRED The inference is as follows:

[0157] qP Y_PRED =(qP Y_A +qP Y_B +1)>>1 (8-932)

[0158] variable Qp Y The inference is as follows:

[0159] Qp Y =

[0160] ((qP Y_PRED +CuQpDeltaVal+64+2*QpBdOffset Y )%(64+QpBdOffset Y ))–QpBdOffsetY(8-932)

[0161] The luminance quantization parameter Qp′Y is inferred as follows:

[0162] Qp′ Y =Qp Y +QpBdOffset Y (8-934)

[0163] When ChromaArrayType is not equal to 0 and treeType is equal to SINGLE_TREE or DUAL_TREE_CHROMA, the following applies: When treeType is equal to DUAL_TREE_CHROMA, the variable Qp Y Set equal to the luma quantization parameter of the luma codec unit covering the luma position (xCb+cbWidth / 2,yCb+cbHeight / 2). Variable qP Cb 、qP Cr and qP CbCr The inference is as follows:

[0164] qPi Chroma =Clip3(-QpBdOffsetC ,63,Qp Y ) (8-935)

[0165] qPi Cb =ChromaQpTable[0][qPi Chroma ] (8-936)

[0166] qPi Cr =ChromaQpTable[1][qPi Chroma ] (8-937)

[0167] qPi CbCr =ChromaQpTable[2][qPi Chroma ] (8-938)

[0168] Cb and Cr components, Qp′ Cb and Qp′ Cr and joint Cb-Cr codec Qp′ CbCr The chromaticity quantization parameters are inferred as follows:

[0169] Qp′ Cb =Clip3(-QpBdOffset C ,63,qP Cb +pps_cb_qp_offset+slice_cb_qp_offset+CuQpOffset Cb )+QpBdOffset C (8-939)

[0170] Qp′ Cr =Clip3(-QpBdOffset C ,63,qP Cr +pps_cr_qp_offset+slice_cr_qp_offset+CuQpOffset Cr )+QpBdOffset C (8-940)

[0171] Qp′ CbCr =Clip3(-QpBdOffset C ,63,qP CbCr +pps_cbcr_qp_offset+slice_cbcr_qp_offset+CuQpOffset CbCr )+QpBdOffset C (8-941)

[0172] VVC Draft 6 supports two encoding and decoding modes for luma and chroma: single tree and dual tree. Under the single tree, the video encoder 200 and the video decoder 300 encode and decode the luma samples and chroma samples of each CTU in an interleaved manner. Under the dual tree, the video encoder 200 and the video decoder 300 encode and decode the luma samples and chroma samples of each CTU continuously. The definition of transform unit in VVC Draft 6 is as follows: Transform Unit (TU): A transform block of luma samples and two corresponding transform blocks of chroma samples of an image, and a syntactic structure for transforming the samples of the transform block.

[0173] In this definition, the phrase "corresponding chroma transform blocks" is used to indicate that the transform blocks (TBs) of the luma and chroma of a TU cover the same area (even if the chroma TBs have different sizes due to chroma subsampling).

[0174] However, in the dual-tree case, it is not easy to define a TU as one luma transform block and two corresponding chroma transform blocks. The decision to separate luma samples and chroma samples in a CTU is separate in the dual-tree. Therefore, one of the following scenarios may apply: one luma TB may correspond to two chroma TBs per component (aligned with the definition); one luma TB may correspond to one chroma TB or more than two chroma TBs per component; multiple luma TBs may correspond to one chroma block per component; or there may be no such correspondence.

[0175] In this case, a video decoder such as the video decoder 300 may not be able to determine what is included in the transform unit, which may cause potential problems for the video decoder in parsing the syntax tables and decoding the video data. Currently, in VVC draft 6, a transform block is associated with a transform unit only under the following conditions: a monochrome image (only one component); or separate_color_plane_flag is equal to 1.

[0176] The following examples illustrate problems that can arise due to (and sometimes in part due to) this ambiguity. Note that some of the examples below also apply to the single-tree case.

[0177] In one example, when the syntax elements are not present in the transform unit, the semantics of tu_cbf_cr[] and tu_cbf_cb[] (these two syntax elements may be referred to herein as chroma CBF values) are inferred to be equal to 0. If the TU contains both luma and chroma samples and the slice is coded using dual-tree mode, the luma tree is coded first, followed by the chroma tree. In this case, the video decoder 300 may not be able to determine whether to use the actual values ​​of the syntax elements tu_cbf_cb and tu_cbf_cr (which may not yet be available) or the inferred values ​​when parsing the luma tree. This may result in improper decoding of the video data. The values ​​of these syntax elements are used to parse tu_cbf_luma (note that there are also inference rules for tu_cbf_luma when tu_cbf_luma does not exist in the current TU). Currently, dual-tree only applies to intra-frame coded slices, in which case the chroma CBF values ​​are not used to parse the luma CBF values.

[0178] In another example, signaling / parsing delta QP values ​​for luma depends on tu_cbf_cb and tu_cbf_cr values. Similar to the previous paragraph, for dual-tree, the video decoder 300 may not be able to determine whether to use actual values ​​of tu_cbf_cb and tu_cbf_cr (when available) or inferred values.

[0179] In yet another example, in a single-tree coded slice, an intra-frame sub-partitioning (ISP) coded block may have N sub-partitions. The N luma TBs are coded first, followed by the chroma (e.g., ISP applies only to luma). Syntax elements associated with the chroma block are coded with the last luma TB. In other words, the chroma CBF value is signaled only for the last intra-frame sub-partition.

[0180] For non-ISP blocks, when the chroma CBF value is non-zero, the delta QP value for luma is signaled for the luma TB. For the first N-1 luma sub-partitions, the video decoder 300 may not be able to determine whether to use the actual or inferred value of the chroma CBF when decoding the delta QP value for luma. This is inconsistent with the design of non-ISP blocks.

[0181] This disclosure describes several techniques for improving chroma QP inference for hybrid transform-based video codecs, such as those implementing a VVC version (including signaling). It should be understood that one or more of these techniques can be used independently or in combination with other techniques.

[0182] Several potential definitions of transform units are as follows. According to the techniques of this disclosure, video encoder 200 and video decoder 300 may be configured to encode and decode video data, including TUs, according to one or more of these definitions.

[0183] In one example, the transform unit may be defined as follows: Transform Unit (TU): a transform block of luma samples, and in some cases two corresponding transform blocks of chroma samples, and a syntax structure for transforming the transform block samples.

[0184] In one example, the transform unit is defined as follows: Transform Unit (TU): One or more transform blocks and syntax structures for transforming the following transform blocks of samples: in a single tree, one transform block of luma samples and two corresponding transform blocks of chroma samples; and in a dual tree, one transform block of luma samples or two corresponding transform blocks of chroma samples.

[0185] In another alternative, the transform unit is defined as follows: Transform Unit (TU): one or more transform blocks and syntax structure for transforming the following transform blocks of samples: in a single tree, when ISP is not applied to the codec block containing the sample, one luma sample transform block and two corresponding chroma sample transform blocks; in a single tree, when ISP is applied to the codec block containing the sample, the luma sample transform block of the first N–1 luma sample transform blocks, and one luma sample transform block and two chroma sample transform blocks, where the chroma sample can correspond to more than one transform block in the codec block; and in a dual tree, one luma sample transform block or two corresponding chroma sample transform blocks.

[0186] In another alternative, the transform unit is defined as follows: Transform Unit (TU): a luma sample transform block, or when single tree encoding and decoding mode is used, a luma sample transform block and two corresponding chroma sample transform blocks of an image, or when dual tree encoding and decoding mode is used, a luma sample transform block or two chroma sample transform blocks, and a syntactic structure for transforming the transform block samples.

[0187] For example, the video encoder 200 or the video decoder 300 may determine whether to use a single tree to encode and decode a first CTU. Based on using a single tree to encode and decode the first CTU, the video encoder 200 or the video decoder 300 may determine a first TU based on a first set of TB samples, the first set of TB samples including a first luma sample TB and a first two corresponding chroma samples TB, and encode and decode a CU of the first CTU based on the first TU. The video encoder 200 or the video decoder 300 may determine whether to use a dual tree to encode and decode a second CTU. Based on using a dual tree to encode and decode the second CTU, the video encoder 200 or the video decoder 300 may determine a second TU based on a second set of TB samples, the second set of TB samples including (I) a second luma sample TB or (II) a second two chroma samples TB, and encode and decode a CU of the second CTU based on the second TU. In some examples, the first TU includes a syntax structure for transforming the first set of TB samples, and the second TU includes a syntax structure for transforming the second set of TB samples.

[0188] It may be beneficial to avoid the dependency of the delta QP signal on the chroma CBF value. For example, for the dual tree case, the video encoder 200 or the video decoder 300 may be configured to determine the delta QP value without reference to the chroma CBF value. In other words, the delta QP value may not depend on the chroma CBF value. In some cases, for both the dual tree and single tree cases, the dependency of the signaling or parsing delta QP on the chroma CBF value may be eliminated. In other examples, in the dual tree case, the video encoder 200 or the video decoder 300 may initialize the value of the chroma CBF syntax element to zero at the beginning of the luma tree. For example, based on a CTU encoded and decoded using a dual tree, the video encoder 200 or the video decoder 300 may initialize the chroma CBF value to zero at the beginning of the luma tree.

[0189] In the case of a single tree, the video encoder 200 can signal the tu_cbf value of chroma and the first luma sub-partition in the ISP CU, and signal the residual associated with the chroma and the first sub-partition. In one example, although the tu_cbf value of chroma is signaled in the first intra sub-partition, the residual can be signaled only in the last intra sub-partition. In some examples, the chroma component is disabled when the ISP is enabled, where the chroma reconstruction depends on the reconstructed luma value (e.g., cross-component linear model (CCLM) prediction, shaping, etc.).

[0190] In some examples, when tu_cbf_cr or tu_cbr_cb is not present in the current TU, the video decoder 300 may not be able to infer that tu_cbf_cr or tu_cbr_cb is equal to 0.

[0191] One or more of the techniques disclosed in this disclosure may also be constrained by one or more of the following characteristics of the sample to which the chroma map is applied: for example, block shape, aspect ratio, prediction mode used, characteristics of neighboring blocks, location of the sample relative to the image (near or far from a boundary, including image boundaries, strip boundaries, strip boundaries, brick boundaries, etc.).

[0192] Some of the techniques described above can be applied by a video encoder (e.g., video encoder 200) or a video decoder (e.g., video decoder 300), or both. Although many of the techniques disclosed are for chrominance components, these techniques can also be applied to components in luma and other color spaces that can be used to represent video. One or more of the techniques disclosed herein can be applied together or independently. Furthermore, although one or more of the techniques described above may be described with respect to a particular type of block, it should be understood that these techniques are applicable to different types of blocks—codec blocks, prediction blocks, transform blocks, or other types of blocks that specify some spatial partitioning of an image.

[0193] In some examples, the video encoder 200 and the video decoder 300 can be configured to determine the luma delta QP without reference to the chroma CBF value. <delete> and< / delete> The following deletions shown between may be made in the syntax and semantics of VVC draft 6. In this example, the dependency of the luma delta QP signal on the chroma CBF value is removed.

[0194] Table 7

[0195]

[0196]

[0197]

[0198] In one example, the video decoder 300 applies the parsing of tu_cbf_cb and tu_cbf_cr only for the case of a single tree. For example, the video decoder 300 may determine whether a CTU is encoded and decoded using a single tree, and parse the chroma CBF value based on the CTU encoded and decoded using a single tree. Table 8 shows <add> and< / add> The syntax may be as follows:

[0199]

[0200]

[0201]

[0202] Table 8

[0203] In another example, the video encoder 200 or the video decoder 300 initializes the values ​​of TuCbfCb and TuCbfCr to 0. The syntax and semantics of VVC draft 6 may have the following changes. <add> and< / add> between, delete the <delete> and< / delete> In addition, one or more occurrences of tu_cbf_cb and tu_cbf_cr in the syntax tables, semantics, and decoding process may be replaced by TuCbfCb and TuCbfCr. In some alternatives, the variables TuCbfCr and TuCbfCb may be initialized to 0 for all samples in the image.

[0204] 7.4.7.1 General Strip Header Semantics

[0205] When the slice syntax element is present, the value of each of the slice syntax elements slice_pic_parameter_set_id, non_reference_picture_flag, color_plane_id, slice_pic_order_cnt_lsb, recovery_poc_cnt, no_output_of_prior_pics_flag, pic_output_flag, and slice_temporal_mvp_enabled_flag shall be the same in all slices of a codec picture.

[0206] The variable CuQpDeltaVal, which specifies the difference between the luma quantization parameter and its predicted value for codecs containing cu_qp_delta_abs, is set equal to 0. The variables CuQpOffsetCb, CuQpOffsetCr, and CuQpOffsetCbCr are all set equal to 0, which specify the difference between the luma quantization parameter and its predicted value for codecs containing cu_chroma_qp_offset_flag. Cb ,Qp′ Cr and Qp′ CbCr The value to use when quantizing the respective values ​​of the parameters.

[0207] <add> The variables TuCbfCr and TuCbfCb, which specify the codec block flag values ​​for the Cb and Cr components, respectively, for the transform block, are set equal to 0.< / add>

[0208] slice_pic_parameter_set_id specifies the value of pps_pic_parameter_set_id of the PPS being used. The value of slice_pic_parameter_set_id should be in the range of 0 to 63 (inclusive). ...

[0210] 7.4.9.10 Transformation unit semantics

[0211] The transform coefficient levels are represented by the array TransCoeffLevel[x0][y0][cIdx][xC][yC]. Array indices x0, y0 specify the position (x0, y0) of the top-left luma sample of the transform block under consideration relative to the top-left luma sample of the picture. Array index cIdx specifies the indicator for the color component; Y is equal to 0, Cb is equal to 1, and Cr is equal to 2. Array indices xC and yC specify the transform coefficient position (xC, yC) within the current transform block. When the value of TransCoeffLevel[x0][y0][cIdx][xC][yC] is not specified in clause 7.3.8.11, it is inferred to be equal to 0.

[0212] tu_cbf_cb[x0][y0] equal to 1 specifies that the Cb transform block contains one or more transform coefficient levels not equal to 0. Array index x0, y0 specifies the position (x0, y0) of the top left corner of the transform block under consideration.

[0213] <delete> When tu_cbf_cb[x0][y0] does not exist in the current TU, its value is inferred to be equal to 0.< / delete>

[0214] <add> When tu_cbf_cb[x0][y0] exists, the variable TuCbfCb[x0][y0] is set equal to tu_cbf_cb[x0][y0].< / add>

[0215] tu_cbf_cr[x0][y0] equal to 1 specifies that the Cr transform block contains one or more transform coefficient levels not equal to 0. Array index x0, y0 specifies the position (x0, y0) of the top left corner of the transform block under consideration.

[0216] <delete> When tu_cbf_cr[x0][y0] does not exist in the current TU, its value is inferred to be equal to 0.< / delete>

[0217] <add> When tu_cbf_cr[x0][y0] exists, the variable TuCbfCr[x0][y0] is set equal to tu_cbf_cr[x0][y0].< / add>

[0218] tu_cbf_luma[x0][y0] equal to 1 specifies that the luma transform block contains one or more transform coefficient levels not equal to 0. Array indices x0, y0 specify the position (x0, y0) of the top-left luma sample of the transform block under consideration relative to the top-left luma sample of the image.

[0219] When tu_cbf_luma[x0][y0] does not exist in the current TU, its value is inferred as follows: ...

[0221] In another example, the video encoder 200 signals the values ​​of tu_cbf_cb and tu_cbf_cr for the ISP block in the single tree in the first sub-partition instead of the last sub-partition, and the video decoder 300 parses the values ​​of tu_cbf_cb and tu_cbf_cr for the ISP block in the single tree in the first sub-partition instead of the last sub-partition. The syntax and semantics of VVC Draft 6 may have the following changes.

[0222] Table 9 shows <add> and< / add> Table 9 shows <delete> and< / delete> Delete between.

[0223]

[0224]

[0225]

[0226]

[0227] Table 9

[0228] In some examples, the video encoder 200 may set the index of the syntax elements such as tu_joint_cbcr_residual_flag[][] to xC and yC instead of x0 and y0. This may be reflected in the syntax table above (Table 9) and the corresponding position in the standard specification.

[0229] In other examples, the variables TuCbfCb and TuCbfCr may be used, and the syntax elements tu_cbf_cb and tu_cbf_cr may still be signaled with the last intra sub-partition.

[0230] In another example, the tu_cbf_cb and tu_cbf_cr syntax elements may be signaled by the video encoder 200 along with the first intra sub-partition, but the corresponding residual may be signaled in the last intra sub-partition. The syntax structure may be similar to the above except that the residual codec structure is called as shown in Table 10 below. <add> and< / add> Delete the display in <delete> and< / delete> between.

[0231]

[0232] Table 10

[0233] In some examples, signaling of other syntax elements (chroma QP offset, joint Cb-Cr flag, etc.) may also be conditional on chromaResidualParseFlag.

[0234] In another example, some of the above examples can be combined as follows:

[0235]

[0236] Table 11

[0237] In another example, parsing of the chroma residual syntax element only applies to some cases where the chroma syntax element does not depend on luma signaling or the chroma CBF that the luma CBF depends on may have already been parsed. For example, when the video decoder 300 has already parsed the chroma syntax element that the luma CBF depends on, the video decoder 300 may only parse the chroma residual syntax element. For example, the video decoder 300 may determine whether the chroma CBF has been parsed, and parse the chroma residual syntax element based on whether the chroma CBF has been parsed and the value of the chroma CBF. In some examples, the luma CBF may depend on the value of the chroma CBF. In some examples, the video encoder 200 or the video decoder 300 may determine whether a TU of a dual-tree coded CTU is based on two chroma sample TBs, and based on the TU being based on two chroma sample TBs, suppress signaling or parsing the luma residual. Changes to the syntax structure and semantics are listed in Table 12 below. Additions shown in <add> and< / add> Delete the display in <delete> and< / delete> between.

[0238]

[0239]

[0240]

[0241]

[0242]

[0243] Table 12

[0244] The semantic changes in VVC draft 6 are as follows. <delete> and< / delete> between.

[0245] tu_cbf_cb[x0][y0] equal to 1 specifies that the Cb transform block contains one or more transform coefficient levels not equal to 0. Array index x0, y0 specifies the position (x0, y0) of the top left corner of the transform block under consideration.

[0246] When the current TU does not exist <delete>tu_cbf_cb[x0][y0], its value is inferred to be equal to 0. In one example, when tu_cbf_cb is not signaled in an instance of the syntax structure transform_unit(), the value may be inferred to be 0.

[0247] tu_cbf_cr[x0][y0] equal to 1 specifies that the Cr transform block contains one or more transform coefficient levels not equal to 0. Array index x0, y0 specifies the position (x0, y0) of the top left corner of the transform block under consideration.

[0248] When the current TU< / delete> does not exist <delete>tu_cbf_cr[x0][y0], its value is inferred to be equal to 0. In one example, when tu_cbf_cr is not signaled in an instance of the syntax structure transform_unit(), the value can be inferred to be 0.

[0249] tu_cbf_luma[x0][y0] equal to 1 specifies that the luma transform block contains one or more transform coefficient levels not equal to 0. Array indices x0, y0 specify the position (x0, y0) of the top-left luma sample of the transform block under consideration relative to the top-left luma sample of the image.

[0250] When the current TU <delete>When tu_cbf_luma[x0][y0] does not exist, its value is inferred as follows:

[0251] tu_cbf_luma[x0][y0] is inferred to be equal to 0 if cu_sbt_flag is equal to 1 and one of the following conditions is true: subTuIndex is equal to 0 and cu_sbt_pos_flag is equal to 1; or subTuIndex is equal to 1 and cu_sbt_pos_flag is equal to 0. Otherwise, tu_cbf_luma[x0][y0] is inferred to be equal to 1.

[0252] For example, the video decoder 300 can determine whether a CBF exists and, based on the absence of the CBF, infer that the CBF is zero. In some examples, the CBF can be a chroma CBF, such as tu_cbf_cb[x0][y0] or tu_cbf_cr[x0][y0]. In some examples, the CBF can be a luma CBF, such as tu_cbf_luma[x0][y0].

[0253] Figure 5 1 is a flow chart illustrating a technique for determining a transform unit (TU) according to the techniques of this disclosure. The video encoder 200 or the video decoder 300 may determine whether to use a single tree to encode and decode a first CTU (330). For example, the video encoder 200 may determine to encode and decode the first CTU in a single tree and signal a syntax element to the video decoder 300 indicating that the first CTU is encoded in a single tree. The video decoder 300 may parse the syntax element to determine that the first CTU is encoded using a single tree.

[0254] The video encoder 200 (e.g., transform processing unit 206) or the video decoder 300 (e.g., inverse transform processing unit 308) may determine a first TU (332) based on a first set of TB samples, the first set of TB samples including a first luma sample TB and first two corresponding chroma samples TB, based on encoding and decoding the first CTU using a single tree. For example, the video encoder 200 or the video decoder 300 may determine a first TU including a first luma sample TB and first two corresponding chroma samples TB.

[0255] The video encoder 200 or the video decoder 300 may encode or decode the CU of the first CTU based on the first TU (334). For example, the video encoder 200 may encode the CU of the first CTU based on the first TU, and the video decoder 300 may decode the CU of the first CTU based on the first TU.

[0256] The video encoder 200 or the video decoder 300 may determine whether to encode the second CTU using the dual tree (336). For example, the video encoder 200 may determine to encode the second CTU in the dual tree and signal a syntax element indicating that the second CTU is encoded in the dual tree to the video decoder 300. The video decoder 300 may parse the syntax element to determine that the second CTU is encoded using the dual tree.

[0257] The video encoder 200 (e.g., transform processing unit 206) or the video decoder 300 (e.g., inverse transform processing unit 308) may determine a second TU (338) based on the second CTU encoded and decoded using the dual-tree and based on a second set of TB samples, the second set of TB samples including (I) a second luma sample TB or (II) second two chroma samples TB. For example, the video encoder 200 or the video decoder 300 may determine the second TU including the second luma sample TB or the second two chroma samples TB.

[0258] The video encoder 200 or the video decoder 300 may encode or decode the CU of the second CTU based on the second TU (340). For example, the video encoder 200 may encode the CU of the second CTU based on the second TU, and the video decoder 300 may decode the CU of the second CTU based on the second TU.

[0259] In some examples, video encoder 200 or video decoder 300 can initialize the chroma CBF value to zero at the beginning of the luma tree based on encoding and decoding the second CTU using a dual tree. In some examples, video decoder 300 can determine whether a CBF exists and, based on the absence of a CBF, infer that the CBF is zero. In some examples, the CBF is a chroma CBF. In some examples, the CBF is a luma CBF.

[0260] In some examples, video decoder 300 may determine whether a chroma CBF has been parsed and parse the chroma residual syntax element based on whether the chroma CBF has been parsed and the value of the chroma CBF. In some examples, the luma CBF depends on the value of the chroma CBF.

[0261] In some examples, video encoder 200 or video decoder 300 may determine whether the second TU is based on a second two chroma sample TB, and may suppress signaling or resolve luma residual based on the second two chroma sample TB based on the second TU.

[0262] In some examples, the video encoder 200 or the video decoder 300 may determine whether to use a single tree to encode the third CTU. The video encoder 200 or the video decoder 300 may determine whether to use intra sub-partitioning to encode the third CTU. Based on encoding the third CTU using a single tree and intra sub-partitioning, the video encoder 200 or the video decoder 300 may determine a third TU based on a third group of TB samples, the third group of TB samples including a third luma sample (TB), and determine a fourth TU based on a fourth group of TB samples, the fourth group of TB samples including a fourth luma sample (TB) and four corresponding chroma samples (TB). The video encoder 200 or the video decoder 300 may encode or decode the CU of the third CTU based on the third TU and the fourth TU.

[0263] Figure 6 is a flowchart illustrating an example method for encoding a current block. The current block may include a current CU. Although the video encoder 200 ( Figure 1 and Figure 3 ) is described, but it should be understood that other devices may be configured to perform similar Figure 6 method.

[0264] In this example, the video encoder 200 initially predicts the current block (350). For example, the video encoder 200 may form a prediction block for the current block. The video encoder 200 may then calculate a residual block for the current block (352). To calculate the residual block, the video encoder 200 may calculate the difference between the original uncoded block and the prediction block for the current block. The video encoder 200 may then transform and quantize coefficients of the residual block (354). In some examples, the video encoder 200 may determine whether to encode a first CTU using a single tree, and based on encoding the first CTU using the single tree, determine a first TU based on a first set of TB samples, the first set of TB samples including a first luma sample TB and first two corresponding chroma samples TB. Additionally or alternatively, the video encoder 200 may determine whether to encode a second CTU using a dual tree, and based on encoding the second CTU using the dual tree, determine a second TU based on a second set of TB samples, the second set of TB samples including (I) a second luma sample TB or (II) a second two chroma samples TB.

[0265] Next, video encoder 200 may scan the quantized transform coefficients of the residual block (356). During or after the scan, video encoder 200 may entropy encode the transform coefficients (358). For example, video encoder 200 may encode the transform coefficients using CAVLC or CABAC. Video encoder 200 may then output the entropy-encoded data for the block (360).

[0266] Figure 7 is a flow chart illustrating an example method for a current block of decoded video data. The current block may include a current CU. Although the video decoder 300 ( Figure 1 and Figure 4 ) is described, but it should be understood that other devices may be configured to perform similar Figure 7 method.

[0267] The video decoder 300 may receive entropy coded data for the current block, such as entropy coded prediction information and entropy coded data corresponding to coefficients of a residual block for the current block (370). The video decoder 300 may entropy decode the entropy coded data to determine prediction information for the current block and reproduce coefficients of the residual block (372). The video decoder 300 may predict the current block (374), such as by calculating a prediction block for the current block using an intra or inter prediction mode indicated by the prediction information for the current block. The video decoder 300 may then inverse scan the reproduced coefficients (376) to create a block of quantized transform coefficients. The video decoder 300 may then inverse quantize and inverse transform the transform coefficients to produce a residual block (378). In some examples, the video decoder 300 may determine whether to encode a first CTU using a single tree, and based on encoding the first CTU using a single tree, determine a first TU based on a first set of TB samples, the first set of TB samples including a first luma sample TB and first two corresponding chroma samples TB. Additionally or alternatively, the video decoder 300 may determine whether to use a dual tree to encode the second CTU, and based on using the dual tree to encode the second CTU, determine the second TU based on a second set of TB samples, the second set of TB samples including (I) a second luma sample TB or (II) a second two chroma samples TB.

[0268] The video decoder 300 may finally decode the current block by combining the prediction block and the residual block ( 380 ).

[0269] This disclosure includes the following examples.

[0270] Example 1. A method for encoding and decoding video data, the method comprising: determining, by one or more processors, whether to use a single tree to encode and decode a first codec tree unit (CTU); determining, by the one or more processors, a first transform unit (TU) based on a first set of transform block (TB) samples based on encoding and decoding the first CTU using the single tree, the first set of transform block samples including a first luma sample TB and first two corresponding chroma samples TB; encoding and decoding a codec unit (CU) of the first CTU based on the first TU; determining, by the one or more processors, whether to use a dual tree to encode and decode a second CTU; determining, by the one or more processors, a second TU based on a second set of TB samples based on encoding and decoding the second CTU using the dual tree, the second set of TB samples including (I) a second luma sample TB or (II) a second two chroma sample TB; and encoding and decoding a CU of the second CTU based on the second TU, wherein the first TU includes a syntax structure for transforming the first set of TB samples, and the second TU includes a syntax structure for transforming the second set of TB samples.

[0271] Example 2. The method of Example 1 further includes: encoding and decoding the second CTU using the dual tree by one or more processors, initializing a chroma codec block flag (CBF) value to zero at the beginning of the luma tree.

[0272] Example 3. The method of any combination of Examples 1-2, further comprising: determining, by the one or more processors, whether a CBF exists; and inferring, by the one or more processors, that the CBF is zero based on the absence of the CBF.

[0273] Example 4. The method of Example 3, wherein the CBF comprises a chromatic CBF.

[0274] Example 5. The method of Example 3, wherein the CBF includes a luminance CBF.

[0275] Example 6. The method of any combination of Examples 1-5, further comprising: determining, by the one or more processors, whether the chroma CBF has been parsed; and parsing, by the one or more processors, a chroma residual syntax element based on whether the chroma CBF has been parsed and the value of the chroma CBF.

[0276] Example 7. The method of any combination of Examples 1-6, further comprising: determining, by the one or more processors, whether the second TU is based on a second two chroma sample TB; and suppressing, by the one or more processors, signaling or resolving a luma residual based on the second TU and the second two chroma sample TB.

[0277] Example 8. A method for encoding and decoding video data, the method comprising: determining whether to use a single tree or a dual tree to encode and decode a codec tree unit (CTU); if a single tree is used to encode and decode the CTU, determining a transform unit (TU) based on one luma sample transform block (TB) and two corresponding chroma sample TBs; if a dual tree is used to encode and decode the CTU, determining the TU based on one luma sample TB or two corresponding chroma sample TBs; and encoding and decoding the video data based on the TU.

[0278] Example 9. A method for encoding and decoding video data, the method comprising: determining whether to apply intra-frame sub-partitioning (ISP) to a codec block containing samples; determining whether to use a single tree or a dual tree to encode and decode the codec block; if the codec block is encoded and decoded using a single tree and the ISP is not applied, determining the TU based on one luma sample TB and two corresponding chroma sample TBs; if the codec block is encoded and decoded using a single tree and the ISP is applied, determining the TU based on one luma sample TB, one luma TB, and two chroma sample TBs of the first N-1 luma TBs, wherein the chroma sample may correspond to more than two TBs in the codec block; if the codec block is encoded and decoded using a dual tree, determining the TU based on one luma sample TB or two corresponding chroma sample TBs; and encoding and decoding the video data based on the TU.

[0279] Example 10. A method for encoding and decoding video data, the method comprising: determining whether to encode and decode a codec block using a single tree or a dual tree; if the codec block is encoded and decoded using a dual tree, signaling or parsing a delta quantization parameter (QP) without considering a chroma codec block flag (CBF) value; and encoding and decoding the video data based on the delta QP.

[0280] Example 11. The method of Example 10, further comprising: if a single tree is used to encode or decode the coding block, signaling or parsing the delta QP without considering the chroma CBF value.

[0281] Example 12. A method for encoding and decoding video data, the method comprising: determining whether to use a dual tree to encode and decode a coding block; if the dual tree is used to encode and decode the coding block, initializing the chroma CBF value to zero at the beginning of the luma tree; and encoding and decoding the video data based on the chroma CBF value.

[0282] Example 13. A method for encoding and decoding video data, the method comprising: determining whether an ISP is applied to a codec block; if an ISP is applied to the codec block, signaling a tu_cbf value of chroma in a first intra-frame sub-partition and signaling a residual associated with chroma in the first intra-frame sub-partition; and encoding and decoding the video data based on the tu_cbf value and the residual.

[0283] Example 14. The method of Example 13 further includes: modifying the index of tu_joint_cbcr_residual_flag[] to xC and yC instead of x0 and y0.

[0284] Example 15. A method for encoding and decoding video data, the method comprising:

[0285] Determining whether an ISP is applied to a codec block; if an ISP is applied to the codec block, signaling a tu_cbf value of chroma in a first intra-frame sub-partition and signaling a residual associated with chroma in a last intra-frame sub-partition; and encoding and decoding the video data based on the tu_cbf value and the residual.

[0286] Example 16. The method of Example 15, further comprising: if ISP is applied to the codec block, signaling the tu_cbf value of the luma in the first intra-frame sub-partition, and signaling the residual associated with the luma in the last intra-frame sub-partition.

[0287] Example 17. A method for encoding and decoding video data, the method comprising: determining whether ISP is enabled for a codec block; determining whether a chroma component depends on a reconstructed luminance value of the codec block; if ISP is enabled and the chroma component depends on the reconstructed luminance value, disabling the chroma component for the codec block; and encoding and decoding the video block based on the determination.

[0288] Example 18. A method for encoding and decoding video data, the method comprising: determining whether tu_cbr_cr and tu_cbr_cb exist in a current TU; if tu_cbr_cr does not exist in the current TU, not inferring tu_cbr_cr to be zero; if tu_cbr_cb does not exist in the current TU, not inferring tu_cbr_cb to be zero; and encoding and decoding the video data based on tu_cbr_cr and tu_cbr_cb.

[0289] Example 19. The method of Example 18, further comprising: initializing TuCfbCb and TuCbfCr to zero; setting TuCbfCr equal to tu_cbr_cr if tu_cbr_cr exists; and setting TuCfbCb equal to tu_cbr_cb if tu_cbr_cb exists.

[0290] Example 20. The method of Example 18 or 19, further comprising: signaling the sub-partitioned tu_cbr_cr and tu_cbr_cb in the last frame.

[0291] Example 21. A method for encoding and decoding video data, the method comprising: determining whether to use a single tree or a dual tree to encode and decode a codec block; parsing tu_cbr_cr and tu_cbr_cb only when the codec block is encoded and decoded using a single tree; and encoding and decoding the video data based on tu_cbr_cr and tu_cbr_cb.

[0292] Example 22. A device for encoding and decoding video data, the device comprising one or more components for performing the method of any one of Examples 1-21.

[0293] Example 23. The device of Example 22, wherein the one or more components include one or more processors implemented in circuitry.

[0294] Example 24. The device of Example 22 or 23 further includes a memory for storing video data.

[0295] Example 25. The device of any combination of Examples 22-24, further comprising a display configured to display the decoded video data.

[0296] Example 26. The device of any combination of Examples 22-25, wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

[0297] Example 27. The device of any combination of Examples 22-26, wherein the device comprises a video decoder.

[0298] Example 28. The device of any combination of Examples 22-27, wherein the device comprises a video encoder.

[0299] Example 29. A computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors to perform the method of any of Examples 1-21.

[0300] It should be appreciated that, depending on the examples, certain actions or events of any of the techniques described herein may be performed in a different order, may be added, combined, or omitted entirely (e.g., not all described actions or events are necessary for implementation of the techniques). Furthermore, in some examples, actions or events may be performed simultaneously, for example, through multithreading, interrupt handling, or multiple processors, rather than sequentially.

[0301] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted through a computer-readable medium as one or more instructions or codes and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which corresponds to tangible media such as data storage media, or communication media, including any media that facilitates the transfer of a computer program from one place to another, for example, according to a communication protocol. In this manner, computer-readable media may generally correspond to (1) non-transitory tangible computer-readable storage media, or (2) communication media such as signals or carrier waves. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures to implement the techniques described in this disclosure. A computer program product may include computer-readable media.

[0302] As an example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. In addition, any connection is properly referred to as a computer-readable medium. For example, if a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave) is used to transmit instructions from a website, server or other remote source, then the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology (such as infrared, radio and microwave) is included in the definition of medium. However, it should be understood that the computer-readable storage medium and data storage medium do not include connections, carrier waves, signals or other temporary media, but rather point to non-temporary tangible storage media. The disk and disc used here include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks and blue-ray discs, wherein the disks usually reproduce data magnetically, while the discs reproduce data optically with lasers. The above combinations should also be included in the scope of computer-readable media.

[0303] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, the terms "processor" and "processing circuitry" as used herein may refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. Furthermore, in some aspects, the functionality described herein may be provided in dedicated hardware and / or software modules configured for encoding and decoding, or incorporated in a combined codec. Similarly, these techniques may be implemented entirely in one or more circuits or logic elements.

[0304] The techniques of this disclosure can be implemented in a variety of devices or apparatuses, including wireless handsets, integrated circuits (ICs), or a set of ICs (e.g., a chipset). Various components, modules, or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but they do not necessarily need to be implemented by different hardware units. Instead, as described above, the various units can be combined in a codec hardware unit, or provided by a collection of interoperable hardware units including one or more processors as described above in combination with appropriate software and / or firmware.

[0305] Various examples have been described. These and other examples are within the scope of the following claims.< / delete> < / delete>

Claims

1. A method for encoding and decoding video data, the method comprising: Determining, by one or more processors, to use single-tree partitioning to encode and decode a first codec tree unit (CTU); Determining, by the one or more processors, a first transform unit TU based on a first set of transform block samples (TB) based on the encoding and decoding of the first CTU using single tree partitioning, where the first set of transform block samples (TB) includes a first luma sample (TB) and first two corresponding chroma samples (TB); encoding and decoding, by the one or more processors, a coding unit CU of the first CTU based on the first TU; Determining, by the one or more processors, to use dual-tree luma partitioning to encode and decode a second CTU; Determining, by the one or more processors, a second TU based on a second set of TB samples based on encoding and decoding a second CTU using dual-tree luma partitioning, the second set of TB samples including second luma samples TB; Initializing, by the one or more processors, a chroma codec block flag (CBF) value to zero at the beginning of a luma tree based on encoding and decoding the second CTU using dual-tree luma partitioning; and The one or more processors encode and decode the CU of the second CTU based on the second TU, The first TU includes a syntax structure for transforming the first set of TB samples, and the second TU includes a syntax structure for transforming the second set of TB samples.

2. The method according to claim 1, further comprising: determining, by the one or more processors, that a first CBF is not present; and The one or more processors infer, based on the absence of the first CBF, that the CBF is zero. The method of claim 2 , wherein the first CBF comprises a chrominance CBF. The method of claim 2 , wherein the first CBF comprises a luminance CBF.

5. The method according to claim 1, further comprising: determining, by the one or more processors, that a chroma CBF for a third CTU has been resolved; and A chroma residual syntax element is parsed, by the one or more processors, based on the chroma CBF of the third CTU having been parsed and a value of the chroma CBF.

6. The method according to claim 1, further comprising: determining, by the one or more processors, that the second set of TB samples includes a second two chroma samples TB; and The one or more processors suppress encoding and decoding of the luma residual based on the second set of TB samples including the second two chroma samples TB. The method of claim 1 , wherein encoding and decoding comprises encoding. The method of claim 1 , wherein encoding and decoding comprises decoding.

9. The method according to claim 1, further comprising: Determining, by the one or more processors, to use single-tree partitioning to encode and decode a third CTU; Determining, by the one or more processors, to use intra-frame sub-partitioning to encode and decode the third CTU; Determining, by the one or more processors, a third TU based on a third group of TB samples including a third luma sample TB based on encoding and decoding a third CTU using the single-tree partitioning and the intra-frame sub-partitioning, and determining a fourth TU based on a fourth group of TB samples including a fourth luma sample TB and four corresponding two chroma samples TB; The one or more processors encode and decode the CU of the third CTU based on the third TU and the fourth TU.

10. The method according to claim 1, further comprising: A delta quantization parameter is determined, by the one or more processors, based at least in part on the chroma CBF value.

11. The method according to claim 1 , further comprising: Determining, by the one or more processors, to encode a third CTU using dual-tree chroma partitioning; as well as A third TU based on the third set of TB samples is determined by the one or more processors to include corresponding second two chroma samples TB.

12. A device for encoding and decoding video data, the device comprising: a memory configured to store the video data; and one or more processors executing in circuitry, communicatively coupled to the memory, the one or more processors configured to: Determine to use single tree partitioning to encode and decode a first codec tree unit CTU; Determining a first transform unit TU based on the encoding and decoding of the first CTU using single tree partitioning, based on a first set of transform block samples TB, where the first set of transform block samples TB includes a first luma sample TB and first two corresponding chroma samples TB; encode and decode the coding unit CU of the first CTU based on the first TU; Determine to use dual-tree luminance partitioning to encode and decode the second CTU; Determining a second TU based on a second group of TB samples based on encoding and decoding the second CTU using dual-tree luma partitioning, wherein the second group of TB samples includes second luma samples TB; Initializing a chroma codec block flag (CBF) value to zero at the beginning of the luma tree based on encoding and decoding the second CTU using the dual-tree luma partitioning; and Encode and decode the CU of the second CTU based on the second TU, The first TU includes a syntax structure for transforming the first set of TB samples, and the second TU includes a syntax structure for transforming the second set of TB samples.

13. The apparatus of claim 12, wherein the one or more processors are further configured to: Determining that the first codec block flag CBF does not exist; and Based on the absence of the first CBF, it is inferred that the CBF is zero. The apparatus of claim 13 , wherein the first CBF comprises a chrominance CBF. The apparatus of claim 13 , wherein the first CBF comprises a luminance CBF.

16. The apparatus of claim 12, wherein the one or more processors are further configured to: Determine whether the chroma codec block flag CBF has been parsed; and Based on the chroma CBF having been parsed and the value of the chroma CBF, a chroma residual syntax element is parsed.

17. The apparatus of claim 12, wherein the one or more processors are further configured to: determining whether the second TB samples include second two chroma samples TB; and Based on the second TB samples including the second two chroma samples TB, encoding and decoding of the luma residual is suppressed.

18. The apparatus of claim 12, wherein the apparatus comprises a video encoder.

19. The apparatus of claim 12, wherein the apparatus comprises a video decoder.

20. The device of claim 12, wherein the one or more processors are further configured to: Determine to use single tree partitioning to encode and decode the third CTU; Determining to use intra-frame sub-division coding and decoding of the third CTU; Determining, based on the encoding and decoding of the third CTU using the single-tree partitioning and the intra-frame sub-partitioning, a third TU based on a third group of TB samples including a third luma sample TB, and determining a fourth TU based on a fourth group of TB samples including a fourth luma sample TB and four corresponding two chroma samples TB; and The CU of the third CTU is encoded and decoded based on the third TU and the fourth TU.

21. A non-transitory computer-readable storage medium having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to: Determine to use single tree partitioning to encode and decode a first codec tree unit CTU; Determine, based on the encoding and decoding of the first CTU using single tree partitioning, a first transform unit TU based on a first set of transform block TB samples, where the first set of transform block TB samples includes a first luma sample TB and first two corresponding chroma samples TB; encode and decode the coding unit CU of the first CTU based on the first TU; Determine to use dual-tree luminance partitioning to encode and decode the second CTU; Determining a second TU based on a second group of TB samples based on encoding and decoding the second CTU using dual-tree luma partitioning, wherein the second group of TB samples includes second luma samples TB; Initializing a chroma codec block flag (CBF) value to zero at the beginning of the luma tree based on encoding and decoding the second CTU using the dual-tree luma partitioning; and Encode and decode the CU of the second CTU based on the second TU, The first TU includes a syntax structure for transforming the first set of TB samples, and the second TU includes a syntax structure for transforming the second set of TB samples.

22. The non-transitory computer-readable storage medium of claim 21 , wherein the instructions further cause the one or more processors to: Determine that the first CBF does not exist; Based on the absence of the first CBF, it is inferred that the CBF is zero.

23. The non-transitory computer-readable storage medium of claim 22, wherein the first CBF comprises a chrominance CBF.

24. The non-transitory computer-readable storage medium of claim 22, wherein the first CBF comprises a luminance CBF.

25. The non-transitory computer-readable storage medium of claim 21 , wherein the instructions further cause the one or more processors to: Determining that the chroma CBF of the third CTU has been resolved; and A chroma residual syntax element is parsed based on that the chroma CBF of the third CTU has been parsed and the value of the chroma CBF.

26. The non-transitory computer-readable storage medium of claim 21 , wherein the instructions further cause the one or more processors to: determining that the second group of TB samples includes second two chroma samples TB; and Based on the second group of TB samples including the second two chroma samples TB, encoding and decoding of the luma residual is suppressed.

27. The non-transitory computer-readable storage medium of claim 21, wherein encoding and decoding comprises encoding.

28. The non-transitory computer-readable storage medium of claim 21, wherein encoding and decoding comprises decoding.

29. The non-transitory computer-readable storage medium of claim 21 , wherein the instructions further cause the one or more processors to: Determine to use single tree partitioning to encode and decode the third CTU; Determining to use intra-frame sub-division coding and decoding for the third CTU; Determining, based on the encoding and decoding of the third CTU using the single-tree partitioning and the intra-frame sub-partitioning, a third TU based on a third group of TB samples including a third luma sample TB, and determining a fourth TU based on a fourth group of TB samples including a fourth luma sample TB and four corresponding two chroma samples TB; and The CU of the third CTU is encoded and decoded based on the third TU and the fourth TU.

30. A device for encoding and decoding video data, the device comprising: A component for determining to use single tree partitioning to encode and decode a first codec tree unit CTU; means for determining a first transform unit (TU) based on the encoding and decoding of the first CTU using single-tree partitioning and based on a first set of transform block (TB) samples, the first set of transform block (TB) samples comprising a first luma sample (TB) and first two corresponding chroma samples (TB); a component configured to encode and decode a coding unit CU of the first CTU based on the first TU; A component for determining to use dual-tree luma partitioning to encode and decode a second CTU; means for determining a second TU based on a second set of TB samples based on encoding and decoding a second CTU using dual-tree luma partitioning, wherein the second set of TB samples includes second luma samples TB; means for initializing a chroma codec block flag (CBF) value to zero at the beginning of a luma tree based on encoding and decoding the second CTU using dual-tree luma partitioning; and a component for encoding and decoding the CU of the second CTU based on the second TU, The first TU includes a syntax structure for transforming the first set of TB samples, and the second TU includes a syntax structure for transforming the second set of TB samples.

31. A computer program product comprising computer readable instructions which, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 11.

Images