Block segmentation of sent images and videos

Abstract

A video system for applying constraints to block partitioning is provided. The system receives a partitioning control parameter from a bitstream that specifies that a maximum block size for enabling ternary tree partitioning is constrained to be 64 or less (1410). The system receives data from the bitstream for a pixel block of a current block of a current picture of a video to be decoded (1420). The system recursively partitions the current block into one or more partitions (1430), wherein ternary tree partitioning is not permitted for a portion of the current block unless the partition is less than or equal to the maximum block size (1435). The system reconstructs the one or more partitions of the current block (1440).

Description

[0001] Cross-references

[0002] This application claims priority to U.S. Provisional Patent Applications Serial Nos. 62 / 914,577, 62 / 915,049, 62 / 916,891, 62 / 930,084, and 62 / 956,359, filed on October 14, 2019, October 15, 2019, October 18, 2019, November 4, 2019, and January 2, 2020, respectively, which are hereby incorporated by reference in their entirety. Technical Field

[0003] The present invention generally relates to video and image coding and decoding. More particularly, the present invention relates to a method for signaling a partitioning structure of blocks (eg, coding units) in a coded video or image. Background Art

[0004] Unless otherwise indicated herein, the approaches described in this section are not prior art to the claims listed below and are not admitted to be prior art by inclusion in this section.

[0005] In High Efficiency Video Codec (HEVC), a coded picture is partitioned into non-overlapping square block regions represented by codec tree units (CTUs). A coded picture can be represented by a number of slices, each of which includes an integer number of CTUs. The individual CTUs in a slice are processed in raster scanning order. Bidirectionally predicted (B) slices can be decoded using intra prediction or inter prediction using up to two motion vectors and reference indices to predict the sample values ​​of each block. Intra (I) slices are decoded using only intra prediction. Predicted (P) slices are decoded using intra prediction or inter prediction using up to one motion vector and reference indices to predict the sample values ​​of each block.

[0006] A CTU can be further split into multiple non-overlapping coding units (CUs) to accommodate various local motion and texture characteristics. A coding tree built on a quadtree (QT) splitting structure is used to represent multiple CU partitions split from a CTU. Regarding a CTU of size M×M pixel samples, where M is a value of 64, 32, and 16, a CTU can consist of a single unit of the same size M×M, or split into four smaller units of size M / 2×M / 2. The CTU and the four M / 2×M / 2 units associated with the root node of the coding tree are child quadtree nodes split from the root node of the coding tree. Each of the four quadtree nodes can become a parent node for another split into four child nodes with the size reduced by half in each spatial dimension. The quadtree splitting can be applied recursively until a leaf size is reached. The leaf nodes of the coding tree are associated with CUs. The minimum and maximum CU sizes can be specified in the sequence parameter set (SPS).

[0007] One or more prediction units (PUs) can be specified for each CU. A prediction unit, together with the associated CU syntax, serves as the basic unit for signaling prediction sub-information. The specified prediction process is applied to predict the values ​​of the associated pixel samples within the PU. A CU can be split into one, two, or four PUs, depending on the selected PU type.

[0008] The CU can be further partitioned using a residual quadtree (RQT) structure to represent the associated prediction residual signal. The leaf nodes of the RQT correspond to the generated transform units (TUs). A transform unit consists of a transform block (TB) of luminance samples of size 8×8, 16×16, or 32×32, or four transform blocks of luminance samples of size 4×4, and two corresponding transform blocks of chrominance samples of an image in a 4:2:0 color format. An integer transform is applied to the transform block and the quantized coefficient values ​​are encoded and decoded in the bitstream. The minimum and maximum transform block sizes are specified in the sequence parameter set.

[0009] In HEVC, the terms coding tree block, coding block, prediction block, and transform block are defined to refer to a 2-D sample array of one color component from the associated CTU, CU, PU, ​​and TU, respectively. Thus, a CTU contains one luma CTB, two chroma CTBs, and associated syntax elements in color pictures that are not coded using three separate color planes. The signaled coding tree partitioning is generally applied to both luma and chroma blocks, but exceptions can be made when certain minimum size constraints are met. Summary of the Invention

[0010] The following summary is illustrative only and is not intended to be limiting in any way. That is, the following summary is intended to introduce the concepts, highlights, benefits, and advantages of the novel and non-obvious technologies described herein. Selected and not all embodiments are further described in the detailed description. Therefore, the following summary is not intended to identify essential features of the claimed subject matter, nor is it intended to determine the scope of the claimed subject matter.

[0011] The present invention provides some embodiments of a video system that applies constraints to block partitioning. The system receives a partitioning control parameter from a bitstream, the partitioning control parameter specifying that a maximum block size for enabling ternary tree (TT) partitioning is constrained to be 64 or less. The system receives data from a bitstream to be decoded as a pixel block of a current block of a current picture of a video. The system recursively partitions the current block into one or more partitions, wherein TT partitioning is not allowed for a partition of the current block unless the partition is less than or equal to the maximum block size. The system reconstructs the one or more partitions of the current block.

[0012] In some embodiments, the partition control parameter is signaled in a strip header of a strip that includes the current block. The partition control parameter is signaled in a picture header of the current picture. The partition control parameter may be signaled in a sequence parameter set (SPS) of a video picture sequence that includes the current picture. In some embodiments, the SPS or picture header sets a default value for the partition control parameter, which may be overwritten by a slice header that includes the current block.

[0013] In some embodiments, the partitioning control parameter specifies the maximum block size by reference to a minimum size of a quadtree split leaf block, for example, by using the difference between (i) the base-2 logarithm of the maximum block size of a block that can be further partitioned by ternary tree splitting and (ii) the base-2 logarithm of the minimum size of the quadtree split leaf block. The minimum size of the quadtree split leaf block is constrained to be a smaller value between 64 and a coding tree block size. The maximum block size for enabling TT splitting is further constrained to be less than a coding tree block size.

[0014] The minimum size of the quadtree split leaf block is specified by reference to a minimum coding block size, and the minimum size of the quadtree split leaf block is specified by using the difference between (i) the base 2 logarithm of the minimum size of the quadtree split leaf block and (ii) the base 2 logarithm of the minimum coding block size.

[0015] In some embodiments, the maximum block size is determined based on the size of a virtual pipeline data unit (VPDU), and the maximum block size is defined to ensure that each VPDU can be completely processed by the same pipeline stage. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are included to provide a further understanding of the present invention and are incorporated into and constitute a part of this invention. The accompanying drawings illustrate embodiments of the present invention and, together with the description, serve to explain the principles of the present invention. It will be understood that the present invention is not necessarily drawn to scale because some parts are shown out of proportion to actual size to clearly illustrate the concepts of the present invention.

[0017] Figure 1 A coding tree structure is shown for representing the partitioning of a CTU into multiple CUs. Solid lines indicate CU boundaries and dashed lines indicate TU boundaries.

[0018] Figure 2 8 partition types defined in HEVC for splitting a CU into multiple PUs are shown.

[0019] Figure 3 Various split types for splitting a CU into two partitions are shown, including different symmetric and asymmetric split types that can split the CU vertically or horizontally.

[0020] Figure 4 An example of a block partition and its corresponding binary tree is shown.

[0021] Figure 5 An example of block partitioning and its corresponding QTBT is shown.

[0022] Figure 6 The splitting of a CU into multiple smaller CUs using one of five split types is shown.

[0023] Figure 7 Shown are consistent and inconsistent partitioning structures of pixel blocks when VPDU-based constraints are applied.

[0024] Figure 8 The constraint parameters used to control the partitioning structure of a pixel block are conceptually shown.

[0025] Figure 9 An exemplary video encoder that signals constraints on block partitioning is shown.

[0026] Figure 10 Part of a video encoder that enforces block partitioning constraints is shown.

[0027] Figure 11 The process for signaling block split constraints is conceptually shown.

[0028] Figure 12 An exemplary video decoder that applies constraints on block partitioning is shown.

[0029] Figure 13 Part of a video decoder that enforces block partitioning constraints is shown.

[0030] Figure 14 The process of applying block partitioning constraints is conceptually illustrated.

[0031] Figure 15 An electronic system is conceptually illustrated that implements some embodiments of the present invention. DETAILED DESCRIPTION

[0032] In the detailed description that follows, numerous specific details are provided by way of example to provide a thorough understanding of the present invention. Any variations, derivatives, and / or extensions of the teachings described herein are within the scope of the present invention. In some cases, well-known methods, processes, components, and / or circuits of one or more exemplary embodiments disclosed herein are described at a relatively high level without further detail in order to unnecessarily obscure aspects of the present invention.

[0033] I. Block Partition Structure

[0034] Figure 1 A coding tree structure is shown for representing the partitioning of a CTU into multiple CUs. Solid lines indicate CU boundaries and dashed lines indicate TU boundaries. Figure 2 8 types of partitioning defined in HEVC for splitting a CU into multiple PUs are shown.

[0035] A. Binary Tree (BT) Block Partitioning

[0036] In some embodiments, a binary tree block partitioning structure is used to partition the block. In the binary tree partitioning structure, a block can be recursively split into two smaller blocks or partitions.

[0037] Figure 3 Various split types for splitting a CU into two partitions are shown, including different symmetric and asymmetric split types that can split the CU vertically or horizontally. For some embodiments, the most efficient and simplest split types are symmetric horizontal and vertical splits, and only these two split types are used in the binary tree split structure.

[0038] For a given block of size M×N, a syntax element bt_split_flag may be signaled to indicate whether the block is to be split into two smaller blocks. If the flag is true, another syntax element bt_split_mode is signaled to indicate which split type is used (i.e., horizontal or vertical). If horizontal splitting is used, the block is split into two smaller blocks of size M×N / 2. Otherwise, if vertical splitting is used, the block is split into two smaller blocks of size M / 2×N. The binary tree splitting process may be recursively applied until the current block size (width or height) reaches a minimum block size (width or height), which may be defined in a higher-level syntax set (e.g., SPS). Because binary trees have two split types (i.e., horizontal and vertical), a minimum block width and height may be indicated. When a split would result in a block height less than the indicated minimum, a non-horizontal split is implied. When a split would result in a width less than the indicated minimum, a non-vertical split is implied.

[0039] Figure 4 An example of a block partition (left) and its corresponding binary tree (right) is shown. For each split (i.e., non-leaf) node of the binary tree, a flag is signaled to indicate which split type (horizontal or vertical) was used, with "0" indicating horizontal splitting and "1" indicating vertical splitting.

[0040] B. Quadtree + Binary Tree (QTBT) Block Partitioning

[0041] In some embodiments, a quadtree plus binary tree (QTBT) structure is adopted to split a CU into smaller CUs. Each CTU can contain one CU or be first split into multiple smaller CUs using a quadtree splitting structure. Quadtree splitting can be recursively applied to the CU until the size of the current CU reaches the minimum quadtree leaf node size. If the current quadtree block is not larger than the maximum binary tree root node size, it can be further split using the binary tree splitting structure. Binary tree splitting can be recursively applied until the current CU size (width or height) reaches the minimum binary tree leaf node size (width or height) or the current binary tree depth reaches the maximum binary tree depth. There are two split types, symmetric horizontal splitting and symmetric vertical splitting. Binary tree leaf nodes are associated with CUs. For each split (i.e., non-leaf) node of the binary tree, a flag is signaled to indicate which split type (horizontal or vertical) is used. "0" indicates horizontal splitting and "1" indicates vertical splitting. The minimum quadtree leaf node size, the maximum binary tree root node size, the minimum binary tree leaf node width and height, and the maximum binary tree depth may be indicated in a high-level syntax set. Figure 5 Examples of block partitioning (left) and its corresponding QTBT (right) are shown. Solid lines indicate quadtree splitting and dashed lines indicate binarytree splitting.

[0042] C. Multi-Type Tree (MTT) Block Partitioning

[0043] In the multi-type tree (MTT) structure, horizontal and vertical center-side ternary trees (TTs) are further added to the QTBT structure for splitting CUs. Figure 6 The splitting of a CU into multiple smaller CUs using one of five split types is shown. Split types (d) and (e) are called ternary or ternary tree split types and can be used to split a block into three smaller blocks, all with reduced size in one spatial dimension while maintaining the same size in the other spatial dimensions. The binary tree depth btDepthC of the current CU is defined in the MTT structure as follows:

[0044] btDepthC=log2((widthR*heightR) / (widthC*heightC))

[0045] Wherein, widthC and heightC represent the width and height of the current CU, respectively, widthR and heightR represent the width and height of the corresponding MTT root node, respectively, and the function log2(x) is the logarithm of x with a base of 2.

[0046] In some embodiments, each coding tree unit (CTU) can be split into one or more smaller CUs by a quadtree with an embedded multi-type tree (using binary and ternary splitting). The syntax element split_cu_flag is first signaled to indicate whether the current coding unit is further split into more than one coding unit. When split_cu_flag is equal to 0, the current CU is a leaf node and is not further split. Otherwise, the current CU is further split, and the syntax element split_qt_flag is further sent to indicate whether the current coding unit is split by quadtree splitting. When split_qta_flag is equal to 1, the current CU is further split into four CUs with half the horizontal and vertical size by quadtree splitting. Otherwise, the syntax elements mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag are signaled to indicate the direction (vertical or horizontal) and type (binary or ternary tree) of the selected MTT split.

[0047] II. Constraints on Block Size for Segmentation

[0048] A virtual pipeline data unit (VPDU) is defined as a non-overlapping M×M luma (L) / N×N chroma (C) unit in a picture. In a hardware decoder, consecutive VPDUs are processed simultaneously by multiple pipeline stages, and different stages process different VPDUs simultaneously. The size of a VPDU is roughly proportional to the buffer size of most pipeline stages. In an HEVC hardware decoder, the VPDU size is typically set to match the maximum transform block (TB) size, which is called the maximum allowed block size, or MaxTbSizeY.

[0049] In some embodiments, to reduce the VPDU size in the encoded video, CU partitioning is performed according to the following constraints: (1) For each VPDU that contains one or more CUs, the CU is completely contained within the VPDU. (2) For each CU that contains one or more VPDUs, the VPDU is completely contained within the CU. (3) The order in which CUs are processed cannot leave a VPDU untouched and accessed later. This ensures that each VPDU can be fully processed by the same pipeline stage.

[0050] Figure 7 The diagram shows consistent and non-compliant partitioning structures of a pixel block when VPDU-based constraints are applied. The diagram shows a block 700 (e.g., a CTU) that can be further partitioned for encoding and decoding. The size of the block 700 is 128×128 luma samples. The block 700 is processed into four VPDUs (701-704) (separated by dashed lines), each corresponding to a 64×64 luma sample region.

[0051] The diagram illustrates various exemplary schemes (a) through (i) for partitioning block 700 into multiple CUs. Some partitioning schemes are compatible with VPDUs, while others are not. Specifically, because each VPDU contains one or more CUs, the CU is completely contained within the VPDU, and for each CU containing one or more VPDUs, the VPDU is completely contained within the CU, partitioning schemes (a), (b), (c), and (d) result in CU-to-VPDU compatibility. This ensures that a VPDU does not need to be partially processed as part of one CU and then re-accessed later as part of another CU.

[0052] Splitting schemes (e), (f), (g), (h), (i), (j), (k), and (i) generate multiple CUs compatible with the VPDU because each scheme requires at least one VPDU to be partially processed as part of one CU and later revisited as part of another CU. In other words, there is at least one VPDU that spans multiple pipeline stages.

[0053] In some embodiments, when the TT split allowed for the current block is determined based on VPDU constraints, such as the maximum allowed block size due to pipeline limitations, the decision on whether TT split is allowed may be based on the following parameters:

[0054] -Ternary Split Mode ttSplit

[0055] - Coding block width cbWidth in luma samples

[0056] - Coding block height cbHeigth in luma samples

[0057] - The position (x0, y0) of the top left luma sample of the considered coding block relative to the top left luma sample of the image

[0058] -Multi-type tree depth mttDepth

[0059] - Maximum multi-type tree depth maxMttDepth with offset

[0060] -Maximum ternary tree size maxTtSize

[0061] - The variable treeType specifies whether a single tree (SINGLE_TREE) or a dual tree is used to partition the coding tree nodes, and when a dual tree is used, whether the luma (DUAL_TREE_LUMA) or chroma components (DUAL_TREE_CHROMA) are currently being processed.

[0062] - The variable modeType specifies whether intra (MODE_INTRA), IBC (MODE_IBC) and inter coding modes can be used (MODE_TYPE_ALL), or whether only intra and IBC coding modes can be used (MODE_TYPE_INTRA), or whether only inter coding mode (MODE_TYPE_INTER) can be used for coding units within the coding tree node.

[0063] These parameters are used to determine the variable "allowTtSplit". The variable cbSize is derived based on ttSplit as follows:

[0064]

[0065]

[0066] The variable allowTtSplit is derived from: (VPDU size indicated by MaxTbSizeY)

[0067] - allowTtSplit is set to FALSE if one or more subsequent conditions are true:

[0068] -cbSize is less than or equal to 2*MinTtSizeY

[0069] -cbWidth is greater than Min(MaxTbSizY, maxTtSize)

[0070] -cbHeight is greater than Min(MaxTbSizY, maxTtSize)

[0071] -mttDepth is greater than or equal to maxMttDepth

[0072] -x0+cbWidth is greater than pic_width_in_luma_samples

[0073] -y0+cbHeight is greater than pic_height_in_luma_samples

[0074] -treeType is equal to DUAL_TREE_CHROMA and (cbWidth / SubWidthC)*(cbHeight / SubHeightC) is less than or equal to 32

[0075] -treeType equals DUAL_TREE_CHROMA and (cbWidth / SubWidthC) equals 8 and TTSplit equals SPLIT_TT_VER

[0076] -treeType equals DUAL_TREE_CHROMA and modeType equals MODE_TYPE_INTRA

[0077] -cbWidth*cbHeight equals 64 and modeType equals MODE_TYPE_INTER

[0078] Otherwise, allow TTSplit to be set equal to TRUE.

[0079] In some embodiments, high-level syntax parameters related to the CU partitioning structure are specified based on pipeline processing constraints, i.e., implementing VPDU-related constraints (e.g., maximum allowed block size) when partitioning a CU. In some embodiments, the video codec system may include one or more syntax elements for deriving the value of the maximum allowed block size for enabling TT splitting in related data structures.

[0080] In some embodiments, the value of the maximum allowed block size for enabling TT splitting is constrained for bitstream conformance requirements. Therefore, to determine whether TT splitting is allowed for the current CU, the video codec system may only test whether the width or height of the current CU is greater than the maximum allowed block size for enabling TT splitting, and not further test the VPDU size for each current CU. For example, when the VPDU size is equal to 64, the video codec system may constrain the maximum allowed block size for enabling TT splitting signaled in the bitstream to be less than or equal to 64.

[0081] In some embodiments, the parameter maxTtSize is constrained to be less than or equal to 64, where the variable maxTtSize is derived from one or more coded syntax element values ​​indicating the maximum allowed block size for enabling TT splitting. In some embodiments, the variable maxTtSize is derived from the coded higher-level syntax element log2_diff_max_tt_min_t according to:

[0082] maxTtSize=1<<(log2_diff_max_tt_min_tt+log2MinMaxTT)

[0083] Under the constraint that the maximum allowed value of log2_diff_tt_min_tt is equal to Min(6, CtbLog2SizeY)-log2MinMaxTT. The variable log2MinMaxTT indicates the base-2 logarithm of the allowed minimum value of the maximum TT size and CtbLog2SizeY indicates the base-2 logarithm of the specified CTB size.

[0084] The process of deciding whether TT split is allowed for the current CU can be further simplified. Therefore, the variable allowTtSplict is derived as follows:

[0085] - allowTtSplit is set equal to FALSE if one or more of the following conditions are true:

[0086] -cbsize is less than or equal to 2*MinTtSizeY

[0087] -cbheight is greater than maxTtSize

[0088] -cbheight is greater than maxTtSize

[0089] -mttDepth is greater than or equal to maxMttDepth

[0090] -x0+cbWidth is greater than pic_width_in_luma_samples

[0091] -y0+cbHeight is greater than pic_height_in_luma_samples

[0092] -treeType is equal to DUAL_TREE_CHROMA and (cbWidth / SubWidthC)*(cbHeight / SubHeightC) is less than or equal to 32

[0093] -treeType equals DUAL_TREE_CHROMA and (cbwidth / subwidthC) equals 8 and ttSplit equals SPLIT_TT_VER

[0094] -treeType equals DUAL_TREE_CHROMA and modeType equals MODE_TYPE_INTRA

[0095] -cbWidth*cbHeight equals 64 and modetype equals MODE_TYPE_INTER

[0096] Otherwise, allowTtSplit is set equal to TRUE.

[0097] To satisfy the VDPU constraint, in some embodiments, variables minQtSizeY and minQtSizeC are derived from one or more coded syntax element values ​​of QT leaf nodes indicating the minimum allowed block size for luma and chroma, respectively, and when intra dual-tree mode is enabled, the values ​​of variables minQtSizeY and minQtSizeC are constrained in intra slices to be less than or equal to 64. In some embodiments, when intra dual-tree mode is enabled, the values ​​of variables minQtSizeY and minQtSizeC are constrained in intra slices to be less than or equal to Min(64, CtbSizeY), where variables minQtSizeY and minQtSizeC are derived from one or more coded syntax element values ​​of QT leaf nodes indicating the minimum allowed block size of luma samples for luma and chroma, respectively, and variable CtbSizeY indicates the CTB size in luma samples in the coded video sequence.

[0098] In some embodiments, when the intra dual-tree mode is enabled, a current CU of size greater than 64 may be further split into multiple smaller CUs, each contained within a VPDU region split by a quadtree.

[0099] In some embodiments, the values ​​of the variables minQtSizeY and minQtSizeC are constrained to be less than or equal to Min(64, CtbSizeY) for all slice types. Therefore, when a current CU with a size greater than 64 partially crosses (or straddles) the right or bottom picture boundary, the current CU can be further split into multiple smaller CUs by quadtree splitting.

[0100] III. Constraints on Segmentation Depth

[0101] In some embodiments, a constraint parameter related to the maximum MTT depth is specified. The constraint parameter is used to control the CU partition structure based on a quadtree with an embedded multi-type tree using binary splitting and / or ternary splitting. According to one aspect of the present invention, when the minimum QT size is greater than the minimum coding block size, the maximum MTT depth is constrained to be greater than 0. In some embodiments, when the minimum QT size is greater than the minimum coding block size, the maximum MTT depth is constrained to be greater than 0.

[0102] In some embodiments, the SPS syntax element log2_ctu_size_minus5 plus 5 specifies the luma coding treeblock size (base 2 logarithm) of each CTU. The syntax element log2_ctu_size_minus5 can be used to derive ctblog2SizeY and ctbSizeY according to the following:

[0103] CtbLog2SizeY=log2_ctu_size_minus5+5

[0104] CtbSizeY=1<<CtbLog2SizeY

[0105] In some embodiments, the SPS syntax element log2_min_luma_coding_block_size_minus plus 2 specifies the base-2 logarithm of the minimum luma coding block size. The syntax element log2_min_luna_coding_block_size_minus2 can be used to derive the minimum size of the CB in luma samples, denoted as MinCbLog2SizeY or MinCbSizeY. In some embodiments, MinCbSizeY is constrained to be less than or equal to Min(64, CtbSizeY). MinCbLog2SizeY and MinCbSizeY can be derived as follows:

[0106] MinCbLog2SizeY=log2_min_luma_coding__block_size_minus2+2

[0107] MinCbSizeY=1<<MinCbLog2SizeY

[0108] In some embodiments, the slice header (SH) syntax element slice_log2_diff_qt_min_cb_luma specifies the difference between the base 2 logarithm of the minimum size of luma samples of the luma block generated from the QT split of the CTU and the base 2 logarithm of the minimum coding block size of luma samples for the luma CU in the current slice.

[0109] In some embodiments, the slice header (SH) syntax element slice_max_mtt_hierarchy_depth_luma specifies the maximum hierarchical depth of coding units generated from a multi-type tree split of the leaves of the quadtree of the current slice. When slice_max_mtt_hierarchy_depth_luma is non-zero, the syntax elements slice_log2_diff_max_bt_min_qt_luma and slice_log2_diff_max_tt_min_qt_luma are encoded in the current slice header. The slice header syntax element slice_log2_diff_max_bt_min_qt_luma specifies the difference between the maximum size (width or height) (or logarithm with base 2) of a luma coding block that can be split using a binary split and the minimum size (width or height in luma samples) (logarithm with base 2) of a luma leaf block generated from a quadtree split of a CTU in the current slice. The SH syntax element slice_log2_diff_max_tt_min_qt_luma specifies the difference between the maximum size (width or height in luma samples) (logarithm with base 2) of a luma coding block that can be split using a ternary split and the minimum size (width or height in luma samples) (logarithm with base 2) of a luma leaf block generated from a quadtree split of CTUs in the current slice.

[0110] The variable MinQtLog2SizeY or MinQtSizeY indicates the minimum size of luma samples of a luma leaf block generated from the QT split of a CTU.

[0111] slice_log2_diff_max_bt_min_qt_luma and

[0112] The value of slice_log2_diff_max_tt_min_qt_luma is constrained to be in the range of 0 to CtbLog2SizeY-MinQtLog2SizeY, including 0 and CtbLog2SizeY-MinQtLog2SizeY. MinQtSizeY is constrained to be in the range of (MinCbSizeY, Min(64, CtbSizeY)), including MinCbSizeY and Min(64, CtbSizeY).

[0113] The syntax element slice_log2_diff_min_qt_min_cb_luma can be used to derive MinQtLog2SizeY and MinQtSizeY as follows:

[0114] MinQtLog2SizeY=MinCbLog2SizeY+slice_log2_diff_min_qt_min_cb_luma

[0115] MinQtSizeY=1< <MinQtLog2SizeY

[0116] Because the value of MinQtSizeY is constrained to be in the range of (MinCbSizeY, Min(64, CtbSizeY)), the range of slice_log2_diff_min_qt_min_cb_luma is 0 to Min(6, CtbLog2SizeY)-MinCbLog2SizeY, including 0 and Min(6, CtbLog2SizeY)-MinCbLog2SizeY.

[0117] The variable MaxBtSizeY indicates the maximum size in luma samples of a luma coding block that can be split using a binary split. The variable MaxBtSizeY is derived as follows:

[0118] MaxBtSizeY=1<<(MinQtLog2SizeY+slice_log2_diff_max_bt_min_qt_luma)

[0119] In some embodiments, the maximum BT size is constrained to be greater than or equal to the minimum QT size. Thus, when the current coding block (as a result of a split) reaches the minimum QT size, the coding block can be further recursively split using binary splits until the current coding block equals the minimum coding block size. In some embodiments, the maximum TT size is constrained to be greater than or equal to the minimum QT size. Thus, when the current coding block (as a result of a split) reaches the minimum QT size, the coding block can be further iteratively split using ternary splits until the current coding block equals the minimum coding block size.

[0120] The variable MaxTtSizeY indicates the maximum size of luma samples of a luma coding block that can be split using a ternary split. The variable MaxTtSizeY may be derived from the syntax element

[0121] slice_log2_diff_max_tt_min_qt_luma is derived as follows:

[0122] MaxTtSizeY=1<<(MinQtLog2SizeY+slice_log2_diff_max_tt_min_qt_luma)

[0123] Because MaxTtSizeY is in the range of MinQtSize to Min(64, CtbSizeY), slice_log2_diff_max_tt_min_qt_luma is in the range of 0 to Min(6, CtbLog2SizeY)-MinQtLog2SizeY.

[0124] In some embodiments, the slice header syntax elements slice_log2_diff_min_qt_min_cb_chroma, slice_log2_diff_max_bt_min_qt_chroma, and slice_log2_diff_max_tt_min_qt_chroma specify the minimum QT size, maximum BT size, and maximum TT size (in units of luma samples) of a chroma coding block, respectively. In some embodiments, the default values ​​of these three SH syntax elements are specified by the corresponding syntax elements in the SPS. For example, in some embodiments, there may be corresponding SPS parameters sps_log2_diff_min_qt_min_cb_chroma, sps_log2_diff_max_bt_min_qt_chroma, and sps_log2_djff_max_tt_min_qt_chroma that specify the minimum QT size, maximum BT size, and maximum TT size (in units of luma samples) of a chroma coding block, respectively, at the SPS level.

[0125] In some embodiments, the derived maximum BT size and maximum TT size may be greater than or equal to the minimum QT size. Therefore, when the current coding block reaches the minimum QT size and binary or ternary splitting is enabled, the current coding block may be further recursively split using binary or ternary splitting until the current coding block equals the minimum coding block size. However, when slice_max_mtt_hierarchy_depth_luma is equal to 0, both binary and ternary splitting are disabled. In some embodiments, if the minimum QT size is greater than the minimum coding block size, no coding block can be split into minimum coding blocks, as this may cause problems for splitting CUs along picture boundaries, where each CU that exceeds the boundary needs to be further iteratively split until the entire CU is within the picture boundary. In some embodiments, when slice_log2_diff_min_qt_min_cb_luma is non-zero, syntaxslice_max_mtt_hierarchy_depth_luma is constrained to be greater than 0.

[0126] Figure 8 The diagram conceptually illustrates constraint parameters for controlling the partitioning structure of a pixel block 800. The pixel block 800 may be a CTU partitioned using both QT and MTT. The diagram illustrates the CTU 800 together with a tree 810 illustrating a partial partitioning structure of the CTU 800.

[0127] The size of CTU 800 is 128×128. CTU 800 is constrained to not allow MTT (e.g., TT) splitting for partitions larger than 64. In other words, MTT splitting is not allowed for CTU 800, but MTT splitting is allowed for QT splitting of CTU 800 that is 64×64 or smaller. To ensure that each VPDU can be completely processed by the same pipeline stage, this maximum allowed block size of the MTT can be determined based on the size of the VPDU.

[0128] The maximum allowed size for the TI (e.g., variable MaxBtSizeY or MaxTtSizeY) can also be signaled by referencing the minimum size of the QT leaf (variable minQTSizeY). For example, the maximum allowed size for the MTT can be expressed as the difference between the base-2 logarithm of MaxTtSizeY and the base-2 logarithm of MinQtSizeY, such as slice_log2_diff_max_tt_min_qt_luma or a corresponding signal in the picture header or SPS. The minimum size of the QT leaf can be signaled by referencing the minimum size of the CB (variable MinCbSizeY). For example, the minimum size of the QT leaf can be expressed as the difference between the base-2 logarithm of MinQtSizeY and the base-2 logarithm of MinCbSizeY, such as slice_log2_diff_min_qt_min_cb_luma or a corresponding signal in the picture header or SPS.

[0129] IV. Transmitting block segmentation structure

[0130] In some embodiments, information about the maximum MTT depth is signaled based on the difference between the base-2 logarithm of the minimum QT size and the base-2 logarithm of the minimum coding block size. In some embodiments, MaxMttDepthY indicates the maximum hierarchical depth of coding units generated from a multi-type tree split of a quadtree leaf in the range of (0, 2*(CtbLog2SizeY-MinCbLog2SizeY)), inclusive. The variable MaxMttDepthY can be derived as follows:

[0131] MaxMttDepthY=slice_max_mtt_hierarchy_depth_luma+((1+slice_log2_diff_min_qt_min_cb_luma)>>1).

[0132] In some embodiments, constraint parameters for controlling the CU partitioning structure are specified. In some embodiments, when the current coding block reaches (or has) the minimum QT size, it is allowed to be further recursively split up to the minimum coding block size as long as at least one MTT split type (BT and TT) is still enabled. The video encoder can signal the maximum BT size or the maximum TT size to be less than the minimum QT size, when the current MTT depth is non-zero, where at least one of the maximum BT size and the maximum TT size is constrained to be greater than or equal to the minimum QT size. Therefore, the specified MTT structure can disable one of the MTT split types for coding blocks larger than the size constraint, and the size constraint is less than the minimum QT size. Specifically, by specifying the corresponding size constraint equal to the minimum coding block size, the video encoder can disable only one MTT type.

[0133] In some embodiments, a first syntax element signaling a maximum size of a first MTT type specifies the difference between the maximum size (logarithm to base 2) of luma samples of the first MTT type and the minimum coding block size (logarithm to base 2) of luma samples of the luma CU in the current slice. When the maximum size of the first MTT type is less than the minimum QT size and the minimum QT size is greater than the minimum coding block size, a second syntax element signaling a maximum size of a second MTT type specifies the difference between the maximum size (logarithm to base 2) of luma samples of the first MTT type and the minimum size (width or height) (logarithm to base 2) of luma samples of luma leaf blocks generated from a quadtree split of a CTU in the current slice. Otherwise, a second syntax element signaling a maximum size of the second MTT type specifies the difference between the maximum size (logarithm to base 2) of luma samples of the second MTT type and the minimum coding block size (logarithm to base 2) of luma samples of the luma CU in the current slice.

[0134] Table 1 provides an exemplary syntax table for a slice header (SH) for specifying a block partitioning structure. The syntax allows for one MTT type to be specified with a maximum size that is smaller than the minimum QT size.

[0135] Table 1

[0136]

[0137] The variables MaxBtSizeY and MaxTtSizeY are derived as follows:

[0138] MaxBtSizeY=1<<(MinCbLog2SizeY+slice_log2_max_bt_luma)

[0139] MaxTtSizeY=1<<

[0140] (((MaxBtSizeY<MinQtSizeY&&MinQtLog2SizeY>MinCbLog2SizeY)? MinQtLog2SizeY:MinCbLog2SizeY)+slice_log2_max_tt_luma).

[0141] In some other embodiments, the variables MaxBtSizeY and MaxTtSizeY are derived as follows:

[0142] MaxBtSizeY=1<<(MinCbLog2SizeY+slice_log2_max_bt_luma)

[0143] MaxTtSizeY=1<<(((MaxBtSizeY <MinQtSizeY)?

[0144] MinQtLog2SizeY:MinCbLog2SizeY)+slice_log2_max_tt_luma).

[0145] In some other embodiments, a first syntax element signaling a maximum size of a first MTT type specifies the difference between the base-2 logarithm of the maximum size of luma samples of the first MTT type and the base-2 logarithm of the minimum coding block size of luma samples of luma CUs in the current slice. A second syntax element signaling a maximum size of a second MTT type specifies the base-2 logarithm of the maximum size of luma samples of the second MTT type and the base-2 logarithm of the minimum coding block size of luma samples of luma CUs in the current slice. The variables MaxBtSizeY and MaxTtSizeY are derived as follows:

[0146] MaxBtSizeY=1<<(MinCbLog2SizeY+sice_log2_max_bt_luma)

[0147] MaxTtSizeY=1<<(MinCbLog2SizeY+slice_log2_max_tt_luma)

[0148] In some embodiments, at least one value of the MaxBtSizeY variable MaxBtSizeY is constrained to be greater than or equal to MinQtSizeY.

[0149] It is worth noting that when dual-tree splitting mode is enabled (qtbtt_dual_tree_intra_flagequal is equal to 1), the proposed method and related embodiments can be further applied to specify other relevant syntax elements to signal the maximum BT size and the maximum TT size in the slice header of the intra slice for splitting the chroma codec block. The proposed method and related embodiments can be further applied to specify other relevant syntax elements to signal the default maximum BT size and the default maximum TT size in the SPS.

[0150] In some embodiments, when the current coding block reaches the minimum QT size, it is allowed to be further recursively split until the minimum coding block size is reached (e.g., by enabling binary or ternary splits for coding block sizes less than or equal to the minimum QT size without any constraints on the maximum size of the binary or ternary splits). In some embodiments, under the constraint that the maximum BT size is greater than or equal to the minimum QT size, the video encoder can signal that the maximum TT size is less than the minimum QT size. Therefore, the specified MTT structure can disable ternary splits for coding blocks of sizes greater than the specified size constraint, i.e., less than the minimum QT size. Specifically, by specifying the corresponding maximum size constraint to be equal to the minimum coding block size, the video encoder can effectively disable ternary splits.

[0151] In some embodiments, a syntax element that signals a maximum size of a binary split type specifies the difference between the maximum size (logarithm to base 2) of a luma coding block (luma samples) that can be split using a binary split and the minimum coding block size (width or height of luma samples) (logarithm to base 2) of a luma leaf block generated from a quadtree split of a CTU of the current slice. Another syntax element that signals a maximum size of a ternary type specifies the difference between the maximum size (width or height of luma samples) that can be split using a ternary split and the minimum block size (logarithm to base 2) of luma samples of the luma CU in the current slice. Table 2 provides an example syntax table for a slice header that can support a ternary split type where the specified maximum size is less than the minimum QT size.

[0152] Table 2

[0153]

[0154]

[0155] The variable MaxBtSizeY indicates the maximum size of the luma CB allowed for BT splitting. The variable MaxTtSizeY indicates the maximum size of the luma CB allowed for TT splitting. MaxBtSizeY and MaxTtSizeY are in the range of MinQtSize to Min(64, CtbSizeY), inclusive. MaxBtSizeY and MaxTtSizeY are derived as follows:

[0156] MaxBtSizeY=1<<(MinQtLog2SizeY+slice_log2_diff_max_bt_min_qt_luma)

[0157] MaxTtSizeY=1<<(MinCbLog2SizeY+slice_log2_diff_min_tt_min_cb_luma).

[0158] The variable MaxBtSizeC indicates the maximum size of the chroma CB allowed for BT splitting. The variable MaxTtSizeY indicates the maximum size of the chroma CB allowed for TT splitting. The variables MaxBtSizeC and MaxTtSizeC are derived as follows:

[0159] MaxBtSizeC=1<<(MinQtLog2SizeC+slice_log2_diff_max_bt_min_qt_chroma)

[0160] MaxTtSizeC=1<<(MinCbLog2SizeY+slice_log2_diff_min_tt_min_cb_chroma)

[0161] It is worth noting that when dual-tree partitioning mode is enabled (qtbtt_dual_tree_intra_flag is equal to 1), the methods described in Sections II to IV and the related embodiments can be further applied to other relevant syntax elements for signaling information for deriving the maximum MTT depth in the slice header of intra slices to split chroma coding blocks. The proposed scheme can be further applied to signaling relevant constraint parameters in the picture header. The proposed scheme and the related embodiments can also be applied to other relevant syntax elements for signaling information for deriving the preset maximum MTT depth in the SPS. For example, there are picture header parameters corresponding to slice header parameters and SPS parameters such as slice_log2_diff_max_tt_min_qt_luma and slice_log2_diff_min_qt_min_cb_luma that set default values ​​for these parameters at the sequence or picture level.

[0162] Any of the aforementioned proposed methods may be implemented in an encoder and / or decoder. For example, any of the proposed methods may be implemented in an entropy encoding (or high-level syntax encoding) module of an encoder and / or an entropy decoding (or high-level syntax decoding) module of a decoder. Alternatively, any of the aforementioned proposed methods may be implemented as circuitry integrated into an entropy encoding (or high-level syntax encoding) module of an encoder and / or an entropy decoding (or high-level syntax decoding) module of a decoder. Any of the aforementioned proposed methods may also be implemented in an image encoder and / or decoder, wherein the generated bitstream corresponds to a coded frame using only intra-frame prediction.

[0163] V. Exemplary Video Encoders

[0164] Figure 9 An exemplary video encoder 900 that signals constraints regarding block partitioning is shown. As shown, the video encoder 900 receives an input video signal from a video source 905 and encodes the signal into a bitstream 995. The video encoder 900 has various components or modules for encoding the signal from the video source 905, including at least some components selected from a transform module 910, a quantization module 911, an inverse quantization module 914, an inverse transform module 915, an intra-frame image estimation module 920, an intra-frame prediction module 925, a motion compensation module 930, a motion estimation module 935, a loop filter 945, a reconstructed image buffer 950, an MV buffer 965, an MV prediction module 975, and an entropy encoder 990. The motion compensation module 930 and the motion estimation module 935 are part of the inter-frame prediction module 940.

[0165] In some embodiments, modules 910-990 are modules of software instructions executed by one or more processing units (e.g., processors) of a computing device or electronic device. In some embodiments, modules 910-990 are modules of hardware circuitry implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 910-990 are shown as separate modules, some modules may be combined into a single module.

[0166] A video source 905 provides a raw video signal representing uncompressed pixel data for each video frame. A subtractor 908 calculates the difference between the raw video pixel data from the video source 905 and the predicted pixel data 913 from the motion compensation module 930 or the intra-frame prediction module 925. The transform module 910 transforms the difference (or residual pixel data or residual signal 909) into transform coefficients (e.g., by performing a discrete cosine transform, or DCT). The quantization module 911 quantizes the transform coefficients into quantized data (or quantized coefficients) 912, which are encoded into a bitstream 995 by the entropy encoder 990.

[0167] The inverse quantization module 914 dequantizes the quantized data (or quantized coefficients) 912 to obtain transform coefficients, and the inverse transform module 915 performs an inverse transform on the transform coefficients to generate a reconstructed residual 919. The reconstructed residual 919 is added to the predicted pixel data 913 to generate reconstructed pixel data 917. In some embodiments, the reconstructed pixel data 917 is temporarily stored in a linear buffer (not shown) for intra-frame image prediction and spatial MV prediction. The reconstructed pixels are filtered by the loop filter 945 and stored in a reconstructed image buffer 950. In some embodiments, the reconstructed image buffer 950 is an external memory area of ​​the video encoder 900. In some embodiments, the reconstructed image buffer 950 is an internal memory area of ​​the video encoder 900.

[0168] The intra-frame image estimation module 920 performs intra-frame prediction based on the reconstructed pixel data 917 to generate intra-frame prediction data. The intra-frame prediction data is provided to the entropy encoder 990 to be encoded into the bitstream 995. The intra-frame prediction data is also used by the intra-frame prediction module 925 to generate predicted pixel data 913.

[0169] The motion estimation module 935 performs inter-frame prediction by generating MVs to parameterize pixel data of previously decoded frames stored in the reconstructed image buffer 950. These MVs are provided to the motion compensation module 930 to generate predicted pixel data.

[0170] The video encoder 900 uses MV prediction to generate a predicted MV instead of encoding the complete actual MV in the bitstream, and the difference between the MV used for motion compensation and the predicted MV is encoded as residual motion data and stored in the bitstream 995 .

[0171] The MV prediction module 975 generates a predicted MV based on a reference MV generated for encoding a previous video frame (i.e., a motion compensated MV used to perform motion compensation). The MV prediction module 975 retrieves the reference MV from the previous video frame from the MV buffer 965. The video decoder 900 stores the MV generated for the current video frame in the MV buffer 965 as a reference MV for generating the predicted MV.

[0172] The MV prediction module 975 uses the reference MV to create a predicted MV. The predicted MV can be calculated by spatial MV prediction or temporal MV prediction. The difference between the predicted MV and the motion-compensated MY (MCMY) of the current frame is encoded into the bitstream 995 by the entropy encoder 990.

[0173] Using entropy coding techniques such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding, the entropy encoder 990 encodes various parameters and data into a bitstream 995. The entropy encoder 990 encodes various header elements and flags along with the quantized transform coefficients 912 and the residual motion data set as syntax elements into the bitstream 995. The bitstream 995 is in turn stored in a storage device or transmitted to a decoder via a communication medium such as a network.

[0174] The loop filter 945 performs a filtering or smoothing operation on the reconstructed pixel data to reduce artifacts in the coded solution, particularly at pixel block boundaries. In some embodiments, the filtering operation performed includes sample adaptive offset (SAO). In some embodiments, the filtering operation includes an adaptive loop filter (ALF).

[0175] Figure 10A video encoder 900 is shown that implements block partitioning constraints. Specifically, a partitioning engine 1010 generates a set of partitioning constraints 1020 for use by an entropy encoder 990 based on hardware constraints (e.g., VPDU size) and a set of rate / distortion information 1015. The entropy encoder 990 encodes or signals the set of partitioning constraints 1020 into the bitstream 995. The partitioning constraints 1020 may include constraints applicable at different levels of the video hierarchy (e.g., sequence, picture, slice, block) and may be signaled by the entropy encoder 990 in a slice header, picture header, SPS, etc. The partitioning constraints 1020 may include a maximum MTT block size, a minimum QT leaf size, etc. The partitioning engine 1010 also provides a partitioning structure 1030 to the transform module 910 so that the transform module can perform a transform operation on the current block of pixels being encoded (e.g., CTU) according to the partitioning structure 1030.

[0176] Figure 11 A process 1100 for signaling a block partition constraint is conceptually illustrated. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing encoder 900 perform process 1100 by executing instructions stored on a computer-readable medium. In some embodiments, an electronic device implementing decoder 900 performs process 1100.

[0177] The encoder signals (at block 1110) a segmentation control parameter in the bitstream specifying that the maximum block size for enabling ternary tree splitting is constrained to 64 or less.

[0178] In some embodiments, the partition control parameters are signaled in a slice header of the slice that includes the current block. The partition control parameters may be signaled in a picture header of the current picture. The partition control parameters may be signaled in a sequence parameter set (SPS) of a sequence of video pictures that includes the current picture. In some embodiments, the SPS or picture header sets a preset value for the partition control parameter, which may be overwritten by the slice header that includes the current block.

[0179] In some embodiments, the partitioning control parameters specify a maximum block size by reference to the minimum size of a quadtree split leaf block, for example, as the difference between (i) the base-2 logarithm of the maximum block size that a block can be further partitioned by ternary tree partitioning and (ii) the base-2 logarithm of the minimum size of a quadtree split leaf block. The minimum size of a quadtree split leaf block is constrained to be the smaller of 64 and the coding tree block size. The maximum block size for enabling TT partitioning is further constrained to be smaller than the coding tree block size.

[0180] The minimum size of a quadtree leaf block size can be specified by reference to the minimum coding block size, and the minimum size of a quadtree split leaf block is specified by the difference between (i) the base 2 logarithm of the minimum size of the quadtree split leaf block and (ii) the base 2 logarithm of the minimum coding block size.

[0181] In some embodiments, the maximum block size is determined based on the size of a virtual pipeline data unit (VPDU), and the maximum block size is defined based on ensuring that each VPDU can be fully processed by the same pipeline stage.

[0182] The encoder receives (at block 1120) raw pixel data for a current block of a current picture of a video to be encoded into a bitstream.

[0183] The encoder recursively splits (at block 1130) the current block into one or more partitions. Unless the partition is less than or equal to the maximum block size, the encoder does not allow (at block 1135) TT splitting for a partition of the current block unless the partition is less than or equal to the maximum block size.

[0184] The encoder encodes (at block 1140) one or more partitions of the current block into a bitstream.

[0185] VI. Exemplary Video Encoders

[0186] Figure 12 An exemplary video decoder 1200 is shown that applies constraints to block segmentation. As shown, video decoder 1200 is an image decoding or video decoding circuit that receives a bitstream 1295 and decodes the contents of the bitstream into pixel data of a video frame for display. Video decoder 1200 has multiple components or modules for decoding bitstream 1295, including selected components from an inverse quantization module 1211, an inverse transform module 1210, an intra-frame prediction module 1225, a motion compensation module 1230, a loop filter 1245, a decoded picture buffer 1250, an MV buffer 1265, an MV prediction module 1275, and a parser 1290. Motion compensation module 1230 is part of inter-frame prediction module 1240.

[0187] In some embodiments, modules 1210-1290 are modules of software instructions executed by one or more processing units (e.g., processors) of a computing device. In some embodiments, modules 1210-1290 are modules of hardware circuitry implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 1210-1290 are shown as separate modules, some modules may be combined into a single module.

[0188] The parser 1290 (or entropy decoder) performs initial parsing according to the syntax defined by the video codec or image codec standard. The parsed syntax elements include various header elements, flags, and quantized data (or quantized coefficients) 1212. The parser 1290 parses the various syntax elements using entropy coding techniques such as context-adaptive arithmetic coding (CABAC) or Huffman coding.

[0189] The inverse quantization module 1211 dequantizes the quantized data (or quantized coefficients) 1212 to obtain transform coefficients, and the inverse transform module 1210 performs an inverse transform on the transform coefficients to generate a reconstructed parameter signal 1219. The reconstructed residual signal 1219 is added to the predicted pixel data 1213 from the intra prediction module 1225 or the motion compensation module 1230 to generate decoded pixel data 1217. The decoded pixel data is filtered by the loop filter 1245 and stored in the decoded picture buffer 1250. In some embodiments, the decoded picture buffer 1250 is an external storage area of ​​the video decoder 1200. In some embodiments, the decoded picture buffer 1250 is an internal storage area of ​​the video decoder 1200.

[0190] The intra prediction module 1225 receives intra prediction data from the bitstream 1295 and generates predicted pixel data 1213 based on the intra prediction data from the decoded pixel data 1217 stored in the decoded picture buffer 1250. In some embodiments, the decoded pixel data 1217 is also stored in a linear buffer (not shown) for intra picture prediction and spatial MV prediction.

[0191] In some embodiments, the contents of the decoded image buffer 1250 are used for display. The display device 1255 retrieves the contents of the decoded image buffer 1250 directly for display or retrieves the contents of the decoded image buffer into a display buffer. In some embodiments, the display device receives pixel values ​​from the decoded image buffer 1250 via pixel transfer.

[0192] The motion compensation module 1230 generates predicted pixel data 1213 from the decoded pixel data 1217 stored in the decoded picture buffer 1250 according to the motion compensated MV (MC MY). These motion compensated MVs are decoded by adding the residual motion data from the bitstream 1295 to the predicted MV from the MV prediction module 1275.

[0193] MV prediction module 1275 generates a predicted MV based on a reference MV generated for decoding a previous video frame, such as a motion-compensated MV for performing motion compensation. MV prediction module 1275 retrieves the reference MV for the previous video frame from MV buffer 1265. Video decoder 1200 stores the motion-compensated MV generated for decoding the current video frame in MV buffer 1265 as a reference MV for generating the predicted MV.

[0194] The loop filter performs a filtering or smoothing operation on the decoded pixel data 1217 to reduce encoding and decoding artifacts, particularly at pixel block boundaries. In some embodiments, the filtering operation performed includes sample adaptive offset (SAO). In some embodiments, the filtering operation includes adaptive loop filtering (ALF).

[0195] Figure 13 A portion of a video decoder 1200 that implements block partitioning constraints is shown. Specifically, an entropy decoder 1290 parses syntax elements related to block partitioning from a bitstream 1295 and generates a set of partitioning constraints 1320. The partitioning constraints may include constraints applicable at different levels of the video hierarchy (e.g., sequence, picture, slice, block) and may be signaled in a slice header, picture header, SPS, etc. The partitioning constraints 1320 may include a maximum MTT block size, a minimum QT leaf size, etc. Based on the partitioning constraints 1320, the decoder may not allow certain splitting modes that exceed the maximum block size.

[0196] The generated partition constraints 1320 are applied to the partition engine 1310 , which generates a partition structure 1330 for the inverse transform module 1210 so that it can perform inverse transform operations on a single partition (eg, CU) according to the partition structure.

[0197] Figure 14 The process 1400 of applying a block partition constraint is conceptually illustrated. In some embodiments, one or more processing units (e.g., processors) of a computing device implement a decoder to perform process 1400 by executing code stored in a computer-readable medium. In some embodiments, an electronic device implementing decoder 1200 performs process 1400.

[0198] The decoder receives (at block 1410) a partitioning control parameter from the bitstream specifying a maximum block size for enabling ternary tree (TT) splitting. In some embodiments, the maximum block size is constrained to 64 or less.

[0199] In some embodiments, the partition control parameter is signaled in a slice header of the slice that includes the current block. The partition control parameter may be signaled in a picture header of the current picture. The partition control parameter may be signaled in a sequence parameter set (SPS) of a sequence of video pictures that includes the current picture. In some embodiments, the SPS or picture header sets a preset value for the partition control parameter, which may be overwritten by the slice header that includes the current block.

[0200] In some embodiments, the partitioning control parameters specify a maximum block size by reference to the minimum size of a quadtree split leaf block, for example, by using the difference between (i) the base-2 logarithm of the maximum block size that a block can be further partitioned by a ternary tree split and (ii) the base-2 logarithm of the minimum size of a quadtree split leaf block. The minimum size of a quadtree split leaf block is constrained to be the smaller of 64 and the coding tree block size. The maximum block size for enabling TT splitting can be further constrained to be smaller than the coding tree block size.

[0201] The minimum size of a quadtree split leaf block can be specified by reference to the minimum coding block size, and the minimum size of a quadtree split leaf block can be specified by using the difference between (i) the base 2 degree of the minimum size of the quadtree split leaf block and (ii) the base 2 logarithm of the minimum coding block size.

[0202] In some embodiments, the maximum block size is determined based on the size of a virtual pipeline data unit (VPDU), and the maximum block size is defined based on ensuring that each VPDU can be completely processed by the same pipeline stage.

[0203] In some embodiments, the partition control parameter is signaled in a slice header of the slice that includes the current block. The partition control parameter may be signaled in a picture header of the current picture. The partition control parameter may be signaled in a sequence parameter set (SPS) of a sequence of video pictures that includes the current picture. In some embodiments, the SPS or picture header sets a preset value for the partition control parameter, which may be overwritten by the slice header that includes the current block.

[0204] The decoder receives (at block 1420) from a bitstream a block of pixels to be decoded as a current block of a current picture of a video.

[0205] The decoder recursively splits (at block 1430) the current block into one or more partitions. The decoder disables (at block 1435) TT splitting for a partition of the current block unless the partition is less than or equal to the maximum block size.

[0206] The decoder reconstructs (at block 1440) one or more partitions of the current block.

[0207] In some embodiments, an encoder may signal (or generate) one or more syntax elements in a bitstream such that a decoder may parse the one or more syntax elements from the bitstream.

[0208] VII. Exemplary Electronic Systems

[0209] Many of the features and applications described above are implemented as software processes, which are specified as a set of instructions recorded on a computer-readable storage medium (also referred to as computer-readable media). When these instructions are executed by one or more computing or processing units (e.g., one or more processors, processor cores, or other processing units), the processing units perform the actions indicated by the instructions. Examples of computer-readable media include, but are not limited to, CD-ROMs, flash drives, random access memory (RAM) chips, hard drives, erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), etc. Computer-readable media does not include carrier waves and electronic signals transmitted wirelessly or over wired connections.

[0210] In this specification, the term "software" is intended to include firmware residing in read-only memory or applications stored in magnetic storage that can be read into memory and processed by a processor. Additionally, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger component while retaining the independence of the software invention. In some embodiments, multiple software inventions can also be implemented as separate programs. Finally, any combination of separate programs together with the software inventions described herein are within the scope of the present invention. In some embodiments, when installed to operate one or more electronic systems, the software component defines one or more specific machine implementations that execute and allow the operation of the software program.

[0211] Figure 15 The electronic system 1500 is conceptually illustrated, in which some embodiments of the present invention may be implemented. The electronic system 1500 may be a computer (e.g., a desktop computer, a personal computer, a tablet computer, etc.), a phone, a PDA, or any other suitable electronic device. Such electronic devices may include various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 1500 includes a bus 1505, a processing unit 1510, a graphics processing unit (GPU) 1515, system memory 1520, a network 1525, read-only memory 1530, a persistent storage device 1535, an input device 1540, and an output device 1545.

[0212] Buses 1505 collectively represent all system, peripheral, and chipset buses that communicatively connect the various internal devices of electronic system 1500. For example, bus 1505 communicatively connects processing unit 1510, GPU 1515, read-only memory 1530, system memory 1520, and persistent storage 1535.

[0213] From these various storage units, processing unit 1510 retrieves instructions to be executed and data to be processed to execute the processes of the present invention. The processing unit can be a single processor or a multi-core processor in different embodiments. Some embodiments are transmitted and executed by GPU 1515. GPU 1515 can offload various calculations provided by processor unit 1510 or perform image processing.

[0214] Read-only memory (ROM) stores static data and instructions used by processing unit 1510 and other modules of the electronic system. Persistent storage device 1535, on the other hand, is a read-write memory device. This device is a non-volatile storage unit that stores instructions and data even when electronic system 1500 is turned off. Some embodiments of the present invention use a mass storage device (such as a magnetic or optical disk and its corresponding hard drive) as persistent storage device 1535.

[0215] Other embodiments use removable storage devices (such as floppy disks, flash storage devices, etc. and their corresponding hard disk drives) as permanent storage devices. Like permanent storage device 1535, system memory 1520 is a read-write storage device. However, unlike storage device 1535, system memory 1520 is a volatile read-write memory, such as random access memory. System memory 1520 stores some instructions and data used by the processor during operation. In some embodiments, processes according to the present invention are stored in system memory 1520, permanent storage device 1535 and / or read-only memory 1530. For example, various storage units include instructions for processing multimedia according to some embodiments. From these memory units, processing unit 1510 retrieves instructions to be executed and data to be processed to execute the processes of some embodiments.

[0216] Bus 1505 also connects to input and output devices 1540 and 1545. Input device 1540 enables a user to communicate information and select commands to the electronic system. Input device 1540 includes an alphanumeric keyboard and pointing device (also known as a cursor control device), a camera (e.g., a webcam), a microphone, or similar device for receiving voice commands. Input device 1545 displays images generated by the electronic system or other output data. Output device 1545 includes a printer and a display device, such as a cathode ray tube (CRT) or liquid crystal display (LCD), as well as speakers or similar audio output devices. Some embodiments include a touch screen as both an input and output device.

[0217] Finally, if Figure 15 As shown, bus 1505 also couples electronic system 1500 to network 1525 via a network interface card (not shown). In this manner, the computer can be part of a network of computers, such as a local area network (LAN), a wide area network (WAN), or an intranet, or a network of networks, such as the Internet. Any or all of the components of electronic system 1500 may be used in conjunction with the present invention.

[0218] Some embodiments include electronic components, such as microprocessors, and computer program instruction storage and memory or computer-readable media (or computer-readable storage media, and readable media or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, compact disc read-only memory (CD-ROM), compact disc recordable memory (CD-R), compact disc rewritable memory (CD-RW), read-only universal optical discs (e.g., DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD card, mini-SD card, micro-SD card, etc.), magnetic and / or solid-state hard drives, read-only and recordable Blu-ray discs, ultra-high-density optical discs, any other optical or magnetic media, and floppy disks. The computer-readable media can be used to store a computer program executed by at least one processing unit and includes a set of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as produced by a compiler, and files including higher-level code that are executed by a computer, electronic component, or microprocessor using an interpreter.

[0219] While the above discussion primarily refers to microprocessors or multi-core processors executing software, many of the features and applications discussed above are implemented by one or more integrated circuits, such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuits themselves. Additionally, some embodiments execute software stored in programmable logic devices (PLDs), ROM, or RAM devices.

[0220] As used in this specification and any claims herein, the terms "computer," "server," "processor," and "memory" refer to electronic or other technological devices. These terms do not include persons or groups of persons. For purposes of this specification, the term "display" means displayed on an electronic device. As used in this specification and any claims herein, the terms "computer-readable medium," "computer-readable media," and "machine-readable medium" are restricted to tangible, physical objects that store information in a computer-readable form. These terms exclude any wireless signals, wired downloaded signals, and any other ephemeral signals.

[0221] Although the present invention has been described with reference to various specific details, a person skilled in the art will appreciate that the present invention may be presented in other specific forms without departing from the spirit of the present invention. Figure 11 as well as Figure 14 ) conceptually illustrates processes. The specific operations of these processes may not be performed in the order shown and described. Specific operations may not be performed as a continuous operation, and different specific operations may be performed in different embodiments. In addition, processes may be implemented using various sub-processes or as part of a larger macro process. Therefore, those skilled in the art will understand that the present invention is not limited by the foregoing illustrative details, but is defined by the appended claims.

[0222] Notes

[0223] The subject matter described herein sometimes shows different elements included in or connected to different other elements. It is understood that the architecture of this description is only an example, and in fact many other architectures that can be implemented to achieve the same function. Conceptually, any arrangement of elements that achieve the same function is effectively "associated" so as to achieve the desired function. Therefore, any two elements that are combined herein to achieve a specific function can be considered to be "associated" to each other so as to achieve the desired function, regardless of the architecture or intermediate elements. Similarly, any two elements that are so associated can also be considered to be "operably connected" or "operably coupled" to each other to achieve the desired function, and any two elements that can be so associated can also be considered to be "operably coupled" to each other to achieve the desired function. The specific examples of operably coupled include but are not limited to physically matchable and / or physically interactive elements and / or wirelessly understandable and / or wirelessly interactive elements and / or logically interactive and / or logically interactive elements.

[0224] Furthermore, with respect to the use of substantially any plural and / or singular terms herein, those skilled in the art may translate from the plural to the singular and / or from the singular to the plural as appropriate, depending on the context and application. For clarity, various singular / plural permutations are expressly set forth herein.

[0225] In addition, those skilled in the art will understand that, generally, the terms used herein, and particularly the terms used in the appended claims (e.g., the body of the appended claims), are generally intended to be "open-ended" terms, e.g., the term "including" should be interpreted as "including but not limited to," the term "having" should be interpreted as "having at least," the term "includes" should be interpreted as "including but not limited to," etc. Those skilled in the art will further understand that if a specific number of cited claim statements is intended, such intent will be explicitly enumerated in the claims, and in the absence of such a statement, such intent is absent. For example, to aid understanding, the appended claims may include the use of the introductory phrases "at least one" and "one or more" to introduce claim statements. However, the use of such phrases should not be construed to imply that a claim statement introduced by the indefinite article "a" or "an" limits any particular claim statement containing such introduced claim statement to embodiments containing only one such statement, even when the same claim statement includes the introductory phrases "one or more" or "at least one" and an indefinite article such as "a" or "an," "a" and / or "an" should be construed to mean "at least one" or "one or more," and the same applies to the definite article introducing the claim statement. Furthermore, even if a specific number of introduced claim statements is explicitly recited, one of ordinary skill in the art will recognize that such recitation should be construed to mean at least one of the recited number, e.g., the mere recitation "two statements" without other modifications means at least two statements, or two or more statements. Furthermore, where a convention similar to "at least one of A, B, and C, etc." is used, generally such construction is intended to be understood by one of ordinary skill in the art, such that "a system having at least one of A, B, and C" would include, but is not limited to, a system having A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B, and C together, etc. In such instances where a convention similar to "at least one of A, B, or C" is used, generally such construction is intended to be understood by one of ordinary skill in the art, such that "a system having at least one of A, B, or C" would include, but is not limited to, a system having A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B, and C together, etc. Those skilled in the art will further understand that, in fact, any separator and / or phrase indicating two or more alternative terms in the description, claims, or figures will be understood to contemplate the possibility of including one, either, or both of the terms. For example, the phrase "A or B" will be understood to include the possibilities of "A or B" or "A and B."

[0226] It will be appreciated from the foregoing that various embodiments of the present invention have been described herein for illustrative purposes, and that various modifications may be made without departing from the scope and spirit of the invention. Therefore, the various embodiments described herein are not intended to be limiting, the true scope and spirit of which are to be determined by the scope of the subsequent patent claims.

Claims

1. A decoding method, comprising: receiving a split control parameter from a bitstream, the split control parameter specifying a minimum size of a leaf block for enabling quadtree splitting; receiving data from a bitstream to be decoded as a pixel block of a current block of a current picture of a video; recursively splitting the current block into one or more partitions, wherein a quadtree split is not allowed for a partition of the current block unless the partition is larger than a minimum size of a leaf block of the quadtree split; and reconstructing the one or more partitions of the current block; A smaller value between 64 and the coding tree block size is used as a specific value, and the specific value is used as the maximum value of the minimum size of the leaf block of the quadtree split.

2. The decoding method according to claim 1, wherein: The split control parameter specifies the maximum block size for a ternary tree split by reference to the minimum size of a leaf block for the quadtree split.

3. The decoding method according to claim 2, wherein: The split control parameter is specified by using the difference between (i) the base 2 logarithm of the maximum block size of a block that can be further split by a ternary tree split and (ii) the base 2 logarithm of the minimum size of the quadtree split leaf block.

4. The decoding method according to claim 2, wherein: The minimum size of the quadtree split leaf block is specified by reference to a minimum coding block size.

5. The decoding method according to claim 4, wherein: The minimum size of the quadtree split leaf block is specified by using the difference between (i) the base 2 logarithm of the minimum size of the quadtree split leaf block and (ii) the base 2 logarithm of the minimum coding block size.

6. The decoding method according to claim 2, wherein: The maximum block size for enabling ternary tree splitting is further constrained to be smaller than a coding tree block size.

7. The decoding method according to claim 2, wherein: The maximum block size is determined based on the size of a virtual pipeline data unit (VPDU).

8. The decoding method according to claim 7, wherein: The maximum block size is defined based on ensuring that each virtual pipeline data unit can be completely processed by a same pipeline stage.

9. The decoding method according to claim 1, wherein: The partition control parameter is signaled in a band header of a band including the current block.

10. The decoding method according to claim 1, wherein: The segmentation control parameters are signaled in a picture header of the current picture.

11. The decoding method according to claim 1, wherein: The segmentation control parameters are signaled in a sequence parameter set for a sequence of video pictures including the current picture.

12. A video encoding method, comprising: Signaling a split control parameter in a bitstream, the split control parameter specifying a minimum size for enabling quadtree splitting; Receiving raw pixel data of a current block of a current picture of a video to be encoded into the bitstream; recursively splitting the current block into one or more partitions, wherein a quadtree split is not allowed for a partition of the current block unless the partition is larger than a minimum size of a leaf block of the quadtree split; and encoding the one or more partitions of the current block into the bitstream; A smaller value between 64 and the coding tree block size is used as a specific value, and the specific value is used as the maximum value of the minimum size of the leaf block of the quadtree split.

13. An electronic device, The device comprises a processor and a memory. The processor executes program instructions stored in the memory so that the device performs a decoding method. The method comprises: Receiving a split control parameter from a bitstream, the split control parameter specifying a minimum size for enabling quadtree splitting; receiving data from a bitstream to be decoded as a pixel block of a current block of a current picture of a video; recursively splitting the current block into one or more partitions, wherein a quadtree split is not allowed for a partition of the current block unless the partition is larger than a minimum size of a leaf block of the quadtree split; and reconstructing the one or more partitions of the current block; A smaller value between 64 and the coding tree block size is used as a specific value, and the specific value is used as the maximum value of the minimum size of the leaf block of the quadtree split.

Images