Method and apparatus for using individual partition trees for color components in video coding system
By employing a separate chroma segmentation tree and segmentation tree structure in video encoding and decoding, and utilizing context formation and entropy encoding and decoding techniques, the problem of low encoding and decoding efficiency in existing technologies is solved, achieving more efficient encoding and decoding performance.
Patent Information
- Application Number
- CN202380033897.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-04-13
- Filing Date
- 2023-03-20
- Publication Date
- 2024-11-26
AI Technical Summary
Existing video encoding and decoding technologies struggle to effectively utilize individual chroma segmentation trees and the entropy of these trees to improve encoding and decoding efficiency when dealing with multi-functional video encoding and decoding standards. This is especially true when processing the local characteristics of different color components, resulting in low encoding and decoding efficiency.
A separate chroma segmentation tree and segmentation tree structure are adopted. Through context formation and entropy encoding and decoding techniques, the luminance and chroma components are segmented and encoded and decoded independently. The information of the luminance segmentation tree is used to adjust the entropy encoding and decoding context of the chroma segmentation tree, thereby improving the encoding and decoding efficiency.
By using an independent chroma segmentation tree structure and entropy encoding/decoding technology, the efficiency of video encoding/decoding is improved, especially when processing different color components, reducing redundant signaling and improving encoding/decoding performance.
Smart Images

Figure CN119032576A_ABST
Abstract
Description
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This disclosure is part of a non-provisional application that claims priority to U.S. Provisional Patent Application No. 63 / 330,342, filed on April 13, 2022. The contents of the above-listed applications are incorporated herein by reference in their entirety. Technical Field
[0003] The present disclosure relates generally to codec systems and in particular to partitioning color blocks using separate chroma partition trees and entropy of the partition trees in a video codec system. Background Art
[0004] Versatile video coding (VVC) is the latest international video coding standard jointly developed by the Joint Video Experts Team (JVET) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG). The standard has been published as an ISO standard: ISO / IEC 23090-3:2021, Information technology - Codec representation of immersive media - Part 3: Versatile video coding, published in February 2021. VVC is developed based on its predecessor HEVC (High Efficiency Video Coding) by adding more codec tools to improve codec efficiency, and can also handle various types of video sources, including 3-dimensional (3D) video signals.
[0005] Figure 1AAn example adaptive inter / intra video codec system including a loop process is shown. For intra prediction, prediction data is derived from previously coded video data in the current picture. For inter prediction 112, motion estimation (ME) is performed on the encoder side and motion compensation (MC) is performed based on the results of ME to provide prediction data derived from other pictures and motion data. Switch 114 selects intra prediction 110 or inter prediction 112 and the selected prediction data is provided to adder 116 to form a prediction error, also known as a residual. The prediction error is then processed by transform (T) 118 and subsequent quantization (Q) 120. Next, the transformed and quantized residual is encoded by entropy encoder 122 to be included in a video bitstream corresponding to the compressed video data. The bitstream associated with the transform coefficients is then packaged together with auxiliary information (e.g., motion and codec modes associated with intra prediction and inter prediction) and other information (e.g., parameters associated with loop filters applied to underlying image areas). Auxiliary information associated with intra prediction 110, inter prediction 112 and loop filter 130 is provided to entropy encoder 122, such as Figure 1A As shown. When the inter-frame prediction mode is used, one or more reference pictures must also be reconstructed at the encoder end. Therefore, the transformed and quantized residual is processed by Inverse Quantization (IQ) 124 and Inverse Transformation (IT) 126 to recover the residual. The residual is then added back to the prediction data 136 at Reconstruction (REC) 128 to reconstruct the video data. The reconstructed video data can be stored in the reference picture buffer 134 and used to predict other frames.
[0006] like Figure 1AAs shown, the input video data undergoes a series of processes in the encoding system. Due to the series of processes, the reconstructed video data from REC 128 may be subject to various impairments. Therefore, the loop filter 130 is often applied to the reconstructed video data before it is stored in the reference picture buffer 134 to improve the video quality. For example, a deblocking filter (DF for short), a sample adaptive offset (SAO for short), and an adaptive loop filter (ALF for short). The loop filter information may need to be incorporated into the bitstream so that the decoder can correctly recover the required information. Therefore, the loop filter information is also provided to the entropy encoder 122 to be incorporated into the bitstream. In Figure 1A In , a loop filter 130 is applied to the reconstructed video before the reconstructed samples are stored in a reference picture buffer 134 . Figure 1A The system in is intended to illustrate an exemplary structure of a typical video encoder. It may correspond to a High Efficiency Video Coding (HEVC) system, VP8, VP9, H.264 or VVC.
[0007] like Figure 1B As shown, the decoder can use similar or the same functional blocks as those in the encoder except for transform 118 and quantization 120, because the decoder only needs inverse quantization 124 and inverse transform 126. The decoder uses entropy decoder 140 instead of entropy encoder 122 to decode the video bitstream into quantized transform coefficients and required coding information (such as ILPF information, intra prediction information and inter prediction information). Intra prediction 150 on the decoder side does not need to perform mode search. Instead, the decoder only needs to generate intra prediction based on the intra prediction information received from entropy decoder 140. In addition, for inter prediction, the decoder only needs to perform motion compensation (MC 152) based on the inter prediction information received from entropy decoder 140 without motion estimation.
[0008] The VVC standard combines various new coding tools to further improve the coding efficiency relative to the HEVC standard. Among the various new coding tools, some coding tools related to the present invention are summarized as follows.
[0009] Divide the image into CTUs
[0010] The picture is divided into a series of coding tree units (CTUs). The concept of CTU is the same as that of HEVC. For a picture with three sample arrays, a CTU consists of an N×N luma sample block and two corresponding chroma sample blocks. Figure 2 An example of a picture being divided into CTUs is shown, where a thick-line frame 210 corresponds to the picture, and each small rectangle (eg, frame 220 ) corresponds to a CTU.
[0011] The maximum allowed size of a luma block in a CTU is specified as 128x128 (although the maximum size of a luma transform block is 64x64).
[0012] Divide the image into sub-images, fragments, and tiles
[0013] A picture is divided into one or more tile rows and one or more tile columns. A tile is a sequence of CTUs covering a rectangular area of the picture.
[0014] A fragment consists of an integer number of complete tiles or an integer number of consecutive complete CTU rows within a block of a picture.
[0015] Two fragment modes are supported, raster-scan fragment mode and rectangular fragment mode. In raster-scan fragment mode, a fragment contains a sequence of complete tiles in a tile raster scan of a picture. In rectangular fragment mode, a fragment contains multiple complete tiles that together form a rectangular area of a picture or multiple consecutive complete CTU rows that together form a tile of a rectangular area of a picture. Fragments within a rectangular fragment are scanned in fragment raster scan order within the rectangular area corresponding to the fragment.
[0016] A sub-picture consists of one or more fragments that together cover a rectangular area of the picture.
[0017] Figure 3 An example of a raster scan segmentation of a picture 310 is shown, where the picture is divided into 12 tiles 314 (and 3 raster scan segments 316). Each small rectangle 312 corresponds to a CTU.
[0018] Figure 4 An example of a rectangular segment partitioning of a picture 410 is shown, where the picture is partitioned into 24 segments 414 (6 segment columns and 4 segment rows) and 9 rectangular segments 416. Each small rectangle 412 corresponds to a CTU.
[0019] Figure 5 An example of a picture 510 being divided into tiles and rectangular segments is shown, where the picture 510 is divided into 4 tiles 514 (2 tile columns and 2 tile rows) and 4 rectangular segments 516. Each small rectangle 512 corresponds to a CTU.
[0020] Figure 6An example of sub-picture partitioning of a picture 610 is shown, where the picture 610 is divided into 18 tiles 614, with 12 tiles on the left, each covering a fragment of a 4x4 CTU, and 6 tiles on the right, each covering 2 vertically stacked fragments of 2x2 CTUs, resulting in a total of 24 fragments 616 and 24 sub-pictures 616 of different sizes (each fragment is also a sub-picture). Each small rectangle 612 corresponds to a CTU.
[0021] Use tree structure to divide CTU
[0022] In HEVC, CTU is divided into CUs to adapt to various local characteristics by using a quadtree (QT) structure represented as a codec tree. The decision on whether to use inter-picture (temporal) prediction or intra-picture (spatial) prediction to encode and decode a picture area is made at the leaf-CU level. Each leaf-CU can be further divided into one, two or four prediction units (PUs) according to the PU partition type. Within a PU, the same prediction process is applied, and related information is transmitted to the decoder on a PU basis. After obtaining the residual block by applying the prediction process based on the PU partition type, the leaf-CU can be divided into transform units (TUs) according to another quadtree structure similar to the codec tree of the CU. One of the key features of the HEVC structure is that it has multiple partition concepts, including CU, PU, and TU.
[0023] In VVC, a quadtree with nested multi-type trees using binary and ternary partition structures replaces the concept of multiple partition unit types, that is, it eliminates the independence of the concepts of CU, PU, and TU, unless the size of the CU is too large for the maximum transform length, and supports more flexible CU partition shapes. In the codec tree structure, the CU can be a square or a rectangle. The coding tree unit (CTU) is first partitioned according to a quadtree (also known as a quadtree) structure. Then the quadtree leaf nodes can be further divided into multi-type tree structures. For example Figure 7As shown, there are four types of splits in the multi-type tree structure, vertical binary split (SPLIT_BT_VER 710), horizontal binary split (SPLIT_BT_HOR 720), vertical ternary split (SPLIT_TT_VER 730), and horizontal ternary split (SPLIT_TT_HOR 740). The multi-type tree leaf node is called a coding unit (CU for short), and this segmentation will be used for prediction and transform processing without further division unless the CU is too large for the maximum transform length. This means that in most cases, CU, PU, and TU have the same block size in a quadtree with a nested multi-type tree coding block structure. An exception occurs when the maximum supported transform length is less than the width or height of the CU color component.
[0024] Figure 8 The signaling mechanism of partition split information in a quadtree with a nested multi-type tree coding tree structure is shown. The coding tree unit (CTU) is regarded as the root of the quadtree and is first divided by the quadtree structure. Then, each quadtree leaf node (when large enough to allow it) is further divided by the multi-type tree structure. In the multi-type tree structure, a first flag (mtt_split_cu_flag) is sent to indicate whether the node is further divided; when a node is further divided, a second flag (mtt_split_cu_vertical_flag) is sent to indicate the split direction, and then a third flag (mtt_split_cu_binary_flag) is sent to indicate whether the split is a binary split or a ternary split. According to the values of mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag, the multi-type tree split mode (MttSplitMode) of a CU is derived, as shown in Table 1.
[0025] Table 1 - Derivation of MttSplitMode based on multi-type tree syntax elements
[0026] MttSplitMode mtt_split_cu_vertical_flag mtt_split_cu_binary_flag SPLIT_TT_HOR 0 0 SPLIT_BT_HOR 0 1 SPLIT_TT_VER 1 0 SPLIT_BT_VER 1 1
[0027] Fig. 9 The CTU is shown to be partitioned into multiple CUs with a quadtree and nested multi-type tree codec block structure, where the bold block edges represent quadtree partitions and the remaining edges represent multi-type tree partitions. The quadtree with nested multi-type tree partitions provides a content-adaptive codec tree structure consisting of CUs. The size of the CU can be as large as the CTU or as small as 4×4 (in units of luma samples). For the 4:2:0 chroma format, the maximum chroma CB size is 64×64 and the minimum size chroma CB consists of 16 chroma samples.
[0028] In VVC, the maximum supported luma transform size is 64 × 64 and the maximum supported chroma transform size is 32 × 32. When the width or height of the CB is larger than the maximum transform width or height, the CB is automatically split in the horizontal and / or vertical direction to meet the transform size limit in that direction.
[0029] The following parameters are defined and specified by the SPS syntax elements for quadtree with nested multi-type tree codec tree scheme.
[0030] –CTU size: The root node size of the quadtree
[0031] –MinQTSize: Minimum allowed quadtree leaf node size
[0032] –MaxBtSize: Maximum allowed binary tree root node size
[0033] –MaxTtSize: Maximum allowed ternary tree root node size
[0034] –MaxMttDepth: Maximum allowed hierarchical depth of multi-type trees split from the leaves of a quadtree
[0035] –MinBtSize: The minimum allowed binary tree leaf node size
[0036] –MinTtSize: The minimum allowed ternary tree leaf node size
[0037] In one example of a quadtree with a nested multi-type tree coding tree structure, the CTU size is set to 128×128 luma samples and two corresponding 64×64 blocks of 4:2:0 chroma samples, MinQTSize is set to 16×16, MaxBtSize is set to 128×128, MaxTtSize is set to 64×64, MinBtSize and MinTtSize (width and height) are set to 4×4, and MaxMttDepth is set to 4. Tree partitioning is first applied to the CTU to generate quadtree leaf nodes. The size of the quadtree leaf node can range from 16×16 (i.e., MinQTSize) to 128×128 (i.e., CTU size). If the leaf QT node is 128×128, the binary tree will not be further partitioned because the size exceeds MaxBtSize and MaxTtSize (i.e., 64×64). Otherwise, the quadtree leaf node may be further partitioned by the multi-type tree. Therefore, the quadtree leaf node is also the root node of the multitree, and its multitree depth (mttDepth) is 0. When the multitree depth reaches MaxMttDepth (i.e. 4), no further splitting is considered. When the width of the multi-type tree node is equal to MinBtSize and less than or equal to 2*MinTtSize, no further horizontal splitting is considered. Similarly, when the height of the multi-type tree node is equal to MinBtSize and less than or equal to 2*MinTtSize, no further vertical splitting is considered.
[0038] In order to allow 64×64 luma blocks and 32×32 chroma pipeline design in the VVC hardware decoder, TT splitting is prohibited when the width or height of the luma codec block is greater than 64, such as Fig.10 As shown, block 1000 corresponds to a 128x128 luma CU. The CU can be partitioned using vertical binary partitioning (1010) or horizontal binary partitioning (1020). The block is partitioned into 4 CUs, each CU is 64x64 in size, and the CU can be further partitioned using partitions including TT. For example, the top left 64x64 CU is partitioned using vertical ternary partitioning (1030) or horizontal ternary partitioning (1040). TT partitioning is also prohibited when the width or height of the chroma codec block is greater than 32.
[0039] In VVC, the codec tree scheme supports the ability to have separate block tree structures for luma and chroma. For P and B slices, the luma and chroma CTBs in one CTU must share the same codec tree structure. However, for I slices, luma and chroma can have separate block tree structures. When separate block tree mode is applied, the luma CTB is partitioned into CUs by one codec tree structure, and the chroma CTB is partitioned into chroma CUs by another codec tree structure. This means that a CU in an I slice may consist of codec blocks for the luma component or codec blocks for two chroma components, while a CU in a P or B slice always consists of codec blocks for all three color components unless the video is monochrome.
[0040] CU segmentation at the edge of the picture
[0041] As done in HEVC, when part of a tree node block exceeds the bottom or right picture boundary, the tree node block is forced to split until all samples of each coded CU are within the picture boundary. The following splitting rules are applied in VVC:
[0042] – If any part of the tree node block exceeds the bottom or right image boundary, and QT, BT and TT splits are not allowed due to block size restrictions, the block is forced to be divided using QT splitting.
[0043] – Otherwise, if part of the tree node block exceeds the bottom and right picture boundaries,
[0044] If the block is a QT node, and the size of the block is larger than the minimum QT size, the block is forced to be split using the QT split mode.
[0045] Otherwise, the block is forced to be split in SPLIT_BT_HOR mode.
[0046] – Otherwise, if part of the tree node block exceeds the bottom image boundary,
[0047] If the block is a QT node, and the block size is greater than the minimum QT size and the block size is greater than the maximum BT size, the block is forced to be split using the QT split mode.
[0048] Otherwise, if the block is a QT node, and the block size is greater than the minimum QT size and the block size is less than or equal to the maximum BT size, the block is forced to be split using the QT split mode or the SPLIT_BT_HOR mode.
[0049] Otherwise (the block is a BTT node or the block size is less than or equal to the minimum QT size), the block is forced to be split using the SPLIT_BT_HOR mode.
[0050] – Otherwise, if part of the tree node block exceeds the right picture boundary,
[0051] If the block is a QT node, and the block size is greater than the minimum QT size and the block size is greater than the maximum BT size, the block is forced to be split using the QT split mode.
[0052] Otherwise, if the block is a QT node, and the block size is greater than the minimum QT size and the block size is less than or equal to the maximum BT size, the block is forced to be split using the QT split mode or the SPLIT_BT_VER mode.
[0053] Otherwise (the block is a BTT node or the block size is less than or equal to the minimum QT size), the block is forced to be split using the SPLIT_BT_VER mode.
[0054] Restrictions on redundant CU splitting
[0055] The quadtree with nested multi-type tree codec block structure provides a highly flexible block segmentation structure. Due to the support of multiple types of tree segmentation types, different segmentation modes may result in the same codec block structure. In VVC, some of these redundant segmentation modes are not allowed.
[0056] Fig.11 The redundant partitioning patterns of binary tree partitioning and ternary tree partitioning are shown. Fig.11 As shown, two levels of continuous binary partitions in one direction (vertical 1110 and horizontal 1130) can have the same codec block structure as ternary tree partitions (vertical 1120 and horizontal 1140), followed by binary tree partitions of the central partition. In this case, binary tree partitions of the central partition of the ternary tree partition (in a given direction) are prohibited. This restriction applies to CUs in all pictures.
[0057] When segmentation is prohibited as described above, the signaling of the corresponding syntax elements is modified to account for the prohibited situation. Fig.11 When any of the cases is identified (ie, binary split is prohibited for CUs of the center partition), the syntax element mtt_split_cu_binary_flag specifying whether the split is binary or ternary is not sent and is inferred to be equal to 0 by the decoder.
[0058] Virtual Pipeline Data Unit (VPDU)
[0059] A virtual pipeline data unit (VPDU) is defined as a non-overlapping unit in a picture. In a hardware decoder, consecutive VPDUs are processed simultaneously by multiple pipeline stages. In most pipeline levels, the VPDU size is roughly proportional to the buffer size, so it is important to keep the VPDU size small. In most hardware decoders, the VPDU size can be set to the maximum transform block (TB) size. However, in VVC, ternary tree (TT) and binary tree (BT) segmentation may cause the VPDU size to increase.
[0060] In order to keep the VPDU size to 64x64 luma samples, the following specification segmentation restrictions (with syntax signaling modifications) are applied to the VTM, as follows: Fig.12 As shown:
[0061] – For CUs with width or height, or width and height equal to 128, TT partitioning (such as Fig.12 ) are not allowed.
[0062] – For 128xNCU with N≤64 (i.e. width equals 128 and height less than 128), horizontal BT is not allowed.
[0063] For Nx128CUs with N≤64 (i.e., height equal to 128 and width less than 128), vertical BT is not allowed. Fig.12 In the example above, the luma block size is 128x128. The dashed line indicates a block size of 64x64. According to the above restrictions, the segmentation examples that are not allowed are indicated by "X", such as Fig.12 As shown in various examples (1210-1280).
[0064] Intra-frame chroma partitioning and prediction constraints
[0065] In typical hardware video encoders and decoders, the processing throughput drops when a picture has more small intra blocks due to the sample processing data dependency between adjacent intra blocks. The predictor generation for an intra block requires the top and left boundary reconstructed samples from the adjacent blocks. Therefore, intra prediction must be processed continuously block by block.
[0066] In HEVC, the smallest intra CU is 8x8 luma samples. The luma component of the smallest intra CU can be further split into 4 4x4 luma intra prediction units (Pus), but the chroma component of the smallest intra CU cannot be further split. Therefore, the worst-case hardware processing throughput occurs when a 4x4 chroma intra block or a 4x4 luma intra block is processed. In VVC, in order to improve the worst-case throughput, by constraining the division of chroma intra CBs, chroma intra CBs with less than 16 chroma samples (size 2x2, 4x2, and 2x4) and chroma intra CBs with a width less than 4 chroma samples (size 2xN) are not allowed.
[0067] In a single codec tree, the smallest chroma intraprediction unit (SCIPU) is defined as a codec tree node whose chroma block size is greater than or equal to 16 chroma samples and has at least one sub-luminance block less than 64 luminance samples, or is defined as a codec tree node whose chroma block size is not 2xN and has at least one sub-luminance block (with 4xN luminance samples). It is required that in each SCIPU, all CBs are inter, or all CBs are non-inter, that is, intra or intra block copy (IBC). In the case of non-inter SCIPU, it is also required that the chroma of the non-inter SCIPU should not be further split and the luminance of the SCIPU is allowed to be further split. In this way, small chroma intra CBs with a size less than 16 chroma samples or a size of 2xN are removed. In addition, in the case of non-inter SCIPU, chroma scaling is not applied. Here, no additional syntax is sent, and whether the SCIPU is non-inter can be derived from the prediction mode of the first luma CB in the SCIPU. If the current segment is an I-segment, or the current SCIPU has a 4x4 luma partition after further splitting once, the type of the SCIPU is inferred to be non-inter (because inter 4x4 is not allowed in VVC); otherwise, the type of the SCIPU (inter or non-inter) is indicated by a flag before parsing the CU in the SCIPU.
[0068] For dual trees in intra pictures, 2xN intra chroma blocks are removed by disabling vertical binary and vertical ternary partitioning for 4xN and 8xN chroma partitions, respectively. Small chroma blocks of size 2x2, 4x2, and 2x4 are also removed by partition restriction.
[0069] In addition, by considering the picture width and height to be multiples of max(8, MinCbSizeY), the limitation on the picture size is taken into account to avoid 2x2 / 2x4 / 4x2 / 2xN intra chroma blocks at the picture corners.
[0070] CST (Chroma Separate Tree) in VVS
[0071] In VVC, the codec tree scheme supports the ability to have separate block tree structures for luma and chroma. For P and B slices, the luma and chroma CTBs in one CTU must share the same codec tree structure. However, for I slices, luma and chroma can have separate block tree structures. When separate block tree mode is applied, the luma CTB is partitioned into CUs by one codec tree structure, and the chroma CTB is partitioned into chroma CUs by another codec tree structure. This means that a CU in an I slice may consist of codec blocks for the luma component or codec blocks for two chroma components, while a CU in a P or B slice always consists of codec blocks for all three color components unless the video is monochrome.
[0072] In VVC, the luminance and chrominance components use separate segmentation trees to segment the CTU into CUs. Although separate segmentation trees can adapt to different local characteristics between luminance and chrominance components, more codec bits will be required to represent separate segmentation trees. Therefore, it is desirable to improve the codec efficiency of separate segmentation trees. In addition, it is also desirable to apply separate segmentation trees to different chrominance components to improve codec efficiency. Summary of the invention
[0073] A method and apparatus for video encoding are disclosed. According to the method, input data associated with a picture region is received, the picture region including a first color picture region and a second color picture region, wherein the input data includes pixel data of the picture region to be encoded at the encoder side, or codec data associated with the image region to be decoded at the decoding side, wherein the first color picture region is divided into one or more first color blocks according to a first color segmentation tree, and the second color picture region is divided into one or more second color picture regions according to a second color segmentation tree. Using context formation, entropy encoding or decoding is applied to the second color segmentation tree, wherein the context formation includes information related to the first color segmentation tree. The one or more first color blocks and the one or more second color blocks are encoded or decoded.
[0074] In one embodiment, the context formation depends on a quadtree depth or a multi-type tree (MTT) depth associated with the first color segmentation tree, or a block size associated with the one or more first color blocks.
[0075] In one embodiment, the context formation for entropy coding and decoding the segmentation decision of the current second color block depends on the segmentation situation in the corresponding first color block.
[0076] In one embodiment, context formation for entropy encoding and decoding of a segmentation flag associated with a current second color block depends on the block size of the corresponding first color block.
[0077] In one embodiment, the context formation for entropy encoding and decoding the multi-type tree (MTT) vertical flag of the current second color block depends on the size of the corresponding first color block. The size of the corresponding first color block may correspond to the width or height of the corresponding first color block, or to the width and height of the first color block.
[0078] In one embodiment, the first color picture region corresponds to a luma picture region and the second color picture region corresponds to a chroma picture region.
[0079] In one embodiment, the picture region includes a third color picture region, and wherein the first color picture region corresponds to the luma picture region, the second color picture region corresponds to the first chroma picture region, and the third color picture region corresponds to the second chroma picture region. In one embodiment, the third color picture region is partitioned into one or more third color blocks according to a third color segmentation tree that is different from the second color segmentation tree partition.
[0080] In one embodiment, the syntax is sent or parsed at a picture level, a slice level, a tile level, a CTU row level, a CTU level, a VPDU level, or a combination thereof, and wherein the syntax relates to indicating whether a third color segmentation tree is used to partition a third color picture region in a corresponding picture, a slice, a tile, a CTU row, a CTU, or a VPDU, the third color segmentation tree being different from the second color segmentation tree. BRIEF DESCRIPTION OF THE DRAWINGS
[0081] Figure 1A An example adaptive inter / intra video coding system incorporating loop processing is shown.
[0082] Figure 1B Show Figure 1A The corresponding decoder of the encoder in .
[0083] Figure 2 An example in which a picture is divided into CTUs is shown.
[0084] Figure 3 An example of raster scan segmentation of a picture is shown, where the picture is divided into 12 tiles and 3 raster scan segments.
[0085] Figure 4 An example showing a picture divided into tiles and rectangular segments;
[0086] Figure 5An example is shown where a picture is divided into four tiles and four rectangular segments.
[0087] Figure 6 An example is shown in which a picture is divided into 24 sub-pictures.
[0088] Figure 7 Examples of multi-type tree structures corresponding to vertical binary segmentation (SPLIT_BT_VER), horizontal binary segmentation (SPLIT_BT_HOR), vertical ternary segmentation (SPLIT_TT_VER), and horizontal ternary segmentation (SPLIT_TT_HOR) are shown.
[0089] Figure 8 An example of a signaling mechanism for partition split information in a quadtree with a nested multi-type tree codec tree structure is shown.
[0090] Fig. 9 An example of a CTU being partitioned into multiple CUs with quadtree and nested multi-type tree codec block structures is shown, where bold block edges represent quadtree partitions and the remaining edges represent multi-type tree partitions.
[0091] Fig.10 An example of prohibiting TT partitioning when the width or height of the luma codec block is greater than 64 is shown.
[0092] Fig.11 Examples of redundant partitioning patterns of binary tree partitioning and ternary tree partitioning are shown.
[0093] Fig.12 Some examples of disabling TT splitting when the width or height of the luma codec block is greater than 64 are shown.
[0094] Fig.13 An example is shown where luma and chroma may have different trees for inter segments, and it would be more efficient to generate the chroma predictor as the entire parent block.
[0095] Fig.14 A flow chart illustrating an example video coding system using separate partitioning trees for luma and chroma components according to an embodiment of the present invention. DETAILED DESCRIPTION
[0096] It is readily understood that the components of the present invention as generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations. Therefore, the following more detailed description of embodiments of the systems and methods of the present invention, as shown in the figures, is not intended to limit the scope of the claimed invention, but is merely representative of selected embodiments of the present invention. References in this specification to "an embodiment," "some embodiments," or similar language mean that specific features, structures, or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present invention. Therefore, the phrases "in an embodiment" or "in some embodiments" appearing in various places throughout this specification do not necessarily all refer to the same embodiment.
[0097] In addition, the described features, structures or characteristics may be combined in any suitable manner in one or more embodiments. However, those skilled in the art will recognize that the present invention may be implemented without one or more specific details or using other methods, components, etc. In other cases, well-known structures or operations are not shown or described in detail to avoid obscuring various aspects of the present invention. The illustrated embodiments of the present invention will be best understood by reference to the accompanying drawings, in which the same components are represented by the same numbers throughout. The following description is by way of example only, and simply describes some selected embodiments of the apparatus and method consistent with the present invention as claimed herein.
[0098] In the following, techniques for improving codec performance associated with a codec tree are disclosed.
[0099] Method A: Chroma Separate Tree for Cb and Cr
[0100] Instead of a VVC CST where Cb and Cr share the same partition tree, according to an embodiment of the present invention, Cb is allowed to have its own partition tree and Cr is allowed to have its own separate partition tree.
[0101] In order to save partition signaling bits, the partition context probability can be referenced between the Cb tree and the Cr tree to save bits for signaling syntax. For example, if the Cb tree shows a high depth for the partition result, it will use a higher probability to obtain a higher depth syntax for Cr tree partition signal encoding and decoding.
[0102] This method is a content-dependent method. Therefore, it is recommended to turn on / off this method for different pictures, slices, tiles, CTU-rows, CTUs or VPDUs, and provide an on / off control flag for each picture, slice, tile, CTU-row, CTU or VPDU. In other words, the syntax is sent or parsed at the picture level, slice level, tile level, CTU-row level, CTU level, VPDU level or a combination thereof, and the syntax is related to indicating whether a Cr partition tree is used to partition the Cr picture area in the corresponding picture, slice, tile, CTU-row, CTU or VPDU, and the Cr partition tree is different from the Cb partition tree.
[0103] In another embodiment, luma and Cb (or chroma component 1) samples may share one segmentation tree and Cr (or chroma component 2) samples may use another separate segmentation tree.
[0104] In another embodiment, luma and Cr (or chroma component 2) samples may share one partition tree, while Cb (or chroma component 1) samples may use another separate partition tree.
[0105] Method B: CST for inter-based and CU-group-based CCLM
[0106] In this method, CST is applied between segments. The main benefit is that the CCLM part can achieve large codec gain for the chroma part. Fig.13 As shown, according to an embodiment of the present invention, luminance and chrominance may have different trees for inter-frame segments. Fig.13 In FIG. 1 , partition 1310 corresponds to a partition of a luminance image, where the image is divided into various blocks for intra-frame coding 1312 and inter-frame coding 1314. Fig.13 As shown, it is more efficient to generate the chroma predictor as the entire parent block 1320. Therefore, chroma can use a large block to generate CCLM predictor to have better coding gain.
[0107] This method is content-dependent. Therefore, it is recommended to enable / disable this method for different pictures, slices, tiles, CTU-rows, CTUs or VPDUs, and provide a control flag for enabling / disabling this method for each picture, slice, tile, CTU-row, CTU or VPDU.
[0108] Method C: CST of predictors and residuals
[0109] In order to improve the coding gain, it is proposed to separate the partition trees for prediction and residual. That is, starting from the root CU, traverse one partition tree to generate all predictors (such as motion compensation and intra prediction), and traverse another partition tree to generate all residual blocks (such as inverse transform). Finally, all prediction samples and residual samples are added to generate the final reconstructed samples.
[0110] In one sub-embodiment, the CST for predictors and residuals is applied only to all inter-regions. In other words, within the root CU, all predictions are inter predictions.
[0111] This method will improve the codec gain. For the predictor, it will give the best predictor MV inheritance of a dedicated tree. For the residual, it will give the best residual block partitioning of a dedicated tree.
[0112] This method is content-dependent. Therefore, it is recommended to enable / disable this method for different pictures, slices, tiles, CTU-rows, CTUs or VPDUs, and provide a control flag for enabling / disabling this method for each picture, slice, tile, CTU-row, CTU or VPDU.
[0113] Method D: Segmentation probability prediction between CST luminance and chrominance
[0114] In VVC CST, luma and chroma have their own segmentation information (due to separate trees) to perform entropy coding and decoding. For entropy coding and decoding, a relevant probability model needs to be established for each segmentation information signal. For a high-density (high segmentation) luma region, the corresponding chroma region is usually also high-density (although the segmentation tree may be different), and the segmentation depth of the luma tree and the segmentation depth of the chroma tree will be correlated.
[0115] In one embodiment, it is proposed to adjust the segmentation signal probability model of the chrominance region with reference to the segmentation density of the luma tree.
[0116] For example, for the current chroma parent node, if the corresponding spatial luma region has a high segmentation depth (ie, segmented into very small CUs), the probability of a higher depth segmentation syntax will be increased.
[0117] For example, if the corresponding spatial luma region has a low degree of partition depth (not partitioned into very small CUs) for the current chroma parent node, the probability of a higher depth partition syntax will decrease.
[0118] In other embodiments, it is proposed to consider the segmentation of brightness together with the chroma segmentation information for context formation. For example, when forming the context for the segmentation flag of the current chroma parent CU / current CU, the context will also include the corresponding brightness area segmentation, such as quadtree depth, MTT depth, block size, etc. For example, the context formation depends on the quadtree depth or MTT (multi-type tree) depth associated with the brightness segmentation tree, or the block size associated with the one or more brightness blocks. The video codec can allocate different context variables corresponding to different segmentation situations, and determine the selected context variables for entropy encoding and decoding of the segmentation decision of the current chroma block. Therefore, the context formation for entropy encoding the segmentation decision of the current chroma block depends on the segmentation situation in the corresponding brightness block. The processing performed by the codec may depend on the segmentation situation in the corresponding brightness area. In one example, the video codec can select a modeling context for entropy encoding and decoding the CU segmentation flag of the current chroma block according to the size of the corresponding brightness block. In another example, the video codec may select a modeling context for entropy encoding and decoding the MTT vertical flag of the current chrominance block according to the size (e.g., width and height) of the corresponding luminance block. For example, the size of the corresponding luminance block may correspond to the width, height, or width and height of the corresponding luminance block.
[0119] Any of the aforementioned CST (chroma separation tree) methods can be implemented in the encoder and / or decoder. For example, any of the proposed methods can be implemented in the decoder frame (e.g. Figure 1B Intra-frame prediction 150), motion compensation module (e.g. Figure 1B MC 152 in ) or entropy coding and decoding module (e.g. Figure 1B In addition, any of the proposed methods can be implemented in the frame of the encoder (e.g. Figure 1A Intra-frame prediction 110), inter-frame coding module (e.g. Figure 1B inter-frame prediction 112 in) or entropy coding and decoding module (e.g. Figure 1A Alternatively, any of the proposed methods may be implemented as one or more circuits or processors coupled to the inter-frame / intra-frame / prediction / entropy coding / decoding modules of the encoder and / or the inter-frame / intra-frame / prediction / entropy coding / decoding modules of the decoder to provide the information required by the inter-frame / intra-frame / prediction modules.
[0120] Fig.14A flowchart of an exemplary video encoding system using separate segmentation trees for luminance and chrominance components according to an embodiment of the present invention is shown. The steps shown in the flowchart can be implemented as program codes that can be executed on one or more processors (e.g., one or more CPUs) on the encoder side. The steps shown in the flowchart can also be implemented based on hardware, such as one or more electronic devices or processors arranged to perform the steps in the flowchart. According to the method, in step 1410, input data associated with a picture area is received, the picture area includes a first color picture area and a second color picture area, wherein the input data includes pixel data of the picture area to be encoded on the encoder side, or codec data associated with the picture area to be decoded at the decoding end, wherein the first color picture area is divided into one or more first color blocks according to the first color segmentation tree, and the second color picture area is divided into one or more second color blocks according to the second color segmentation tree. In step 1420, entropy coding or decoding is applied to the second color segmentation tree using context formation, wherein the context formation includes information related to the first color segmentation tree. In step 1430, the one or more first color blocks and the one or more second color blocks are encoded or decoded.
[0121] The flowchart shown is intended to illustrate an example of video coding according to the present invention. Those skilled in the art may modify each step, rearrange the steps, split the steps, or combine the steps to implement the present invention without departing from the spirit of the present invention. In the present disclosure, specific syntax and semantics are used to illustrate examples to implement embodiments of the present invention. A technician may implement the present invention by replacing the above syntax and semantics with equivalent syntax and semantics without departing from the spirit of the present invention.
[0122] The above description is presented to enable one of ordinary skill in the art to implement the present invention provided in the context of a specific application and its requirements. Various modifications to the described embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the specific embodiments shown and described, but to conform to the widest scope consistent with the principles and novel features disclosed herein. In the above detailed description, various specific details are described in order to provide a thorough understanding of the present invention. However, it will be understood by those skilled in the art that the present invention may be implemented.
[0123] The embodiments of the present invention as described above can be implemented in various hardware, software codes or a combination of the two. For example, one embodiment of the present invention can be one or more electronic circuits integrated into a video compression chip or integrated into video compression software to perform the program code described herein. The embodiment of the present invention can also be a program code to be executed on a digital signal processor (Digital Signal Processor, referred to as DSP) to perform the processing described herein. The present invention can also be related to many functions performed by a computer processor, a digital signal processor, a microprocessor or a field programmable gate array (field programmable gate array, referred to as FPGA). These processors can be configured to perform specific tasks according to the present invention by executing machine-readable software code or firmware code that defines the specific method embodied in the present invention. The software code or firmware code can be developed in different programming languages and different formats or styles. The software code can also be compiled for different target platforms. However, the different code formats, styles and languages of the software code and other ways of configuring the code to perform tasks according to the present invention will not depart from the spirit and scope of the present invention.
[0124] The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples are to be considered in all respects as illustrative only and not restrictive. Therefore, the scope of the present invention is indicated by the appended claims rather than by the foregoing description. All changes within the equivalent meaning and scope of the claims should be included within their scope.
Claims
1. A video encoding and decoding method for multiple color pictures, the method comprising: Receiving input data associated with a picture region, the picture region comprising a first color picture region and a second color picture region, wherein the input data comprises pixel data of the picture region to be encoded at an encoder side, or codec data associated with the picture region to be decoded at a decoder side, and wherein the first color picture region is divided into one or more first color blocks according to a first color segmentation tree, and the second color picture region is divided into one or more second color blocks according to a second color segmentation tree; applying entropy encoding or decoding to the second color segmentation tree using context formation, wherein the context formation includes information related to the first color segmentation tree; and The one or more first color blocks and the one or more second color blocks are encoded or decoded.
2. The video encoding and decoding method according to claim 1, characterized in that: The context formation depends on a quadtree depth or a multi-type tree depth associated with the first color segmentation tree, or a block size associated with the one or more first color blocks.
3. The video encoding and decoding method according to claim 1, characterized in that: The context formation for entropy coding and decoding the segmentation decision of the current second color block depends on the segmentation situation in the corresponding first color block.
4. The video encoding and decoding method according to claim 1, characterized in that: Said context formation for entropy coding and decoding a segmentation flag associated with a current second color block depends on a block size of the corresponding first color block.
5. The video encoding and decoding method according to claim 1, characterized in that: The context formation for entropy coding and decoding the multi-type tree vertical flag of the current second color block depends on the size of the corresponding first color block.
6. The video encoding and decoding method according to claim 5, characterized in that: The size of the corresponding first color block corresponds to the width or height of the corresponding first color block, or corresponds to the width and the height of the corresponding first color block.
7. The video encoding and decoding method according to claim 1, characterized in that: The first color picture region corresponds to a luma picture region, and the second color picture region corresponds to a chroma picture region.
8. The encoding and decoding method according to claim 1, characterized in that: The picture region includes a third color picture region, wherein the first color picture region corresponds to a luma picture region, the second color picture region corresponds to a first chroma picture region, and the third color picture region corresponds to a second chroma picture region.
9. The encoding and decoding method according to claim 8, characterized in that: The third color picture area is divided into one or more third color blocks according to a third color segmentation tree, and the third color segmentation tree is different from the second color segmentation tree.
10. The encoding and decoding method according to claim 9, characterized in that: The syntax is sent or parsed at a picture level, a slice level, a tile level, a codec tree unit row level, a codec tree unit level, a virtual pipeline data unit level or a combination thereof, and wherein the syntax is related to indicating whether the third color segmentation tree is used to partition the third color picture area in the corresponding picture, slice, tile, codec tree unit row, codec tree unit or virtual pipeline data unit, and the third color segmentation tree is different from the second color segmentation tree.
11. A video coding and decoding apparatus, the apparatus comprising one or more electronic devices or processors, arranged to: receiving input data associated with a picture region, the picture region comprising a first color picture region and a second color picture region, wherein: The input data comprises pixel data of the picture area to be encoded at the encoder side, or codec data associated with the picture area to be decoded at the decoder side, and wherein the first color picture area is divided into one or more first color blocks according to a first color segmentation tree, and the second color picture area is divided into one or more second color blocks according to a second color segmentation tree; applying entropy encoding or decoding to the second color segmentation tree using context formation, wherein the context formation includes information related to the first color segmentation tree; as well as The one or more first color blocks and the one or more second color blocks are encoded or decoded.
