Method and apparatus for video encoding and decoding using partially shared luma and chroma coding trees - Patents.com

A partially shared partitioning tree structure for luminance and chroma blocks addresses the challenge of high coding efficiency and hardware decoding limitations, optimizing video compression by allowing separation trees for smaller blocks and ensuring compatibility with decoding pipelines.

JP7595462B2Active Publication Date: 2024-12-06INTERDIGITAL VC HOLDINGS INC
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
JP2020561859
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2018-05-29
Filing Date
2019-05-21
Publication Date
2024-12-06
Estimated Expiration
2039-05-21

AI Technical Summary

Technical Problem

Existing video compression systems face challenges in achieving high coding efficiency while accommodating hardware decoding pipeline limitations, particularly with the separation of luminance and chroma coding trees in complex partitioning schemes.

Method used

Implement a partially shared partitioning tree structure for luminance and chroma blocks, where a single split tree structure is shared from the root node to a switch node, and a double split tree structure is used from the switch node to the leaf nodes, optimizing coding efficiency by allowing separation trees for smaller blocks and limiting decoding pipeline complexity.

Benefits of technology

This approach enhances coding efficiency by ensuring compatibility with decoding pipelines, such as 64x64 block-based decoding, while maintaining high compression performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007595462000003
    Figure 0007595462000003
  • Figure 0007595462000004
    Figure 0007595462000004
  • Figure 0007595462000005
    Figure 0007595462000005
Patent Text Reader

Abstract

Different implementations are described, and in particular, an implementation for video encoding and decoding based on a partition tree structure partially shared between luma blocks and chroma blocks is presented. According to the implementation, a single partition tree structure is shared between luma blocks and chroma blocks from the root node of the partition tree to the switching node, and a double partition tree structure is determined for luma blocks and chroma blocks from the switching node to the leaf node of the partition tree. Therefore, this implementation optimizes coding efficiency by enabling partition trees for smaller blocks while limiting the complexity of the decoding pipeline. [Selection diagram] Figure 9
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] At least one of the present embodiments generally relates to, for example, a method or apparatus for video encoding or decoding, and more particularly, to a method or apparatus in which a partition tree structure is partially shared between luma blocks and chroma blocks. [Background technology]

[0002] The technical field of one or more implementations generally relates to video compression. At least some embodiments relate to improving compression efficiency compared to existing video compression systems such as HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2, described in "ITU-T H.265 Telecommunication standardization sector of ITU (10 / 2014) Series H: Audiovisual and multimedia systems, infrastructure for audiovisual services - Coding of moving images, High Efficiency Video Coding, Recommendation ITU-T H.265"), or compared to developing video compression systems such as VVC (Versatile Video Coding, a new standard being developed by JVET, Joint Video Experts Team).

[0003] To achieve high compression efficiency, image and video coding schemes usually employ image partitioning, prediction including motion vector prediction, and transformation to exploit spatial and temporal redundancy in video content. In general, intra- or inter-prediction is used to exploit intra- or inter-frame correlation, and then the difference between the original and predicted image, often called prediction error or prediction residual, is transformed, quantized, and entropy coded. To reconstruct the video, the compressed data is decoded by an inverse process corresponding to entropy decoding, inverse quantization, inverse transform, and prediction.

[0004] With the emergence of new video coding schemes, the partitioning schemes have become more complex, allowing dual-tree partitioning / coding of luma and chroma to achieve high compression. However, the dual-tree coding scheme seems to have problems in hardware implementation at the decoding stage.

[0005] Therefore, a new method for coding and decoding video is needed that jointly ensures high coding efficiency while taking into account the constraints of some hardware decoding pipelines. Summary of the Invention

[0006] It is an object of the present invention to overcome at least one of the disadvantages of the prior art. To this end, according to a general aspect of at least one embodiment, a method for video coding is presented, comprising: determining a partitioning tree structure corresponding to a partitioning process of a block of an image of a video, the partitioning tree structure being partially shared between luma blocks and chroma blocks; and encoding the block based on at least the partitioning tree structure.

[0007] According to another general aspect of at least one embodiment, a method for video decoding is presented, the method including determining a partitioning tree structure corresponding to a partitioning process of a block of an image of a video, the partitioning tree structure being partially shared between luma blocks and chroma blocks, and decoding the block based on at least the partitioning tree structure.

[0008] According to another general aspect of at least one embodiment, an apparatus for video encoding is presented, the apparatus including means for performing any one of the embodiments of the encoding method.

[0009] According to another general aspect of at least one embodiment, an apparatus for video decoding is presented, the apparatus including means for performing any one of the embodiments of the decoding method.

[0010] According to another general aspect of at least one embodiment, there is provided an apparatus for video encoding, the apparatus including one or more processors and at least one memory, the one or more processors configured to implement any one of the embodiments of the encoding method.

[0011] According to another general aspect of at least one embodiment, there is provided an apparatus for video decoding, the apparatus including one or more processors and at least one memory, the one or more processors configured to implement any one of the embodiments of the decoding method.

[0012] According to another general aspect of at least one embodiment, a single partition tree structure is shared between luma and chroma blocks from the root node of the partition tree to the switch node, and a double partition tree structure is determined for luma and chroma blocks from the switch node to the leaf node of the partition tree. Advantageously, this embodiment optimizes coding efficiency by allowing partition trees for smaller blocks while limiting the complexity of the decoding pipeline.

[0013] According to another general aspect of at least one embodiment, the partially shared partitioning tree structure is used to code or decode a portion of an intra-coded video image, e.g., a block belonging to a tile or a group of tiles.

[0014] According to another general aspect of at least one embodiment, the partially shared partitioning tree structure is used to code or decode a portion of an inter-coded video image, e.g., a block belonging to a tile or a group of tiles.

[0015] According to another general aspect of at least one embodiment, the switching node is determined implicitly. According to a variant of this embodiment, the switching block node is determined based on a partition block size. Advantageously, the partition block size corresponds to the size of the decoding pipeline unit, for example, set to 64x64. According to another variant of this embodiment, the switching node is determined based on a partition depth level. Advantageously, the partition depth level is set to one depth of a quadtree division of the partition tree, corresponding to a quadtree division of a 128x128 CTU into four sub-CUs of size 64x64, and thus compatible with the size of the decoding pipeline unit.

[0016] The decoding pipeline unit may also be called a Virtual Decoding Pipeline Unit (VPDU).

[0017] According to another general aspect of at least one embodiment, the switching node is signaled. Advantageously, the embodiment can adapt the switching node to each CTU based on, for example, RD cost.

[0018] According to another general aspect of at least one embodiment, a non-transitory computer-readable medium is presented that includes data content generated according to any of the methods or apparatus of the preceding description.

[0019] According to another general aspect of at least one embodiment, a signal is provided, the signal data including video data generated according to any of the methods or apparatuses described above. According to another general aspect of at least one embodiment, the signal data is interleaved based on a partition block size. Thus, the signal data is advantageously arranged for a decoding pipeline unit.

[0020] One or more of the present embodiments also provide a computer readable storage medium having instructions stored thereon for encoding or decoding video data according to any of the methods described above.The present embodiments also provide a computer readable storage medium having a bitstream generated according to the methods described above stored thereon.The present embodiments also provide a method and apparatus for transmitting a bitstream generated according to the methods described above.The present embodiments also provide a computer program product including instructions for performing any of the methods described. [Brief description of the drawings]

[0021] [Figure 1] 1 illustrates an example of the concepts of Coding Tree Unit (CTU) and Coding Tree (CT) for representing a compressed HEVC picture. [Diagram 2] We present an example of a coding unit and associated coding tree division in the Quad-Tree plus Binary-Tree (QTBT) method. [Diagram 3-4] Here are some examples of binary or ternary tree division of CUs: [Figure 5-6] 1 illustrates an example division of a non-limiting embodiment of a CTU in its luma and chroma components, respectively, in accordance with a general aspect of at least one embodiment. [Figure 7-8] 1 illustrates various examples of encoding methods in accordance with a general aspect of at least one embodiment. [Figure 9-12] 1 illustrates various examples of decoding methods in accordance with a general aspect of at least one embodiment. [Figure 13] 1 shows a block diagram of an embodiment of a video encoder in which various aspects of the embodiments may be implemented; [Figure 14] 1 shows a block diagram of an embodiment of a video decoder in which various aspects of the embodiments may be implemented; [Figure 15] 1 shows a block diagram of an exemplary device in which various aspects of the embodiments may be implemented; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] It should be understood that the figures and descriptions are simplified for clarity to illustrate relevant elements for a clear understanding of the principles of the present invention, while excluding many other elements found in a typical encoding and / or decoding device. Although the present specification may use terms such as first, second, etc. to describe various elements, it should be understood that these elements should not be limited by these terms. These terms are used only to distinguish one element from another.

[0023] Various embodiments are described for encoding / decoding an image, which may be applied to encode / decode a portion of an image, such as a slice or tile image, a tile group image, or an entire sequence of images.

[0024] Various methods have been described above, each method including one or more steps or actions for achieving the described method. Insofar as a specific order of steps or actions is not required for the proper operation of the method, the order and / or use of specific steps and / or actions may be modified or combined.

[0025] At least some embodiments relate to methods for encoding or decoding video in which a tree coding / partitioning structure is partially shared between luma and chroma blocks.

[0026] In the HEVC video compression standard, an image is divided into so-called coding tree units (CTUs), whose size is usually 64x64, 128x128 or 256x256 pixels. Each CTU is represented by a coding tree in the compressed domain. This coding tree is a quadtree partition of the CTUs, where each leaf is called a coding unit (CU). Each CU is then given some intra or inter prediction parameters (prediction information). To that end, the CU is spatially divided into one or more Prediction Units (PUs), and each PU is assigned some prediction information. As shown in Figure 1, the intra or inter coding mode is assigned at the CU level.

[0027] Emerging video compression tools include more flexible coded tree unit representations of the compressed domain, which improve compression efficiency compared to the CU / PU / TU arrangement of the HEVC standard. First, the coded tree unit representation includes a quadtree plus binary tree (QTBT), a coding structure that allows a coded unit to be split in both quadtree and binary tree ways. An exemplary coded tree representation of a coded tree unit is shown in Figure 2. The splitting of a coded unit is determined at the encoder side through a rate-distortion optimization procedure, which consists of determining a QTBT representation of a CTU with the lowest rate-distortion cost. In QTBT technology, the shape of a CU is square or rectangular. The size of a coded unit is always a power of 2, typically between 4 and 128.

[0028] In addition to this multiplicity of rectangular coded units, this new CTU representation has the following different properties compared to HEVC: First, the QTBT decomposition of a CTU consists of two stages where the CTU is first split in a quadtree manner, and then each quadtree leaf can be further split in a binary manner. This is shown on the right side of Figure 2, where the solid lines represent the quadtree decomposition phase and the dashed lines represent the binary decomposition spatially embedded in the quadtree leaves.

[0029] Second, in intra slices, the partitioning structures of luma blocks and chroma blocks are separated and determined independently.

[0030] Third, no further partitioning of CUs into prediction units or transform units is adopted. In other words, each coding unit is systematically composed of a single prediction unit (2N×2N prediction unit partition type) and a single transform unit (no partitioning into transform trees).

[0031] Finally, some other CU 2-ary or 3-ary tree partitions can also be adopted for the representation of the coding tree of the CTU, as shown in Figure 3 or Figure 4. Figure 3 shows the respective rectangular sizes

number

number

[0032] Using the new topology described above, the coding efficiency is significantly improved. In particular, a significant gain is obtained in chroma. This large gain in chroma components is largely due to the separation of the coding trees of luma and chroma in the intra slice. However, this separation of the coding trees of luma and chroma for the CTU level has some problems with hardware implementation. In at least one embodiment, the CTU is usually of size 128x128, or 256x256. Furthermore, completely separating the coding trees of the luma and chroma components also means that these luma and chroma are also completely separated in the compressed domain and therefore are represented in a separated manner in the coded bitstream. This causes some problems in at least one decoder implementation, where we want to ensure that the decoding pipeline is achieved with a maximum decoding unit size that may be smaller than the CTU size.

[0033] Typically, a 64x64 based decoder pipeline is desired for at least one decoder implementation. For that purpose, a maximum transform block size equal to 64x64 is selected. In addition, the separation of luma and chroma coding trees is removed since it implies a coded bitstream in which all luma blocks of a CTU appear before the chroma blocks of the same CTU. In such a case, the decoding process leads to parsing, thus resulting in decoded quantization block data that spatially covers the entire CTU, i.e., 128x128 or 256x256 picture area. The removal of the luma / chroma coding separation results in a reduced coding efficiency compared to the case in which these trees are selected, coded and decoded independently.

[0034] It is therefore desirable to have a coding tree representation of luma and chroma in a way that optimizes coding efficiency while ensuring that a decoding pipeline, for example a 64x64 block-based decoding pipeline, is feasible.

[0035] To that end, at least one embodiment comprises: -Having a joint luma / chroma coding tree from the maximum allowed block size (CTU size), determined according to the pipeline decoding unit, down to a predefined block size or predefined CU depth level. Then, when the block size is equal to or smaller than the pipeline decoding unit, the luma and chroma coding trees are selected, encoded, and decoded separately. This means that a part of the entire CTU coding tree is shared by the luma and chroma components, and then, beyond a certain depth level of the CTU coding tree, the luma and chroma trees are independent and therefore may be different. Hereinafter, the term partially shared luma / chroma coding tree is used to refer to this aspect. - Switching from an interleaved luma / chroma coding process to a separated luma / chroma coding process according to the block size associated with the processed coding tree node or according to the depth level of the coding tree node being encoded / decoded.

[0036] In practice, when encoding the luma and chroma coding trees, assuming a 64x64 decoding pipeline unit, the coding trees are jointly determined and coded for block sizes larger than 64x64. Then, for block sizes smaller than 64x64, the luma / chroma coding trees may diverge from each other.

[0037] Advantageously, this process also ensures that in the bitstream configuration, for each picture area equal to the considered decoding pipeline unit (64x64), luma and chroma compressed signals appear one after the other. Thus, when parsing the bitstream, it is guaranteed that the block residual signal associated with the picture area equal to the pipeline decoding unit 64x64 is fully decoded for both luma and chroma components before starting the process of the next pipeline decoding unit.

[0038] Moreover, the coding efficiency is improved compared to a completely separate luma / chroma coding tree.

[0039] In Section 1, several embodiments of partially shared luma / chroma coding trees and associated bitstream structures are disclosed.

[0040] In section 2, several embodiments of the encoding method are disclosed.

[0041] In sections 3 and 4, several embodiments of the decoding method are disclosed.

[0042] In Section 5, additional information and general embodiments are disclosed.

[0043] 1. An embodiment of a partially shared luma / chroma coding tree and associated bitstream configuration 5 and 6 show non-limiting example embodiment divisions of a CTU in luma and chroma components, respectively, according to a general aspect of at least one embodiment. In both examples, the CTU size considered is 128×128, and the decoding pipeline unit considered is 64×64.

[0044] FIG. 5 shows a first example where a CTU is first split in a quadtree fashion. This results in a 64×64 coded unit in the luma component, as shown on the left side of the figure, which is aligned with a corresponding 32×32 CU in the chroma component, as shown on the right side of the figure. As shown in the figure, when the size of the CU to be split is larger than the size of the decoding unit, 64×64, the two luma and chroma split types are identical. Then, for luma CUs smaller than or equal to 64×64, the split mode may be different for luma and chroma. This is the case in FIG. 5, except for the 64×64 CUs (3 and 4) in the top right corner.

[0045] At the bottom of Fig. 5, the bitstream organization proposed herein is shown for the luma and chroma coding trees of Fig. 5. As shown, according to an embodiment of the bitstream arrangement, the bitstream is organized in consecutive parts, each of which includes a sequence of coded luma blocks and a sequence of coded chroma blocks. The bitstream is organized such that the sequence of coded luma or chroma blocks corresponds to a spatial region of 64x64 (in the luma domain) or less, i.e., the luma size imposed on the decoding pipeline unit.

[0046] In a particular embodiment, the bitstream portion containing a given sequence of coded luma or chroma blocks contains exactly the coded data of the spatial region of the picture under consideration and its size is equal to the size of the decoding pipeline unit under consideration, which is typically 64x64.

[0047] 6 shows another embodiment of a partially shared luma / chroma coding tree. The 128x128 first undergoes a vertical bisection split, resulting in two 64x128 coded units. For CU sizes larger than 64x64, the CTU is split in the same way in the chroma component, resulting in two 64x32 CUs, since luma and chroma are synchronized. The CU on the right is not split any more. The 128x64 on the left is split in a bisection horizontal split mode, synchronously in the luma and chroma components (the shared / synchronized part of the coding tree arrangement, according to at least one embodiment).

[0048] Furthermore, for the CU resulting from this second bipartite split stage, since it has a luma of size 64x64, it can be further split in a separated manner between luma and chroma components. Finally, the resulting bitstream arrangement and ordering is shown at the bottom of FIG. 6. A particular aspect to note here is that since the size of the maximum decoding unit is 64x64, the maximum transform size is also 64x64. Thus, for the 64x128 CU in FIG. 6, the CU is split into two transform units 64x64 in the luma component. Synchronously, it is split into two 32x32 CUs in the two chroma components. Then, in terms of bitstream ordering, the 64x64 luma transform block and the 32x32 chroma transform block resulting from this spatial division are interleaved in the bitstream to ensure that all luma and chroma coding data corresponding to the spatial region of size 64x64 are packed together in the coded bitstream. This corresponds to coded data blocks 5, 6, 7, and 8 in FIG. 6.

[0049] 2 At least one embodiment of the encoding FIG. 7 illustrates an example of an encoding method 700 in accordance with a general aspect of at least one embodiment.

[0050] In a preliminary step 710, a block is accessed for coding. A block, typically a CU, belongs to a spatial division of an image of a video, usually a CTU from the previous section. The coding method 700 recursively determines a division or split into sub-blocks for the block to be coded. Typically, the method starts with a block that is a CTU from the previous section. For example, the block is split into sub-blocks according to any of the split modes described in Fig. 2, Fig. 3 or Fig. 4. The split tree structure, also called coding tree or split information, indicates how the image is split. Thus, in step 720, a split tree structure corresponding to the split process of the block is determined. Advantageously, the split tree structure is partially shared between luma and chroma blocks. A split tree structure partially shared between luma and chroma blocks means that for some blocks of the split tree, a single split tree structure is used for both luma and chroma blocks, while for other blocks of the split tree, a separate split tree structure is used for luma and chroma blocks. For example, the luma block corresponding to the luma component of the CTU in FIG. 5 shares a first quadtree split of the partition tree structure with a chroma block corresponding to any of the chroma components of the CTU. Next, luma block 1 corresponding to the luma component of CU1 in FIG. 5 and chroma block 2 corresponding to any of the chroma components of CU2 are split in a different manner, and thus a dual partition tree structure is determined for luma block 1 and for the co-located chroma block 2. In the following, the terms separation tree and dual tree are used indifferently to define the two trees, while the terms common tree, shared tree, joint tree, and single tree are used indifferently to define the common tree for both the luma component and the chroma component. In step 730, the block is coded based on at least the determined partition tree structure.

[0051] According to the embodiment, a single partition tree structure is shared between luma and chroma blocks from the root node of the partition tree to the switching node, and a dual partition tree structure is determined for luma and chroma blocks from the switching node to the leaf node of the partition tree. Thus, the shared coding tree is used for larger blocks, while the split tree is used for smaller blocks, and the switching between the shared coding tree and the dual coding tree is defined by a switching point or switching level. For example, the root node corresponds to a CTU, while the leaf node corresponds to a leaf CU. Advantageously, the embodiment optimizes the coding efficiency by allowing split trees for some blocks while limiting the complexity of the decoding pipeline.

[0052] According to an embodiment, the partially shared partitioning tree structure is used to code blocks that belong to a portion of an intra-coded video image, for example the portion of the video image is a tile or a group of tiles, and the partially shared luma and chroma coding trees are used only for the intra-tile or the group of intra-tiles.

[0053] According to another embodiment, a partially shared partitioning tree structure is used for coding blocks that belong to a portion of an inter-coded video image.

[0054] According to an embodiment, the switching node is determined implicitly. Advantageously, this embodiment avoids signaling the switching node from the encoder to the decoder. In a first variant, the switching block node is determined based on the partition block size. The partition block size is therefore a parameter predefined in the encoder and in the decoder. For example, the switching from a shared luma / chroma coding tree to two separated luma and chroma sub-trees is determined based on the size of the coding unit, for the size of the decoding unit under consideration. According to a preferred variant, the partition block size is set to 64x64, thus enabling a decoding pipeline on a 64x64 block basis. As shown in FIG. 5, CTU128x128 shares the coding tree for both chroma and luma components, while for CUs of size 64x64 or less, the sub-trees are separated for chroma and luma components.

[0055] In a second variant, the switching block node is determined based on the partition depth level. Thus, the partition block size is a parameter predefined in the encoder and the decoder. According to a preferred embodiment, the partition depth level is set to the depth of one of the quadtree divisions of the partition tree. As shown in FIG. 5, the shared tree of the CTU includes a first quadtree division, and the lower subtree is separated for the CU. For example, the switching from the shared luma / chroma coding tree to the two separated luma and chroma subtrees is determined based on the depth level of the coding unit, relative to the depth threshold of the CU. In a particular variant, the first quadtree division is inferred. This variant is well adapted to a CTU size of 128x128, where the first QT division generates four CUs of size 64x64, and then the double tree is processed for luma and chroma components. In other words, the switching from the shared luma / chroma coding tree to the two separated luma and chroma subtrees is determined based only on the quadtree depth level, relative to the quadtree depth threshold of the CU. Indeed, in the coding structure of QTBT, a CTU first undergoes a quadtree split process before entering the so-called binary / ternary tree split process. Thus, two types of CU depth levels can be considered: quadtree-related depth levels and binary / ternary tree depth levels. The binary / ternary tree level of a CU that undergoes a quadtree split is equal to 0, and the quadtree split leads to four sub-CUs, and the quadtree-related depth level increases by 1 compared to the parent CU.

[0056] In a third variant, the switching block node is also determined based on the split depth level, where a composite quadtree / binary tree / ternary tree depth level is associated with each CU. This composite depth is exemplarily calculated as follows: if the CU is a quadtree split, the composite depth increases by 3 between the parent CU and the child CU; if the CU is a binary tree split, the composite depth increases by 2 between the parent CU and the child CU; if the CU is a ternary tree split, the composite depth increases by 1 between the parent CU and the middle child CU, and increases by 3 between the parent CU and the other two (not middle) child CUs. In the scope of this embodiment, the switching between the dual luma / chroma coding tree and the split tree is determined according to the composite depth level of the current CU relative to the composite depth threshold.

[0057] According to an embodiment, the switching node is signaled. Advantageously, the embodiment allows adapting the switching node to each CTU, for example based on the RD cost.

[0058] FIG. 8 illustrates another example of an encoding method according to at least one embodiment. FIG. 8 illustrates an embodiment of a partially shared luma and chroma coding tree determination step 720 for a given CTU in an encoder. The embodiment of FIG. 8 corresponds to an embodiment in which a switching node is determined based on a split block size. This decision process includes a recursive rate-distortion optimization procedure to search for two luma and chroma coding trees that jointly minimize the overall rate-distortion cost at the CTU level. To that end, at each depth level of the recursive process, a set (or a part of it) of candidate split modes for the considered coding unit (corresponding to the entire CTU at the start of the process) is assumed. Two main cases are illustrated in FIG. 8 on the left side (yes branch) and on the right side (no branch) of the figure, respectively.

[0059] The left side corresponds to the case where the size of the current CU is strictly larger than the size of the decoding pipeline unit. In that case, according to at least one embodiment, a single split mode commonly used for luma and chroma components must be selected. To do so, each split mode s or CurrentSplitMode in the set of allowed candidate split modes for the current CU is evaluated, which results in a rate-distortion cost RDcost_YCbCr(CurrentSplitMode) associated with the best coding of the considered CU that can be made when selecting the split model s at the current depth level. Thus, the evaluation of the split mode s involves splitting the current CU according to the split mode s, and then looping over each resulting sub-coded unit (sub-CU). For each sub-CU, the current RDO process is recursively called such that the split mode of the considered sub-CU is determined. This loop over the candidate split modes then leads to the determination of the split mode that minimizes the joint luma / chroma rate-distortion cost associated with the current CU.

[0060] For a 128x128 CTU, the quad-tree split operation splits the 128x128 CTU into four 64x64 coded units. This split is signaled or inferred as part of a general chroma / luminance coding tree to save bits. Note that in the case where the quad-tree split mode is inferred for a CU larger than the decoding pipeline unit size 64x64, the left side of the process in FIG. 8 simply consists of splitting the input CU into four sub-CUs and recursively applying the process in FIG. 8 to each resulting sub-CU in turn.

[0061] Regarding the right side of the algorithm in Fig. 8, the entire RD search process corresponds to the case where a coding tree depth level is reached, which corresponds to a coding unit size equal to or smaller than the size of the decoding pipeline unit under consideration, i.e., a switching node. In that case, at least one embodiment proposes to switch to luma and chroma sub-coding trees separated from the joint luma / chroma coding tree from the considered depth level in the entire coding tree representation being optimized. Thus, the determination of the luma and chroma coding trees associated with the current CU takes the form of two separate successive optimization procedures to find the best luma coding tree and the best chroma coding tree for the current CU, respectively.

[0062] This corresponds to the two RDO steps following in FIG. 8. The optimization steps are known from the skilled person. The next step involves calculating the overall joint luma / chroma rate-distortion cost RDcost_YCbCr(s_luma, s_chroma) resulting from the obtained separated luma and chroma sub-coding trees. s_luma and s_chroma represent the best split mode found for the current CU in luma (Y) and chroma (Cb+Cr) components, respectively. This joint RD cost can be calculated simply by summing the rates of the luma and chroma components (giving an overall rate R) and calculating the overall distortion D as the sum of the luma and chroma distortions resulting from the two separate coding tree searches. The overall RD cost is then obtained according to the classical formula, where λ is the known Lagrangian parameter. Rdcost=D+λR

[0063] Note that in the case where the quadtree split mode is inferred for a CU larger than the size of the decoding pipeline unit 64×64, the rate-distortion optimal split modes s_luminance and s_chroma are assigned to the current CU, currCU, in the luma and chroma channels, respectively. In fact, for CUs larger than the size of the decoding pipeline unit, there is no rate-distortion conflict between the split modes.

[0064] The first module / operation in FIG. 8 (whether there is a possibility of sharing luma / chroma coding trees) can be located, for example, in module 102 (image segmentation) in FIG. 13 (shown later).

[0065] 3. Decoding embodiment FIG. 9 illustrates an example of a decoding method 900 in accordance with a general aspect of at least one embodiment.

[0066] In a preliminary step 910, a bitstream or signal of a coded block is accessed for decoding. The coded block is typically a CU or a CTU. The decoding method 900 recursively determines a division or split into sub-blocks for the block to be decoded. For example, the coded block is split into sub-blocks according to any of the split modes described in FIG. 2, FIG. 3 or FIG. 4. Thus, in step 920, a split tree structure corresponding to the split process of the block is determined. Advantageously, the split tree structure is partially shared between the luma block and the chroma block. For example, the split tree structure is obtained from the decoding process and syntax elements associated with the coded block. This is explained later in the variant embodiment examples of FIG. 10 to FIG. 13. In step 930, the block is decoded based at least on the determined split tree structure.

[0067] Those skilled in the art will easily adapt the example variants described for the encoding method to the decoding method 900 which will not be repeated here.

[0068] FIG. 10 illustrates another example of a decoding method according to a general aspect of at least one embodiment. FIG. 10 illustrates an embodiment of a step 920 of determining the partially shared luma and chroma coding trees of a given CTU in a decoder. The embodiment of FIG. 10 corresponds to an embodiment in which the switching node is determined based on the split block size. As for encoding, the process also consists of two parts, respectively shown on the left and right sides of the diagram of FIG. 10. The left side of FIG. 10 illustrates the processing of a CU whose width or height is larger than the size of the considered decoding pipeline unit. The right side of FIG. 10 illustrates the decoding of a coding unit whose size is equal to or smaller than the size of the considered decoding pipeline unit. Thus, the left side refers to a recursive analysis of the split mode, and if in step 921 the size of the CU is larger than the size of the decoding unit, the decoding of the CU is possible. The decoding 921 of such a larger CU is illustrated in FIG. 11. According to a variant, the quad-tree split operation divides the CU into sub-blocks equal to the DPU size (for example, for a 128x128 CTU, the quad-tree split operation divides the 128x128 CTU into four 64x64 coding units).

[0069] According to another variant, the quadtree splitting mode may be inferred for CUs larger than the DPU size on the encoder side.

[0070] With respect to the right hand side of FIG. 10, this involves the analysis, decoding and reconstruction of the coded units contained in the picture region under consideration, successively and independently, for the luma component and then for the chroma component.

[0071] The modules / operations of Figures 10 and 11 may, for example, be located between modules 230 (entropy decoding including parsing syntax elements) and 235 (segmentation) of Figure 14 (described below).

[0072] 4 Alternative Decoding Embodiments This section presents an alternative embodiment of the decoding process of Section 3. The difference here is that the block decoding process is done on a decoding pipeline basis in order to explicitly comply with the constraints taken into account in the decoding process. In fact, a decoder compatible with an embodiment of the present principles can advantageously perform all the decoding processes of a decoding unit for luma and chroma components before starting the process of the next decoding unit.

[0073] 12 shows a decoding unit-based process used to decode and reconstruct luma and chroma blocks according to an embodiment of the present invention, which first spatially divides the CTU into decoding units of size equal to the decoding pipeline unit, typically 64×64.

[0074] For example, this division is performed by a quadtree split operation, which divides a 128x128 CTU into four 64x64 coded units.

[0075] This process involves a loop over all decoding units (of typical size 64×64) contained in the CTU under consideration. For each decoding unit, the luma blocks contained in the decoding unit are first decoded, then the chroma blocks contained in the decoding unit are decoded and reconstructed.

[0076] 5 Additional Embodiments and Information This section describes a wide variety of aspects, including tools, features, embodiments, models, approaches, and the like. Many of these aspects are described specifically and often in a manner that may sound limiting, at least to indicate their individual characteristics. However, this is for clarity of description and does not limit the application or scope of these aspects. Indeed, all of the various aspects can be combined and interchanged to provide further aspects. Additionally, these aspects can also be combined and interchanged with aspects described in previous applications.

[0077] The aspects described and contemplated in this document can be implemented in many different forms. Figures 13, 14, and 15 below provide some embodiments, but other embodiments are contemplated, and discussion of Figures 13, 14, and 15 is not intended to limit the breadth of implementations. At least one of these aspects generally relates to video coding and decoding, and at least one other aspect generally relates to transmitting a generated or coded bitstream. These and other aspects can be implemented as a method, an apparatus, a computer-readable storage medium having stored thereon instructions for coding or decoding video data according to any of the described methods, and / or a computer-readable storage medium having stored thereon a bitstream generated according to any of the described methods.

[0078] In this application, the terms "reconstructed" and "decoded" may be used interchangeably, the terms "pixel" and "sample" may be used interchangeably, and the terms "image", "picture" and "frame" may be used interchangeably. Usually, but not necessarily, the term "reconstructed" is used on the encoder side, while "decoded" is used on the decoder side.

[0079] Various methods have been described above, each method including one or more steps or actions for achieving the described method. Insofar as a specific order of steps or actions is not required for the proper operation of the method, the order and / or use of specific steps and / or actions may be modified or combined.

[0080] Various methods and other aspects described herein can be used to modify modules such as, for example, the splitting module (102, 235) and the entropy decoding module (230) of the JVET or HEVC encoder 100 and decoder 200, as shown in Figures 13 and 14. Moreover, the aspects are not limited to JVET or HEVC, but can be applied to, for example, other standards and recommendations, whether existing or developed in the future, and extensions of such standards and recommendations (including JVET and HEVC). Unless otherwise specified or technically excluded, the aspects described herein can be used individually or in combination.

[0081] In this document, various numerical values ​​are used, such as, for example, a 64x64 decoding unit pipeline or block division (CU, TU) size, etc. The specific values ​​are for illustrative purposes, and the described aspects are not limited to these specific values.

[0082] 13 illustrates an exemplary encoder 100. Although variations of this encoder 100 are contemplated, the encoder 100 is described below for clarity without describing all possible variations.

[0083] Before being coded, a video sequence may undergo a pre-coding process (101), such as applying a color transformation to the input color picture (e.g., from RGB 4:4:4 to YCbCr 4:2:0) or performing a remapping of the input picture components to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and may be attached to the bitstream.

[0084] In the exemplary encoder 100, a picture is coded by the encoder elements as described below. The picture to be coded is divided (102) and processed, for example, in units of CUs. Each unit is coded, for example, using either intra mode or inter mode. When a unit is coded in intra mode, intra prediction is performed (160). In inter mode, motion estimation (175) and motion compensation (170) are performed. The encoder decides (105) whether intra mode or inter mode is used to code the unit, and indicates the intra / inter decision, for example, by a prediction mode flag. A prediction residual is calculated, for example, by subtracting (110) the predicted block from the original image block.

[0085] The prediction residual is then transformed (125) and quantized (130). The quantized transform coefficients, plus motion vectors and other syntax elements, are entropy coded (145) to output a bitstream. The encoder may skip the transform and apply quantization directly to the untransformed residual signal. The encoder may also bypass both the transform and quantization, i.e., the residual is directly coded without applying a transform or quantization process.

[0086] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual is combined (155) with the predicted block to reconstruct an image block. An in-loop filter (165) is applied to the reconstructed picture, for example to perform deblocking / SAO (sample adaptive offset) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer (180).

[0087] Figure 14 shows a block diagram of an exemplary video decoder 200. In the exemplary decoder 200, the bitstream is decoded by a decoder element as described below. The video decoder 200 generally performs a decoding pass that is the inverse of the encoding pass as described in Figure 13. The encoder 100 also generally performs video decoding as part of coding the video data.

[0088] In particular, the input of the decoder includes a video bitstream, which may be generated by the video encoder 100. First, the bitstream is entropy decoded (230) to obtain transform coefficients, motion vectors, and other coded information. Picture partition information indicates how the picture is partitioned. Thus, the decoder can partition the picture according to the decoded picture partition information (235). The transform coefficients are dequantized (240) and inverse transformed (250), and the prediction residual is decoded. The decoded prediction residual is combined with a predicted block (255) to reconstruct an image block. The predicted block may be obtained from intra prediction (260) or motion compensated prediction (i.e., inter prediction) (275) (270). An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (280).

[0089] The decoded picture may further undergo post-decoding processing (285), such as an inverse color conversion (e.g., converting from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping that performs the inverse of the remapping process performed in the pre-coding process (101). The post-decoding process may use metadata derived in the pre-coding process and signaled in the bitstream.

[0090] FIG. 15 illustrates a block diagram of an exemplary system in which various aspects and exemplary embodiments are implemented. System 1000 may be embodied as a device including various components described below and configured to perform one or more of the aspects described in this document. Examples of such devices include, but are not limited to, personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. System 1000 may be communicatively coupled to other similar systems and to displays via communication channels as shown in FIG. 15 and known to those skilled in the art, to implement various aspects described in this document.

[0091] The system 1000 may include at least one processor 1010 configured to execute instructions loaded therein to implement various aspects described herein. The processor 1010 may include embedded memory, input / output interfaces, and various other circuits as known in the art. The system 1000 may include at least one memory 1020 (e.g., volatile memory device, non-volatile memory device). The system 1000 may include a storage device 1040, which may include non-volatile memory including, but not limited to, EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash, magnetic disk drive, and / or optical disk drive. The storage device 1040 may include, as non-limiting examples, an internal storage device, an attached storage device, and / or a network-accessible storage device. The system 1000 may include an encoder / decoder module 1030 configured to process data to provide encoded or decoded video, which may include its own processor and memory.

[0092] The encoder / decoder module 1030 represents a module or modules that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of a coding and decoding module. Furthermore, the encoder / decoder module 1030 may be implemented as a separate element of the system 1000 or may be incorporated within the processor 1010 as a combination of hardware and software as known to those skilled in the art.

[0093] Program code that is loaded onto the processor 1010 to perform various aspects described herein may be stored in the storage device 1040 and then loaded onto the memory 1020 for execution by the processor 1010. According to an example embodiment, one or more of the processor(s) 1010, the memory 1020, the storage device 1040, and the encoder / decoder module 1030 may store one or more of a variety of items, including, but not limited to, input video, decoded video, bitstreams, equations, formulas, matrices, variables, operations, and computational logic, during execution of the processes described herein.

[0094] The system 1000 may include a communication interface 1050 that enables communication with other devices over a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to transmit and receive data from the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or a network card, and the communication channel may be implemented in a wired and / or wireless medium. The various components of the system 1000 may be connected or communicatively coupled to one another using a variety of suitable connections, including, but not limited to, an internal bus, wires, and printed circuit boards.

[0095] Exemplary embodiments may be implemented by computer software implemented by the processor 1010, or by hardware, or by a combination of hardware and software. As a non-limiting example, exemplary embodiments may be implemented by one or more integrated circuits. The memory 1020 may be of any type suitable for the technology environment, and may be implemented using any suitable data storage technology, such as, as non-limiting examples, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memories, and removable memories. The processor 1010 may be of any type suitable for the technology environment, and may include, as non-limiting examples, one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0096] The implementations and aspects described herein may be implemented, for example, in a method or process, an apparatus, a software program, a data stream, or a signal. Even if discussed only in the context of a single implementation (e.g., discussed only as a method), the implementation of the discussed features may also be implemented in other forms (e.g., an apparatus or a program). An apparatus may be implemented, for example, in appropriate hardware, software, and firmware. A method may be implemented in an apparatus such as, for example, a processor, which generally refers to a processing device including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, computers, mobile phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate the transfer of information between end users.

[0097] References to "one embodiment" or "an embodiment" or "one implementation" or "an implementation," as well as other variations thereof, mean that a particular feature, structure, characteristic, etc. described in connection with that embodiment is included in at least one embodiment. Thus, appearances of the phrases "in one embodiment" or "in an embodiment" or "in one implementation" or "in an implementation" in various places throughout this document, as well as any other variations, are not necessarily all referring to the same embodiment.

[0098] Additionally, this document may refer to "determining" various portions of information. Determining the information may include, for example, one or more of estimating the information, calculating the information, predicting the information, or retrieving the information from a memory.

[0099] Additionally, this document may refer to "accessing" various portions of information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or inferring information.

[0100] Additionally, this document may refer to "receiving" various portions of information. Receiving, like "accessing," is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Additionally, "receiving" is typically included in some manner, such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0101] Additionally, this document may refer to optimization, and various embodiments need not be fully optimized; some embodiments simply select a mode that is not optimized in the normal sense.

[0102] As will be apparent to one of ordinary skill in the art, implementations can generate a wide variety of signals formatted to carry information that can be, for example, stored or transmitted. Information can include, for example, instructions for performing a method or data generated by one of the described implementations. For example, a signal may be formatted to carry a bit stream of the described embodiments. Such a signal can be formatted, for example, as an electromagnetic wave (e.g., using a radio frequency portion of the spectrum) or as a baseband signal. Formatting can include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information that the signal carries can be, for example, analog or digital information. The signal can be transmitted over a variety of different wired or wireless links, as is known. The signal can be stored on a processor-readable medium.

[0103] Having described several embodiments, these embodiments provide at least the following generalized invention and claims, including all combinations thereof, across a variety of different claim categories and types. Modifying the luma and chroma coding trees applied to the decoder and / or encoder. Combining interleaved luma / chroma coding with separate luma / chroma coding applied in the decoder and / or encoder. Combining interleaved luma / chroma coding with separate luma / chroma coding, where the separate luma / chroma coding is controlled by information about the constraints of the decoding unit pipeline applied to the decoder and / or encoder. · Jointly determining, encoding, and signaling luma and chroma coding trees from a maximum allowable block size (CTU size) down to a given block size or depth level for a given CU, and independently determining, encoding, and signaling luma and chroma coding trees for blocks whose size is smaller than the given block size or exceeds the depth level for a given CU. Switching from interleaved luma / chroma coding to separated luma / chroma coding according to the block size associated with the processed coding tree node or according to the depth level of the coding tree node being encoded / decoded. · Allowing some partially shared luma and chroma coding tree methods in the decoder and / or encoder. Selecting which luma and chroma coding trees to apply at the decoder based on some syntax elements. Creating and / or transmitting and / or receiving and / or decoding a bitstream or signal that includes one or more of the described encoded luma / chroma components or variations thereof. A television, set-top box, mobile phone, tablet, or other electronic device that performs encoding / decoding to enable a partially shared luma and chroma coding tree method according to any of the described embodiments. A television, set-top box, mobile phone, tablet, or other electronic device that performs the encoding / decoding enabling the partially shared luma and chroma coding tree method according to any of the described embodiments and displays the resulting image (e.g., using a monitor, screen, or other type of display device). A television, set-top box, mobile phone, tablet, or other electronic device that tunes to a channel (e.g., using a tuner) to receive a signal including an encoded image and performs decoding to enable a partially shared luma and chroma coding tree method according to any of the described embodiments. A television, set-top box, mobile phone, tablet, or other electronic device that receives a signal containing a coded image over the air (e.g., using an antenna) and performs decoding to enable a partially shared luma and chroma coding tree method according to any of the described embodiments.

[0104] Various other generalized and specific inventions and claims are also supported and contemplated throughout this disclosure.

Claims

1. 1. A method for decoding a video, comprising: decoding a partition tree structure corresponding to a partition process of blocks of images of said video; and decoding the block based on at least the partition tree structure; the partition tree structure has a switch node indicating a partition block size, such that above the switch node, the luma blocks and the chroma blocks share a single partition tree structure, and at or below the switch node, the luma blocks and the chroma blocks have separate partition tree structures; The method of claim 1, wherein decoding a partition tree structure at or below the switch node includes decoding at least an indication that a chroma block is further partitioned.

2. 1. A method for encoding video, comprising: determining a partitioning tree structure corresponding to a partitioning process of blocks of images of said video; encoding the block based on at least the partitioning tree structure; the partition tree structure has a switch node indicating a partition block size, such that above the switch node, the luma blocks and the chroma blocks share a single partition tree structure, and at or below the switch node, the luma blocks and the chroma blocks have separate partition tree structures; The method of claim 1, wherein encoding a partition tree structure at or below the switch node includes encoding at least an indication that a chroma block is further partitioned.

3. 1. A video decoding apparatus comprising one or more processors, the one or more processors comprising: decoding a partition tree structure corresponding to a partition process of blocks of images of said video; and decoding the block based on at least the partition tree structure; the partition tree structure has a switch node indicating a partition block size, such that above the switch node, the luma blocks and the chroma blocks share a single partition tree structure, and at or below the switch node, the luma blocks and the chroma blocks have separate partition tree structures; The video decoding apparatus, wherein decoding the partition tree structure at or below the switch node includes decoding at least an indication that a chroma block is further partitioned.

4. 1. A video encoding apparatus comprising one or more processors, the one or more processors comprising: determining a partitioning tree structure corresponding to a partitioning process of blocks of images of said video; and encoding the block based on at least the partitioning tree structure; the partition tree structure has a switch node indicating a partition block size, such that above the switch node, the luma blocks and the chroma blocks share a single partition tree structure, and at or below the switch node, the luma blocks and the chroma blocks have separate partition tree structures; 11. A video encoding apparatus, wherein encoding a partitioning tree structure at or below the switch node includes encoding at least an indication that a chroma block is further partitioned.

5. The method of claim 1 , wherein the partitioning tree structure is used to decode blocks that belong to a portion of the video image that is intra-coded.

6. The method of claim 2 , wherein the partitioning tree structure is used to code blocks that belong to a portion of the video image that is intra-coded.

7. The method of claim 1 , 2 , 5 or 6 , wherein the switching node is determined implicitly.

8. The method of claim 1 , 2, 5, 6, or 7, wherein the partition block size is set to 64×64.

9. The method of claim 1, 2, 5 or 6, wherein the switching node is signaled.

10. The apparatus of claim 3 , wherein the partitioning tree structure is used to decode blocks belonging to a portion of the video image that is intra-coded.

11. The apparatus of claim 4 , wherein the partitioning tree structure is used to code blocks that belong to a portion of the video image that is intra-coded.

12. The apparatus of claim 3, 4, 10, or 11, wherein the switching node is determined implicitly.

13. 13. The apparatus of claim 3, 4, 10, 11, or 12, wherein the division block size is set to 64x64.

14. 12. The apparatus of claim 3, 4, 10 or 11, wherein the switching node is signaled.

15. A computer program comprising software code instructions for performing the method according to any one of claims 1, 2 or 5 to 9, when said computer program is executed by a processor.

Citation Information

Patent Citations

  • Method and apparatus for encoding / decoding an image unit containing image data represented by a luminance channel and at least one chrominance channel

    JP2019509662A

  • Method and device for encoding / decoding an image unit comprising image data represented by a luminance channel and at least one chrominance channel

    WO2017137311A1