Method, apparatus and medium for decoding and encoding video

By adopting a partially shared brightness and chromaticity encoding tree structure in video encoding, the problem of hardware implementation in the decoding stage in the prior art is solved, and efficient video encoding and decoding is achieved, ensuring the feasibility and encoding efficiency of the decoding pipeline.

CN118900329APending Publication Date: 2024-11-05INTERDIGITAL VC HOLDINGS INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411142617.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2018-05-29
Filing Date
2019-05-21
Publication Date
2024-11-05

AI Technical Summary

Technical Problem

The existing video encoding scheme has problems with hardware implementation methods during the decoding stage, and it is difficult to ensure high encoding efficiency and complexity limitations of the decoding pipeline at the same time.

Method used

Using a partially shared brightness and chromaticity coding tree structure, optimizes encoding efficiency and limits the complexity of the decoding pipeline by switching down from the maximum allowable block size or CU depth level to the separate encoding tree.

Benefits of technology

Improved encoding efficiency, ensured that the decoding pipeline based on 64x64 blocks is feasible, and improved the organizational efficiency of encoding bitstreams.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118900329A_ABST
    Figure CN118900329A_ABST
Patent Text Reader

Abstract

A method, an apparatus, and a medium for decoding and encoding a video are provided. The method for decoding a video comprises: decoding (920) a partition tree structure corresponding to a partition process of a block of an image of the video; the block is decoded (930) based at least on the partition tree structure.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This application is a divisional application of an invention patent application with an application date of May 21, 2019, application number 201980035564.X, and invention name “Method and device for video encoding and decoding using partially shared luminance and chrominance coding trees”. Technical Field

[0002] At least one of the present embodiments generally relates to a method or apparatus, for example, for video encoding or decoding, and more particularly to a method or apparatus in which a partition tree structure is partially shared between luma and chroma blocks. Background Art

[0003] The technical field of one or more implementations is generally related to video compression. Compared to existing video compression systems, such as HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2 described in "ITU-T H.265 Telecommunication Standardization Sector (10 / 2014), Series H: Audiovisual and Multimedia Systems, Infrastructure for Audiovisual Services – Coding of Mobile Video, High Efficiency Video Coding, Recommendation ITU-T H.265"), or compared to developed video compression systems, such as VVC (Versatile Video Coding, a new standard developed by the Joint Video Experts Group JVET), at least some embodiments relate to improving compression efficiency.

[0004] In order to achieve high compression efficiency, image and video coding schemes usually use partitioning, prediction (including motion vector prediction) and transformation of images to exploit spatial and temporal redundancy in video content. In general, intra-frame or inter-frame prediction is used to exploit intra-frame or inter-frame correlation, and then the difference between the original image and the predicted image (often expressed as prediction error or prediction residual) is transformed, quantized and entropy encoded. In order to reconstruct the video, the compressed data is decoded by the inverse process corresponding to entropy decoding, inverse transformation, inverse quantization and prediction.

[0005] With the advent of new video coding schemes, the partitioning schemes have become more sophisticated and allow dual-tree partitioning / coding of luma and chroma to achieve high compression. However, it seems that dual-tree coding schemes create hardware implementation issues at the decoding stage.

[0006] Therefore, there is a need for a new method for encoding and decoding video that jointly ensures high coding efficiency while taking into account some hardware decoding pipeline constraints. Summary of the invention

[0007] The object of the present invention is to overcome at least one disadvantage of the prior art. To this end, according to a general aspect of at least one embodiment, a method for video coding is proposed, comprising: determining a partition tree structure corresponding to a partition processing of a block of an image of the video, wherein the partition tree structure is partially shared between luminance and chrominance blocks; and encoding the block based on at least the partition tree structure.

[0008] According to another general aspect of at least one embodiment, a method for video decoding is proposed, comprising: determining a partition tree structure corresponding to a partition processing of a block of an image of a video, wherein the partition tree structure is partially shared between luminance and chrominance blocks; and decoding the block based at least on the partition tree structure.

[0009] According to another general aspect of at least one embodiment, an apparatus for video encoding is provided, comprising means for implementing any one embodiment of the encoding method.

[0010] According to another general aspect of at least one embodiment, an apparatus for video decoding is provided, comprising means for implementing any one embodiment of the decoding method.

[0011] According to another general aspect of at least one embodiment, an apparatus for video encoding is provided, comprising one or more processors and at least one memory. The one or more processors are configured to implement any one of the embodiments of the encoding method.

[0012] According to another general aspect of at least one embodiment, an apparatus for video decoding is provided, comprising one or more processors and at least one memory. The one or more processors are configured to implement any one of the embodiments of the decoding method.

[0013] According to another general aspect of at least one embodiment, a single partition tree structure is shared between luma and chroma blocks from the root node of the partition tree down to the switch node, and a dual partition tree structure is determined for luma and chroma blocks from the switch node down to the leaf nodes of the partition tree. Advantageously, this embodiment optimizes coding efficiency by allowing separate trees to be used for smaller blocks while limiting the complexity of the decoding pipeline.

[0014] According to another general aspect of at least one embodiment, a partially common partition tree structure is used for encoding or decoding a block, such as a tile or a group of tiles, that is part of an intra-coded video picture.

[0015] According to another general aspect of at least one embodiment, a partially common partition tree structure is used for encoding or decoding a block, such as a tile or a group of tiles, that is part of an inter-coded video picture.

[0016] According to another general aspect of at least one embodiment, the switch node is implicitly determined. According to a variation of this embodiment, the switch block node is determined based on the partition block size. Advantageously, the partition block size corresponds to the decoding pipeline unit size and is set to 64×64, for example. According to another variation of this embodiment, the switch node is determined based on the partition depth level. Advantageously, the partition depth level is set to a depth of the quadtree partitioning of the partition tree and corresponds to a quadtree partitioning of a 128x128 CTU into 4 sub-CUs of size 64x64, thereby being compatible with the decoding pipeline unit size.

[0017] The decoding pipeline unit may also be referred to as a virtual decoding pipeline unit (VPDU).

[0018] According to another general aspect of at least one embodiment, the switching node is signaled. Advantageously, this embodiment allows adapting the switching node for each CTU, for example based on RD cost.

[0019] According to another general aspect of at least one embodiment, a non-transitory computer-readable medium is provided, which contains data content generated according to any of the methods or apparatus described above.

[0020] According to another general aspect of at least one embodiment, a signal is provided, the signal comprising video data generated according to any of the methods or apparatus of the foregoing descriptions. According to another general aspect of at least one embodiment, the signal data is interleaved based on the partition block size. Thus, the signal data is advantageously arranged for use in a decoding pipeline unit.

[0021] According to yet another general aspect of at least one embodiment, a method for decoding a video is provided, comprising:

[0022] - decoding (920) a partition tree structure corresponding to the partition processing of blocks of an image of the video;

[0023] - decoding (930) said block based at least on said partition tree structure.

[0024] According to another general aspect of at least one embodiment, a decoding device is provided, comprising a memory and one or more processors configured to decode a video, wherein the one or more processors are further configured to:

[0025] - decoding a partition tree structure corresponding to the partitioning process of blocks of an image of a video;

[0026] - decoding said block based at least on said partition tree structure.

[0027] According to yet another general aspect of at least one embodiment, a method is provided comprising encoding a video, the encoding further comprising:

[0028] - determining (720) a partition tree structure corresponding to the partition processing of blocks of the images of the video;

[0029] - encoding (730) said block based at least on said partition tree structure.

[0030] According to another general aspect of at least one embodiment, an encoding apparatus is provided, comprising a memory and one or more processors configured to encode a video, wherein the one or more processors are further configured to:

[0031] - determining a partition tree structure corresponding to the partitioning process of blocks of the images of the video;

[0032] - encoding said block based at least on said partition tree structure.

[0033] According to yet another general aspect of at least one embodiment, a non-transitory computer-readable storage medium is provided, on which computer program instructions are stored, the computer program instructions being executed by a processor to implement the method according to yet another general aspect.

[0034] One or more of the embodiments further provide a computer-readable storage medium on which instructions for encoding or decoding video data according to any of the above methods are stored. The embodiment further provides a computer-readable storage medium on which a bitstream generated according to the above method is stored. The embodiment further provides a method and apparatus for sending a bitstream generated according to the above method. The embodiment further provides a computer program product including instructions for performing any of the described methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 The concepts of coding tree unit (CTU) and coding tree (CT) are illustrated to represent an example of a compressed HEVC picture.

[0036] Figure 2 An example of partitioning of coding units and associated coding trees in a quadtree plus binary tree (QTBT) scheme is illustrated.

[0037] Figure 3 and Figure 4 Some examples of CU binary tree or ternary tree partitioning are illustrated.

[0038] Figure 5 and Figure 6 An example illustrating a non-limiting embodiment of partitioning of a CTU in luma and chroma components respectively, in accordance with a general aspect of at least one embodiment.

[0039] Figure 7 and Figure 8 Various examples of encoding methods according to a general aspect of at least one embodiment are illustrated.

[0040] Fig. 9 , 10 , 11 and 12 illustrate various examples of decoding methods according to general aspects of at least one embodiment.

[0041] Fig.13 A block diagram illustrating an embodiment of a video encoder in which various aspects of the embodiments may be implemented.

[0042] Fig.14 Block diagram illustrating an embodiment of a video decoder in which various aspects of the embodiments may be implemented.

[0043] Fig.15 A block diagram illustrating an example apparatus in which various aspects of the embodiments may be implemented. DETAILED DESCRIPTION

[0044] It is to be understood that the drawings and descriptions have been simplified to illustrate the elements relevant to a clear understanding of the present principles, while for the sake of clarity, many other elements found in typical encoding and / or decoding devices are eliminated. It will be understood that although the terms first and second can be used to describe various elements in this article, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element.

[0045] Various embodiments are described with respect to encoding / decoding of images. They may be applied to encoding / decoding a portion of an image, such as a slice or tile, a group of tiles or an entire image sequence.

[0046] Various methods are described above, and each method includes one or more steps or actions for implementing the described method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions can be modified or combined.

[0047] At least some embodiments relate to methods for encoding or decoding video, wherein a tree coding / partitioning structure is partially shared between luma and chroma blocks.

[0048] In the HEVC video compression standard, images are divided into so-called Coding Tree Units (CTUs), which are typically 64x64, 128x128 or 256x256 pixels in size. Each CTU is represented by a coding tree in the compressed domain. This is a quadtree partitioning of the CTU, where each leaf is called a Coding Unit (CU). Each CU is then given some intra or inter prediction parameters (prediction information). For this purpose, it is spatially partitioned into one or more Prediction Units (PUs), each of which is assigned some prediction information. Intra or inter coding modes are assigned at CU level, e.g. Figure 1 shown.

[0049] Emerging video compression tools include a more flexible coding tree unit representation in the compression domain that provides higher compression efficiency compared to the CU / PU / TU arrangement of the HEVC standard. First, the coding tree unit representation includes a quadtree plus binary tree (QTBT), which is a coding structure in which coding units can be partitioned both in a quadtree manner and in a binary tree manner. Figure 2 An example of a coding tree representation of a coding tree unit is illustrated. The partitioning of the coding unit is determined at the encoder side by a rate-distortion optimization process, which includes determining the QTBT representation of the CTU with the minimum rate-distortion cost. In the QTBT technique, the CU has a square or rectangular shape. The size of the coding unit is always a power of 2, typically from 4 to 128.

[0050] In addition to the multiple rectangular shapes used for coding units, this new CTU representation has the following different features compared to HEVC. First, the QTBT decomposition of a CTU is divided into two stages: the CTU is first split in a quadtree manner, and then the leaves of each quadtree can be further split in a binary manner. This Figure 2 , where the solid line represents the quadtree decomposition stage and the dashed line represents the binary decomposition of the spatially embedded quadtree leaves.

[0051] Second, within an intra slice, the luma and chroma block partition structures are separated and determined independently.

[0052] Third, CU partitioning into prediction units or transform units is no longer used. In other words, each coding unit is systematically composed of a single prediction unit (2N×2N prediction unit partition type) and a single transform unit (not divided into transform trees).

[0053] Finally, if Figure 3 or Figure 4 For example, some other CU binary tree or ternary tree partitions may also be used in the representation of the CTU coding tree. Figure 3 Illustrate CU partitioning in an asymmetric binary tree, where a square or rectangular coding unit with size (w,h) (width and height) is split by one of the asymmetric binary partitioning modes, such as HOR_UP (horizontally up), resulting in respective rectangular sizes and In addition, Figure 4 Represents the so-called ternary tree partitioning of a CU. The ternary tree consists in splitting a CU into tree sub-CUs of size (1 / 4, 1 / 2, 1 / 4) relative to the parent UC in the considered orientation.

[0054] Using the new topology described above, significant coding efficiency improvements are achieved. In particular, significant gains are achieved in chroma. This large gain in chroma components is largely due to the separation of luminance and chroma coding trees in intra-frame strips. However, this separation of luminance and chroma coding trees at the CTU level has some problems in hardware implementation. In at least one embodiment, the typical size of a CTU is 128×128 or 256×256. In addition, completely separating the coding trees for luminance and chroma components means that these luminance and chroma components are also completely separated in the compressed domain, and therefore also appear in the encoded bitstream in a separated manner. This causes some problems for at least one decoder implementation, which is to ensure that the decoding pipeline is implemented at the maximum decoding unit size that may be smaller than the CTU size.

[0055] Typically, at least one decoder implementation expects a 64x64 based decoder pipeline. For this purpose, a maximum transform block size equal to 64x64 has been selected. In addition, the separation of luma and chroma coding trees is removed, because this means that all luma blocks of a CTU appear in the encoded bitstream before the chroma blocks in the same CTU. In this case, the decoding process will result in parsing and therefore generating decoded quantized block data that will spatially cover the entire CTU, i.e., a 128x128 or 256x256 picture area. The result of removing the luma / chroma coding separation is a loss in coding efficiency compared to the case where these trees are selected, encoded and decoded independently.

[0056] Therefore, it is desirable to have the luma and chroma coding tree representation in such a way that the coding efficiency is optimized while ensuring that a decoding pipeline, for example a 64x64 block based decoding pipeline is feasible.

[0057] To this end, at least one embodiment includes:

[0058] - Have a joint luma / chroma coding tree from the maximum allowed block size (CTU size) down to a predefined block size or up to a predefined CU depth level, which is determined according to the pipeline decoding unit. Then, when the block size is smaller than or equal to the pipeline decoding unit, the luma and chroma coding trees are selected, encoded and decoded in a separate manner. This means that the luma and chroma components share part of the entire CTU coding tree, and then, beyond a certain depth level in the CTU coding tree, the luma and chroma trees are independent and can therefore be different. In the following of this specification,

[0059] To refer to this aspect, the term partially shared luma / chroma coding tree is used.

[0060] - Switching from interleaved luma / chroma coding process to separate luma / chroma coding process depending on the block size associated with the processed coding tree node, or depending on the depth level of the encoded / decoded coding tree node.

[0061] In practice, assuming a 64x64 decoding pipeline unit, when encoding the luma and chroma coding trees, the coding trees are jointly determined and encoded down to block sizes larger than 64x64. Then, for block sizes smaller than 64x64, the luma / chroma coding trees may deviate from each other.

[0062] Advantageously, this process also ensures that in the bitstream organization, for each picture area equal to the considered decoding pipeline unit (64x64), the luminance and chrominance compressed signals appear one after another. Thus, when parsing the bitstream, it is ensured that the block residual signals associated with a picture area equal to the pipeline decoding unit 64x64 are completely decoded for both luminance and chrominance components before starting to process the next pipeline decoding unit.

[0063] Furthermore, the coding efficiency is improved compared to a complete separation of the luma / chroma coding trees.

[0064] In Section 1, several embodiments of partially common luma / chroma coding trees and associated bitstream organization are disclosed.

[0065] In Section 2, several embodiments of the encoding method are disclosed.

[0066] In Sections 3 and 4, several embodiments of decoding methods are disclosed.

[0067] In Section 5, additional information and general embodiments are disclosed.

[0068] 1. Embodiment of partially shared luma / chroma coding tree and associated bitstream organization

[0069] Figure 5 and Figure 6 Examples illustrating non-limiting embodiment partitioning of CTUs in luma and chroma components respectively according to a general aspect of at least one embodiment. On both examples, the considered CTU size is 128x128 and the considered decoding pipeline unit is 64x64.

[0070] Figure 5 A first example is illustrated, where the CTU is first split in a quadtree manner. This results in a 64x64 coding unit in the luma component (as shown on the left), which is aligned with the corresponding 32x32 CU in the chroma component (as shown on the right). It can be seen that when the size of the CU to be split is larger than the decoding unit size 64×64, the two luma and chroma split types are exactly the same. Next, for luma CUs less than or equal to 64x64, the split mode may be different between luma and chroma. Figure 5 This is the case except for the top right 64x64 CUs (3 and 4).

[0071] exist Figure 5 In the case of luma and chroma coding trees, Figure 5 The bottom of shows the bitstream organization proposed here. It can be seen that according to an embodiment of the bitstream arrangement, the bitstream is formed by consecutive parts comprising a series of encoded luminance blocks and a series of encoded chrominance blocks, respectively. The bitstream is organized in such a way that a series of encoded luminance or chrominance blocks corresponds to a spatial area not larger than 64x64 (in the luminance domain), i.e., the luminance size applied for the decoding pipeline unit.

[0072] In a particular embodiment, the portion of the bitstream comprising a given series of coded luminance or chrominance blocks contains exactly the coded data of the spatial region of the picture under consideration, the size of which is equal to the size of the decoding pipeline unit under consideration. Typically, this size is 64x64.

[0073] Figure 6 Another embodiment of a partially shared luma / chroma coding tree is illustrated. First the 128x128 undergoes a vertical binary split, which results in two 64x128 coding units. Since luma and chroma are synchronized for CU sizes larger than 64x64, the CTU is split in the same way in the chroma component, resulting in two 64x32 CUs. The right CU is no longer split. The left 128x64 is divided by a binary horizontal split pattern in a synchronized manner for luma and chroma components (a shared / synchronized portion of the coding tree arrangement according to at least one embodiment).

[0074] Furthermore, regarding the CUs produced by this second binary segmentation stage, thus having a size of 64x64 in luma, they can be further segmented in a separate manner between luma and chroma components. Figure 6 The resulting bitstream arrangement and ordering is shown at the bottom of . One particular aspect to note here is that since the maximum decoding unit size is 64x64, the maximum transform size is also 64x64. Therefore, Figure 6 In the case of a 64x128 CU on a 4K resolution, the CU is split into 2 transform units 64x64 in the luma component. Simultaneously, it is split into two 32x32 CUs in the 2 chroma components. Then, in terms of bitstream ordering, the 64x64 luma and 32x32 chroma transform blocks resulting from this spatial partitioning are interleaved in the bitstream to ensure that all luma and chroma coded data corresponding to a 64x64 spatial region are packed together in the coded bitstream. This corresponds to Figure 6 The encoded data blocks 5, 6, 7 and 8.

[0075] 2. At least one embodiment of the encoding

[0076] Figure 7 An example of an encoding method 700 is illustrated in accordance with general aspects of at least one embodiment.

[0077] In a preliminary step 710, a block for encoding is accessed. The block (typically, a CU) belongs to a spatial partition of a picture of the video, typically a CTU in the previous section. The encoding method 700 recursively determines the partition or split into sub-blocks for the block to be encoded. Typically, the method starts with the block that was a CTU in the previous sections. For example, according to Figure 2 , Figure 3 or Figure 4 The block is partitioned into sub-blocks according to any of the partitioning modes described. The partition tree structure (also referred to as coding tree or partition information) indicates how the image is partitioned. Therefore, in step 720, a partition tree structure corresponding to the partitioning process of the block is determined. Advantageously, the partition tree structure is partially shared between luma and chroma blocks. The partition tree structure that is partially shared between luma and chroma blocks means that for some blocks in the partition tree, a single partition tree structure is used for both luma and chroma blocks, while for other blocks in the partition tree, separate partition tree structures are used for luma blocks and chroma blocks. For example, corresponding to Figure 5 The luminance block corresponding to the luminance component of the CTU in the partition tree structure shares the first quadtree partition with the chrominance block corresponding to any chrominance component of the CTU. Figure 5 In step 730, the luma block 1 of the luma component of CU 1 and the chroma block 2 corresponding to any chroma component of CU 2 are partitioned in different ways, so a dual partition tree structure is determined for the luma block 1 and the chroma block 2 at the same position. In the following, the terms "separate tree" and "dual tree" are used indiscriminately to define the two trees, while the terms common tree, shared tree, joint tree and single tree are used indiscriminately to define the common tree for both luma and chroma components. In step 730, the block is encoded based on at least the determined partition tree structure.

[0078] According to one embodiment, a single partition tree structure is shared between luma and chroma blocks from the root node of the partition tree down to the switch node, and a dual partition tree structure is determined for luma and chroma blocks from the switch node down to the leaf nodes of the partition tree. Thus, a common coding tree is used for larger blocks, while a separate tree is used for smaller blocks, and the switch between the common coding tree and the dual coding tree is defined by a switch point or level. For example, the root node corresponds to a CTU, while the leaf node corresponds to a leaf CU. Advantageously, this embodiment optimizes coding efficiency by allowing separate trees to be used for some blocks while limiting the complexity of the decoding pipeline.

[0079] According to an embodiment, a partially shared partition tree structure is used to encode a block that is part of an intra-coded video image. For example, the part of the video image is a tile or a tile group, and the partially shared luminance and chrominance coding trees are only used in the intra-frame tile or the intra-frame tile group.

[0080] According to another embodiment, a partially common partition tree structure is used for encoding blocks that are part of an inter-coded video picture.

[0081] According to an embodiment, the switching node is determined implicitly. Advantageously, this embodiment allows the switching node to be signaled from the encoder to the decoder.

[0082] In a first variant, the switching block node is determined based on the partition block size. Thus, the partition block size is a predefined parameter in the encoder and decoder. For example, the switch from a common luma / chroma coding tree to two separate luma and chroma subtrees is decided based on the size of the coding unit relative to the size of the considered decoding unit. According to a preferred variant, the partition block size is set to 64×64, thereby allowing a decoding pipeline based on 64×64 blocks. Figure 5 For example, CTU128x128 shares the coding tree of chroma and luma components, while for CUs with a CU size less than or equal to 64x64, the subtrees of chroma and luma components are separated.

[0083] In a second variant, the switching block nodes are determined based on the partition depth level. Therefore, the partition block size is a parameter predefined in the encoder and decoder. According to a preferred example, the partition depth level is set to a depth of the quadtree partitioning of the partition tree. Figure 5 As an example, the common tree of a CTU comprises a first quadtree partition, below which the subtrees are separated for the CU. For example, the switch from a common luma / chroma coding tree to two separate luma and chroma subtrees is decided based on the depth level of the coding unit relative to the CU depth threshold. In a specific variant, the first quadtree partition is inferred. This variant is very suitable for a CTU size of 128x128, where the first QT partition results in 4 CUs of size 64x64, where a dual tree is then processed for luma and chroma components. In other words, the switch from a common luma / chroma coding tree to two separate luma and chroma subtrees is decided only based on the quadtree depth level relative to the CU quadtree depth threshold. In fact, in the QTBT coding structure, the CTU first undergoes a quadtree partition process before entering the so-called binary / ternary tree partition process. Therefore, two types of CU depth levels can be considered: quadtree-related depth levels and binary / ternary tree depth levels. A CU that undergoes quadtree partitioning has a binary / ternary tree level equal to 0, and the quadtree partitioning results in 4 sub-CUs whose quadtree-related depth level is increased by 1 compared to the parent CU.

[0084] In a third variant, the switching block node is also determined based on the partition depth level, wherein a composite quadtree / binary tree / ternary tree depth level is associated with each CU. The composite depth is exemplarily calculated as follows. If the CU is partitioned by a quadtree, the composite depth between the parent and child CU is increased by 3. If the CU is partitioned by a binary tree, the composite depth between the parent and child CU is increased by 2. If the CU is partitioned by a ternary tree, the composite depth between the parent and the middle child CU is increased by 1, and between the parent and the other two (non-middle) child CUs, the composite depth is increased by 3. Within the scope of this embodiment, the switching between the dual brightness / chrominance coding tree and the separate tree is determined based on the composite depth level of the current CU relative to the composite depth threshold.

[0085] According to an embodiment, the switching node is signaled. Advantageously, this embodiment allows adapting the switching node for each CTU, for example based on the RD cost.

[0086] Figure 8 Another example of an encoding method according to at least one embodiment is illustrated. Figure 8 An embodiment of a determination step 720 of partially common luma and chroma coding trees for a given CTU at an encoder is shown. Figure 8 The embodiment of corresponds to an embodiment in which the switching node is determined based on the partition block size. The decision process involves a recursive rate-distortion optimization process to search for two luma and chroma coding trees that jointly minimize the overall rate-distortion cost at the CTU level. To this end, at each depth level of the recursive process, a set (or part) of candidate partitioning modes is envisioned for the considered coding unit (corresponding to the entire CTU at the start of the process). Figure 8 There are two main cases shown in , on the left side (branch is yes) and on the right side (branch is no) of the figure.

[0087] The left side corresponds to the case where the current CU has a size strictly larger than the decoding pipeline unit size. In that case, according to at least one embodiment, a single partitioning mode must be selected to be used jointly in the luminance and chrominance components. To this end, each partitioning mode s or CurrentSplitMode in the set of allowed candidate partitioning modes of the current CU is evaluated, which results in the associated rate-distortion cost RDcost_YCbCr(CurrentSplitMode) being associated with the best encoding of the considered CU, which can be performed when the partitioning mode s of the current depth level is selected. Therefore, the evaluation of the partitioning mode s involves the partitioning of the current CU according to the partitioning mode s and then looping on each resulting sub-coding unit (sub-CU). For each sub-CU, the current RDO process is called in a recursive manner in order to determine the partitioning mode of the sub-CU considered. This looping on the candidate partitioning modes then leads to the determination of the partitioning mode that minimizes the joint luminance / chrominance rate-distortion cost associated with the current CU.

[0088] In case of 128x128 CTU, the quadtree splitting operation divides the 128x128 CTU into four 64x64 coding units. This splitting is either signaled as part of the common chroma / luminance coding tree or inferred to save bits. Note that if the quadtree splitting mode is inferred for a CU larger than the decoding pipeline unit size of 64x64, then Figure 8 The left side of the process consists simply in splitting the input CU into 4 sub-CUs and recursively applying on each resulting sub-CU in turn Figure 8 processing.

[0089] about Figure 8 The right side of the algorithm of , which corresponds to the case where the entire RD search process has reached a depth level in the coding tree, which corresponds to a coding unit size that is less than or equal to the size of the considered decoding pipeline unit, i.e., a switching node. In that case, at least one embodiment proposes switching from a joint luma / chroma coding tree to separate luma and chroma sub-coding trees, depending on the considered depth level in the overall coding tree representation being optimized. Therefore, the determination of the luma and chroma coding trees associated with the current CU takes the form of two separate, continuous optimization processes, which find the best luma coding tree and the best chroma coding tree for the current CU, respectively.

[0090] This corresponds to Figure 8 Two RDO steps follow. The optimization steps are known to those skilled in the art. The next step consists in calculating the entire joint luminance / chrominance rate-distortion cost RDcost_YCbCr(s_luma, s_chroma), which is derived from the obtained separate luminance and chrominance sub-coding trees. s_luma and s_chroma represent the best partitioning mode found for the current CU in the Luma (Y) and Chroma (Cb+Cr) components, respectively. This joint RD cost can be simply calculated by adding the ratios of the luminance and chrominance components (resulting in the total ratio R) and by calculating the total distortion D as the sum of the luminance and chrominance distortions produced by the two separate coding tree searches. The total RD cost is then obtained according to the classical formula, where λ is a known Lagrangian parameter:

[0091] RDcost=D+λ.R

[0092] Note that in the case of inferring quadtree partitioning mode for a CU larger than the decoding pipeline unit size 64x64, the rate-distortion best partitioning modes s_luma and s_chroma are assigned to the current CUcurrCU in the Luma and Chroma channels, respectively. In fact, for CUs larger than the decoding pipeline unit size, no rate-distortion competition occurs between partitioning modes.

[0093] From Figure 8 The first module / operation (which may or may not share the luma / chroma coding tree) may be located at e.g. Fig.13 In module 102 (image partitioning) (shown later).

[0094] 3 Decoding Example

[0095] Fig. 9 An example of a decoding method 900 is illustrated in accordance with a general aspect of at least one embodiment.

[0096] In a preliminary step 910, a bitstream or signal of a coding block is accessed for decoding. The coding block is typically a CU or CTU. The decoding method 900 recursively determines, partitions or splits the block to be decoded into sub-blocks. For example, according to Figure 2 , Figure 3 or Figure 4 Any partitioning mode described above partitions the coding block into sub-blocks. Therefore, in step 920, a partition tree structure corresponding to the partitioning process of the block is determined. Advantageously, the partition tree structure is partially shared between luma and chroma blocks. For example, the partition tree structure is obtained from the decoding process and from the syntax elements associated with the coding block. Figure 10-13 In step 930, the block is decoded based at least on the determined partition tree structure.

[0097] A person skilled in the art will easily adapt the variant embodiments described for the encoding method to the decoding method 900 , and therefore they will not be repeated here.

[0098] Fig.10 Another example of a decoding method according to a general aspect of at least one embodiment is illustrated. Fig.10 An embodiment of a determination step 920 of partially common luma and chroma coding trees for a given CTU at a decoder is shown. Fig.10 The embodiment corresponds to the embodiment in which the switching node is determined based on the partition block size. For encoding, this process also consists of two parts, respectively in Fig.10 The left and right sides are shown. Fig.10 The left side illustrates the processing of a CU whose width or height size is larger than the decoding pipeline unit under consideration. Fig.10 The right side of illustrates the decoding of coding units that are smaller than or equal to the size of the decoding pipeline unit under consideration. Therefore, in step 921, the left side implies a recursive resolution of the partitioning mode, and if the size of the CU is larger than the size of the decoding unit, the decoding of the CU can be performed. Fig.11Decoding of such a large CU is depicted above 921. According to a variant, the quadtree partitioning operation divides the CU into sub-blocks equal to the DPU size (eg, in case of a 128x128 CTU, the quadtree partitioning operation divides the 128x128 CTU into 4 64x64 coding units).

[0099] According to another variation, on the encoder side, for CUs larger than the DPU size, a quadtree partitioning mode may be inferred.

[0100] about Fig.10 , which comprises sequentially and independently parsing, decoding and reconstruction of the coding units contained in the considered picture area for the luminance component and then for the chrominance components.

[0101] From Fig.10 and Fig.11 The modules / operations may be located, for example, in Fig.14 Between modules 230 (entropy decoding including parsing of syntax elements) and 235 (partitioning) (shown later).

[0102] 4. Alternative Embodiments of Decoding

[0103] This section illustrates an alternative embodiment of the decoding process of Section 3. The difference here is that the block decoding process is performed on a decoding pipeline basis in order to explicitly comply with the constraints considered for the decoding process. In fact, a decoder compatible with an embodiment of the present principles can advantageously perform all the decoding processes for a decoding unit for luma and chroma components before starting to process the next decoding unit.

[0104] Fig.12 A decoding unit based process for decoding and reconstructing luma and chroma blocks according to an embodiment of the present invention is shown. The process first spatially divides the CTU into decoding units of size equal to the decoding pipeline unit, typically 64x64.

[0105] For example, the partitioning is performed by a quadtree partitioning operation that partitions the 128x128 CTU into four 64x64 coding units.

[0106] The process involves a loop over all decoding units (of typical size 64x64) contained in the considered CTU. For each decoding unit, first the luma blocks contained in the decoding unit are decoded, then the chroma blocks contained in the decoding unit are decoded and reconstructed.

[0107] 5. Other Examples and Information

[0108] This section describes various aspects, including tools, features, embodiments, models, methods, etc. Many of these aspects are described specifically and, at least to illustrate the various characteristics, are described in a manner that may sound restrictive. However, this is for clarity of description and does not limit the application or scope of those aspects. In fact, all different aspects can be combined and interchanged to provide further aspects. In addition, these aspects can also be combined and interchanged with aspects described in previous applications.

[0109] The aspects described and contemplated herein may be implemented in many different forms. Fig.13 , Fig.14 and Fig.15 Some embodiments are provided, but other embodiments are contemplated, and Fig.13 , 14 The discussion of 15 does not limit the scope of the embodiments. At least one of these aspects generally relates to video encoding and decoding, and at least another aspect generally relates to transmitting a generated or encoded bitstream. These and other aspects can be implemented as a method, an apparatus, a computer-readable storage medium having stored thereon instructions for encoding or decoding video data according to any of the described methods, and / or a computer-readable storage medium having stored thereon a bitstream generated according to any of the described methods.

[0110] In this application, the terms "reconstruction" and "decoding" are used interchangeably, the terms "pixel" and "sample" are used interchangeably, and the terms "image", "picture" and "frame" are used interchangeably. Usually, but not necessarily, the term "reconstruction" is used on the encoder side, while "decoding" is used on the decoder side.

[0111] Various methods are described above, and each method includes one or more steps or actions for implementing the described method. Unless a specific step or action sequence is required for proper operation of the method, the order and / or use of specific steps and / or actions may be modified or combined.

[0112] Various methods and other aspects described herein can be used to modify modules, such as, for example, Fig.13 and Fig.14 The partitioning module (102, 235) and entropy decoding module (230) of the JVET or HEVC encoder 100 and decoder 200 are shown. Furthermore, the present aspects are not limited to JVET or HEVC, and may be applied, for example, to other standards and recommendations, whether pre-existing or developed in the future, as well as extensions of any such standards and recommendations (including JVET and HEVC). Unless otherwise specified or technically excluded, the various aspects described herein may be used alone or in combination.

[0113] Various numerical values ​​are used in this document, such as a decoding unit pipeline of 64x64 or a block partition (CU, TU) size. The specific values ​​are for exemplary purposes, and the described aspects are not limited to these specific values.

[0114] Fig.13 An exemplary encoder 100 is illustrated. Variations of the encoder 100 are contemplated, but for clarity, the encoder 100 is described below without describing all contemplated variations.

[0115] Before being encoded, the video sequence may undergo a pre-encoding process (101), for example, applying a color transform to the input color picture (e.g., conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing a remapping of the input picture components in order to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and appended to the bitstream.

[0116] In an exemplary encoder 100, a picture is encoded by an encoder element as described below. The picture to be encoded is partitioned (102) and processed, for example, in units of CUs. Each unit is encoded, for example, using intra or inter mode. When a unit is encoded in intra mode, it performs intra prediction (160). In inter mode, motion estimation (175) and compensation (170) are performed. The encoder decides (105) which of intra mode or inter mode is used to encode the unit, and indicates the intra / inter decision, for example, by a prediction mode flag. The prediction residual is calculated by subtracting (110) the predicted block from the original image block.

[0117] The prediction residual is then transformed (125) and quantized (130). The quantized transform coefficients are entropy encoded (145) along with motion vectors and other syntax elements to output a bitstream. The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can also bypass both the transform and quantization, i.e., directly encode the residual without applying the transform or quantization process.

[0118] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (155) to reconstruct the image block. An in-loop filter (165) is applied to the reconstructed picture to, for example, perform deblocking / SAO (sample adaptive offset) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer (180).

[0119] Fig.142 is a block diagram of an exemplary video decoder 200. In the exemplary decoder 200, a bitstream is decoded by decoder elements, as described below. The video decoder 200 generally performs the same Fig.13 The decoding path is the inverse of the encoding path described in . The encoder 100 also typically performs video decoding as part of encoding the video data.

[0120] Specifically, the input to the decoder includes a video bitstream that may be generated by the video encoder 100. The bitstream is first entropy decoded (230) to obtain transform coefficients, motion vectors, and other encoding information. Picture partition information indicates how the picture is partitioned. Thus, the decoder can divide (235) the picture according to the decoded picture partition information. The transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (255) to reconstruct the image block. The prediction block can be obtained (270) from intra-frame prediction (260) or motion compensated prediction (i.e., inter-frame prediction) (275). An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored at a reference picture buffer (280).

[0121] The decoded picture may further undergo post-decoding processing (285), such as inverse color conversion (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping, which performs the inverse of the remapping process performed in the pre-encoding process (101). The post-decoding process may use metadata derived in the pre-encoding process and signaled in the bitstream.

[0122] Fig.15 A block diagram illustrating an example system in which various aspects and exemplary embodiments are implemented. System 1000 may be embodied as a device including the various components described below and configured to perform one or more aspects described herein. Examples of such devices include, but are not limited to, personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. System 1000 may be communicatively coupled to other similar systems and coupled to displays via communication channels, such as Fig.15 As shown, and as known to those skilled in the art, various aspects described in this document can be implemented.

[0123] The system 1000 includes at least one processor 1010 configured to execute instructions loaded therein for implementing, for example, various aspects described in this document. The processor 1010 may include embedded memory, input-output interfaces, and various other circuits known in the art. The system 1000 also includes at least one memory 1020 (e.g., a volatile memory device, a non-volatile memory device). The system 1000 includes a storage device 1040, which may include a non-volatile memory, including but not limited to EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash memory, a disk drive, and / or an optical drive. As a non-limiting example, the storage device 1040 may include an internal storage device, an attached storage device, and / or a network accessible storage device. The system 1000 may also include an encoder / decoder module 1030 configured to process data to provide encoded video or decoded video, and the encoder / decoder module 1030 may include its own processor and memory.

[0124] The encoder / decoder module 1030 represents a module(s) that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of the encoding and decoding modules. In addition, as known to those skilled in the art, the encoder / decoder module 1030 may be implemented as a separate element of the system 1000, or may be incorporated into the processor 1010 as a combination of hardware and software known to those skilled in the art.

[0125] Program code to be loaded onto the processor 1010 to perform various aspects described herein may be stored in the storage device 1040 and subsequently loaded onto the memory 1020 for execution by the processor 1010. According to an exemplary embodiment, one or more of the processor(s) 1010, the memory 1020, the storage device 1040, and the encoder / decoder module 1030 may store one or more of various items during execution of the processes described herein, including, but not limited to, input video, decoded video, bitstreams, equations, formulas, matrices, variables, operations, and operation logic.

[0126] The system 1000 includes a communication interface 1050 that enables communication with other devices via a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to send and receive data over the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or a network card, and the communication channel 1060 may be implemented, for example, within a wired and / or wireless medium. The various components of the system 1000 may all be connected or communicatively coupled together using various suitable connections, including, but not limited to, internal buses, wires, and printed wiring boards.

[0127] The exemplary embodiments may be performed by computer software implemented by the processor 1010, or by hardware, or by a combination of hardware and software. As a non-limiting example, the exemplary embodiments may be implemented by one or more integrated circuits. As a non-limiting example, the memory 1020 may be of any type suitable for the technical environment and may be implemented using any appropriate data storage technology, such as, optical storage devices, magnetic storage devices, semiconductor-based storage devices, fixed memory, and removable memory. As a non-limiting example, the processor 1010 may be of any type suitable for the technical environment and may include one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0128] The implementations and aspects described herein can be implemented in, for example, methods or processes, devices, software programs, data streams, or signals. Even if only discussed in the context of a single implementation (e.g., discussed only as a method), the implementation of the features discussed can also be implemented in other forms (e.g., devices or programs). The device can be implemented in, for example, appropriate hardware, software, and firmware. The method can be implemented in, for example, a device (e.g., a processor), which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes communication devices, such as computers, cellular phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate information communication between end users.

[0129] References to "one embodiment" or "an embodiment" or "one implementation" or "an implementation" and other variations thereof mean that a particular feature, structure, characteristic, etc. described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "in one implementation" or "in an implementation" and any other variations appearing in various places throughout this document are not necessarily all referring to the same embodiment.

[0130] Additionally, this document may refer to "determining" various information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from a memory.

[0131] Additionally, this document may refer to "accessing" various information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0132] Additionally, this document may refer to "receiving" various information. As with "accessing," receiving is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory device). Furthermore, "receiving" typically involves, in one way or another, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information during an operation.

[0133] Additionally, this document may refer to optimizations, and the various embodiments need not be fully optimized, with some embodiments simply selecting patterns without optimization in the usual sense.

[0134] It will be apparent to those skilled in the art that implementations may generate various signals formatted to carry information that may be stored or transmitted, for example. The information may include, for example, instructions for executing a method or data generated by one of the described embodiments. For example, a signal may be formatted to carry a bitstream of the described embodiments. Such a signal may be formatted, for example, as an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or as a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor readable medium.

[0135] A number of embodiments have been described above. These embodiments provide at least the following general inventions and claims, including all combinations across various claim categories and types:

[0136] Modify the luma and chroma coding trees applied in the decoder and / or encoder.

[0137] Combined interleaved luma / chroma coding and separated luma / chroma coding applied in the decoder and / or encoder

[0138] Chroma coding.

[0139] Combined interleaved luminance / chrominance coding and separated luminance / chrominance coding, where separated luminance

[0140] Chroma coding is controlled by information about constraints on the decoding unit pipeline applied in the decoder and / or encoder.

[0141] Jointly determine, encode and signal luma and chroma coding trees from the maximum allowed block size (CTU size) down to a given block size or up to a given CU depth level, and independently determine, encode and signal luma and chroma coding trees for blocks smaller than a given block size or beyond a given CU depth level.

[0142] Switching from interleaved luma / chroma coding to separated luma / chroma coding depending on the block size associated with the processed coding tree node, or depending on the depth level of the coding tree node to be encoded / decoded.

[0143] • Enabling several partially common luma and chroma coding tree approaches in the decoder and / or encoder.

[0144] Based on certain syntax elements, the luma and chroma coding trees to be applied at the decoder are selected.

[0145] · Create and / or send and / or receive and / or decode a coded brightness including one or more descriptions

[0146] / chrominance components or their variants.

[0147] A television, set-top box, mobile phone, tablet or other electronic device that performs encoding / decoding according to any of the described embodiments with the partially common luma and chroma coding tree approach enabled.

[0148] A television, set-top box, mobile phone, tablet, or other electronic device that performs encoding and decoding according to any of the described embodiments that enables the partially common luma and chroma coding tree approach, and displays (e.g., using a monitor, screen, or other type of display) the resulting image.

[0149] a television, set-top box, mobile phone, tablet or other electronic device, tuning (e.g., using a tuner) to a channel to receive a signal including an encoded image according to any of the described embodiments,

[0150] And performs decoding enabling partially shared luma and chroma coding trees.

[0151] a television, set-top box, mobile phone, tablet or other electronic device that wirelessly receives (e.g., using an antenna) a signal including an encoded image according to any of the embodiments described,

[0152] And performs decoding that enables partially shared luma and chroma coding tree methods.

[0153] Various other general and specific inventions and claims are also supported and contemplated throughout this disclosure.

Claims

1. A method for decoding a video, comprising: - decoding (920) a partition tree structure corresponding to the partition processing of blocks of an image of the video; - decoding (930) said block based at least on said partition tree structure.

2. A decoding device comprising a memory and one or more processors configured to decode a video, wherein: The one or more processors are further configured to: - decoding a partition tree structure corresponding to the partitioning process of blocks of an image of a video; - decoding said block based at least on said partition tree structure.

3. A method comprising encoding a video, the encoding further comprising: - determining (720) a partition tree structure corresponding to the partition processing of blocks of the images of the video; - encoding (730) said block based at least on said partition tree structure.

4. An encoding device comprising a memory and one or more processors configured to encode a video, wherein: The one or more processors are further configured to: - determining a partition tree structure corresponding to the partitioning process of blocks of the images of the video; - encoding said block based at least on said partition tree structure.

5. A non-transitory computer-readable storage medium having computer program instructions stored thereon, the computer program instructions being executed by a processor to implement the method according to any one of claims 1 to 3.