Method and apparatus for video encoding and decoding using partially shared luminance and saturation coding trees

Abstract

To provide a method for video encoding and decoding based on a partition tree structure that is partially shared between luma and chroma blocks.SOLUTION: A video decoding method includes partitioning an image of a video into a plurality of coding tree units, and for an I slice, each coding tree unit is partitioned using an implicit quadtree partition into coding units having 64×64 luma samples, decoding an indication that the coding unit is the root of two separate coding tree syntax structures for luma and chroma, and decoding the plurality of coding tree units on the basis of the indication.SELECTED DRAWING: Figure 9

Description

TECHNICAL FIELD

[0001] At least one of the present embodiments generally relates to, for example, a method or apparatus for video encoding or decoding, and more particularly, to a method or apparatus in which a partition tree structure is partially shared between a luma block and a chroma block. BACKGROUND ART

[0002] The technical field of one or more implementations generally relates to video compression. At least some embodiments relate to improving compression efficiency compared to existing video compression systems such as HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2 described in "ITU-T H.265 Telecommunication standardization sector of ITU (10 / 2014) Series H: Audiovisual and Multimedia Systems, Infrastructure of Audiovisual Services - Coding of Moving Video, High Efficiency Video Coding, Recommendation ITU-T H.265"), or compared to under-development video compression systems such as VVC (Versatile Video Coding, a new standard being developed by JVET, the Joint Video Experts Team).

[0003] To achieve high compression efficiency, image and video coding schemes usually employ picture partitioning, prediction including motion vector prediction, and transforms to leverage spatial and temporal redundancy of video content. In general, intra or inter prediction is used to exploit the correlation of intra or inter frames, then the difference between the original picture and the predicted picture, often called prediction error or prediction residual, is transformed, quantized, and entropy coded. To reconstruct the video, the compressed data is decoded by inverse processes corresponding to entropy decoding, inverse quantization, inverse transform, and prediction.

[0004] With the emergence of new video encoding schemes, the segmentation methods have become more complex, and dual-tree segmentation / encoding of luminance and saturation has become possible to achieve high compression. However, dual-tree encoding schemes seem to have problems with hardware implementation during the decoding stage.

[0005] Therefore, a new method is needed for encoding and decoding video that, while taking into account the constraints of several hardware decoding pipelines, jointly guarantees high encoding efficiency. [Overview of the project]

[0006] An object of the present invention is to overcome at least one of the disadvantages of the prior art. To this end, according to a general aspect of at least one embodiment, a method for video encoding is presented, which includes determining a partition tree structure corresponding to a process of partitioning blocks of images in a video, wherein the partition tree structure is partially shared between luminance blocks and saturation blocks, and encoding the blocks based at least on the partition tree structure.

[0007] According to another general aspect of at least one embodiment, a method for video decoding is presented, which includes determining a partition tree structure corresponding to a process of dividing blocks of images in a video, wherein the partition tree structure is partially shared between luminance blocks and saturation blocks, and decoding the blocks based at least on the partition tree structure.

[0008] According to another general aspect of at least one embodiment, an apparatus for video coding is presented, which includes means for carrying out any one of the embodiments of the coding method.

[0009] According to another general aspect of at least one embodiment, an apparatus for video decoding is presented, which includes means for carrying out any one of the embodiments of the decoding method.

[0010] According to another general aspect of at least one embodiment, an apparatus for video encoding is provided, comprising one or more processors and at least one memory. The one or more processors are configured to be implemented in any one of the embodiments of the encoding method.

[0011] According to another general aspect of at least one embodiment, an apparatus for video decoding is provided, comprising one or more processors and at least one memory. The one or more processors are configured to be implemented in any one of the embodiments of the decoding method.

[0012] According to another general aspect of at least one embodiment, a single partitioned tree structure is shared between luminance and chrominance blocks from the root node of the partitioned tree to the switching node, while a double partitioned tree structure is determined for luminance and chrominance blocks from the switching node to the leaf nodes of the partitioned tree. Advantageously, this embodiment optimizes encoding efficiency by enabling separate trees for smaller blocks while limiting the complexity of the decoding pipeline.

[0013] According to another general aspect of at least one embodiment, a partially shared partitioned tree structure is used to encode or decode a portion of an intra-encoded video image, for example, a block belonging to a tile or group of tiles.

[0014] According to another general aspect of at least one embodiment, a partially shared partitioned tree structure is used to encode or decode a portion of an intercoded video image, for example, a block belonging to a tile or group of tiles.

[0015] According to another general aspect of at least one embodiment, the switching node is implicitly determined. According to a variation of this embodiment, the switching block node is determined based on the partition block size. Advantageously, the partition block size corresponds to the size of the decoding pipeline unit and is set to, for example, 64x64. According to another variation of this embodiment, the switching node is determined based on the partition depth level. Advantageously, the partition depth level is set to one depth of the quadtree split of the partition tree and corresponds to a quadtree split into four subCUs of size 64x64, with a CTU size of 128x128, and is therefore compatible with the size of the decoding pipeline unit.

[0016] A decoding pipeline unit is sometimes also called a Virtual Decoding Pipeline Unit (VPDU).

[0017] According to another general aspect of at least one embodiment, the switching node is signal-notified. Advantageously, this embodiment allows the switching node to be adapted to each CTU, for example, based on RD cost.

[0018] According to another general aspect of at least one embodiment, a non-temporary computer-readable medium is presented containing data content generated according to any of the methods or apparatus described above.

[0019] According to another general aspect of at least one embodiment, a signal is provided, which includes video data generated according to any of the methods or apparatus described above. According to another general aspect of at least one embodiment, the signal data is interleaved based on the division block size. Thus, the signal data is arranged favorably for decoding pipeline units.

[0020] Furthermore, one or more of these embodiments provide a computer-readable storage medium in which instructions for encoding or decoding video data according to any of the methods described above are stored. These embodiments also provide a computer-readable storage medium in which a bitstream generated according to any of the methods described above is stored. These embodiments also provide a method and apparatus for transmitting a bitstream generated according to any of the methods described above. These embodiments also provide a computer program product containing instructions for performing any of the methods described. [Brief explanation of the drawing]

[0021] [Figure 1] This provides an example of the concepts of a Coding Tree Unit (CTU) and a Coding Tree (CT) for representing a compressed HEVC picture. [Figure 2] This shows the coding unit and related coding tree partitioning examples in the Quad-Tree plus Binary-Tree (QTBT) method. [Figure 3-4] Here are some examples of binary or ternary tree partitioning for several CUs. [Figure 5-6] The following are exemplary divisions of a non-limiting embodiment of the CTU in terms of luminance and chrominance components, respectively, according to a general aspect of at least one embodiment. [Figure 7-8] Various examples of encoding methods are shown, according to a general aspect of at least one embodiment. [Figure 9-12] Various examples of decoding methods are shown, according to a general aspect of at least one embodiment. [Figure 13] A block diagram of an embodiment of a video encoder is shown, in which various aspects of the embodiment can be carried out. [Figure 14] A block diagram of an embodiment of a video decoder is shown, in which various aspects of the embodiment can be implemented. [Figure 15] A block diagram of an exemplary apparatus is shown, illustrating various embodiments of the model. [Mode for Carrying Out the Invention]

[0022] It should be understood that the figures and the description have been simplified to illustrate relevant elements for a clear understanding of the principles of the present invention, while excluding many other elements found in typical encoding devices and / or decoding devices for the sake of clarity. Although various elements may be described herein using terms such as first and second, it will be appreciated that these elements should not be limited by these terms. These terms are only used to distinguish one element from another.

[0023] Various embodiments are described with respect to image encoding / decoding. These embodiments may be applied to encode / decode part of an image, such as a slice image, a tile image, a tile group image, or the entire sequence of images.

[0024] Various methods have been described above, each of which includes one or more steps or operations for achieving the described method. Unless the order of specific steps or operations is required for the correct operation of the present method, the order and / or use of specific steps and / or operations may be modified or combined.

[0025] At least some embodiments relate to a method for encoding or decoding video, in which a tree coding / partitioning structure is partially shared between luma blocks and chroma blocks.

[0026] In the HEVC video compression standard, images are divided into so-called coding tree units (CTUs), typically 64x64, 128x128, or 256x256 pixels in size. Each CTU is represented by a coding tree of the compression domain. This coding tree is a quadtree partition of the CTUs, with each leaf being called a coding unit (CU). Each CU is then assigned several intra or intercoding parameters (prediction information). To do this, the CU is spatially divided into one or more prediction units (PUs), and each PU is assigned several pieces of prediction information. As shown in Figure 1, the intra or intercoding mode is assigned at the CU level.

[0027] Newly emerging video compression tools include a more flexible coded tree unit representation of the compression domain, offering improved compression efficiency compared to the HEVC standard's CU / PU / TU arrangement. First, the coded tree unit representation involves adding a binary tree to a quadtree (QTBT), a coded structure that allows the coded unit to be split in both quadtree and binary tree configurations. An exemplary coded tree representation of the coded tree unit is shown in Figure 2. The splitting of the coded unit is determined on the encoder side through a rate-distortion optimization procedure, which involves determining the QTBT representation of the CTU with the lowest rate-distortion cost. In QTBT technology, the shape of the CU is either square or rectangular. The size of the coded unit is always a power of 2, typically between 4 and 128.

[0028] In addition to this diverse range of rectangles for the coding unit, this new CTU representation has the following different characteristics compared to HEVC. Firstly, the QTBT decomposition of CTU consists of two stages, in which the CTU is first split in a quadtree manner, and then each quadtree leaf can be further divided in a binary manner. This is shown on the right side of Figure 2, where the solid line represents the quadtree decomposition phase and the dashed line represents the binary decomposition spatially embedded in the quadtree leaf.

[0029] Secondly, in intra-slice, the division structure of luminance blocks and saturation blocks is separated and determined independently.

[0030] Thirdly, the division of CUs into prediction units or transformation units will no longer be adopted. In other words, each coding unit is systematically composed of a single prediction unit (2N × 2N prediction unit division type) and a single transformation unit (without division into a transformation tree).

[0031] Finally, as shown in Figure 3 or Figure 4, several other CU2 subtree or ternary subtrees can also be employed in the representation of the CTU's coded tree. Figure 3 shows the respective rectangular sizes.

number

number

[0032] The new topology described above significantly improves coding efficiency, particularly in saturation. This large gain in the saturation component is largely attributable to the separation of the luminance and saturation coding trees within the intra-slice. However, this separation of the luminance and saturation coding trees relative to the CTU level presents several challenges in hardware implementation. In at least one embodiment, the CTU is typically 128x128 or 256x256 in size. Furthermore, completely separating the coding trees of the luminance and saturation components also means that these luminance and saturation components are also completely separated in the compressed domain and therefore appear separated in the coded bitstream. This presents several challenges in at least one decoder implementation, where we want to ensure that the decoding pipeline is achieved at a size that may be smaller than the CTU size for the maximum decoding unit.

[0033] Typically, at least one decoder implementation requires a 64x64 based decoder pipeline. For this purpose, a maximum conversion block size equal to 64x64 is selected. Additionally, the separation of luminance and saturation coding trees is removed because it implies a coded bitstream where every luminance block in a CTU appears before the saturation blocks of the same CTU. In such cases, the decoding process leads to analysis, thus yielding coded quantized block data that spatially covers the entire CTU, i.e., a 128x128 or 256x256 picture area. The removal of the luminance / saturation coding separation results in reduced coding efficiency compared to the case where these trees are selected, coded, and decoded independently.

[0034] Therefore, it is desirable to have coded tree representations of luminance and saturation while ensuring that a decoding pipeline, such as a 64x64 block-based decoding pipeline, is feasible in a way that optimizes coding efficiency.

[0035] To that end, at least one embodiment includes the following: -The system has a shared luminance / chroma coding tree ranging from the maximum allowable block size (CTU size), determined according to the pipeline decoding unit, to a predefined block size, or to a predefined CU depth level. When the block size is less than or equal to the pipeline decoding unit, the luminance and chroma coding trees are selected, encoded, and decoded separately. This means that a portion of the entire CTU coding tree is shared by the luminance and chroma components, and then beyond a certain depth level of the CTU coding tree, the luminance and chroma trees become independent and therefore can be different. Hereafter, the term partially shared luminance / chroma coding tree is used to refer to this embodiment. - Switch from an interleaved luminance / saturation encoding process to a separate luminance / saturation encoding process, depending on the block size associated with the processed encoded tree node, or according to the depth level of the encoded tree node being encoded / decoded.

[0036] In practice, when encoding luminance and saturation coding trees assuming a 64x64 decoding pipeline unit, the coding trees are jointly determined and encoded into block sizes larger than 64x64. Then, for block sizes smaller than 64x64, the luminance / saturation coding trees may branch off from each other.

[0037] Advantageously, this process also ensures that, in the bitstream configuration, the luminance and saturation compression signals appear successively for each picture region equal to the decoding pipeline unit (64x64) being considered. Therefore, when analyzing the bitstream, it is guaranteed that the block residual signals associated with each picture region equal to the 64x64 pipeline decoding unit are fully decoded for both the luminance and saturation components before the process for the next pipeline decoding unit begins.

[0038] Furthermore, coding efficiency is improved compared to completely separating the coding trees for luminance and saturation.

[0039] Section 1 discloses several embodiments of partially shared luminance / saturation coding trees and associated bitstream configurations.

[0040] Section 2 discloses several embodiments of the encoding method.

[0041] Sections 3 and 4 disclose several embodiments of the decryption method.

[0042] Section 5 discloses additional information and general embodiments.

[0043] 1. Embodiments of partially shared luminance / saturation coding trees and associated bitstream configurations. Figures 5 and 6 show exemplary divisions of non-limiting embodiments of the CTU in terms of luminance and chrominance components, respectively, according to a general aspect of at least one embodiment. In both examples, the CTU size considered is 128 × 128, and the decoding pipeline units considered are 64 × 64.

[0044] Figure 5 shows the first example of a CTU splitting in a quadtree manner. This results in a 64x64 coded unit within the luminance component, as shown on the left side of the figure, which is aligned with the corresponding 32x32 CU within the chroma component, as shown on the right side of the figure. As shown in the figure, when the size of the splitting CU is larger than the size of the decoding unit (64x64), the splitting types of the two luminance and chroma components are identical. Next, for luminance CUs smaller than 64x64, the splitting modes may differ between luminance and chroma. This is the case in Figure 5, except for the 64x64 CUs (3 and 4) in the upper right.

[0045] The bottom of Figure 5 shows the bitstream configuration proposed herein for the luminance and saturation coding tree of Figure 5. As shown in the figure, according to the embodiment of the bitstream configuration, the bitstream consists of consecutive portions, each containing a series of coded luminance blocks and a series of coded saturation blocks. The bitstream is configured such that each series of coded luminance blocks or saturation blocks corresponds to a spatial area of ​​64 × 64 (within the luminance domain) or less, i.e., the luminance size imposed on the decoding pipeline unit.

[0046] In certain embodiments, a bitstream portion containing a given set of coded luminance or saturation blocks precisely contains coded data for a spatial region of the picture under consideration, the size of which is equal to the size of the decoding pipeline unit under consideration. Typically, this size is 64x64.

[0047] Figure 6 shows another embodiment of a partially shared luminance / saturation coding tree. A 128×128 initially undergoes a vertical bifurcation, yielding two 64×128 coding units. For CU sizes larger than 64×64, since luminance and saturation are synchronized, the CTU splits similarly on the saturation component, yielding two 64×32 CUs. The right-hand CU does not split further. The left-hand 128×64 is split synchronously on the luminance and saturation components (the shared / synchronized portion of the coding tree arrangement in at least one embodiment) in a bifurcation horizontal split mode.

[0048] Furthermore, the CUs resulting from this second bifidation stage have a luminance size of 64×64, and can therefore be further split in a manner separated between the luminance and chroma components. Finally, the resulting bitstream arrangement and ordering are shown at the bottom of Figure 6. A particular aspect to note here is that since the size of the maximum decoding unit is 64×64, the maximum transformation size is also 64×64. Thus, in the case of the 64×128 CU in Figure 6, the CU splits into two 64×64 transformation units by its luminance component. Synchronously, it splits into two 32×32 CUs by its two chroma components. Next, with respect to bitstream ordering, the 64×64 luminance transformation blocks and 32×32 chroma transformation blocks resulting from this spatial division are interleaved into the bitstream so that all the encoded luminance and chroma data corresponding to the 64×64 spatial region are packed together into the encoded bitstream. This corresponds to the encoded data blocks 5, 6, 7, and 8 in Figure 6.

[0049] 2. At least one embodiment of encoding Figure 7 shows an example of an encoding method 700 according to a general aspect of at least one embodiment.

[0050] In the preliminary step 710, the blocks are accessed for encoding. A block, typically a CU, is a spatial division of the video image, usually belonging to the CTU in the previous section. The encoding method 700 recursively determines, for the block to be encoded, whether to divide or split it into subblocks. Typically, the method begins with a block that is the CTU in the previous section. For example, a block is split into subblocks according to one of the splitting modes described in Figures 2, 3, or 4. A split tree structure, also called a coded tree or splitting information, shows how the image is divided. Thus, in step 720, a split tree structure corresponding to the block splitting process is determined. Conveniently, the split tree structure is partially shared between the luminance block and the saturation block. A split tree structure partially shared between the luminance block and the saturation block means that for some blocks of the split tree, a single split tree structure is used for both the luminance block and the saturation block, while for other blocks of the split tree, separate split tree structures are used for both the luminance block and the saturation block. For example, the luminance block corresponding to the luminance component of CTU in Figure 5 shares the first quadtree split of the split tree structure with the chroma block corresponding to one of the chroma components of CTU. Next, luminance block 1 corresponding to the luminance component of CU1 in Figure 5 and chroma block 2 corresponding to one of the chroma components of CU2 are split in different ways, and thus a double split tree structure is determined for luminance block 1 and for the co-located chroma block 2. Hereafter, the terms separate tree and double tree are used indiscriminately to define two trees, while the terms common tree, shared tree, joint tree and single tree are used indiscriminately to define a common tree for both the luminance and chroma components. In step 730, the blocks are coded based on at least the determined split tree structure.

[0051] According to the embodiment, a single partitioned tree structure is shared between luminance and chrominance blocks from the root node to the switching node of the partitioned tree, while a double partitioned tree structure is determined for luminance and chrominance blocks from the switching node to the leaf nodes of the partitioned tree. Thus, the shared coding tree is used for larger blocks, while the separated tree is used for smaller blocks, and the switching between the shared coding tree and the double coding tree is defined by the switching point or switching level. For example, the root node corresponds to the CTU, while the leaf nodes correspond to the leaf CU. Advantageously, this embodiment optimizes coding efficiency by enabling separated trees for some blocks while limiting the complexity of the decoding pipeline.

[0052] According to one embodiment, a partially shared partitioned tree structure is used to encode blocks belonging to a portion of an intra-encoded video image. For example, the portion of the video image is a tile or group of tiles, and the partially shared luminance and saturation encoding trees are used only for intra-tiles or groups of intra-tiles.

[0053] According to another embodiment, a partially shared partitioned tree structure is used to encode blocks that belong to a portion of the video image encoded in the interface.

[0054] According to the embodiment, the switching node is implicitly determined. Advantageously, this embodiment avoids sending a signal to the switching node from the encoder to the decoder. In the first modification, the switching block node is determined based on the partition block size. Thus, the partition block size is a predefined parameter in the encoder and decoder. For example, the switching from a shared luminance / saturation coding tree to two separate luminance and saturation subtrees is determined based on the size of the coding unit, with respect to the size of the decoding unit being considered. According to the preferred modification, the partition block size is set to 64x64, thus enabling a decoding pipeline on a 64x64 block basis. As shown in Figure 5, a CTU 128×128 shares a coding tree for both saturation and luminance components, but for CUs of 64×64 or smaller, the subtrees are separated for saturation and luminance components.

[0055] In the second modification, the switching block node is determined based on the splitting depth level. Thus, the splitting block size is a parameter predefined in the encoder and decoder. According to the preferred example, the splitting depth level is set to one depth of the quadtree split of the splitting tree. As shown in Figure 5, the shared tree of the CTU includes a first quadtree split, and below the subtrees are separated for the CUs. For example, the switching from a shared luminance / saturation coding tree to two separate luminance and saturation subtrees is determined based on the depth level of the coding unit with respect to the depth threshold of the CUs. In a particular modification, the first quadtree split is inferred. This modification fits well with a CTU size of 128x128, where the first QT split generates four CUs of size 64x64, and then a double tree is processed for the luminance and saturation components. In other words, the switching from a shared luminance / saturation coding tree to two separate luminance and saturation subtrees is determined based only on the quadtree depth level with respect to the quadtree depth threshold of the CUs. In fact, in the coded structure of QTBT, a CTU first undergoes a quadtree splitting process before entering the so-called binary / ternary splitting process. Thus, we can consider two types of CU depth levels: the quadtree-related depth level and the binary / ternary depth level. The binary / ternary level of a CU undergoing a quadtree split is equal to 0, the quadtree split leads to four subCUs, and the quadtree-related depth level increases by 1 compared to the parent CU.

[0056] In the third variation, the switching block node is also determined based on the split depth level, where the depth level of the composite quad / binary / ternary tree is associated with each CU. This composite depth is calculated exemplarily as follows: If the CU is a quad branch, the composite depth increases by 3 between the parent CU and the child CU. If the CU is a binary branch, the composite depth increases by 2 between the parent CU and the child CU. If the CU is a ternary branch, the composite depth increases by 1 between the parent CU and the meson CU, and by 3 between the parent CU and the other two (non-intermediate) child CUs. Within the scope of this embodiment, the switching between the dual luminance / chrominance coding tree and the decoupling tree is determined according to the current composite depth level of the CU relative to the composite depth threshold.

[0057] According to this embodiment, the switching node is signaled. Advantageously, this embodiment allows for the switching node to be adapted to each CTU based, for example, on RD cost.

[0058] Figure 8 shows another example of the encoding method according to at least one embodiment. Figure 8 shows an embodiment of the determination step 720 of the partially shared luminance and chroma coding trees of a given CTU in an encoder. The embodiment in Figure 8 corresponds to an embodiment in which the switching node is determined based on the split block size. This determination process involves a recursive rate-distortion optimization procedure to find two luminance and chroma coding trees that jointly minimize the overall rate-distortion cost at the CTU level. To this end, at each depth level of the recursive process, a set (or part thereof) of candidate split modes of the coding unit under consideration (corresponding to the entire CTU at the start of the process) is assumed. Two main cases are shown in Figure 8, on the left side (yes branching) and on the right side (no branching), respectively.

[0059] The left side corresponds to the case where the size of the current CU is strictly larger than the size of the decoding pipeline unit. In this case, according to at least one embodiment, a single splitting mode commonly used for the luminance and chroma components must be selected. To this end, each splitting mode s or CurrentSplitMode in the set of acceptable candidate splitting modes for the current CU is evaluated, thereby yielding the rate distortion cost RDcost_YCbCr(CurrentSplitMode) associated with the best encoding of the CU under consideration, which can be done when the splitting model s is selected at the current depth level. Thus, the evaluation of the splitting modes s involves splitting the current CU according to the splitting modes s and then looping through each resulting subcoding unit (subCU). For each subCU, the current RDO process is recursively called so that the splitting mode for the subCU under consideration is determined. This loop for the candidate splitting modes then leads to the determination of the splitting mode that minimizes the rate distortion cost of the co-luminance / chroma associated with the current CU.

[0060] For a 128x128 CTU, the quadtree splitting operation divides the 128x128 CTU into four 64x64 coding units. This split is signaled as part of a general saturation / luminance coding tree or inferred to save bits. Note that when the quadtree splitting mode is inferred for CUs larger than the size of the decoding pipeline unit (64x64), the left side of the process in Figure 8 simply consists of splitting the input CU into four sub-CUs and sequentially recursively applying the process in Figure 8 to each resulting sub-CU.

[0061] The right side of the algorithm in Figure 8 corresponds to the case where the entire RD search process reaches the depth level of the coding tree, which corresponds to a coding unit size less than or equal to the size of the decoding pipeline unit under consideration, i.e., a switching node. In that case, at least one embodiment proposes that, in the entire optimized coding tree representation, from the depth level under consideration, switch to luminance and chroma subcoding trees separated from the joint luminance / chroma coding tree. Thus, the determination of the luminance and chroma coding trees associated with the current CU takes the form of two separate, sequential optimization steps to find the best luminance coding tree and the best chroma coding tree for the current CU, respectively.

[0062] This corresponds to the two RDO steps following Figure 8. The optimization step is known to those skilled in the art. The next step involves calculating the overall joint luminance / chroma rate distortion cost RDcost_YCbCr(s_luma, s_chroma) arising from the obtained, separated luminance and chroma subcoding trees. s_luma and s_chroma are the luminance (Y) component and chroma (Cb+Cr) component, respectively, representing the best splitting mode found for the current CU. This joint RD cost can be easily calculated by summing the rates of the luminance and chroma components (giving an overall rate R) and calculating the overall distortion D as the sum of the luminance and chroma distortions arising from two separate coding tree searches. The overall RD cost is then obtained according to a classical formula, where λ is a known Lagrangian parameter. Rdcost = D + λR

[0063] Note that when a quadtree splitting mode is inferred for a CU larger than the decoding pipeline unit size of 64x64, the rate-distorted optimal splitting modes s_luminance and s_chrominance are assigned to the current CU and currCU in the luminance and chrominance channels, respectively. In fact, for CUs larger than the decoding pipeline unit size, there is no rate-distortion competition between splitting modes.

[0064] The first module / operation in Figure 8 (whether there is a possibility of sharing the luminance / saturation coding tree) can be located, for example, in module 102 (image segmentation) in Figure 13 (shown later).

[0065] 3. Decryption Embodiment Figure 9 shows an example of a decoding method 900 according to a general aspect of at least one embodiment.

[0066] In preliminary step 910, the bitstream or signal of the coded block is accessed for decoding. The coded block is typically a CU or CTU. The decoding method 900 recursively determines, for the block to be decoded, whether to split or divide into subblocks. For example, the coded block is split into subblocks according to one of the splitting modes described in Figures 2, 3, or 4. Thus, in step 920, a split tree structure corresponding to the block splitting process is determined. Advantageously, the split tree structure is partially shared between the luminance block and the saturation block. For example, the split tree structure is derived from the decoding process and syntactic elements associated with the coded block. This will be described later in the modified embodiments shown in Figures 10-13. In step 930, the block is decoded based on at least the determined split tree structure.

[0067] Those skilled in the art will readily adapt the modified embodiments of the encoding method described herein to the decoding method 900, which is not repeated herein.

[0068] Figure 10 shows another example of a decoding method according to a general aspect of at least one embodiment. Figure 10 shows an embodiment of step 920 of determining the coding tree of partially shared luminance and saturation of a given CTU in a decoder. The embodiment in Figure 10 corresponds to an embodiment in which the switching node is determined based on the split block size. With respect to coding, this process also consists of two parts, shown on the left and right sides of the drawing in Figure 10, respectively. The left side of Figure 10 shows the processing of CUs whose width or height is larger than the decoding pipeline unit under consideration. The right side of Figure 10 shows the decoding of coding units less than or equal to the size of the decoding pipeline unit under consideration. Thus, the left side represents a recursive analysis of the split mode, and if the size of the CU is larger than the size of the decoding unit in step 921, the CU can be decoded. Decoding of such a larger CU 921 is shown in Figure 11. According to the modified version, the quadtree splitting operation divides the CU into subblocks equal to the DPU size (for example, in the case of a 128x128 CTU, the quadtree splitting operation divides the 128x128 CTU into four 64x64 coded units).

[0069] According to another variation, the quadtree splitting mode may be inferred to be for CUs larger than the DPU size, with respect to the encoder side.

[0070] The right side of Figure 10 involves the analysis, decoding, and reconstruction of coded units contained within the considered picture region, continuously and independently, for the luminance component, and then for the saturation component.

[0071] The modules / operations in Figures 10 and 11 can be positioned, for example, between modules 230 (entropy decoding including parsing of syntactic elements) and 235 (partitioning) in Figure 14 (described later).

[0072] 4. Alternative Embodiments of Decryption This section presents an alternative embodiment of the decoding process in Section 3. The difference here is that the block decoding process is performed on a decoding pipeline basis to explicitly comply with the constraints considered in the decoding process. In fact, a decoder compatible with embodiments of this principle can favorably perform all decoding processes for the luminance and chrominance components of a decoding unit before initiating the process for the next decoding unit.

[0073] Figure 12 shows a decoding unit-based process used to decode and reconstruct luminance blocks and chrominance blocks according to an embodiment of the present invention. In this process, the CTU is first spatially divided into decoding units equal in size to the decoding pipeline unit, typically 64 × 64.

[0074] For example, this splitting is performed by a quadtree splitting operation, which divides a 128x128 CTU into four 64x64 coded units.

[0075] This process involves looping through all decoding units (typically 64x64 in size) contained within the CTU being considered. For each decoding unit, the luminance blocks contained within the decoding unit are decoded first, followed by the chroma blocks contained within the decoding unit, and then reconstructed.

[0076] 5. Additional embodiments and information This section describes a wide variety of embodiments, including tools, features, embodiments, models, and approaches. Many of these embodiments are described specifically, often in a way that sounds restrictive, at least to illustrate their individual characteristics. However, this is for clarity and not to limit the application or scope of these embodiments. In fact, all of the various embodiments can be combined and interchangeable to provide further embodiments. Furthermore, these embodiments can also be combined and interchangeable with embodiments described in previous applications.

[0077] The embodiments described and intended in this document can be implemented in many different forms. Figures 13, 14, and 15 below provide some embodiments, but other embodiments are intended, and the consideration of Figures 13, 14, and 15 is not intended to limit the scope of implementation forms. At least one of these embodiments generally relates to video encoding and decoding, and at least one other embodiment generally relates to transmitting a generated or encoded bitstream. These and other embodiments can be implemented as a computer-readable storage medium storing instructions for encoding or decoding video data according to any of the methods, apparatus, or described methods, and / or a computer-readable storage medium storing a bitstream generated according to any of the described methods.

[0078] In this application, the terms “reconstructed” and “decoded” may be used interchangeably, the terms “pixel” and “sample” may be used interchangeably, and the terms “image,” “picture,” and “frame” may be used interchangeably. While not always the case, the term “reconstructed” is typically used on the encoder side, while “decoded” is used on the decoder side.

[0079] Various methods are described above, each method including one or more steps or actions to achieve the described method. Unless a specific order of steps or actions is required for the proper operation of this method, any particular order of steps or actions and / or use may be modified or combined.

[0080] Using the various methods and other embodiments described herein, modules such as the split modules (102, 235) and the entropy decoding module (230) of the JVET or HEVC encoder 100 and decoder 200 can be modified, as shown in Figures 13 and 14. Furthermore, these embodiments are not limited to JVET or HEVC and can be applied to other standards and recommendations, whether existing or future, as well as extensions of such standards and recommendations (including JVET and HEVC). Unless otherwise specified or technically excluded, the embodiments described herein can be used individually or in combination.

[0081] This book uses various numerical values, such as the pipeline or block partitioning (CU, TU) size for a 64x64 decoding unit. Specific values ​​are provided for illustrative purposes only, and the embodiments described are not limited to these specific values.

[0082] Figure 13 shows an exemplary encoder 100. While variations of this encoder 100 are intended, the encoder 100 is described below for clarity without explaining all expected variations.

[0083] Before encoding, the video sequence may undergo pre-coding (101), for example, applying a color conversion to the input color picture (e.g., from RGB 4:4:4 to YCbCr 4:2:0), or performing a remapping of the input picture components to obtain a signal distribution that is more resilient to compression (e.g., using one histogram equalization of the color components). Metadata may be associated with the pre-processing and attached to the bitstream.

[0084] In an exemplary encoder 100, a picture is encoded by encoder elements as described below. The encoded picture is divided (102) and processed, for example, in units of CUs. Each unit is encoded, for example, using either intra-mode or inter-mode. When a unit is encoded in intra-mode, intra-prediction is performed (160). In inter-mode, motion estimation (175) and motion compensation (170) are performed. The encoder determines whether to use intra-mode or inter-mode for encoding the unit (105), and indicates the intra / inter decision, for example, by a prediction mode flag. The prediction residual is calculated, for example, by subtracting the predicted blocks from the original image blocks (110).

[0085] Next, the predicted residuals are transformed (125) and quantized (130). In addition to the quantized transformed coefficients, the motion vector and other syntactic elements are entropy encoded to output a bitstream (145). The encoder may skip the transformation and apply quantization directly to the untransformed residual signal. The encoder may also bypass both the transformation and quantization, i.e., the residuals are encoded directly without applying any transformation or quantization process.

[0086] The encoder decodes the coded blocks and provides a reference for further prediction. The quantized transformation coefficients are dequantized (140), inversely transformed (150), and the prediction residuals are decoded. The decoded prediction residuals and the predicted blocks are combined (155) to reconstruct the image blocks. An in-loop filter (165) is applied to the reconstructed picture to reduce coded artifacts, for example, by performing deblocking / SAO (sample adaptive offset) filtering. The filtered image is stored in a reference picture buffer (180).

[0087] Figure 14 shows a block diagram of an exemplary video decoder 200. In the exemplary decoder 200, the bitstream is decoded by the decoder elements as described below. The video decoder 200 generally performs a decoding pass which is the reverse of the encoding pass as described in Figure 13. The encoder 100 also generally performs video decoding as part of encoding the video data.

[0088] In particular, the decoder input includes a video bitstream that may be generated by the video encoder 100. First, the bitstream is entropy-decoded (230) to obtain transformation coefficients, motion vectors, and other encoded information. Picture segmentation information indicates how the picture is segmented. Thus, the decoder can segment the picture according to the decoded picture segmentation information (235). The transformation coefficients are dequantized (240), inversely transformed (250), and the prediction residuals are decoded. The decoded prediction residuals and the predicted blocks are combined (255) to reconstruct the image blocks. The predicted blocks may be obtained from intra-prediction (260) or motion-compensated prediction (i.e., inter-prediction) (275) (270). An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (280).

[0089] The decoded picture may undergo further decoded post-processing (285), such as reverse color conversion (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4), or reverse remapping, which performs the reverse of the remapping process performed in the pre-coding process (101). The decoded post-processing may use metadata derived in the pre-coding process and signaled in the bitstream.

[0090] Figure 15 shows a block diagram of an exemplary system in which various embodiments and exemplary designs are implemented. System 1000 can be embodied as a device comprising various components described below and configured to perform one or more of the embodiments described herein. Examples of such devices include, but are not limited to, personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected consumer electronics, and servers. System 1000 can be communicatively coupled to other similar systems and to displays via communication channels as shown in Figure 15 and known to those skilled in the art, and can implement the various embodiments described herein.

[0091] System 1000 may include at least one processor 1010 configured to execute internally loaded instructions in order to implement the various embodiments described in this document. Processor 1010 may include embedded memory, input / output interfaces, and various other circuits, as known in the Art. System 1000 may include at least one memory 1020 (e.g., a volatile memory device, a non-volatile memory device). System 1000 may include a storage device 1040, which may include, but is not limited to, non-volatile memory, including EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash, magnetic disk drives, and / or optical disk drives. Storage device 1040 may include, as an unspecified example, an internal storage device, an attached storage device, and / or a network-accessible storage device. System 1000 may include an encoder / decoder module 1030 configured to process data to provide encoded or decoded video, which may include its own processor and memory.

[0092] The encoder / decoder module 1030 represents a module(s) that may be included in the device to perform encoding and / or decoding functions. As is well known, the device may include one or both of the encoding and decoding modules. Furthermore, the encoder / decoder module 1030 may be implemented as a separate element of system 1000, or it may be incorporated into processor 1010 as a combination of hardware and software known to those skilled in the art.

[0093] Program code loaded onto the processor 1010 to perform the various actions described herein is stored in the storage device 1040 and can then be loaded onto the memory 1020 for execution by the processor 1010. According to an exemplary embodiment, one or more of the processor(s) 1010, memory 1020, storage device 1040, and encoder / decoder module 1030 may store one or more of a variety of items, including but not limited to input video, decoded video, bitstream, equations, formulas, matrices, variables, operations, and operation logic, during the execution of the processes described herein.

[0094] System 1000 may include a communication interface 1050 that enables communication with other devices via a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to send and receive data from the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or a network card, and the communication channel may be implemented within a wired and / or wireless medium. Various components of System 1000 may be connected to or communicated with one another using a variety of suitable connectors, including, but not limited to, internal buses, wires, and printed circuit boards.

[0095] Exemplary embodiments can be implemented by computer software implemented by processor 1010, by hardware, or by a combination of hardware and software. In a non-limiting example, exemplary embodiments may be implemented by one or more integrated circuits. Memory 1020 can be of any type appropriate to the technical environment and, in a non-limiting example, can be implemented using any suitable data storage technology such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. Processor 1010 can be of any type appropriate to the technical environment and, in a non-limiting example, may include one or more microprocessors, general-purpose computers, dedicated computers, and processors based on multi-core architectures.

[0096] The implementation forms and embodiments described herein may be implemented, for example, in methods or processes, apparatus, software programs, data streams, or signals. Even when considered only in the context of a single implementation form (e.g., considered only as a method), the implementation form of the considered feature may also be implemented in other forms (e.g., apparatus or program). Apparatus may be implemented, for example, in appropriate hardware, software, and firmware. Methods may be implemented in apparatus such as a processor, which generally refers to a processing device including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as computers, mobile phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate the transmission of information between end users.

[0097] The phrases "one embodiment," "an embodiment," "one implementation," or "an implementation," as well as references to other variations thereof, mean that the specific features, structures, characteristics, etc., described in relation to the embodiment are included in at least one embodiment. Therefore, the appearance of phrases such as "in one embodiment," "in one embodiment," or "in one implementation," as well as any other variations, found in various places throughout this document, do not necessarily all refer to the same embodiment.

[0098] Furthermore, this document may refer to "determining" various parts of information. Determining information may include, for example, one or more of the following: estimating information, calculating information, predicting information, or retrieving information from memory.

[0099] Furthermore, this document may refer to "accessing" various parts of information. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0100] Furthermore, this document may refer to "receiving" various parts of information. Receiving is intended to be a broad term, similar to "accessing." Receiving information may include, for example, one or more of the following: accessing information, or retrieving information (for example, from memory). Moreover, "receiving" is typically involved in some way during operations such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0101] Furthermore, this document may refer to optimization, and various embodiments do not need to be fully optimized; some embodiments simply select a mode that is not optimized in the usual sense.

[0102] As will be apparent to those skilled in the art, the implementation can generate a wide variety of signals, for example, that are formatted to carry information that can be stored or transmitted. The information may include, for example, instructions for performing a method, or data generated by one of the implementations described. For example, the signal may be formatted to carry the bitstream of the embodiment described. Such a signal may be formatted, for example, as an electromagnetic wave (using, for example, the radio frequency portion of the spectrum) or as a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. The signal may be transmitted over a variety of different wired or wireless links, as is known. The signal may be stored on a processor-readable medium.

[0103] Several embodiments have been described. These embodiments provide at least the following generalized inventions and claims, including all combinations across various different claim categories and types. - Modify the luminance and saturation coding trees applied to the decoder and / or encoder. - Combining interleaved luminance / saturation coding with isolated luminance / saturation coding applied to decoders and / or encoders. - A combination of interleaved luminance / saturation coding and isolated luminance / saturation coding, where the isolated luminance / saturation coding is controlled by information regarding the constraints of the decoding unit pipeline applied to the decoder and / or encoder. - To jointly determine, encode, and signal luminance and chroma encoding trees from the maximum allowable block size (CTU size) down to a given block size or a given CU depth level, and to independently determine, encode, and signal luminance and chroma encoding trees for blocks smaller than the given block size or exceeding a given CU depth level. Switching from interleaved luminance / saturation encoding to isolated luminance / saturation encoding, according to the block size associated with the processed encoding tree node, or according to the depth level of the encoding tree node being encoded / decoded. • To enable several partially shared luminance and saturation encoding tree methods in decoders and / or encoders. • Select the luminance and saturation encoding trees to apply in the decoder based on several syntactic elements. • To create and / or transmit and / or receive and / or decode a bitstream or signal containing one or more of the encoded luminance / chrominance components described therein, or variations thereof. A television, set-top box, mobile phone, tablet, or other electronic device that performs encoding / decoding to enable a partially shared luminance and saturation coding tree method according to any of the embodiments described. A television, set-top box, mobile phone, tablet, or other electronic device that performs encoding / decoding that enables a partially shared luminance and saturation coding tree method according to any of the embodiments described, and displays the resulting image (e.g., using a monitor, screen, or other type of display device). A television, set-top box, mobile phone, tablet, or other electronic device that tunes to a channel (e.g., using a tuner) to receive a signal containing an encoded image and performs decoding that enables a partially shared luminance and saturation encoded tree method according to any of the embodiments described. A television, set-top box, mobile phone, tablet, or other electronic device that receives a signal containing an encoded image via terrestrial broadcast (for example, using an antenna) and performs decoding that enables a partially shared luminance and saturation encoded tree method according to any of the embodiments described.

[0104] Various other generalized and specific inventions and claims are also supported and contemplated throughout this disclosure.

Claims

1. The process involves dividing the video image into multiple coded tree units, The process involves decoding an indication that, for each I-slice, each coded tree unit is divided into coded units having 64 x 64 luminance samples using a quadtree partition, and that said coded unit is the root of two separate coded tree syntactic structures for luminance and saturation. Decoding the multiple coded tree units based on the aforementioned indication, A method that includes this.

2. The method according to claim 1, further comprising dividing the current coding unit into four equal-sized square coding units, based on the condition that the current coding unit has more than 64 x 64 luminance samples.

3. The method according to claim 1, further comprising dividing the luminance component of the current coded unit using a luminance coded tree structure and dividing the chroma component of the current coded unit using a chroma coded tree structure, based on the condition that the current coded unit has luminance samples of 64 × 64 or less.

4. The method according to claim 3, wherein the division of the chroma component of the current coding unit using a chroma coding tree structure further comprises at least decoding an indication that the chroma component of the current coding unit is further divided.

5. The method according to claim 1, wherein the indication is decoded from a syntactic element.

6. It is a device, Memory and Equipped with one or more processors, The aforementioned one or more processors The process involves dividing the video image into multiple coded tree units, The process involves decoding an indication that, for each I-slice, each coded tree unit is divided into coded units having 64 x 64 luminance samples using a quadtree partition, and that said coded unit is the root of two separate coded tree syntactic structures for luminance and saturation. Decoding the multiple coded tree units based on the aforementioned indication, A device configured to perform the following actions.

7. The apparatus according to claim 6, wherein one or more processors are further configured to divide the current coded unit into four equal-sized square coded units, based on the condition that the current coded unit has more than 64 x 64 luminance samples.

8. The apparatus according to claim 6, wherein the one or more processors are further configured to divide the luminance component of the current coded unit using a luminance coded tree structure and to divide the chroma component of the current coded unit using a chroma coded tree structure, based on the condition that the current coded unit has luminance samples of 64 × 64 or less.

9. The apparatus according to claim 8, wherein, in order to partition the chroma components of the current coding unit using a chroma coding tree structure, one or more processors are further configured to decode an indication that the chroma components of the current coding unit are further partitioned.

10. The apparatus according to claim 6, wherein the indication is decoded from syntactic elements.

11. Determine a partition tree structure that corresponds to the division of a video image from one encoded tree unit into multiple encoded units, For each I-slice, an indication is encoded that each coded tree unit is divided into coded units having 64 x 64 luminance samples using a quadtree partition, and that the coded unit is the root of two separate coded tree syntactic structures for luminance and saturation. Encoding the coded tree unit based on the aforementioned indication, A method that includes this.

12. The method according to claim 11, further comprising determining that the current coding unit is divided into four equal-sized square coding units based on the condition that the current coding unit has more than 64 x 64 luminance samples.

13. The method according to claim 11, further comprising determining, based on the condition that the current coded unit has luminance samples of 64 × 64 or less, that the luminance components of the current coded unit are divided using a luminance coded tree structure, and determining that the chroma components of the current coded unit are divided using a chroma coded tree structure.

14. The method according to claim 13, further comprising encoding an indication that the saturation component of the current coded unit is further divided.

15. The method according to claim 11, wherein the indication is encoded as a syntactic element.

16. It is a device, Memory and Equipped with one or more processors, The aforementioned one or more processors Determine a partition tree structure that corresponds to the division of a video image from one encoded tree unit into multiple encoded units, For each I-slice, an indication is encoded that each coded tree unit is divided into coded units having 64 x 64 luminance samples using a quadtree partition, and that the coded unit is the root of two separate coded tree syntactic structures for luminance and saturation. Encoding the coded tree unit based on the aforementioned indication, A device configured to perform the following actions.

17. The apparatus according to claim 16, wherein one or more processors are further configured to determine that the current coded unit is divided into four equal-sized square coded units, on the condition that the current coded unit has more than 64 x 64 luminance samples.

18. The apparatus according to claim 16, wherein the one or more processors are further configured to determine, based on the condition that the current coded unit has luminance samples of 64 × 64 or less, that the luminance component of the current coded unit is divided using a luminance coded tree structure, and to determine that the chroma component of the current coded unit is divided using a chroma coded tree structure.

19. The apparatus according to claim 18, wherein the one or more processors are further configured to encode at least an indication that the saturation component of the current coded unit is further divided.

20. The apparatus according to claim 16, wherein the indication is encoded as a syntactic element.

Images