Relationships between partition constraint elements

Abstract

To provide video coding and picture partitioning methods.SOLUTION: In particular, the invention is concerned with relations between partition constraint elements, by setting partitioning rules for different picture partitioning methods. The invention provides devices and corresponding methods for generating or processing a bitstream including encoded pictures, particularly for setting and including partition constraint element into the bitstream. One of the devices is configured to determine a minimum luma size of a leaf block resulting from quadtree splitting (MinQtSizeY), determine a maximum luma size of a coding block to be split using a binary tree splitting (MaxBtSizeY) based on the MinQtSizeY, and include information for the determined MinQtSizeY into the bitstream.SELECTED DRAWING: Figure 13

Description

[Technical Field]

[0001] TECHNICAL FIELD This disclosure relates generally to video encoding, decoding, and picture partitioning methods. [Background technology]

[0002] The amount of video data required to depict even a relatively short video is substantial, which can create challenges when the data is streamed or otherwise communicated across communication networks with limited bandwidth capacity. As such, video data is typically compressed before being communicated across today's communication networks. Because memory resources may be limited, the size of the video can also become an issue when the video is stored on a storage device. Video compression devices often use software and / or hardware to encode the video data at the source prior to transmission or storage, thereby reducing the amount of data required to represent a digital video image. The compressed data is then received by a video decompression device, which decodes the video data at the destination. With limited network resources and an increasing demand for higher video quality, improved compression and decompression techniques that increase compression ratios with little or no sacrifice in image quality are desirable.

[0003] Conventionally, there are many partition constraint elements, each of which constrains the partitioning of a picture (video partitioning) for various types of partitioning methods. These various types of partitioning methods include binary tree partitioning, quad tree partitioning, and ternary tree partitioning, among others. The partition constraint elements are usually signaled in a bitstream containing coded pictures. In the following, some important partition constraint elements are described.

[0004] A minimum coding block size (MinCbSizeY) may be defined. As an example, MinCbSizeY may be equal to 8, meaning that a parent block of size 8x8 cannot be divided using any of the partitioning (splitting) methods, since the resulting child blocks are guaranteed to be smaller (in either width or height) than MinCbSizeY. According to a second example, if MinCbSizeY is equal to 8, a parent block of size 8x16 cannot be divided using quadtree splitting (partitioning), for example, since the resulting four child blocks would have sizes of 4x8 (width equal to 4 and height equal to 8), and the widths of the resulting child blocks would be smaller than MinCbSizeY. In the second example, it was assumed that MinCbSizeY applies to both the width and height of the block, but two different syntax elements can be used to restrict the width and height separately.

[0005] The maximum coding tree block size (CtbSizeY) may indicate the size of the largest coding block in units of the number of luma samples.

[0006] The maximum binary tree size (MaxBtSizeY) may be defined as the maximum luma size (width or height) in units of number of samples of a coding block that can be divided using a binary division method. As an example, if MaxBtSizeY is equal to 64, coding blocks that are larger in size in either width or height cannot be divided using binary division. This means that a block of size 128x128 cannot be divided using binary division, but a block of size 64x64 can be divided using binary division.

[0007] The minimum binary tree size (MinBtSizeY) may be defined as the minimum luma size (width or height) in units of number of samples of a coding block that can be divided using a binary division method. As an example, if MinBtSizeY is equal to 16, coding blocks that are smaller or equal in size in either width or height cannot be divided using binary division. This means that a block of size 8x8 cannot be divided using binary division, but a block of size 32x32 can be divided using binary division.

[0008] The minimum quadtree size (MinQtSizeY) may be defined as the minimum luma size of a leaf block resulting from a quadtree division of a coding tree unit (CTU). The size can indicate either the width or height of the block in number of samples. It may indicate the width and height together in the case of a square block. As an example, if MinQtSizeY is equal to 16, coding blocks of size 16 or less cannot be divided into child blocks using the quadtree division method. Conventionally, MinQtSizeY (as well as the signaled syntax elements "log2_min_qt_size_intra_slices_minus2" and "log2_min_qt_size_inter_slices_minus2") are used to indicate the minimum quadtree block size. In particular, the size syntax element may be an indirect syntax element, meaning that log2_min_qt_size_intra_slices_minus2 may be the binary logarithm (base 2) of the number of luma samples in the minimum quadtree block.

[0009] The minimum transform block size (MinTbSizeY) may be defined as the smallest transform block size, in units of number of samples, of a coding block that can be divided using the ternary partitioning method. As an example, if MinTbSizeY is equal to 16, coding blocks that are smaller or equal in size in either width or height cannot be divided using ternary partitioning. This means that a block of size 8x8 cannot be divided using ternary partitioning, but a block of size 32x32 can be divided using ternary partitioning.

[0010] The maximum multitype tree depth (MaxMttDepth) may be defined as the maximum hierarchical depth for a coding unit resulting from a multitype tree division of a quadtree leaf or CTU. A CTU or coding tree block (CTB) describes the maximum block size used to divide a picture frame. MaxMttDepth describes an upper bound on the number of consecutive binary or ternary divisions that can be applied to obtain child blocks. As an example, assuming a CTU size of 128x128 (width equals 128 and height equals 128) and MaxMttDepth is 1, each parent block (size 128x128) can first be divided into two 128x64 child blocks using binary division. However, the child block cannot apply consecutive binary divisions (to become either a 128x32 or 64x64 child block) because the maximum number of allowed binary divisions has been reached. Note that MaxMttDepth can control the maximum binary split depth, the maximum ternary split depth, or both simultaneously. If it controls both binary and ternary split depth simultaneously, one binary split followed by one ternary split can count as two hierarchical splits. Conventionally, MaxMttDepth (and its syntax elements "max_mtt_hierarchy_depth_inter_slices" and "max_mtt_hierarchy_depth_intra_slices") is used to indicate the maximum hierarchical depth for coding units resulting from a multitype tree.

[0011] Furthermore, "pic_width_in_luma_samples" is a picture size element, i.e., a syntax element that specifies the width of each decoded picture in units of luma samples. pic_width_in_luma_samples must typically be non-equal to 0 and an integer multiple of MinCbSizeY.

[0012] Similarly, "pic_height_in_luma_samples" is a picture size element, i.e., a syntax element that specifies the height of each decoded picture in units of luma samples. pic_height_in_luma_samples must typically be non-equal to 0 and an integer multiple of MinCbSizeY.

[0013] One of the goals of video coding is to provide the highest possible quality at the lowest possible rate, and one of the factors that can contribute to this goal is to increase the efficiency of the bitstream structure. Summary of the Invention

[0014] In view of the above problems, embodiments of the present invention aim to improve current implementations of picture partitioning. In particular, the aim is to increase the availability and flexibility of various picture partitioning methods, namely, binary tree partitioning, quad tree partitioning, and ternary tree partitioning methods. The goal is to enable encoding and decoding of more picture sizes. [Means for solving the problem]

[0015] Embodiments of the invention are provided in the accompanying independent claims. Advantageous implementations of the invention are further defined in the dependent claims.

[0016] These and other objects are achieved by the subject matter of the independent claims. Further embodiments are evident from the dependent claims, the description and the figures.

[0017] Particular embodiments are outlined in the accompanying independent claims, while other embodiments are outlined in the dependent claims.

[0018] In embodiments of the present invention, new partitioning rules are established, particularly by setting relationships between partitioning constraint elements and signaling them in a bitstream along with the coded pictures, which may be used to encode video data into a bitstream and decode the bitstream into decoded video data.

[0019] According to a first aspect, the present invention relates to a device for decoding or processing a bitstream containing coded pictures, the device including circuitry configured to: obtain syntax elements from the bitstream (101), obtain information about a minimum size in luma samples of a luma root block resulting from quadtree partitioning, MinQtSizeY (105), and determine, based on the information about MinQtSizeY (105) and the obtained syntax elements, a maximum size in luma samples of a luma root block that can be partitioned using binary tree partitioning, MaxBtSizeY (102).

[0020] By defining new partitioning rules through setting relationships between these partitioning constraint elements, the decoding device facilitates increased availability and flexibility of various picture partitioning methods, particularly quad-tree partitioning and binary tree partitioning.

[0021] In a possible implementation of the method according to the first aspect, the circuitry is configured to determine MaxBtSizeY taking into account that its lower bound is MinQtSizeY.

[0022] In a possible implementation of the method according to the first aspect or the above implementation, the syntax element is a syntax element (301) of the difference between the base 2 logarithm of MinQtSizeY (105) and the base 2 logarithm of MaxBtSizeY (102).

[0023] In this way, the relevant partitioning constraint elements can be easily inferred, for example, at the decoder side, while reducing the information overhead in the bitstream. A difference is an example of a relationship. However, the relationship could also be a proportionality factor, a calculation method, etc., which allows inferring MaxBtSizeY from MinQtSizeY.

[0024] In a possible implementation of the method according to the first aspect or the above implementation, the syntax element is a difference syntax element (301) between MinQtSizeY (105) and MaxBtSizeY (102), i.e. the difference syntax element may be log2_diff_max_bt_size_min_qt_size, signaling said difference in a logarithmic scale to the base 2. Such a difference syntax element provides a compact way of signaling the difference.

[0025] In a possible implementation of the method according to the first aspect or the above implementation, the circuit is configured to obtain syntax elements from the bitstream that depend on the maximum hierarchical depth (MaxMttDepth) for coding units resulting from multi-type tree partitioning of quadtree leaf blocks. In this way, the relevant partitioning constraint elements can be easily inferred, for example, at the decoder side, while reducing information overhead in the bitstream.

[0026] Additionally or alternatively, if MaxMttDepth is equal to zero, the circuitry may be configured to not retrieve any MaxBtSizeY syntax elements from the bitstream.

[0027] Additionally or alternatively, if MaxMttDepth (103) is not equal to zero, the circuitry may be configured to not retrieve any syntax elements from the bitstream (101).

[0028] According to a second aspect, the invention relates to a device for generating or processing a bitstream comprising coded pictures, the device being configured to determine a minimum luma size MinQtSizeY of leaf blocks resulting from quadtree partitioning, to determine based on MinQtSizeY a maximum luma size MaxBtSizeY of coding blocks partitioned using binary tree partitioning, and to include information on the determined MinQtSizeY in the bitstream.

[0029] This method allows for the implementation of an efficient encoder where the generated stream has a compact syntax and allows the decoder to efficiently infer the constraint parameters. The above advantages on the decoder side also apply to the encoder side where the bitstream is generated.

[0030] In a possible implementation of the method according to the second aspect or the above implementations, the device (its processing circuitry) is configured to determine MaxBtSizeY taking into account that its lower limit is MinQtSizeY.

[0031] In a possible implementation of the second aspect or the method according to the above implementation, the device for generating or processing a bitstream may further be configured to include in the bitstream (101) a syntax element of the difference between the base 2 logarithm of MinQtSizeY (105) and the base 2 logarithm of MaxBtSizeY (102).

[0032] In a possible implementation of the method according to the second aspect or the above implementation, the syntax element is a syntax element (301) of the difference between MinQtSizeY (105) and MaxBtSizeY (102), the difference syntax element signaling said difference on a logarithmic scale to the base 2.

[0033] In a possible implementation form of the method according to the second aspect or the above implementation, the device is configured to include in the bitstream a syntax element of MaxBtSizeY that depends on the maximum hierarchical depth MaxMttDepth for the coding unit resulting from the multi-type tree split.

[0034] Additionally or alternatively, if MaxMttDepth is equal to zero, the device is configured not to include any MaxBtSizeY syntax elements in the bitstream.

[0035] Additionally or alternatively, if MaxMttDepth is not equal to zero, the device is configured to include MaxBtSizeY optional syntax elements in the bitstream.

[0036] According to a third aspect, the invention relates to a method provided for generating or processing a bitstream containing coded pictures, comprising the steps of determining a minimum luma size MinQtSizeY of leaf blocks resulting from quadtree partitioning, determining based on MinQtSizeY a maximum luma size MaxBtSizeY of coding blocks partitioned using binary tree partitioning, and including information about the determined MinQtSizeY (105) in the bitstream.

[0037] According to a fourth aspect, the present invention relates to a method provided for decoding or processing a bitstream containing coded pictures, comprising the steps of: obtaining syntax elements from a bitstream (101); obtaining information on a minimum size in luma samples of a luma root block resulting from quadtree partitioning, MinQtSizeY (105); and determining a maximum size in luma samples of a luma root block, MaxBtSizeY (102), that can be partitioned using binary tree partitioning based on the information on MinQtSizeY (105) and the obtained syntax elements.

[0038] The method may further include determining whether binary partitioning is allowed to be applied to the picture block based on MaxBtSizeY, obtaining a coding block of the picture block based on the determination result, and obtaining reconstructed sample values ​​of the coding block.

[0039] The picture block may be a luma block of a coded picture.

[0040] The syntax element can specify the difference between the base 2 logarithm of MaxBtSizeY (102) and the base 2 logarithm of MinQtSizeY (105), or the syntax element can specify the difference between MaxBtSizeY and MinQtSizeY.

[0041] The syntax element may be from a slice header of the bitstream.

[0042] The method according to the third aspect of the invention can be performed by the apparatus according to the first aspect of the invention. Further features and implementations of the method according to the first aspect of the invention correspond to the features and implementations of the apparatus according to the third aspect of the invention.

[0043] The method according to the fourth aspect of the invention can be performed by the apparatus according to the second aspect of the invention. Further features and implementations of the method according to the second aspect of the invention correspond to the features and implementations of the apparatus according to the fourth aspect of the invention.

[0044] The method according to the second aspect can be extended to an implementation form corresponding to an implementation form of the first device according to the first aspect, and thus the implementation form of the method includes the features of the corresponding implementation form of the first device.

[0045] The advantages of the method according to the fourth aspect are the same as those for the corresponding implementation of the method according to the third aspect.

[0046] According to a fifth aspect, the present invention relates to an apparatus for decoding a video stream, comprising a processor and a memory, the memory storing instructions for causing the processor to carry out the method according to the third aspect.

[0047] According to a sixth aspect, the present invention relates to an apparatus for encoding a video stream, comprising a processor and a memory, the memory storing instructions for causing the processor to carry out the method according to the fourth aspect.

[0048] According to a seventh aspect, there is proposed a computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors to code video data, the instructions causing the one or more processors to perform a method according to the third or fourth aspect or any possible embodiment of the third or fourth aspect.

[0049] According to an eighth aspect, the present invention relates to a computer program comprising a program code for performing, when the computer program is run on a computer, the method according to the third or fourth aspect or any possible embodiment of the third or fourth aspect.

[0050] The details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will become apparent from the description, drawings, and claims.

[0051] According to one embodiment there is provided a computer program product comprising program code for controlling a device to perform any of the above methods when the program code is executed by one or more processors of the device.

[0052] For clarity, any one of the embodiments disclosed herein may be combined with any one or more of the other embodiments to create new embodiments within the scope of the present disclosure.

[0053] These and other features will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings and claims.

[0054] It should be noted that all devices, elements, units, and means described in this application can be implemented in software elements or hardware elements, or any kind of combination thereof. All steps performed by various entities described in this application, and functions described as being performed by various entities, mean that the respective entities are adapted or configured to perform the respective steps and functions. In the following description of specific embodiments, even if a specific function or step performed by an external entity is not reflected in the description of the specific detailed element of that entity that performs the specific step or function, it should be apparent to those skilled in the art that these methods and functions can be implemented in the respective software elements or hardware elements, or any kind of combination thereof.

[0055] The above aspects and embodiments are explained in the following description of specific embodiments in connection with the accompanying drawings. [Brief explanation of the drawings]

[0056] [Figure 1] FIG. 1 illustrates a device according to one embodiment of the present invention. [Figure 2] FIG. 10 illustrates an SPS RBSP syntax for a device according to a second specific embodiment of the present invention. [Figure 3] FIG. 10 illustrates a slice header syntax for a device according to a second specific embodiment of the present invention. [Figure 4] FIG. 10 illustrates an SPS RBSP syntax for a device according to a third specific embodiment of the present invention. [Figure 5] FIG. 10 illustrates a slice header syntax for a device according to a fourth specific embodiment of the present invention. [Figure 6]FIG. 10 illustrates a slice header syntax for a device according to a fourth specific embodiment of the present invention. [Figure 7] FIG. 1 illustrates a method according to one embodiment of the present invention. [Figure 8] FIG. 1 illustrates a conventional SPS RBSP syntax. [Figure 9] FIG. 1 illustrates a conventional slice header syntax. [Figure 10] FIG. 1 illustrates a device according to one embodiment of the present invention. [Figure 11A] FIG. 1 is a block diagram illustrating an exemplary coding system in which embodiments of the present disclosure can be implemented. [Figure 11B] FIG. 1 is a block diagram illustrating another exemplary coding system in which embodiments of the present disclosure may be implemented. [Figure 12] FIG. 1 is a block diagram illustrating an exemplary video encoder capable of implementing embodiments of the present disclosure. [Figure 13] FIG. 2 is a block diagram illustrating an example of a video decoder capable of implementing embodiments of the present disclosure. [Figure 14] FIG. 1 is a schematic diagram of a network device according to one embodiment of the present disclosure. [Figure 15] 11B is a simplified block diagram of an apparatus that may be used as either or both of source device 12 and destination device 14 from FIG. 11A, according to an exemplary embodiment. [Figure 16A] A diagram showing various CU partition modes in VVC. [Figure 16B] A diagram showing various CU partition modes in VVC. [Figure 16C] A diagram showing various CU partition modes in VVC. [Figure 16D] A diagram showing various CU partition modes in VVC. [Figure 16E] A diagram showing various CU partition modes in VVC. [Figure 16F] A diagram showing various CU partition modes in VVC. [Figure 17A]HD (1920x1080) lower boundary CTU (128x128) forced QT segmentation. [Figure 17B] FIG. 10 illustrates a HD (1920×1080) bottom-bound CTU (128×128) forced BT split according to one embodiment of the present disclosure. [Figure 18] FIG. 1 illustrates an exemplary boundary definition. [Figure 19A] FIG. 10 illustrates an example of a corner case forced QTBT split, according to one embodiment of the present disclosure. [Figure 19B] FIG. 10 illustrates an example of a forced QTBT split for a block located at a corner, according to one embodiment of the present disclosure. [Figure 20] FIG. 1 illustrates one embodiment of a boundary definition. [Figure 21] FIG. 1 is an illustrative diagram of an example of block division using a quadtree-binary tree (QTBT) structure. [Figure 22] FIG. 7 is an illustrative diagram of an example of a tree structure corresponding to block partitioning using the QTBT structure of FIG. 6. [Figure 23] FIG. 10 is an illustrative diagram of an example of a horizontal ternary tree division type. [Figure 24] FIG. 10 is an illustrative diagram of an example of a vertical ternary tree division type. [Figure 25] 1 is a block diagram illustrating an example of a video encoder configured to implement embodiments of the present invention. [Figure 26] 1 is a block diagram illustrating an exemplary configuration of a video decoder configured to implement embodiments of the present invention. [Figure 27] 31 is a block diagram illustrating an exemplary structure of a content supply system 3100 for implementing a content distribution service. [Figure 28] FIG. 2 is a block diagram illustrating the structure of an example of a terminal device. DETAILED DESCRIPTION OF THE INVENTION

[0057] While exemplary implementations of one or more embodiments are provided below, it should be understood at the outset that the disclosed systems and / or methods may be implemented using any number of techniques, whether currently known or in existence. The present disclosure should in no way be limited to the exemplary implementations, drawings, and techniques shown below, including the exemplary designs and implementations shown and described herein, but may be modified within the scope of the appended claims along with their full scope of equivalents.

[0058] In the following description, reference is made to the accompanying drawings, which form a part of this disclosure and which show, by way of illustration, specific aspects of embodiments of the present invention or in which embodiments of the present invention may be used. It is understood that embodiments of the present invention may be used in other ways and may include structural or logical changes not depicted in the drawings. Therefore, the following detailed description is not to be taken in a limiting sense, and the scope of the present invention is defined by the appended claims.

[0059] For example, it is understood that disclosure related to a described method may also apply to a corresponding device or system configured to perform the method, and vice versa. For example, when one or more particular method steps are described, a corresponding device may include one or more units, e.g., functional units, for performing the described one or more method steps (e.g., one unit that performs one or more steps, or multiple units that each perform one or more of the steps), even if such one or more units are not explicitly described or shown in a figure. Conversely, for example, when a particular apparatus is described based on one or more units, e.g., functional units, a corresponding method may include one step for performing the function of the one or more units (e.g., one step that performs the function of one or more units, or multiple steps that each perform one or more functions of multiple units), even if such one or more steps are not explicitly described or shown in a figure. Furthermore, it is understood that features of various exemplary embodiments and / or aspects described herein may be combined with each other, unless otherwise specified.

[0060] Video coding typically refers to the processing of a video or a series of pictures that form a video sequence. Instead of the term "picture," the terms "frame" or "image" may be used as synonyms in the field of video coding. Video coding, as used in this application (or this disclosure), refers to either video encoding or video decoding. Video encoding is performed at the source side and typically involves processing of an original video picture (e.g., by compression) to reduce the amount of data needed to represent the video picture (for more efficient storage and / or transmission). Video decoding is performed at the destination side and typically involves the reverse process compared to an encoder to reconstruct the video picture. Embodiments that refer to "coding" a video picture (or, as described later, pictures in general) should be understood to relate to either "encoding" or "decoding" a video picture. The combination of the encoding and decoding parts is also referred to as CODEC (Coding and Decoding).

[0061] In the case of lossless video coding, the original video picture can be restored, i.e., the restored video picture has the same quality as the original video picture (assuming there is no transmission loss or other data loss during storage or transmission). In the case of lossy video coding, further compression is performed, e.g., by quantization, to reduce the amount of data representing the video picture, but the video picture cannot be perfectly restored at the decoder, i.e., the quality of the restored video picture is low or poor compared to the quality of the original video picture.

[0062] Several video coding standards since H.261 belong to the group of "lossy hybrid video codecs" (i.e., they combine spatial and temporal prediction in the sample domain with 2D transform coding to apply quantization in the transform domain). Each picture of a video sequence is typically divided into a set of non-overlapping blocks, and coding is typically performed at the block level. In other words, at the encoder, video is typically processed, i.e., encoded, at the block (video block) level, for example, by using spatial (intra-picture) and temporal (inter-picture) prediction to generate a predictive block, subtracting the predictive block from a current block (the block currently being / to be processed) to obtain a residual block, transforming the residual block, and quantizing the residual block in the transform domain to reduce the amount of data to be transmitted (compression); whereas at the decoder, in order to reconstruct the current block for display, a reverse process is partially applied to the coded or compressed block compared to the encoder. Additionally, the encoder replicates the decoder processing loop so that both produce the same prediction (eg, intra-prediction and inter-prediction) and / or reconstruction for processing, i.e., coding, subsequent blocks.

[0063] The term "block" as used herein may refer to a portion of a picture or a frame. For ease of explanation, embodiments of the present invention are described herein with reference to the High Efficiency Video Coding (HEVC) or Versatile Video Coding (VVC) reference software developed by the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG) Joint Research Team on Video Coding (JCT-VC). Those skilled in the art will understand that embodiments of the present invention are not limited to HEVC or VVC. It may refer to CUs (coding units), PUs (prediction units), and TUs (transform units). In HEVC, coding tree units (CTUs) are divided into CUs using a quadtree structure referred to as a coding tree. The decision of whether to code a picture region using inter-picture (temporal) prediction or intra-picture (spatial) prediction is made at the CU level. Each CU can be further divided into one, two, or four PUs depending on the PU partition type. Within one PU, the same prediction process is applied, and related information is transmitted to the decoder on a PU-by-PU basis. After applying a prediction process based on the PU partition type to obtain residual blocks, the CUs can be divided into transform units (TUs) according to another quadtree structure similar to the coding tree for CUs. In the latest developments in video compression technology, quadtree and binary tree (QTBT) partitioning frames are used to divide coding blocks. In the QTBT block structure, CUs can have either square or rectangular shapes. For example, coding tree units (CTUs) are first divided using a quadtree structure. The quadtree leaf nodes are further divided using a binary tree structure. The binary tree leaf nodes are called coding units (CUs), and their segmentation is used for prediction and transform processing without further division. This means that CUs, PUs, and TUs have the same block size in the QTBT coding block structure. In parallel, multiple divisions, such as ternary tree (TT) division, have also been proposed to be used with the QTBT block structure.The term "device" may also be "apparatus," "decoder," or "encoder."

[0064] In the following embodiments, the encoder 20, the decoder 30, and the coding system 10 are described based on FIGS.

[0065] In the following description, reference is made to the accompanying drawings, which form a part of this disclosure and which show, by way of illustration, specific aspects of embodiments of the present invention or in which embodiments of the present invention may be used. It is understood that embodiments of the present invention may be used in other ways and may include structural or logical changes not depicted in the drawings. Therefore, the following detailed description is not to be taken in a limiting sense, and the scope of the present invention is defined by the appended claims.

[0066] For example, it is understood that disclosure related to a described method may also apply to a corresponding device or system configured to perform the method, and vice versa. For example, when one or more particular method steps are described, a corresponding device may include one or more units, e.g., functional units, for performing the described one or more method steps (e.g., one unit that performs one or more steps, or multiple units that each perform one or more of the steps), even if such one or more units are not explicitly described or shown in a figure. Conversely, for example, when a particular apparatus is described based on one or more units, e.g., functional units, a corresponding method may include one step for performing the function of the one or more units (e.g., one step that performs the function of one or more units, or multiple steps that each perform one or more functions of multiple units), even if such one or more steps are not explicitly described or shown in a figure. Furthermore, it is understood that features of various exemplary embodiments and / or aspects described herein may be combined with each other, unless otherwise specified.

[0067] Video coding typically refers to the processing of a video or a series of pictures that form a video sequence. Instead of the term "picture," the terms "frame" or "image" may be used as synonyms in the field of video coding. Video coding, as used in this application (or this disclosure), refers to either video encoding or video decoding. Video encoding is performed at the source side and typically involves processing of an original video picture (e.g., by compression) to reduce the amount of data needed to represent the video picture (for more efficient storage and / or transmission). Video decoding is performed at the destination side and typically involves the reverse process compared to an encoder to reconstruct the video picture. Embodiments that refer to "coding" a video picture (or, as described later, pictures in general) should be understood to relate to either "encoding" or "decoding" a video picture. The combination of the encoding and decoding parts is also referred to as CODEC (Coding and Decoding).

[0068] In the case of lossless video coding, the original video picture can be restored, i.e., the restored video picture has the same quality as the original video picture (assuming there is no transmission loss or other data loss during storage or transmission). In the case of lossy video coding, further compression is performed, e.g., by quantization, to reduce the amount of data representing the video picture, but the video picture cannot be perfectly restored at the decoder, i.e., the quality of the restored video picture is low or poor compared to the quality of the original video picture.

[0069] Several video coding standards since H.261 belong to the group of "lossy hybrid video codecs" (i.e., they combine spatial and temporal prediction in the sample domain with 2D transform coding to apply quantization in the transform domain). Each picture of a video sequence is typically divided into a set of non-overlapping blocks, and coding is typically performed at the block level. In other words, at the encoder, video is typically processed, i.e., encoded, at the block (video block) level, for example, by using spatial (intra-picture) and temporal (inter-picture) prediction to generate a predictive block, subtracting the predictive block from a current block (the block currently being / to be processed) to obtain a residual block, transforming the residual block, and quantizing the residual block in the transform domain to reduce the amount of data to be transmitted (compression); whereas at the decoder, in order to reconstruct the current block for display, a reverse process is partially applied to the coded or compressed block compared to the encoder. Additionally, the encoder replicates the decoder processing loop so that both produce the same prediction (eg, intra-prediction and inter-prediction) and / or reconstruction for processing, i.e., coding, subsequent blocks.

[0070] The term "block" as used herein may refer to a portion of a picture or a frame. For ease of explanation, embodiments of the present invention are described herein with reference to the High Efficiency Video Coding (HEVC) or Versatile Video Coding (VVC) reference software developed by the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG) Joint Research Team on Video Coding (JCT-VC). Those skilled in the art will understand that embodiments of the present invention are not limited to HEVC or VVC. It may refer to CUs (coding units), PUs (prediction units), and TUs (transform units). In HEVC, coding tree units (CTUs) are divided into CUs using a quadtree structure referred to as a coding tree. The decision of whether to code a picture region using inter-picture (temporal) prediction or intra-picture (spatial) prediction is made at the CU level. Each CU can be further divided into one, two, or four PUs depending on the PU partition type. Within one PU, the same prediction process is applied, and related information is transmitted to the decoder on a PU-by-PU basis. After applying a prediction process based on the PU partition type to obtain residual blocks, the CUs can be divided into transform units (TUs) according to another quadtree structure similar to the coding tree for CUs. In the latest developments in video compression technology, quadtree and binary tree (QTBT) partitioning frames are used to divide coding blocks. In the QTBT block structure, CUs can have either square or rectangular shapes. For example, coding tree units (CTUs) are first divided using a quadtree structure. The quadtree leaf nodes are further divided using a binary tree structure. The binary tree leaf nodes are called coding units (CUs), and their segmentation is used for prediction and transform processing without further division. This means that CUs, PUs, and TUs have the same block size in the QTBT coding block structure. In parallel, multiple divisions, such as ternary tree (TT) division, have also been proposed to be used with the QTBT block structure.The term "device" may also be "apparatus," "decoder," or "encoder."

[0071] In the following embodiments, the encoder 20, the decoder 30, and the coding system 10 are described based on FIGS.

[0072] 11A is a conceptual or schematic block diagram illustrating an example coding system 10, e.g., a video coding system 10 that can utilize the techniques of this application (this disclosure). An encoder 20 (e.g., video encoder 20) and a decoder 30 (e.g., video decoder 30) of video coding system 10 represent examples of devices that may be configured to perform techniques according to various examples described herein. As shown in FIG. 11A, coding system 10 includes a source device 12 configured to provide encoded data 13, e.g., encoded pictures 13, to a destination device 14, e.g., for decoding encoded data 13.

[0073] Source device 12 comprises an encoder 20 and may further, i.e., optionally, comprise a picture source 16 , a pre-processing unit 18 , eg, a picture pre-processing unit 18 , and a communication interface or unit 22 .

[0074] Picture source 16 may comprise or be, for example, any kind of picture capturing device for capturing real-world pictures and / or any kind of picture or commentary (in the case of screen content coding, some text on the screen is considered part of the picture or image to be coded) generation device, such as a computer graphics processor for generating computer-animated pictures, or any kind of device for obtaining and / or providing real-world pictures, computer-animated pictures (e.g., screen content, virtual reality (VR) pictures), and / or any combination thereof (e.g., augmented reality (AR) pictures). Picture source may be any kind of memory or storage for storing any of the aforementioned pictures.

[0075] A (digital) picture is, or can be considered as, a two-dimensional array or matrix of samples with intensity values. The samples in the array are sometimes called pixels (a short form of picture element) or pels. The number of samples in the horizontal and vertical directions (or axes) of the array or picture defines the size and / or resolution of the picture. To represent color, three color components are usually used, i.e., a picture may be represented by or contain three sample arrays. In an RBG format or color space, a picture contains corresponding red, green, and blue sample arrays. However, in video coding, each pixel is usually represented in a luma / chroma format or color space, such as YCbCr, which contains a luma component denoted Y (sometimes L is used instead) and two chroma components denoted Cb and Cr. The luma (or luma for short) component Y represents brightness or gray-level intensity (e.g., as in a grayscale picture), and the two chroma (or chroma for short) components Cb and Cr represent color or color information components. Thus, a picture in YCbCr format includes a luma sample array of luma sample values ​​(Y) and two chroma sample arrays of chroma values ​​(Cb and Cr). A picture in RGB format may be converted or transformed to YCbCr format, and vice versa; this process is also known as color conversion or color translation. If a picture is monochrome, the picture may include only a luma sample array.

[0076] Picture source 16 (e.g., video source 16) may be, for example, a camera for capturing a picture, a memory, e.g., a picture memory, including or storing a previously captured or generated picture, and / or any kind of interface (internal or external) for acquiring or receiving a picture. The camera may be, for example, a local camera or an integrated camera integrated into the source device, and the memory may be a local memory or, for example, an integrated memory integrated into the source device. The interface may be, for example, an external interface for receiving pictures from an external video source, e.g., an external picture capturing device such as a camera, an external memory, or an external picture generating device, e.g., an external computer graphics processor, computer, or server. The interface may be any kind of interface, e.g., a wired or wireless interface, an optical interface, following any proprietary or standardized interface protocol. The interface for acquiring picture data 17 may be the same interface as or part of communication interface 22.

[0077] As distinct from preprocessing unit 18 and the processing performed by preprocessing unit 18, pictures or picture data 17 (eg, video data 16) may also be referred to as raw pictures or raw picture data 17.

[0078] The preprocessing unit 18 is configured to receive (raw) picture data 17 and perform preprocessing on the picture data 17 to obtain a preprocessed picture 19 or preprocessed picture data 19. The preprocessing performed by the preprocessing unit 18 may include, for example, cropping, color format conversion (e.g., from RGB to YCbCr), color correction, or noise removal. It can be understood that the preprocessing unit 18 may be an optional component.

[0079] An encoder 20 (e.g., video encoder 20) is configured to receive pre-processed picture data 19 and provide encoded picture data 21 (further details are described below, e.g., with reference to Figure 12 or Figure 14).

[0080] The communication interface 22 of the source device 12 may be configured to receive the encoded picture data 21 and transmit the encoded picture data 21 (or any further processed version thereof) via the communication channel 13 to another device for storage or direct restoration, e.g., the destination device 14 or any other device.

[0081] The communications interface 22 of the source device 12 may be configured to receive the encoded picture data 21 and transmit it to another device for storage or direct restoration, such as the destination device 14 or any other device, or to process the encoded picture data 21 before storing the encoded data 13 and / or transmitting the encoded data 13 to another device, such as the destination device 14 or any other device for decoding or storage, respectively.

[0082] Destination device 14 includes a decoder 30 (eg, a video decoder 30) and may further include, i.e., possibly a communications interface or unit 28, a post-processing unit 32, and a display device 34.

[0083] The communications interface 28 of the destination device 14 is configured to receive the encoded picture data 21 (or any further processed version thereof) directly from the source device 12 or from any other source, e.g., a storage device, e.g., an encoded picture data storage device, and to provide the encoded picture data 21 to the decoder 30.

[0084] Communications interface 28 of destination device 14 is configured to receive coded picture data 21 or coded data 13, for example directly from source device 12, or from any other source, for example a storage device, for example a coded picture data storage device.

[0085] The communication interface 22 and the communication interface 28 may be configured to transmit and receive the encoded picture data 21 or the encoded data 13 via a direct communication link between the source device 12 and the destination device 14, e.g., a direct wired or wireless connection, or via any type of network, e.g., a wired or wireless network or any combination thereof, or via any type of private and public network, or any type of combination thereof.

[0086] The communications interface 22 may be configured, for example, to package the encoded picture data 21 into a suitable format, e.g., packets, and / or process the encoded picture data using any type of transmission encoding or processing for transmission over a communications link or network.

[0087] A communications interface 28 forming the counterpart of communications interface 22 may for example be arranged to unpackage encoded data 13 to obtain encoded picture data 21 .

[0088] The communications interface 28, which forms the counterpart of the communications interface 22, may for example be configured to receive transmitted data and process the transmitted data using any type of corresponding transmission decoding or processing and / or unpackaging to obtain the coded picture data 21.

[0089] Both communication interface 22 and communication interface 28 may be configured as unidirectional communication interfaces, as indicated by the arrow in coded picture data 13 in FIG. 11A pointing from source device 12 to destination device 14, or as bidirectional communication interfaces, e.g., configured to send and receive messages, e.g., to set up connections, confirm and exchange communication links and / or any other information related to data transmission, e.g., coded picture data transmission.

[0090] The decoder 30 is configured to receive the coded picture data 21 and to provide decoded picture data 31 or decoded pictures 31 (further details are described below, for example, with reference to Figure 13 or Figure 15).

[0091] Post-processor 32 of destination device 14 is configured to post-process decoded picture data 31 (also referred to as reconstructed picture data), e.g., decoded picture 31, to obtain post-processed picture data 33, e.g., post-processed picture 33. The post-processing performed by post-processing unit 32 may include, for example, color format conversion (e.g., from YCbCr to RGB), color correction, cropping or resampling, or any other processing, for example, to prepare decoded picture data 31 for display by display device 34.

[0092] Display device 34 of destination device 14 is configured to receive post-processed picture data 33, for example for displaying the picture to a user or viewer. Display device 34 may be or comprise any type of display, e.g., an integrated or external display or monitor, for presenting the restored picture. The display may include, for example, a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display.

[0093] 11A depicts source device 12 and destination device 14 as separate devices, an embodiment of the device may include both source device 12 or corresponding functionality and destination device 14 or corresponding functionality, or both. In such an embodiment, source device 12 or corresponding functionality and destination device 14 or corresponding functionality may be implemented using the same hardware and / or software, or by separate hardware and / or software, or any combination thereof.

[0094] As will be apparent to those skilled in the art based on the description, the presence and (exact) division of functions of various units or functions within source device 12 and / or destination device 14 shown in FIG. 11A may vary depending on the actual device and application.

[0095] Encoder 20 (e.g., video encoder 20) and decoder 30 (e.g., video decoder 30) may each be implemented as any of a variety of suitable circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If the techniques are implemented partially in software, a device may store software instructions on a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the techniques of this disclosure. Any of the foregoing (including hardware, software, a combination of hardware and software, etc.) may be considered to be one or more processors. Video encoder 20 and video decoder 30 may each be included in one or more encoders or decoders, any of which may be integrated as part of a combined encoder / decoder (codec) within the respective device.

[0096] Encoder 20 may be implemented via processing circuitry 46 to embody the various modules described with respect to encoder 20 of FIG. 12 and / or any other encoder system or subsystem described herein. Decoder 30 may be implemented via processing circuitry 46 to embody the various modules described with respect to decoder 30 of FIG. 13 and / or any other decoder system or subsystem described herein. The processing circuitry may be configured to perform various operations described below. If the techniques are implemented partially in software, as shown in FIG. 15, a device may store software instructions on a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the techniques of this disclosure. Either video encoder 20 and video decoder 30 may be integrated as part of a combined encoder / decoder (codec) in a single device, as shown in FIG. 11B, for example.

[0097] Source device 12 may be referred to as a video encoding device or video encoding apparatus. Destination device 14 may be referred to as a video decoding device or video decoding apparatus. Source device 12 and destination device 14 may be examples of video coding devices or video coding apparatuses.

[0098] The source device 12 and the destination device 14 may include any of a wide range of devices, including any type of handheld or fixed device, such as a notebook or laptop computer, a mobile phone, a smartphone, a tablet or tablet computer, a camera, a desktop computer, a set-top box, a television, a display device, a digital media player, a video gaming console, a video streaming device (such as a content service server or content distribution server), a broadcast receiver device, a broadcast transmitter device, etc., and may use no operating system or any type of operating system.

[0099] In some cases, source device 12 and destination device 14 may be capable of wireless communication. As such, source device 12 and destination device 14 may be wireless communication devices.

[0100] 11A is merely an example, and the techniques of the present application may be applied to video coding settings (e.g., video encoding or video decoding) that do not necessarily include any data communication between encoding and decoding devices. In other examples, data may be retrieved from local memory, streamed over a network, etc. A video encoding device may encode data and store it in memory, and / or a video decoding device may retrieve and decode data from memory. In some examples, encoding and decoding are performed by devices that do not communicate with each other, but simply encode data to memory and / or retrieve and decode data from memory.

[0101] For ease of explanation, embodiments of the present invention are described herein with reference to the High Efficiency Video Coding (HEVC) or Versatile Video Coding (VVC) reference software, next-generation video coding standards, developed by, for example, the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG) Joint Research Team on Video Coding (JCT-VC). Those skilled in the art will understand that embodiments of the present invention are not limited to HEVC or VVC. For each of the above examples described with reference to video encoder 20, video decoder 30 may be configured to perform the reverse process. With respect to notification of syntax elements, video decoder 30 may be configured to receive and parse such syntax elements and decode associated video data accordingly. In some examples, video encoder 20 may entropy encode one or more syntax elements into the encoded video bitstream. In such examples, video decoder 30 may parse such syntax elements and decode associated video data accordingly.

[0102] 11B is an illustrative diagram of another example video coding system 40 including the encoder 20 of FIG. 12 and / or the decoder 30 of FIG. 13 according to an example embodiment. System 40 may implement techniques according to various examples described herein. In the illustrated embodiment, video coding system 40 may include an imaging device 41, a video encoder 100, a video decoder 30 (and / or a video coder implemented via logic 47 of a processing unit 46), an antenna 42, one or more processors 43, one or more memory stores 44, and / or a display device 45.

[0103] As shown, imaging device 41, antenna 42, processing unit 46, logic circuitry 47, video encoder 20, video decoder 30, processor 43, memory store 44, and / or display device 45 may be capable of communicating with one another. As described, although shown with both video encoder 20 and video decoder 30, video coding system 40 may include only video encoder 20 or only video decoder 30 in various examples.

[0104] As shown, in some examples, video coding system 40 may include antenna 42. Antenna 42 may be configured, for example, to transmit and receive encoded bitstreams of video data. Further, in some examples, video coding system 40 may include display device 45. Display device 45 may be configured to present the video data. As shown, in some examples, logic circuitry 47 may be implemented via processing unit 46. Processing unit 46 may include application specific integrated circuit (ASIC) logic, a graphics processor, a general-purpose processor, etc. Video coding system 40 may also include an optional processor 43, which may also include application specific integrated circuit (ASIC) logic, a graphics processor, a general-purpose processor, etc. In some examples, logic circuitry 47 may be implemented via hardware, dedicated video coding hardware, etc., and processor 43 may implement general-purpose software, an operating system, etc. Additionally, memory store 44 may be any type of memory, such as volatile memory (e.g., static random access memory (SRAM), dynamic random access memory (DRAM), etc.) or non-volatile memory (e.g., flash memory, etc.). In a non-limiting example, memory store 44 may be implemented by a cache memory. In some examples, logic circuitry 47 may access memory store 44 (e.g., for implementing an image buffer). In other examples, logic circuitry 47 and / or processing unit 46 may include a memory store (e.g., a cache, etc.) for implementing an image buffer, etc.

[0105] In some examples, video encoder 100 implemented via logic circuitry may include an image buffer (e.g., via either processing unit 46 or memory store 44) and may include a graphics processing unit (e.g., via processing unit 46). The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include various modules described with respect to FIG. 12 and / or video encoder 100 implemented via logic circuitry 47 to embody any other encoder system or subsystem described herein. The logic circuitry may be configured to perform various operations described herein.

[0106] Video decoder 30 may be implemented similarly via processing circuitry 47 to embody the various modules described with respect to decoder 30 of FIG. 13 and / or any other decoder system or subsystem described herein. In some examples, video decoder 30, which may be implemented via logic circuitry, may include an image buffer (e.g., via either processing unit 420 or memory store 44) and may include a graphics processing unit (e.g., via processing unit 46). The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include video decoder 30 implemented via logic circuitry 47 to embody the various modules described with respect to FIG. 13 and / or any other decoder system or subsystem described herein.

[0107] In some examples, antenna 42 of video coding system 40 may be configured to receive an encoded bitstream of video data. As described, the encoded bitstream may include data associated with the encoding of video frames described herein, such as data associated with a coding partition (e.g., transform coefficients or quantized transform coefficients, optional indicators (as described), and / or data defining the coding partition), indicators, index values, mode selection data, etc. Video coding system 40 may also include a video decoder 30 coupled to antenna 42 and configured to decode the encoded bitstream. A display device 45 configured to present the video frames.

[0108] Figure 12 shows a schematic / conceptual block diagram of an exemplary video encoder 20 configured to implement the techniques of the present application. In the example of Figure 12, the video encoder 20 includes a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a buffer 216, a loop filter unit 220, a decoded picture buffer (DPB) 230, a prediction processing unit 260, and an entropy coding unit 270. The prediction processing unit 260 may include an inter prediction unit 244, an intra prediction unit 254, and a mode selection unit 262. The inter prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown). The video encoder 20 shown in Figure 12 may also be referred to as a hybrid video encoder or a video encoder using a hybrid video codec.

[0109] For example, the residual calculation unit 204, the transform processing unit 206, the quantization unit 208, the prediction processing unit 260, and the entropy coding unit 270 form the forward signal path of the encoder 20, while, for example, the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the buffer 216, the loop filter 220, the decoded picture buffer (DPB) 230, and the prediction processing unit 260 form the backward signal path of the encoder, which corresponds to the signal path of the decoder (see decoder 30 in Figure 13).

[0110] The inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the loop filter 220, the decoded picture buffer (DPB) 230, the inter prediction unit 244, and the intra prediction unit 254 are also referred to as forming a “built-in decoder” of the video encoder 20. The encoder 20 is configured to receive, for example, via an input 202, a picture 201 or a block 203 of a picture 201, e.g., a picture of a series of pictures forming a video or a video sequence. The picture block 203 may also be referred to as a current picture block or a picture block to be coded, and the picture 201 may also be referred to as a current picture or a picture to be coded (particularly in video coding to distinguish the current picture from other pictures, e.g., previously encoded and / or decoded pictures of the same video sequence, i.e., the video sequence that also includes the current picture).

[0111] A (digital) picture is, or can be considered as, a two-dimensional array or matrix of samples with intensity values. The samples in the array are sometimes called pixels (a short form of picture element) or pels. The number of samples in the horizontal and vertical directions (or axes) of the array or picture defines the size and / or resolution of the picture. Three color components are typically used to represent color, i.e., a picture may be represented by or contain three sample arrays. In an RBG format or color space, a picture contains corresponding red, green, and blue sample arrays. However, in video coding, each pixel is typically represented in a luma and chroma format or color space, e.g., YCbCr, which contains a luma component denoted Y (sometimes L is used instead) and two chroma components denoted Cb and Cr. The luma (or luma for short) component Y represents brightness or gray-level intensity (e.g., as in a grayscale picture), and the two chroma (or chroma for short) components Cb and Cr represent color or color information components. Thus, a picture in YCbCr format includes a luma sample array of luma sample values ​​(Y) and two chroma sample arrays of chroma values ​​(Cb and Cr). A picture in RGB format may be converted or transformed into YCbCr format, or vice versa; this process is also known as color conversion or color translation. If a picture is monochrome, the picture may include only a luma sample array. Thus, a picture may be, for example, an array of luma samples in monochrome format, or an array of luma samples and two corresponding arrays of chroma samples in 4:2:0, 4:2:2, and 4:4:4 color formats.

[0112] Split An embodiment of encoder 20 may include a partitioning unit (not depicted in FIG. 12 ) configured to partition picture 201 into multiple (usually non-overlapping) picture blocks 203. These blocks may also be called root blocks, macroblocks (H.264 / AVC) or coding tree blocks (CTBs) or coding tree units (CTUs) (H.265 / HEVC and VVC). The partitioning unit may be configured to use the same block size for all pictures of a video sequence and the corresponding grid that defines the block size, or to vary the block size among pictures or subsets or groups of pictures, and to partition each picture into the corresponding blocks.

[0113] In further embodiments, the video encoder may be configured to directly receive blocks 203 of picture 201, e.g., one, some, or all of the blocks forming picture 201. Picture blocks 203 may also be referred to as current picture blocks or picture blocks to be coded. In one example, prediction processing unit 260 of video encoder 20 may be configured to perform any combination of the partitioning techniques described above.

[0114] Similar to picture 201, block 203 may also be, or may be considered to be, a two-dimensional array or matrix of samples having intensity values ​​(sample values), albeit with smaller dimensions than picture 201. In other words, block 203 may include, for example, one sample array (e.g., a luma array in the case of monochrome picture 201), or three sample arrays (e.g., a luma array and two chroma arrays in the case of color picture 201), or any other number and / or type of arrays depending on the applied color format. The number of samples in the horizontal and vertical directions (or axes) of block 203 define the size of block 203. Thus, a block may be, for example, an M×N (M columns by N rows) array of samples or an M×N array of transform coefficients.

[0115] The encoder 20 shown in FIG. 12 is configured to code a picture 201 block by block, eg, coding and prediction is performed block by block 203 .

[0116] The embodiment of video encoder 20 shown in FIG. 12 may be further configured to divide and / or encode pictures using slices (also called video slices), where a picture may be divided or encoded using one or more slices (typically non-overlapping), each of which may include one or more blocks (e.g., CTUs) or one or more groups of blocks (e.g., tiles (H.265 / HEVC and VVC) or bricks (VVC)).

[0117] The embodiment of video encoder 20 shown in FIG. 12 may be further configured to divide and / or encode a picture using slices / tile groups (also referred to as video tile groups) and / or tiles (also referred to as video tiles), where a picture may be divided or encoded using one or more slice / tile groups (typically non-overlapping), each of which may include, for example, one or more blocks (e.g., CTUs) or one or more tiles, and each tile may be, for example, rectangular in shape and may include one or more blocks (e.g., CTUs), e.g., full or partial blocks.

[0118] Residual calculation The residual calculation unit 204 is configured to calculate the residual block 205 based on the picture block 203 and the prediction block 265 (further details regarding the prediction block 265 will be provided later), for example, by subtracting sample values ​​of the prediction block 265 from sample values ​​of the picture block 203, to obtain the residual block 205 in the sample domain on a sample-by-sample (pixel-by-pixel) basis.

[0119] conversion The transform processing unit 206 is configured to apply a transform, for example, a discrete cosine transform (DCT) or a discrete sine transform (DST), to the sample values ​​of the residual block 205 to obtain transform coefficients 207 in a transform domain. The transform coefficients 207, which may also be referred to as transform residual coefficients, may represent the residual block 205 in the transform domain.

[0120] The transform processing unit 206 may be configured to apply an integer approximation of a DCT / DST, such as a transform specified for HEVC / H.265. Compared to an orthogonal DCT transform, such an integer approximation is typically scaled by a particular factor. To preserve the norm of the residual block processed by the forward and inverse transforms, an additional scaling factor is applied as part of the transform process. The scaling factor is typically selected based on particular constraints, such as the scaling factor being a power of two due to shift operations, the bit depth of the transform coefficients, a trade-off between accuracy and implementation cost, etc. A particular scaling factor may be specified, for example, for the inverse transform, e.g., by the inverse transform processing unit 212 in the decoder 30 (and the corresponding inverse transform, e.g., by the inverse transform processing unit 212 in the encoder 20), and a corresponding scaling factor for the forward transform, e.g., by the transform processing unit 206 in the encoder 20, may be specified accordingly.

[0121] An embodiment of the video encoder 20 (respectively the transform processing unit 206) may be configured to output transform parameters, e.g., one or more types of transform, directly or may be encoded or compressed via the entropy coding unit 270, so that, for example, the video decoder 30 can receive and use the transform parameters for decoding.

[0122] quantization The quantization unit 208 is configured to quantize the transform coefficients 207 to obtain quantized transform coefficients 209, for example, by applying scalar quantization or vector quantization. The quantized transform coefficients 209 may also be referred to as quantized residual coefficients 209. The quantization process may reduce the bit depth associated with some or all of the transform coefficients 207. For example, an n-bit transform coefficient may be truncated to an m-bit transform coefficient during quantization, where n is greater than m. The degree of quantization may be modified by adjusting a quantization parameter (QP). For example, in the case of scalar quantization, different scaling may be applied to achieve finer or coarser quantization. A smaller quantization step size corresponds to finer quantization, and a larger quantization step size corresponds to coarser quantization. The applicable quantization step size may be indicated by the quantization parameter (QP). The quantization parameter may, for example, be an index into a predefined set of applicable quantization step sizes. For example, a small quantization parameter may correspond to finer quantization (smaller quantization step size), a large quantization parameter may correspond to coarser quantization (larger quantization step size), or vice versa. Quantization may involve division by the quantization step size and corresponding or inverse inverse quantization, e.g., by inverse quantization 210, which may involve multiplication by the quantization step size. Some standards, e.g., HEVC, embodiments may be configured to use the quantization parameter to determine the quantization step size. Generally, the quantization step size may be calculated based on the quantization parameter using a fixed-point approximation of an equation involving division. To restore the norm of the residual block, an additional scaling factor may be introduced in the quantization and inverse quantization, which may be corrected for the scaling used in the fixed-point approximation of the equations for the quantization step size and the quantization parameter. In one exemplary implementation, the scaling of the inverse transform and inverse quantization may be combined. Alternatively, customized quantization tables may be used and signaled from the encoder to the decoder, e.g., in the bitstream.Quantization is a lossy operation, and the loss increases with increasing quantization step size.

[0123] An embodiment of video encoder 20 (respectively quantization unit 208) may be configured to output a quantization parameter (QP) directly or may be encoded via entropy coding unit 270 so that, for example, video decoder 30 can receive and apply the quantization parameter for decoding.

[0124] The inverse quantization unit 210 is configured to apply the inverse quantization of the quantization unit 208 to the quantized coefficients by applying the inverse of the quantization scheme applied by the quantization unit 208, e.g., based on or using the same quantization step size as the quantization unit 208, to obtain the inverse quantized coefficients 211. The inverse quantized coefficients 211, also referred to as the inverse quantized residual coefficients 211, typically correspond to the transform coefficients 207, although they are not identical to the transform coefficients due to loss due to quantization.

[0125] The inverse transform processing unit 212 is configured to apply an inverse transform of the transform applied by the transform processing unit 206, for example, an inverse discrete cosine transform (DCT) or an inverse discrete sine transform (DST), to obtain an inverse transform block 213 in the sample domain. The inverse transform block 213 may also be referred to as an inverse transform inverse quantization block 213 or an inverse transform residual block 213.

[0126] The reconstruction unit 214 (e.g., adder 214) is configured to add the inverse transform block 213 (i.e., the reconstructed residual block 213) to the prediction block 265 to obtain the reconstruction block 215 in the sample domain, for example, by adding the sample values ​​of the reconstructed residual block 213 and the sample values ​​of the prediction block 265.

[0127] An optional buffer unit 216 (or "buffer" 216 for short), e.g., a line buffer 216, is configured to buffer or store the reconstructed blocks 215 and their respective sample values, e.g., for intra-prediction. In further embodiments, the encoder may be configured to use the unfiltered reconstructed blocks and / or their respective sample values ​​stored in the buffer unit 216 for any kind of estimation and / or prediction, e.g., intra-prediction.

[0128] Embodiments of encoder 20 may be configured, for example, such that buffer unit 216 is used to store reconstructed blocks 215 not only for intra prediction 254 but also for loop filter unit 220 (not shown in FIG. 12 ), and / or such that buffer unit 216 and decoded picture buffer unit 230 form one buffer. Further embodiments may be configured to use filtered blocks 221 and / or blocks or samples from decoded picture buffer 230 (both not shown in FIG. 12 ) as input or basis for intra prediction 254.

[0129] Loop filter unit 220 (or “loop filter” 220 for short) is configured to filter reconstruction block 215 to obtain filtered block 221, e.g., to smooth pixel transitions or otherwise improve video quality. Loop filter unit 220 represents one or more loop filters, such as a deblocking filter, a sample adaptive offset (SAO) filter, or other filters, e.g., a bilateral filter, an adaptive loop filter (ALF), a sharpening filter, a smoothing filter, or a collaborative filter. Although loop filter unit 220 is shown in FIG. 12 as being an in-loop filter, in other configurations, loop filter unit 220 may be implemented as a post-loop filter. Filtered block 221 may also be referred to as filtered reconstruction block 221. Decoded picture buffer 230 may store the reconstruction coding block after loop filter unit 220 performs filtering operations on the reconstruction coding block.

[0130] The loop filter unit 220 (or "loop filter" 220 for short) is configured to filter the reconstruction block 215 to obtain a filtered block 221, or generally, to filter the reconstruction samples to obtain filtered sample values. The loop filter unit is configured, for example, to smooth pixel transitions or otherwise improve video quality. The loop filter unit 220 may include one or more loop filters, such as a deblocking filter, a sample adaptive offset (SAO) filter, or one or more other filters, for example, an adaptive loop filter (ALF), a noise suppression filter (NSF), or any combination thereof. In one example, the loop filter unit 220 may include a deblocking filter, an SAO filter, and an ALF filter. The order of filtering processes may be deblocking filter, SAO, and ALF. In another example, a process called luma mapping with chroma scaling (LMCS) (i.e., adaptive in-loop reshaper) is added. This process is performed before deblocking. In another example, the deblocking filter process may also be applied to interior sub-block edges, e.g., affine sub-block edges, ATMVP sub-block edges, sub-block transform (SBT) edges, and intra-sub-partition (ISP) edges. Although loop filter unit 220 is shown in FIG. 12 as being an in-loop filter, in other configurations, loop filter unit 220 may be implemented as a post-loop filter. Filtered block 221 may also be referred to as filtered reconstruction block 221.

[0131] An embodiment of video encoder 20 (respectively loop filter unit 220) may be configured to output loop filter parameters (such as SAO filter parameters or ALF filter parameters or LMCS parameters), e.g., directly or encoded via entropy coding unit 270, so that, e.g., decoder 30 can receive and apply the same loop filter parameters or respective loop filters for decoding.

[0132] The decoded picture buffer (DPB) 230 may be a reference picture memory that stores reference picture data for use in encoding video data by the video encoder 20. The DPB 230 may be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. The DPB 230 and the buffer 216 may be provided by the same memory device or separate memory devices. In some examples, the decoded picture buffer (DPB) 230 is configured to store filtered blocks 221. The decoded picture buffer 230 may further be configured to store other previously filtered blocks, e.g., previously reconstructed and filtered blocks 221 of the same current picture or a different picture, e.g., a previously reconstructed picture, and may provide a fully previously reconstructed, i.e., decoded, picture (and corresponding reference blocks and samples) and / or a partially reconstructed current picture (and corresponding reference blocks and samples), e.g., for inter-prediction. In some examples, if the reconstruction block 215 is reconstructed but without in-loop filtering, the decoded picture buffer (DPB) 230 is configured to store one or more unfiltered reconstruction blocks 215, or generally unfiltered reconstruction samples, for example, if the reconstruction block 215 is not filtered by the loop filter unit 220 or any other further processed version of the reconstructed block or sample.

[0133] The prediction processing unit 260, also referred to as block prediction processing unit 260, is configured to receive or obtain block 203 (current block 203 of current picture 201) and reconstructed picture data, e.g., reference samples of the same (current) picture from buffer 216 and / or reference picture data 231 from one or more previously decoded pictures from decoded picture buffer 230, and process such data for prediction, i.e., to provide a prediction block 265, which may be an inter-prediction block 245 or an intra-prediction block 255.

[0134] The mode selection unit 262 may be configured to select a prediction mode (e.g., intra-prediction or inter-prediction mode) and / or the corresponding prediction block 245 or 255 to be used as the prediction block 265 for calculating the residual block 205 and reconstructing the reconstruction block 215.

[0135] Embodiments of mode selection unit 262 may be configured to select a prediction mode (e.g., from those supported by prediction processing unit 260) that provides the best match, or in other words, the smallest residual (smallest residual means better compression for transmission or storage), or the smallest signaling overhead (smallest signaling overhead means better compression for transmission or storage), or that considers or balances both. Mode selection unit 262 may be configured to determine the prediction mode based on rate-distortion optimization (RDO), i.e., to select a prediction mode that provides the smallest rate-distortion optimization or whose associated rate-distortion at least meets a prediction mode selection criterion.

[0136] The prediction process (eg, prediction unit 260) and mode selection (eg, by mode selection unit 262) performed by exemplary encoder 20 are described in more detail below.

[0137] 25 , the mode select unit 260 includes a partitioning unit 262, an inter prediction unit 244, and an intra prediction unit 254, and is configured to receive or obtain original picture data, e.g., original block 203 (current block 203 of current picture 17), and reconstructed picture data, e.g., filtered and / or unfiltered reconstructed samples or blocks of the same (current) picture and / or from one or more previously decoded pictures, e.g., from the decoded picture buffer 230 or other buffer (e.g., a line buffer, not shown). The reconstructed picture data is used as reference picture data for prediction, e.g., inter prediction or intra prediction, to obtain a prediction block 265 or predictor 265.

[0138] The mode selection unit 260 may be configured to determine or select a current block prediction mode (including no partitioning) and a partitioning for the prediction mode (e.g., intra prediction mode or inter prediction mode) and generate a corresponding prediction block 265, which is used to calculate the residual block 205 and reconstruct the reconstruction block 215.

[0139] Embodiments of mode selection unit 260 may be configured to select a partitioning and prediction mode (e.g., from those supported by or available to mode selection unit 260) that provides the best match, or in other words, the smallest residual (smallest residual means better compression for transmission or storage), or the smallest signaling overhead (smallest signaling overhead means better compression for transmission or storage), or that considers or balances both. Mode selection unit 260 may be configured to determine the partitioning and prediction mode based on rate-distortion optimization (RDO), i.e., to select the prediction mode that provides the smallest rate-distortion. Terms such as “best,” “minimum,” and “optimum” in this context do not necessarily refer to overall “best,” “minimum,” “optimum,” etc., but may refer to termination and selection criteria such as values ​​above or below a threshold, or other constraints that potentially lead to a “suboptimal selection” but reduce complexity and processing time.

[0140] In other words, the division unit 262 may be configured to divide a picture from a video sequence into a series of coding tree units (CTUs), and the CTUs 203 may be further divided into smaller block partitions or sub-blocks (which again form blocks), for example by repeatedly using quadtree partitioning (QT), binary tree partitioning (BT), or ternary tree partitioning (TT), or any combination thereof, and performing prediction of each of the block partitions or sub-blocks, and the mode selection includes selecting a tree structure of the division block 203, and a prediction mode is applied to each of the block partitions or sub-blocks.

[0141] Below, the partitioning (eg, by partitioning unit 260) and prediction processes (by inter-prediction unit 244 and intra-prediction unit 254) performed by exemplary video encoder 20 are described in more detail.

[0142] Split The division unit 262 may be configured to divide a picture from a video sequence into a series of coding tree units (CTUs), and the division unit 262 may partition (or split) the coding tree units (CTUs) 203 into smaller partitions, e.g., smaller blocks of square or rectangular size. For a picture with a three-sample array, a CTU consists of an N×N block of luma samples along with two corresponding blocks of chroma samples. The maximum allowable size of a luma block within a CTU is specified as 128×128 in the developing Versatile Video Coding (VVC) standard, but it may be specified as a value other than 128×128, e.g., 256×256, in the future. The CTUs of a picture may be clustered / grouped as slices / tile groups, tiles, or bricks. A tile covers a rectangular area of ​​the picture, and a tile can be divided into one or more bricks. A brick consists of several CTU rows within the tile. A tile that is not divided into multiple bricks can be called a brick. However, a brick is a true subset of a tile and is not called a tile. There are two modes of tile groups supported in VVC: raster scan slice / tile group mode and rectangular slice mode. In raster scan tile group mode, a slice / tile group contains a set of tiles in the tile raster scan of the picture. In rectangular slice mode, a slice contains several bricks of the picture that collectively form a rectangular region of the picture. The bricks within a rectangular slice are in the order of the raster scan of the slice. These smaller blocks (sometimes called sub-blocks) may be further divided into even smaller partitions.This is also called tree or hierarchical tree partitioning, in which a root block, e.g., root tree level 0 (hierarchical level 0, depth 0), may be recursively partitioned, e.g., into two or more blocks at the next lower tree level, e.g., tree level 1 (hierarchical level 1, depth 1), and these blocks may be partitioned again into two or more blocks at the next lower level, e.g., tree level 2 (hierarchical level 2, depth 2), until the partitioning terminates, e.g., because a termination criterion is met, e.g., a maximum tree depth or a minimum block size has been reached. Blocks that are not further partitioned are also called leaf blocks or leaf nodes of the tree. A tree that uses a partition into two partitions is called a binary tree (BT), a tree that uses a partition into three partitions is called a ternary tree (TT), and a tree that uses a partition into four partitions is called a quad tree (QT).

[0143] For example, a coding tree unit (CTU) may be or include a CTB of luma samples for a picture having a three-sample arrangement, two corresponding CTBs of chroma samples, or a CTB of samples for a monochrome picture or picture coded using a syntax structure used to code three separate color planes and samples. Correspondingly, a coding tree block (CTB) may be an N×N block of samples for some value of N, such that the partitioning of the components into CTBs is a partition. A coding unit (CU) may be or include a coding block of luma samples for a picture having a three-sample arrangement, two corresponding coding blocks of chroma samples, or a coding block of samples for a monochrome picture or picture coded using a syntax structure used to code three separate color planes and samples. Correspondingly, a coding block (CB) may be an M×N block of samples for some value of M and N, such that the partitioning of the CTB into coding blocks is a partition.

[0144] In an embodiment, for example, according to HEVC, coding tree units (CTUs) may be divided into CUs using a quadtree structure, denoted as a coding tree. The decision of whether to code a picture region using intra-picture (temporal) or inter-picture (spatial) prediction is made at the leaf CU level. Each leaf CU can be further divided into one, two, or four PUs depending on the PU partition type. Within one PU, the same prediction process is applied, and related information is transmitted to the decoder on a PU-by-PU basis. After obtaining residual blocks by applying the prediction process based on the PU partition type, the leaf CUs can be divided into transform units (TUs) according to another quadtree structure similar to the coding tree for CUs.

[0145] In an embodiment, for example, according to the latest video coding standard currently under development, called Versatile Video Coding (VVC), a hybrid quadtree-nested multitype tree using a binary and ternary split segmentation structure is used to split a coding tree unit. In the coding tree structure within a coding tree unit, a CU can have either a square or rectangular shape. For example, a coding tree unit (CTU) is first split by a quadtree. Then, the quadtree leaf node can be further split by a multitype tree structure. There are four split types in the multitype tree structure: vertical binary split (SPLIT_BT_VER), horizontal binary split (SPLIT_BT_HOR), vertical ternary split (SPLIT_TT_VER), and horizontal ternary split (SPLIT_TT_HOR). The multitype tree leaf node is called a coding unit (CU), and this segmentation is used for prediction and transform processing without further splitting, as long as the CU is not too large for the maximum transform length. This means that in most cases, CUs, PUs, and TUs have the same block size within a quadtree with a nested multi-type tree coding block structure. An exception occurs when the maximum supported transform length is smaller than the width or height of a CU's color components. VVC develops its own partition notification mechanism to partition information within a quadtree with a nested multi-type tree coding tree structure. In the notification mechanism, a coding tree unit (CTU) is treated as the root of the quadtree and is first partitioned by the quadtree structure. Then, each quadtree leaf node (when large enough to permit it) is further partitioned by the multi-type tree structure.In a multi-type tree structure, a first flag (mtt_split_cu_flag) is signaled to indicate whether a node is further split; when the node is further split, a second flag (mtt_split_cu_vertical_flag) is signaled to indicate the split direction; and then a third flag (mtt_split_cu_binary_flag) is signaled to indicate whether the split is binary or ternary. Based on the values ​​of mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag, the multi-type tree split mode (MttSplitMode) of the CU can be derived by the decoder based on a predefined rule or table. For a specific design, for example, a design of 64x64 luma blocks and a 32x32 chroma pipeline in a VVC hardware decoder, it should be noted that TT splitting is prohibited when either the width or height of the luma coding block is greater than 64, as shown in FIG. 6. TT splitting is also prohibited when either the width or height of a chroma coding block is greater than 32. The pipeline design divides a picture into virtual pipeline data units (VPDUs), which are defined as non-overlapping units within a picture. In a hardware decoder, consecutive VPDUs are processed simultaneously by multiple pipeline stages. Because the VPDU size is roughly proportional to the buffer size in most pipeline stages, it is important to keep the VPDU size small. In most hardware decoders, the VPDU size can be set to the maximum transform block (TB) size. However, in VVC, ternary tree (TT) and binary tree (BT) splitting can lead to an increase in VPDU size.

[0146] In addition, note that when a portion of a tree node block extends beyond the bottom or right picture boundary of a picture, the tree node block is forced to be split until all samples of every coded CU are located within the picture boundary.

[0147] As an example, an intra sub-partition (ISP) tool can divide a luma intra-predicted block into two or four sub-partitions vertically or horizontally depending on the block size.

[0148] In one example, mode selection unit 260 of video encoder 20 may be configured to perform any combination of the partitioning techniques described herein. As described above, encoder 20 is configured to determine or select a best or optimal prediction mode from a set of (predetermined) prediction modes. The set of prediction modes may include, for example, intra-prediction modes and / or inter-prediction modes.

[0149] The set of intra-prediction modes may include 35 different intra-prediction modes, such as omnidirectional modes such as DC (or average) mode and planar mode, or directional modes defined, for example, in H.265, or 67 different intra-prediction modes, such as omnidirectional modes such as DC (or average) mode and planar mode, or directional modes defined, for example, for VVC. As an example, some conventional angular intra-prediction modes are adaptively replaced with wide-angle intra-prediction modes for non-square blocks, for example, as defined in VVC. As another example, to avoid division operations for DC prediction, only the longer side of a non-square block is used to calculate the average. In addition, the results of planar mode intra-prediction may be further modified by a position-dependent intra-prediction concatenation (PDPC) method.

[0150] The intra prediction unit 254 is configured to generate the intra prediction block 265 using reconstructed samples of neighboring blocks of the same current picture according to an intra prediction mode from a set of intra prediction modes.

[0151] The intra prediction unit 254 (or generally the mode selection unit 260) is further configured to output intra prediction parameters (or generally information indicating the intra prediction mode selected for the block) to the entropy coding unit 270 in the form of syntax elements 266 for inclusion in the coded picture data 21, so that, for example, the video decoder 30 can receive and use the prediction parameters for decoding.

[0152] The set (or possible) inter prediction modes depends on the available reference pictures (i.e., previously at least partially decoded pictures, e.g., stored in DBP 230) and other inter prediction parameters, e.g., whether the entire reference picture or only a portion thereof, e.g., a search window area around the area of ​​the current block of the reference picture, is used to search for the best matching reference block, and / or whether pixel interpolation, e.g., half / semi-pel, quarter-pel, and / or 1 / 16-pel interpolation, is applied.

[0153] In addition to the prediction modes mentioned above, skip mode, direct mode, and / or other inter prediction modes may be applied.

[0154] For example, in enhanced merge prediction, the merge candidate list for such a mode is constructed by sequentially including the following five types of candidates: spatial MVP from spatially adjacent CUs, temporal MVP from co-located CUs, history-based MVP from a FIFO table, pairwise average MVP, and zero MV. To improve the accuracy of the MV in the merge mode, bilateral-matching-based decoder-side motion vector refinement (DMVR) may be applied. The MVD-based merge mode (MMVD) is derived from the motion vector difference-based merge mode. The MMVD flag is signaled immediately after sending the skip flag and merge flag to specify whether the MMVD mode is used for the CU. The CU-level adaptive motion vector resolution (AMVR) scheme may also be applied. AMVR enables the MVD of a CU to be coded with various accuracies. The MVD of the current CU may be adaptively selected depending on the prediction mode for the current CU. When a CU is coded in merge mode, a combined inter / intra prediction (CIIP) mode may be applied to the current CU. To obtain CIIP prediction, a weighted average of the inter-prediction signal and the intra-prediction signal is performed. Affine motion compensation prediction: The affine motion field of a block is described by motion information from two control points (four parameters) or three control point motion vectors (six parameters). Subblock-based temporal motion vector prediction (SbTMVP): It is similar to temporal motion vector prediction (TMVP) in HEVC, but now predicts motion vectors for sub-CUs within a CU. Bidirectional optical flow (BDOF), formerly known as BIO, is a simpler version that requires much less computation, particularly in terms of the number of multiplications and the size of the multipliers. Triangulation mode: In such a mode, the CU is evenly divided into two triangular-shaped partitions using either diagonal or anti-diagonal partitioning. Furthermore, bi-prediction mode is extended beyond simple averaging to allow for a weighted average of the two prediction signals.

[0155] In addition to the above prediction modes, skip mode and / or direct mode may be applied.

[0156] The prediction processing unit 260 may be further configured to divide the block 203 into smaller block partitions or sub-blocks, for example, by repeatedly using quadtree partitioning (QT), binary tree partitioning (BT), or ternary tree partitioning (TT), or any combination thereof, and performing prediction of each of the block partitions or sub-blocks, where the mode selection includes selecting a tree structure of the divided block 203 and a prediction mode to be applied to each of the block partitions or sub-blocks.

[0157] The inter prediction unit 244 may include a motion estimation (ME) unit (not shown in FIG. 2) and a motion compensation (MC) unit (not shown in FIG. 2). The motion estimation unit is configured to receive or obtain the picture block 203 (the current picture block 203 of the current picture 201) and the decoded picture 231, or at least one or more previously reconstructed blocks, e.g., reconstructed blocks of one or more other / different previously decoded pictures 231, for motion estimation. For example, a video sequence may include the current picture and the previously decoded picture 231, or in other words, the current picture and the previously decoded picture 231 may be part of a series of pictures forming a video sequence or can form a series of pictures.

[0158] The encoder 20 may be configured, for example, to select a reference block from multiple reference blocks of the same or different ones of multiple other pictures, and provide the reference picture (or reference picture index, ...) and / or an offset (spatial offset) between the position (x, y coordinates) of the reference block and the position of the current block to a motion estimation unit (not shown in FIG. 2) as an inter-prediction parameter. This offset is also called a motion vector (MV).

[0159] The motion compensation unit is configured to obtain, e.g., receive, inter prediction parameters and perform inter prediction based on or using the inter prediction parameters to obtain an inter prediction block 265. The motion compensation performed by the motion compensation unit may include fetching or generating a prediction block based on motion / block vectors determined by motion estimation, possibly performing interpolation to sub-pixel accuracy. Interpolation filtering generates additional pixel samples from known pixel samples, thus potentially increasing the number of candidate prediction blocks that can be used to code the picture block. Upon receiving a motion vector for the PU of the current picture block, the motion compensation unit may locate the prediction block pointed to by the motion vector in one of the reference picture lists.

[0160] The intra prediction unit 254 is configured to obtain, e.g., receive, the picture block 203 (current picture block) and one or more previously reconstructed blocks of the same picture, e.g., reconstructed neighboring blocks, for intra estimation. The encoder 20 may, for example, be configured to select an intra prediction mode from a plurality of (predetermined) intra prediction modes.

[0161] An embodiment of the encoder 20 may be configured to select an intra-prediction mode based on an optimization criterion, such as minimum residual (e.g., the intra-prediction mode that provides the predicted block 255 that is most similar to the current picture block 203) or minimum rate distortion.

[0162] The intra prediction unit 254 is further configured to determine the intra-predicted block 255 based on intra prediction parameters, e.g., a selected intra prediction mode. In either case, after selecting an intra prediction mode for the block, the intra prediction unit 254 is also configured to provide the intra prediction parameters, i.e., information indicating the selected intra prediction mode for the block, to the entropy coding unit 270. In one example, the intra prediction unit 254 may be configured to perform any combination of the intra prediction techniques described below.

[0163] Entropy encoding unit 270 is configured to apply an entropy encoding algorithm or scheme (e.g., a variable length coding (VLC) scheme, a context-adaptive VLC scheme (CALVC), an arithmetic coding scheme, context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy encoding method or technique) to the quantized residual coefficients 209, the inter-prediction parameters, the intra-prediction parameters, and / or the loop filter parameters, individually or together (or not at all), to obtain coded picture data 21, which may be output by output 272, e.g., in the form of coded bitstream 21. Coded bitstream 21 may be transmitted to video decoder 30 or archived for later transmission or retrieval by video decoder 30. Entropy encoding unit 270 may further be configured to entropy encode other syntax elements of the current video slice being coded.

[0164] Other structural variations of the video encoder 20 can be used to encode the video stream. For example, a non-transform-based encoder 20 can quantize the residual signal directly for a particular block or frame without the transform processing unit 206. In another implementation, the encoder 20 can combine the quantization unit 208 and the inverse quantization unit 210 into a single unit.

[0165] 13 shows an example video decoder 30 configured to implement the techniques of the present application. Video decoder 30 is configured to receive coded picture data (e.g., coded bitstream) 21, for example, coded by encoder 100, to obtain decoded picture 131. During the decoding process, video decoder 30 receives from video encoder 100 a coded video bitstream representing picture blocks and associated syntax elements of coded video slices.

[0166] 13, decoder 30 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transform processing unit 312, a reconstruction unit 314 (e.g., adder 314), a buffer 316, a loop filter 320, a decoded picture buffer 330, and a prediction processing unit 360. Prediction processing unit 360 may include an inter prediction unit 344, an intra prediction unit 354, and a mode selection unit 362. Video decoder 30, in some examples, may perform a decoding pass that is generally the reverse of the encoding pass described with respect to video encoder 100 from FIG. 12.

[0167] As described with respect to encoder 20, inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, loop filter 220, decoded picture buffer (DPB) 230, inter prediction unit 344, and intra prediction unit 354 are also referred to as forming a “built-in decoder” of video encoder 20. Accordingly, inverse quantization unit 310 may be functionally identical to inverse quantization unit 110, inverse transform processing unit 312 may be functionally identical to inverse transform processing unit 212, reconstruction unit 314 may be functionally identical to reconstruction unit 214, loop filter 320 may be functionally identical to loop filter 220, and decoded picture buffer 330 may be functionally identical to decoded picture buffer 230. Accordingly, the descriptions provided for the respective units and functions of video encoder 20 correspondingly apply to the respective units and functions of video decoder 30.

[0168] Entropy decoding unit 304 is configured to perform entropy decoding on coded picture data 21 to obtain, e.g., quantized coefficients 309 and / or decoded coding parameters (not shown in FIG. 13 ), e.g., any or all of inter-prediction parameters, intra-prediction parameters, loop filter parameters, and / or other syntax elements (decoded). Entropy decoding unit 304 is further configured to forward the inter-prediction parameters, intra-prediction parameters, and / or other syntax elements to prediction processing unit 360. Video decoder 30 may receive the syntax elements at the video slice level and / or the video block level.

[0169] The entropy decoding unit 304 is configured to parse the bitstream 21 (or coded picture data 21 in general), e.g., perform entropy decoding on the coded picture data 21, e.g., to obtain quantization coefficients 309 and / or decoded coding parameters (not shown in FIG. 13 ), e.g., any or all of inter-prediction parameters (e.g., reference picture indices and motion vectors), intra-prediction parameters (e.g., intra-prediction modes or indices), transform parameters, quantization parameters, loop filter parameters, and / or other syntax elements. The entropy decoding unit 304 may be configured to apply a decoding algorithm or scheme corresponding to the encoding scheme described with respect to the entropy coding unit 270 of the encoder 20. The entropy decoding unit 304 may be further configured to provide the inter-prediction parameters, intra-prediction parameters, and / or other syntax elements to the mode application unit 360 and to provide other parameters to other units of the decoder 30. The video decoder 30 may receive syntax elements at a video slice level and / or a video block level. In addition to or as an alternative to slices and their respective syntax elements, tile groups and / or tiles and their respective syntax elements may be received and / or used.

[0170] The inverse quantization unit 310 may be functionally identical to the inverse quantization unit 110, the inverse transform processing unit 312 may be functionally identical to the inverse transform processing unit 112, the reconstruction unit 314 may be functionally identical to the reconstruction unit 114, the buffer 316 may be functionally identical to the buffer 116, the loop filter 320 may be functionally identical to the loop filter 120, and the decoded picture buffer 330 may be functionally identical to the decoded picture buffer 130.

[0171] An embodiment of decoder 30 may include a partitioning unit (not depicted in FIG. 13). In one example, prediction processing unit 360 of video decoder 30 may be configured to perform any combination of the partitioning techniques described above.

[0172] Prediction processing unit 360 may include an inter prediction unit 344 and an intra prediction unit 354, which may be similar in function to inter prediction unit 144 and intra prediction unit 354, which may be similar in function to intra prediction unit 154. Prediction processing unit 360 is typically configured to perform block prediction and / or obtain a prediction block 365 from encoded data 21 and to receive or obtain (explicitly or implicitly) prediction-related parameters and / or information regarding a selected prediction mode, for example, from entropy decoding unit 304.

[0173] When a video slice is coded as an intra-coded (I) slice, intra prediction unit 354 of prediction processing unit 360 is configured to generate a prediction block 365 for a picture block of the current video slice based on the signaled intra prediction mode and data from previously decoded blocks of the current frame or picture. When a video frame is coded as an inter-coded (i.e., B or P) slice, inter prediction unit 344 (e.g., a motion compensation unit) of prediction processing unit 360 is configured to generate a prediction block 365 for a video block of the current video slice based on the motion vector and other syntax elements received from entropy decoding unit 304. For inter prediction, the prediction block may be generated from one of the reference pictures in one of the reference picture lists. Video decoder 30 may construct the reference frame lists, List 0 and List 1, using a default construction technique based on the reference pictures stored in DPB 330.

[0174] Prediction processing unit 360 parses the motion vectors and other syntax elements to determine prediction information for the video blocks of the current video slice and uses the prediction information to generate a predictive block for the current video block being decoded. For example, motion compensation unit 360 uses some of the received syntax elements to determine the prediction mode (e.g., intra-prediction or inter-prediction) used to code the video blocks of the video slice, the inter-prediction slice type (e.g., B slice, P slice, or GPB slice), construction information for one or more of the reference picture lists for the slice, motion vectors for each inter-coded video block of the slice, inter-prediction states for each inter-coded video block of the slice, and other information for decoding video blocks in the current video slice.

[0175] The inverse quantization unit 310 is configured to inverse quantize, i.e., dequantize, the quantized transform coefficients provided in the bitstream and decoded by the entropy decoding unit 304. The inverse quantization process may include using quantization parameters calculated by the video encoder 100 for each video block in a video slice to determine the degree of quantization, and similarly the degree of inverse quantization, to be applied.

[0176] The inverse quantization unit 310 may be configured to receive a quantization parameter (QP) (or information generally related to inverse quantization) and quantized coefficients from the coded picture data 21 (e.g., by parsing and / or decoding by the entropy decoding unit 304), and to apply inverse quantization to the decoded quantized coefficients 309 based on the quantization parameter to obtain inverse quantized coefficients 311, which may also be referred to as transform coefficients 311.

[0177] The inverse transform processing unit 312 is configured to apply an inverse transform, e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process, to transform the coefficients to produce residual blocks in the pixel domain.

[0178] The inverse transform processing unit 312 may be configured to receive the inverse quantized coefficients 311, also referred to as transform coefficients 311, and apply a transform to the inverse quantized coefficients 311 to obtain the reconstructed residual blocks 213 in the sample domain. The reconstructed residual blocks 213 may also be referred to as transform blocks 313. The transform may be an inverse transform, e.g., an inverse DCT, an inverse DST, an inverse integer transform, or a conceptually similar inverse transform process. The inverse transform processing unit 312 may further be configured to receive transform parameters or corresponding information from the coded picture data 21 (e.g., by parsing and / or decoding by the entropy decoding unit 304) to determine the transform to apply to the inverse quantized coefficients 311.

[0179] The reconstruction unit 314 (e.g., adder 314) is configured to add the inverse transform block 313 (i.e., the reconstructed residual block 313) to the prediction block 365 to obtain the reconstruction block 315 in the sample domain, for example, by adding the sample values ​​of the reconstructed residual block 313 and the sample values ​​of the prediction block 365.

[0180] The loop filter unit 320 (either within the coding loop or after the coding loop) is configured to filter the reconstruction block 315 to obtain a filtered block 321, e.g., to smooth pixel transitions or otherwise improve video quality. The loop filter unit 320 may include one or more loop filters, such as a deblocking filter, a sample adaptive offset (SAO) filter, or one or more other filters, e.g., an adaptive loop filter (ALF), a noise suppression filter (NSF), or any combination thereof. In one example, the loop filter unit 220 may include a deblocking filter, an SAO filter, and an ALF filter. The order of the filtering processes may be deblocking filter, SAO, and ALF. In another example, a process called luma mapping with chroma scaling (LMCS) (i.e., adaptive in-loop reshaper) is added. This process is performed before deblocking. In another example, the deblocking filter process may also be applied to interior sub-block edges, e.g., affine sub-block edges, ATMVP sub-block edges, sub-block transform (SBT) edges, and intra-sub-partition (ISP) edges. Although loop filter unit 320 is shown in FIG. 13 as being an in-loop filter, in other configurations, loop filter unit 320 may be implemented as a post-loop filter.

[0181] The decoded video blocks 321 in a given frame or picture are then stored in a decoded picture buffer 330, which stores reference pictures used for subsequent motion compensation.

[0182] The decoded video blocks 321 of the picture are then stored in a decoded picture buffer 330, which stores the decoded picture 331 as a reference picture for subsequent motion compensation for other pictures and / or output for respective displays.

[0183] Decoder 30 is configured to output decoded pictures 331, for example via output 332, for presentation or display to a user.

[0184] Other variations of the video decoder 30 can be used to decode the compressed bitstream. For example, the decoder 30 can generate an output video stream without the loop filtering unit 320. For example, a non-transform-based decoder 30 can inverse quantize the residual signal directly for a particular block or frame without the inverse transform processing unit 312. In another implementation, the video decoder 30 can combine the inverse quantization unit 310 and the inverse transform processing unit 312 into a single unit.

[0185] Additionally or alternatively to the above embodiment, in another embodiment according to Figure 26, in function, the inter prediction unit 344 may be identical to the inter prediction unit 244 (in particular the motion compensation unit), and the intra prediction unit 354 may be identical to the inter prediction unit 254, and perform split or partitioning decision and prediction based on split parameters and / or prediction parameters or respective information received from the coded picture data 21 (e.g., by parsing and / or decoding by the entropy decoding unit 304). The mode application unit 360 may be configured to perform prediction (intra prediction or inter prediction) for each block based on the reconstructed picture, block, or respective samples (filtered or unfiltered) to obtain a prediction block 365.

[0186] When a video slice is coded as an intra-coded (I) slice, intra prediction unit 354 of mode application unit 360 is configured to generate a predictive block 365 for a picture block of the current video slice based on the signaled intra prediction mode and data from a previously decoded block of the current picture. When a video picture is coded as an inter-coded (i.e., B or P) slice, inter prediction unit 344 (e.g., a motion compensation unit) of mode application unit 360 is configured to generate a predictive block 365 for a video block of the current video slice based on the motion vector and other syntax elements received from entropy decoding unit 304. For inter prediction, the predictive block may be generated from one of the reference pictures in one of the reference picture lists. Video decoder 30 may construct the reference frame lists, List 0 and List 1, using a default construction technique based on the reference pictures stored in DPB 330. The same or similar may apply to or by embodiments that use tile groups (e.g., video tile groups) and / or tiles (e.g., video tiles) in addition to or instead of slices (e.g., video slices), e.g., video may be coded using I, P, or B tile groups and / or tiles.

[0187] Mode application unit 360 is configured to determine prediction information for video blocks of the current video slice by parsing the motion vectors or related information and other syntax elements, and to use the prediction information to generate predictive blocks for the current video block being decoded. For example, mode application unit 360 uses some of the received syntax elements to determine a prediction mode (e.g., intra-prediction or inter-prediction) to be used for coding the video blocks of the video slice, an inter-prediction slice type (e.g., B slice, P slice, or GPB slice), construction information for one or more of the reference picture lists for the slice, motion vectors for each inter-coded video block of the slice, an inter-prediction state for each inter-coded video block of the slice, and other information for decoding video blocks in the current video slice. The same or similar may apply to or by embodiments that use tile groups (e.g., video tile groups) and / or tiles (e.g., video tiles) in addition to or instead of slices (e.g., video slices), e.g., video may be coded using I, P, or B tile groups and / or tiles.

[0188] The embodiment of video decoder 30 shown in FIG. 13 may be configured to divide and / or decode pictures using slices (also called video slices), where a picture may be divided or decoded using one or more slices (typically non-overlapping), each of which may include one or more blocks (e.g., CTUs) or one or more groups of blocks (e.g., tiles (H.265 / HEVC and VVC) or bricks (VVC)).

[0189] The embodiment of video decoder 30 shown in FIG. 13 may be configured to divide and / or decode a picture using slices / tile groups (also referred to as video tile groups) and / or tiles (also referred to as video tiles), where a picture may be divided or decoded using one or more slice / tile groups (typically non-overlapping), where each slice / tile group may include, for example, one or more blocks (e.g., CTUs) or one or more tiles, where each tile may, for example, be rectangular in shape and may include one or more blocks (e.g., CTUs), e.g., full or partial blocks.

[0190] Other variations of the video decoder 30 may be used to decode the coded picture data 21. For example, the decoder 30 may generate an output video stream without the loop filtering unit 320. For example, a non-transform-based decoder 30 may inverse quantize the residual signal directly for a particular block or frame without the inverse transform processing unit 312. In another implementation, the video decoder 30 may combine the inverse quantization unit 310 and the inverse transform processing unit 312 into a single unit.

[0191] It should be understood that in the encoder 20 and the decoder 30, the processing result of the current step may be further processed and then output to the next step. For example, after interpolation filtering, motion vector derivation or loop filtering, further operations such as clipping or shifting may be performed on the processing result of the interpolation filtering, motion vector derivation or loop filtering.

[0192] 14 is a schematic diagram of a video coding device 400 according to one embodiment of the present disclosure. The video coding device 400 is suitable for implementing the disclosed embodiments described herein. In one embodiment, the video coding device 400 may be a decoder, such as the video decoder 30 of FIG. 11A, or an encoder, such as the video encoder 20 of FIG. 11A. In one embodiment, the video coding device 400 may be one or more components of the video decoder 30 of FIG. 11A or the video encoder 20 of FIG. 11A, described above.

[0193] Video coding device 400 comprises an ingress port 410 and a receiver unit (Rx) 420 for receiving data, a processor, logic unit, or central processing unit (CPU) 430 for processing the data, a transmitter unit (Tx) 440 and an egress port 450 for transmitting the data, and a memory 460 for storing the data. Video coding device 400 may also comprise optical-to-electrical (OE) and electrical-to-optical (EO) components coupled to ingress port 410, receiver unit 420, transmitter unit 440, and egress port 450 for the egress or ingress of optical or electrical signals.

[0194] The processor 430 is implemented by hardware and software. The processor 430 may be implemented as one or more CPU chips, cores (e.g., as a multi-core processor), FPGA, ASIC, and DSP. The processor 430 communicates with the ingress port 410, the receiver unit 420, the transmitter unit 440, the egress port 450, and the memory 460. The processor 430 includes a coding module 470. The coding module 470 implements the above-disclosed embodiments. For example, the coding module 470 implements, processes, prepares, or provides various coding operations. Thus, the inclusion of the coding module 470 provides substantial improvements to the functionality of the video coding device 400 and provides transformations of the video coding device 400 into different states. Alternatively, the coding module 470 is implemented as instructions stored in the memory 460 and executed by the processor 430.

[0195] Memory 460 may include one or more disks, tape drives, and solid-state drives, and may be used as an overflow data storage device to store programs when such programs are selected for execution and to store instructions and data read during the execution of the programs. Memory 460 may be volatile and / or non-volatile, and may be read-only memory (ROM), random access memory (RAM), ternary content addressable memory (TCAM), and / or static random access memory (SRAM).

[0196] 15 is a simplified block diagram of an apparatus 500 that may be used as either or both of the source device 310 and the destination device 320 from FIG. 11 according to an exemplary embodiment. The apparatus 500 may implement the techniques of the present application described above. The apparatus 500 may be in the form of a computing system including multiple computing devices, or in the form of a single computing device, such as a mobile phone, tablet computer, laptop computer, notebook computer, desktop computer, etc.

[0197] The processor 502 in the device 500 may be a central processing unit. Alternatively, the processor 502 may be any other type of device or devices, now existing or later developed, capable of manipulating or processing information. While the disclosed implementations may be practiced with a single processor, e.g., processor 502, as shown, advantages of speed and efficiency may be realized using two or more processors.

[0198] The memory 504 in the device 500, in one implementation, may be a read-only memory (ROM) device or a random-access memory (RAM) device. Any other suitable type of storage device may be used as the memory 504. The memory 504 may include code and data 506 accessed by the processor 502 using a bus 512. The memory 504 may further include an operating system 508 and application programs 510, which include at least one program that enables the processor 502 to perform the methods described herein. For example, the application programs 510 may include applications 1-N, which further include a video coding application that performs the methods described herein. The device 500 may also include additional memory in the form of secondary storage 514, which may be, for example, a memory card used with a mobile computing device. Because video communication sessions may contain a significant amount of information, they may be stored in whole or in part in the secondary storage 514 and loaded into the memory 504 as needed for processing. The device 500 may also include one or more output devices, such as a display 518. The display 518 may be a touch-sensitive display that combines a display with a touch-sensitive element operable to detect touch input. The display 518 may be coupled to the processor 502 via the bus 512.

[0199] The device 500 may also include one or more output devices, such as a display 518. The display 518, in one example, may be a touch-sensitive display that combines a display with a touch-sensitive element operable to detect touch input. The display 518 may be coupled to the processor 502 via the bus 512. Other output devices that enable a user to program or otherwise use the device 500 may be provided in addition to or instead of the display 518. When the output device is or includes a display, the display may be implemented in a variety of ways, including a liquid crystal display (LCD), a cathode ray tube (CRT) display, a plasma display, or a light emitting diode (LED) display, such as an organic LED (OLED) display.

[0200] The device 500 may also include or communicate with an image sensing device 520, such as a camera or any other now existing or later developed image sensing device 520 that is capable of sensing images, such as an image of a user operating the device 500. The image sensing device 520 may be positioned to be pointed at a user operating the device 500. In one example, the position and optical axis of the image sensing device 520 may be configured to be directly adjacent to the display 518 and have a field of view that includes the area from which the display 518 is viewable.

[0201] The device 500 may also include or communicate with an acoustic wave sensing device 522, such as a microphone or any other now existing or later developed acoustic wave sensing device that can detect acoustic waves in the vicinity of the device 500. The acoustic wave sensing device 522 may be positioned to face a user operating the device 500 and may be configured to receive acoustic waves, such as voice or other speech, made by the user while the user is operating the device 500.

[0202] While FIG. 15 depicts the processor 502 and memory 504 of device 500 as integrated into a single unit, other configurations may be utilized. The operations of processor 502 may be distributed across multiple machines (each machine having one or more processors) that may be coupled directly or via a local area network or other network. Memory 504 may be distributed across multiple machines, such as a network-based memory or memory within multiple machines that perform the operations of device 500. While depicted here as a single bus, bus 512 of device 500 may be comprised of multiple buses. Additionally, secondary storage 514 may be directly coupled to other components of device 500 or may be accessible over a network and may comprise a single integrated unit, such as a memory card, or multiple units, such as multiple memory cards. Thus, device 500 may be implemented in a wide variety of configurations.

[0203] Embodiments relating to relationships between partition constraint elements The present disclosure relates to relationships between partition constraint elements (which set partition rules for different picture partitioning methods) that can be signaled via a bitstream containing multiple coded pictures. Accordingly, the present invention provides devices and methods for generating, decoding, or processing such bitstreams, and in particular for including partition constraint elements in the bitstream and extracting partition constraints according to the new partition rules.

[0204] It should be noted that the names of the above syntax elements are used (throughout this specification) as they are conventionally used. However, it should be clear that these names may be changed without changing the technical context. Therefore, it is the logical meaning of the syntax elements that should be considered important.

[0205] Currently, partition constraint elements (e.g., MaxBtSizeY, MaxMttDepth, and MinQtSizeY) are signaled individually within their definition ranges. In particular, currently, the syntax element for CtbSizeY (i.e., log2_ctu_size_minus2), and the syntax elements for MinQtSizeY (i.e., log2_min_qt_size_intra_slices_minus2 and log2_min_qt_size_inter_slices_minus2), and the syntax element for MaxMttDepth (i.e., max_mtt_hierarchy_depth_inter_slices and max_mtt_hierarchy_depth_intra_slices) are signaled within the sequence parameter set (SPS). Additionally, the syntax element for the difference between the luma CTB size and MaxBtSizeY (i.e., log2_diff_ctu_max_bt_size) is signaled within the slice header. The SPS and slice headers may be included in the picture bitstream.

[0206] An exemplary SPS syntax, specifically the SPS Raw Byte Sequence Payload (RBSP) syntax, is shown in Figure 8. The SPS RBSP semantics of this syntax are as follows:

[0207] "pic_width_in_luma_samples" specifies the width of each decoded picture in units of luma samples and must be non-equal to 0 and an integer multiple of MinCbSizeY.

[0208] "pic_height_in_luma_samples" specifies the height of each decoded picture in units of luma samples and must be non-equal to 0 and an integer multiple of MinCbSizeY.

[0209] "log2_ctu_size_minus2" plus 2 specifies the luma CTB size for each CTU.

[0210] The elements CtbLog2SizeY, CtbSizeY, MinCbLog2SizeY, MinCbSizeY, MinTbLog2SizeY, MaxTbLog2SizeY, PicWidthInCtbsY, PicHeightInCtbsY, PicSizeInCtbsY, PicWidthInMinCbsY, PicHeightInMinCbsY, PicSizeInMinCbsY, PicSizeInSamplesY, PicWidthInSamplesC, and PicHeightInSamplesC are typically derived as follows. · CtbLog2SizeY = log2_ctu_size_minus2 + 2 · CtbSizeY = 1 << CtbLog2SizeY · MinCbLog2SizeY = 2 · MinCbSizeY = 1 << MinCbLog2SizeY · MinTbSizeY = 4 · MaxTbSizeY = 64 · PicWidthInCtbsY = Ceil(pic_width_in_luma_samples ÷ CtbSizeY) · PicHeightInCtbsY = Ceil(pic_height_in_luma_samples ÷ CtbSizeY) · PicSizeInCtbsY = PicWidthInCtbsY * PicHeightInCtbsY · PicWidthInMinCbsY = pic_width_in_luma_samples / MinCbSizeY · PicHeightInMinCbsY = pic_height_in_luma_samples / MinCbSizeY · PicSizeInMinCbsY = PicWidthInMinCbsY * PicHeightInMinCbsY · PicSizeInSamplesY = pic_width_in_luma_samples * pic_height_in_luma_samples ·PicWidthInSamplesC=pic_width_in_luma_samples / SubWidthC ·PicHeightInSamplesC=pic_height_in_luma_samples / SubHeightC

[0211] "log2_min_qt_size_intra_slices_minus2" plus 2 specifies the minimum luma size of leaf blocks resulting from quadtree decomposition of CTUs in slices with slice_type equal to 2(I). The value of log2_min_qt_size_intra_slices_minus2 must be in the range 0 to CtbLog2SizeY-2, inclusive. ·MinQtLog2SizeIntraY=log2_min_qt_size_intra_slices_minus2+2

[0212] "log2_min_qt_size_inter_slices_minus2" plus 2 specifies the minimum luma size of leaf blocks resulting from the quadtree decomposition of CTUs in slices with slice_type equal to 0 (B) or 1 (P). The value of log2_min_qt_size_inter_slices_minus2 must be in the range 0 to CtbLog2SizeY-2, inclusive. ·MinQtLog2SizeInterY=log2_min_qt_size_inter_slices_minus2+2

[0213] 'max_mtt_hierarchy_depth_inter_slices' specifies the maximum hierarchical depth of coding units resulting from multi-type tree partitioning of quadtree leaves in slices with slice_type equal to 0 (B) or 1 (P). The value of max_mtt_hierarchy_depth_inter_slices must be in the range 0 to CtbLog2SizeY - MinTbLog2SizeY, inclusive.

[0214] 'max_mtt_hierarchy_depth_intra_slices' specifies the maximum hierarchical depth of coding units resulting from multitype tree partitioning of quadtree leaves in slices with slice_type equal to 2(I). The value of max_mtt_hierarchy_depth_intra_slices must be in the range 0 to CtbLog2SizeY-MinTbLog2SizeY, inclusive.

[0215] Additionally, Figure 9 shows an example slice header syntax. The slice header semantics of this syntax are as follows:

[0216] 'log2_diff_ctu_max_bt_size' specifies the difference between the luma CTB size and the maximum luma size (width or height) of a coding block that can be split using binary splitting. The value of log2_diff_ctu_max_bt_size must be in the range 0 to CtbLog2SizeY - MinCbLog2SizeY, inclusive.

[0217] If log2_diff_ctu_max_bt_size is not present, the value of log2_diff_ctu_max_bt_size is inferred to be equal to 2.

[0218] The elements MinQtLog2SizeY, MaxBtLog2SizeY, MinBtLog2SizeY, MaxTtLog2SizeY, MinTtLog2SizeY, MaxBtSizeY, MinBtSizeY, MaxTtSizeY, MinTtSizeY, and MaxMttDepth are typically derived as follows: ·MinQtLog2SizeY=(slice_type==I)?MinQtLog2SizeIntraY:MinQtLog2SizeInterY ·MaxBtLog2SizeY=CtbLog2SizeY-log2_diff_ctu_max_bt_size ·MinBtLog2SizeY=MinCbLog2SizeY ·MaxTtLog2SizeY=(slice_type==I)?5:6 ·MinTtLog2SizeY=MinCbLog2SizeY MinQtSizeY=1< <MinQtLog2SizeY MaxBtSizeY=1< <MaxBtLog2SizeY MinBtSizeY=1< <MinBtLog2SizeY MaxTtSizeY=1< <MaxTtLog2SizeY MinTtSizeY=1< <MinTtLog2SizeY ·MaxMttDepth=(slice_type==I)?max_mtt_hierarchy_depth_intra_slices:max_mtt_hierarchy_depth_inter_slices

[0219] As shown in Figure 8, currently, the width and height of a picture sequence (e.g., a picture in a video sequence) are indicated using the syntax elements "pic_width_in_luma_samples" and "pic_height_in_luma_samples". Traditionally, these syntax elements need to be multiples of MinCbSizeY to ensure that the picture can be divided into minimum-sized coding blocks. However, although pic_height_in_luma_samples and pic_width_in_luma_samples must be multiples of MinCbSizeY, it is still not guaranteed that the picture can be perfectly divided into blocks using the available quadtree, binary tree, and ternary tree partitioning methods, respectively. An example of this problem is explained as follows:

[0220] For example, MinCbSizeY is equal to 4 MinQtSizeY equals 32 MaxBtSizeY equals 16 Assume that:

[0221] In this example, if pic_width_in_luma_samples=132 and pic_width_in_luma_samples=132, the picture width and height are multiples of 4, but it is still not possible to divide the picture perfectly. The reason is that a parent block of size, for example, 32x32, cannot be divided using binary or quadtree division; if quadtree division is applied, it would violate the MinQtSizeY limit, and if binary division is applied, it would violate the MaxBtSizeY limit. In other words, it is not possible to generate a block whose width or height is equal to 4, but that is necessary to divide the picture perfectly.

[0222] Thus, as illustrated above, there is currently a problem of splitting pictures in a way that fully respects the traditional splitting rules, which is actually a big problem since it means that some picture sizes cannot be coded and decoded.

[0223] 1 illustrates a device 100 according to a general embodiment of the present invention. The device 100 is suitable for generating or processing, i.e., manipulating, a bitstream 101 including coded pictures, e.g., a video bitstream. The device 100 may be included in or comprise an encoder configured to encode pictures and generate the bitstream 101. The device 100 may determine relationships between partitioning constraint elements and may include these partitioning constraint elements and / or their relationships in the bitstream 101. The device 100 may thereby add the partitioning constraint elements and / or the relationships to SPS syntax and / or slice header syntax in the bitstream 101.

[0224] In a similar manner, a decoder device parses the bitstream and / or infers from it constraint parameters associated with the partitions by applying predefined rules. The constraint parameters then serve to accurately decode and reconstruct the partitions. The encoder and decoder operate on (process) the same syntax.

[0225] In particular, device 100 is configured to determine MaxBtSizeY 102 and / or to determine MaxMttDepth 103 and MinCbSizeY 104. Then, device 100 is configured to determine MinQtSizeY 105 based on MaxBtSizeY 102 and / or based on MaxMttDepth 103 and MinCbSizeY 104. Finally, device 100 is configured to include the determined MinQtSizeY 105 in bitstream 101 either indirectly (i.e., by including information from which MinQtSizeY 105 can be derived) or directly.

[0226] In a device 100 according to a first particular embodiment of the present invention, which is based on the device 100 shown in FIG. 1 , the range of values ​​for MinQtSizeY 105 may be limited based on the value of MaxBtSizeY 102. For example, the upper limit of MinQtSizeY 105 may be limited to MaxBtSizeY 102. In other words, in a device 100 according to the first particular embodiment, the minimum value of MinQtSizeY 105 cannot be greater than MaxBtSizeY 102.

[0227] Alternatively, or in addition, in a device 100 according to a first particular embodiment, the range of values ​​for MinQtSizeY 105 may be limited based on MaxMttDepth 103, for example: · When MaxMttDepth 103 is equal to 0, MinQtSizeY 105 may be equal to (or be inferred to be) MinCbSizeY 104. · When MaxMttDepth 103 is greater than 0, the upper limit of MinQtSizeY 105 may be equal to (MinCbSizeY 104 << MaxMttDepth 103). In other words, MinQtSizeY 105 must not be greater than (MinCbSizeY 104 << MaxMttDepth 103).

[0228] Specifically, throughout this specification, the operation x << y can be described mathematically as x y where x is a power of 2 and n is a non - negative integer. In other words, x << y represents shifting x left by y bits.

[0229] Thus, the device 100 according to the first specific embodiment is configured to set the relationship between MinQtSizeY 105 and MinCbSizeY 104, MaxBtSizeY 102, and / or MaxMttDepth 103 such that, in particular, the smallest partition block can be realized by recursively dividing the parent block using a combination of a quadtree splitting method or a binary splitting method. The smallest block (whose size is indicated by MinCbSizeY 104) can be generated by the available splitting methods, so it is possible to completely divide a picture with a size that is a multiple of MinCbSizeY 104.

[0230] 2 and 3 show an SPS RPBS syntax 200 and a slice header syntax 300 for a device according to a second specific embodiment of the present invention. In particular, the conventional SPS RBSP syntax shown in FIG. 8 is modified to the syntax 200 shown in FIG. 2 (new elements are marked in bold and deleted elements are crossed out). Furthermore, the conventional slice header syntax shown in FIG. 9 is modified to the syntax 300 shown in FIG. 3 (new elements are marked in bold and deleted elements are crossed out). The device according to the second specific embodiment may be based on the device 100 shown in FIG. 1 or may be a separate embodiment of the present invention. In the device according to the second specific embodiment, the following are implemented: MaxBtSizeY 102 is signaled in the bitstream 101 relative to MinQtSizeY 105. In other words, as shown in Figure 3, a syntax element 301 of the difference between MaxBtSizeY 102 and MinQtSizeY 105 may be signaled in the bitstream 101 (e.g., using a syntax element such as log2_diff_max_bt_size_min_qt_size), and MaxBtSizeY 102 can be derived (e.g., at a decoder of the bitstream 101) based on MinQtSizeY 105 and log2_diff_max_bt_size_min_qt_size. In this case, in one example, ·MaxBtSizeY 102=MinQtSizeY 105< <log2_diff_max_bt_size_min_qt_size and In particular, in this example, a syntax element 301 of the difference between MaxBtSizeY 102 and MinQtSizeY 105 is signaled in a logarithmic scale (in particular in base 2). In this example, log2_diff_max_bt_size_min_qt_size can only have a positive integer value or a zero value. MaxMttDepth 103 is signaled in the bitstream 101 in relation to MinQtSizeY 105 and MinCbSizeY 104. As shown in FIG. 2, the syntax element 201 of the difference between the log2 values ​​of MaxMttDepth 103 and MinQtSizeY 105 is (for example, the syntax element: diff_max_mtt_hierarchy_depth_log2_min_qt_size) in the bitstream 101. Two such syntax elements 201 are shown in Figure 2, one for inter_slices and one for intra_slices. In this case, in the example: ·MaxMttDepth 103 = diff_max_mtt_hierarchy_depth_log2_min_qt_size + log2(MinQtSizeY) - log2(MinCbSizeY). In particular, in this example it is again assumed that diff_max_mtt_hierarchy_depth_log2_min_qt_size is signaled in a logarithmic scale: the function log2(x) corresponds to the logarithm of x in base 2.

[0231] In Figure 3, "log2_diff_max_bt_size_min_qt_size" specifies the difference between the maximum luma size (width or height) of a coding block that can be divided using binary division and the minimum luma size (width or height) of a coding block that can be divided using quadtree division. The value of log2_diff_ctu_max_bt_size must be in the range 0 to CtbLog2SizeY - MinQtLog2SizeY, inclusive.

[0232] 4 shows SPS RPBS syntax 400 for a device according to a third specific embodiment of the present invention. The device according to the third specific embodiment may be based on device 100 shown in FIG. 1 or may be a separate embodiment of the present invention. As mentioned previously, conventionally, signaled picture size elements 405 and 406 (pic_width_in_luma_samples and pic_height_in_luma_samples) are each specified as an integer multiple of MinCbSizeY 104.

[0233] In contrast, in a first implementation of a device according to the third specific embodiment, picture size elements 405 and 406 may be constrained to have only values ​​that are integer multiples of MinQtSizeY 105. The advantage is that boundary blocks can always have quadtree partitioning as an available partitioning method.

[0234] In a second implementation of a device according to the third particular embodiment, the picture width and height may be signaled in the bitstream 101 based on MinQtSizeY 105. In particular, the conventional SPS RBSP syntax shown in Figure 8 is modified according to the syntax 400 shown in Figure 4 (new elements are marked in bold and removed elements are crossed out).

[0235] 4, four syntax elements 401-404 may be signaled, in particular two syntax elements (height / width) per intra_slice and two syntax elements (height / width) per inter_slice (e.g., log2_diff_pic_height_min_Qt and log2_diff_pic_width_min_Qt). Preferably, these syntax elements 401-404 are signaled instead of the actual picture size elements 405 and 406, and the picture width and height may be determined using the following relationships: Picture width in luma samples = MinQtSizeY 105< <log2_diff_pic_width_min_Qt Picture height in luma samples = MinQtSizeY 105< <log2_diff_pic_height_min_Qt

[0236] The syntax elements 401-404, especially the deltas, can be represented in the SPS syntax 400 based on a logarithmic scale.

[0237] 5 and 6 show two slice header syntaxes 5000 and 600, respectively, for a device according to a fourth specific embodiment of the present invention, which may be based on device 100 shown in FIG. 1 or may be a separate embodiment of the present invention.

[0238] A device according to a fourth particular embodiment of the present invention is concerned with the problem that if MaxMttDepth 103 is inferred, signaled, or indicated to be equal to 0, MaxBtSizeY 102 (or MaxTtSizeY) can still be present in the bitstream 101 and can have a value greater than MinCbSizeY 104, and MinBtSizeY (or MinTtSizeY) can still have a value equal to MinCbSizeY 104. This condition can create ambiguity in the operation of the encoder and / or decoder, with the result that complete division of the picture frame may still not be possible.

[0239] In a first implementation of a device according to the fourth particular embodiment, MaxBtSizeY 102 (or MaxTtSizeY) is signaled or indicated in the bitstream 101 based on the value of MaxMttDepth 103. In particular, the conventional slice header syntax shown in Figure 9 is changed to syntax 5000 shown in Figure 5 (new elements are marked in bold). That is, the device is configured to include in the bitstream 101 a syntax element 5001 for MaxBtSizeY 102 (or MaxTtSizeY) that depends on MaxMttDepth 103.

[0240] Specifically, when MaxMttDepth 103 is equal to 0, MaxBtSizeY 102 (or MaxTtSizeY) may not be signaled in the bitstream 101 but may be inferred (e.g., at a decoder) to be equal to MinCbSizeY 104. Alternatively, MaxBtSizeY 102 (or MaxTtSizeY) may be inferred (e.g., at a decoder) to be equal to a default predefined value, such as 4 or 8, when MaxMttDepth 103 is equal to 0.

[0241] 'log2_diff_ctu_max_bt_size' again specifies the difference between the luma CTB size and the maximum luma size (width or height) of a coding block that can be split using binary splitting. The value of log2_diff_ctu_max_bt_size must be in the range 0 to CtbLog2SizeY - MinCbLog2SizeY, inclusive.

[0242] When log2_diff_ctu_max_bt_size is not present, the following may apply: if slice_type is equal to 2(I) and max_mtt_hierarchy_depth_intra_slices is equal to 1, the value of log2_diff_ctu_max_bt_size may be inferred to be equal to 2. Otherwise, the value of log2_diff_ctu_max_bt_size may be inferred to be equal to CtbLog2SizeY - MinCbLog2SizeY.

[0243] In a second implementation of the device according to the fourth particular embodiment, MaxMttDepth 103 is signaled or indicated in the bitstream 101 based on the value of MaxBtSizeY 102 (or MaxTtSizeY). In particular, the conventional slice header syntax shown in Figure 9 is modified to syntax 600 shown in Figure 6 (new elements are marked in bold). That is, the device is configured to include in the bitstream 101 a syntax element 601 for MaxMttDepth 103 that depends on MaxBtSizeY 102 or MaxTtSizeY.

[0244] Specifically, when MaxBtSizeY 102 (or MaxTtSizeY) is equal to 0, MaxMttDepth 103 may not be signaled but may be inferred to be equal to 0, for example, in a decoder. If the value of MaxMttDepth 103 is equal to 0, it means that binary splitting is not allowed to be applied. In this solution, the MaxBtSizeY 102 and MaxTtSizeY syntax elements should be signaled before MaxMttDepth 103, but are not restricted in any parameter set header.

[0245] "MaxTtSizeY" is defined as the maximum luma size (width or height) in number of samples of a coding block that can be divided using ternary partitioning. "MinTtSizeY" is defined as the minimum luma size (width or height) in number of samples of a coding block that can be divided using ternary partitioning.

[0246] In Figure 6, "max_mtt_hierarchy_depth" specifies the maximum hierarchical depth of coding units resulting from multitype tree partitioning of quadtree leaves within a slice. The value of max_mtt_hierarchy_depth_inter_slices must be in the range 0 to CtbLog2SizeY-MinTbLog2SizeY, inclusive. When max_mtt_hierarchy_depth is not present, the value of max_mtt_hierarchy_depth is inferred to be 0.

[0247] Figure 7 shows a method 7000 according to one embodiment of the present invention. The method 7000 is specifically for manipulating the bitstream 101 and may be performed by the device 100 shown in Figure 1. The method 7000 may also be performed by an encoder that encodes pictures of the bitstream 101 into the bitstream 101, i.e., generates the bitstream 101.

[0248] The method 7000 includes a step 7001 of determining MaxBtSizeY 102 and / or determining MaxMttDepth 103 and MinCbSizeY 104. Further, the method 7000 includes a step 7002 of determining MinQtSizeY 105 based on MaxBtSizeY 102 and / or based on MaxMttDepth 103 and MinCbSizeY 104. Finally, the method 7000 includes a step 7003 of including the determined MinQtSizeY 105 in the bitstream 101.

[0249] FIG. 10 illustrates a device 1000 according to a general embodiment of the present invention. The device 1000 is suitable for generating or processing, i.e., manipulating, a bitstream 101 including coded pictures, e.g., a video bitstream. The device 1000 may be included in or comprise an encoder configured to encode pictures and generate the bitstream 101. The device 1000 may determine relationships between partitioning constraint elements and include these partitioning constraint elements and / or their relationships in the bitstream 101. The device 1000 may thereby add the partitioning constraint elements and / or relationships to SPS syntax and / or slice header syntax in the bitstream 101. The device 1000 is an alternative to the device 100 illustrated in FIG. 1. However, features of the device 1000 described below with respect to FIG. 10 may be combined with the above-described features of devices according to the first, second, third, or fourth specific embodiments (if they are not based on the device 100 of FIG. 1).

[0250] In particular, the device 1000 is configured to determine MinQtSizeY (105). Furthermore, the device is configured to determine MaxBtSizeY 102 and / or determine MaxMttDepth 103 based on MinQtSizeY 105. Finally, the device is configured to include the determined MaxBtSizeY 102 and / or the determined MaxMttDepth 103 in the bitstream (101) either indirectly (i.e., by including information from which MaxBtSizeY 102 and / or MaxMttDepth 103 can be derived) or directly.

[0251] For example, the device 1000 may determine MaxBtSizeY 102 taking into account that its lower limit is MinQtSizeY 105. That is, the range of values ​​for MaxBtSizeY 102 may be limited by the value of MinQtSizeY 105. For example, the lower limit of MaxBtSizeY 102 may be limited to MinQtSizeY 105. In other words, in the device 1000, the minimum value of MaxBtSizeY 102 cannot be smaller than MinQtSizeY 105.

[0252] Alternatively, or in addition, the device 1000 may determine MaxMttDepth 103 by considering that its upper limit is the difference between the log2 value of MinQtSizeY 105 and the log2 value of MinCbSizeY 104. That is, the maximum value of MaxMttDepth 103 cannot be greater than the difference between the log2 value of MinQtSizeY 105 and the log2 value of MinCbSizeY 104.

[0253] In summary, a first aspect of the present invention provides a device for generating or processing a bitstream including coded pictures, the device being configured to determine a MaxBtSizeY and / or determine a MaxMttDepth and a MinCbSizeY, determine a MinQtSizeY based on the MaxBtSizeY and / or based on the MaxMttDepth and MinCbSizeY, and include the determined MinQtSizeY in the bitstream.

[0254] By determining MinQtSizeY based on MaxBtSizeY and / or based on MaxMttDepth and MinCbSizeY, i.e., by defining new partitioning rules through setting the relationship between these partitioning constraint elements, the device of the first aspect realizes increased availability and flexibility of various picture partitioning methods, particularly quadtree partitioning and binary tree partitioning.

[0255] In an implementation form of the first aspect, the device is configured to determine MinQtSizeY taking into account that its upper limit is MaxBtSizeY, and / or to determine MinQtSizeY taking into account that its upper limit is MinCbSizeY raised to the power MaxMttDepth.

[0256] By establishing such relationships between the partitioning constraints MinQtSizeY, MinCbSizeY, MaxBtSizeY, and MaxMttDepth, respectively, it is guaranteed that the smallest partition blocks can be achieved by recursively partitioning the parent block using a combination of quadtree or binary partitioning methods. Since the smallest blocks (whose size is indicated by MinCbSizeY) can be generated by any available partitioning method, it is possible to completely partition a picture whose size is a multiple of MinCbSizeY.

[0257] In a further implementation of the first aspect, the device is configured to determine that MinQtSizeY is MinCbSizeY when MaxMttDepth is equal to zero, and to determine that MinQtSizeY is MinCbSizeY raised to the power of MaxMttDepth when MaxMttDepth is greater than zero.

[0258] This provides an efficient implementation of the previous implementation to allow for full division of the picture using quadtree and binary tree division, respectively.

[0259] A second aspect of the present invention provides a device for generating or processing a bitstream including coded pictures, the device being configured to determine a MinQtSizeY, determine a MaxBtSizeY, and / or determine a MaxMttDepth based on the MinQtSizeY, and include the determined MaxBtSizeY and / or the determined MaxMttDepth in the bitstream.

[0260] In an implementation form of the second aspect, the device is configured to determine MaxBtSizeY taking into account that its lower limit is MinQtSizeY, and / or to determine MaxMttDepth taking into account that its upper limit is the difference between the log2 value of MinQtSizeY and the log2 value of MinCbSizeY.

[0261] Similar to the first aspect, the second aspect defines new partitioning rules by setting relationships between partitioning constraint elements. In this way, the device of the second aspect realizes improved availability and flexibility of various picture partitioning methods, particularly quadtree partitioning and binary tree partitioning.

[0262] In a further implementation of the first or second aspect, the device is configured to include in the bitstream an indication of the relationship, in particular the difference, between MinQtSizeY and MaxBtSizeY.

[0263] In this way, the relevant partitioning constraint elements can be easily inferred, for example, at the decoder side, while reducing the information overhead in the bitstream. A difference is an example of a relationship. However, the relationship could also be a proportionality factor, a calculation method, etc., which allows inferring MaxBtSizeY from MinQtSizeY.

[0264] In a further implementation of the first or second aspect, the device is configured to include in the bitstream an indication of the relationship, in particular the difference, between the log2 value of MinQtSizeY and MaxMttDepth.

[0265] In this way, the relevant partitioning constraint elements can be inferred, for example, at the decoder side, while reducing the information overhead in the bitstream.

[0266] In a further implementation of the first or second aspect, the device is configured to determine one or more picture size elements indicating the size, in particular the height and width, of pictures in the bitstream to be integer multiples of MinQtSizeY and include the one or more picture size elements in the bitstream.

[0267] As a result, the boundary blocks always have quadtree partitioning as an available partitioning method.

[0268] In a further implementation of the first or second aspect, the device is configured to determine one or more picture size elements that indicate the size, in particular the height and width, of pictures in the bitstream, and to include in the bitstream an indication of the relationship between the picture size elements and MinQtSizeY.

[0269] In this way, the relevant partitioning constraint elements can be inferred, for example, at the decoder side, while reducing the information overhead in the bitstream.

[0270] In a further implementation of the first or second aspect, the indication of the relationship between the picture size factor and MinQtSizeY is based on a logarithmic scale.

[0271] In a further implementation of the first or second aspect, the device is configured to include in the bitstream an indication of MaxBtSizeY or MaxTtSizeY that depends on MaxMttDepth.

[0272] In this way, ambiguities previously introduced in the operation of the encoder and / or decoder are eliminated, allowing for perfect division of picture frames.

[0273] In a further implementation of the first or second aspect, the device is configured to not include any indication of MaxBtSizeY or MaxTtSizeY in the bitstream if MaxMttDepth is equal to zero.

[0274] This allows for a reduction in information overhead in the bitstream.

[0275] In a further implementation of the first or second aspect, the device is configured to include in the bitstream an indication of MaxMttDepth that depends on MaxBtSizeY or MaxTtSizeY.

[0276] In this way, ambiguities previously introduced in the operation of the encoder and / or decoder are eliminated, allowing for perfect division of picture frames.

[0277] In a further implementation of the first or second aspect, the device is configured to not include any indication of MaxMttDepth in the bitstream if MaxBtSizeY or MaxTtSizeY is equal to zero.

[0278] This allows for a reduction in information overhead in the bitstream.

[0279] In a further implementation of the first or second aspect, the device comprises or is included in an encoder configured to encode pictures of the bitstream.

[0280] A third aspect of the present invention provides a method for generating or processing a bitstream including coded pictures, the method comprising the steps of determining a MaxBtSizeY and / or determining a MaxMttDepth and a MinCbSizeY, determining a MinQtSizeY based on the MaxBtSizeY and / or based on the MaxMttDepth and MinCbSizeY, and including the determined MinQtSizeY in the bitstream.

[0281] In an implementation form of the third aspect, the method includes a step of determining MinQtSizeY taking into account that its upper limit is MaxBtSizeY, and / or determining MinQtSizeY taking into account that its upper limit is MinCbSizeY raised to the power MaxMttDepth.

[0282] In a further implementation of the third aspect, the method includes the steps of determining that MinQtSizeY is MinCbSizeY if MaxMttDepth is equal to zero, and determining that MinQtSizeY is MinCbSizeY to the power of MaxMttDepth if MaxMttDepth is greater than zero.

[0283] In a further implementation of the third aspect, the method includes the step of including in the bitstream an indication of the relationship, in particular the difference, between MinQtSizeY and MaxBtSizeY.

[0284] In a further implementation of the third aspect, the method includes including in the bitstream an indication of the relationship, in particular the difference, between MinQtSizeY and MaxMttDepth.

[0285] In a further implementation of the third aspect, the method includes determining one or more picture size elements indicating the size, in particular the height and width, of pictures in the bitstream to be integer multiples of MinQtSizeY, and including the one or more picture size elements in the bitstream.

[0286] In a further implementation of the third aspect, the method includes determining one or more picture size elements indicating the size, in particular the height and width, of pictures in the bitstream, and including in the bitstream an indication of the relationship between the picture size elements and MinQtSizeY.

[0287] In a further implementation of the third aspect, the indication of the relationship between the picture size factor and MinQtSizeY is based on a logarithmic scale.

[0288] In a further implementation of the third aspect, the method includes including in the bitstream an indication of MaxBtSizeY or MaxTtSizeY that depends on MaxMttDepth.

[0289] In a further implementation of the third aspect, the method includes the step of not including any indication of MaxBtSizeY or MaxTtSizeY in the bitstream if MaxMttDepth is equal to zero.

[0290] In a further implementation of the third aspect, the method includes including in the bitstream an indication of MaxMttDepth that is dependent on MaxBtSizeY or MaxTtSizeY.

[0291] In a further implementation of the third aspect, the method includes not including any indication of MaxMttDepth in the bitstream if MaxBtSizeY or MaxTtSizeY is equal to zero.

[0292] In a further implementation of the third aspect, the method is performed in an encoder that encodes pictures of a bitstream.

[0293] The method of the third aspect and its implementations may achieve all the advantages and effects described above for the corresponding device of the first aspect and its respective implementations. A further aspect of the present invention is a method for generating or processing a bitstream, corresponding to the device of the second aspect.

[0294] A fourth aspect of the present invention provides a device for generating or processing a bitstream comprising coded pictures, the device being configured to include in the bitstream an indication of the relationship, in particular the difference, between MinQtSizeY and MaxBtSizeY and / or to include in the bitstream an indication of the relationship, in particular the difference, between MinQtSizeY and MaxMttDepth.

[0295] The device of the fourth aspect allows for signaling relevant partitioning constraint elements while reducing information overhead in the bitstream. The partitioning constraint elements can be inferred, for example, at the decoder side. The device of the fourth aspect provides increased availability and flexibility of various picture partitioning methods, particularly quadtree and binary tree partitioning.

[0296] A fifth aspect of the present invention provides a device for generating or processing a bitstream comprising coded pictures, the device being configured to: determine one or more picture size elements indicating the size, in particular the height and width, of pictures in the bitstream to be integer multiples of MinQtSizeY and include the one or more picture size elements in the bitstream; or determine one or more picture size elements indicating the size, in particular the height and width, of pictures in the bitstream and include an indication of the relationship between the picture size elements and MinQtSizeY.

[0297] The device of the fifth aspect allows boundary blocks to always have quadtree partitioning as an available partitioning method. Furthermore, the relevant partitioning constraint elements can be inferred, for example, at the decoder side, while reducing information overhead in the bitstream. The device of the fifth aspect realizes improved availability and flexibility of various picture partitioning methods, especially quadtree partitioning and binary tree partitioning.

[0298] A sixth aspect of the present invention provides a device for generating or processing a bitstream including coded pictures, the device being configured to include in the bitstream an indication of MaxBtSizeY or MaxTtSizeY that depends on MaxMttDepth, or to include in the bitstream an indication of MaxMttDepth that depends on MaxBtSizeY or MaxTtSizeY.

[0299] The device of the sixth aspect reduces or eliminates ambiguities traditionally introduced in the operation of an encoder and / or decoder, thus supporting full partitioning of picture frames. The device of the sixth aspect realizes increased availability and flexibility of various picture partitioning methods, particularly quadtree partitioning and binary tree partitioning.

[0300] A seventh aspect of the present invention provides a computer program product comprising program code for controlling a device to perform the method of the third aspect and its respective implementations when the program code is executed by one or more processors of the device. The device may be a device of the first, second, fourth, fifth or sixth aspect, or any implementation thereof.

[0301] An eighth aspect of the present invention provides a device for decoding a bitstream generated or processed by a device according to the first, second, fourth, fifth or sixth aspect, or any implementation thereof.

[0302] Embodiments Related to Boundary Splitting The next-generation standard for Versatile Video Coding (VVC) is the latest collaborative video project between the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) Video Coding Experts Group (VCEG) and the International Organization for Standardization / International Electrotechnical Commission (ISO / IEC) Moving Picture Experts Group (MPEG) standardization organizations, working together in a partnership known as the Joint Video Research Team (JVET). In VVC, a multi-type (binary / ternary / quaternary) tree (BT / TT / QT or binary tree / ternary tree / quad tree) segmentation structure replaces or potentially replaces the concept of multiple segmentation unit types; i.e., it removes the separation of the concepts of CU, PU, ​​and TU, except when needed for CUs whose size is too large for the maximum transform length, and supports flexibility for CU segmentation shapes [JVET-J1002].

[0303] Figures 6A-6F illustrate, by way of example, partitioning modes currently used within a VTM. Figure 16A illustrates an unpartitioned block (no partition), Figure 16B illustrates quaternary or quadtree (QT) partitioning, Figure 16C illustrates horizontal binary or binary tree (BT) partitioning, Figure 16D illustrates vertical binary or binary tree (BT) partitioning, Figure 16E illustrates horizontal ternary or ternary tree (TT) partitioning, and Figure 16F illustrates vertical ternary or ternary tree (TT) partitioning of a block such as a CU or CTU. An embodiment may be configured to implement the partitioning modes illustrated in Figures 16A-16F.

[0304] In an embodiment, the following parameters may be defined and specified by a sequence parameter set (SPS) syntax element for the BT / TT / QT coding tree scheme: CTU size: Root node size of the quadtree MinQTSize: The minimum allowable quadtree leaf node size MaxBTTSize: Maximum allowed binary and ternary tree root node size MaxBTTDepth: Maximum allowable binary and ternary tree depth MinBTTSize: The minimum allowable binary and ternary tree leaf node size

[0305] In other embodiments, the minimum allowed quadtree leaf node size MinQTSize parameter may also be included in other headers or sets, for example, the slice header (SH) or the picture parameter set (PPS).

[0306] In the HEVC standard, a coding tree unit (CTU) or coding unit (CU) located on a slice / picture boundary is forced to split using a quadtree (QT) until the bottom-right sample of the leaf node is located within the slice / picture boundary. Because both the encoder and decoder, e.g., video encoder 20 and video decoder 30, know when to apply a forced QT, the forced QT partition or split does not need to be signaled in the bitstream. The purpose of the forced split is to allow boundary CTUs / CUs by video encoder 20 / video decoder 30.

[0307] International Patent Publication No. WO2016 / 090568 discloses a QTBT (quadtree plus binary tree) structure, and VTM1.0 also inherits the boundary CTU / CU forced splitting process from HEVC. That is, CTUs / CUs located on frame boundaries are forced to split by the quadtree (QT) structure without considering rate-distortion (RD) optimization until the entire current CU fits within the picture boundary. These forced splits are not signaled in the bitstream.

[0308] Figure 7A shows an example of forced splitting for a lower-boundary CTU (128x128) of a high-definition (HD) (1920x1080 pixels) picture split by forced QT. In Figure 17, the HD picture has or is 1920x1080 pixels, and the CTU has or is 128x128 pixels.

[0309] In the San Diego Conference (April 2018), SubCE2 (Picture Boundary Processing) of CE1 (Splitting) [JVET-J1021] proposed 15 tests for picture boundary processing using BT, TT, or ABT (Asymmetric BT). For example, in JVET-K0280 and JVET-K0376, the boundary is defined as shown in Figure 18. Figure 18 shows the picture boundary by the dotted hash line and the regions of the straight boundary cases: bottom boundary case, corner boundary case, and right boundary case. The bottom boundary can be split by horizontally forced BT or forced QT, the right boundary can be split by vertically forced BT or forced QT, and the corner cases can only be split by forced QT. The decision of whether to use forced BT or forced QT splitting is based on a rate-distortion optimization criterion and is signaled in the bitstream. Forced splitting means that a block must be split; for example, forced splitting applies to boundary blocks that may not be coded using the "no split" option shown in Figure 16A.

[0310] When forced QT splitting is used with forced boundary splitting, the MinQTSize splitting constraint is ignored. For example, in Figure 19A, if MinQTSize is reported as 32 in the SPS, a QT split with block size 8x8 is required to match the boundaries with the forced QT method, thereby ignoring the MinQTSize constraint of 32.

[0311] According to embodiments of the present disclosure, when forced QT is used for picture boundary partitioning, the forced QT partitioning follows, e.g., does not ignore, the partitioning constraint signaled, for example, in the SPS. If further forced partitioning is required, only forced BT is used, which may be referred to in combination as forced QTBT. In embodiments of the present disclosure, for example, the partitioning constraint MinQTSize is taken into account for forced QT partitioning at picture boundaries, and further notification for forced BT partitioning is not required. The embodiments also enable harmonization of partitioning of regular (non-border) blocks and boundary blocks. For example, conventional solutions require two "MinQTSize" parameters, one for regular block partitioning and the other for boundary block partitioning. The embodiments require only one common "MinQTSize" parameter for both regular block and boundary block partitioning, which may be flexibly set between the encoder and the decoder, for example, by signaling one "MinQTSize" parameter. Furthermore, the embodiments require fewer partitions than, for example, forced QT.

[0312] Solutions for the lower and right bounding cases In the bottom and right boundary cases, if the block size is larger than MinQTSize, the partitioning mode for picture boundary partitioning can be selected between forced BT partitioning and forced QT partitioning, for example, based on RDO (rate distortion optimization). Otherwise (i.e., if the block size is equal to or smaller than MinQTSize), only forced BT partitioning is used for picture boundary partitioning. More specifically, horizontal forced BT is used for the bottom boundary of the border block located at the bottom boundary of the picture, and vertical forced BT is used for the right boundary of the border block located at the right boundary of the picture, respectively.

[0313] The forced BT partitioning may include recursively dividing the current block by horizontal forced boundary partitioning until a sub-partition of the current block is located at the bottom boundary of the picture, and recursively dividing the sub-partitions by vertical forced boundary partitioning until the leaf node is located entirely at the right boundary of the picture. Alternatively, the forced BT partitioning may include recursively dividing the current block by vertical forced boundary partitioning until a sub-partition of the current block is located at the bottom boundary, and recursively dividing the sub-partitions by horizontal forced boundary partitioning until the leaf node is located entirely at the right boundary. MinQTSize may also be applied to control the partitioning of non-border blocks.

[0314] For example, in the case shown in Figure 17A, if MinQTSize is 32 or is limited to 32 and the size of a rectangular (non-square) block of 8 samples in height or width must match the picture boundary, forced BT partitioning is used to divide blocks located on 32x32 boundaries. The BT partition may be further divided using the same type of forced BT partitioning; for example, if forced vertical BT partitioning is applied, only further forced vertical BT partitioning is applied, and if forced horizontal BT partitioning is applied, only further forced horizontal BT partitioning is applied. The forced BT partitioning continues until the leaf node falls entirely within the picture.

[0315] Figure 17B shows an exemplary division of a bottom-boundary CTU having a size of 128x128 samples according to one embodiment of the present invention. The bottom-boundary CTU, which forms the root block or root node of the partition tree, is divided into smaller partitions, e.g., small blocks of square or rectangular size. These smaller partitions or blocks may be further divided into even smaller partitions or blocks. In Figure 17B, the CTU is a quadtree first divided into four square blocks 710, 720, 730, and 740, each having a size of 64x64 samples. Of these blocks, blocks 710 and 720 are again bottom-boundary blocks, while blocks 730 and 740 are outside the picture (located outside the picture, respectively) and are not processed.

[0316] Block 710 is further divided using quadtree division into four square blocks 750, 760, 770, and 780, each having a size of 32x32 samples. Blocks 750 and 760 are located inside the picture, while blocks 770 and 780 again form bottom boundary blocks. Because the size of block 770 is no larger than MinQTSize, e.g., 32, recursive horizontal forcing binary divisions are applied to block 770 until the leaf node is entirely within the picture or is entirely within the picture—for example, (after one horizontal binary division) leaf node block 772, a rectangular non-square block with 32x16 samples, is within the picture, or (after two horizontal binary divisions) leaf node block 774, a rectangular non-square block located at the bottom boundary of the picture and with 32x8 samples, is within the picture. The same applies to block 780.

[0317] The embodiments of the present disclosure enable the division of boundary blocks to be harmonized with the division of normal blocks that are completely inside a picture. A boundary block is a block that is neither completely inside nor completely outside a picture. In other words, a boundary block is a block that includes a portion that is inside the picture and a portion that is outside the picture. Furthermore, the embodiments of the present disclosure enable reduced notification because forced BT divisions below MinQTSize do not need to be notified.

[0318] Solutions for corner cases In corner cases, some techniques only allow forced QT division, and the MinQTSize constraint is also ignored. Embodiments of the present disclosure provide two solutions for the corner case. The corner case occurs when the currently processed block is at a corner of the picture. This is the case when the current block crosses or is adjacent to two picture boundaries (vertical and horizontal).

[0319] Solution 1: The corner cases are considered to be the bottom boundary case or the right boundary case. Figure 20 shows one embodiment of the boundary definition. Figure 20 shows the picture boundary by dot hash lines and the region of the straight line boundary case. As shown, the corner case is defined as the bottom boundary case. Thus, the solution is the same as that described for the bottom boundary case and the right boundary case above. In other words, first, horizontal division is applied (as described for the bottom boundary case) until the block or partition is completely (vertically) within the picture, and then vertical division is applied (as described for the right boundary case) until the leaf node is completely (horizontally) within the picture.

[0320] Solution 2: The definition of the boundary case remains the same: if the forced QT is constrained by MinQTSize (current block size less than or equal to MinQTSize), then the horizontal forced BT is used to match the bottom boundary, and once the bottom boundary is matched, the vertical forced BT is used to match the right boundary.

[0321] For example, in Figure 19A, which shows one embodiment of forced QTBT for blocks located at the corners of a picture, if MinQTSize is 32 or limited to 32 for the corner case forced QT split, further BT splits are used after splitting the 32x32 block until the forced splitting is completed.

[0322] 9B shows further details of an exemplary partitioning of a boundary CTU at or within a corner of a picture, according to one embodiment of the present invention, where the CTU has a size of 128x128 samples. The CTU is initially quadtree partitioned into four square blocks, each having a size of 64x64 samples. Of these blocks, only the top-left block 910 is a boundary block; the other three are located outside (completely outside) the picture and are not processed further. Block 910 is further partitioned using quadtree partitioning into four square blocks 920, 930, 940, and 950, each having a size of 32x32 samples. Block 920 is located inside the picture, while blocks 930, 940, and 950 again form boundary blocks. Because the sizes of these blocks 930, 940, and 950 are not larger than MinQTSize, which is 32, a forced binary partitioning is applied to blocks 930, 940, and 950.

[0323] Block 930 is split using recursive vertical forced binary splitting until it is located on the right boundary and the leaf node falls within the picture, e.g., block 932 is located on the right boundary of the picture (here after two vertical binary splits).

[0324] Block 940 is split using recursive horizontal forcing binary splitting until it is located on the bottom boundary and the leaf node falls within the picture, e.g., block 942 is located on the right boundary of the picture (here after two horizontal binary splits).

[0325] Block 950 is located at a corner boundary and is first split using recursive horizontal forced binary splitting until a sub-partition or block, here block 952, is located at the bottom boundary of the picture (here after two horizontal binary splits), and then the sub-partition is recursively split by vertical forced boundary splitting until a leaf node or block, for example block 954, is located at the right boundary of the picture (here after two vertical binary splits), or until the leaf node is located within the picture.

[0326] The above approach may apply to both decoding and encoding. For decoding, MinQTSize may be received via SPS. For encoding, MinQTSize may be transmitted via SPS. Embodiments may use the boundary definitions shown in Figure 18 or Figure 20, or other boundary definitions.

[0327] Further embodiments of the present disclosure are provided below. Please note that the numbering used in the following sections does not necessarily conform to the numbering used in the previous sections.

[0328] Embodiment 1: determining whether a current block of a picture is a border block; If the current block is a boundary block, determining whether the size of the current block is greater than the minimum allowed quadtree leaf node size; if the size of the current block is not greater than the minimum allowable quadtree leaf node size, applying a forced binary tree split to the current block; A division method including:

[0329] Embodiment 2: The partitioning method of embodiment 1, wherein the forced binary tree partitioning is a recursive horizontal forced binary partitioning when the current block is located at the bottom boundary of the picture, or a recursive vertical forced boundary partitioning when the current block is located at the right boundary of the picture.

[0330] Embodiment 3: The partitioning method of embodiment 1 or 2, wherein the forced binary partitioning includes recursively partitioning the current block by horizontal forced boundary partitioning until a sub-partition of the current block is aligned exactly on the bottom boundary of the picture, and recursively partitioning the sub-partition by vertical forced boundary partitioning until a leaf node is aligned exactly on the right boundary of the picture.

[0331] Embodiment 4: The splitting method of any of embodiments 1 to 3, wherein the minimum allowed quadtree leaf node size is a minimum allowed quadtree leaf node size that also applies to control splitting of non-border blocks.

[0332] Embodiment 5: A decoding method for decoding a block by dividing the block according to the division method of any one of embodiments 1 to 4.

[0333] Embodiment 6: The decoding method of embodiment 5, wherein the minimum allowable quadtree leaf node size is received via an SPS.

[0334] Embodiment 7: A coding method for coding a block by dividing the block according to the division method of any one of embodiments 1 to 4.

[0335] Embodiment 8: The encoding method of embodiment 7, wherein the minimum allowed quadtree leaf node size is transmitted via SPS.

[0336] Embodiment 9: A decoding device comprising logic circuitry configured to perform any one of the methods of embodiment 5 or 6.

[0337] Embodiment 10: An encoding device comprising logic circuitry configured to perform any one of the methods of embodiment 7 or 8.

[0338] Embodiment 11: A non-transitory storage medium for storing instructions that, when executed by a processor, cause the processor to perform any of the methods according to embodiments 1 to 8.

[0339] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which correspond to tangible media such as data storage media, or communication media, including any medium that facilitates transfer of a computer program from one place to another, for example, according to a communications protocol. As such, computer-readable media may generally correspond to (1) tangible computer-readable storage media that is non-transitory, or (2) a communication medium such as a signal or carrier wave. Data storage media may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, or data structures for implementation of the techniques described in this disclosure. A computer program product may include computer-readable media.

[0340] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, Electrically Erasable Programmable Read-Only Memory (EEPROM), CD-ROM or other optical disk storage, magnetic disk storage, other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included within the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but instead cover non-transitory tangible storage media. As used herein, disk and disc include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically and discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0341] The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term "processor," as used herein, may refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding or incorporated into a composite codec. Also, these techniques may be implemented entirely in one or more circuits or logic elements.

[0342] The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units are described in this disclosure to highlight functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require realization by different hardware units. Rather, as noted above, the various units may be combined within a codec hardware unit or provided by a collection of interoperable hardware units, including one or more processors as described above, in conjunction with appropriate software and / or firmware.

[0343] The apparatus comprises a memory element; and a processor element coupled to the memory element and configured to determine whether a current block of the picture is a border block, and when the current block is a border block, determine whether a size of the current block is greater than a minimum allowed quadtree (QT) leaf node size (MinQTSize), and apply a forced binary tree (BT) split to the current block when the size of the current block is not greater than MinQTSize.

[0344] While several embodiments are provided in this disclosure, it should be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the disclosure. The examples should be considered illustrative rather than limiting, and the intention is not to be limited to the details given herein. For example, various elements or components may be combined or integrated in another system, and some features may be omitted or not implemented.

[0345] Additionally, techniques, systems, subsystems, and methods described and illustrated in various embodiments as separate or singular may be combined or integrated with other systems, modules, techniques, or methods without departing from the scope of the present disclosure. Other items illustrated or described as coupled or directly coupled or in communication with each other may also be indirectly coupled or in communication through some interface, device, or intermediate component, whether electrical, mechanical, or otherwise. Other examples of changes, substitutions, and alterations will be ascertainable by those skilled in the art and can be made without departing from the spirit and scope disclosed herein.

[0346] SUMMARY OF THE INVENTION Embodiments of the present application (or the present disclosure) provide apparatuses and methods for encoding and decoding.

[0347] A first aspect relates to a partitioning method, comprising the steps of determining whether a current block of a picture is a border block and whether the size of the current block is greater than a minimum allowed quadtree leaf node size, and applying a forced binary tree (BT) partition to the current block if the current block is a border block and the size of the current block is not greater than the minimum allowed quadtree leaf node size (MinQTSize).

[0348] In a first implementation form of the method according to such a first aspect, the forced binary tree partitioning is a recursive horizontal forced binary partitioning in the case where the current block is located at the bottom boundary of the picture, or a recursive vertical forced boundary partitioning in the case where the current block is located at the right boundary of the picture.

[0349] In a second implementation of the method according to such first aspect or any preceding implementation of the first aspect, the forced binary tree splitting continues until the leaf node block fits within the picture.

[0350] In a third implementation of a method according to such a first aspect or any preceding implementation of the first aspect, the forced binary splitting includes recursively splitting the current block by horizontal forced boundary splitting until a sub-partition of the current block is located at the bottom boundary of the picture, and recursively splitting the sub-partition by vertical forced boundary splitting until a leaf node is located entirely at the right boundary of the picture.

[0351] In a fourth implementation of a method according to such a first aspect or any preceding implementation of the first aspect, the forced BT split includes recursively splitting the current block by vertical forced boundary splitting until a subpartition of the current block is located at the lower boundary, and recursively splitting the subpartition by horizontal forced boundary splitting until the leaf node is located entirely at the right boundary.

[0352] In a fifth implementation of the method according to such first aspect or any previous implementation of the first aspect, the method further includes applying a minimum allowable quadtree leaf node size to control splitting of non-border blocks.

[0353] In a sixth embodiment of the method according to such first aspect or any preceding embodiment of the first aspect, the border blocks are blocks that are neither entirely inside the picture nor entirely outside the picture.

[0354] A second aspect relates to a decoding method for decoding a block by dividing the block according to such first aspect or any preceding implementation of the first aspect.

[0355] In a first implementation of such a method according to the second aspect, the method further comprises receiving a minimum allowed quadtree leaf node size via a sequence parameter set (SPS).

[0356] A third aspect relates to an encoding method for encoding a block by dividing the block according to such first aspect or any preceding implementation of the first aspect.

[0357] In a first implementation of such a method according to the third aspect, the method further comprises transmitting the minimum allowed quadtree leaf node size via a sequence parameter set (SPS).

[0358] A fourth aspect relates to a decoding device comprising a logic circuit configured to decode a block by dividing the block according to the division method of such first aspect or any preceding implementation of the first aspect.

[0359] In a first implementation of such a decoding device according to the fourth aspect, the logic circuitry is further configured to receive a minimum allowed quadtree leaf node size via a sequence parameter set (SPS).

[0360] A fifth aspect relates to an encoding device comprising a logic circuit configured to encode a block by dividing the block according to the division method of such first aspect or any preceding implementation of the first aspect.

[0361] In such a first implementation of a decoding device according to the fifth aspect, the logic circuitry is further configured to transmit the minimum allowed quadtree leaf node size via a sequence parameter set (SPS).

[0362] A sixth aspect relates to a non-transitory storage medium for storing instructions that, when executed by a processor, cause the processor to perform any of such first, second, or third aspects, or any preceding implementation of the first, second, or third aspects.

[0363] A seventh aspect relates to a method that includes determining that a current block of a picture is a border block and that the size of the current block is less than or equal to a minimum allowable quad-tree (QT) leaf node size (MinQTSize), and applying a forced binary tree (BT) split to the current block in response to the determination.

[0364] In such a first implementation form of the method according to the seventh aspect, the current block is located at the bottom boundary of the picture, and the forced BT division is a recursive horizontal forced BT division.

[0365] In a second implementation form of the method according to such seventh aspect or any preceding implementation form of the seventh aspect, the current block is located at the right boundary of the picture, and the forced BT division is a recursive vertical forced BT division.

[0366] In a third implementation of a method according to such seventh aspect or any preceding implementation of the seventh aspect, the forced BT split includes recursively splitting the current block by horizontal forced boundary splitting until a subpartition of the current block is located at the lower boundary, and recursively splitting the subpartition by vertical forced boundary splitting until the leaf node is located entirely at the right boundary.

[0367] In a fourth implementation of a method according to such seventh aspect or any preceding implementation of the seventh aspect, the forced BT split includes recursively splitting the current block by vertical forced boundary splitting until a subpartition of the current block is located at the lower boundary, and recursively splitting the subpartition by horizontal forced boundary splitting until the leaf node is located entirely at the right boundary.

[0368] In a fifth implementation of the method according to such seventh aspect or any preceding implementation of the seventh aspect, the method further includes a step of applying MinQTSize to control the division of non-border blocks.

[0369] In a sixth implementation of the method according to such seventh aspect or any preceding implementation of the seventh aspect, the method further includes receiving MinQTSize via a sequence parameter set (SPS).

[0370] In a seventh implementation of the method according to such seventh aspect or any preceding implementation of the seventh aspect, the method further includes transmitting MinQTSize via a sequence parameter set (SPS).

[0371] An eighth aspect relates to an apparatus comprising: a memory; and a processor coupled to the memory and configured to determine whether a current block of a picture is a border block; if the current block is a border block, determine whether a size of the current block is greater than a minimum allowed quadtree (QT) leaf node size (MinQTSize); and apply a forced binary tree (BT) split to the current block when the size of the current block is not greater than MinQTSize.

[0372] In a first implementation form of such an apparatus according to the eighth aspect, the forced BT split is a recursive horizontal forced BT split when the current block is located at the bottom boundary of the picture, or a recursive vertical forced BT split when the current block is located at the right boundary of the picture.

[0373] In a second implementation of an apparatus according to such an eighth aspect or any preceding implementation of the eighth aspect, the forced BT splitting includes recursively splitting the current block by horizontal forced boundary splitting until a subpartition of the current block is located at the lower boundary, and recursively splitting the subpartition by vertical forced boundary splitting until the leaf node is located entirely at the right boundary.

[0374] In a third implementation of an apparatus according to such an eighth aspect or any preceding implementation of the eighth aspect, the forced BT splitting includes recursively splitting the current block by vertical forced boundary splitting until a subpartition of the current block is located at the lower boundary, and recursively splitting the subpartition by horizontal forced boundary splitting until the leaf node is located entirely at the right boundary.

[0375] In a fourth implementation of the apparatus according to such eighth aspect or any previous implementation of the eighth aspect, the processor is further configured to apply MinQTSize to control splitting of non-border blocks.

[0376] In a fifth implementation of an apparatus according to such an eighth aspect or any previous implementation of the eighth aspect, the apparatus further comprises a receiver coupled to the processor and configured to receive MinQTSize via a sequence parameter set (SPS).

[0377] In a sixth implementation of the device according to such eighth aspect or any previous implementation of the eighth aspect, the device further includes a transmitter coupled to the processor and configured to receive MinQTSize via a sequence parameter set (SPS).

[0378] A ninth aspect relates to a computer program product including computer-executable instructions stored on a non-transitory medium that, when executed by a processor, causes the device to determine whether a current block of a picture is a border block; if the current block is a border block, determine whether a size of the current block is greater than a minimum allowed quadtree (QT) leaf node size (MinQTSize); and, if the size of current block 0 is not greater than MinQTSize, apply a forced binary tree (BT) split to the current block.

[0379] In a first implementation form of such an apparatus according to the eighth aspect, the forced BT split is a recursive horizontal forced BT split when the current block is located at the bottom boundary of the picture, or a recursive vertical forced BT split when the current block is located at the right boundary of the picture.

[0380] In a second implementation of an apparatus according to such a ninth aspect or any preceding implementation of the ninth aspect, the forced BT splitting includes recursively splitting the current block by horizontal forced boundary splitting until a subpartition of the current block is located at the lower boundary, and recursively splitting the subpartition by vertical forced boundary splitting until the leaf node is located entirely at the right boundary.

[0381] In a third implementation of an apparatus according to such a ninth aspect or any preceding implementation of the ninth aspect, the forced BT splitting includes recursively splitting the current block by vertical forced boundary splitting until a subpartition of the current block is located at the lower boundary, and recursively splitting the subpartition by horizontal forced boundary splitting until the leaf node is located entirely at the right boundary.

[0382] In a fourth implementation of the device according to such ninth aspect or any previous implementation of the ninth aspect, the instructions further cause the device to apply MinQTSize to control splitting of non-border blocks.

[0383] In a fifth implementation of the device according to such ninth aspect or any previous implementation of the ninth aspect, the instructions cause the device to further receive MinQTSize via a sequence parameter set (SPS).

[0384] In a sixth implementation of the device according to such ninth aspect or any previous implementation of the ninth aspect, the instructions further cause the device to transmit MinQTSize via a sequence parameter set (SPS).

[0385] Embodiments Regarding Relationship Between Splitting Constraints for Non-Border and Border Blocks Next Generation Video Coding (NGVC) removes the separation of the concepts of CU, PU, ​​and TU and supports flexibility in CU partition shape. The size of a CU corresponds to the size of a coding node and can be square or non-square (e.g., rectangular) in shape.

[0386] In J. An et al., "Block partitioning structure for next generation video coding," International Telecommunication Union, COM16-C966, September 2015 (hereafter "VCEG Proposal COM16-C966"), a quadtree-binary tree (QTBT) partitioning technique was proposed for future video coding standards beyond HEVC. Simulations show that the proposed QTBT structure is more efficient than the quadtree structure used in HEVC. In HEVC, inter prediction for small blocks is restricted to reduce memory accesses for motion compensation, and inter prediction for 4x4 blocks is not supported. JEM's QTBT removes these restrictions.

[0387] In QTBT, CUs can have either a square or rectangular shape. As shown in Figure 21, a coding tree unit (CTU) is first partitioned using a quadtree structure. The quadtree leaf nodes can be further partitioned using a binary tree structure. There are two types of binary tree partitioning: symmetric horizontal partitioning and symmetric vertical partitioning. In both cases, the node is partitioned by splitting it horizontally or vertically in the middle. The binary tree leaf nodes are called coding units (CUs), and their segmentation is used for prediction and transformation processes without further partitioning. This means that CUs, PUs, and TUs have the same block size in the QTBT coding block structure. CUs are sometimes composed of coding blocks (CBs) of different color components. For example, in the case of P slices and B slices in 4:2:0 chroma format, one CU contains one luma CB and two chroma CBs. Sometimes, they are composed of a CB of a single component. For example, in the case of an I slice, one CU contains only one luma CB or only two chroma CBs.

[0388] The following parameters are defined for the QTBT splitting method: -CTU size: Root node size of the quadtree, same concept as HEVC -MinQTSize: The minimum allowed quadtree leaf node size -MaxBTSize: Maximum allowed binary tree root node size -MaxBTDepth: Maximum allowed binary tree depth -MinBTSize: The minimum allowable binary tree leaf node size

[0389] In one example of a QTBT partitioning structure, if a quadtree node has a size less than or equal to MinQTSize, no further quadtree splits are considered. Once the size (MinQTSize) exceeds MaxBTSize, it is not further split by the binary tree. Otherwise, the leaf quadtree node may be further split by the binary tree. Therefore, the quadtree leaf node is also the root node for the binary tree, and it has a binary tree depth of 0 (zero). Once the binary tree depth reaches MaxBTDepth (i.e., 4), no further splits are considered. When a binary tree node has a width equal to MinBTSize (i.e., 4), no further horizontal splits are considered. Similarly, when a binary tree node has a height equal to MinBTSize, no further vertical splits are considered. The leaf node of the binary tree is further processed by the prediction and transform processes without further splits. In JEM, the maximum CTU size is 256x256 luma samples. Leaf nodes of the binary tree may be further processed (eg, by performing prediction and transformation processes) without further splitting.

[0390] FIG. 21 shows an example of a block 30 (e.g., CTB) divided using the QTBT partitioning technique. As shown in FIG. 21, using the QTBT partitioning technique, each of the blocks is divided symmetrically through the center of each block. FIG. 22 shows a tree structure corresponding to the block division of FIG. 21. The solid lines in FIG. 22 indicate quadtree partitioning, and the dotted lines indicate binary tree partitioning. In one example, at each partition (i.e., non-leaf) node of the binary tree, a syntax element (e.g., a flag) is signaled to indicate the type of partitioning performed (e.g., horizontal or vertical), with 0 indicating a horizontal partition and 1 indicating a vertical partition. In the case of quadtree partitioning, there is no need to indicate the partition type because quadtree partitioning always divides a block horizontally and vertically into four sub-blocks of equal size.

[0391] As shown in Figure 22, at node 50, block 30 (corresponding to root 50) is split into four blocks 31, 32, 33, and 34 shown in Figure 21 using QT splitting. Block 34 is not split further and is therefore a leaf node. At node 52, block 31 is further split into two blocks using BT splitting. As shown in Figure 22, node 52 is marked with a 1, indicating a vertical split. Thus, the split at node 52 results in a block that includes block 37 and both blocks 35 and 36. Blocks 35 and 36 are created by a further vertical split at node 54. At node 56, block 32 is further split into two blocks 38 and 39 using BT splitting.

[0392] At node 58, block 33 is divided into four equally sized blocks using QT partitioning. Blocks 43 and 44 are created from this QT partitioning and are not further divided. At node 60, the top left block is first divided using vertical binary tree partitioning, resulting in block 40 and the vertical block on the right. The vertical block on the right is then divided into blocks 41 and 42 using horizontal binary tree partitioning. The bottom right block created from the quad tree partitioning at node 58 is divided into blocks 45 and 46 at node 62 using horizontal binary tree partitioning. As shown in Figure 22, node 62 is marked with a 0, indicating a horizontal partition.

[0393] In addition to QTBT, a block partitioning structure named Multi-Type Tree (MTT) is proposed to replace BT in the QTBT-based CU structure, i.e., a CTU may be first partitioned by QT partitioning to obtain blocks of CTU, and then the blocks may be secondly partitioned by MTT partitioning.

[0394] The MTT partitioning structure is also a recursive tree structure. In MTT, multiple different partitioning structures (e.g., two or more) are used. For example, according to the MTT technique, at each depth of the tree structure, two or more different partitioning structures may be used for each non-leaf node of the tree structure. The depth of a node in the tree structure may refer to the length of the path (e.g., the number of splits) from the node to the root of the tree structure.

[0395] In MTT, there are two partition types: BT partition and ternary tree (TT) partition. The partition type can be selected from BT partition and TT partition. The TT partition structure differs from the QT structure or the BT structure in that the TT partition structure does not split the block in half. The central region of the block remains together in the same subblock. Unlike QT, which generates four blocks, or binary tree, which generates two blocks, a partition by the TT partition structure generates three blocks. Exemplary partition types by the TT partition structure include symmetric partition types (both horizontally and vertically) and asymmetric partition types (both horizontally and vertically). Furthermore, the symmetric partition type by the TT partition structure can be unequal / uneven or equal / uniform. The asymmetric partition type by the TT partition structure is unequal / uneven. In one example, the TT partition structure may include at least one of the following partition types: horizontal equal / uniform symmetric ternary tree partition type, vertical equal / uniform symmetric ternary tree partition type, horizontal uneven / uneven symmetric ternary tree partition type, vertical uneven / uneven symmetric ternary tree partition type, horizontal uneven / uneven asymmetric ternary tree partition type, or vertical uneven / uneven asymmetric ternary tree partition type.

[0396] In general, an uneven / uneven symmetric ternary tree division type is a division type that is symmetrical about the centerline of the block, but at least one of the three resulting blocks is not the same size as the other two. One preferred example is when the side blocks are 1 / 4 the size of the block and the center block is 1 / 2 the size of the block. An even / uniform symmetric ternary tree division type is a division type that is symmetrical about the centerline of the block, and the resulting blocks are all the same size. Such a division is possible when the height or width of the block is a multiple of three, depending on the vertical or horizontal division. An uneven / uneven asymmetric ternary tree division type is a division type that is not symmetrical about the centerline of the block, and at least one of the resulting blocks is not the same size as the other two.

[0397] FIG. 23 is a conceptual diagram illustrating an optional exemplary horizontal ternary tree division type. FIG. 24 is a conceptual diagram illustrating an optional exemplary vertical ternary tree division type. In both FIG. 23 and FIG. 24, h represents the height of the block in luma or chroma samples, and w represents the width of the block in luma or chroma samples. Note that the center lines of each of the blocks do not represent the block boundaries (i.e., ternary tree division does not divide the blocks through the center lines). Rather, the center lines are used to depict whether a particular division type is symmetric or asymmetric with respect to the center line of the original block. The center lines also follow the direction of the division.

[0398] As shown in Figure 23, block 71 is divided using the horizontal equal / uniform symmetric division type. The horizontal equal / uniform symmetric division type generates an upper and lower half that are symmetrical about the centerline of block 71. The horizontal equal / uniform symmetric division type generates three sub-blocks of the same size, each with a height of h / 3 and a width of w. The horizontal equal / uniform symmetric division type is possible when the height of block 71 is divisible by 3.

[0399] The block 73 is divided using a horizontally unequal / uneven symmetrical division type. The horizontally unequal / uneven symmetrical division type generates an upper half and a lower half that are symmetrical about the centerline of the block 73. The horizontally unequal / uneven symmetrical division type generates two blocks of the same size (e.g., an upper block and a lower block having a height of h / 4) and a central block of a different size (e.g., a central block having a height of h / 2). In one example, according to the horizontally unequal / uneven symmetrical division type, the area of ​​the central block is equal to the combined area of ​​the upper and lower blocks. In some examples, the horizontally unequal / uneven symmetrical division type may be preferred for blocks having heights that are powers of two (e.g., 2, 4, 8, 16, 32, etc.).

[0400] Block 75 is divided using a horizontally unequal / uneven asymmetric division type. The horizontally unequal / uneven asymmetric division type does not produce symmetrical upper and lower halves about the centerline of block 75 (i.e., the upper and lower halves are asymmetric). In the example of FIG. 23, the horizontally unequal / uneven asymmetric division type produces an upper block having a height of h / 4, a middle block having a height of 3h / 8, and a lower block having a height of 3h / 8. Of course, other asymmetric arrangements may be used.

[0401] As shown in Figure 24, block 81 is divided using the vertical equal / uniform symmetric division type. The vertical equal / uniform symmetric division type generates left and right halves that are symmetrical about the centerline of block 81. The vertical equal / uniform symmetric division type generates three sub-blocks of the same size, each with a width of w / 3 and a height of h. The vertical equal / uniform symmetric division type is possible when the width of block 81 is divisible by 3.

[0402] Block 83 is divided using a vertical unequal / uneven symmetrical division type. The vertical unequal / uneven symmetrical division type generates left and right halves that are symmetrical about the centerline of block 83. The vertical unequal / uneven symmetrical division type generates left and right halves that are symmetrical about the centerline of block 83. The vertical unequal / uneven symmetrical division type generates two blocks of the same size (e.g., a left block and a right block having a width of w / 4) and a central block of a different size (e.g., a central block having a width of w / 2). In one example, according to the vertical unequal / uneven symmetrical division type, the area of ​​the central block is equal to the combined area of ​​the left and right blocks. In some examples, the vertical unequal / uneven symmetrical division type may be preferred for blocks having widths that are powers of two (e.g., 2, 4, 8, 16, 32, etc.).

[0403] Block 85 is divided using a vertical unequal / uneven asymmetric division type. The vertical unequal / uneven asymmetric division type does not produce left and right halves that are symmetrical about the centerline of block 85 (i.e., the left and right halves are asymmetric). In the example of Figure 24, the vertical unequal / uneven asymmetric division type produces a left block having a width of w / 4, a middle block having a width of 3w / 8, and a right block having a width of 3w / 8. Of course, other asymmetric arrangements may be used.

[0404] In addition to (or instead of) the parameters for QTBT defined above, the following parameters are defined for the MTT splitting method: -MaxBTSize: Maximum allowed binary tree root node size -MinBtSize: The minimum allowable binary tree root node size -MaxMttDepth: Maximum multitype tree depth -MaxMttDepthOffset: Maximum multitype tree depth offset -MaxTtSize: Maximum allowed ternary tree root node size -MinTtSize: The minimum allowable ternary tree root node size -MinCbSize: Minimum allowable coding block size

[0405] Embodiments of the present disclosure may be implemented by a video encoder or a video decoder, such as the video encoder 20 of Figure 12 or the video decoder 30 of Figure 13, according to embodiments of the present application. One or more structural elements of the video encoder 20 or the video decoder 30, including the splitting unit, may be configured to perform the techniques of embodiments of the present disclosure.

[0406] In one embodiment of the present disclosure: In JVET-K1001-v4, log2_ctu_size_minus2, log2_min_qt_size_intra_slices_minus2, and log2_min_qt_size_inter_slices_minus2 are signaled in the SPS (as syntax elements). The parameter log2_ctu_size_minus2 plus 2 specifies the luma coding tree block size of each CTU. CtbLog2SizeY=log2_ctu_size_minus2+2 (7-5) CtbSizeY=1< <CtbLog2SizeY (7-6) is.

[0407] In other words, CtbLog2SizeY specifies the log2 value of the CTU size CtbSizeY, which corresponds to the coding tree block (CTB) size for luma (Y). Further settings are provided below. MinCbLog2SizeY=2 (7-7) MinCbSizeY=1< <MinCbLog2SizeY (7-8) MinTbSizeY=4 (7-9) MaxTbSizeY=64 (7-10)

[0408] The parameter log2_min_qt_size_intra_slices_minus2 plus 2 specifies the minimum luma size of leaf blocks resulting from the quadtree decomposition of CTUs in slices with slice_type equal to 2(I), i.e., intra slices. The value of log2_min_qt_size_intra_slices_minus2 must be in the range 0 to CtbLog2SizeY-2, inclusive. MinQtLog2SizeIntraY=log2_min_qt_size_intra_slices_minus2+2(7-22)

[0409] The parameter log2_min_qt_size_inter_slices_minus2 plus 2 specifies the minimum luma size of leaf blocks resulting from the quadtree decomposition of CTUs in slices with slice_type equal to 0 (B) or 1 (P), i.e., inter-slices. The value of log2_min_qt_size_inter_slices_minus2 must be in the range 0 to CtbLog2SizeY-2, inclusive. MinQtLog2SizeInterY=log2_min_qt_size_inter_slices_minus2+2(7-23)

[0410] MinQtSizeY is defined as (7-30), which means the minimum allowed quadtree division size in luma samples. If the coding block size is less than or equal to MinQtSizeY, quadtree division is not allowed. Further settings are provided below. MinQtLog2SizeY=(slice_type==I)?MinQtLog2SizeIntraY:MinQtLog2SizeInterY (7-25) MaxBtLog2SizeY=CtbLog2SizeY-log2_diff_ctu_max_bt_size (7-26) MinBtLog2SizeY=MinCbLog2SizeY (7-27) MaxTtLog2SizeY=(slice_type==I)?5:6 (7-28) MinTtLog2SizeY=MinCbLog2SizeY (7-29) MinQtSizeY=1< <MinQtLog2SizeY (7-30) MaxBtSizeY=1< <MaxBtLog2SizeY (7-31) MinBtSizeY=1< <MinBtLog2SizeY (7-32) MaxTtSizeY=1< <MaxTtLog2SizeY (7-33) MinTtSizeY=1< <MinTtLog2SizeY (7-34) MaxMttDepth=(slice_type==I)?max_mtt_hierarchy_depth_intra_slices:max_mtt_hierarchy_depth_inter_slices (7-35)

[0411] The parameters max_mtt_hierarchy_depth_intra_slices and max_mtt_hierarchy_depth_inter_slices denote the maximum hierarchical depth of MTT type partitioning for intra-slices and inter-slices, respectively.

[0412] Based on the semantics of log2_min_qt_size_intra_slices_minus2 and log2_min_qt_size_inter_slices_minus2, the range of log2_min_qt_size_intra_slices_minus2 and log2_min_qt_size_inter_slices_minus2 is from 0 to CtbLog2SizeY-2.

[0413] Here, CtbLog2SizeY is defined by the semantics of log2_ctu_size_minus2, which means the log2 value of the luma coding tree block size of each CTU, and CtbLog2SizeY in VTM2.0 is equal to 7.

[0414] Based on (7-22) and (7-23), the ranges of MinQtLog2SizeIntraY and MinQtLog2SizeInterY are from 2 to CtbLog2SizeY.

[0415] Based on (7-25), the range of MinQtLog2SizeY is from 2 to CtbLog2SizeY.

[0416] Based on (7-30), in JVET-K1001-v4, the range of MinQtSizeY is from (1<<2) to (1<<CtbLog2SizeY), and in VTM2.0, the range is from (1<<2) to (1<<7), which is equal to 4 to 128.

[0417] In JVET-K1001-v4, log2_diff_ctu_max_bt_size is conditionally signaled in the slice header.

[0418] The parameter log2_diff_ctu_max_bt_size specifies the difference between the luma CTB size of the coding block that can be split using binary splitting and the maximum luma size (width or height). The value of log2_diff_ctu_max_bt_size must be in the range of 0 to CtbLog2SizeY - MinCbLog2SizeY, including both ends.

[0419] If log2_diff_ctu_max_bt_size does not exist, the value of log2_diff_ctu_max_bt_size is presumed to be equal to 2.

[0420] MinCbLog2SizeY is defined by (7 - 7), which means the minimum allowable coding block size.

[0421] Based on the semantics of log2_diff_ctu_max_bt_size, the range of log2_diff_ctu_max_bt_size is from 0 to CtbLog2SizeY - MinCbLog2SizeY.

[0422] Based on (7 - 26), the range of MaxBtLog2SizeY is from CtbLog2SizeY to MinCbLog2SizeY.

[0423] Based on (7 - 31), the range of MaxBtSizeY is from (1 << CtbLog2SizeY) to (1 << MinCbLog2SizeY).

[0424] Based on (7 - 7), in JVET - K1001 - v4, the range of MaxBtSizeY is from (1 << CtbLog2SizeY) to (1 << 2), and in VTM2.0, since CtbLog2SizeY is equal to 7, the range of MaxBtSizeY in VTM2.0 is from 128 to 4.

[0425] Therefore, MinQtSizeY has a range from 4 to (1 << CtbLog2SizeY), from 4 to 128 in VTM2.0, and MaxBtSizeY has a range from (1 << CtbLog2SizeY) to 4, from 128 to 4 in VTM2.0.

[0426] Therefore, MinQtSizeY can be larger than MaxBtSizeY.

[0427] [[ID=二十八]]Moreover, based on the current boundary processing of VVC2.0, only QT splitting and BT splitting are allowed for blocks located at the boundary (TT is not allowed, and no splitting is not allowed).

[0428] Currently, the coding block is located on the boundary, and the current coding block size cbSizeY satisfies the condition: MinQtSizeY>cbSizeY>MaxBtSizeY When this condition is satisfied, neither QT splitting nor BT splitting is possible for the current coding block. Therefore, there is no available splitting mode for the current block.

[0429] Embodiment 1 The solutions to the above problems (embodiments of the present invention), including the problem of boundary cases, are described in more detail below.

[0430] According to one embodiment, to solve the described problem, the lower limit of MaxBtSizeY should be restricted to MinQtSizeY to ensure that MaxBtSizeY is not smaller than MinQtSizeY. Specifically, since the lower limit of MaxBtSizeY may be equal to MinQtSizeY, the range of MaxBtSizeY should be from (1<<CtbLog2SizeY) to (1<<MinQtLog2SizeY), and therefore, the range of MaxBtLog2SizeY should be from CtbLog2SizeY to MinQtLog2SizeY, and therefore, the range of log2_diff_ctu_max_bt_size should be from 0 to CtbLog2SizeY - MinQtLog2SizeY.

[0431] The corresponding changes in the draft text (of the video standard) are within the semantics of log2_diff_ctu_max_bt_size as follows.

[0432] log2_diff_ctu_max_bt_size specifies the difference between the luma CTB size and the maximum luma size (width or height) of a coding block that can be split using binary splitting. The value of log2_diff_ctu_max_bt_size must be in the range 0 to CtbLog2SizeY-MinQtLog2SizeY, inclusive. Therefore, information about MinQtSizeY may be used to determine the validity of MaxBtSizeY. In other words, MaxBtSizeY may be determined based on information about MinQtSizeY.

[0433] The corresponding method of coding performed by a coding device (decoder or encoder) may be as follows: determining whether a current block of a picture is a border block; determining whether the size of the current block is greater than the minimum allowable quadtree leaf node size; applying binary splitting to the current block if the current block is a boundary block and the size of the current block is not greater than the minimum allowable quadtree leaf node size, where the minimum allowable quadtree leaf node size is not greater than the maximum allowable binary tree root node size;

[0434] Applying binary splitting to the current block may include applying forced binary splitting to the current block.

[0435] The current block may be obtained by dividing an image or a coding tree unit (CTU).

[0436] The method may include two cases: 1) treeType is equal to SINGLE_TREE or DUAL_TREE_LUMA; 2) treeType is equal to DUAL_TREE_CHROMA. In case 1), the current block is a luma block, and in case 2), the current block is a chroma block.

[0437] The maximum allowed binary tree root node size may be the maximum luma size in luma samples of the luma coding root block that can be split using binary tree splitting.

[0438] The maximum allowed ternary tree root node size may be the maximum luma size in luma samples of the luma coding root block that can be divided using ternary tree division.

[0439] The minimum allowed quadtree roof node size may be the minimum luma size in luma samples of a luma leaf block resulting from the quadtree division.

[0440] In this document, coding corresponds to image, video or moving image coding.

[0441] Being a boundary block means that the image / frame boundary cuts the block, or in other words, the block is at the image / frame boundary. In the above embodiment, if the current block is a boundary block (condition 1) and its size is not larger than the minimum allowed quadtree leaf node size (condition 2), binary splitting is applied to the current block. Note that in some embodiments, ternary splitting or other splitting may be used instead of binary splitting. Moreover, in some embodiments, binary splitting may be applied under condition 2, regardless of condition 1. In other words, condition 1 does not need to be determined. If the size of the current block is actually larger than the minimum allowed quadtree leaf node size (i.e., condition 2 is not met), quadtree splitting may be applied.

[0442] Note that there are embodiments in which binary splitting is used only for boundary blocks (Condition 1). For non-boundary blocks, quadtree splitting may be the only splitting used. Applying binary (or ternary) splitting at image / frame boundaries potentially offers the advantage of more efficient splitting, e.g., horizontal binary / ternary splitting at horizontal boundaries and vertical binary / ternary splitting at vertical boundaries.

[0443] Another corresponding coding method implemented by a coding device (decoder or encoder) may be as follows: determining whether the size of the boundary block is greater than the minimum allowed quad-tree leaf node size, if the size of the boundary block is not greater than the minimum allowed quad-tree leaf node size, and if (e.g., by standard) the minimum allowed quad-tree leaf node size is not greater than the maximum allowed binary tree root node size, then binary splitting is applied to the boundary block.

[0444] In some cases, border blocks may not include corner blocks: in other words, corner blocks cut off by both vertical and horizontal image / frame boundaries are not considered border blocks for the purposes of condition 1 above.

[0445] Embodiment 2 Other embodiments of the present disclosure (which may be combined with the above embodiments) are described below.

[0446] In JVET-K1001-v4, max_mtt_hierarchy_depth_inter_slices and max_mtt_hierarchy_depth_intra_slices are signaled within the SPS. In other words, max_mtt_hierarchy_depth_inter_slices and max_mtt_hierarchy_depth_intra_slices are syntax elements that mean their values ​​are included in the bitstream that also contains the coded image or video.

[0447] In particular, max_mtt_hierarchy_depth_inter_slices specifies the maximum hierarchical depth of coding units resulting from multi-type tree partitioning of quadtree leaves in slices with slice_type equal to 0 (B) or 1 (P). The value of max_mtt_hierarchy_depth_inter_slices must be in the range 0 to CtbLog2SizeY - MinTbLog2SizeY, inclusive.

[0448] max_mtt_hierarchy_depth_intra_slices specifies the maximum hierarchical depth of coding units resulting from multitype tree partitioning of quadtree leaves in slices with slice_type equal to 2(I). The value of max_mtt_hierarchy_depth_intra_slices must be in the range 0 to CtbLog2SizeY-MinTbLog2SizeY, inclusive.

[0449] MinTbSizeY is defined in (7-9) and is fixed to 4, so MinTbLog2SizeY=log2 MinTbSizeY is fixed to 2.

[0450] MaxMttDepth is defined, which means the maximum allowed depth of a multitype tree split. If the depth of the current multitype tree split is equal to or greater than MaxMttDepth, the multitype tree split is not allowed (applied).

[0451] Based on the semantics of max_mtt_hierarchy_depth_inter_slices and max_mtt_hierarchy_depth_intra_slices, the range of max_mtt_hierarchy_depth_inter_slices and max_mtt_hierarchy_depth_intra_slices is from 0 to CtbLog2SizeY-MinTbLog2SizeY.

[0452] Based on (7-35), the range of MaxMttDepth is from 0 to CtbLog2SizeY-MinTbLog2SizeY. In VTM2.0, CtbLog2SizeY is equal to 7, so the range of MaxMttDepth is from 0 to 5.

[0453] Therefore, the range of MaxMttDepth is 0 to CtbLog2SizeY-MinTbLog2SizeY, which is 0 to 5 in VTM2.0.

[0454] Based on the current boundary handling in VVC2.0, only QT and BT splits are allowed for blocks located on boundaries (TT is not allowed, and no splitting is not allowed).

[0455] If the first problem above is resolved (MaxBtSizeY>=MinQtSizeY), then the following additional conditions are true: cbSizeY<=MinQtSizeY MaxMttDepth=0

[0456] There is no sufficient level of BT (and generally any MTT including TT) division for boundary processing.

[0457] For example, MinQtSizeY is equal to 16, MinTbSizeY is equal to 4, and MaxMttDepth is 0.

[0458] If a boundary block has cbSizeY=16, the parent split is QT, and this block is still located on the boundary, then no further splits can be performed since the Mttdepth of the current block reaches MaxMttDepth.

[0459] Solution to this problem for boundary cases (one embodiment of the present invention): To solve the described problem, the lower bound of MaxMttDepth should be limited to 1 (in other words, it cannot take on a value of zero) to ensure that there is a sufficient level of multitype tree splitting for boundary cases after QT splitting. Alternatively, the lower bound of MaxMttDepth should be further limited to (MinQtLog2SizeY - MinTbLog2SizeY) to ensure that there is a sufficient level of multitype tree splitting for both boundary and non-boundary cases after QT splitting.

[0460] The corresponding changes in the draft text (of the standard) are in the semantics of max_mtt_hierarchy_depth_inter_slices and max_mtt_hierarchy_depth_intra_slices: max_mtt_hierarchy_depth_inter_slices specifies the maximum hierarchical depth of coding units resulting from multi-type tree splitting of quadtree leaves in slices with slice_type equal to 0 (B) or 1 (P). The value of max_mtt_hierarchy_depth_inter_slices must be in the range 1 to CtbLog2SizeY-MinTbLog2SizeY, inclusive.

[0461] max_mtt_hierarchy_depth_intra_slices specifies the maximum hierarchical depth of coding units resulting from multitype tree partitioning of quadtree leaves in slices with slice_type equal to 2(I). The value of max_mtt_hierarchy_depth_intra_slices must be in the range 1 to CtbLog2SizeY-MinTbLog2SizeY, inclusive, or or max_mtt_hierarchy_depth_inter_slices specifies the maximum hierarchical depth of coding units resulting from multi-type tree splitting of quadtree leaves in slices with slice_type equal to 0 (B) or 1 (P). The value of max_mtt_hierarchy_depth_inter_slices must be in the range MinQtLog2SizeY-MinTbLog2SizeY to CtbLog2SizeY-MinTbLog2SizeY, inclusive.

[0462] max_mtt_hierarchy_depth_intra_slices specifies the maximum hierarchical depth of coding units resulting from multitype tree splitting of quadtree leaves in slices with slice_type equal to 2(I). The value of max_mtt_hierarchy_depth_intra_slices must be in the range MinQtLog2SizeY-MinTbLog2SizeY to CtbLog2SizeY-MinTbLog2SizeY, inclusive.

[0463] The corresponding coding method implemented by a coding device (decoder or encoder) may be as follows:

[0464] Dividing an image into blocks, the blocks including boundary blocks, and applying binary partitioning to the boundary blocks having a maximum boundary multi-type partition depth, the maximum boundary multi-type partition depth being at least the sum of a maximum multi-type tree depth and a maximum multi-type tree depth offset, and the maximum multi-type tree depth being greater than 0. This embodiment may be combined with embodiment 1 or may be applied without embodiment 1.

[0465] In some cases, the maximum multitype tree depth is greater than 0 when applying binary splitting to boundary blocks.

[0466] In some cases, the border blocks may not include the corner blocks.

[0467] Embodiment 3 In another embodiment of the present disclosure: For JVET-K1001-v4, if MinQtSizeY>MaxBtSizeY and MinQtSizeY>MaxTtSizeY.

[0468] If cbSize=MinQtsizeY, the division cannot reach MinCbSizeY because there are no possible division modes available (MinTbSizeY and MinCbsizeY are fixed and equal to 4).

[0469] Solutions to this problem for non-boundary or boundary cases: To solve the described problem, the lower bound of MaxBtSizeY should be constrained to MinQtSizeY to ensure that MaxBtSizeY is not smaller than MinQtSizeY, or the lower bound of MaxTtSizeY should be constrained to MinQtSizeY to ensure that MaxTtSizeY is not smaller than MinQtSizeY.

[0470] The corresponding changes in the draft text are: log2_diff_ctu_max_bt_size specifies the difference between the luma CTB size and the maximum luma size (width or height) of a coding block that can be split using binary splitting Within semantics: The value of log2_diff_ctu_max_bt_size must be in the range 0 to CtbLog2SizeY-MinQtLog2SizeY, inclusive.

[0471] and / or log2_min_qt_size_intra_slices_minus2 plus 2 specifies the minimum luma size of leaf blocks resulting from quadtree decomposition of CTUs in slices with slice_type equal to 2(I). The value of log2_min_qt_size_intra_slices_minus2 must be in the range 0 to MaxTtLog2SizeY-2, inclusive.

[0472] log2_min_qt_size_inter_slices_minus2 plus 2 specifies the minimum luma size of leaf blocks resulting from the quadtree decomposition of CTUs in slices with slice_type equal to 0 (B) or 1 (P). The value of log2_min_qt_size_inter_slices_minus2 must be in the range 0 to MaxTtLog2SizeY-2, inclusive.

[0473] The corresponding coding method implemented by a coding device (decoder or encoder) may be as follows: determining whether the size of the current block is greater than the minimum allowable quadtree leaf node size; and applying multitype tree partitioning to the current block if the size of the current block is not greater than the minimum allowable quadtree leaf node size, the minimum allowable quadtree leaf node size is not larger than the maximum allowable binary tree root node size, or the minimum allowable quadtree leaf node size is not larger than the maximum allowable ternary tree root node size, Step.

[0474] In some cases, the minimum allowable quadtree leaf node size is no larger than the maximum allowable binary tree root node size, and the minimum allowable quadtree leaf node size is no larger than the maximum allowable ternary tree root node size.

[0475] In some cases, applying a multi-type tree partition to the current block comprises applying a ternary partition to the current block, or applying a binary partition to the current block.

[0476] In some cases, the border blocks may not include the corner blocks.

[0477] Embodiment 4 In another embodiment of the present disclosure: If MaxBtSizeY>=MinQtSizeY, MinQtSizeY>MinTbLog2SizeY, and MaxMttDepth<(MinQtLog2SizeY-MinTbLog2SizeY), If cbSize=MinQtsizeY, then no splitting can reach MinCbSizeY because there are not enough levels of multitype tree splitting allowed.

[0478] Solution to this problem for non-boundary or boundary cases: To solve the described problem, the lower bound of MaxMttDepth should be limited to (MinQtLog2SizeY - MinTbLog2SizeY) to ensure that after QT splitting there is a sufficient level of multitype tree splitting for both boundary and non-boundary cases.

[0479] The corresponding changes in the draft text are in the semantics of max_mtt_hierarchy_depth_inter_slices and max_mtt_hierarchy_depth_intra_slices: max_mtt_hierarchy_depth_inter_slices specifies the maximum hierarchical depth of coding units resulting from multi-type tree splitting of quadtree leaves in slices with slice_type equal to 0 (B) or 1 (P). The value of max_mtt_hierarchy_depth_inter_slices must be in the range MinQtLog2SizeY-MinTbLog2SizeY to CtbLog2SizeY-MinTbLog2SizeY, inclusive.

[0480] max_mtt_hierarchy_depth_intra_slices specifies the maximum hierarchical depth of coding units resulting from multitype tree splitting of quadtree leaves in slices with slice_type equal to 2(I). The value of max_mtt_hierarchy_depth_intra_slices must be in the range MinQtLog2SizeY-MinTbLog2SizeY to CtbLog2SizeY-MinTbLog2SizeY, inclusive.

[0481] The corresponding coding method implemented by a coding device (decoder or encoder) may be as follows: Dividing the image into blocks. applying multitype tree partitioning to blocks of blocks having a final maximum multitype tree depth, wherein the final maximum multitype tree depth is at least the sum of the maximum multitype tree depth and the maximum multitype tree depth offset, and wherein the maximum multitype tree depth is greater than or equal to the Log2 value of the minimum allowed quadtree leaf node size minus the Log2 value of the minimum allowed transform block size, or the maximum multitype tree depth is greater than or equal to the Log2 value of the minimum allowed quadtree leaf node size minus the Log2 value of the minimum allowed coding block size.

[0482] In some cases, the block is a non-border block.

[0483] In some cases, the maximum multitype tree depth offset is 0.

[0484] In some cases, the blocks are boundary blocks and the multi-type tree partition is a binary partition.

[0485] In some cases, the multitype tree partition is (or includes) a ternary partition.

[0486] In some cases, the border blocks may not include the corner blocks.

[0487] Embodiments 1-4 can be applied at the encoder side to divide images / frames into coding units and to code the coding units. Embodiments 1-4 can be applied at the decoder side to provide image / frame partitions, i.e., coding units, and to decode the coding units accordingly (e.g., correctly parse coding units from a stream and decode them).

[0488] According to some embodiments, there is provided a decoder comprising one or more processors and a non-transitory computer-readable storage medium coupled to the processors and storing programming for execution by the processors, the programming, when executed by the processors, configuring the decoder to perform any of the methods described above with reference to embodiments 1 to 4.

[0489] Moreover, there is provided an encoder comprising one or more processors and a non-transitory computer-readable storage medium coupled to the processors and storing programming for execution by the processors, the programming, when executed by the processors, configuring the encoder to perform any of the methods described above with reference to embodiments 1 to 4.

[0490] In summary, there is provided a method for coding performed by a decoding device, the method comprising the steps of determining whether a size of a current block is greater than a minimum allowed quadtree leaf node size, and if the size of the current block is not greater than the minimum allowed quadtree leaf node size, applying a multi-type tree split to the current block, wherein the minimum allowed quadtree leaf node size is not greater than the maximum allowed binary tree root node size, or the minimum allowed quadtree leaf node size is not greater than the maximum allowed ternary tree root node size.

[0491] This approach facilitates efficient partitioning of image / video blocks and signaling of partitioning parameters.

[0492] Moreover, in some implementations, the method also includes determining whether a current block of the picture is a border block. If the current block is a border block and the size of the current block is not larger than the minimum allowed quad-tree leaf node size, the method also includes applying binary partitioning to the current block. Note that in this case, the minimum allowed quad-tree leaf node size is not larger than the maximum allowed binary tree root node size. For example, if the size of the current block is not larger than the minimum allowed quad-tree leaf node size, the above step of applying multi-type tree partitioning to the current block includes applying binary partitioning to the current block if the current block is a border block and the size of the current block is not larger than the minimum allowed quad-tree leaf node size.

[0493] Providing binary partitioning may be particularly advantageous for blocks at image / video frame boundaries, e.g., for blocks cut off by boundaries. Thus, in some implementations, it may be beneficial to apply measures to boundary blocks and not to the remaining blocks. However, the present disclosure is not limited thereto, and as described above, techniques for applying binary partitioning to greater partition depths may also be applied to and efficiently signaled for non-border blocks.

[0494] Additionally or alternatively to the above embodiments, the minimum allowable quadtree leaf node size is no larger than the maximum allowable binary tree root node size, and the minimum allowable quadtree leaf node size is no larger than the maximum allowable ternary tree root node size.

[0495] Applying multi-type tree partitioning to the current block may include applying ternary partitioning to the current block or applying binary partitioning to the current block, although the disclosure is not limited thereby and in general, multi-type tree partitioning may also include further or other different types of partitioning.

[0496] The method may further include determining a maximum allowable binary tree root node size based on a minimum allowable quad-tree leaf node size, which facilitates efficient signaling / storage of the parameter. For example, the maximum allowable binary tree root node size may be considered to be equal to the minimum allowable quad-tree leaf node size. However, the present disclosure is not limited thereby, and other relationships may be envisioned for deriving the maximum allowable binary tree root node size.

[0497] According to an example embodiment, in addition to or in place of the above embodiments, a method may include dividing an image into blocks, the blocks including a current block. Applying binary partitioning to the current block includes applying binary partitioning to a boundary block having a maximum boundary multitype partition depth, the maximum boundary multitype partition depth being at least the sum of the maximum multitype tree depth and the maximum multitype tree depth offset, and the maximum multitype tree depth being greater than 0. Moreover, in some embodiments, the maximum multitype tree depth is greater than 0 when applying binary partitioning to the boundary block.

[0498] According to one embodiment, a method includes dividing an image into blocks (including a current block). Applying multitype tree partitioning to the current block includes applying multitype tree partitioning to the current block among blocks having a final maximum multitype tree depth, the final maximum multitype tree depth being at least the sum of the maximum multitype tree depth and a maximum multitype tree depth offset, the maximum multitype tree depth being equal to or greater than the Log2 value of the minimum allowed quadtree leaf node size minus the Log2 value of the minimum allowed transform block size, or the maximum multitype tree depth being equal to or greater than the Log2 value of the minimum allowed quadtree leaf node size minus the Log2 value of the minimum allowed coding block size. This facilitates further partitioning even when the partitioning depth is large.

[0499] The current block may be a non-border block. The maximum multitype tree depth offset may be 0. The current block may alternatively or additionally be a border block, and the multitype tree partition is a binary partition. The multitype tree partition may be or include a ternary partition.

[0500] According to one embodiment, there is provided an encoding method comprising the steps of determining whether a size of a current block is greater than a minimum allowed quadtree leaf node size, and if the size of the current block is not greater than the minimum allowed quadtree leaf node size, applying a multi-type tree split to the current block, wherein the minimum allowed quadtree leaf node size is not greater than the maximum allowed binary tree root node size, or the minimum allowed quadtree leaf node size is not greater than the maximum allowed ternary tree root node size.

[0501] The encoding method can apply any of the above rules and constraints described for the decoding method, because the encoder and decoder sides need to share a bitstream. In particular, the encoding side generates a bitstream after encoding the partitions resulting from the above-mentioned division, and the decoding side parses the bitstream and restores the decoded partitions accordingly. The same applies to the embodiments related to the encoding device (encoder) and the decoding device (decoder) described below.

[0502] According to one embodiment, there is provided a decoding device comprising: a circuit configured to determine whether a size of a current block is larger than a minimum allowed quad-tree leaf node size; and if the size of the current block is not larger than the minimum allowed quad-tree leaf node size, apply multi-type tree splitting to the current block, wherein the minimum allowed quad-tree leaf node size is not larger than the maximum allowed binary tree root node size, or the minimum allowed quad-tree leaf node size is not larger than the maximum allowed ternary tree root node size. It should be noted that determining whether the size of the current block is larger than the minimum allowed quad-tree leaf node size may be performed at the decoding side based on a notification in the bitstream.

[0503] Also provided is an encoding device comprising circuitry configured to determine whether a size of a current block is greater than a minimum allowable quadtree leaf node size, and if the size of the current block is not greater than the minimum allowable quadtree leaf node size, apply multi-type tree splitting to the current block, wherein the minimum allowable quadtree leaf node size is not greater than the maximum allowable binary tree root node size, or the minimum allowable quadtree leaf node size is not greater than the maximum allowable ternary tree root node size.

[0504] According to one embodiment, a non-transitory computer-readable storage medium is provided that stores programming for execution by a processing circuit, the programming, when executed by the processing circuit, configures the processing circuit to perform any of the methods described above.

[0505] The devices described in this disclosure may include processing circuitry for performing the various operations and methods described herein. The processing circuitry may include hardware and software. For example, the processing circuitry may include one or more processors and non-volatile memory coupled to the one or more processors. The memory may carry program code that, when executed by the one or more processors, causes the device to perform the operations or methods.

[0506] The present invention has been described with various embodiments as examples and implementations. However, other variations can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the independent claims. In the claims and the description, the word "comprises" does not exclude other elements or steps, and the indefinite articles "a" or "an" do not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in a claim. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation.

[0507] The following is a description of applications of the encoding and decoding methods shown in the above embodiments and a system using them.

[0508] 27 is a block diagram showing a content supply system 3100 for realizing a content distribution service. The content supply system 3100 includes a capture device 3102, a terminal device 3106, and, in some cases, a display 3126. The capture device 3102 communicates with the terminal device 3106 via a communication link 3104. The communication link may include the communication channel 13 described above. The communication link 3104 may include, but is not limited to, WIFI, Ethernet, cable, wireless (3G / 4G / 5G), USB, or any combination thereof.

[0509] The capture device 3102 can generate data and encode it using the encoding method described in the above embodiment. Alternatively, the capture device 3102 can deliver the data to a streaming server (not shown), which then encodes the data and transmits the encoded data to the terminal device 3106. The capture device 3102 may include, but is not limited to, a camera, a smartphone or smartpad, a computer or laptop, a video conferencing system, a PDA, an in-vehicle device, or any combination thereof. For example, the capture device 3102 may include the source device 12 described above. When the data includes video, the video encoder 20 included in the capture device 3102 can actually perform the video encoding process. When the data includes audio (i.e., voice), the audio encoder included in the capture device 3102 can actually perform the audio encoding process. In some practical scenarios, the capture device 3102 delivers the encoded video data and audio data by multiplexing them together. In other practical scenarios, for example, in a video conferencing system, the encoded audio data and the encoded video data are not multiplexed. The capture device 3102 delivers the encoded audio data and the encoded video data separately to the terminal device 3106 .

[0510] In the content delivery system 3100, a terminal device 3106 receives and plays encoded data. The terminal device 3106 may be a device capable of receiving and restoring data, such as a smartphone or smart pad 3108, a computer or laptop 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a television 3114, a set-top box (STB) 3116, a video conferencing system 3118, a video surveillance system 3120, a personal digital assistant (PDA) 3122, an in-vehicle device 3124, or any combination thereof, capable of decoding the encoded data. For example, the terminal device 3106 may include the destination device 14 described above. When the encoded data includes video, a video decoder 30 included in the terminal device is prioritized to perform video decoding. When the encoded data includes audio, an audio decoder included in the terminal device is prioritized to perform audio decoding.

[0511] In the case of a terminal device having its display, for example, a smartphone or smart pad 3108, a computer or laptop 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a television 3114, a personal digital assistant (PDA) 3122, or an in-vehicle device 3124, the terminal device may provide the decoded data to its display. In the case of a terminal device without a display, for example, an STB 3116, a video conferencing system 3118, or a video surveillance system 3120, an external display 3126 is contacted therein to receive and display the decoded data.

[0512] When each device in this system performs encoding or decoding, it can use a picture encoding device or a picture decoding device as shown in the above embodiments.

[0513] 28 is a diagram illustrating the structure of an example of a terminal device 3106. After the terminal device 3106 receives a stream from the capture device 3102, a protocol progression unit 3202 analyzes the transmission protocol of the stream. The protocol may include, but is not limited to, Real Time Streaming Protocol (RTSP), HyperText Transfer Protocol (HTTP), HTTP Live Streaming Protocol (HLS), MPEG-DASH, Real Time Transport Protocol (RTP), Real Time Messaging Protocol (RTMP), or any combination thereof.

[0514] After the protocol progression unit 3202 processes the stream, a stream file is generated. The file is output to the demultiplexing unit 3204. The demultiplexing unit 3204 can separate the multiplexed data into encoded audio data and encoded video data. As mentioned above, in some practical scenarios, for example, in a video conferencing system, the encoded audio data and encoded video data are not multiplexed. In this situation, the encoded data is sent to the video decoder 3206 and the audio decoder 3208 without passing through the demultiplexing unit 3204.

[0515] Through the demultiplexing process, a video elementary stream (ES), an audio ES, and in some cases subtitles are generated. The video decoder 3206, which includes the video decoder 30 described in the above embodiment, decodes the video ES using the decoding method shown in the above embodiment to generate video frames, and supplies this data to the synchronization unit 3212. The audio decoder 3208 decodes the audio ES to generate audio frames, and supplies this data to the synchronization unit 3212. Alternatively, the video frames can be stored in a buffer (not shown in FIG. 28) before being supplied to the synchronization unit 3212. Similarly, the audio frames can be stored in a buffer (not shown in FIG. 28) before being supplied to the synchronization unit 3212.

[0516] The synchronization unit 3212 synchronizes the video and audio frames and provides the video / audio to the video / audio display 3214. For example, the synchronization unit 3212 synchronizes the presentation of video and audio information. The information can be coded in the syntax using timestamps for the presentation of the coded audio and visual data, as well as timestamps for the delivery of the data stream itself.

[0517] If subtitles are included in the stream, the subtitle decoder 3210 decodes the subtitles, synchronizes them with the video and audio frames, and provides the video / audio / subtitles to the video / audio / subtitle display 3216.

[0518] The present invention is not limited to the above system, and either the picture encoding device or the picture decoding device in the above embodiments can be incorporated into other systems, for example, in-vehicle systems.

[0519] Although embodiments of the present invention are described primarily in terms of video coding, it should be noted that embodiments of coding system 10, encoder 20, and decoder 30 (and corresponding system 10), as well as other embodiments described herein, may also be configured for still image processing or coding, i.e., processing or coding of individual pictures independent of preceding or successive images, as in video coding. Generally, when image processing coding is limited to a single picture 27, only inter-prediction units 244 (encoder) and 344 (decoder) may not be available. All other functions (also called tools or techniques) of video encoder 20 and video decoder 30 may equally be used for still picture processing, e.g., residual calculation 204 / 304, transform 206, quantization 208, inverse quantization 210 / 310, (inverse) transform 212 / 312, segmentation 262 / 362, intra-prediction 254 / 354, and / or loop filtering 220, 320, as well as entropy coding 270 and entropy decoding 304.

[0520] For example, embodiments of the encoder 20 and the decoder 30, and functions described herein with reference to the encoder 20 and the decoder 30, may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on a computer-readable medium or transmitted over a communication medium as one or more instructions or code and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which correspond to tangible media such as data storage media, or communication media, including any medium that facilitates transfer of a computer program from one place to another, for example, according to a communication protocol. Thus, computer-readable media can generally correspond to (1) tangible computer-readable storage media that are non-transitory, or (2) a communication medium such as a signal or carrier wave. Data storage media may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementation of the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0521] The following logical or mathematical operators are defined as follows: The mathematical operators used in this application are similar to those used in the C programming language. However, the results of integer division and arithmetic shift operations are more precisely defined, and additional operations such as exponentiation and real division are defined. Numbering and counting conventions generally start from 0, e.g., "first" corresponds to 0, "second" corresponds to 1, etc.

[0522] Arithmetic operators The following arithmetic operators are defined as follows:

[0523] [Table 1]

[0524] Logical operators The following logical operators are defined as follows: x&&y Boolean logic "and" between x and y x||y Boolean logic "or" of x and y ! Boolean logic "not" x?y:zIf x is true or not equal to 0, it is evaluated as the value of y, otherwise it is evaluated as the value of z.

[0525] Relational operators The following relational operators are defined as follows: > Greater than >= greater than or equal to < Less than <= Less than or equal to == Equal != Not equal to

[0526] When a relational operator is applied to a syntax element or variable that has been assigned the value "na" (not applicable), the value "na" is treated as a separate value for the syntax element or variable. The value "na" is not considered equal to any other value.

[0527] Bitwise Operators The following bitwise operators are defined as follows: & Bitwise "and". When operating on an integer argument, it operates on the two's complement representation of the integer value. When operating on a binary argument that contains fewer bits than another argument, it extends the shorter argument by adding the more significant bits equal to 0. Bitwise "or". When operating on integer arguments, it operates on the two's complement representation of the integer value. When operating on a binary argument that contains fewer bits than another argument, it extends the shorter argument by adding the more significant bits equal to 0. When acting on an integer argument of "exclusive OR" on a bit-by-bit basis, it acts on the two's complement representation of the integer value. When acting on a binary argument that contains fewer bits than another argument, the short argument is extended by adding higher-order bits equal to 0. x >> y Arithmetic right shift of only y bits of the two's complement integer representation of x This function is defined only for non-negative integer values of y. The bit shifted into the most significant bit (MSB) as a result of the right shift has a value equal to the MSB of x before the shift operation. x << y Arithmetic left shift of only y bits of the two's complement integer representation of x This function is defined only for non-negative integer values of y. The bit shifted into the least significant bit (LSB) as a result of the left shift has a value equal to 0.

[0528] Assignment operator The following arithmetic operators are defined as follows. = Assignment operator ++ Increment, i.e., x++ is equivalent to x = x + 1, and when used as an array index, the value of the variable is determined before the increment operation. -- Decrement, i.e., x-- is equivalent to x = x - 1, and when used as an array index, the value of the variable is determined before the decrement operation. += Increment by the specified amount, i.e., x += 3 is equivalent to x = x + 3, and x += (-3) is equivalent to x = x + (-3). -= Decrement by the specified amount, i.e., x -= 3 is equivalent to x = x - 3, and x -= (-3) is equivalent to x = x - (-3).

[0529] Range notation The following notations are used to specify a range of values. x = y..z x takes integer values from y to z, inclusive, where x, y, and z are integer values and z is greater than y.

[0530] Mathematical functions The following mathematical functions are defined. Abs(x) =

number

number

number

number

number

number

number

number

[0531] Operation precedence When the precedence of an expression is not explicitly stated using parentheses, the following rules apply: -Operations with higher priority are considered before operations with lower priority. -Operations of equal precedence are considered in order from left to right.

[0532] The table below specifies the priority of operations from highest to lowest, with higher positions in the table indicating higher priority.

[0533] For operators that are also used in the C programming language, the precedence used herein is the same as that used in the C programming language.

[0534] [Table 2]

[0535] Text description of logical operations In the text, statements of logical operations written mathematically in the following form: if(condition0) Statement 0 else if(condition 1) Statement 1 … else / *Informative comments about the remaining conditions* / Statement n may be written as: …as follows / …the following applies: -If condition 0, statement 0 -Otherwise,If condition1,statement1 -… -Otherwise (helpful comments about the remaining conditions), statement n

[0536] Each "If…Otherwise,if…Otherwise,…" statement in the text is immediately followed by "…as" or "…the following applies" and then "if…". The last condition of "if…Otherwise,if…Otherwise,…" is always "Otherwise,…". Interleaved "If…Otherwise,if…Otherwise,…" statements can be indicated by matching "…as" or "…the following applies", ending with "Otherwise,…".

[0537] In the text, statements of logical operations written mathematically in the following form: if(condition0a&&condition0b) Statement 0 else if(condition 1a||condition 1b) Statement 1 … else Statement n may be written as: …as follows / …the following applies: -Statement 0 if all of the following conditions are true: -Condition 0a -condition 0b - Otherwise, if one or more of the following conditions are all true, then statement 1: -Condition 1a -Condition 1b -… - Otherwise, statement n

[0538] In the text, statements of logical operations written mathematically in the following form: if(condition0) Statement 0 if(condition1) Statement 1 may be written as: If condition 0, then statement 0 If condition 1, then statement 1

[0539] Acronym Definitions and Glossary HEVC - High Efficiency Video Coding VVC - Versatile Video Coding VTM-VVC test model JEM - Joint Exploration Model CTU - Coding Tree Unit CU - Coding Unit BT-binary tree TT-ternary tree QT - Quad Tree or Quaternary Tree ABT - Asymmetric BT MTT - Multitype Tree AMP - Asymmetric Partition SH-slice header SPS - Sequence Parameter Set PPS - Picture Parameter Set CE-Core Experiment SubCE - Sub-core experiment (part of the core experiment) [Explanation of symbols]

[0540] 10. Video Coding System 12 Source Devices 13 Encoded Data 14 Destination Device 16 Picture Source 17 Picture Data 18 Pre-treatment Unit 19 Preprocessed Picture Data 20 Encoder 21 Encoded Picture Data 22 Communication Interface 28 Communication Interface 30 Decoder 30 blocks 31 Decoded Pictures 31 blocks 32 Aftertreatment Unit 32 blocks 33 Post-processed pictures 33 blocks 34 Display Devices 34 blocks 35 blocks 36 blocks 37 blocks 38 blocks 39 blocks 40 Video Coding System 40 blocks 41 Imaging Device 41 blocks 42 Antenna 42 blocks 43 processors 43 blocks 44 Memory Store 44 blocks 45 Display Devices 45 blocks 46 Processing Unit 46 blocks 47 Logic Circuits 50 nodes 52 nodes 54 nodes 56 nodes 58 nodes 60 nodes 62 nodes 100 devices 101 Bitstream 102 MaxBtSizeY 103 MaxMttDepth 104 MinCbSizeY 105 MinQtSizeY 200 SPS RBSP Syntax 201 Syntax Elements 202 Syntax Elements 202 Input 203 Picture Block 204 Residual Calculation Unit 205 Residual Blocks 206 Conversion Processing Unit 207 Conversion Factor 208 quantization units 209 Quantized Transform Coefficients 210 Inverse Quantization Unit 211 Inverse quantization coefficients 212 Inverse Transformation Processing Unit 213 Inverse Transform Block 214 Restoration Unit 215 Restoration Block 216 buffers 220 Loop Filter Unit 221 Filtered Blocks 230 Decoded Picture Buffer (DPB) 231 Decoded Picture 244 Inter Prediction Units 245 Inter Prediction Blocks 254 intra prediction units 255 intra-prediction blocks 260 Prediction Processing Unit 262 Mode Selection Unit 265 predicted blocks 266 Syntax Elements 270 Entropy Coding Unit 271 Encoded Picture Data 272 output 300 Slice Header Syntax 301 Syntax Elements 304 Entropy Decoding Unit 309 Quantization Coefficients 310 Inverse Quantization Unit 311 Inverse quantization coefficients 312 Inverse Transformation Processing Unit 313 Inverse Transform Block 314 Restoration Unit 315 Restoration Block 316 buffers 317 Reference Sample 320 Loop Filter 321 Filtered Blocks 330 Decoded Picture Buffer 331 Decoded Picture 332 output 344 Inter Prediction Unit 354 intra prediction units 360 Prediction Processing Unit 362 Mode Selection Unit 365 predicted blocks 366 Syntax Elements 400 SPS RBSP Syntax 400 Video Coding Device 401 Syntax Elements 402 Syntax Elements 403 Syntax Elements 404 Syntax Elements 405 Picture Size Element 406 Picture Size Element 410 Inlet Port 420 Receiver Unit (Rx) 430 processor 440 Transmitter Unit (Tx) 450 outlet port 460 memory 470 Coding Module 500 devices 502 processor 504 memory 506 Code and Data 508 Operating Systems 510 Application Program 512 Bus 514 Secondary Storage 518 Display 520 Image sensing device 522 Sonic detection device 600 Slice Header Syntax 601 Syntax Elements 710 Square Block 720 square blocks 730 square blocks 740 square blocks 750 square blocks 760 square blocks 770 square blocks 772 leaf node blocks 774 leaf node blocks 780 square blocks 910 Block 920 square block 930 square block 932 blocks 940 square block 942 blocks 950 square blocks 952 blocks 954 blocks 1000 devices 3100 Contents Supply System 3102 Capture Device 3104 Communication Links 3106 Terminal Device 3108 Smartphone or Smartpad 3110 Computer or Laptop 3112 Network Video Recorder (NVR) / Digital Video Recorder (DVR) 3114 Television 3116 Set-top box (STB) 3118 Video Conference System 3120 Video Surveillance System 3122 Personal Digital Assistant (PDA) 3124 In-Vehicle Devices 3126 Display 3202 Protocol Progression Unit 3204 Demultiplexing Unit 3206 Video Decoder 3208 Audio Decoder 3210 Subtitle Decoder 3212 Synchronous Unit 3214 Video / Audio Display 3216 Video / Audio / Subtitle Display 5000 Slice Header Syntax 5001 Syntax Elements 7000 ways 7001 steps 7002 steps 7003 steps

Claims

1. A device for decoding or processing a video bitstream (101), said device comprising: Obtaining syntax elements from the video bitstream (101); Obtain information about the minimum size in luma samples of a luma leaf block resulting from the quadtree decomposition, MinQtSizeY(105); determining a maximum size, MaxBtSizeY (102), in luma samples of the luma root block that can be divided using binary tree division based on the information about MinQtSizeY (105) and the obtained syntax element, wherein MaxBtSizeY is equal to MinQtSizeY left-shifted according to the value of the syntax element; determining whether binary partitioning is allowed to be applied to the picture block based on the MaxBtSizeY; obtaining a coding block for the picture block based on the result of the determination; 2. A device comprising a circuit configured to:

2. The device of claim 1 , wherein the value of the syntax element is a positive integer value or a value of 0.

3. The device described in claim 1, wherein the circuit is configured to obtain the syntax element from the video bitstream (101) if the maximum hierarchical depth (MaxMttDepth) for a coding unit resulting from multi-type tree splitting is not equal to 0.

4. A device (1000) for generating a video bitstream (101), said device (1000) comprising: determining the minimum size in luma samples of a luma leaf block resulting from the quadtree decomposition, MinQtSizeY(105); determining a maximum size in luma samples of a luma root block, MaxBtSizeY(102), that can be divided using binary tree division; including information about the determined MinQtSizeY (105) in the video bitstream (101); configured to include syntax elements in the video bitstream (101); The device, wherein the MaxBtSizeY is equal to the MinQtSizeY shifted left according to the value of the syntax element.

5. The device (1000) of claim 4, wherein the value of the syntax element can only have a positive integer value or a value of 0.

6. The device (1000) of claim 5, configured to include the syntax element in the video bitstream (101) if the MaxMttDepth (103) is not equal to 0.

7. A method for generating a video bitstream (101), said method comprising: determining a minimum size in luma samples of a leaf block resulting from the quadtree decomposition, MinQtSizeY(105); determining a maximum size in luma samples of a luma root block that can be divided using binary tree division, MaxBtSizeY(102); and including syntax elements in the video bitstream; The method, wherein the MaxBtSizeY is equal to the MinQtSizeY left-shifted according to the value of the syntax element.

8. A method for decoding or processing a video bitstream (101), said method comprising: obtaining syntax elements from the video bitstream (101); obtaining information about the minimum size in luma samples of a luma leaf block resulting from the quadtree decomposition, MinQtSizeY(105); determining a maximum size in luma samples of a luma root block that can be divided using binary tree partitioning, MaxBtSizeY (102), based on the information about MinQtSizeY (105) and the obtained syntax element, wherein MaxBtSizeY is equal to MinQtSizeY left-shifted according to the value of the syntax element; determining whether binary partitioning is allowed to be applied to the picture block based on the MaxBtSizeY; and obtaining a coding block for the picture block based on a result of the determination.

9. A computer program product that, when executed by one or more processors of a device, performs the method of claim 7.

10. A computer program product that, when executed by one or more processors of a device, performs the method of claim 8.

Images