Multi-type tree depth extension for picture boundary processing

Abstract

To provide an apparatus and a method for dividing an image into coding units.SOLUTION: An image is divided into hierarchically partitioned coding tree units (CTUs). Hierarchical partitioning includes multi-type partitioning, such as binary tree or quad tree partitioning. For CTUs that are entirely within the image and CTUs on the boundary, respective multi-type partition depths are selected. The present disclosure provides the flexibility of multi-type partitioning in the boundary portion of the image.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] TECHNICAL FIELD This disclosure relates to the field of video processing, and in particular to the topic commonly referred to as hybrid video coding and compression. [Background technology]

[0002] The Versatile Video Coding (VVC) next-generation standard is the latest collaborative video project between the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG) standards organizations, which collaborate in a partnership known as the Joint Video Exploration Team (JVET).

[0003] Current block-based hybrid video codecs use predictive coding. Pictures of a video sequence are subdivided into blocks of pixels, and these blocks are then coded. Instead of coding the block pixel by pixel, the entire block is predicted using already encoded pixels that are spatially or temporally neighboring to the block. The encoder further processes only the difference between the block and its prediction. Further processing typically involves transforming the block pixels into coefficients in the transform domain. The coefficients may then be further compressed (e.g., by quantization) and further compacted (e.g., by entropy coding) to form a bitstream. The bitstream may further include any signaling information that enables a decoder to decode the encoded video. For example, the signaling may include settings regarding encoder settings such as the input picture size, frame rate, quantization step instructions, or prediction applied to blocks of the picture.

[0004] The difference between a block and its prediction is known as the residual of the block. More specifically, each pixel of the block has a residual, which is the difference between that pixel's intensity level and its predicted intensity level. A pixel's intensity level is called its pixel value or pixel's value. The residuals of all pixels of a block are collectively called the residual of the block. In other words, a block has a residual, which is a set or matrix of residuals of all pixels of the block. The residual is then transformed, quantized, and coded along with signaling information. Coding can include various forms of fixed-length and variable-length coding, including arithmetic coding or other entropy coding types.

[0005] In block-based hybrid video coding, each picture is partitioned into blocks of samples, and multiple blocks within a picture are aggregated to form slices as independently decodable entities. A block to which prediction and / or transformation is applied is called a coding unit (CU) or coding block (CB). Coding units may have different sizes.

[0006] For example, in High Efficiency Video Coding (HEVC, also known as H.265), a video frame is subdivided into coding tree units (CTUs, also called coding tree blocks, or CTBs). CTBs are disjoint square blocks of the same size, e.g., 64x64 samples. Each CTB serves as the root of a block partitioning quadtree structure, the coding tree. CTBs may be further subdivided into coding blocks along the coding tree structure. For the coding blocks, a prediction type is determined. The coding blocks may be further divided into smaller transform blocks, to which transform and quantization are applied.

[0007] More details regarding partitioning in HEVC can be found in V. Sze et al. (Ed.), High Efficiency Video Coding (HEVC): Algorithms and Architectures, Springer, 2014, Chapter 3.2.

[0008] Additionally, WO 2016 / 090568 discloses a binary tree structure for partitioning a unit into multiple smaller units using a quadtree and binary tree structure, whereby the root unit is first partitioned by the quadtree structure, and then the leaf nodes of the quadtree are further partitioned by the binary tree structure. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] International Publication No. 2016 / 090568 [Non-patent literature]

[0010] [Non-Patent Document 1] V. Sze et al (Ed.), High Efficiency Video Coding (HEVC): Algorithms and Architectures, Springer, 2014, Chapter 3.2 Summary of the Invention [Means for solving the problem]

[0011] Embodiments of the invention are defined by the features of the independent claims, further preferred developments of the embodiments are defined by the features of the dependent claims.

[0012] According to a general aspect, this disclosure provides an apparatus for partitioning an image into coding units, the apparatus including a processing circuit configured to: subdivide an image into coding tree units (CTUs), the coding tree units (CTUs) including non-border CTUs having predetermined horizontal and vertical sizes and boundary CTUs having portions within the image separated by a horizontal or vertical image boundary, the portions having a size in a direction perpendicular to the image boundary that is smaller than the predetermined size; hierarchically partition the non-border CTUs and boundary CTUs into respective coding units, the hierarchical partitioning of the non-border CTUs including multi-type partitioning with a non-border multi-type partitioning maximum depth, where the partitioning direction is either vertical or horizontal; and the hierarchical partitioning of the boundary CTUs including multi-type partitioning with a boundary multi-type partitioning maximum depth.

[0013] This offers the advantage of increased flexibility in boundary partitioning.

[0014] In a further embodiment of the device, the maximum boundary multi-type partition depth is the sum of at least an adaptive boundary multi-type partition depth and a predefined multi-type partition depth, and the adaptive boundary multi-type partition depth is a depth of a multi-type partition whose partition direction is in the direction of the image boundary.

[0015] This allows for adaptive determination of partition depth when using multi-type partitioning for boundary coding tree units or partition blocks.

[0016] For example, the predefined multitype partition depth is equal to the unbounded multitype partition maximum depth.

[0017] This allows for reuse of the unbounded multitype partition maximum depth.

[0018] In a further embodiment of the device, the sum further includes a function of the ratio of the sizes of the boundary partition blocks of the boundary CTU in the direction of the image boundary and in the direction perpendicular to the image boundary, and the boundary partition blocks are blocks of adaptive boundary multi-type partition depth.

[0019] This further increases the maximum depth of multi-type boundary partitioning, thereby providing greater partitioning flexibility.

[0020] For example, the function is the binary logarithm.

[0021] This is beneficial as it allows for a practical implementation.

[0022] In some further embodiments, a boundary multi-type partition maximum depth is predefined.

[0023] This facilitates reducing the computational cost in determining the hierarchical partitioning.

[0024] For example, the hierarchical partitioning of the boundary CTU further includes quadtree partitioning.

[0025] This allows for flexible selection from a variety of modes.

[0026] In a further embodiment of the apparatus, the bounded multi-type partition maximum depth is greater than or equal to the non-bounded multi-type partition maximum depth.

[0027] This allows for an increased maximum depth of boundary partitioning possible.

[0028] There is further provided an apparatus for encoding images of a video sequence, comprising an apparatus for dividing images into coding units according to any of the above examples and embodiments, the apparatus further comprising: an image coding unit configured to encode the coding units; and a bitstream formation unit configured to generate a bitstream including the encoded coding units and partitioning information indicating how the coding tree units have been partitioned.

[0029] In a further embodiment, the apparatus for encoding an image includes an apparatus for segmenting the image, wherein a boundary multi-type partitioning maximum depth is predefined, and the bitstream further includes an encoded sequence parameter set including the boundary multi-type partitioning maximum depth.

[0030] A method for decoding images of a video sequence, comprising: a bitstream parser for parsing a bitstream including encoded coding units; an apparatus for determining a division of an image according to any of the above examples and embodiments; and an image decoding unit for decoding the encoded coding units based on the determined division of the image. Equipment for is further provided.

[0031] In a further embodiment, the apparatus for decoding an image includes an apparatus for determining a partition of the image, wherein a boundary multi-type partitioning maximum depth is predefined, the bitstream further includes an encoded sequence parameter set including the boundary multi-type partitioning maximum depth, and the apparatus for determining the partition of the image is further configured to obtain a second multi-type partitioning maximum depth from the sequence parameter set.

[0032] According to another general aspect, a method for partitioning an image into coding units is provided, the method including the steps of: subdividing an image into coding tree units (CTUs), the coding tree units (CTUs) including non-border CTUs having predetermined horizontal and vertical sizes; and boundary CTUs having portions within the image separated by a horizontal or vertical image boundary, the portions having a size in a direction perpendicular to the image boundary that is smaller than the predetermined size; and hierarchically partitioning the non-border CTUs and boundary CTUs into respective coding units, the hierarchical partitioning of the non-border CTUs including multi-type partitions with a non-border multi-type partitioning maximum depth, where the partitioning direction is either vertical or horizontal, and the hierarchical partitioning of the boundary CTUs including multi-type partitions with a boundary multi-type partitioning maximum depth.

[0033] In a further embodiment of the method, the boundary multi-type partition maximum depth is the sum of at least an adaptive boundary multi-type partition depth and a predefined multi-type partition depth, where the adaptive boundary multi-type partition depth is the depth of a multi-type partition whose partition direction is in the direction of the image boundary.

[0034] For example, the predefined multitype partition depth is equal to the unbounded multitype partition maximum depth.

[0035] In a further embodiment of the method, the sum further includes a function of the ratio of the sizes of the boundary partition blocks of the boundary CTU in the direction of the image boundary and in the direction perpendicular to the image boundary, and the boundary partition blocks are blocks of adaptive boundary multi-type partition depth.

[0036] For example, the function is the binary logarithm.

[0037] In another embodiment, a boundary multi-type partition maximum depth is predefined.

[0038] In a further embodiment, the hierarchical partitioning of the boundary CTU further comprises quadtree partitioning.

[0039] For example, the bounded multitype partition maximum depth is greater than or equal to the non-bounded multitype partition maximum depth.

[0040] There is further provided a method for encoding images of a video sequence, the method comprising: dividing the images into coding units according to any of the above embodiments; an image coding step of encoding the coding units; and a bitstream formation step of generating a bitstream including the encoded coding units and partitioning information indicating how the coding tree units have been partitioned.

[0041] In a further embodiment, the method for encoding an image includes a method for partitioning the image, wherein a boundary multi-type partitioning maximum depth is predefined, and the bitstream further includes an encoded sequence parameter set including the boundary multi-type partitioning maximum depth.

[0042] There is further provided a method for decoding images of a video sequence, the method comprising the steps of analyzing a bitstream including encoded coding units, determining a partition of the images according to any of the above embodiments, and an image decoding step of decoding the encoded coding units based on the determined partition of the images.

[0043] In a further embodiment, a method for decoding an image includes a method for determining a partition of the image, wherein a boundary multi-type partitioning maximum depth is predefined, and the bitstream further includes an encoded sequence parameter set including the boundary multi-type partitioning maximum depth, and the method for determining a partition of the image further includes a step of obtaining a second multi-type partitioning maximum depth from the sequence parameter set.

[0044] In a further aspect, the present disclosure provides a computer-readable medium having stored thereon instructions that, when executed by a processing circuit, cause the processing circuit to perform a method for dividing an image into coding units, a method for encoding an image of a video sequence, or a method for decoding an image of a video sequence according to any of the above embodiments.

[0045] The details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will become apparent from the description, drawings, and claims.

[0046] The following embodiments of the present invention will be explained in more detail with reference to the accompanying figures and drawings. [Brief explanation of the drawings]

[0047] [Figure 1] 1 is a block diagram illustrating an exemplary structure of a video encoder configured to implement embodiments of the present invention; [Figure 2] 1 is a block diagram illustrating an exemplary structure of a video decoder configured to implement embodiments of the present invention. [Figure 3] FIG. 1 is a schematic diagram illustrating an example of quadtree partitioning used by HEVC. [Figure 4] FIG. 2 illustrates several modes of partitioning a coding unit. [Figure 5] FIG. 1 is a schematic diagram illustrating an example of quadtree / binary tree partitioning. [Figure 6] FIG. 10 is a schematic diagram showing forced quadtree division of a boundary portion. [Figure 7] FIG. 10 is a schematic diagram showing binary tree division of a boundary portion. [Figure 8] 1 is a block diagram of an apparatus for dividing an image into coding units; [Figure 9] FIG. 10 is a schematic diagram showing division of a boundary portion. [Figure 10] FIG. 1 illustrates an example of boundary partitioning using binary tree partitioning. [Figure 11]FIG. 10 illustrates a further example of boundary partitioning using binary tree partitioning. [Figure 12] FIG. 10 illustrates a comparison of boundary segmentation using quadtree and binary tree partitioning. [Figure 13] 10 is a flowchart illustrating an implementation of multitype tree maximum depth expansion. [Figure 14] 1 is a block diagram illustrating an example of a video coding system configured to implement embodiments of the present invention. [Figure 15] 1 is a schematic diagram of a video coding device. DETAILED DESCRIPTION OF THE INVENTION

[0048] The present invention relates to dividing (i.e., partitioning) an image into smaller units for further processing. Such division may be suitably used in coding and decoding still or video images. In the following, exemplary video coders and decoders that may implement division according to the present disclosure are described.

[0049] FIG. 1 illustrates an encoder 100 having an input for receiving an input block of a frame or picture of a video stream and an output for providing an encoded video bitstream. The term "frame" in this disclosure is used as a synonym for picture. However, it should be noted that this disclosure is also applicable to fields where interlacing is applied. Generally, a picture includes m×n pixels. These correspond to image samples, each of which may include one or more color components. For simplicity, the following description refers to pixels as samples of luminance. However, it should be noted that the segmentation approach of this disclosure can be applied to any color component, including chrominance, or components of a color space such as RGB. On the other hand, it may be beneficial to perform the segmentation on only one component and then apply the determined segmentation to more (or all) of the remaining components.

[0050] The encoder 100 is configured to apply partitioning, prediction, transformation, quantization, and entropy coding to the video stream.

[0051] In the division unit 110, the input video frame is further divided before coding. The blocks to be coded do not necessarily have the same size. A picture may contain blocks of different sizes, and the block rasters of different pictures in a video sequence may also be different. In particular, each video image (picture) is first subdivided into CTUs of the same fixed size. The CTU size may be fixed and predefined, for example, in a standard. In HEVC, a size of 64x64 is used. However, the present disclosure is not limited to standard fixed sizes. It may be preferable to provide a CTU size that can be set in the encoder and provided as a signaling parameter in the bitstream. For example, different CTU sizes may be beneficial for different picture sizes and / or content types. The CTU size may be signaled at any signaling level; for example, it may be common to the entire video sequence or parts thereof (i.e., multiple pictures) or for each individual picture. Correspondingly, it may be signaled, for example, in the picture parameter set PPS or sequence parameter set SPS or video parameter set VPS or similar parameter sets known from current codecs (H.264 / AVC, H.265 / HEVC). Alternatively, it may be specified in the slice header or at any other level. The CTU size may take values ​​different from 64x64. It may have, for example, a sample size of 128x128. In general, to perform a hierarchical partitioning by binary or quadtree, a power of 2 is used, i.e., 2 where n is an integer greater than 2. n It may be beneficial to provide CTU sizes that are in the form

[0052] The partitioning of a picture into CTUs and the partitioning of CTUs into CUs is shown in Figure 3 from V. Sze et al. (Eds.), High Efficiency Video Coding (HEVC): Algorithms and Architectures, Springer, 2014. The partitioning follows a quadtree structure to accommodate various local characteristics. On the left, Figure 3 shows a CTU hierarchically partitioned according to the quadtree structure on the right. In particular, the coding tree defines a syntax that specifies the subdivision of a CTU into CUs. Like a CTU, a CU consists of square blocks of samples and the syntax associated with these sample blocks. Thus, the partitioning is performed hierarchically, starting from a CTU (hierarchical depth 0) that can, but does not have to, be subdivided into four CUs (in the quadtree) of hierarchical depth 1. In Figure 3, the CTU is partitioned into CUs 8 and 16 at level 1, which are not further divided and therefore form quadtree leaves, and two further CUs that are further divided into CUs at level 2. In particular, the depth 1 CU at the top left is further subdivided into depth 2 CUs 1, 2, and 7, which form quadtree leaves, and another CU that is further divided into depth 3 CUs 3, 4, 5, and 6, which are all leaves. Similarly, the depth 1 CU at the bottom left is further divided into depth 2 CUs 13, 14, and 15, which are also quadtree leaves, and the remaining CUs that are further divided into level 3 CUs 9, 10, 11, and 12, which are all leaves and therefore not further divided.

[0053] An example syntax for quadtree partitioning in HEVC is shown below in Table 1.

[0054] [Table 1]

[0055] In particular, at the CTU level, a flag named split_cu_flag is included in the bitstream, which indicates whether the complete CTU forms a CU or whether it is split into four equally sized blocks corresponding to square sample blocks. If a CTU is split, another split_cu_flag is sent for each resulting block, specifying whether the block represents a CU or whether it is further split into four equally sized blocks. This hierarchical subdivision continues until the resulting block cannot be further subdivided. The minimum size of a CU is signaled in the sequence parameter set, which can range from 8x8 luma samples to the size of the CTU. When the minimum CU size is reached in the hierarchical subdivision process, no split flag is sent for the corresponding block; instead, it is inferred that these blocks will not be further split. In typical HEVC encoder settings, the maximum range of supported CU sizes is utilized, so that CUs ranging from 8x8 to 64x64 samples can be used. CUs within a CTU are coded in depth-first order. This coding order is also called z-scanning. It ensures that for each CU except those located on the top or left boundary of the slice, all samples above the CU and to the left of the CU have already been coded, so that the corresponding samples can be used for intra prediction and the associated coding parameters can be used to predict the coding parameters of the current CU.

[0056] In other words, split_cu_flag[x0][y0] specifies whether the coding unit is split into coding units with half the size in the horizontal and vertical directions. The array index x0,y0 specifies the position (x0,y0) of the top-left luma sample of the coding block in question relative to the top-left luma sample of the picture. If split_cu_flag[x0][y0] is not present, the following applies to the decoder: If log2CbSize (a parameter specifying the coding block size) is greater than MinCbLog2SizeY (a parameter specifying the minimum configurable coding unit size), the value of split_cu_flag[x0][y0] is inferred to be equal to 1. Otherwise (log2CbSize equals MinCbLog2SizeY), the value of split_cu_flag[x0][y0] is inferred to be equal to 0.

[0057] The array CtDepth[x][y] specifies the coding tree depth of the luma coding block covering position (x,y). If split_cu_flag[x0][y0] is equal to 0, CtDepth[x][y] is inferred to be equal to cqtDepth for x=x0..x0+nCbS-1 and y=y0..y0+nCbS-1.

[0058] In Versatile Video Coding (VVC), segmentation structures, including quadtree (QT) segmentation and multi-type (binary / tripartite / asymmetric binary) tree (BT / TT / ABT) segmentation structures, replace the concept of multiple partitioning unit types. In other words, the new segmentation structures eliminate the separation of the concepts of CU (coding unit), PU (prediction unit), and TU (transform unit), except as needed for CUs with sizes too large for the maximum transform length, and support greater flexibility regarding CU partition shapes [JVET-J1002]. Figure 4 shows the partitioning modes currently used in the VVC Test Model (VTM). Part (a) of Figure 4 shows a CTU or CU to which no further partitioning is applied (no partitioning). Part (b) shows a CTU or CU to which no further partitioning is applied (no partitioning). CUParts (c) and (d) show vertical and horizontal binary tree splitting modes, respectively. Furthermore, parts (e) and (f) show vertical and horizontal ternary tree splitting modes. It can be seen that in the ternary tree splitting, there are two blocks of size 1 / 4 and one block of size 1 / 2.

[0059] The following parameters are relevant for the hierarchical partitioning of CTUs: CTU size: The root node size of the quaternary tree MinQTSize: The minimum allowed quaternary tree leaf node size. MaxBTTSize: The maximum allowed binary and ternary tree root node size. MaxBTTDepth: Maximum allowed binary and ternary tree depth MinBTTSize: The minimum allowed binary and ternary tree leaf node size MinCUSize: The minimum allowed CU size is defined and specified by the syntax elements of the Sequence Parameter Set (SPS) of the BT / TT / QT coding tree scheme.

[0060] Figure 5 shows a mixed quadtree and binary tree partitioning. Quadtree partitioning is shown with solid lines, while binary tree partitioning is shown with dashed lines. The labels 1 or 0 on the nodes representing coding units that are further divided by the binary tree indicate whether the binary partitioning is done vertically or horizontally, respectively.

[0061] The horizontal and vertical size of a video picture in luma samples must be an integer multiple of the smallest CU size in luma samples signaled in the sequence parameter set, but it does not need to be an integer multiple of the CTU size. If the horizontal or vertical size of a video picture does not represent an integer multiple of the CTU size, boundary CTUs are inferred to be split until the resulting block boundaries coincide with the picture boundaries. For this forced split, no split flag is signaled, but the resulting block may be further split using the quadtree syntax described above. CUs that are outside the picture area are not coded.

[0062] This partitioning is shown in Figure 6, which is an illustration of a CTU (128x128) constrained QT partition of the lower boundary of an HD sequence (1920x1080). In particular, Figure 6 shows the frame boundary, above which 56 lines (128 samples long) are the boundary portion of the slice or picture. The portion of the CTU below the frame boundary may belong to another slice, or may not exist at all, for example, if the frame boundary is the lower boundary of the picture. As can be seen, a constrained quadtree partitioning is applied to 128x56 samples.

[0063] The subdivision of the chroma CTB is always aligned with the subdivision of the respective luma CTB in HEVC. Note that the present disclosure may process the chroma components in the same way, but is not limited thereto. There may also be independent division of different color components.

[0064] Returning to FIG. 1, after image segmentation is performed by segmentation unit 110, transformation, quantization, and entropy coding are performed by transform unit 130, quantization unit 140, and entropy encoding unit 150, respectively, to generate the output as an encoded video bitstream.

[0065] A video stream may include multiple frames. For example, a block of a first frame of the video stream is intra-coded by intra prediction unit 190. An intra-frame is coded using information from only that frame, and thus, it can be decoded independently from other frames. Thus, an intra-frame may provide an entry point into the bitstream, e.g., for random access. Blocks of other frames of the video stream may be inter-coded by inter prediction unit 195, where each block of an inter-coded frame is predicted from a block in another frame (a reference frame), e.g., a previously coded frame. Mode selection unit 180 is configured to select whether a block of a frame is intra-predicted or inter-predicted, i.e., whether it is processed by intra prediction unit 190 or inter prediction unit 195. Mode selection unit 180 also Intra or Inter Prediction Nopa Inter-coded frames control the frame rate and frame rate parameters. To allow for refreshing of image information, inter-coded frames may contain not only inter-coded blocks but also one or more intra-coded blocks. In contrast, intra-frames contain only intra-coded blocks, but no inter-coded blocks. Intra-frames may be inserted into a video sequence (e.g., periodically, i.e., after every certain number of inter-frames) to provide an entry point for decoding, i.e., a point at which a decoder can begin decoding without using information from previous frames.

[0066] The intra prediction unit 190 is a block prediction unit. To perform spatial or temporal prediction, the coded block may be further processed by the inverse quantization unit 145 and the inverse transform unit 135. After the reconstruction of the block by the reconstructor 125, the loop filtering unit 160 may be applied to further improve the quality of the decoded image. The reconstructor 125 adds the decoded residual to the predictor to obtain a reconstructed block. The filtered block then forms a reference frame, which is then stored in the frame buffer 170. Such a decoding loop (decoder) on the encoder side provides the advantage of generating the same reference frame as the reference picture reconstructed on the decoder side. Therefore, the encoder side and the decoder side operate in a corresponding manner. The term "reconstruction" here refers to obtaining a reconstructed block by adding the decoded residual block to the predictive block.

[0067] The inter prediction unit 195 receives as input a block of the current frame or picture to be inter-coded and one or several reference frames or pictures from the frame buffer 170. Motion estimation and motion compensation are performed by the inter prediction unit 195. Motion estimation is used to obtain a motion vector and a reference frame, for example, based on a cost function. Motion compensation then describes the current block of the current frame in terms of the translation of a reference block of the reference frame to the current frame, i.e., by a motion vector. The inter prediction unit 195 selects a predictive block (i.e., a predictor) for the current block from a set of candidate blocks (i.e., candidate predictors) in one or several reference frames such that the predictive block minimizes the cost function. In other words, the candidate block with the smallest cost function is used as the predictive block for the current block.

[0068] For example, the cost function may be a measure of the difference between the current block and the candidate block, i.e., a measure of the residual error of the current block relative to the candidate block. For example, the cost function may be the sum of absolute differences (SAD) between all pixels (samples) of the current block and all pixels of the candidate block in the candidate reference picture. However, in general, similarity metrics such as mean square error (MSE) or structural similarity metric (SSIM) may be used.

[0069] However, the cost function may also be the number of bits required to code such inter-blocks and / or the distortion resulting from such coding. Thus, a rate-distortion optimization procedure may be used to determine the encoding parameters, such as the selection of motion vectors and / or generally whether to use inter- or intra-prediction for a block and its settings.

[0070] The intra prediction unit 190 receives as input a block of the current frame or picture to be intra-coded and one or several reference samples from already reconstructed regions of the current frame. Intra prediction then describes the pixels of the current block of the current frame in terms of a function of the reference samples of the current frame. The intra prediction unit 190 outputs a prediction block of the current block, which preferably minimizes the difference between the current block to be coded and its prediction block, i.e., it minimizes the residual block. The minimization of the residual block may be based, for example, on a rate-distortion optimization procedure. In particular, the prediction block is obtained as a directional interpolation of the reference samples. The direction may be determined by rate-distortion optimization and / or by calculating a similarity measure, as mentioned above in connection with inter prediction.

[0071] Next, the difference between the current block and its prediction, i.e., residual block, is transformed by transform unit 130. The transform coefficients are quantized by quantization unit 140 and entropy coded by entropy encoding unit 150. The encoded video bitstream thus generated includes intra- and inter-coded blocks and corresponding signaling (such as mode indication, motion vector indication, and / or intra-prediction direction). Transform unit 130 may apply a linear transform, such as a discrete Fourier transform (DFT) or a discrete cosine transform (DCT). Such a transformation to the spatial frequency domain offers the advantage that the resulting coefficients typically have higher values ​​at lower frequencies. Thus, after effective coefficient scanning (such as zigzag) and quantization, the resulting sequence of values ​​typically has a few large values ​​at the beginning and a run of zeros at the end. This enables more efficient coding. Quantization unit 140 performs lossy compression by reducing the resolution of the coefficient values. Entropy coding unit 150 then assigns binary codewords to the coefficient values. The codewords are written into a bitstream called the encoded bitstream. The entropy coder also codes signaling information (not shown in FIG. 1), which may include coding according to the split flag syntax shown above.

[0072] 2 shows an example of a video decoder 200. The video decoder 200 includes, among other things, a reference picture buffer 270 and an intra-prediction unit 290, which is a block prediction unit. The reference picture buffer 270 ,workman The video decoder 200 is configured to store at least one reference frame reconstructed from the encoded video bitstream. The intra prediction unit 290 is configured to generate a prediction block that is an estimate of a block to be decoded. The intra prediction unit 290 is configured to generate this prediction based on reference samples obtained from the reference picture buffer 270.

[0073] Decoder 200 is configured to decode the encoded video bitstream generated by video encoder 100, and preferably both decoder 200 and encoder 100 generate identical predictions for each block being encoded / decoded. The features of reference picture buffer 270 and intra prediction unit 290 are similar to the features of reference picture buffer 170 and intra prediction unit 190 of FIG. 1.

[0074] The video decoder 200 includes further units that are also present in the video encoder 100, such as an inverse quantization unit 240, an inverse transform unit 230, and a loop filtering unit 260, which correspond to the inverse quantization unit 140, the inverse transform unit 150, and the loop filtering unit 160 of the video encoder 100, respectively.

[0075] The bitstream parsing, entropy decoding, and partitioning unit 250 is configured to parse and decode the received encoded video bitstream to obtain quantized residual transform coefficients and signaling information. The quantized residual transform coefficients are provided to the inverse quantization unit 240 and the inverse transform unit 230 to generate residual blocks. The residual blocks are added to the prediction blocks in the reconstructor 225, and the resulting sums are provided to the loop filtering unit 260 to obtain decoded video blocks. Frames of decoded video are stored in the reference picture buffer 270 and may serve as reference frames for inter-prediction. The signaling information parsed and decoded from the bitstream may generally include control information related to frame partitioning. To further correctly parse and decode the image, the control information is used to restore the division of the image into coding units and correctly assign subsequent decoded data to the respective coding units.

[0076] In general, the intra prediction units 190 and 290 in Figures 1 and 2 may use reference samples from already encoded regions to generate a prediction signal for a block that needs to be encoded or decoded.

[0077] The bitstream parsing, entropy decoding, and splitting unit 250 receives an encoded bitstream as its input. The bitstream may first be parsed, i.e., signaling parameters and residuals are extracted from the bitstream. The syntax and semantics of the bitstream may be defined in a standard so that encoders and decoders can operate in an interoperable manner.

[0078] In the HEVC standard, a coding tree unit (CTU) or coding unit (CU) located on a slice / picture boundary is split using forced quadtree partitioning (QT) until the bottom-right sample of the leaf node is located within the slice / picture boundary. Forced QT partitioning does not need to be signaled in the bitstream. The purpose of forced partitioning is to enable encoding / decoding of boundary CTUs / CUs by the encoder / decoder. In other words, it is agreed upon between the encoding and decoding sides that QT partitioning will be used without requiring further signaling of the partitioning mode.

[0079] In both the patent [WO2016090568] on the QTBT structure and VTM-1.0, the boundary CTU / CU forced segmentation process is inherited from HEVC. This is because the CTU / CU located on the frame boundary, especially the C TCTUs / CUs through which boundaries pass such that part of the CTU / CU is outside the picture / frame (in this disclosure, such CTUs / CUs are also referred to as "boundary CTUs" and "boundary CUs," respectively) are first forced to be partitioned by a quadtree (QT) structure without rate-distortion (RD) optimization until the entire current CU is located inside the slice / picture boundary. These forced partitions do not need to be signaled in the bitstream. Further partitioning is optionally achieved based on RD optimization. Figure 6 shows an example of one forced partition of a lower-boundary CTU (128x128) of an HD (1920x1080 pixel) sequence with forced QT.

[0080] Since only the QT partition structure is used in boundary partitioning, the limit for multi-type trees (BT / TT / ABT) in VTM is given by MaxBTTDepth in SPS. Therefore, in the case of partitioning a boundary CTU (i.e., a CTU with parts on both sides of the boundary, as shown in Figure 6), the limit for the total depth (TotalDepthLim), i.e., the maximum depth, is given by the following equation (1): TotalDepthLim=QTDepth+MaxBttDepth(1)

[0081] In equation (1), MaxBttDepth is the maximum allowed binary and ternary tree depth specified in the SPS. QTDepth is the hierarchical depth of the partitioned block resulting from quadtree partitioning. That is, at each partitioning step of the hierarchical partitioning where quadtree division is applied, the value of QTDepth is increased by 1. For example, QTDepth may correspond to the parameter cqtDepth in the syntax example of Table 1. However, although the QTDepth limit is incremented at each QT partitioning step, the hierarchical depth is ultimately limited by a predetermined allowed minimum CU size (e.g., corresponding to the SPS parameter minCUSize).

[0082] Picture boundaries can be processed using BT, TT, or ABT (Asymmetric Binary Tree). Both constrained and adaptive methods can be used. If MTT (Multi-Type Tree) such as BT / TT is used for boundary segmentation instead of QT segmentation, the BTT (binary and ternary tree) limit MaxBTTDepth from the SPS can easily be exceeded. Figure 7 shows an example of BT used in a CTU at the bottom boundary of an HD sequence. If BT starts to be used from the CTU level down to the leaf node (coding unit) that is completely inside the boundary, a depth of 4 is used for BT segmentation, which exceeds MaxBTTDepth from the SPS (in VTM-1.0, VVC Test Model Version 1.0, MaxBTTDepth is set to 3).

[0083] The objective of this disclosure is to improve the flexibility of boundary portion partitioning and provide a perspective for picture boundary processing. The approach of this disclosure is to define a boundary partition depth limit in addition to the conventional MaxBTTdepth from the SPS used for partitioning. For this purpose, a depth value is set specifically for boundary partitioning (which may be called, for example, ExtdMaxBTTDepth) rather than MaxBttDepth from Equation (1). ExtdMaxBTTDepth may be predefined and fixed in the SPS and signaled in the SPS bitstream, or it may be calculated. Therefore, when multi-type tree (MTT) partitioning (such as BT, TT, or ABT) is used in picture boundary processing (mandatory or adaptive), the maximum MTT depth limit may be extended in both the decoder and the encoder.

[0084] In the following aspects and embodiments of the present disclosure, possible ways of obtaining and calculating ExtdMaxBTTDepth are described.

[0085] According to a general aspect, an apparatus 800 for partitioning an image into coding units (which may be implemented in the encoder or decoder units 110, 250 described above) is provided, as shown in FIG. 8 . The apparatus includes a processing circuit configured to subdivide 810 the image into coding tree units (CTUs). The CTUs include non-boundary CTUs having predetermined sizes in the horizontal and vertical directions and boundary CTUs having a portion within the image separated by a horizontal or vertical image boundary, the portion having a size smaller than the predetermined size in the direction perpendicular to the image boundary. The circuit is further configured to partition the non-boundary CTUs and boundary CTUs into respective coding units. Among these, the hierarchical partitioning of the non-boundary CTUs includes multi-type partitioning with a non-boundary multi-type partitioning maximum depth, where the multi-type partitioning is partitioning in either the vertical or horizontal direction. Furthermore, the hierarchical partitioning 820 of the boundary CTUs includes multi-type partitioning with a boundary multi-type partitioning maximum depth.

[0086] Therefore, the circuitry of the apparatus 800 for dividing an image into coding units is configured to partition non-boundary CTUs into coding units using a first partitioning depth limit of an MTT partitioning step included in the partitioning, i.e., a non-boundary multi-type partitioning maximum depth. The circuitry is further configured to partition 820 boundary CTUs into coding units using a second partitioning depth limit of an MTT partitioning step included in the partitioning, i.e., a boundary multi-type partitioning maximum depth. In other words, according to the present disclosure, the circuitry of the apparatus 800 uses respective MTT partitioning maximum depths (MTT depth limits) for non-boundary CTUs and boundary CTUs during operation.

[0087] The CTUs obtained by frame segmentation 810 may be further hierarchically segmented, including hierarchical segmentation of boundary CTUs 820. This segmentation may be performed, for example, in the manner shown in and described above with reference to Figures 3 to 5.

[0088] 8 shows the internal structure of the circuit of the device 800. This circuit can be any kind of hardware and software having functional units 810 and 820 for the respective division (subdivision) of an image (or frame) into CTUs, segmentation of CTUs including non-boundary CTUs, and in particular boundary CTUs. These units can be implemented, for example, on a single processor. However, the invention is not limited to such application, and these units can also be implemented by separate hardware components.

[0089] In this disclosure, the term "border CTU" is used to refer to a coding tree unit that is divided by an image boundary into a portion of an image that is partitioned (within the image boundary) and a portion that is not located inside the image (i.e., that is located beyond the image boundary). A border CTU exists when the size of the image to be coded is not an (integer) multiple of the CTU size in at least one direction.

[0090] FIG. 9 visualizes an example of a bottom picture boundary 900 and a corresponding boundary portion 910 (shaded) that includes a boundary CTU. Portion 950 indicates the remainder of the image, which has a size that is an integer multiple of the CTU in the vertical and horizontal directions. Furthermore, the vertical size of the CTU is indicated as 970V, and the horizontal size of the CTU is indicated as 970H. As can be seen in FIG. 9, the boundary portion in this example is an integer multiple of the CTU size 970H in the horizontal direction. However, in the vertical direction, boundary portion 910 has the size that is the remainder after dividing the vertical picture size by the vertical CTU size 970V. Portion 920 is only imaginary and indicates the difference between the height of the boundary portion and the CTU size. Note that in this embodiment, the CTU is square, and therefore, sizes 970H and 970V are the same. However, the present disclosure is not limited thereto, and the vertical and horizontal sizes of the CTU may differ.

[0091] To encode (and correspondingly decode) the boundary portion, boundary portion 910 in FIG. 9 is divided into incomplete CTUs, i.e., CTU portions having a horizontal size of CTU 970H and a vertical size smaller than CTU size 970V. These incomplete CTUs correspond to "boundary CTUs" in this application, and "incomplete CTUs" correspond to portions of CTUs within an image. While horizontal boundary portions are shown in FIG. 9, vertical boundary portions may exist in addition to or instead of the horizontal boundary portions. In particular, if the image size (i.e., the image width and height) is not a multiple of the CTU size in both the vertical and horizontal directions, there may also be incomplete CTUs, which are horizontally and vertically boundary CTUs, and the vertical and horizontal sizes of the CTU portions within the image are smaller than the size of the complete CTUs.

[0092] In contrast to a "boundary CTU," the term "non-boundary CTU" is used to refer to a CTU that fits entirely into a frame or picture, i.e., a CTU inside portion 950 in the example shown in FIG. 9. That is, a non-boundary CTU is located inside an image that is encoded / decoded at its full size vertically and horizontally. However, non-boundary CTUs may be adjacent to at least one image boundary, or they may be additional CTUs between the non-boundary CTU and the image boundary.

[0093] It should be further noted that non-boundary CTUs and boundary CTUs do not constitute different types of CTUs that are coded or specified differently from one another. The difference between non-boundary CTUs and boundary CTUs is whether they are located on a boundary (i.e., a boundary that passes through them) or inside a boundary that delimits an image.

[0094] Furthermore, whether a CTU is a boundary CTU is determined, for example, by comparing the CTU's position (particularly, an appropriate pixel position within the CTU) with the boundary position (or the vertical / horizontal image size of the sample). In the code, a CTU has a predefined fixed size, for example, 128x128 or 64x64 as in the case of HEVC, as described above. A picture is divided into CTUs without overlapping. The lower-right corner may be selected as an appropriate pixel position because it allows the CTU to be determined to be a boundary CTU with respect to each of the vertical and horizontal picture boundaries when the image is processed from left to right and from top to bottom (generally, which pixel position is optimal depends on the processing direction). The encoder / decoder checks the pixel in the lower-right corner of the CTU and compares it with the vertical and horizontal sizes of the image. If the lower-right pixel is located inside the picture boundary, the CTU is a non-boundary CTU; otherwise, it is a boundary CTU. This method of determining whether a partitioned block is on a boundary not only applies to CTUs, but can also be used for CUs and any partitioned blocks that result from splitting a CTU or any block in the partitioning hierarchy.

[0095] Furthermore, other methods for determining whether a CTU is a boundary CTU may be used. For example, as mentioned above, dividing the image size (width / height) by the CTU size may determine the number of CTUs and whether boundary CTUs exist vertically and horizontally. The CTUs may be indexed, and if boundary CTUs exist, the last k CTUs may be determined to be bottom boundary CTUs, or every kth CTU may correspond to a right boundary CTU (k is the number of CTUs per row (including boundary CTUs), i.e., the upper limit of the ratio of image width to CTU size).

[0096] The hierarchical depth of hierarchical partitioning is described above with respect to Figure 3. Thus, the partition depth corresponds to the number of partitioning steps performed to obtain a particular partitioning level, starting with a CTU having a partitioning depth of 0, which corresponds to level 0 (or layer 0), and going up to the depth of the coding unit.

[0097] For unbounded CTUs, the hierarchical depth of the coding unit is limited so that it cannot exceed the maximum hierarchical depth according to settings such as the SPS settings. For example, the settings may set different limits for the partitioning depth of quadtree partitioning and multi-type partitioning. For example, the partitioning of a CTU may include both QT partitioning and MTT partitioning (such as BT, TT, and ABT partitioning) at different levels or different subblocks at the same level, since depth limits may be set separately for the QT partitioning step and the MTT partitioning. For example, the unbounded MTT maximum depth may correspond to the current VTM parameter MaxBTTDepth, or it may be redefined and named, for example, MaxMTTDepth, and applied to all MTT partition types such as BT and TT. Therefore, the depth limit, i.e., the maximum overall depth of CTU partitioning, may be the sum of the respective depth limits for QT partitioning and MTT (e.g., BTT).

[0098] On the other hand, for boundary CTUs, the present disclosure provides a depth constraint given by equation (2) below: TotalBPDepthLim=QTDepth+ExtdMaxBTTDepth(2)

[0099] As can be seen from equation (2), the depth limit of the boundary CTU, also referred to in this disclosure as the boundary partition depth, generally consists of the depth of the QT division and the depth of the MTT division (this is not limited to bisection and trisection divisions, and the name ExtdMaxBttDepth should be considered merely an example). The QT depth may be the same QT depth used in equation (1) in the case of a forced QT division. However, equation (2) differs from equation (1) by a boundary multitype partition maximum depth ExtdMaxBTTDepth (extended binary / triangular tree maximum depth), which may alternatively be referred to, for example, as "ExtdMaxMTTDepth."

[0100] Corresponding to the device 800, a further aspect of the present disclosure provides a method for partitioning an image into coding units. The method includes subdividing an image into coding tree units (CTUs). The CTUs include non-boundary CTUs having predetermined horizontal and vertical sizes and boundary CTUs having a portion within the image separated by a horizontal or vertical image boundary, the portion having a size smaller than the predetermined size in the direction perpendicular to the image boundary. The method further includes hierarchically partitioning the non-boundary CTUs and boundary CTUs into respective coding units. The hierarchical partitioning of the non-boundary CTUs includes multi-type partitioning with a non-boundary multi-type partitioning maximum depth, where the multi-type partitioning is partitioning in either the vertical or horizontal direction. The hierarchical partitioning of the boundary CTUs includes multi-type partitioning with a boundary multi-type partitioning maximum depth.

[0101] In the following, several example embodiments of the present disclosure are described that further demonstrate both the apparatus 800 and the corresponding method for dividing an image into coding units.

[0102] Embodiment 1 As explained above, HEVC or VTM-1.0 uses forced QT for boundary CTU partitioning, so there is no need to define a separate BTT depth limit for boundary partitioning. In HEVC or VTM-1.0, the parameter MaxBttDepth from equation (1) applies to both boundary and non-boundary cases.

[0103] However, since CE1 begins to include BT or TT as a boundary segment, the restriction of MaxBTTDepth of the boundary according to equation (1) may not be inappropriate for the reasons explained above in relation to SubCE2. In embodiment 1, a new MaxBTTDepth is defined that is used only for the boundary segment when the MTT segment structure is included in the boundary segment.

[0104] This disclosure provides a new limit definition for boundary partition MTT depth in hybrid video coding. As mentioned, this boundary multi-type partition maximum depth can be calculated or predefined. This embodiment 1 provides several exemplary methods for calculating the boundary multi-type partition maximum depth.

[0105] In embodiment 1, the boundary multi-type partition maximum depth is at least the sum of the adaptive boundary multi-type partition depth and the predefined multi-type partition depth. The adaptive boundary multi-type partition depth is the depth of the multi-type partition whose partition direction is along the image boundary.

[0106] Therefore, in order to maintain the flexibility of boundary block partitioning, it is proposed to perform fair processing of the overall depth restriction of blocks located on the boundary CTU, and the fairness is compared with the QT partitioning in JEM picture boundary processing as defined by Eq. (1). , exactly MTT division of border CTU Yes do.

[0107] For this purpose, the boundary MTT partition maximum depth used in equation (2) is redefined as follows: ExtdMaxBTTDepth=BTTBPDepth+MaxBTTDepth(3)

[0108] Therein, the boundary MTT partition maximum depth, denoted as ExtdMaxBTTDepth, can be derived from equation (1) or (2). Thus, the overall partition depth limit of the boundary CTU is redefined as follows: TotalBPDepthLim=QTDepth+(BTTBPDepth+MaxBTTDepth)(4)

[0109] In equations (3) and (4), the adaptive boundary multi-type partition depth BTTBPDepth in this embodiment is determined for each layer, and thus ExtdMaxBTTDepth also varies depending on the partitioning layer, i.e., the layer of the coding block of each partitioning step and the layer of the resulting CU that is finally coded and decoded. For example, the adaptive boundary multi-type partition depth can be selected to be the same for different MTT partitioning modes, such as BT / TT / ABT boundary partitioning. The adaptive boundary multi-type partition depth is re-determined (i.e., increased) for each partitioning layer, regardless of whether BT, TT, ABT, or some other MTT partitioning type is performed in this layer. As in the case of QT partitioning, the absolute depth limit can result from a predetermined minimum CU size, for example, the SPS parameter minCUSize mentioned above. For a BT partitioning step, the depth is increased by 1 for each partitioned block resulting from the step. For TT partitioning, according to VTM-1.0, the depth is increased by 1 for the resulting half-size blocks and by 2 for the quarter-size blocks to establish the relationship between coding unit depth and size and to ensure compatibility between BT and TT partitioning.

[0110] As mentioned above, the adaptive multi-type partition depth is a depth whose division direction is in the direction of the image boundary. This means that the division line dividing a block of a corresponding layer into two, three, or more blocks of a subsequent, deeper layer has the same direction, i.e., is parallel to the image boundary passing through the boundary CTU to be divided. However, division / partitioning steps with a division direction perpendicular to the picture boundary do not contribute to the value of the adaptive multi-type partition depth. In particular, if the picture boundary is a horizontal boundary, only steps in the horizontal division direction are counted by the adaptive multi-type partition depth. Alternatively, if the picture boundary is a vertical boundary, only steps whose division direction is vertical are counted. This can be seen in the partitioning examples shown in Figures 10 and 11.

[0111] However, the present disclosure is also applicable to boundary CUs and boundary partition blocks in the partitioning hierarchy of boundary CTUs located on both horizontal (e.g., bottom) and vertical (e.g., right) boundaries, such as the bottom-right CTU. In this case, the MTT steps in both partitioning directions are counted by the BTTBPDepth term in equation (3) or by separate respective variables.

[0112] On the other hand, as can be understood from equations (3) and (4), the non-adaptive term in the bounded multi-type partition maximum depth (i.e., the predefined multi-type partition depth) may be equal to, for example, the non-bounded multi-type partition maximum depth. However, the present disclosure is not limited thereto. For example, if an additional predefined multi-type bounded partition depth parameter is to be defined, it may be beneficial to set this parameter to a smaller value (e.g., to save one or more bits).

[0113] FIG. 10 shows a first example of boundary partitioning using binary tree partitioning, particularly an example of bottom boundary partitioning using BT. The starting point shown on the left side of the upper half of the figure indicates the boundary CTU and the bottom picture boundary (thick solid line) that passes through the boundary CTU. Furthermore, in the upper right corner of the CTU, there is a marked block object obtained by image partitioning. The right side of the upper half of the figure shows the final partitioning (i.e., partitioning pattern) in which the boundary CTU is partitioned. However, in this first example and the example shown in FIG. 10, only the MTT (particularly BTT) partitioning step is considered. According to Equation (4), QT partitioning can be performed in a further step preceding the MTT partitioning step. Therefore, the starting point does not necessarily have to be a CTU, but can also be, for example, a square coding block (with equal vertical and horizontal sizes) of depth greater than 0 resulting from the QT partitioning.

[0114] The bottom half of Figure 10 shows segmentation steps 1 through 3 for segmenting a boundary CTU from the starting point to the final segmentation. In it, the thick solid lines are picture boundaries, BTT boundaries and CTU segments are shown as solid lines (inside the boundary) or dashed lines (outside the boundary), and the dotted lines (in step 3) are "normal" segments of the BTT (i.e., segments that are vertical are not boundary segments whose division direction is along the image boundary). If MaxBTTDepth (the limit is 3 for VTM) is used according to equation (1), the object block in the upper right corner of the CTU cannot be further segmented because there are three MTT segmentation steps, and therefore CurrentBTTDepht(3) >= MaxBTTDepth(3) (CurrentBTTDepth is the depth of BTT segmentation at the current step, corresponding to the number of steps in which BTT segmentation is performed).

[0115] According to embodiment 1, in step 1, there is one layer of (BTT) boundary partitioning (i.e., partitioning in which the split direction is the direction of the boundary). Therefore, BTTBPDepth=1. Therefore, from equation (3), we obtain ExtdMaxBTTdepth=BTTBPDepth+MaxBTTDepth=1+3=4 (MaxBTTDepth is set to 3 in SPS). Since the maximum BTT depth layer (boundary multi-type partition maximum depth) in step 1 is 4 and the block has already been partitioned once, the partitioned block can be MTT partitioned three more times according to ExtdMaxBTTdepth set in step 1 according to the boundary multi-type partition maximum depth.

[0116] In step 2, there are two layers of boundary segmentation obtained by the BP (boundary segmentation) performed so far, and BTTBPDepth=2. Therefore, from equation (3), we obtain ExtdMaxBTTdepth=BTTBPDepth+MaxBTTDepth=2+3=5 (MaxBTTDepth is set to 3 in SPS). The new maximum BTT depth layer (boundary multi-type segmentation maximum depth) in step 2 is 5 (calculated from the starting point (a)), and since the block has already been segmented twice, the segmented block can be segmented three more times according to the boundary multi-type segmentation maximum depth. It can be seen that for boundary segmentation (i.e., the division direction is horizontal and in the same direction as the image boundary direction), BTTBPDepth, and therefore ExtdMaxBttDepth, has increased by 1 from step 1 to step 2.

[0117] In step 3, a further BT segmentation is performed. However, this final segmentation has a segmentation direction that is not in the direction of the image boundary. In particular, the segmentation direction in step 3 is perpendicular to the image boundary. Therefore, step 3 does not contribute to the value of BPPBPDepth. Therefore, there are still two layers of BTT BP, and BTTBPDepth = 2. As follows, ExtdMaxBTTdepth = BTTBPDepth + MaxBTTDepth = 2 + 3 = 5 (MaxBTTDepth is set to 3 in SPS). As explained above, in adaptive multi-type segmentation depth, only the depth (number of steps / layers) of multi-type segmentation whose segmentation direction is toward the image boundary contributes. However, the boundary multi-type segmentation maximum depth is a limitation of MTT boundary segmentation. Therefore, the maximum depth layer in step 3 remains 5 (calculated from the starting point), and the segmented block can be BTT segmented two more times because the block has already been segmented three times (here, all segments, not just BP, are considered).

[0118] FIG. 11 shows a second example of MTT partition maximum depth calculation. In this figure, similar to FIG. 10, the thick solid line is the picture boundary, and the solid line (inside) or dashed line (outside) is the CTU boundary or BTT boundary partition. There is a marked object (i.e., the target coding block obtained by MTT partitioning) located inside the boundary CTU. To obtain the final partition pattern in the upper right part of the figure, four partitioning steps need to be performed, as shown in steps 1 to 4 of the figure.

[0119] If MaxBTTDepth is used according to Equation 1 and limited to 3 as in the case of VTM, the marked target block (at MTT depth 4) cannot be obtained and cannot be further partitioned because CurrentBTTDepht(4)>=MaxBTTDepth(3).

[0120] In step 1 of Figure 11, there is one layer of BTT BP, and BTTBPDepth = 1. Therefore, ExtdMaxBTTdepth = BTTBPDepth + MaxBTTDepth = 1 + 3 = 4 (MaxBTTDepth is set to 3 in SPS). Since the maximum BTT depth layer in step 1 is 4 and the block has already been partitioned once, the partitioned block can be BTT partitioned three more times.

[0121] In step 2 of Figure 11, there are two layers of BTT BP, and BTTBPDepth = 2. Therefore, ExtdMaxBTTdepth = BTTBPDepth + MaxBTTDepth = 2 + 3 = 5. Because the maximum BTT depth layer in step 2 is 5 and the block has already been partitioned once, the partitioned block can be BTT partitioned three more times.

[0122] In step 3 of Figure 11, there are three layers of BTT BP, and BTTBPDepth = 3. Therefore, ExtdMaxBTTdepth = BTTBPDepth + MaxBTTDepth = 3 + 3 = 6. Since the maximum BTT depth layer in step 2 is 6 and the block has already been partitioned once, the partitioned block can be BTT partitioned three more times.

[0123] Furthermore, in step 4, there are four layers of BTT partitioning: ExtdMaxBTTdepth=BTTBPDepth+MaxBTTDepth=4+3=7. Since the maximum depth layer in step 4 is 7 (calculated from starting point (a)), and the block has already been partitioned four times, the partitioned block can be partitioned at most three more times.

[0124] In the boundary multi-type partition maximum depth, all partitions are considered and limited, not just BPs whose partitioning direction is along the boundary. However, in all of the steps in the example shown in Figure 11, multi-type partitions are performed whose partitioning direction is along the image boundary. Therefore, in each of these steps, the value of the adaptive boundary multi-type partition depth (BTTBPDepth in this example) is increased by 1.

[0125] By adapting the depth limit of boundary multi-type partitioning to the number of partitioning steps whose partitioning direction is the same as the boundary, the present disclosure facilitates a coding unit partitioning pattern that more realistically approximates the remaining portions of CTUs near image boundaries. In particular, the number of coding units in boundary CTUs / portions may be reduced, as opposed to using only QT partitioning.

[0126] Embodiment 2 According to the above-described embodiment 1, the boundary multi-type partition maximum depth (for example, given as ExtdMaxBTTDepth) is the sum of at least the adaptive boundary multi-type partition depth and the predefined multi-type partition depth. Here, according to the present embodiment 2, this sum further includes a function of the ratio of the size of the boundary partition block of the boundary CTU in the direction of the image boundary to the direction perpendicular to the image boundary. Among them, the boundary partition block is a block of the adaptive boundary multi-type partition depth.

[0127] In this disclosure, the term "partition block" refers to a CTU or block resulting from dividing a lower-level block / unit. Thus, a partition block is not limited to a CTU at the top of a hierarchical partitioning or coding unit that is the final result of the hierarchical partitioning to be encoded / decoded, but also includes blocks obtained in intermediate partitioning steps, such as steps 1 and 2 in FIG. 10 or steps 1 to 3 in FIG. 11. Furthermore, a "boundary partition block" is a partition block that is located at an image boundary (an image boundary through a block), similar to how a boundary CTU is located at a boundary.

[0128] If the boundary in question is a horizontal image boundary, the size ratio referred to above is the ratio of the horizontal size (width) to the vertical size (height) of the divided boundary segment blocks, whereas if the image boundary is vertical, the ratio is the height divided by the width of the boundary segment blocks.

[0129] For example, the ratio function may be a binary logarithm log2 ratio. In embodiment 1, the overall depth limit is regarded as the criterion for defining the BTT limit of the boundary block. Compared with the calculation of the MTT BP depth limit using Equation 1, embodiment 2 particularly considers the achievable size of the coding unit as a fairness criterion (i.e., fairness between QT division and MTT division). Therefore, the boundary multi-type partition maximum depth is extended according to the following Equation (5). ExtdMaxBTTDepth=log2Ratio+BTTBPDepth+MaxBTTDepth(5) where Ratio indicates the ratio of width to height (width / height if the border is horizontal, or height / width if the border is vertical).

[0130] An example of boundary partitioning according to embodiment 2 is shown in FIG. 12. Part (a) of the figure shows boundary partitioning of VTM-1.0 using forced QT, while part (b) of FIG. 1 shows boundary partitioning using BT. As in FIGS. 10 and 11, solid lines (inside the boundary). Dashed lines indicate CTU and boundary partitioning (the partitioning direction is the boundary direction), and dotted lines indicate partitioning where the partitioning direction is not the same as the boundary direction. In part (a) of FIG. 12, because MaxBTTDepth from the SPS is set to 3 in the VTM-1.0 configuration, the top-left marked block of the CTU can potentially be further partitioned with three additional levels of BT, and no BT partitioning has been performed so far. In part (b), the top-left marked block of the CTU can also be partitioned with three more levels of BT (or other MTT types), and therefore ExtdMaxBTTDepht of the marked block is obtained from equation (5) as follows: ExtdMaxBTTDepth=log2(boundary partition block ratio(4))+BTTBPdepth(2)+MaxBTTDepth(3)=7

[0131] For the current embodiment 2 shown in part (b) of Figure 12, according to ExtendedMaxBTTDepth, which indicates the boundary multi-type partition maximum depth, the marked block can potentially be divided into three further levels of BT (ExtdMaxBTTDepht(7)-CurrentBTDepht(4)) = 3.

[0132] Here, it is assumed that the marked block is obtained by two BT partition steps in which the partition direction is the boundary direction, followed by two BT steps in which the partition direction is perpendicular to the boundary partition, and the resulting current BT depth is equal to 4. However, the present disclosure is not limited to MTT partitioning, which is BTT partitioning. For example, a partitioning mode in which four partitioned blocks are obtained in a single step in which a partitioned block is divided three or more times in the same partitioning direction may also be used (such a partitioning mode in which four partitioned blocks are obtained is sometimes called SplitInto4, which is different from QT partitioning, although four partitioned blocks are also obtained in one step). In the example of FIG. 12, if the last two BT partitioning steps were replaced with one SplitInto4 partitioning step, the resulting current MTT depth would be 3 instead of 4.

[0133] Furthermore, in MTT, the counted hierarchical depth can be determined with respect to the resulting block size of the smallest partition. For example, in the current TT implementation of VTM-1.0, blocks are divided into 1 / 4, 1 / 2, and 1 / 4 subblocks / partition blocks. For the first and third partition blocks, the depth is counted twice in VTM-1.0 to ensure that the depth is equally valid for both BT and TT (and the ABT step can be counted twice for the smaller resulting partition block). Therefore, to ensure compatibility with BT partitioning, a single SplitInto4 step can also be counted as two steps. In this way, the resulting block size of the partition block / coding unit can be understood from the depth. However, in some other possible ways, each step can be counted once, for example, if it is guaranteed that the respective partitioning type is known from several variables.

[0134] Note that in equation (5), especially when BT partitioning is applied (as shown in part (b) of Figure 12), the binary logarithm of the right-side boundary partition block ratio (first term) can be the same as the MTT BP depth (second term).

[0135] It should be further noted that the present disclosure is not limited to a function of the boundary partition block that is a binary logarithm. For example, the quantification itself (identity function) may be used, or, for example, the ratio may be multiplied by a constant or some adaptive parameter.

[0136] Embodiment 3 In the above-described embodiments 1 and 2, how the boundary multi-type partition maximum depth can be calculated is described. Meanwhile, in the present embodiment 3, the boundary multi-type partition maximum depth is predefined (i.e., it is fixed). The predetermined value can be provided as the term BTTBPDepth in equation (4), or it can replace MaxBttDepth in an equation such as equation (1) for the boundary CTU.

[0137] For example, such a fixed boundary multi-type partition maximum depth may be obtained from bitstream signaling. For example, it may be signaled within a set of control parameters common to one or more video frames, such as PPS, SPS, or VPS in HEVC. It may also be signaled once for the entire video sequence. The signaling may include further coding of the parameters, such as differential coding, predictive coding, entropy coding, or any other embedding into the bitstream.

[0138] However, as described above, the present disclosure, including embodiments 1 to 3, is not limited to a fixed, bitstream-signaled boundary multitype partition maximum depth. It may also be derived, for example, based on one or more parameters conveyed in the bitstream (such as MaxBTTDepth in SPS) and / or internal variables such as BTTBPDepth or QTDepth mentioned above. However, the derivation may also be based on a standard or signaled predefined relationship.

[0139] Although the descriptions of some examples (including Equations 1 through 5) refer to BTT partitioning, and variables are named accordingly (e.g., BTTBPDepth, MaxBTTDepth), the present disclosure is also applicable to ABT (Asymmetric Bisection Partitioning) partitioning (wherein a partitioned block is partitioned into two or more partitioned blocks having different sizes, such as 1 / 4 and 3 / 4, in one partitioning step) or other types of MTT partitioning, such as SplitInto4Mode mentioned in connection with part (b) of FIG. 12. Thus, the variables may be named differently, e.g., BTTBPDepth, MaxBTTDepth, etc. The present invention thus facilitates providing flexibility for picture boundary processing using alternative multi-type partition modes in addition to QT partitioning, regardless of the particular MTT partitioning mode used.

[0140] However, in some embodiments, in addition to MTT partitioning, the hierarchical partitioning of bounded CTUs (and the hierarchical partitioning of non-bounded CTUs) may further include QT partitioning. This can be understood from Equation (4), which defines the overall bounded partition maximum depth as the sum of the bounded multitype partition maximum depth (the term in parentheses) and the QT partition depth (internal variable QTDepth). For example, as in VTM-1.0, quaternary partitioning may be performed before MTT partitioning. That is, a QT partitioning step is performed, followed by an MTT partitioning step. However, the present disclosure is not limited thereto and is also applicable to configurations in which QT partitioning is performed after MTT partitioning. Thus, as mentioned, the starting point in FIGS. 10 and 11 need not be a CTU, but may be a partition block obtained from one or more previous partitioning steps.

[0141] Furthermore, according to the present disclosure, the boundary multi-type partition maximum depth may be equal to or greater than the non-boundary multi-type partition maximum depth. For example, if a boundary CTU does not perform multi-type partitioning with a partitioning direction along the boundary, the boundary multi-type partition maximum depth may be equal to the non-boundary multi-type partition maximum depth. In this case, there is no partitioning step in which the internal variable BTTBPDepth (or MTTBPDepth) is incremented. Nevertheless, the apparatus for partitioning an image into coding units performs the determination of the non-boundary multi-type partition depth and the boundary multi-type partition maximum depth in different ways, as described above.

[0142] However, in embodiment 3, where the boundary multi-type partition maximum depth is predetermined, the predetermined value must be greater than the non-boundary multi-type partition maximum depth to allow greater flexibility in MTT partitioning in boundary CTUs.

[0143] A flowchart of the BT / TT (or MTT in general) maximum depth extension of the present disclosure is shown in Figure 13. The condition "Is there a CTU on the boundary?" indicates whether the current CTU / CU is located inside a CTU located on the boundary. If this condition is false (N), the regular MaxBTTDepth from the SPS is used as the non-boundary multitype partition depth to limit the possible BT / TT (or other MTT) maximum depth. Otherwise, MaxBttdepth is extended and determined according to any of the embodiments of the present disclosure.

[0144] As mentioned, the apparatus 800 for partitioning an image may be incorporated into a video encoding apparatus 100 or a decoding apparatus 200 such as those shown in Figures 1 and 2. Accordingly, the present invention further provides an apparatus 100 for encoding an image of a video sequence, i.e., an encoding apparatus, comprising: the apparatus 800 for partitioning an image according to any of the embodiments described in this disclosure; an image coding unit that encodes coding units; and a bitstream formation unit configured to generate a bitstream including the encoded coding units and partitioning information indicating how the coding tree units have been partitioned.

[0145] In particular, the partition information may include information indicating which partitioning type mode is applied for each CTU or for a partitioned block of an intermediate step, from among QT and several MTT partitioning modes. For example, for each partition of a partitioned block or CTU, a parameter indicating the partitioning mode may be included in the bitstream. Alternatively, a specific partitioning mode may be forced to be used.

[0146] Note that the partitioning modes of boundary CTUs and non-boundary CTUs may be different. For example, as illustrated above, the partitioning mode may be signaled on a CTU-by-CTU basis (and / or on a partition block basis) for both boundary and non-boundary CTUs. Alternatively, the partitioning mode of all CTUs on a particular boundary (vertical or horizontal) may be the same and set in signaling associated with one or more video pictures.

[0147] However, the partitioning mode of a boundary CTU may be determined differently from the partitioning mode of a non-boundary CTU. For example, a pre-determined partitioning mode may be enforced for boundary CTUs (and partition blocks), i.e., the standard may define a fixed partitioning mode, or an algorithm for determining the partitioning mode of a boundary CTU based, for example, on the boundaries and / or size and / or its position within the partitioning mode of adjacent CTUs.

[0148] Correspondingly, the present disclosure further provides a method for encoding images of a video sequence, the method including steps of a method for partitioning an image according to any of the embodiments described in the present disclosure, an image coding step for encoding coding units, and a bitstream formation step for generating a bitstream including the encoded coding units and partitioning information indicating how the coding tree units have been partitioned.

[0149] Also provided is an apparatus 200 and a method for decoding images of a video sequence. When incorporated into the decoding apparatus 200, the apparatus 800 is used to determine the division of the (decoded) images into coding units. The decoding apparatus 200 includes a bitstream parser for parsing a bitstream including encoded coding units, the apparatus 800 for determining the division of the images according to any of the embodiments, and an image decoding unit for decoding the encoded coding units based on the determined division of the images.

[0150] Correspondingly, a method for decoding an image includes a step of analyzing a bitstream including encoded coding units, a step of determining a division of the image according to any of the embodiments of the present disclosure, and an image decoding step of decoding the encoded coding units based on the determined division of the image.

[0151] At the decoder side, the maximum BT / TT depth is extended by the boundary CTU / CU, i.e., the boundary multi-type partition maximum depth is used for the decoded boundary CTU. The extension may be simply analyzed from the SPS, or may be derived based on specific conditions. Based on embodiment 1, the maximum BTT depth may be extended by the BTT boundary partition depth. Also, based on embodiment 2, the maximum BTT depth may be extended by the ratio of boundary partition blocks and the BTT boundary partition depth.

[0152] In encoding and decoding video images by the encoder / decoder and the respective encoding / decoding methods, if the partitioning (or partition determination) is performed according to embodiment 3, the boundary multi-type partition maximum depth is predefined. In this case, the encoded / decoded bitstream may further include an encoding sequence parameter set including the boundary multi-type partitioning maximum depth. On the decoder side, the apparatus 800 for determining image partitioning may then be further configured to obtain a second multi-type partitioning maximum depth from the sequence parameter set.

[0153] FIG. 14 is a conceptual or schematic block diagram illustrating one embodiment of a coding system 300, e.g., a picture coding system 300, comprising a source device 310 configured to provide encoded data 330, e.g., an encoded picture 330, to a destination device 320, e.g., for decoding the encoded data 330.

[0154] The source device 310 comprises an encoder 100 or encoding unit 100, and may additionally, i.e. optionally, comprise a picture source 312, a pre-processing unit 314, e.g., a picture pre-processing unit 314, and a communication interface or communication unit 318.

[0155] Picture source 312 may include or be, for example, any kind of picture capture device for capturing real-world pictures and / or any kind of picture generation device, such as a computer graphics processor for generating computer-animated pictures, or any kind of device for acquiring and / or providing real-world pictures, computer-animated pictures (e.g., screen content, virtual reality (VR) pictures), and / or combinations thereof (e.g., augmented reality (AR) pictures). Hereinafter, all these kinds of pictures, as well as any other kind of pictures, will be referred to as “pictures” or “images” unless otherwise specified, and the previous discussion of the term “picture,” which encompasses “video pictures” and “still pictures,” will also remain applicable unless explicitly specified otherwise.

[0156] A (digital) picture is or can be considered as a two-dimensional array or matrix of samples with intensity values. The samples in the array may also be called pixels (short for picture element) or pels. The number of samples in the horizontal and vertical directions (or axes) of the array or picture defines the size and / or resolution of the picture. For color representation, three color components are usually used, i.e., a picture can be represented by or can contain three sample arrays. In an RBG format or color space, a picture comprises corresponding arrays of red, green, and blue samples. However, in video coding, each pixel is usually represented in a luminance / chrominance format or color space, such as YCbCr, which includes a luminance component denoted by Y (although in some cases L is used instead) and two chrominance components denoted by Cb and Cr. The luminance (or luma for short) component Y represents brightness or gray-level intensity (e.g., in the case of a grayscale picture), and the two chrominance (or chroma for short) components Cb and Cr represent chromaticity or color information components. Thus, a picture in YCbCr format includes a luminance sample array of luminance sample values ​​(Y) and two chrominance sample arrays of chrominance values ​​(Cb and Cr). A picture in RGB format can be converted or translated to YCbCr format, and vice versa; this process is also known as color conversion or color translation. If a picture is monochrome, the picture may include only a luminance sample array.

[0157] The picture source 312 may be, for example, a camera for capturing a picture, a memory containing or storing previously captured or generated pictures, e.g., a picture memory, and / or any kind of interface (internal or external) for acquiring or receiving pictures. The camera may be, for example, a local or integrated camera integrated into the source device, and the memory may be, for example, a local or integrated memory integrated into the source device. The interface may be, for example, an external interface for receiving pictures from an external video source, e.g., an external picture capturing device such as a camera, an external memory, or an external picture generating device, e.g., an external computer graphics processor, computer, or server. The interface may be any kind of interface, e.g., a wired or wireless interface conforming to a proprietary or standard interface protocol, an optical interface. The interface for acquiring picture data 313 may be the same interface as or part of the communication interface 318. The communication interface may be any interface, such as Ethernet, WLAN, Bluetooth, LTE, or a wired or non-wired interface such as a satellite or optical interface. Transmission may be peer-to-peer or broadcast or multicast.

[0158] To distinguish between the preprocessing unit 314 and the processing performed by the preprocessing unit 314, the picture or picture data 313 may also be referred to as a raw picture or raw picture data 313.

[0159] The preprocessing unit 314 is configured to receive the (raw) picture data 313 and perform preprocessing on the picture data 313 to obtain a preprocessed picture 315 or preprocessed picture data 315. The preprocessing performed by the preprocessing unit 314 may include, for example, cropping, color format conversion (e.g., from RGB to YCbCr), color correction, or noise removal.

[0160] The encoder 100 is configured to receive pre-processed picture data 315 and to provide encoded picture data (further details are described, for example, with reference to FIG. 1).

[0161] The communications interface 318 of the source device 310 may be configured to receive the encoded picture data and transmit it directly to another device, such as the destination device 320 or any other device, for storage or direct reconstruction, or to process the encoded picture data 330 before storing and / or transmitting the encoded data 330 to another device, such as the destination device 320 or any other device, for decoding or storage, respectively.

[0162] The destination device 320 comprises a decoder 200 or decoding unit 200, and may additionally, i.e. optionally, comprise a communication interface or communication unit 322, a post-processing unit 326, and a display device 328.

[0163] The communication interface 322 of the destination device 320 is configured to receive the encoded picture data or encoded data 330, for example directly from the source device 310 or from any other source, for example a memory, for example a memory of encoded picture data.

[0164] The communication interface 318 and the communication interface 322 may be configured to transmit and receive the encoded picture data or the encoded data 330, respectively, via a direct communication link between the source device 310 and the destination device 320, e.g., a direct wired or wireless connection, or via any type of network, e.g., a wired (such as based on optical, power line, coaxial, or any other medium) or wireless network, or any combination thereof, or any type of private network and public network, or any type of combination thereof.

[0165] The communications interface 318 may be configured, for example, to package the encoded picture data into an appropriate format, e.g., packets, for transmission over a communications link or network, and may further include data loss protection and data loss recovery.

[0166] Communications interface 322, which forms a counterpart of communications interface 318, may be configured to, for example, depackage encoded data 330 to obtain encoded picture data, and may further be configured to perform data loss protection and data loss recovery, including, for example, error concealment.

[0167] Both communication interface 318 and communication interface 322 may be configured as unidirectional communication interfaces, as indicated by the arrows in encoded picture data 330 of FIG. 14 pointing from source device 310 to destination device 320, or as bidirectional communication interfaces, and may be configured, for example, to send and receive messages, for example, to set up connections, to confirm and / or retransmit lost or delayed data including picture data, and to exchange any other information related to the communication link and / or data transmission, e.g., encoded picture data transmission.

[0168] The decoder 200 is configured to receive encoded picture data and provide decoded picture data or decoded pictures (further details are described, for example, with reference to FIG. 2).

[0169] The post-processor 326 of the destination device 320 is configured to post-process the decoded picture data, e.g., a decoded picture, to obtain post-processed picture data 327, e.g., a post-processed picture 327. The post-processing performed by the post-processing unit 326 may include, for example, color format conversion (e.g., from YCbCr to RGB), color correction, cropping, or resampling, or any other processing, for example, to prepare the decoded picture data for display, e.g., by a display device 328.

[0170] Display device 328 of destination device 320 is configured to receive post-processed picture data 327 for displaying the picture, for example, to a user or viewer. Display device 328 may be or comprise any type of display for presenting the reconstructed picture, for example, an integrated or external display or monitor. The display may comprise, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or any type of other display, including beamer, hologram, or 3D / VR glasses.

[0171] 14 depicts source device 310 and destination device 320 as separate devices, an embodiment of the devices may also include both or both functionality, source device 310 or corresponding functionality and destination device 320 or corresponding functionality. In such an embodiment, source device 310 or corresponding functionality and destination device 320 or corresponding functionality may be implemented using the same hardware and / or software, or by separate hardware and / or software, or any combination thereof.

[0172] As will be clear to those skilled in the art based on the description, the presence and (exact) division of functions of different units or functions within the source device 310 and / or destination device 320 as shown in FIG. 14 may vary depending on the actual device and application.

[0173] Therefore, the source device 310 and destination device 320 as shown in FIG. 14 are merely exemplary embodiments of the present invention, and embodiments of the present invention are not limited to those shown in FIG.

[0174] The source device 310 and the destination device 320 may include any of a wide range of devices, including any type of handheld or fixed device, such as, for example, a notebook or laptop computer, a mobile phone, a smartphone, a tablet or tablet computer, a camera, a desktop computer, a set-top box, a television, a display device, a digital media player, a video gaming console, a video streaming device, or a broadcast receiver device, and may use no operating system or any type of operating system.

[0175] 15 is a schematic diagram of a video coding device 1000 according to one embodiment of the present disclosure. The video coding device 1000 is suitable for implementing the disclosed embodiments as described herein, for example as an encoder or decoder. The video coding device 1000 comprises an input port 1010 and a receiver unit (Rx) 1020 for receiving data, a processor, logic unit, or central processing unit (CPU) 1030 for processing data, a transmitter unit (Tx) 1040 and an output port 1050 for transmitting data, and a memory 1060 for storing data. The video coding device 1000 may also comprise optical-electrical (OE) and electro-optical (EO) components coupled to the input port 1010, the receiver unit 1020, the transmitter unit 1040, and the output port 1050 for inputting or outputting optical or electrical signals.

[0176] The present disclosure may be implemented in an apparatus. Such an apparatus may be a combination of software and hardware. For example, intra-prediction and deblocking filtering may be performed by a chip such as a general-purpose processor, a digital signal processor (DSP), or a field programmable gate array (FPGA). However, the present invention is not limited to implementation on programmable hardware. It may be implemented on an application-specific integrated circuit (ASIC) or by a combination of the above-mentioned hardware components.

[0177] The present disclosure further provides a computer-readable medium having stored thereon instructions that, when executed by a processing circuit, cause the processing circuit to perform any of the disclosed methods for segmenting, encoding, and decoding an image. The computer-readable medium may be any medium having a program stored thereon, such as a DVD, a CD, a USB (flash) drive, a hard disk, or server storage available over a network.

[0178] The encoder and / or decoder may be implemented in various devices including a TV set, a set-top box, a PC, a tablet, or a smartphone etc. It may also be software, an app, that performs the method steps.

[0179] In summary, the present disclosure provides an apparatus and method for dividing an image into coding units. The image is divided into hierarchically partitioned coding tree units (CTUs). The hierarchical partitioning includes multi-type partitioning, such as binary tree or quad tree partitioning. Respective multi-type partition depths are selected for CTUs that are entirely within the image and CTUs on the boundary. The present disclosure provides flexibility for multi-type partitioning in boundary portions of the image. [Explanation of symbols]

[0180] 100 Encoder, video encoding device 110 Split Unit 120 residual blocks 125 Reconstructor 130 Conversion Unit 135 Reverse conversion unit 140 quantization units 145 Inverse Quantization Unit 150 Entropy Encoding Units, Entropy Coding Units 160 Loop Filtering Unit 170 Frame buffer, reference picture buffer 180 Mode Selection Unit 190 intra prediction units 195 Inter Prediction Units 200 Video decoder, decoding device 225 Reconstructor 230 Reverse conversion unit 240 Inverse Quantization Unit 250 Bitstream Parsing, Entropy Decoding, and Splitting Unit 260 Loop Filtering Unit 270 Reference Picture Buffer 290 intra prediction units 295 Inter Prediction Units 300 Picture Coding System 310 Source Device 312 Picture Source 313 Raw Picture Data 314 Preprocessor, Picture Preprocessing Unit 315 Preprocessed Picture Data 318 Communication Interface, Communication Unit 320 Destination Device 322 Communication Interface, Communication Unit 326 Postprocessor, Post-processing unit 327 Post-Processed Pictures 328 Display Devices 330 Encoded Data 800 Apparatus for dividing an image into coding units 810 Image Segmentation, Frame Segmentation, Functional Unit 820 Boundary Segmentation 900 bottom picture border 910 Boundary part 920 parts 950 pieces 1000 Video Coding Devices 1010 Input Port 1020 receiver unit 1030 processor 1040 Transmitter Unit 1050 output port 1060 memory 1070 Coding Module

Claims

1. 1. A bitstream storage method comprising: receiving or transmitting a bitstream, the bitstream including partition information indicating how coding tree units (CTUs) of an image are partitioned, the CTUs including non-border CTUs having a predetermined size and boundary CTUs having portions within the image separated by an image boundary, the portions having a size smaller than the predetermined size in a direction perpendicular to the image boundary, the boundary CTUs being hierarchically partitioned into coding units; The hierarchical partitioning of the boundary CTU includes a multi-type partition having a boundary multi-type partition maximum depth; storing said bitstream on one or more storage media; Including, The method, wherein the bounded multi-type partition maximum depth is the sum of an adaptive bounded multi-type partition depth and a predefined multi-type partition depth, the predefined multi-type partition depth being within a sequence parameter set (SPS).

2. The method of claim 1 , wherein the adaptive boundary multi-type partition depth is a multi-type partition depth having a partition direction that is a direction of a horizontal or vertical image boundary.

3. The method of claim 1 , wherein the predefined multitype partition depth is equal to a maximum unbounded multitype partition depth.

4. The method of claim 1 , wherein the adaptive boundary multi-type partition depth is increased by 1 if one layer of binary tree (BT) boundary partitioning (BP) exists.

5. The method of claim 1 , wherein the adaptive boundary multi-type partition depth is not increased if the partition direction is perpendicular to a picture boundary.

6. The method of claim 1 , wherein the adaptive boundary multi-type partition depth is increased by N if there are N layers of binary and ternary tree (BTT) boundary partitioning (BP).

7. 7. The method of claim 6, wherein N is equal to 1, 2, 3, or 4.

8. The method of claim 1 , wherein the boundary CTU is determined by comparing sample positions within the CTU with vertical or horizontal image sizes within the samples.

9. 1. An apparatus for dividing an image into coding units, the apparatus comprising: a transceiver and one or more storage media; The transceiver is configured to receive or transmit a bitstream, the bitstream comprising: the partition information indicating how coding tree units (CTUs) of an image are partitioned, the CTUs including non-border CTUs having a predetermined size and boundary CTUs having portions within the image separated by an image boundary, the portions having a size smaller than the predetermined size in a direction perpendicular to the image boundary, the boundary CTUs being hierarchically partitioned into coding units, the hierarchical partitioning of the boundary CTUs including a multi-type partition having a boundary multi-type maximum partition depth; the one or more storage media are configured to store the bitstream; The apparatus, wherein the boundary multi-type partition maximum depth is the sum of an adaptive boundary multi-type partition depth and a predefined multi-type partition depth, the predefined multi-type partition depth being within a sequence parameter set (SPS).

10. The apparatus of claim 9 , wherein the adaptive boundary multi-type partition depth is a multi-type partition depth having a partition direction that is a direction of a horizontal or vertical image boundary.

11. The apparatus of claim 9 , wherein the predefined multi-type partition depth is equal to a non-bounded multi-type partition maximum depth.

12. The apparatus of claim 9 , wherein the adaptive boundary multi-type partition depth is increased by 1 if one layer of binary tree (BT) boundary partitioning (BP) exists.

13. The apparatus of claim 9 , wherein the adaptive boundary multi-type partition depth is not increased if the partition direction is perpendicular to a picture boundary.

14. 14. The apparatus of claim 9, wherein the adaptive boundary multi-type partition depth is increased by N if there are N layers of binary and ternary tree (BTT) boundary partitioning (BP).

15. 15. The apparatus of claim 14, wherein N is equal to 1, 2, 3, or 4.

16. The apparatus of claim 9 , wherein the boundary CTU is determined by comparing a sample position within the CTU with a vertical or horizontal image size within the sample.

Images