Improved boundary enforcement

Abstract

To provide a method and device of partitioning an image or an image slice.SOLUTION: In particular, an image or image slice is split into an integer number of coding tree units in both vertical and horizontal directions. The remaining boundary portion is then at least partly forced-split so that the deepest partitioning level is a result of binary partitioning, i.e. includes a coding unit which is longer on one side than on the other side. The longer side is parallel or concurrent to the image or image slice boundary.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] FIELD OF THE DISCLOSURE This disclosure relates to image and / or video encoding and decoding, and in particular to dividing an image into coding units. [Background technology]

[0002] Current block-based hybrid video codecs employ predictive coding. Pictures of a video sequence are subdivided into blocks of pixels, and then these blocks are coded. Instead of coding a block pixel by pixel, an entire block is predicted using already coded pixels that are spatially or temporally close to the block. The encoder further processes only the difference between a block and its prediction. Further processing generally involves transforming the block's pixels into coefficients in the transform domain. The coefficients may then be further compressed (e.g., by means of quantization) and further compacted (e.g., by entropy coding) to form a bitstream. The bitstream may further include any signaling information that enables a decoder to decode the coded video. For example, the signaling may include settings regarding encoder settings such as the input picture size, frame rate, quantization step indication, prediction applied to blocks of the picture, etc.

[0003] The difference between a block and a prediction of that block is known as the residual of the block. More specifically, each pixel of the block has a residual, which is the difference between that pixel's intensity level and the predicted intensity level of that pixel. The intensity level of a pixel is called a pixel value or a pixel's value. The residuals of all pixels of a block are collectively called the residual of the block. In other words, a block has a residual, which is a set or matrix of residuals of all pixels of the block. The residual is then transformed, quantized, and coded along with signaling information. Coding may include various forms of fixed and variable length coding, including arithmetic coding or other types of entropy coding.

[0004] In block-based hybrid video coding, each picture is partitioned into blocks of samples, and multiple blocks within a picture are aggregated to form slices as independently decodable entities. A block to which prediction and / or transformation is applied is called a coding unit (CU) or coding block (CB). Coding units may have different sizes.

[0005] For example, in High Efficiency Video Coding (HEVC, also known as H.265), a video frame is subdivided into coding tree units (CTUs, also called coding tree blocks, CTBs). CTBs are disjoint square blocks of the same size, e.g., 64 x 64 samples. Each CTB serves as the root of a coding tree, a quadtree structure that partitions blocks. CTBs may be further subdivided into coding blocks along the coding tree structure. For coding blocks, a type of prediction is determined. Coding blocks may be further divided into smaller transform blocks, to which transform and quantization are applied.

[0006] More information on partitioning in HEVC can be found in V. Sze et al. (eds.), High Efficiency Video Coding (HEVC): Algorithms and Architectures, Springer, 2014, Chapter 3.2.

[0007] Furthermore, WO 2016 / 090568 shows a binary tree structure for partitioning a unit into multiple smaller units using a binary tree structure in addition to a quad tree. Thus, first, the root unit is partitioned into a quad tree structure, and then the leaf nodes of the quad tree are further partitioned by the binary tree structure. Summary of the Invention [Means for solving the problem]

[0008] In the HEVC standard, CTUs and CUs at slice or picture boundaries are forced to split using quadtree partitioning until the bottom-right sample of the leaf node falls within the slice or picture boundary. The forced quadtree partitioning does not need to be signaled in the bitstream as long as the partitioning rules are fixed (predefined). The purpose of the forced partitioning is to allow encoders and decoders to process pictures and slices with horizontal and / or vertical sizes that are not integer multiples of the respective horizontal and / or vertical sizes of the CTUs.

[0009] In HEVC and WO 2016 / 090568, the forced partitioning process hierarchically splits CTUs and CUs at frame boundaries using a quadtree approach without rate-distortion (RC) optimization until the entire current CU falls within a slice or picture boundary. These forced partitionings do not need to be signaled in the bitstream. However, splitting into very small coding units may introduce additional complexity and signaling requirements related to further processing (encoding, decoding) of the small coding units.

[0010] According to the present disclosure, forced partitioning applies binary tree partitioning at least at the deepest hierarchical level of partitioning, or at least at levels of partitioning other than the deepest hierarchical level of partitioning where no coding unit remains completely within a boundary portion.

[0011] Such a division provides greater efficiency as it results in a smaller number of coding units at the deepest hierarchical levels.

[0012] According to an aspect of the invention, an apparatus is provided for dividing an image portion into coding units, the apparatus including processing circuitry that, during operation, subdivides the image portion into an integer number of coding tree units having a predetermined size and boundary portions having a size in at least a first direction, either horizontally or vertically, that is less than the predetermined size, and hierarchically partitions the boundary portions into coding units including a deepest coding unit having different sizes in each horizontal and vertical direction, the deepest coding unit being smaller in the first direction.

[0013] The advantage of this approach is that it avoids too many small CUs on the picture boundaries, which can also result in more efficient coding of the boundary parts.

[0014] According to an example, the processing circuitry is configured to partition the boundary portion by binary tree partitioning at least at the deepest partitioning level.

[0015] In addition to or as an alternative to binary tree partitioning, the processing circuitry may also be configured to partition the boundary portion by quadtree partitioning.

[0016] According to one example, during operation, the processing circuit partitions the boundary portion by quad-tree partitioning at the N shallowest levels, where N is an integer greater than or equal to zero and less than the number of hierarchical levels of the partitioning, and partitions the boundary portion by binary tree partitioning at the remaining hierarchical levels of the partitioning.

[0017] This approach offers greater flexibility in construction and further ensures that the boundaries are coded efficiently by avoiding a large number of small square boundary CUs.

[0018] In one implementation, the processing circuitry segments the boundary portion during operation until the deepest level coding unit remains entirely within the boundary portion.

[0019] In one implementation, the processing circuitry operates by partitioning the boundary portion horizontally and vertically using binary tree partitioning.

[0020] Additionally or alternatively, the processing circuitry may, during operation, partition the corners of the boundary at each hierarchical level by quadtree partitioning.

[0021] As an alternative to mixing quadtree and binary tree partitioning for the CTUs of the boundary portion, the processing circuitry may be configured to partition the boundary portion by binary tree partitioning at all levels of partitioning.

[0022] According to an aspect of the invention, an apparatus for encoding images of a video sequence is provided, the apparatus including: an apparatus for dividing an image portion into coding units according to any of the above-mentioned examples and implementations; an image coding unit configured to code the coding units; and a bitstream forming unit configured to generate a bitstream including the encoded coding units and partitioning information indicating how the coding tree units are partitioned.

[0023] For example, the encoder further includes a rate-distortion optimization unit configured to determine hierarchical partitioning of the boundary portions by rate-distortion optimization for any of the levels of partitioning apart from the deepest level.

[0024] Another aspect of the present invention includes an apparatus for decoding images of a video sequence, the apparatus including: an apparatus for determining a division of an image portion to be decoded into coding units according to any of the above examples and implementations; a bitstream parser for parsing a bitstream including coded coding units and division information indicating how coding tree units are divided and based on the determined division of the image portion; and an image decoding unit for decoding the coded coding units.

[0025] According to an exemplary implementation, the division of the boundary portion into coding units is a predefined division, and the partitioning information is not related to the division of the boundary portion.

[0026] The processing circuit of the decoding device and / or the encoding device may be configured to partition the boundary portion by quadtree partitioning at the N shallowest levels, where N is an integer greater than or equal to zero and less than the number of hierarchical levels of partitioning, and partition the boundary portion by binary tree partitioning at the remaining hierarchical levels of partitioning. Furthermore, in an exemplary implementation, the partitioning information includes N, or N is derived based on a type of prediction applied to encode the processed image. N may be derived based on the size of boundary portions having a size less than a predetermined size in either the horizontal or vertical direction.

[0027] According to an aspect of the present invention, there is provided a method for dividing an image portion into coding units, the method comprising the steps of: subdividing the image portion into an integer number of coding tree units having a predetermined size and boundary portions having a size in at least a first direction, either horizontally or vertically, that is less than the predetermined size; and hierarchically partitioning the boundary portions into coding units including a deepest coding unit having different sizes in each horizontal and vertical direction, the deepest coding unit being smaller in the first direction.

[0028] In an exemplary embodiment, the partitioning of the boundary portion further includes the steps of starting from a current partitioning level, which is the shallowest partitioning level; sequentially dividing the boundary portion into coding units of the current partitioning level until the boundary coding units of the current partitioning level fit entirely within the boundary portion; and as soon as a coding unit of the current partitioning level does not fit within the boundary portion, increasing the partitioning level and further partitioning said coding units, wherein quadtree partitioning is used for the N shallowest partitioning levels and binary tree partitioning is used otherwise, and N is an integer greater than or equal to zero and less than the number of hierarchical levels of partitioning.

[0029] According to another aspect of the present invention, there is provided a method for encoding images of a video sequence, the method comprising the steps of dividing an image portion into coding units according to any of the methods specified above, encoding the coding units, and generating a bitstream comprising the encoded coding units and partitioning information indicating how the coding tree units are partitioned.

[0030] According to another aspect of the present invention, a method for decoding images of a video sequence, comprising the steps of determining a division of an image portion to be decoded into coding units according to any of the methods specified above, analyzing a bitstream containing partitioning information based on the determined division of the image portion, indicating how the coded coding units and coding tree units are partitioned, and decoding the coded coding units.

[0031] According to an example, the partitioning step partitions the boundary portion by binary tree partitioning at least at the deepest partitioning level.

[0032] In addition to or as an alternative to binary tree partitioning, the partitioning step may also partition the boundary portion by quadtree division.

[0033] According to one example, the partitioning step includes partitioning the boundary portion by quad-tree partitioning at the N shallowest levels, where N is an integer greater than or equal to zero and less than the number of hierarchical levels of the partitioning, and partitioning the boundary portion by binary tree partitioning at the remaining hierarchical levels of the partitioning.

[0034] In one implementation, the partitioning step partitions the boundary portion until the deepest level coding unit remains entirely within the boundary portion.

[0035] In one implementation, the processing circuitry operates by partitioning the boundary portion horizontally and vertically using binary tree partitioning.

[0036] Additionally or alternatively, the processing step partitions the corners of the boundary at each hierarchical level by quadtree decomposition.

[0037] As an alternative to mixing quadtree and binary tree partitioning for the CTUs of the boundary portion, the partitioning step may partition the boundary portion by binary tree partitioning at all partitioning levels.

[0038] According to another aspect of the present invention, a computer-readable medium is provided that stores instructions that, when executed by a processor, cause the processor to perform a method according to any of the above aspects and examples.

[0039] In the following, exemplary embodiments will be explained in more detail with reference to the accompanying figures and drawings. [Brief explanation of the drawings]

[0040] [Figure 1] FIG. 2 is a block diagram illustrating an example structure of a video encoder. [Figure 2] FIG. 2 is a block diagram illustrating an exemplary structure of a video decoder. [Figure 3] FIG. 1 is a schematic diagram illustrating an example of quadtree partitioning adopted by HEVC. [Figure 4] FIG. 1 is a schematic diagram showing an example of quadtree / binary tree division. [Figure 5] FIG. 10 is a schematic diagram showing forced quadtree division of a boundary portion. [Figure 6] FIG. 1 is a block diagram illustrating an example of a device that may act as a splitting unit of an encoder or decoder. [Figure 7] FIG. 10 is a schematic diagram showing division of a boundary portion. [Figure 8] FIG. 1 is a schematic diagram illustrating an example of forced binary tree splitting of vertical boundaries. [Figure 9] 1A-1C are schematic diagrams showing examples of partitioning on vertical boundaries, horizontal boundaries, and combined boundaries. [Figure 10] 10 is a flow diagram illustrating an exemplary partitioning syntax determination. DETAILED DESCRIPTION OF THE INVENTION

[0041] The present invention relates to the division of images into smaller units for further processing. Such division may be advantageously used in encoding and decoding still or video images. In the following, exemplary video coders and decoders capable of implementing the division according to the present disclosure are described.

[0042] FIG. 1 shows an encoder 100 including an input for receiving an input block of a frame or picture of a video stream and an output for providing an encoded video bitstream. The term "frame" in this disclosure is used as a synonym for picture. However, it is noted that this disclosure is also applicable to fields when interlacing is applied. Generally, a picture includes m by n pixels, which correspond to image samples and may each include one or more color components. For simplicity, the following description refers to pixels representing luminance samples. However, it is noted that the segmentation techniques of this disclosure can be applied to any color component, including chrominance or components of a color space such as RGB. Alternatively, it may be beneficial to perform the segmentation on only one component and then apply the determined segmentation to further (or all) remaining components.

[0043] The encoder 100 is configured to apply segmentation, prediction, transformation, quantization, and entropy coding to the video stream.

[0044] In the division unit 110, the input video frame is further divided before encoding. The blocks to be coded are not necessarily of the same size. A picture may include blocks of different sizes, and the block rasters of different pictures in a video sequence may also be different. In particular, each video image (picture) is first subdivided into CTUs of the same fixed size. The CTU size may be, for example, standardized and predefined. In HEVC, a size of 64 x 64 is used. However, the present disclosure is not limited to standardized fixed sizes. It may be advantageous to provide a CTU size that can be set in the encoder and provided as a signaling parameter in the bitstream. For example, different CTU sizes may be beneficial for different picture sizes and / or content types. The CTU size may be signaled at any signaling level; for example, it may be common to the entire video sequence or part of it (i.e., multiple pictures), or individual for each picture. Correspondingly, the CTU size may be signaled, for example, in a picture parameter set (PPS) known from current codecs (H.264 / AVC, H.265 / HEVC), or in a sequence parameter set (SPS), or in a video parameter set (VPS), or in a similar parameter set. Alternatively, the CTU size may be specified in a slice header or at any other level. The CTU size may take values ​​different from 64 x 64. The CTU size may, for example, be 128 x 128 samples in size. Generally, to perform a hierarchical division of a quadtree by a binary tree, it may be beneficial to provide a CTU size that is a power of 2, i.e., in the format 2^n, where n is an integer greater than 2.

[0045] The partitioning of a picture into CTUs and the partitioning of CTUs into CUs from V. Sze et al. (eds.), High Efficiency Video Coding (HEVC): Algorithms and Architectures, Springer, 2014, is shown in Figure 3. The partitioning follows a quadtree structure to accommodate various local characteristics. On the left, Figure 3 shows a CTU hierarchically partitioned according to the quadtree structure on the right. In particular, the coding tree defines syntax that specifies the subdivision of a CTU into CUs. Like a CTU, a CU consists of square blocks of samples and syntax associated with these sample blocks. Thus, the partitioning is performed hierarchically, starting with a CTU (hierarchical depth 0) that may or may not be subdivided into four (quadtree) CUs at hierarchical depth 1. 3, the CTU is divided into CUs 8 and 16 at the first hierarchical depth (level), which are not further divided and therefore form leaves of the quadtree, and two further CUs that are further divided into CUs at depth 2 of the hierarchy (depth 2 CUs). In particular, the top-left depth 1 CU is further subdivided into depth 2 CUs 1, 2, and 7, which form leaves of the quadtree, and another CU that is further divided into depth 3 CUs 3, 4, 5, and 6, which are all leaves. Similarly, the bottom-left depth 1 CU is further divided into depth 2 CUs 13, 14, and 15, which are also leaves of the quadtree, and the remaining CUs that are further divided into level 3 CUs 9, 10, 11, and 12, which are all leaves and therefore are not further divided.

[0046] An exemplary syntax for HEVC quadtree partitioning is shown in Table 1 below.

[0047] [Table 1]

[0048] In particular, at the CTU level, a flag named split_cu_flag is included in the bitstream to indicate whether the complete CTU forms a CU or whether the CTU is to be split into four equally sized blocks corresponding to square sample blocks. If a CTU is split, another split_cu_flag is transmitted for each of the resulting blocks, specifying whether the block represents a CU or whether the block is to be further split into four equally sized blocks. This hierarchical subdivision continues until none of the resulting blocks are further subdivided. The minimum size of a CU is signaled in the sequence parameter set, and the minimum size can range from 8 x 8 luma samples to the size of the CTU. When the minimum CU size is reached in the hierarchical subdivision process, no split flag is transmitted for the corresponding block; instead, it is inferred that these blocks will not be further split. In a typical HEVC encoder configuration, the maximum range of supported CU sizes is utilized, so that CUs ranging from 8 x 8 samples to 64 x 64 samples can be used. CUs within a CTU are coded in depth-first order. This coding order is also called z-scanning. It ensures that for each CU, except for CUs at the top or left boundary of the slice, all samples above and to the left of the CU have already been coded, so that the corresponding samples can be used for intra prediction and the associated coding parameters can be used to predict the coding parameters of the current CU.

[0049] In other words, split_cu_flag[x0][y0] specifies whether the coding unit is split into coding units with half the horizontal and vertical size. The array indices x0, y0 specify the position (x0, y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture. When split_cu_flag[x0][y0] is not present, the following applies at the decoder: - If log2CbSize (a parameter that specifies the coding block size) is greater than MinCbLog2SizeY (a parameter that specifies the smallest configurable coding unit size), the value of split_cu_flag[ x0 ][ y0 ] is inferred to be equal to 1. - Otherwise (log2CbSize is equal to MinCbLog2SizeY), the value of split_cu_flag[ x0 ][ y0 ] is inferred to be equal to 0.

[0050] The array CtDepth[ x ][ y ] specifies the depth of the coding tree for the luma coding block covering position ( x, y ). When split_cu_flag[ x0 ][ y0 ] is equal to 0, CtDepth[ x ][ y ] is inferred to be equal to cqtDepth for x = x0..x0 + nCbS - 1 and y = y0..y0 + nCbS - 1.

[0051] Figure 4 shows mixed quadtree and binary tree partitioning. Quadtree partitioning is indicated by solid lines, while binary tree partitioning is indicated by dashed lines. The label 1 or 0 of the node representing the coding unit that is further divided by the binary tree indicates whether the binary division is applied vertically or horizontally, respectively.

[0052] The horizontal and vertical dimensions of a video picture in luma samples must be an integer multiple of the smallest CU size in luma samples signaled in the sequence parameter set, but need not be an integer multiple of the CTU size. If the horizontal or vertical dimensions of a video picture do not represent an integer multiple of the CTU size, it is inferred that the CTUs at the boundaries are split until the resulting block boundaries coincide with the picture boundaries. For this forced split, no split flag is signaled, but the resulting block may be further split using the quadtree syntax described above. CUs that fall outside the picture area are not coded.

[0053] This division is shown in Figure 5. In particular, Figure 5 shows the frame boundary, and the 56 lines (128 samples long) above the frame boundary are the slice or image boundary portion. The portion of the CTU below the frame boundary may belong to another slice, or may not exist at all, for example, if the frame boundary is the bottom boundary of the picture. As can be seen, the forced quadtree division is applied to 128 x 56 samples.

[0054] The subdivision of the chroma CTB is always aligned with the respective luma CTB subdivision in HEVC. It is noted that this disclosure may treat chroma components in the same way, but is not limited to it. There may also be independent division of different color components.

[0055] After performing image segmentation in segmentation unit 110, transformation, quantization, and entropy coding are performed by transform unit 130, quantization unit 140, and entropy coding unit 150, respectively, to generate an encoded video bitstream as output.

[0056] A video stream may include multiple frames. For example, a block of a first frame of the video stream is intra-coded by intra prediction unit 190. An intra-frame is coded using only information from that frame and, therefore, can be decoded independently of other frames. Thus, an intra-frame can provide an entry point into the bitstream, for example, for random access. Blocks of other frames of the video stream may be inter-coded by inter prediction unit 195, and each block of the inter-coded frame is predicted from a block of another frame (reference frame), for example, a previously coded frame. Mode selection unit 180 is configured to select whether a block of a frame should be intra-predicted or inter-predicted, i.e., whether the block is processed by intra prediction unit 190 or inter prediction unit 195. Mode selection unit 180 also controls intra parameters of the inter prediction. To enable refreshing of image information, an inter-coded frame may include not only an inter-coded block but also one or more intra-coded blocks. In contrast, intra-frames contain only intra-coded blocks, and no inter-coded blocks. Intra-frames may be inserted into a video sequence (e.g., regularly, i.e., every time after a certain number of inter-frames) to provide an entry point for decoding, i.e., a point at which a decoder can begin decoding without using information from previous frames.

[0057] The intra prediction unit 190 is a block prediction unit. To perform spatial or temporal prediction, the coded block may be further processed by the inverse quantization unit 145 and the inverse transform unit 135. After the block reconstruction by the reconstructor 125, the loop filtering unit 160 may be applied to further improve the quality of the decoded image. The reconstructor 125 adds the decoded residual to the predictor to obtain a reconstructed block. The filtered block then forms a reference frame, which is then stored in the frame buffer 170. Such a decoding loop (decoder) on the encoder side provides the advantage of generating a reference frame that is the same as the reference picture reconstructed on the decoder side. Therefore, the encoder and decoder sides operate in a corresponding manner. The term "reconstruction" in this specification refers to obtaining a reconstructed block by adding a decoded residual block to a prediction block.

[0058] The inter prediction unit 195 receives as input a block of the current frame or picture to be inter-coded and one or more reference frames or pictures from the frame buffer 170. Motion estimation and motion compensation are performed by the inter prediction unit 195. Motion estimation is used, for example, to obtain a motion vector and a reference frame based on a cost function. Motion compensation is then performed by translating a reference block of the reference frame to the current frame, that is, describing the current block of the current frame by a motion vector. The inter prediction unit 195 selects a predictive block (i.e., a predictor) for the current block from a set of candidate blocks (i.e., candidate predictors) in one or more reference frames such that the predictive block minimizes the cost function. In other words, the candidate block with the smallest cost function is used as the predictive block for the current block.

[0059] For example, the cost function may be a measure of the difference between the current block and the candidate block, i.e., a measure of the residual error of the current block relative to the candidate block. For example, the cost function may be the sum of absolute differences (SAD) between all pixels (samples) of the current block and all pixels of the candidate block in the candidate reference picture. However, in general, any similarity metric, such as mean square error (MSE) or structural similarity metric (SSIM), may be employed.

[0060] However, the cost function may also be the number of bits required to encode such an inter block and / or the distortion resulting from such encoding. Thus, a rate-distortion optimization procedure may be used to determine coding parameters such as the selection of motion vectors and / or whether and in what settings inter or intra prediction should be used for a block in general.

[0061] The intra prediction unit 190 receives as input a block of the current frame or picture to be intra-coded and one or more reference samples from an already reconstructed region of the current frame. Intra prediction then describes the pixels of the current block of the current frame as a function of the reference samples of the current frame. The intra prediction unit 190 outputs a prediction block for the current block, which advantageously minimizes the difference between the current block to be coded and its prediction block, i.e., minimizes the residual block. The minimization of the residual block can be based, for example, on a rate-distortion optimization procedure. In particular, the prediction block is obtained as a directional interpolation of the reference samples. The direction may be determined by rate-distortion optimization and / or by calculating the similarity measure described above in connection with inter prediction.

[0062] The difference between the current block and its prediction, i.e., the residual block, is then transformed by transform unit 130. The transform coefficients are quantized by quantization unit 140 and entropy coded by entropy coding unit 150. The coded video bitstream thus generated includes intra-coded and inter-coded blocks and corresponding signaling (such as mode indication, motion vector indication, and / or intra-prediction direction). Transform unit 130 may apply a linear transform, such as a discrete Fourier transform (DFT) or a discrete cosine transform (DCT). Such a transformation to the spatial frequency domain provides the advantage that the resulting coefficients generally have higher values ​​at lower frequencies. Thus, after effective coefficient scanning (e.g., zigzag) and quantization, the resulting sequence of values ​​generally has some larger values ​​at the beginning and end of the zero runs. This allows for more efficient coding. Quantization unit 140 performs lossy compression by reducing the resolution of the coefficient values. The entropy coding unit 150 then assigns binary codewords to the coefficient values. The codewords are written into a bitstream, called the coded bitstream. The entropy coder also encodes signaling information (not shown in FIG. 1), which may include encoding the split flag syntax shown above.

[0063] 2 illustrates an example of a video decoder 200. The video decoder 200 includes, among other things, a reference picture buffer 270 and an intra-prediction unit 290, which is a block prediction unit. The reference picture buffer 270 is configured to store at least one reference frame reconstructed from an encoded video bitstream. The intra-prediction unit 290 is configured to generate a predictive block, which is an estimate of a block to be decoded. The intra-prediction unit 290 is configured to generate this prediction based on reference samples obtained from the reference picture buffer 270.

[0064] Decoder 200 is configured to decode the encoded video bitstream generated by video encoder 100, but preferably both decoder 200 and encoder 100 generate identical predictions for each encoded / decoded block. The features of reference picture buffer 270 and intra prediction unit 290 are similar to the features of reference picture buffer 170 and intra prediction unit 190 of FIG. 1.

[0065] The video decoder 200 includes further units that are also present in the video encoder 100, such as, for example, an inverse quantization unit 240, an inverse transform unit 230, and a loop filtering unit 260, which correspond to the inverse quantization unit 140, the inverse transform unit 150, and the loop filtering unit 160, respectively, of the video coder 100.

[0066] The bitstream parsing, entropy decoding, and partitioning unit 250 is configured to parse and decode the received coded video bitstream to obtain quantized residual transform coefficients and signaling information. The quantized residual transform coefficients are provided to the inverse quantization unit 240 and the inverse transform unit 230 to generate residual blocks. The residual blocks are added to the prediction blocks in the reconstructor 225, and the resulting sum is provided to the loop filtering unit 260 to obtain decoded video blocks. Frames of decoded video are stored in the reference picture buffer 270 and can serve as reference frames for inter-prediction. The signaling information parsed and decoded from the bitstream may generally include control information related to frame partitioning. To further correctly parse and decode the image, the control information is used to restore the division of the image into coding units to correctly assign subsequent decoded data to the respective coding units.

[0067] Generally, the intra prediction units 190 and 290 of Figures 1 and 2 can use reference samples from already coded regions to generate a prediction signal for a block that needs to be coded or decoded.

[0068] The bitstream parsing, entropy decoding, and splitting unit 250 receives the encoded bitstream as its input. First, the bitstream may be parsed, that is, signaling parameters and residuals are extracted from the bitstream. The syntax and semantics of the bitstream may be defined by a standard so that encoders and decoders may work interoperably.

[0069] This disclosure presents a new approach for forced boundary partitioning in block-based hybrid video coding. In particular, the forced boundary partitioning is performed, at least in part, by using a binary tree (BT) structure. The inventors have observed that since most boundary blocks may use the same prediction mode, forced quad tree (QT) partitioning for frame boundary CTUs is not always necessary. Instead of forced QT partitioning, in one embodiment, forced horizontal BT for the bottom boundary part and / or forced vertical BT for the right boundary part are introduced (assuming that the upper and lower neighboring parts have already been reconstructed in the processing order of the left-to-right and top-to-bottom CTUs).

[0070] 6, according to an embodiment, an apparatus 600 (which may be implemented in units 110, 250 described above) for dividing an image portion into coding units (610) includes a processing circuit that, during operation, subdivides the image portion into an integer number of coding tree units having a predetermined size and boundary portions having a size in at least a first direction, either horizontally or vertically, that is less than the predetermined size. Further, during operation, the processing circuit hierarchically partitions (630) the boundary portions into coding units having different sizes in each horizontal and vertical direction, with the deepest coding unit being smaller in the first direction.

[0071] The CTUs obtained by frame division 610 may be further hierarchically partitioned 620. This partitioning may be performed in any manner, for example, as shown in and described above with reference to Figures 3 and 4.

[0072] 6 shows the internal structure of the circuit of the device 600. This circuit may be any kind of hardware and software having functional units 610, 620, and 630 for respective division of the frame into CTUs, partitioning of the CTUs, and partitioning of the boundary portions. These units may be implemented, for example, in a single processor. However, the invention is not limited to such application, and these units may also be implemented by separate hardware parts.

[0073] The border portions may be on vertical boundaries only (if the vertical picture size is not an integer multiple of the vertical CTU size), or on horizontal boundaries only (if the horizontal picture size is not an integer multiple of the horizontal CTU size), or on both vertical and horizontal boundaries (if the horizontal and vertical picture sizes are not integer multiples of the respective horizontal and vertical CTU sizes).

[0074] A typical picture to be coded has a size that is a multiple of the CTU. FIG. 7 visualizes an example of a picture boundary 700 below and the corresponding (filled) boundary portion 710. Portion 750 indicates the remaining portion of the image that has a size that is an integer multiple of the CTU in the vertical and horizontal directions. In particular, the vertical size of the CTU is denoted as 770V, while the horizontal size of the CTU is denoted as 770H. As can be seen in FIG. 7, the boundary portion in this example is an integer multiple of the CTU size 770H in the horizontal direction. However, in the vertical direction, boundary portion 710 has a size that is the remainder of dividing the picture size in the vertical direction by the CTU size 770V in the vertical direction. Portion 720 is only virtual and indicates the difference between the height of the boundary portion and the CTU size. It is noted that in this implementation, the CTU is square, and therefore, sizes 770H and 770V are the same. However, this disclosure is not limited thereto, and the vertical and horizontal sizes of the CTU may be different.

[0075] To encode (and correspondingly decode) the boundary portion, the boundary portion 710 in Figure 7 is divided into incomplete CTUs, i.e., portions having a horizontal size 770H of the CTU and a vertical size 770V that is smaller than the CTU size. These incomplete CTUs are further forced to be partitioned as shown in Figure 8.

[0076] 8 shows an example in which a boundary portion is at a horizontal boundary, and therefore division is performed vertically. In particular, the left side of FIG. 7 shows a boundary portion that corresponds to the width of the CTU (in this example, the width is 128 samples), but corresponds to a height (only 56 samples) that is lower than the height of the CTU. Vertical boundaries are forced to be partitioned by binary tree division. In particular, since a 1-unit depth with 64 samples does not fit into a 56-sample high boundary portion, the largest unit size that fits a length of 56 samples is a 2-unit depth with a size of 32 samples. The remaining 24 samples are further divided into a 3-unit depth with a size of 16 samples and a remainder of 8 samples (corresponding to the size of a 4-unit depth), and the remainder reaches the picture boundary, so no further division is required. In this example, division is not applied to horizontal boundaries. In summary, a boundary unit having the size of a CTU in one direction and a smaller size in another direction is partitioned by binary tree partitioning, i.e., by hierarchically partitioning the boundary unit into two parts, one having the size of the next shallower CU to fit the smaller size, and the remaining part being further divided until the deepest CU reaches the frame boundary. This partitioning of the 128x56 sized CTU part here results in three boundary coding units having sizes of 128x32, 128x16, and 128x8, respectively.

[0077] In summary, according to one implementation, the processing circuitry is configured to partition the boundary portions at all levels of partitioning by binary tree partitioning.

[0078] The right side of Figure 8 shows an example of partitioning where quad-tree partitioning is first applied at hierarchical depth 1 (hierarchical depth 0 corresponding to the CT size), resulting in two CUs of size 64 x 56, which are then further bi-tree partitioned into respective pairs of CUs of size 64 x 32, 64 x 16, and 64 x 8.

[0079] The example of Figure 8 shows the partitioning of the bottom boundary. However, the present disclosure is equally applicable to the partitioning of any other boundary, such as the right boundary. If the subdivision of a picture into CTUs is performed from left to right and from top to bottom, then the boundary portion is at the bottom and / or right side of the picture. However, the partitioning may also be performed from bottom to top and from right to left, in which case the top and left boundaries need to be partitioned, which is possible in the same way as described with reference to Figure 8. Similarly, the above example shows the partitioning of such an incomplete CTU starting from the top of the CTU. However, it is also generally possible to partition an incomplete CTU from the picture (frame) boundary upwards.

[0080] As shown on the right side of FIG. 8, the incomplete CTU partitioning may be a mixed QT and BT partitioning. The processing circuit may be configured to partition boundary portions by binary tree partitioning at least at the deepest partitioning level. This provides the advantage that larger boundaries that overlap frame boundaries do not need to be partitioned into the smallest units. Too many, too small CUs on frame boundaries may result in increased coding complexity and a larger bitstream. In this embodiment, it is noted that as long as the deepest CU is larger on one side than the other (i.e., resulting from binary tree partitioning), the remaining depth hierarchy may be determined in any manner and may not be constrained in a predefined manner. Rather, the partitioning may be performed by RD optimization, accompanied by corresponding control information that allows a decoder to restore the same partitioning. The partitioning may also be partially constrained and partially available for RD optimization.

[0081] In other words, the processing circuitry may be configured to partition the boundary portion also by quadtree partitioning. Thus, there may be a predefined mandatory QT or BT partitioning for each depth, either defined by the standard or configurable by control information included in the bitstream.

[0082] For example, forced BT partitioning is performed only after a certain number of forced QT levels. This may be achieved in that the processing circuitry is configured to partition the boundary portion by quadtree partitioning at the N shallowest levels, where N is an integer greater than or equal to zero and less than the number of hierarchical levels of partitioning. The processing circuitry is further configured to partition the boundary portion by binary tree partitioning at the remaining hierarchical levels of partitioning. Referring to Figure 8, there is no QT partitioning performed on the left side.

[0083] Thus, the control information in the bitstream may include a QT / BT indicator that specifies whether QT is applied and, if so, how many levels are QT partitioned. Correspondingly, such a QT / BT indicator may take multiple values, such as a value of 0 when QT partitioning is not applied and a value corresponding to N (as shown on the left in FIG. 8). Such signaling is shorter than the signaling for each CU at each level, regardless of whether QT partitioning is applied, BT partitioning is applied, or no partitioning is applied. However, the present disclosure may use any type of signaling.

[0084] Alternatively, or in combination with control signaling, the level of forced QT can be adaptively selected depending on the time ID or slice type. This approach offers the advantage that no explicit signaling is required (or a smaller amount of explicit signaling is required). Furthermore, the forced partitioning can be adaptively set in relation to at least the type of prediction.

[0085] HEVC defines temporal identifiers (IDs) that indicate levels in a hierarchical temporal prediction structure (dependencies between inter-predicted frames). Slice types can be, for example, intra-predicted slices, inter-predicted slices that apply prediction based on only one reference picture, inter-predicted slices that apply bidirectional prediction, etc.

[0086] For example, if the temporal identifier is 0 (meaning 1 picture) or if the slice type is an intra-predicted slice, the partitioning needs to be more accurate than for higher temporal IDs (inter-predicted pictures) or for P / B slices (inter-predicted slices).

[0087] The above parameters, time ID or slice type, are merely exemplary. Generally, additional and / or different coding parameters may be applied. Furthermore, instead of slice type, prediction modes of CUs in neighboring CTUs may be used to determine the partitioning of boundary portions based on CTUs (CTUs with incomplete boundaries).

[0088] On the other hand, signaling the partitioning may provide more freedom to choose an appropriate partition so that overall coding efficiency is increased. For example, the partitioning can be adjusted by optimizing a cost function (e.g., by RD optimization) and signaled in the bitstream to indicate how many levels of forced QT are used for the decoder (i.e., signaling whether QT is used at all or signaling N as described above).

[0089] Alternatively, boundary (incomplete) CTUs are first partitioned by RD optimization up to the penultimate hierarchical level (depth) or up to the signaled hierarchical level M. Then, the remaining deepest level blocks at the frame boundary use forced BT partitioning or a combination of forced QT and forced BT partitioning.

[0090] FIG. 9 shows three examples of at least partially enforced boundary demarcations.

[0091] When a CTU / CU is at the bottom boundary, horizontal forced BT partitioning may be used without iterative RD optimization (by splitting the vertical boundary) and without signaling a partition flag. The forced horizontal BT partitioning is terminated when the bottom-right sample of the leaf node is within the bottom boundary of the slice / picture. After the forced horizontal BT partitioning, the forcedly partitioned CU from the boundary CTU may be further split based on the RD cost at the encoder, and a further partition flag may be signaled from the encoder to the decoder. Part (a) of Figure 9 shows an example of a bottom boundary of 128 x 56 samples partitioned by forced horizontal BT (i.e., splitting the vertical boundary).

[0092] Similarly, for CTUs / CUs at the right boundary, vertical forced BT partitioning is used, possibly repeatedly, without checking the RD cost and without signaling a split flag until the bottom-right sample of the leaf node falls within the right boundary of the slice / picture. After forced vertical BT partitioning (for horizontal boundaries), the forcedly partitioned CUs from the boundary CTUs may be further split by RD optimization at the encoder, and further split flags are signaled from the encoder to the decoder. Part (b) of Figure 8 shows an example of a right boundary with a length of 56 x 128 samples partitioned by forced horizontal BT.

[0093] According to another example, for a CTU / CU at the bottom right corner boundary, first, a forced hierarchical QT partitioning is repeatedly used without any signaling. When the bottom right sample of the current CU is at the bottom or right boundary, further forced horizontal or vertical BT partitioning is repeatedly performed until the bottom right sample of the leaf node falls within the slice / picture boundary. Otherwise, when the bottom right sample of the current CU is further at the corner boundary, further forced QT partitioning is used until the bottom right sample of the leaf node falls within the slice / picture frame. The forcedly partitioned CU obtained by partitioning from the boundary CTU may be further partitioned by RD optimization in the encoder, and a further partition flag may be signaled from the encoder to the decoder. Part (c) of Figure 9 shows an example of a bottom right corner boundary of 56 x 56 samples partitioned by forced QT and BT.

[0094] In particular, part (c) of Figure 9 shows an incomplete corner boundary CTU of 56 x 56 samples. The first depth level of 64 x 64 samples after QT division is not within the boundary. The second depth level of 32 x 32 samples after QT is within the boundary, but leaves a horizontal corner (the lower left CU of 32 x 24 samples), a vertical corner (the upper right CU of 24 x 32 samples), and another corner (the lower right CU of 24 x 24 samples) to be further divided.

[0095] In this example, the horizontally bounding CUs and the vertically bounding CUs are further binarized so that the deepest level units have their longer sides aligned with the picture / slice boundaries. Corner parts may be further divided in the same way as described above for higher level corner parts, i.e., by quadtree partitioning.

[0096] In other words, during operation, the processing circuitry may partition the boundary portion horizontally and vertically by binary tree partitioning, especially when the boundary portion extends along both picture / slice boundaries, vertical boundaries and horizontal boundaries, and includes corner portions.

[0097] For the corner case, the processing circuitry operates by partitioning the corner portions of the boundary at each hierarchical level by quadtree decomposition. For example, in part (c) of Figure 9, for a corner boundary CU 960 with a size of 56 samples on both sides, four levels of forced QT are required (a QT partition of the 128x128 CTU resulting in four 64x64 CUs with the top-left CU containing a 56x56 corner; a QT partition of the top-left 64x64 CU resulting in four 32x32 CUs with the bottom-right CU containing a 24x24 corner; a QT partition of the bottom-right 32x32 CU resulting in four 16x16 CUs with the bottom-right of those 16x16 CUs containing an 8x8 corner; and a QT partition of four 8x8 CUs with the bottom-right 16x16 incomplete CU, i.e., the final step of QT partitioning into minimum-sized partitions where the top-left partition is within the picture while the other partitions are not).

[0098] On the other hand, for bottom border CU 950, two levels of forced QT followed by two levels of forced BT are required (QT split of the 128x128 CTU, QT split of the top-left 64x64 CU, bisecting of the 32x32 CU with its 32x24 border portion into two 32x16 CUs on the bottom border, and a final step of BT split of the bottom 32x16 CU into two 32x8 CUs with only the top CU within the picture border).

[0099] For corner CUs 960, the level N of the set QT is not taken into account, and corner CUs are forced to be split by QT at all hierarchical levels. In particular, for corners, if the bottom right sample of a leaf node (an incomplete CU, i.e., partially within the boundary area) is outside both the right and bottom boundaries, this current incomplete CU is recognized as bCorner ( FIG. 10 , step 1050, “Yes”). If bCorner applies to the current CU, the current CU is further split using the forced QT. Segmentation ends when the resulting leaf node is completely within the boundary area, or when a corner falls into the bottom or right boundary case (bBottom, bRight, i.e., the bottom right boundary of the current CU is outside the boundary area, i.e., only either the bottom or right boundary of the picture or slice).

[0100] In this example, when N=0, 1, or 2, CU 960 always requires a forced QT, and the forced QT splitting scheme is the same because CU 950 is split by QT at the first two levels anyway. This is because the depth 0 CU (CTU), the depth 1 CU (64 x 64 samples), is still a corner CU. Only after splitting the 64 x 64 CUs, at depth 2, there are four CUs: one that is not a boundary CU at all (top left), two that are bottom boundary CUs and right boundary CUs, and only one that is a corner CU.

[0101] If N=3, another forced QT is required, and then the forced splitting scheme is changed for CU 950 so that the bottom left 32 x 32 CU is further QT split instead of BT split. The splitting of CU 960 remains the same.

[0102] According to the present disclosure, a combination of forced QT and forced BT may be used, as already briefly discussed above. Particularly for some complex textures or motion on boundaries, detailed segmentation may be beneficial for image quality. Therefore, combined forced segmentation by QT and BT is also provided. In particular, in one implementation of a combination of forced QT and forced BT, forced BT segmentation is applied only after a certain number N of forced QT levels. Under other settings, if QT segmentation is allowed after BT segmentation, forced QT segmentation may occur after forced BT.

[0103] In the combined implementation of forced BT and forced QT, there are several ways to define the number of forced QT levels. When predefined forced QT levels are used, all boundary CTUs are first forced to be partitioned by the same QTBT structure, and further partitioning may be performed based on RD optimization. The right side of Figure 8 shows an example of partitioning one level of QT and three levels of BT for the lower boundary of 128 x 56 samples in an incomplete 128 x 128 CTU.

[0104] The level of enforced QT may be adaptively selected by encoder parameters such as the temporal ID or slice type. Specific slice types and lower temporal IDs may require more detailed segmentation. Furthermore, the level of enforced QT may be adjusted by rate-distortion optimization and signaled in the bitstream to indicate how many levels of enforced QT are used. Signaling the number of levels in the CTU level may be used to indicate how many levels of enforced QT were used for the CTU. Additionally, signaling the number of levels in the slice header may be used to indicate how many levels of enforced QT were used for the current slice. Signaling the number of levels in the SPS may be used to indicate how many levels of enforced QT were used for the coded video sequence. Signaling the number of levels in the PPS may be used to indicate how many levels of enforced QT were used for the coded picture.

[0105] The number of levels of enforced QT may be determined based on any other technique. For example, the entropy of the original samples within a slice / frame / border region may be used. The entropy is calculated by summing the term p_i * log2(p_i) over i, where p_i is the probability of the intensity value I of a sample within a particular region. The symbol "*" denotes multiplication and "log2" denotes the logarithm to the base 2. The higher the entropy, the more likely that structure is present within the slice / frame / border region, and therefore a smaller CU may be appropriate. However, these are merely examples, and in principle the present invention is not limited to any particular way in which the number of levels of enforced QT is determined. Any other measure, such as the variance of the presence of edges, may also be used.

[0106] In operation, the processing circuitry segments the boundary portion until the deepest level coding unit remains within the boundary portion.

[0107] According to the present disclosure, there is provided a method for dividing an image portion into coding units. The method includes subdividing the image portion into an integer number of coding tree units having a predetermined size and boundary portions having a size in at least a first direction, either horizontally or vertically, that is less than the predetermined size. A hierarchical partitioning of the boundary portions into coding units is then performed, including a deepest coding unit having different sizes in each horizontal and vertical direction, the deepest coding unit being smaller in the first direction.

[0108] In one implementation, the partitioning of the boundary portion further includes the steps of starting from the current partitioning level, which is the shallowest partitioning level; dividing the boundary portion into coding units of the current partitioning level in order, as long as the coding units of the current partitioning level do not reach the boundary portion; and as soon as the coding units of the current partitioning level do not fit into the boundary portion, increasing the partitioning level and further partitioning the coding units, wherein quadtree partitioning is used for the N shallowest partitioning levels, and binary tree partitioning is used in other cases.

[0109] Sequential partitioning means that the partitioning is performed from the shallowest level to the deepest level. In particular, a forced partitioning is performed at each hierarchical level for boundary coding units (boundary CUs), i.e., such CUs contain samples within (within the boundary part of) the picture / slice boundary and parts outside the picture / slice boundary. Here, the term CU refers to a CU that has a size resulting from the partitioning, but contains fewer samples than given by its size, i.e., is incomplete because part of it is outside the picture / slice into which it is partitioned.

[0110] Therefore, a forced split is performed to partition the boundary portion roughly from the largest CU to the smallest CU so that the entire boundary portion is partitioned. As already discussed above, no partitioning information is required for the forced split, and the partitioning information may be predefined. If there are several levels of QT to be forced, N may be signaled or predefined. A predefined N means that both the encoder and decoder have knowledge of such N, which may be defined by a standard, for example.

[0111] However, after performing the forced split, the CUs in the forced split boundary portion may be further partitioned. This may be performed based on rate-distortion optimization and therefore may be further signaled in the bitstream. In particular, for CUs in the boundary portion that are larger than the minimum CU size, one or more split flags may be signaled to indicate whether and how such CUs are further partitioned.

[0112] It is noted that the partitioning information included in the bitstream may include a QT split flag and / or a BT split flag. QT and BT splitting may also be mixed as long as which of the two types of splitting is used is signaled or derivable. At the decoder side, after the forced splitting, further splitting is then performed based on the decoded partitioning information such as qt_split_cu_flag and bt_split_cu_flag. Figure 10 shows a detailed exemplary implementation of boundary partitioning, and in particular, the implementation of forced BT and one combination of forced QT and forced BT. The variables employed in the figure have the following meanings: - bBottom indicates whether the bottom right sample of the current CTU or CU is outside the bottom boundary. - bRight indicates whether the bottom right sample of the current CTU or CU is outside the right boundary. - bCorner indicates whether the bottom right sample of the current CTU or CU is outside the bottom right corner boundary. - FQT_LV indicates the level of forced QT before forced BT (corresponding to N above). - uiDepth indicates the hierarchical level currently partitioned.

[0113] As mentioned above, the level of enforced QT can be predefined and adaptively selected based on encoder parameters such as slice type / temporal ID or based on RD cost.

[0114] In particular, in step 1010, partitioning of a boundary CTU (level 0) begins. If in step 1020 the CTU is a CTU on the bottom or right boundary (but not a corner boundary) and if uiDepth is less than or equal to FQT_LV-1, the CTU is partitioned in step 1040 by forced BT. Otherwise, forced QT partitioning step 1060 is applied. Partitioning continues by further partitioning boundary CUs. For example, in the example of FIG. 8(b), FQT_LV of the CTU (level 1) is equal to 1. Since uiDepth=0 equals FQT_LV-1=0 in step 1030, the incomplete CTU is forced to be partitioned by QT in step 1060, resulting in two CUs (top-left and top-right) that need to be further partitioned. Each of these CUs is further partitioned iteratively. The top-left CU (level 1) is still incomplete and therefore a boundary CU. As the bottom boundary CU in step 1020, since uiDepth=1 is greater than FQT_LV-1=0 in step 1030, a mandatory BT is applied to the top-left CU in step 1040. The top-right CU is partitioned in the same way. In step 1040, the boundary (incomplete) CU (64 x 24) is further partitioned at level 2 by BT. The remaining 64 x 8 CU in level 3 reaches the picture boundary, thus resulting in "No" results in steps 1020 and 1050. The size 8 partition as the smallest partition is not further partitioned in step 1070; instead, it results in "yes" result in step 1080, which stops the partitioning in step 1090. The second 64 x 24 CU is partitioned in the same way.

[0115] On the other hand, the 56 x 56 CTU in part (c) of Figure 9 is a corner CTU. In step 1020 (level 0), the CTU proceeds to step 1050 and is partitioned by QT as a corner CU in step 1060. In level 1, the top-left 56 x 56 CU is treated in the same way, resulting in three boundary CUs (bottom-left, top-right, and corner). Assuming FQT_LV = 0, i.e., no forced QT split for the bBottom and bRight CUs, the bottom-left and top-right CUs are split by forced BT in step 1040 (after "yes" in step 1020 and "no" in step 1030). The corner 24x24 CU (level 2) passes from step 1020 to step 1050 and is partitioned by QT in step 1060, again yielding three boundary CUs with sizes 16x8, 8x8, and 8x16, which do not need to be further divided, passing "no" to step 1020, "no" to step 1050, step 1070, to stop condition 1080 and end 1090. The stop condition may be, for example, a minimum size equal to 8 in this example.

[0116] Below, Table 2 shows an example syntax that allows signaling the QT partition flag for normal (non-border) CUs. In addition, the BT partition flag may be signaled for CUs other than border CUs. The coding_quadtree syntax is iterative, i.e., partitioning is hierarchical. Partitioning is performed if the following conditions are true: x0 + cuWidth <= pic_width_in_luma_samples and y0 + cuHeight <= pic_height_in_luma_samples

[0117] In other words, as long as the bottom right corner of the current CU (i.e., the CU currently being processed by the partitioning) is within the picture, the CTU / CU is partitioned by a quadtree or a binary tree. It is noted that this syntax is merely exemplary, and the present disclosure may also be applied to other cases, such as using only QT, or only BT, or selectable further partitioning types for CTUs that are completely within the picture / slice. The values ​​cuMinBTSiz, cuMaxBTSize, and cuMaxBTDepth adopted in the syntax are predefined values.

[0118] In the above condition, x0 and y0 specify the location of the top-left corner sample of the current CU, cuWidth and cuHeight specify the horizontal and vertical CU size, and the parameters pic_width_in_luma_samples and pic_width_in_luma_samples specify the horizontal and vertical picture size. Note that in this example, the entire picture is partitioned. However, the same approach is applicable to parts of a picture that can be decoded independently, such as slices, tiles, or any other part of a picture that has a boundary.

[0119] Similar to the HEVC standard, when qt_split_cu_flag[x0][y0] is not present (boundary case), the following applies (at the encoder and decoder): - If cuWidth (CU width of the current level) is greater than MinQTSize (minimum QT size, here 8) and cuHeight (CU height of the current level) is greater than MinQTSize, the value of qt_split_cu_flag[x0][y0] is inferred to be equal to 1. In other words, the border is forced to be split. Otherwise (log2CbSize is equal to MinCbLog2SizeY), the value of qt_split_cu_flag[x0][y0] is inferred to be equal to 0. In other words, if the current CU has the smallest allowed size MinCbLog2SizeY, the decoder assumes that the CU will not be split any further.

[0120] This is similar to the case in Table 1 above.

[0121] [Table 2A] [Table 2B]

[0122] As can be seen, bt_split_cu_flag may be signaled. This flag is signaled when BT splitting of a CTU is configured for a non-boundary CTU / CU (see condition if(! (bBBoundary || bRBoundary))). In the boundary case, a forced split does not require an additional flag since the value of the flag is derived. In particular, if bt_split_cu_flag[x0][y0] is not present (boundary case), the following applies: - If bBBoundary is true (meaning the current CU / CTU is at the bottom boundary) and at the same time cuWidth is greater than MinBTSize or cuHeight is greater than MinBTSize (the current CU / CTU has no minimum size and therefore could in principle be further split), the value of bt_split_cu_flag[x0][y0] is inferred to be equal to 1 (meaning a further bisection is performed, i.e. a binary tree split that would result in the longer side of the split CU being parallel / overlapping the bottom, i.e. horizontal boundary). - bRBoudary is true (meaning the current CU / CTU is on the right boundary) while cuWidth is greater than MinBTSize or cuHeight is greater than MinBTSize and the value of bt_split_cu_flag[x0][y0] is inferred to be equal to 2 (meaning a further binary split will be performed, i.e. a binary tree split will be performed that will result in the longer side of the split CU being parallel / overlapping with the right, i.e. vertical boundary). - Otherwise, the value of bt_split_cu_flag[x0][y0] is inferred to be equal to 0, meaning that no further binary splitting is performed.

[0123] Being on the boundary means that there are no CUs between the current CU and the boundary that are at or below the hierarchical level of the current partition, in other words, the current CU belongs to the boundary part at the current hierarchical level (corresponding to bBBoundary being true or bRBoundary being true). If the current CU is not a boundary CU, BT partitioning is used if QT is not forced (cqtDepth > FQT_LV-1) and the BT flag is included in the bitstream.

[0124] Example values ​​for a JEM implementation are cuMinBTSize for Inter frames having a value of 4, cuMAXBTSize for Inter frames being 128, and cuMAXBTDepth being 3. However, this disclosure is not limited to these sizes and is applicable to any selected values.

[0125] It is noted that the above syntax is only exemplary and assumes knowledge of the variable FQT_LV, which indicates the number of forced QT division levels. If the value of FQT_LV is zero, then there are no forced QT levels and the entire division of the forced boundaries is performed by binary tree division. The syntax and semantics may be implemented differently without departing from this disclosure.

[0126] Another solution based on JEM software (JEM5 and 6) could be to specify the following semantics: When qt_split_cu_flag[x0][y0] is not present (boundary case), the value of qt_split_cu_flag[x0][y0] is inferred to be equal to 1, which means further QT split in case of corner CUs or forced QT (if cqtDepth <= FQT_LV-1).

[0127] If bt_split_cu_flag[x0][y0] is not present (boundary case), the following applies: - If bBBoundary is true, the value of bt_split_cu_flag[x0][y0] is inferred to be equal to 1, meaning that the bottom boundary CU is split vertically by a bisection, i.e., to form two horizontal CUs (i.e., CUs that are longer horizontally than vertically). - If bRBoundary is true, the value of bt_split_cu_flag[x0][y0] is inferred to be equal to 2, meaning that the right-bounding CU is split horizontally by a bisection, i.e., to form two vertical CUs (i.e., CUs that are longer vertically than horizontally).

[0128] The above example shows a forced split using a parameter that defines the number of forced QT levels before BT splitting is applied. Semantic rules define the split that is met when no split flag is present, i.e., at the boundary. However, this disclosure is not limited to such an approach.

[0129] Alternatively, first, the (incomplete) CTU at the boundary can be partitioned by RD optimization, and then the remaining blocks at the frame boundary can use forced BT partitioning or a combination of forced QT and forced BT partitioning. Partitioning by RB optimization implies that a split flag is signaled in the bitstream for the corresponding CU. Partitioning by RD may be specified as QT (i.e., in the standard or by signaling common to a sequence of pictures or pictures) or BT, so that the choice between BT and QT does not have to be signaled for each CU. Meanwhile, the split flag may be extended to indicate not only whether a given CU is split or not, but also whether the CU is split by QT or BT.

[0130] At the decoder side, forced partitioning is performed on boundary CTUs / CUs. For forced BT partitioning, forced horizontal BT (vertical division to form horizontal partitions) is used for CTUs / CUs at the bottom boundary until the bottom right sample of the leaf node falls within the bottom boundary of the slice / picture. Forced vertical BT is used for CTUs / CUs at the right boundary until the bottom right sample of the leaf node falls within the right boundary of the slice / picture. The bottom right corner CTUs / CUs are partitioned using forced QT until the bottom right sample of the leaf node falls within the slice / picture boundary or the leaf node falls in the right / bottom boundary case. As already described above with reference to the syntax, further partitioning is performed based on the decoded partition flag, which may be a QT or BT partition flag depending on whether there are any levels of forced QT partitioning defined. The syntax elements are generated by the encoder and processed by the decoder as described above.

[0131] For a combination of forced QT and forced BT, the number of forced QT levels for a boundary CTU (corresponding to the parameter FQT_LV) may be adaptively selected based on the temporal ID / slice type or decoded from the bitstream. The decoded forced QT level may be extracted from the slice header, SPS, or PPS. A boundary CTU / CU undergoes forced BT partitioning only after forced QT partitioning. The forced partitioning is terminated when the bottom-right sample of the leaf node is within the boundary. Further partitioning is performed based on the decoded partition flag.

[0132] In other words, according to the first implementation, in the decoder, when a CTU is on a slice / picture boundary, binary tree partitioning is performed without a binary tree partitioning indicator (flag) and / or a binary tree partitioning type indicator (indication of whether BT or QT should be performed). The binary partitioning is terminated when the bottom-right sample of the leaf node is within the slice / picture boundary (i.e., does not cross the picture boundary).

[0133] Therefore, instead of forced QT boundary partitioning, forced BT for boundary CTUs is introduced. A forced horizontal BT for the bottom border and a forced vertical BT for the right border are implemented. For the bottom right corner border, forced QT is used until the current CU no longer needs to be forced partitioned (because the minimum size has been reached) or until the current CU is at the bottom or right border.

[0134] In another implementation of the present disclosure, a combination of forced QT and forced BT partitioning is also possible. Therefore, forced BT partitioning is performed only after a certain number of forced QT levels. For the combination of forced QT and forced BT partitioning, the level of forced QT may be adaptively selected according to the time ID or slice type. Alternatively, the level of forced QT may be adjusted according to cost and included in the bitstream to indicate how many levels of forced QT are used for the decoder.

[0135] First, even more flexibility is achieved when the boundary CTUs are partitioned by RD optimization and only the remaining blocks at the frame boundary (i.e., incomplete CUs that are partly in the picture and partly outside the picture) can use the proposed forced BT partitioning or a combination of forced QT and forced BT partitioning as described above.

[0136] The partitioning described so far may be employed in still image or video coding, in particular in encoders and decoders, shown in Figures 1 and 2 as partitioning units 110 and 250, respectively.

[0137] Therefore, the present disclosure also provides an apparatus for encoding images of a video sequence, the apparatus comprising: an apparatus (division unit) 110 for dividing image portions according to any of the above examples into coding units; image coding units 120-145, 160-195 configured to encode the coding units; and a bitstream formation unit 150 configured to generate a bitstream including the coded coding units and partitioning information indicating how the coding tree units are partitioned.

[0138] The encoder may further include a rate-distortion optimization unit (not shown in FIG. 1 ) configured to determine hierarchical partitioning of boundary portions by rate-distortion optimization for any level of partitioning except the deepest level. The RD optimization unit may perform encoding using different encoder settings related to the partitioning and select the setting that results in the smallest cost function. The cost function may be the smallest rate for a given level of distortion or similarity between the prediction and the original block to be coded, etc.

[0139] Further provided is an apparatus for decoding images of a video sequence, the apparatus comprising: an apparatus 250 for determining a division into coding units of an image portion to be decoded as described above; a bitstream parser 250 for parsing the coded coding units and a bitstream indicating how the coding tree units are divided and containing division information based on the determined division of the image portions; and image decoding units 225-240, 260-295 for decoding the coded coding units.

[0140] It is noted that the splitting and parsing are performed iteratively together, with the parsing including separating necessary control information from the bitstream for the current CU, and the splitting using the parsed information and / or semantic knowledge to derive the location, size, and data of the CU. In particular, the parser may send a split flag based on which the splitter either further splits the current CU or maintains the size of the CU. Depending on the split flag, the parser also either further parses the split information for CUs at further depths, or parses the split information for other CUs at the same depth, or parses the decoded data for the CU. Furthermore, the splitter is configured to partition the boundary portions as described above according to mandatory partitioning rules, even without further bitstream signaling.

[0141] In other words, both the encoder and / or decoder may adopt the division of the boundary portion into coding units as a predefined division. Then, the partitioning information in the bitstream is not related to the division of the boundary portion at all. This is the case when the boundary is divided in a predefined manner, which may further include deriving the division based on other encoder settings such as picture type / temporal ID / slice type.

[0142] Alternatively, the encoder and / or decoder may be further configured to partition the boundary portion by quadtree partitioning at the N shallowest levels, where N is an integer greater than or equal to zero and less than the number of hierarchical levels of partitioning, and partition the boundary portion by binary tree partitioning at the remaining hierarchical levels of partitioning, and the partitioning information may include an indication of N. The partitioning information may further include a partition flag to indicate whether and / or with what type of partitioning the CTU / CU in the boundary portion should be partitioned (which may include binary tree or quad tree and / or other options). The partition flag may be signaled for all partitions (CUs) or only for non-boundary partitions at those hierarchical levels.

[0143] The encoder and / or decoder may be implemented (embedded) on a chip or by software, but may also be implemented in multiple hardware and software components.

[0144] In the above, mainly two types of partitioning have been considered: quad-tree partitioning and binary-tree partitioning. However, the cost-based or pre-defined boundary constrained partitioning approach provided by the present disclosure is suitable not only for constrained QT or constrained BT, but also for other constrained partitioning structures.

[0145] The above mainly describes the case where the deepest CU in the forced division of the boundary portion is a rectangular and not a square CU, i.e., a CU with one side longer than the other.

[0146] In general, however, a boundary portion may be partitioned by binary tree partitioning at least at one level of partitioning where boundary coding units do not remain entirely within the boundary portion. In other words, according to the present disclosure, an apparatus for dividing an image portion into coding units may include processing circuitry that, during operation, subdivides the image portion into an integer number of coding tree units having a predetermined size and a boundary portion having a size less than the predetermined size in at least a first direction, either horizontally or vertically, and hierarchically partitions the boundary portion into coding units including coding units having different sizes in each horizontal and vertical direction, where the coding units are partially within the boundary portion and partially outside the image portion. In particular, binary tree partitioning may be used for any boundary CU.

[0147] It is noted that the boundary portion shown in FIG. 7 includes several CTUs. The above exemplary embodiments and implementations have been described with respect to partitioning one CTU. This is because it may be advantageous and lead to better results in terms of coding efficiency and / or complexity even if only one of the (incomplete) CTUs in the boundary portion is partitioned as described above. However, two or more CTUs of the boundary portion may be partitioned as described above. The partitioning may be performed sequentially, one CTU at a time. In some implementations, it may be beneficial to perform partitioning in parallel for different CTUs. Parallel processing, when employed by the encoder / decoder, respectively, may reduce encoding / decoding delay.

[0148] In other words, it may be advantageous to apply the above-described division to all CTUs in the boundary portion. While the present disclosure is not so limited, the division may be applied to one or more boundary CTUs. The remaining CTUs may be divided by any other division, such as QT only.

[0149] Corresponding methods for partitioning are also provided. Corresponding encoding and decoding apparatus and methods employing such partitioning are further provided.

[0150] The present disclosure may be implemented in an apparatus. Such an apparatus may be a combination of software and hardware. For example, the intra-prediction and deblocking filters may be implemented by a chip such as a general-purpose processor, a digital signal processor (DSP), or a field programmable gate array (FPGA). However, the present invention is not limited to implementation on programmable hardware. It may be implemented in an application-specific integrated circuit (ASIC) or by a combination of the above hardware components.

[0151] The determination of the intra-prediction and deblocking filters may also be implemented by program instructions stored in a computer-readable medium, which, when executed by a computer, performs the steps of the method as described above. The computer-readable medium can be any medium on which a program is stored, such as a DVD, a CD, a USB (flash) drive, a hard disk, or a server storage available over a network.

[0152] The encoder and / or decoder may be implemented in a variety of devices including a television set, a set-top box, a PC, a tablet, a smartphone, etc. The encoder and / or decoder may be software, an app that implements the steps of the method.

[0153] In summary, this disclosure relates to partitioning an image or image slice. In particular, the image or image slice is partitioned both vertically and horizontally into an integer number of coding tree units. The deepest partitioning level is then the result of a bipartite partition, meaning that the remaining boundary portion is at least partially forced to be partitioned so that one side contains longer coding units than the other side. The longer side is parallel to or overlaps the boundary of the image or image slice. [Explanation of symbols]

[0154] 100 Encoder 110 Split Unit 120 Image Encoding Unit 125 Reconstructor 130 Conversion Unit 135 Reverse conversion unit 140 quantization units 145 Inverse Quantization Unit 150 entropy coding units 160 Loop Filtering Unit 170 Frame Buffer 180 Mode Selection Unit 190 intra prediction units 195 Inter Prediction Units 200 Video Decoder 225 Reconstructor 230 Reverse conversion unit 240 Inverse Quantization Unit 250 Bitstream Analysis, Entropy Decoding, and Splitting Unit 260 Loop Filtering Unit 270 Reference Picture Buffer 290 Intra Prediction Units 295 Image Decoding Unit 600 equipment 610 Functional Unit 620 Functional Units 630 Functional Units 700 bottom picture border 710 Boundary part 720 parts Vertical size of 770V CTU Horizontal size of 770H CTU 950 Lower Boundary CU 960 CU of the corner boundary

Claims

1. 1. A method for transmitting a bitstream containing encoded coding units of pictures of a video, comprising: subdividing the picture into an integer number of coding tree units and boundary coding tree units at picture boundaries; partitioning the boundary coding tree units into coding units using a quadtree partition and a forced binary tree partition for the boundary coding units resulting from the quadtree partition; encoding the coding unit based on intra prediction or inter prediction; generating a bitstream including the encoded coding units; transmitting the bitstream including the encoded coding unit; wherein the forced binary tree partitioning is a predefined partitioning, and no partitioning flag is signaled in the bitstream to indicate the partitioning of the forced boundary coding unit.

2. The method of claim 1 , wherein the forced binary tree partitioning partitions the boundary coding unit into two partitions of equal size.

3. 3. The method of claim 1, further comprising: partitioning the boundary coding units resulting from the quadtree partitioning by forced binary tree partitioning until the deepest level coding unit remains entirely within the picture.

4. 4. The method according to claim 1, comprising a step of partitioning the boundary coding units resulting from the quadtree partitioning by a forced binary tree partitioning horizontally if the boundary coding unit is on the bottom boundary of the picture, or vertically if the boundary coding unit is on the right boundary of the picture.

5. A computer readable medium storing instructions that, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 4.

Images