Apparatus and method for encoding and decoding images using image boundary processing

Abstract

This invention provides an encoding method and apparatus for images by recursively dividing them into leaf blocks using a multi-tree partitioning method. [Solution] The method includes the steps of: determining whether the sum of the x-coordinate position and width of a given block is greater than the picture width, and determining whether the sum of the y-coordinate position and height of a given block is greater than the picture height, and determining a set of available split modes for a given block that extends beyond the boundary of an image; and selecting from the set of available split modes at least one of four divisions or two divisions in a vertical direction or two divisions in a horizontal direction, which divides the given block into two subblocks along a split line that is parallel to the boundary of the image within the given block and at least one of which has a split line that is juxtaposed with the boundary of the image and is centered inside the given block.

Description

[Technical Field]

[0001] The present invention relates to the encoding of images and videos. Embodiments of the present invention relate to specific methods for dividing input image and video data of a particular size into smaller elements (entities). In particular, embodiments of the present invention relate to image boundary processing in recursive image signal partitioning. [Background technology]

[0002] In modern video coding applications, an input video signal of a given size is divided into smaller chunks.[1] This division consists of a number of structures, each with specific information and description associated with it. In the state-of-the-art video codec H.265 / HEVC[1], the most important subdivision is into macroblocks. These macroblocks, or coding tree units (CTUs), are secondary structures of a predefined size (such as 32x32 or 64x64 luminance samples) that span a fixed grid on the encoded image. All other subdivision levels are defined in units of this fixed subdivision. For example, coarser, higher-level subdivisions into slices and tiles are defined in units of the CTUs they contain.

[0003] In H.265 / HEVC[1], starting from each CTU, the quad-tree structure is signaled to provide a means for recursively dividing a fixed CTU into flexible substructures. At each level, within a defined parameter range, a block is inferred or signaled that needs to be divided into four subblocks or encoded at a specified level. This process is repeated recursively until further division is impossible as defined by the current high-level parameters, or a flag is read indicating that further division is not needed at the current level. In this case, the signals of the encoding unit and its substructures are read.

[0004] The signals for quad-tree splitting may sometimes be omitted and inferred as either true or false. Most notably, due to their fixed nature, CTUs are not always aligned with the video image boundary. In this case, if the CTU or subblock of the quad-tree splitting structure is not entirely contained within the image boundary, the splitting flag is inferred as positive without explicit signaling. If, after splitting, the subblock lies entirely outside the video image boundary, no further signaling for this block is read.

[0005] In the development of future video standards with capabilities beyond H.265 / HEVC[1], the quad-tree recursive partitioning structure will be extended by various partitioning configurations[2]. If a block is not partitioned into four secondary subparts, a 2-part partition is signaled to indicate that this block should be partitioned into two rectangular blocks using a 1 / 2 partitioning ratio. This signal also contains information on whether the partition should be applied horizontally or vertically. Such rectangular blocks can be further recursively partitioned into smaller secondary or rectangular blocks using binary tree partitioning syntax. This extended partitioning tree is called QTBT[2]. QTBT handles image boundaries using an implicit 4-part partition, just as in the H.265 / HEVC standard[1]. In QTBT, if a block crosses an image boundary, a 4-part partition is inferred and no signal is read.

[0006] Patent Document 5 describes a new splitting method called Generalized Binary Splitting (GBS), which may or may not include quadruple splitting, and may be used in conjunction with quadruple splitting. In this method, quadruple splitting can be signaled to perform splitting at a splitting ratio other than 1 / 2 (in QTBT, quadruple splitting performs splitting at the center of the current block). This splitting may also be signaled using syntax related to previous splitting (e.g., vertical and parallel instead of horizontal and vertical).

[0007] However, conventional partitioning methods are quite strict in handling image boundaries. As explained above, conventional partitioning methods can use an implicit 4-part partition at the image boundary. In other words, the conventional technique uses a completely implicit partitioning derivation at the image boundary. Such implicit signal processing can sometimes enable good signal processing efficiency. However, this can result in low flexibility. [Overview of the Initiative] [Problems that the invention aims to solve]

[0008] Therefore, an object of the present invention is to extend existing division methods to make image boundary processing more flexible while simultaneously reducing the bit allocation for signal processing in image boundary processing. [Means for solving the problem]

[0009] The present invention provides a solution by providing an image encoding apparatus according to claim 1, an image decoding apparatus according to claim 10, an image encoding method according to claim 19, an image decoding method according to claim 20, and a computer program according to claim 21.

[0010] The first aspect relates to an image encoding apparatus. The apparatus is configured to divide an image into leaf blocks using a recursive multitree partitioning. That is, the apparatus is configured to divide an image into smaller chunks using the recursive multitree partitioning. These smaller chunks may be blocks of a specific size. By performing multitree steps, the partitioning of an image may begin with a first-level tree root block and end with a leaf block, which is the last block of the multitree and therefore the smallest element of the partition. Between the tree root block and the partitioned leaf blocks, the apparatus can perform stepwise from one partitioning tree level to one or more subsequent partitioning tree levels, at each tree level, the current block of a particular tree level is partitioned into two or more smaller blocks. For example, in HEVC, the tree root block is referred to as a so-called macroblock or coding tree unit (CTU), and the leaf blocks are referred to as so-called subblocks or coding units (CU). Thus, a CTU can be partitioned into one or more CUs, and any block at a tree level between the tree root block and the leaf blocks can be called a leaf block or subblock. Furthermore, in this disclosure, a tree root block, or a leaf block, or any block at the tree level between a tree root block and a leaf block may be referred to as a given block. Thus, a given block may correspond to a given tree level in a multi-tree partition. The device may utilize a block-based coding system; that is, the device may be configured to encode an image into a data stream in a block-based manner by using the partitioning of the image into leaf blocks. Partitioning is sometimes referred to as splitting. To partition a tree root block into smaller leaf blocks, a specific partitioning procedure may be used to divide the block into smaller subblocks. These partitioning procedures may be referred to as partitioning modes, and the partitioning mode may differ for each tree level. Furthermore, a given block may include a given size.When a given block is arranged as a grid on an image, due to its size, some of the given block may extend beyond the image boundary. For example, a first portion of a given block may be located inside the image, while a second portion of the given block may be located outside the image. Thus, the image boundary may intersect with the given block. In general, the image boundary may intersect with a given block at different locations. For example, if a corner of the image is included in a given block, the image boundary may intersect with the given block horizontally, vertically, or both. To handle these situations, the present invention provides the following solution: While dividing the image into leaf blocks, the device is configured to divide a given block corresponding to a given tree level of a multi-tree division and extending beyond the image boundary by using a reduced set of division modes compared to the fully implicit division derivation used by the prior art. According to the present invention, the device is configured to reduce the set of available division modes for dividing a given block depending on the above-mentioned locations where the image boundary intersects with the given block. Thus, the device obtains a reduced set of one or more division modes. The reduced set may include a cardinality indicating the number of division modes available in the reduced set of division modes. Therefore, if the cardinality of the reduced set is 1, i.e., the reduced set contains only one division mode, the device is configured to apply the division mode of this reduced set to divide a given block. If the cardinality of the reduced set is greater than 1, i.e., the reduced set contains two or more different division modes, the device is configured to select one of these division modes of the reduced set and apply the selected division mode to divide a given block, and the device signals each selection in the data stream. In other words, the device of the present invention can pre-select a reduced set of division modes for dividing a given block at present. This pre-selection may depend on the current position of the block relative to the image boundary.As a result, the device of the present invention may only need to select a suitable partition mode from a pre-selected reduced set of partition modes. This reduces the bit allocation required to signal the selected partition mode in the data stream. This is because there is no need to transmit information about the partition itself, and if the reduced set of partition modes includes two or more partition modes, for example, in one bin, only the remaining uncertainty needs to be signaled.

[0011] The second aspect relates to an image decoding apparatus. The apparatus is configured to divide an image into leaf blocks using a recursive multitree partitioning. The apparatus is further configured to decode an image from a data stream in a block-based manner using the division of the image into leaf blocks. When dividing an image into leaf blocks, the apparatus is further configured to reduce the set of available partitioning modes for a given block, depending on where the boundary intersects the given block, corresponding to a given tree level of the multitree partitioning and for a given block that extends beyond the boundary of the image, such that the boundary obtains a reduced set of one or more partitioning modes. If the cardinality of the reduced set is 1, the apparatus is configured to apply the partitioning modes of the reduced set for partitioning the given block; if the cardinality of the reduced set is greater than 1, the apparatus is configured to select one of the partitioning modes of the reduced set and apply the selected one for partitioning the given block according to the signal transmission in the data stream. For the advantages of the image decoding apparatus, see the section above that describes the advantages of an image encoding apparatus.

[0012] A third aspect relates to a method for encoding an image, the method comprising the step of dividing an image into leaf blocks using a recursive multitree partition. The method further comprises the step of encoding an image into a data stream in a block-based manner using the division of the image into leaf blocks. When dividing an image into leaf blocks, the method further comprises the step of reducing the set of available partition modes for partitioning a given block, depending on the position where the boundary intersects the given block, such that a reduced set of one or more partition modes is obtained, for a given block corresponding to a given tree level of the multitree partition and extending beyond the boundary of the image. If the cardinality of the reduced set is 1, the method comprises the step of applying a partition mode from the reduced set to partition the given block; if the cardinality of the reduced set is greater than 1, the method comprises the steps of selecting one of the partition modes from the reduced set, applying one of the selected partition modes to partition the given block, and signaling the selection in the data stream. For the advantages of the method for encoding an image, see the above section describing the advantages of the device for encoding an image.

[0013] A fourth aspect relates to a method of decoding an image, the method including the step of dividing the image into leaf blocks using recursive multi-tree partitioning. The method further includes the step of block-based decoding of the image from a data stream using the division of the image into leaf blocks. When dividing the image into leaf blocks, for a predetermined tree level of the multi-tree partitioning and for a predetermined block that extends beyond the boundaries of the image, the method further includes the step of reducing the set of available partitioning modes for dividing the predetermined block according to the position where the boundary intersects the predetermined block so as to obtain a reduced set of one or more partitioning modes. If the cardinality of the reduced set is 1, the method further includes the step of applying the reduced set of partitioning modes for dividing the predetermined block, and if the cardinality of the reduced set is greater than 1, the method further includes the step of selecting one of the partitioning modes of the reduced set and the step of applying the selected one of the partitioning modes for dividing the predetermined block according to the signaling in the data stream. Regarding the advantages of the method of decoding the image, reference may be made to the above section that describes the advantages of the apparatus for encoding the image.

[0014] According to a fifth aspect, a computer program is provided, each of the computer programs being configured to implement the above method when executed on a computer or a signal processor, and as a result, the above method is implemented by one of the computer programs.

[0015] Advantageous aspects of the present invention are the subject matter of the independent claims. Preferred embodiments of the present application defined in the dependent claims are exemplarily described below with reference to the figures.

Brief Description of the Drawings

[0016] [Figure 1] A block diagram of an apparatus for predictive coding of an image is shown as an example of an encoder capable of implementing an intra prediction concept according to an embodiment of the present application. [Figure 2]As an example of a decoder that can implement the intra-predictive concept according to the embodiments of this application, a block diagram of an image predictively decoding device that fits the device in Figure 1 is shown. [Figure 3] A schematic diagram is shown illustrating an example of the relationship between the predicted residual signal, the predicted signal, and the reconstructed signal, demonstrating that coding mode selection, transformation selection, and transformation performance subdivision can be configured, respectively. [Figure 4A] This document illustrates an embodiment of an encoder based on this concept. [Figure 4B] This document illustrates an embodiment of a decoder based on this concept. [Figure 5A] This shows a completely unrestricted decision tree that includes the full set of available splitting modes. [Figure 5B] This shows a restricted decision tree that includes a reduced set of partitioning modes based on this concept. [Figure 5C] This shows an even more completely unrestricted decision tree, including the full set of available split modes. [Figure 5D] This shows a restricted decision tree that includes a reduced set of partitioning modes based on this concept. [Figure 6] This diagram illustrates various cases where blocks extend beyond the image boundary. [Figure 7A] An example of the 4-way split mode is shown. [Figure 7B] An example of a vertical two-part split mode is shown. [Figure 7C] An example of a horizontal two-part split mode is shown. [Figure 7D] An example of a vertical three-part division mode is shown. [Figure 7E] An example of a horizontal three-part division mode is shown. [Figure 8A] This example demonstrates how to divide a macroblock into leaf blocks using only four divisions. [Figure 8B] This example demonstrates how to divide a macroblock into leaf blocks using a selected quad split, vertical quad split, and horizontal quad split from a reduced set of split modes. [Figure 9]A block diagram of the method for encoding images using this concept is shown. [Figure 10] A block diagram of the image decoding method based on this concept is shown. [Modes for carrying out the invention]

[0017] In the following description, one or more elements that have equal or equivalent functions are indicated by equal or equivalent reference numbers.

[0018] The following description of the figures may only illustrate some of the many illustrative and non-limiting examples and embodiments of the concepts described herein. A CTU may be described as a non-limiting example of a macroblock, while a CU may be described as a non-limiting example of a subblock. Furthermore, an image (e.g., rectangular) may consist of, for example, a large number of tiles, each representing the image itself. In HEVC[1], such tiling is used and processed by a high-level concept of image tiles, for example, for omnidirectional video. Such tiling is available at CTU resolution. If such limitations are relaxed, an "encoded image" may consist of a large number of tiles, each having its own boundary and its own content. In that sense, an encoded image may contain more image boundaries, and each tile may constitute its own "image data" with its own "image data" boundary, and the same rules described herein may apply. Thus, where the term "image boundary" is used herein, this may also include, for example, "current image data boundaries" belonging to one or more tiles.

[0019] The following description of the figures begins with a presentation of a block-based predictive coding encoder and decoder for encoding video images, in order to form an example of a coding framework into which embodiments can be incorporated. The encoder and decoder are described in Figures 1 to 3. The following description of embodiments of the concept of the present application is presented with a description of how such concepts can be incorporated into the encoder and decoder of Figures 1 and 2, respectively, but it is also possible to form encoders and decoders that do not operate according to the underlying coding framework of the encoder and decoder of Figures 1 and 2 using the embodiments described in subsequent Figure 4 and below.

[0020] Figure 1 shows an example of a device that predictively encodes image 12 into a data stream 14 using transform-based residual coding. The encoding device, or encoder, is denoted by reference numeral 10. Figure 2 shows a corresponding decoding device, or decoder 20, which is also configured to predictively decode image 12' from the data stream 14 using transform-based residual decoding, where an apostrophe is used to indicate that image 12' reconstructed by decoder 20 may deviate from image 12 initially encoded by device 10 in terms of coding loss resulting from quantization of the predictive residual signal. While Figures 1 and 2 exemplify transform-based predictive residual coding, embodiments of the present application are not limited to this type of predictive residual coding. This also applies to other details described with respect to Figures 1 and 2, as outlined below.

[0021] The encoder 10 may be configured to apply a spatial-spectral transform to the predicted residual signal and encode the resulting predicted residual signal into a data stream 14. Similarly, the decoder 20 may be configured to decode the predicted residual signal from the data stream 14 and apply a spectral-spatial transform to the resulting predicted residual signal.

[0022] Internally, the encoder 10 may include a predictive residual signal generator 22 that generates a predictive residual 24 for measuring the deviation between the original signal, i.e., the signal from image 12, and the predicted signal 26. The predictive residual signal generator 22 may be, for example, a subtractor that subtracts the predicted signal from the original signal, i.e., the signal from image 12. The encoder 10 further includes a converter 28 that performs a spatial-spectral transform on the predictive residual signal 24 to obtain a spectral domain predictive residual signal 24', which is quantized by a quantizer 32 included in the encoder 10. The thus quantized predictive residual signal 24'' is encoded into a bitstream 14. For this purpose, the encoder 10 may optionally include an entropy encoder 34 that entropically encodes the predictive residual signal that is transformed and quantized into a datastream 14. The predictive residual 26 is generated by the prediction stage 36 of the encoder 10 based on the predictive residual signal 24'' that is encoded into the datastream 14 and decodeable from the datastream 14. For this purpose, the prediction stage 36 may internally include, as shown in Figure 1, an inverse quantizer 38 that inversely quantizes the predicted residual signal 24'' to obtain a spectral domain predicted residual signal 24'''' corresponding to signal 24', excluding quantization losses, and then an inverse transformer 40 that applies an inverse transform, i.e., spectral-spatial transform, to the predicted residual signal 24'''' to obtain a predicted residual signal 24'''' corresponding to the original predicted residual signal 24, excluding quantization losses. Next, the combiner 42 of the prediction stage 36 adds the prediction signal 26 and the predicted residual signal 24'''' to resynthesize them to obtain a reconstructed signal 46, i.e., a reconstruction of the original signal 12. The reconstructed signal 46 may correspond to signal 12'. Next, the prediction module 44 of the prediction stage 36 generates a predicted signal 26 based on signal 46, for example by using spatial prediction, i.e., intra-prediction, and / or temporal prediction, i.e., inter-prediction.

[0023] Similarly, as shown in Figure 2, the decoder 20 may be internally composed of components corresponding to the prediction stage 36 and interconnected in a manner corresponding to the prediction stage 36. In particular, the entropy decoder 50 of the decoder 20 can entropy decode the quantized spectral domain prediction residual signal 24'' from the data stream, and then the inverse quantizer 52, inverse converter 54, combiner 56, and prediction module 58 interconnect and cooperate in the manner described above with respect to the module of the prediction stage 36, and output a reconstructed signal based on the prediction residual signal 24'' such that the output of the combiner 56 yields a reconstructed signal, i.e., image 12', as shown in Figure 2.

[0024] Although not specifically described above, it is readily apparent that encoder 10 can set several coding parameters, including, for example, prediction mode and motion parameters, according to several optimization schemes, such as criteria related to a certain rate or distortion, i.e., a method for optimizing coding cost. For example, encoder 10 and decoder 20 and corresponding modules 44, 58 can support different prediction modes, such as intra-coding mode and inter-coding mode, respectively. The granularity at which the encoder and decoder switch between these prediction mode types may correspond to the subdivision of each coding segment or coding block of image 12 and 12'. In units of these coding segments, for example, the image may be subdivided into intra-coded blocks and inter-coded blocks. As outlined in more detail below, intra-coded blocks are predicted based on the spatial, already coded / decoded neighborhood of each block. Several intra-coding modes exist and may be selected for each intra-coded segment, including directional or angular intra-coding modes, so that each segment is filled by extrapolating neighborhood sample values ​​along a specific direction specific to the intra-coding mode of each direction to each intra-coded segment. The intra-coding mode may include one or more further modes, such as a DC coding mode (where the prediction for each intra-coded block assigns a DC value to all samples within each intra-coded segment) and / or a planar intra-coding mode (where, based on adjacent samples, the prediction for each block is approximated or determined to be a spatial distribution of sample values ​​described by a two-dimensional linear function over the sample locations of each intra-coded block, using the slope and offset of a plane driven by a two-dimensional linear function). In comparison, inter-coded blocks may be predicted, for example, in time.In the case of inter-coded blocks, motion vectors may be signaled within the data stream, and the motion vectors indicate the spatial displacement of the previously encoded portion of the video to which image 12 belongs, at the location where the previously encoded / decoded image is sampled in order to obtain the predicted signal for each inter-coded block. This means that, in addition to the residual signal coding included in the data stream 14, such as the entropy coding transformation coefficient level representing the quantized spectral domain predicted residual signal 24'', the data stream 14 may also be coded into optional additional parameters, such as coding mode parameters that assign coding modes to different blocks, prediction parameters for some of the blocks such as motion parameters for inter-coded segments, and parameters that control and signal the subdivision of images 12 and 12' to the segments, respectively. The decoder 20 uses these parameters to subdivide the image in the same way that the encoder did, assign the same prediction modes to the segments, and perform the same predictions to produce the same predicted signals.

[0025] Figure 3 shows the relationship between the reconstructed signal of the reconstructed image 12', the predicted residual signal 24'''' transmitted in the data stream, and the combination of the predicted signal 26. As mentioned above, additional combinations are possible. The predicted signal 26 is shown in Figure 3 as a subdivision of the image region into intra-encoded blocks, illustrated using hatching, and inter-encoded blocks, illustrated without hatching. The subdivision can be any subdivision, such as a normal subdivision of the image region into rows and columns of blocks, or a multi-tree subdivision of the image 12 into leaf blocks of varying sizes, such as a quad-tree subdivision into blocks, a mixture of which is shown in Figure 3, where the image region is first subdivided into rows and columns of a tree root block, and then further subdivided according to a recursive multi-tree subdivision. In this case as well, the data stream 14 may have an intra-encoded mode for the intra-encoded blocks 80, assigning one of several supported intra-encoded modes to each intra-encoded block 80. Further details are described below. In the case of inter-encoded block 82, the data stream 14 may have one or more motion parameters encoded within it. Generally speaking, inter-encoded block 82 is not limited to temporal encoding. Alternatively, inter-encoded block 82 could be any block predicted from a previously encoded portion beyond the current image 12 itself, such as a previously encoded image of the video to which image 12 belongs, or another view, or, in the case of encoder 10 and decoder 20, which are scalable encoders and decoders respectively, a hierarchically lower layer of image. The predicted residual signal 24'''' in Figure 3 is also shown as a subdivision of the image region into block 84. These blocks may be called transform blocks to distinguish them from encoded blocks 80 and 82. In effect, Figure 3 shows encoder 10 and decoder 20 being divided into blocks using different subdivisions of image 12 and image 12', respectively. That is, one can be used for subdivision into encoded blocks 80 and 82, respectively, and the other for subdivision into transform block 84.Both subdivisions may be the same, i.e., each coding block 80 and 82 can simultaneously form a transform block 84, although Figure 3 shows, for example, the case where the subdivision into a transform block 84 forms an extension of the subdivision into coding blocks 80 and 82, so that any boundary between the two blocks 80 and 82 coincides with the boundary between the two blocks 84, or, to put it another way, each block 80 and 82 coincides with one of the transform blocks 84, or coincides with a cluster of transform blocks 84. However, the subdivisions can also be determined or selected independently of each other so that the transform block 84 can alternately traverse the block boundaries between blocks 80 and 82. Thus, as far as the subdivision into a transform block 84 is concerned, similar statements as those raised for the subdivision into blocks 80 and 82 apply, i.e., block 84 can be the result of a normal subdivision into blocks arranged in rows and columns of an image region, the result of a recursive multi-tree subdivision of an image region, or a combination thereof, or other types of blocking. As a side note, please be aware that blocks 80, 82, and 84 are not limited to quadratic, rectangular, or any other shape.

[0026] Figure 3 shows that the combination of the predicted signal 26 and the predicted residual signal 24'''' results in the directly reconstructed signal 12'. However, it should be noted that, according to an alternative embodiment, two or more predicted signals 26 can be combined with the predicted residual signals 24'''' to obtain the image 12' as a result.

[0027] In Figure 3, the transformation block 84 is defined as follows: The converter 28 and inverse converter 54 perform their transformations in units of these transformation blocks 84. For example, many codecs use some kind of DST or DCT for all transformation blocks 84. Some codecs allow skipping transformations so that for some of the transformation blocks 84, the predicted residual signal is encoded directly in the spatial domain. However, according to the embodiments described below, the encoder 10 and decoder 20 are configured to support several transformations. For example, the transformations supported by the encoder 10 and decoder 20 may include: DCT-II (or DCT-III), DCT stands for Discrete Cosine Transform. DST-IV, DST stands for Discrete Sine Transform. DCT-IV DST-VII Identity transformation (IT)

[0028] Naturally, the converter 28 will support all forward conversion versions of these conversions, while the decoder 20 or the inverse converter 54 will support its corresponding reverse or inverse version. Reverse DCT-II (or Reverse DCT-III) Reverse DST-IV Inverse DCT-IV Inverse DST-VII Identity transformation (IT)

[0029] The following description provides further details about the transformations that may be supported by encoder 10 and decoder 20. Note that in any case, the set of supported transformations may include only one transformation, such as a single spectral-to-space or spatial-to-spectral transformation.

[0030] As outlined above, Figures 1 to 3 are presented as examples that can implement concepts further described below to form specific examples of encoders 10 and decoders 20 according to this application. So far, encoders 10 and decoders 20 in Figures 1 and 2 represent possible implementations of the encoders and decoders described below in this specification, respectively. However, Figures 1 and 2 are merely examples. Encoders according to embodiments of this application can perform block-based coding of images 12 using different concepts than the encoder shown in Figure 1, as will be detailed later. For example, it is a still-image encoder, not a video encoder, and does not support interprediction, but the subdivision into blocks 80 is performed in a different manner than illustrated in Figure 3. Furthermore, this encoder does not use transform predictive residual coding, but instead directly codes the predictive residuals in the spatial domain, for example. Similarly, decoders according to embodiments of this application can perform block-based decoding of images 12' from a data stream 14, as will be outlined further below, but may differ from, for example, decoder 20 in Figure 2, and may be a still-image decoder, not a video decoder. Similarly, it does not support intra-prediction, or subdivides image 12' into blocks in a way different from the method described with respect to Figure 3, and / or similarly does not derive prediction residuals from data stream 14 in the transformation domain, but does derive them in the spatial domain, for example.

[0031] Figure 4A shows an embodiment of the device 10 that encodes image 12 into a data stream 14. Image 12 can be subdivided into blocks, or, for example, in the case of an HEVC-based codec, into macroblocks, also known as CTUs, 101a, 101b, ..., 101o.

[0032] The device 10, also referred to below as encoder 10, is configured to further divide macroblocks 101a, 101b, ..., 101o into smaller subblocks 102a, 102b, 102c, 102d, and finally into leaf blocks (leaf blocks are not yet shown in Figure 4A, but will be explained later with reference to Figures 8A and 8B). Encoder 10 is configured to divide image 12 into leaf blocks using recursive multitree partitioning. To further process these leaf blocks, encoder 10 is configured to encode image 12 into a data stream 14 in a block-based manner, that is, to encode the leaf blocks.

[0033] Block-based encoding of image 12 as data stream 14 may include each signal in the data stream 14, which may include, among other things, instructions for splitting into data stream 14, for example, instructions indicating one or more splitting parameters, such as the splitting mode and splitting ratio used for splitting the current block.

[0034] With respect to multi-tree partitioning, a block can be partitioned along a partitioning tree (for example, Figures 5A to 5D), and each node of the partitioning tree can belong to a respective block in the image. If a block is partitioned, the partitioned tree can be followed by subsequent tree nodes in subsequent partitioning levels, also called partitioning levels. The above signals in the data stream 14 can be done at one or more partitioning levels. According to some examples, partitioning parameters (e.g., partitioning mode, partitioning ratio, etc.) can be transmitted in the data stream 14 at one or more partitioning levels, and in some other examples, partitioning parameters (e.g., partitioning mode, partitioning ratio, etc.) can be transmitted in the data stream 14 at each partitioning level. According to further examples, partitioning parameters can be transmitted in the data stream 14 at partitioning levels of the partitioning tree, and the tree nodes at such partitioning levels can belong to one or more blocks placed on the image boundary of the image 12 being encoded. The above signals in the data stream 14 can be used in connection with each of the concepts of the present invention and corresponding embodiments described herein.

[0035] Returning to Figure 4A, we can see that macroblocks 101a, 101b, ..., 101o are secondary structures of a predetermined size that span a fixed grid on image 12. Because this fixed grid may exceed the size of image 12, it is sometimes possible that some of the macroblocks 101a, 101b, ..., 101o of that grid may extend beyond the boundaries of image 12. In Figure 4A, such a scenario is illustrated by macroblocks 101e, 101m, and 101o.

[0036] For example, with respect to macroblock 101e, the upper left and lower left portions of macroblock 101e are placed inside image 12, while the upper right and lower right portions of macroblock 101e are placed outside image 12. Therefore, macroblock 101e extends beyond the boundary of image 12, and the boundary of image 12 intersects with macroblock 101e at a specific position, namely, the upper left and lower left portions of macroblock 101e are placed inside image 12, while the upper right and lower right portions of macroblock 101e are placed outside image 12.

[0037] For example, with respect to macroblock 101o, only the upper left portion of macroblock 101o is placed inside image 12, while the upper right, lower right, and lower left portions of macroblock 101o are placed outside image 12. Furthermore, with respect to macroblock 101m, the upper left and upper right portions of macroblock 101m are placed inside image 12, while the lower left and lower right portions of macroblock 101m are placed outside image 12. Thus, each boundary of image 12 intersects with the respective macroblocks 101e, 101m, and 101o at different positions. Details regarding the positions of the macroblocks, particularly the positions of macroblocks 101e, 101m, and 101o relative to the boundaries of image 12, will be explained later in the text with reference to Figure 6.

[0038] As described above, macroblocks 101a, 101b, ..., 101o can be divided into smaller subblocks, and finally into leaf blocks. Non-restrictive examples are shown with respect to subblocks 102a, 102b, 102c, 102d. The subblocks themselves can be subjected to division until leaf blocks are reached. Division is also called splitting, and such splits of macroblocks and subblocks can be performed by using different splitting modes. Examples include quad splitting, horizontal bisplitting (also called bisplitting), vertical bisplitting, horizontal trisplitting (also called trisplitting), and vertical trisplitting. Split signaling can be performed using a binary decision tree, and such split signaling can be shown in data stream 14 in the manner described above.

[0039] For a brief explanation of such decision trees, please refer to Figures 5A to 5D, which compare split signaling at the image boundary with split signaling within the image. Figures 5A and 5C show a completely unrestricted decision tree that can be used for split signaling within the image, while Figures 5B and 5D show an exemplary restricted decision tree based on this concept that can be used for split signaling at the image boundary.

[0040] The fully unrestricted decision tree in Figure 5A may be used in this concept as a set of available partitioning modes that can be obtained based, for example, on a quad-tree plus binary tree (QTBT) structure as described in Non-Patent Literature 2. The QTBT scheme includes a set of available partitioning modes for dividing a block into smaller blocks, and this set of available partitioning modes includes four indicated partitioning modes: quad partitioning, no partitioning, vertical partitioning, or horizontal partitioning. Thus, whenever a block is to be divided into smaller blocks, a prior art encoder may always have to perform a fully unrestricted decision tree, i.e., the entire set of available partitioning modes. As can be seen, reaching a horizontal or vertical partition (i.e., a leaf block) in the decision tree requires spending three bins. The encoder then needs to signal the partitioning mode selected in each data stream. However, at image boundaries, the prior art may perform an implicit quad partitioning and refrain from signaling the relevant instructions in the data stream. This makes the handling of blocks at image boundaries very inflexible.

[0041] The fully unrestricted decision tree in Figure 5C may be used in this concept as a set of available splitting modes that can be obtained based on a quad-tree plus binary tree (QTBT) plus multi-type tree (MTT) structure, for example, as described in Non-Patent Literature 2 and Non-Patent Literature 3. The QTBT+MTT scheme includes a set of available splitting modes for dividing a block into smaller blocks, and this set of available splitting modes includes six indicated splitting modes: quad-split, no split, vertical 2-split, vertical 3-split, horizontal 2-split, and horizontal 3-split. Thus, whenever a block is to be divided into smaller blocks, a prior art encoder may always have to perform a fully unrestricted decision tree, i.e., the entire set of available splitting modes. In this example, reaching a horizontal or vertical split (i.e., a leaf block) in the decision tree requires spending four bins. The encoder then needs to signal the respective selected splitting mode in the data stream. However, at the image boundary, the prior art can perform an implicit quad-split and refrain from signaling the corresponding instruction in the data stream. This makes the handling of blocks at image boundaries less flexible compared to Figure 5A.

[0042] Accordingly, according to the present invention, the encoder 10 is configured to reduce the set of available subdivision modes to provide a reduced set of subdivision modes for a given block (macroblock or subblock) that is at least located at the image boundary (i.e., at least partially inside the image 12), as previously illustrated with reference to the given blocks 101e, 101m, and 101o shown in Figure 4A.

[0043] As illustrated in Figures 5B and 5D, the reduced set of the aforementioned split modes may, for example, include exactly two split modes. Alternatively, the reduced set of split modes may include three or more split modes, or exactly one split mode.

[0044] For example, a reduced set of split modes may include a quad-split (Figure 7A) and a vertical split. A vertical split could be, for example, a 2-split vertical (Figure 7B) or a 3-split vertical (Figure 7D).

[0045] Additionally or alternatively, a reduced set of division modes may include, for example, quad division and horizontal division. A horizontal division may be, for example, a horizontal 2-division (Figure 7C) or a horizontal 3-division (Figure 7E).

[0046] Additionally or alternatively, a reduced set of splitting modes may include, for example, vertical and horizontal splitting. Horizontal splitting could be, for example, a horizontal 2-way split or a horizontal 3-way split, and vertical splitting could be, for example, a vertical 2-way split or a vertical 3-way split.

[0047] Additionally or alternatively, a reduced set of split modes may include, for example, only four splits.

[0048] Additionally or alternatively, a reduced set of split modes may include, for example, a horizontal (2 or 3) split or a vertical (2 or 3) split when a 4-split is unavailable.

[0049] Additionally or alternatively, a reduced splitting mode may include exactly one predetermined splitting mode, which could be, for example, either a horizontal split or a vertical split. The choice of split may depend on the position of a given block relative to each image boundary; for example, a horizontal split may be selected for the bottom boundary of the image, while a vertical split may be selected for the right boundary of the image. A horizontal split can be a horizontal 2-split or a horizontal 3-split. A vertical split can be a vertical 2-split or a vertical 3-split. As can be seen from the decision tree, for example in Figure 5D, only one of the horizontal or vertical splits may be available in the reduced set.

[0050] In each of the above cases, one or more of the two divisions may include, for example, a division ratio of 1 / 2.

[0051] To summarize in more general terms, the reduced set of splits based on the innovative principles described herein may include the following splits: (a) If such a split exists, an unrestricted split that, when applied, does not prevent further subdivision of a given block. (b) If such a split exists, a split that sets an edge parallel to the image boundary, and i. If such a split exists and is permitted in a given block, the split located in the middle of the split block, or ii. If such a split exists and is permitted in a given block, a split that sets a split edge juxtaposed on the image boundary.

[0052] Based on QTBT, but also applicable to Non-Patent Documents 3 and 4, and other splitting schemes that offer the possibility of different splits as described above: 4-way splitting, 3-way vertical splitting, 2-way vertical splitting (e.g., splitting ratio of 1 / 2), 3-way horizontal splitting, and 2-way horizontal splitting (e.g., splitting ratio of 1 / 2) (see Figures 7A to 7E), in certain non-limiting embodiments of the present invention, the reduced split set according to the innovative principle described herein may be as follows: • 4 divisions (a) and 2 vertical divisions (e.g., division ratio 1 / 2) (b)-i or 3 vertical divisions (b)-ii • 4 divisions (a) and 2 horizontal divisions (e.g., division ratio 1 / 2) (b)-i or 3 horizontal divisions (b)-ii • Four divisions (a), or two or three divisions horizontally or vertically, for example, a division ratio of 1 / 2 (for example, if four divisions may not be available due to certain constraints).

[0053] Therefore, the reduced set of division modes based on the innovative principle may include a pre-selected set of division modes for dividing blocks at image boundaries. Encoder 10 can select one of the division modes included in the reduced set of division modes for dividing blocks at image boundaries, and encoder 10 can signal the selected division mode in the data stream 14. Thus, this concept can extend the signaling of blocks at image boundaries compared to the implicit quad division in the prior art. However, encoder 10 may only need to signal the uncertainty of the data stream 14, which keeps the signaling effort at a non-negligible level. Nevertheless, this concept offers much greater flexibility when dividing blocks at image boundaries compared to the prior art. This increased flexibility may compensate for and / or outweigh the slightly higher signaling effort.

[0054] Figure 5B may illustrate a reduced set of division modes that includes two different division modes, namely quad division and vertical 2 division, but the reduced set may also include quad division and horizontal 2 division, or vertical 2 division and horizontal 2 division, or any other combination of the above. In one example, the reduced set of division modes may include at least one division mode. In a further example, the reduced set of division modes may include at least two different division modes. In a further example, the reduced set of division modes may include exactly two different division modes, even if the set of available division modes may include more than these two different division modes.

[0055] The selection of a division mode to be included in the reduced set of division modes may depend on the position of each predetermined block 101e, 101m, and 101o relative to the boundary of image 12, that is, on the position where the boundary of image 12 intersects with the predetermined blocks 101e, 101m, and 101o.

[0056] In other words, when the encoder 10 divides the image 12 into leaf blocks, it is configured to reduce the set of available division modes for dividing the predetermined blocks 101e, 101m, and 101o, corresponding to a predetermined tree level of multi-tree division and extending beyond the boundary of the image 12, depending on the position where the boundary of the image 12 intersects with the predetermined blocks 101e, 101m, and 101o, so that the boundary of the image 12 obtains a reduced set of one or more division modes.

[0057] According to the present invention, if the cardinality of the reduced set is 1, that is, if the reduced set of division modes can contain only one division mode, the encoder 10 is configured to apply the division mode of this reduced set to divide a given block 101e, 101m, 101o.

[0058] However, if the cardinality of the reduced set is greater than 1, i.e., if the reduced set of division modes contains two or more division modes, the encoder 10 is configured to select one of these division modes of the reduced set and apply the selected one of the division modes to divide a given block 101e, 101m, 101o. Furthermore, the encoder 10 is configured to signal each selection, i.e., the selected division mode, within the data stream 14 in the manner described above. For example, the encoder 10 can signal the selection by setting a 1-bit flag. However, as described above, there may be cases where only the uncertainty between the division modes included in the already reduced set of division modes needs to be signaled in the data stream 14.

[0059] As described above, the cardinality of the reduced set can be, for example, exactly one or exactly two.

[0060] Figure 4B shows a corresponding device 20 for decoding the image 12 in the data stream 14, which will also be referred to as the decoder 20 below.

[0061] The decoder 20 is configured to use recursive multi-tree partitioning to divide the image 12 into leaf blocks, and to use the division of the image 12 into leaf blocks to decode the image 12 from the data stream 14 in a block-based manner.

[0062] When the decoder 20 divides the image 12 into leaf blocks, it is further configured to correspond to a predetermined tree level of multi-tree division and, for a predetermined block that extends beyond the boundary of the image 12, to reduce the set of available division modes for dividing the predetermined block, depending on the position where the boundary of the image 12 intersects the predetermined block, such that the boundary of the image 12 obtains a reduced set of one or more division modes.

[0063] In this case as well, if the cardinality of the reduced set is 1, that is, if the reduced set of splitting modes can contain only one splitting mode, the decoder 20 is configured to apply the splitting mode of the reduced set for splitting a given block.

[0064] If the cardinality of the reduced set is greater than 1, i.e., if the reduced set of split modes contains two or more split modes, the decoder 20 is configured to select one of the split modes of the reduced set and apply the selected split mode to split a given block according to the signal transmission in the data stream 14. That is, the decoder 20 can derive a signal from the data stream 14 to indicate to the decoder 20 which split mode to use for splitting. This signal may be the 1-bit flag described above, which may have been previously included in the data stream by the encoder 10.

[0065] Therefore, the concepts described herein provide a novel mechanism for boundary processing. This may be applicable to all recursive partitioning methods that typically allow two or more partitions at a given level. In this case as well, the concept can extend the signal processing of blocks at image boundaries compared to the implicit partitioning of prior art. However, the concept offers far greater flexibility in partitioning blocks at image boundaries compared to prior art. This increased flexibility may compensate for and / or outweigh the slightly higher signaling effort of the reduced set of partitioning modes by the concept.

[0066] A non-restrictive example of this concept is shown in Figure 6, which is a segmented image 12 similar to Figure 4A. However, Figure 6 shows some further details, namely, size w f ×h f Figure 12 shows an image with dimensions (width × height), which is subdivided into blocks, e.g., macroblocks 101a, 101b, ..., 101o, smaller subblocks, and finally leaf blocks. Three specific macroblocks are highlighted and numbered 1, 2, and 3, circled. These exemplary highlighted macroblocks may correspond to the aforementioned macroblocks 101e, 101m, and 101o shown in Figure 4. Furthermore, leaf blocks will be discussed in more detail later in the text with reference to Figures 8A and 8B.

[0067] FIG. 6 shows the splitting of image 12 into a strict macroblock structure of fixed size (e.g., CTU splitting in H.265 / HEVC [1] or a future video standard with features beyond H.265 / HEVC [2]). In this example, it can be assumed that the resolution of the video signal may not be spatially limited to a multiple of the size of a single macroblock 101a, 101b, …, 101o. Given these preconditions, it can be speculated that the last macroblock in a macroblock row and / or all macroblocks in the last macroblock row may cross at least one boundary of image 12. In such a case, the low-level splitting into coding units (CUs), prediction units (PUs), and transform units (TUs) of this macroblock, or any other type of sub-CTU level splitting, needs to ensure that the coded split is placed inside the boundaries of image 12. The concepts described herein can enable the definition of a reduced set of splits for splitting blocks. For example, macroblock 101m has a width w b , and a height h b at position x b , y b . This block may cross the boundary of image 12 of size w f ×h f in at least one of the following six cases.

[0068] C1: Corresponding to block 101e, i.e., the upper right corner of block 101e is not included in image 12, and the lower left corner is included in image 12 (the upper left corner is inside image 12): x b +w b >w f ; y b +h b ≦h f ; x b ≧0; y b ≧0, C2: Corresponding to block 101m, i.e., the upper right corner of block 101m is included in image 12, and the lower left corner is not included in image 12 (the upper left corner is inside image 12): x b +wb ≤w f ;y b +h b > h f ;x b ≥0;y b ≥0, C3: Corresponds to block 101o, i.e., neither the upper right corner nor the lower left corner is included in image 12 (the upper left corner is inside image 12): x b + w b >w f ;y b +h b > h f ;x b ≥0;y b ≥0, C4: The top-left corner of the block is not included in the image, the bottom-left corner is not included in image 12, and the top-right corner is included in image 12: x b <0;y b ≥0;x b + w b >0 (This case is treated the same as Case 1 and may not be specified individually), C5: The top-left corner of the block is not included in Image 12, the top-right corner is not included in Image 12, and the bottom-left corner is included in Image 12: x b ≥0;y b <0;y b +h b >0 (This case is treated the same as case 2 and may not be specified individually), C6: Only one of the following block corners is included in Image 12: bottom right, top right, or bottom left (this case is treated the same as case 3 and is not specified individually).

[0069] If none of cases C1 through C6 apply to a particular given block and image 12, the given block may lie outside the boundary of image 12, and there may be no further interest in the concepts described herein. Two or more cases may apply simultaneously, specifically, case pairs C1 and C4, C2 and C5, or C3 and C6 may apply simultaneously. All of these case pairs are treated equally, and there is no need to treat a particular constellation individually.

[0070] To address at least one of these cases C1-C6, the concept provides a reduced set of split modes, which may include one or more split modes selected from the set of split modes actually available.

[0071] This set of actually available partitioning modes can be selected by the encoder 10 depending on the block-based coding scheme used. For example, the encoder 10, · Independent of the location of a given block relative to the image boundary, and / or • Depending on the sequence of preceding tree-level partitioning modes in which a given block appears, It may be configured to determine the set of available partitioning modes for a given block.

[0072] The same applies to decoder 20. For example, decoder 20 is • Independent of the location of a given block relative to the image boundary, and / or • Depending on the sequence of preceding tree-level partitioning modes in which a given block appears, It may be configured to determine the set of available partitioning modes for a given block.

[0073] In this case as well, in order to handle at least one of the above cases C1 to C6, the concept provides a reduced set of split modes which may include one or more split modes selected from the set of split modes actually available.

[0074] For example, a reduced split set might consist of the following splits: (a) If such a split exists: an unrestricted split that, when applied, does not impose any restrictions on further splits of a given block. (b) In cases C1 and C2, if such a split exists: a split that sets an edge parallel to the boundary of image 12. i. If such a split exists and is permitted in a given block, the split is located in the middle of the split block. ii. If such a split exists and is permitted in a given block, the split sets a dividing edge juxtaposed on the boundary of image 12.

[0075] In other words, the encoder 10 reduces the set of available split modes to a reduced set of one or more split modes, A first splitting mode that includes a split that does not impose restrictions on one or more subsequent splits of subsequent tree levels following a given tree level (see, for example, split (a) above), and It may be configured to select at least one of a second splitting mode, which includes a split that divides a given block into two or three subblocks along at least one split line. The dividing line is parallel to the position of the boundary within the given block, and the dividing line is at least one of the following: one that is centrally located inside the given block (see, for example, split(b)-i above), and one that is juxtaposed with the boundary of image 12 (see, for example, split(b)-ii above).

[0076] The same applies to the decoder 20. That is, the decoder 20 reduces the set of available split modes to a reduced set of one or more split modes, A first splitting mode that includes a split that does not impose restrictions on one or more subsequent splits of subsequent tree levels following a given tree level (see, for example, split (a) above), and The system may be configured to select at least one of a second splitting mode, which includes splitting a given block into two or three subblocks along at least one dividing line. The dividing line is parallel to the position of the boundary within the given block, and the dividing line is at least one of the following: one that is centrally located inside the given block (see, for example, split(b)-i above), and one that is juxtaposed with the boundary of image 12 (see, for example, split(b)-ii above).

[0077] If split(b)-ii is not available in the partitioning method used for the specified block, split(b)-i may be included in the reduced split set. Split(b)-i may generally take precedence over split(b)-ii. As described above, at least one split may be included in the reduced split set.

[0078] For example, split (a) could be a four-way split, and split (b)-i could be at least one of a horizontal two-way split or a vertical two-way split. For further example, one of the additionally introduced splits, as described in Non-Patent Documents 3 and 4, could be used for (b)-ii if the boundary of image 12 is at 1 / 4 or 3 / 4 of the divided block. In certain implementations, a vertical or horizontal split with a division ratio of 1 / 2 may be preferred over the additional splits introduced in Non-Patent Documents 3 and / or Non-Patent Documents 4.

[0079] The above divisions are schematically shown in Figures 7A to 7E. Figure 7A shows a four-way division (four-split) in which a given block 701 (macroblock or subblock) is divided into four equal sections 71a, 71b, 71c, and 71d. Figure 7B shows a two-way vertical division (two-split) with a division ratio of 1 / 2, in which a given block 702 (macroblock or subblock) is divided into two equal vertical sections 72a and 72b. Figure 7C shows a two-way horizontal division with a division ratio of 1 / 2, in which a given block 703 (macroblock or subblock) is divided into two equal horizontal sections 73a and 73b. Figure 7D shows a three-way vertical division as described in Non-Patent Literature 3, in which a given block 704 (macroblock or subblock) is divided into three vertical sections 74a, 74b, and 74c. Figure 7E shows a horizontal three-part division as described in Non-Patent Literature 3, in which a predetermined block 705 (macroblock or subblock) is divided into three horizontal sections 75a, 75b, and 75c.

[0080] For example, a vertical bisection as shown in Figure 7B may be applicable when each boundary of image 12 intersects with a given block 702, and as a result, the boundaries of image 12 placed inside each block 702 are also vertical, i.e., parallel to the vertical split line. Such a case may apply to block 101e, for example, as shown in Figure 6.

[0081] A horizontal bisection, as shown in Figure 7C, can be applied when each boundary of image 12 intersects with a given block 703, and as a result, the boundaries of image 12 placed inside each block 703 are also horizontal, i.e., parallel to the horizontal split line. Such a case may apply to block 101m, for example, as shown in Figure 6.

[0082] As a further example of the use of Generalized Binary Splitting (GBS), if a generalized GBS tree is used as a subtree in a quad-tree structure, split (a) may be a vertical split with a split ratio of 1 / 2 (which may be the default split for GBS, provided the split tree is not limited so far) or a quad split. Vertical and horizontal splits that can be converted to vertical / parallel semantics can be used as split (b)-i or (b)-ii (presumably using a split modifier adjusted accordingly). In this case as well, a split with modifier 1 / 2 may be preferred even if other splits provide a split juxtaposed on the boundary of image 12.

[0083] In a further example, three exemplary reduced split sets (RSS) may be based on QTBT, but may also be valid for schemes described in Non-Patent Documents 3 and 4, and any other splitting schemes that offer the possibility of the following three splits (four-way split, vertical split with a split ratio of 1 / 2, and horizontal split with a split ratio of 1 / 2), RSS1: 4-part split (a) and vertical split with a split ratio of 1 / 2 (b)-i, RSS2: 4-part split (a) and horizontal split with a split ratio of 1 / 2 (b)-i, RSS3: This may be either a 4-way split (a) or a vertical or horizontal split (b)-i with a split ratio of 1 / 2.

[0084] These exemplary reduced sets of splitting modes, namely RSS1, RSS2, and RSS3, may be one non-restrictive example of a subset of the set of available splitting modes. That is, a reduced set of splitting modes may include one or more available splits that are available in the set of available splitting modes (for a given block at a given tree level). The number of splits included in the set of available splitting modes may vary depending on the encoding scheme, and the number of splits included in the reduced set of splitting modes may vary depending on the position of each given block (i.e., split at the current tree level) relative to the boundary of image 12.

[0085] As described above, the encoder 10 can determine the set of available split modes. That is, the encoder 10 can selectively select which split modes (e.g., 4-way split, 2-way vertical split, 2-way horizontal split, etc.) are included in the set of available split modes.

[0086] For example, the block-based coding scheme described above may include a basic set of partitioning modes, which may include a quad partition, at least one horizontal 2-part partition (possibly using different modifiers), and at least one vertical 2-part partition (possibly using different modifiers).

[0087] However, some splits, particularly some bipartite splits, can be restrictive. For example, if a bipartite split is applied to a given block, a quadripartite split may not be permitted as a subsequent split. In other words, a quadripartite split may only be permitted at certain tree levels if the preceding split at the previous tree level does not include one of the restrictive splits, particularly one of the restrictive bipartite splits.

[0088] Therefore, if one of the restrictive splits, particularly one of the restrictive bipartite splits, is applied to a given block at a given tree level, subsequent splits may include only the set of primitives described above and not include the quaternite split.

[0089] Therefore, according to one embodiment, the encoder 10 is If all the preceding tree levels in which a given block appears are not a restricted set of two-part divisions, then the set of primitive division modes will be equal to a set of four-part divisions, at least one horizontal two-part division, and at least one vertical two-part division. • If the preceding tree-level partitioning mode in which a given block appears is one of the restricted sets of 2 partitions, then the set of primitives for the partitioning mode will be equal to the set of primitives for the partitioning mode minus the set of 4 partitions. It may be configured to determine the set of available split modes.

[0090] The same applies to decoder 20. Therefore, decoder 20 is If all the preceding tree levels in which a given block appears are not a restricted set of two-part divisions, then the set of primitive division modes will be equal to a set of four-part divisions, at least one horizontal two-part division, and at least one vertical two-part division. If the preceding tree-level partitioning mode in which a given block appears is one of the restricted sets of 2 partitions, then the set of primitives for the partitioning mode will be equal to the set of primitives for the partitioning mode minus the set of 4 partitions. It may be configured to determine the set of available split modes.

[0091] Additionally or alternatively, the above set of primitives for split modes may include at least one horizontal trinity (possibly using different modifiers) and at least one vertical trinity (possibly using different modifiers).

[0092] As described above, the encoder 10 can reduce the set of available split modes in order to provide a reduced set of split modes. Thus, for each given block, the encoder 10 can selectively select a reduced subset of split modes (i.e., one or more split modes) from the above set of available split modes. Only these selectively selected split modes may be included in the reduced set of split modes. Thus, the reduced set of split modes may include the above four splits by split(a) and at least one of the splits by split(b)-i or split(b)-ii.

[0093] Therefore, according to one embodiment, the encoder 10 reduces the set of available split modes to a reduced set of one or more split modes, • Divide into 4 sections (a), and The split line may be configured to select at least one of two divisions that divide a given block 702, 703 into two subblocks 72a, 72b, 73a, 73b along the split line, wherein the split line is parallel to the position of the image boundary within the given block 702, 703, and at least one of the following is the split line being centrally located inside the given block according to split (b)-i, and the split line being juxtaposed with the boundary according to split (b)-ii. That is the case.

[0094] In this case as well, the same applies to decoder 20. Therefore, decoder 20 reduces the set of available split modes to a reduced set of one or more split modes, • Divide into 4 sections (a), and The system may be configured to select at least one of two splits that split a given block 702, 703 into two subblocks 72a, 72b, 73a, 73b along a split line, wherein the split line is parallel to the position of the image boundary within the given block 702, 703, and at least one of the split lines being centrally located inside the given block according to split (b)-i, and juxtaposed with the boundary according to split (b)-ii. That is the case.

[0095] In this case as well, split(b)-i may include a 2-part split (perhaps using different modifiers, but preferably using a split ratio of 1 / 2), and split(b)-ii may include a 2-part split and / or a 3-part split (perhaps using different modifiers).

[0096] Therefore, according to one embodiment, the encoder 10 reduces the set of available split modes to a reduced set of one or more split modes, • Divide into 4 sections (a), and The system may be configured to select at least one of three divisions that divide a given block 704, 705 into three subblocks 74a, 74b, 74c, 75a, 75b, 75c along a split line, wherein the split line is parallel to the position of the image boundary within the given block 704, 705, and the split line is juxtaposed with the boundary according to split (b)-ii.

[0097] In this case as well, the same applies to decoder 20. Therefore, decoder 20 reduces the set of available split modes to a reduced set of one or more split modes, • Divide into 4 sections (a), and The system may be configured to select at least one of three divisions that divide a given block 704, 705 into three subblocks 74a, 74b, 74c, 75a, 75b, 75c along a split line, wherein the split line is parallel to the position of the image boundary within the given block 704, 705, and the split line is juxtaposed with the boundary according to split (b)-ii.

[0098] Figures 8A and 8B illustrate an example of how the concepts described herein can be applied to a block-based coding scheme along with image boundary processing.

[0099] Figure 8A shows a division of a given block that in this example may correspond to macroblock 101o as shown in Figure 6. The hatched portion of block 101o represents the position of the lower right corner of image 12. This example shows the division of macroblock 101o into smaller subblocks by applying only a 4-way division.

[0100] The numbers shown for each subblock may indicate the number of 4 divisions that need to be applied to arrive at the leaf block division of macroblock 101o, as shown in Figure 8A. The first 4 divisions divide macroblock 101o into four equal subblocks 801a, 801b, 801c, and 801d. As can be seen, only the two subblocks on the left, 801a and 801d, may still contain the boundary of image 12. The two subblocks on the right, 801b and 801c, are placed outside of image 12 and no longer contain the boundary of image 12. They may be discarded.

[0101] The second quad division further subdivides the two left subblocks 801a and 801d into four smaller subblocks. As can be seen, the two left subblocks 802a and 802b (derived from the second quad division) are placed inside image 12 and no longer contain the image boundary. Thus, these two subblocks 802a and 802b can be further subdivided using an unrestricted split set, depending on the subdivision method used. Further subdivisions of these blocks may not be described in further detail herein with respect to this concept.

[0102] However, a third division is necessary because the remaining subblocks (derived from the second four-part division) may all contain image boundaries. This third division leads to the leaf block indicated by the number "3".

[0103] Figure 8B shows an alternative split to which the concept of reduced split sets described herein is applied. The numbers in each subblock indicate the number of quadrilaterals and non-quadrilaterals (e.g., 2-quadrilaterals) that need to be applied to arrive at the leaf block split of macroblock 101o, as shown in Figure 8B. The first number may indicate the number of quadrilaterals, and the second number may indicate the number of non-quadrilaterals.

[0104] For example, the upper left leaf block 802a(1,1) has one quad division and one vertical 2 division applied. The adjacent right leaf block 803a(1,2) has one quad division and two vertical 2 divisions applied. The adjacent lower leaf block 803b(2,1) has two quad divisions and one horizontal 2 division applied. The leaf block 803c(3,0) in the lower right corner of Image 12 has three quad divisions applied, and no non-quad divisions were applied.

[0105] In Figure 8B, solid lines indicate edges (split lines) defined by a four-way division, dashed lines indicate edges (split lines) defined by a two-way division in the vertical direction, and dotted lines indicate edges (split lines) defined by a two-way division in the horizontal direction.

[0106] More specifically, a quadrilateral split is applied as the first split at the first tree level, resulting in four equal subblocks 801a, 801b, 801c, and 801d. As can be seen, only the two left subblocks 801a and 801d can still contain the boundary of image 12. The two right subblocks 801b and 801c are placed outside of image 12 and no longer contain the boundary of image 12. They can be discarded.

[0107] The upper-left subblock 801a may be further divided at a second tree level. A reduced split set may be provided for the division of subblock 801a. For example, a reduced split set may be provided that includes at least one of a quadruple split and a vertical bipup, or that includes only a vertical bipup. In the example shown in Figure 8B, subblock 801a may be subjected to a vertical bipup with modifier 1 / 2 at the second tree level, and the split line 811 is centered inside block 801a. Thus, as a result, the upper-left subblock 802a can be formed.

[0108] In this concept, only one quadrilateral and one vertical bilateral (and thus only two tree levels) are needed to reach the leaf block subblock 802a. Instead, in Figure 8A, two blocks 802a and 802b are needed to cover the same area of ​​Image 12 as the single block 802a in Figure 8B.

[0109] Furthermore, the right-adjacent block 802b in Figure 8B is then further divided at a third tree level. For the division of subblock 802b, a reduced set of splits may be provided. For example, since the previous split is already a vertical bipartite division, which may not allow a subsequent quadripartite division, a reduced set of splits including at least a vertical bipartite division may be provided. In the example shown in Figure 8B, subblock 802b may also be subjected to a vertical bipartite division at the third tree level using modifier 1 / 2, with the split line 812 centered inside block 802b and juxtaposed with the image boundary. Thus, the left subblock 803a may consequently be set at the third tree level, and the left subblock 803a is a leaf block, but it may also be further divided by the division method used.

[0110] In this concept, only one quadruple division and two vertical bifurcations (and thus only three tree levels) need to be applied to reach the leaf block, subblock 803a. Alternatively, in Figure 8A, four blocks are needed to cover the same area of ​​image 12 as the single block 803a in Figure 8B. However, if the encoder 10 determines so, it may be applied as in Figure 8A, provided that the quadruple division is always selected from a reduced split set.

[0111] Furthermore, the lower adjacent block 802d in Figure 8B may be derived from two consecutive quad divisions at the first and second tree levels, respectively, and block 802d may be further divided at the third tree level. For the division of subblock 802d, a reduced split set may be provided. For example, a reduced split set may be provided that includes at least a quad division and a horizontal 2 division, or a reduced split set that includes only a horizontal 2 division. In the example shown in Figure 8B, subblock 802d may be subjected to a horizontal 2 division at the third tree level using modifier 1 / 2, with the split line 813 centered inside block 802d and juxtaposed with the image boundary. Thus, subblock 803b may be set at the third tree level as a result, where subblock 803b is a leaf block.

[0112] In this concept, only two quadrilateral divisions and one horizontal bilateral division (and thus only three tree levels) are needed to reach the leaf block, subblock 803b. Instead, in Figure 8A, two blocks are needed for only one to cover the same area of ​​image 12 as block 803b in Figure 8B.

[0113] Block 803c may also be a leaf block that can be derived from three consecutive quad divisions. This example of leaf block 803c may be identical to the quad division of the same block shown in Figure 8A. Since the corners in Figure 12 are contained within the block that leads to leaf block 803c, the syntax may allow providing quad divisions only in a reduced set of division modes.

[0114] For example, the encoder 10 may be configured to select a 4-way split from the set of available split modes only when a given block extends beyond the corners of the image 12, so as to reduce the set of available split modes to a reduced set of one or more split modes.

[0115] The same applies to decoder 20. Therefore, decoder 20 may be configured to select a 4-way split from the set of available split modes only when a given block extends beyond the corners of image 12, so as to reduce the set of available split modes to a reduced set of one or more split modes.

[0116] As shown in Figure 8B, the resulting leaf blocks 802a, 803a, 803b, and 803c represent the minimum division required to process the boundaries of image 12. Additionally or alternatively, the resulting leaf blocks 802a, 803a, 803b, and 803c may be further divided within the given limitations of the division method used.

[0117] In general, if the last split contains at least one valid split, further restrictions may be imposed on the reduced split set. These restrictions are: • The possibility of dividing a specified block size (see Figure 8B), • Divisibility to a specified depth These are possible, but not limited to them.

[0118] The latter bullet point, "divisibility at a specified depth," is briefly explained using the following example and with reference to Figures 5A and 5B.

[0119] To ensure optimal efficiency, some signaling is used instead of purely implicit splitting derivation, but only limited information needs to be signaled. In particular, there is no need to transmit information about the splitting itself. In the above image boundary cases C1 to C3 (see Figure 6), splitting is performed. If the reduced split set contains two or more elements, only the remaining uncertainty needs to be signaled. In the particular embodiment based on the above QTBT, one binary signal needs to be transmitted for cases C1 (see Figure 5B) and C2, and C1 may contain a reduced split set containing a 4-split and a vertical 2-split, C2 may contain a reduced split set containing a 4-split and a horizontal 2-split, and C3 may contain a reduced split set containing only a 4-split, so no signaling is required in case C3.

[0120] Further syntactic restrictions may apply to this concept. For example, if the division imposes additional restrictions, the divided blocks may be completely or partially discarded if they overlap with image boundaries. For instance, the number of subsequent 2-part divisions may be limited (in contrast to 4-part divisions) (e.g., if it is part of a tree with division restrictions, if the division is not the default division, or if this method is used to complement a quad-tree structure).

[0121] If a bifurcation is applied across image boundaries as part of a reduced set, it may not be counted in the enforcement of this limit.

[0122] Therefore, for example, if a 2-part division from a reduced set is applied and the number of consecutive 2-part divisions is limited to a predetermined maximum number of consecutive 2-part divisions, the encoder 10 may be configured not to count consecutive 2-part divisions if the application of consecutive 2-part divisions crosses an image boundary.

[0123] The same applies to decoder 20. Therefore, if a 2x division from a reduced set is applied and the number of consecutive 2x divisions is limited to a predetermined maximum number of consecutive 2x divisions, decoder 20 may be configured not to count consecutive 2x divisions if the consecutive 2x divisions are applied across image boundaries.

[0124] Other restrictions, such as the prohibition of quad splits following splits other than quad splits, may be further enforced even across image boundaries (for example, when used to complement a quad tree structure).

[0125] Figure 9 shows a block diagram of the method for encoding image 12 using this concept.

[0126] In block 901, the image is divided into leaf blocks using recursive multi-tree partitioning.

[0127] Block 902 uses the division of the image into leaf blocks to encode the image into a data stream in a block-based manner.

[0128] In block 903, for a given block, the set of available division modes for dividing the given block is reduced according to the position where the image boundary intersects the given block, resulting in a reduced set of one or more division modes.

[0129] If the cardinality of the reduced set is 1 (C=1), the splitting mode of the reduced set is applied to split a given block, as shown in block 904.

[0130] If the cardinality of the reduced set is greater than 1, one of the splitting modes of the reduced set is selected, as shown in block 905, and the selected splitting mode is applied to split the given block.

[0131] In block 906, the selection, i.e., one of the selected division modes from a reduced set of division modes, is signaled in the data stream.

[0132] Figure 10 shows a block diagram of the method for decoding image 12 according to this concept.

[0133] In block 1001, the image is divided into leaf blocks using recursive multi-tree partitioning.

[0134] Block 1002 uses the division of the image into leaf blocks to decode the image from the data stream in a block-based manner.

[0135] In block 1003, for a given block, the set of available division modes for dividing the given block is reduced according to the position where the image boundary intersects the given block, to obtain a reduced set of one or more division modes.

[0136] If the cardinality of the reduced set is 1 (C=1), the splitting mode of the reduced set is applied to split a given block, as shown in block 1004.

[0137] If the cardinality of the reduced set is greater than 1, one of the splitting modes of the reduced set is selected, as shown in block 1005, and the selected splitting mode is applied to split the given block.

[0138] As shown in block 1006, the selection of one splitting mode from a reduced set of splitting modes is based on signal transfer in the data stream.

[0139] The method steps represented by the blocks in the block diagrams of Figures 9 and 10 can also be performed in a different order than those shown.

[0140] While several embodiments have been described in the context of apparatus, it is clear that these embodiments also represent descriptions of corresponding methods, where blocks or devices correspond to method steps or features of method steps. Similarly, embodiments described in the context of method steps also represent descriptions of corresponding blocks, items, or features of the corresponding apparatus. Some or all of the method steps can be performed by (or using) hardware devices such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more of the most important method steps can be performed by such devices.

[0141] Depending on specific implementation requirements, embodiments of the present invention may be implemented in hardware, in software, or at least partially in hardware or at least partially in software. The implementation may be carried out using a digital storage medium containing electronically readable control signals, such as a floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM, or flash memory, which cooperates (or can cooperate) with a programmable computer system to perform the respective method. Thus, the digital storage medium may be computer-readable.

[0142] Some embodiments of the present invention include a data carrier having electronically readable control signals that can cooperate with a programmable computer system so that one of the methods described herein can be performed.

[0143] Generally, embodiments of the present invention can be implemented as a computer program product having program code, the program code operates to execute one of the methods when the computer program product is run on a computer. The program code may be stored, for example, in a machine-readable carrier.

[0144] Other embodiments include a computer program stored in a machine-readable carrier that performs one of the methods described herein.

[0145] Therefore, in other words, embodiments of the method of the present invention are computer programs having program code that performs one of the methods of the present invention when the computer program is executed on a computer.

[0146] Accordingly, a further embodiment of the method of the present invention is a (recorded) data carrier (or digital storage medium, or computer-readable medium) containing a computer program that performs one of the methods described herein. The data carrier, digital storage medium, or recorded medium is typically tangible and / or non-temporary.

[0147] Therefore, a further embodiment of the method of the present invention is a data stream or sequence of signals representing a computer program that performs one of the methods described herein. The data stream or sequence of signals may be configured to be transmitted over a data communication connection, such as the Internet.

[0148] Further embodiments include process means configured or adapted to perform one of the methods described herein, such as a computer or a programmable logic device.

[0149] Further embodiments include a computer on which a computer program is installed that performs one of the methods described herein.

[0150] Further embodiments of the present invention include an apparatus or system configured to transfer (e.g., electronically or optically) a computer program that performs one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system may include, for example, a file server for transferring the computer program to the receiver.

[0151] In some embodiments, a programmable logic device (e.g., an FPGA) can be used to perform some or all of the functions of the method described herein. In some embodiments, the FPGA may work with a microprocessor to perform one of the methods described herein. Generally, the method is preferably performed by any hardware device.

[0152] The apparatus described herein can be implemented using hardware devices, a computer, or a combination of hardware devices and a computer.

[0153] The methods described herein can be performed using hardware devices, or using a computer, or using a combination of hardware devices and a computer.

[0154] The embodiments described above are merely illustrative of the principles of the present invention. Modifications and variations of the arrangements and details described herein will be obvious to those skilled in the art. Therefore, it is intended that the invention is limited only by the claims and not by the description of the embodiments herein and the specific details presented herein.

[0155] References [1]ITU-T and ISO / IEC.High efficiency video coding.ITU-T Recommendation H.265|ISO / IEC23008 10(HEVC)、edition1、2013;edition2、2014. [2]J.Chen、E.Alshina、GJSullivan、JR.Ohm、J.Boyce、Algorithm Description of Joint Exploration Test Model 6(JEM 6)、JVET、doc.JVET-F1001、April 2017. [3]X.Li、H.-C.Chuang、J.Chen、M.Karczewicz、L.Zhang、X.Zhao、A.Said、Multi-Type-Tree、JVET、doc.JVET-D0117、Oct.2016. [4]F.Le Leannec、T.Poirier、F.Urban、Asymmetric Coding Units in QTBT、JVET、doc.JVET-D0064、Oct.2016. [5]The 17182209.1-1718209.

Claims

1. A method for determining a division corresponding to a video data stream, the method is: The step of determining a set of splitting modes corresponding to a block of an image, wherein the set of splitting modes includes an unrestricted split that, when applied, does not prevent further subdivision of the block; If the block extends beyond the boundary of the image, the step of applying the unrestricted split to the block, The steps include encoding a subblock obtained from the block into the video data stream, or decoding it from the video data stream, Methods that include...

2. The method according to claim 1, wherein the set of split modes is a reduced set of split modes including the unrestricted split.

3. The method according to claim 2, wherein the unrestricted split is a split that sets a dividing edge parallel to the boundary of the image.

4. The method according to claim 3, wherein the unrestricted split is a split located in the center of the block.

5. The method according to claim 3, wherein the unrestricted split is a split that sets a dividing edge juxtaposed on the boundary of the image.

6. A device for determining divisions corresponding to a video data stream, the device, At least one memory, The system comprises one or more processors configured to execute instructions stored in at least one memory, The one or more processors described above are Determining a set of splitting modes corresponding to a block of an image, wherein the set of splitting modes includes an unrestricted split that, when applied, does not prevent further subdivision of the block; If the block extends beyond the boundary of the image, the unrestricted split is applied to the block. A device configured to perform the following actions: encoding a subblock obtained from the block into the video data stream, or decoding from the video data stream.

7. The apparatus according to claim 6, wherein the set of split modes is a reduced set of split modes including the unrestricted split.

8. The apparatus according to claim 7, wherein the unrestricted split is a split that sets a dividing edge parallel to the boundary of the image.

9. The apparatus according to claim 8, wherein the unrestricted split is a split located in the center of the block.

10. The apparatus according to claim 8, wherein the unrestricted split is a split that sets a dividing edge juxtaposed on the boundary of the image.

Images