Decoder, encoder, and method including encoding of intra sub-partitions

Abstract

A decoder (20) for decoding a picture is configured to: derive, from a data stream (112) in which the picture (110) is encoded, an assignment of the picture (110) to a set (114) of prediction types including intra prediction (116) and inter prediction (118) at a granularity of blocks (120) into which the picture (110) is subdivided, such that each block (120) is assigned to an associated prediction type in the set (114) of prediction types; and for each intra prediction block (120) assigned intra prediction (116), derive an associated intra prediction mode (125) from the set (122) of intra prediction modes from the data stream (112). For each predetermined intra prediction block (120) whose associated intra prediction mode (125) is included in a predetermined subset (124) of predetermined intra prediction modes, the decoder is configured to: derive information (126) from the data stream (112) regarding a split (128) of the corresponding predetermined intra prediction block (120) into partitions (130); and perform intra prediction on each partition (130) of the corresponding predetermined intra prediction block (120) in a manner depending on the predetermined intra prediction mode (125) assigned to the predetermined intra prediction block (120). The decoder is configured to: for each block (120), derive information regarding the prediction residual of the corresponding block (120) from the data stream (112) by deriving the prediction residual of each predetermined intra prediction block (120), wherein the prediction residual of each predetermined intra prediction block (120) is derived by: for each partition (130) of the corresponding predetermined intra prediction block (120), deriving a partition-specific prediction residual signal (132) related to a spatial prediction residual signal (134) of the corresponding partition (130) of the corresponding predetermined intra prediction block (120) from the data stream (112) via a predetermined transform (T); and deriving information (136) of a predetermined transform in a set (138) of transforms from the data stream (112), the set (138) of transforms including a first transform (T1) and a second transform (T2), the second transform (T2) being equal to a concatenation of a primary transform and a secondary transform (Ts) applied to a subset (140) of the coefficients of the primary transform. Further, the decoder is configured to reconstruct each block (120) using the information regarding the prediction residual of the corresponding block (120) and a prediction signal obtained using the prediction type (125) assigned to the corresponding block (120).

Description

Technical Field

[0001] This application relates to picture encoding / decoding or video encoding / decoding.

[0002] Introductory Note

[0003] Next, different inventive embodiments and aspects will be described.

[0004] In addition, other embodiments will be defined by the appended claims.

[0005] It should be noted that any embodiment defined by the claims can be supplemented by any details (features and functions) described in the following embodiments.

[0006] In addition, embodiments can be used alone and can also be supplemented by any feature in another embodiment or any feature included in the claims.

[0007] In addition, it should be noted that the various aspects described herein can be used alone or in combination. Therefore, details can be added to each of the individual aspects without adding details to another of the aspects.

[0008] It should also be noted that the present disclosure explicitly or implicitly describes features that can be used in an encoder (a device for providing an encoded representation of an input signal, such as a picture or video) and a decoder (a device for providing a decoded representation of a signal based on the encoded representation). Therefore, any feature described herein can be used in the context of an encoder and in the context of a decoder.

[0009] In addition, the features and functions related to the method disclosed herein can also be used in a device (configured to perform such functions). In addition, any features and functions regarding the device disclosed herein can also be used in the corresponding method. In other words, the methods disclosed herein can be supplemented by any features and functions described regarding the device.

[0010] In addition, as will be described in the "Implementation Alternatives" section, any features and functions described herein can be implemented in hardware or software, or using a combination of hardware and software. Background Art

[0011] Currently, in order to encode or decode a picture or video, there are encoding restrictions on blocks using intra sub-partitions.

[0012] The purpose of the gist of this application is to provide a video codec using block-based predictive coding, which has improved encoding efficiency. It is desired to obtain a more flexible encoding concept for blocks using intra sub-partitions or an alternative encoding concept for blocks using intra sub-partitions.

[0013] This is achieved by the gist of the independent claims of the present application.

[0014] Other embodiments according to the present invention are defined by the gist of the dependent claims of the present application. Summary of the Invention

[0015] According to a first aspect, the inventors of the present application have realized that a problem encountered when encoding or decoding a block using intra sub-partition (ISP) stems from the fact that the low-frequency non-separable transform (LFNST) (i.e., the secondary transform) cannot currently be used in the VTM-5.0 intra sub-partition. According to the first aspect of the present application, this difficulty is overcome by enabling the use of the secondary transform on blocks using ISP, that is, by allowing the use of a two-level transform at the transform unit level. The use of a first transform on the intra sub-partition, as well as the cascade of the primary transform and the secondary transform, introduces additional syntax elements, but it has been found that this additional signaling overhead is overcompensated due to the fact that the secondary transform after the primary transform can better compress non-zero coefficients, thereby resulting in a reduced amount of data for representing the partition-specific residual block. It has been found that higher coding efficiency and a reduced bit rate can be achieved by also using the secondary transform for the intra sub-partition. Hereinafter, the second transform may be equal to the cascade of the primary transform and the secondary transform applied to a subset of the coefficients of the first transform. The primary transform may be the first transform. Implementing the second transform for the intra partition may be related to certain conditions. For example, the second transform is enabled for a partition in the following cases: there are no non-zero transform coefficients (i.e., non-zero coefficients of the primary transform) outside a predetermined region, and / or the number of non-zero transform coefficients is equal to or lower than a predetermined limit, and / or the position of the last non-zero coefficient along the scan path from the DC coefficient position to the highest frequency coefficient position (indicated by the last position syntax element in the data stream) is equal to or lower than another predetermined threshold.

[0016] Therefore, according to the first aspect of the present application, an encoder for encoding a picture and a decoder for decoding a picture are proposed. The decoder / encoder is configured to: derive / encode from / to the data stream into which the picture is encoded an assignment of the picture, in terms of the granularity of the blocks into which the picture is divided, to a set of prediction types including intra prediction and inter prediction, such that each block is assigned to an associated prediction type in the set of prediction types. Thus, each block of the picture is assigned, for example, to intra prediction or inter prediction. This assignment may define the prediction type for each block. In addition, the decoder / encoder is configured to: for each intra prediction block assigned intra prediction, derive / encode from / to the data stream an associated intra prediction mode in the set of intra prediction modes.

[0017] For each predetermined intra prediction block for which its associated intra prediction mode is included in a predetermined subset of predetermined intra prediction modes, the decoder / encoder is configured to derive / encode information regarding the splitting of the corresponding predetermined intra prediction block into partitions from / to the data stream. The information regarding the splitting may indicate a horizontal split, a vertical split, or a quad split. Additionally, for each predetermined intra prediction block, the encoder / decoder is configured to perform intra prediction on each partition of the corresponding predetermined intra prediction block in a manner that depends on the predetermined intra prediction mode assigned to the corresponding predetermined intra prediction block. The predetermined intra prediction mode of the corresponding predetermined intra prediction block may be represented by an intra prediction mode from a set of intra prediction modes of the predetermined intra prediction block. For each partition of the corresponding predetermined intra prediction block, the same intra prediction mode (i.e., the predetermined intra prediction mode) may be used for intra prediction. The predetermined subset of predetermined intra prediction modes is, for example, a subset of the set of intra prediction modes. For each predetermined intra prediction block, the predetermined subset of predetermined intra prediction modes defines, for example, a list of the most likely intra prediction modes from the set of intra prediction modes of the corresponding predetermined intra prediction block.

[0018] The decoder / encoder is configured to, for each block, derive / encode information about the prediction residual of the corresponding block from / to the data stream by deriving / encoding the prediction residual of each intra-prediction block within a predetermined frame, wherein the prediction residual of each intra-prediction block within a predetermined frame is derived / encoded by: for each partition of the corresponding intra-prediction block within a predetermined frame, deriving / encoding a partition-specific prediction residual signal related to the spatial-domain prediction residual signal of the corresponding partition of the corresponding intra-prediction block within a predetermined frame from / to the data stream via a predetermined transform. For each partition of the corresponding intra-prediction block within a predetermined frame, the partition-specific prediction residual signal may be in the transform domain. In addition, the decoder / encoder is configured to, for each block, derive / encode information about the prediction residual of the corresponding block from / to the data stream by deriving / encoding the prediction residual of each intra-prediction block within a predetermined frame, and the prediction residual of each intra-prediction block within a predetermined frame is derived / encoded by: deriving / encoding information about a predetermined transform in a set of transforms that identifies the transform, the set of transforms including a first transform and a second transform, the second transform being equal to the concatenation of a primary transform and a secondary transform applied to a subset of the coefficients of the primary transform. The primary transform may be the first transform. In this case, the second transform may be equal to the concatenation of the first transform and a secondary transform applied to a subset of the coefficients of the first transform. In the case where the predetermined transform is the second transform, the decoder / encoder may be configured to use the primary transform and the secondary transform separately, or use one transform that combines the primary transform and the secondary transform, which results in the same outcome. Alternatively, the predetermined transform is the first transform, and the predetermined transform of the corresponding partition does not include a secondary transform. The decoder / encoder may be configured to: derive / encode information about a predetermined transform in a set of transforms that identifies the transform from / to the data stream by reading / transmitting a transform syntax element for the corresponding intra-prediction block transmitted in the data stream. The information about a predetermined transform in a set of transforms that identifies the transform may be an index pointing to a list of transforms, wherein the list of transforms may be equal to the set of transforms or include a subset of the set of transforms. The transforms included in the decoder transform set may be the inverse transforms of the transforms included in the encoder transform set.

[0019] In addition, the decoder is configured to reconstruct each block using the prediction signal obtained by using the information about the prediction residual of the corresponding block and the prediction type assigned to the corresponding block. For each block encoded by the encoder, the corresponding block is reconstructible using the prediction signal obtained by using the information about the prediction residual of the corresponding block and the prediction type assigned to the corresponding block.

[0020] Information identifying a predetermined transform in a set of transforms may be derived / encoded either for the respective intra prediction block or for each partition of the respective intra prediction block individually. In other words, the decoder / encoder may be configured to perform the derivation / encoding of information identifying a predetermined transform in a set of transforms from / to the data stream on a per partition basis or globally for the block. If the information is derived / encoded globally for the block, the predetermined transform is the same for each partition of the respective intra prediction block.

[0021] According to an embodiment, information identifying a predetermined transform in a set of transforms may be derived / encoded by reading / transmitting from / to the data stream a secondary transform flag in the data stream (transmitted) for the respective intra prediction block in the data stream, the secondary transform flag indicating whether the predetermined transform is a first transform or a second transform. The secondary transform flag may globally for the block indicate whether the second transform is used. In the case where the use of the second transform is indicated, the decoder / encoder may be configured to derive / encode information identifying the predetermined transform by deriving / encoding information identifying the second transform for each partition. The information identifying the second transform may indicate a primary transform and / or a secondary transform. If the primary transform and / or the secondary transform cannot be set or selected by default or based on an associated intra prediction mode, block dimensions, partition dimensions, partition processing order, and / or splitting, the information identifying the second transform may be derived from the data stream / signaled into the data stream. According to an embodiment, the primary transform is set and the secondary transform is selected based on the information identifying the second transform, or the secondary transform is set and the primary transform is selected based on the information identifying the second transform.

[0022] According to a second aspect, the inventors of the present application have recognized that one problem encountered when splitting a block stems from the fact that the current design of intra sub-partitions (ISPs) only allows horizontal or vertical splits that result in partitions having different horizontal and vertical dimensions. According to the second aspect of the present application, this difficulty is overcome by introducing a four-way split of the block in both the horizontal and vertical dimensions. Such a four-way split may result in partitions having the same horizontal dimension and the same vertical dimension. Thus, partitions of the same size may be achieved. Compared to only selecting between horizontal and vertical splits, introducing a third split, which may be referred to as a four-way split, requires additional signaling overhead by having to distinguish between horizontal splits, vertical splits, and four-way splits, and thus may require additional syntax elements or syntax elements with an increased number of states, but it has been found that higher coding efficiency and a reduced bit rate may be achieved.

[0023] Accordingly, in a second aspect of the present application, there is provided an encoder for encoding a picture and a decoder for decoding a picture. The decoder / encoder is configured to derive / encode, from / in the data stream into which the picture is encoded, an assignment of the picture, at the granularity of the blocks into which the picture is subdivided, to a set of prediction types including intra prediction and inter prediction, such that each block is assigned to an associated prediction type in the set of prediction types. Thus, each block of the picture is assigned, for example, to intra prediction or inter prediction. This assignment can define the prediction type for each block. Further, the decoder / encoder is configured to derive / encode, from / in the data stream, an associated intra prediction mode in the set of intra prediction modes for each intra prediction block to which intra prediction is assigned.

[0024] For each predetermined intra prediction block whose associated intra prediction mode is included in a predetermined subset of predetermined intra prediction modes, the decoder / encoder is configured to derive / encode, from / in the data stream, information identifying the partitioning of the respective predetermined intra prediction block to partitions in a set of partitioning patterns, the set of partitioning patterns including: a first partitioning, a second partitioning, and a third partitioning, wherein according to the first partitioning, the respective predetermined intra prediction block is horizontally split such that the partitions of the respective predetermined intra prediction block are as wide as the respective predetermined intra prediction block; according to the second partitioning, the respective predetermined intra prediction block is vertically split such that the partitions of the respective predetermined intra prediction block are as high as the respective predetermined intra prediction block; and according to the third partitioning, the respective predetermined intra prediction block is horizontally and vertically split such that the partitions of the respective predetermined intra prediction block are arranged in partition rows and partition columns. Under the horizontal split, the respective predetermined intra prediction block is split along the vertical dimension, resulting in n partitions that have the same horizontal size as the respective predetermined intra prediction block and a reduced vertical size equal to 1 / n of the vertical size of the respective predetermined intra prediction block. Under the vertical split, the respective predetermined intra prediction block is split along the horizontal dimension, resulting in n partitions that have the same vertical size as the respective predetermined intra prediction block and a reduced horizontal size equal to 1 / n of the horizontal size of the respective predetermined intra prediction block. Under the third partitioning, the respective predetermined intra prediction block may be split into n parts along the horizontal dimension and n parts along the vertical dimension, resulting in n 2 partitions, the n 2 partitions having a reduced horizontal size equal to 1 / n of the horizontal size of the respective predetermined intra prediction block and a reduced vertical size equal to 1 / n of the vertical size of the respective predetermined intra prediction block.

[0025] In addition, the decoder / encoder is configured to, for each predetermined intra prediction block whose associated intra prediction mode is included in a predetermined subset of predetermined intra prediction modes, perform intra prediction on each partition of the corresponding predetermined intra prediction block in a manner depending on the predetermined intra prediction mode assigned to the corresponding predetermined intra prediction block.

[0026] In addition, the decoder / encoder is configured to, for each block, derive / encode information on the prediction residual of the corresponding block from / to the data stream by deriving / encoding the prediction residual of each predetermined intra prediction block, wherein the prediction residual of each predetermined intra prediction block is derived / encoded by: for each partition of the corresponding predetermined intra prediction block, deriving / encoding a partition-specific prediction residual signal related to the spatial prediction residual signal of the corresponding partition of the corresponding predetermined intra prediction block from / to the data stream via a predetermined transform.

[0027] In addition, the decoder is configured to reconstruct each block using the information on the prediction residual of the corresponding block and a prediction signal obtained using the prediction type assigned to the corresponding block. For each block encoded by the encoder, the corresponding block is reconstructable using the information on the prediction residual of the corresponding block and a prediction signal obtained using the prediction type assigned to the corresponding block.

[0028] According to a third aspect, the inventors of the present application have realized that a problem encountered when encoding or decoding a block using intra sub-partition (ISP) stems from the fact that the current design of intra sub-partition (ISP) only allows the same intra prediction mode to be used for all partitions of a block. According to the third aspect of the present application, this difficulty is overcome by determining a partition-specific intra prediction mode for each partition of a predetermined intra prediction block using a predetermined rule. According to the predetermined rule, the partition-specific intra prediction mode is determined based on the block global intra prediction mode. With this feature, local variations between the partitions of a block can be considered when performing intra prediction on the block. The determination of individual intra prediction for each partition is guided by information in the data stream, such as an additional syntax element indicating the use of an individual intra prediction mode and / or an additional syntax element indicating a predetermined rule in a set of rules. Although there is an associated additional signaling overhead, it has been found that higher coding efficiency and coding quality can be achieved.

[0029] Accordingly, in a third aspect of the present application, there is provided an encoder for encoding a picture and a decoder for decoding a picture. The decoder / encoder is configured to derive / encode, from / in the data stream into which the picture is encoded, an assignment of the picture, in terms of the blocks into which the picture is divided, to a set of prediction types including intra prediction and inter prediction, such that each block is assigned to an associated prediction type in the set of prediction types. Thus, each block of the picture is assigned, for example, to intra prediction or inter prediction. This assignment can define a prediction type for each block. Further, the decoder / encoder is configured to derive / encode, from / in the data stream, an associated intra prediction mode in the set of intra prediction modes for each intra prediction block to which intra prediction is assigned.

[0030] For each predetermined intra prediction block whose associated intra prediction mode is included in a predetermined subset of predetermined intra prediction modes, the decoder / encoder is configured to derive / encode, from / in the data stream, information about the division of the corresponding predetermined intra prediction block into partitions, and to derive / encode, from / in the data stream, information of a predetermined rule in a set of identification rules, using which, for each partition of the corresponding predetermined intra prediction block, a partition-specific intra prediction mode is determined from the predetermined subset of predetermined intra prediction modes based on the associated intra prediction mode of the corresponding predetermined intra prediction block. The predetermined intra prediction modes in the predetermined subset of predetermined intra prediction modes can be sorted in a list, for example. It is possible to group similar intra prediction modes together. Using the predetermined rule, for each partition, a predetermined intra prediction mode close to the block-global predetermined intra prediction mode in the list can be determined. Thus, local variations of the intra prediction mode can be considered for each partition. Further, the decoder / encoder is configured to perform intra prediction on each partition of the corresponding predetermined intra prediction block using the partition-specific intra prediction mode determined for each partition of the corresponding predetermined intra prediction block.

[0031] In addition, the decoder / encoder is configured to derive / encode, from / in the data stream, information about the prediction residual of the corresponding block for each block by deriving / encoding the prediction residual of each predetermined intra prediction block, wherein the prediction residual of each predetermined intra prediction block is derived / encoded by deriving / encoding, from / in the data stream, a partition-specific prediction residual signal associated with the spatial-domain prediction residual signal of the corresponding partition of the corresponding predetermined intra prediction block via a predetermined transform.

[0032] In addition, the decoder is configured to reconstruct each block using the prediction residual information for the corresponding block and the prediction signal obtained using the prediction type assigned to the corresponding block. For each block encoded by the encoder, the corresponding block is reconstructable using the prediction residual information for the corresponding block and the prediction signal obtained using the prediction type assigned to the corresponding block. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The drawings are not necessarily to scale, and instead generally focus on illustrating the principles of the present invention. In the following description, various embodiments of the present invention are described with reference to the following drawings, in which:

[0034] Figure 1 A schematic diagram of an encoder is shown;

[0035] Figure 2 A schematic diagram of a decoder is shown;

[0036] Figure 3 A schematic diagram of reconstructing an image by a decoder based on blocks is shown;

[0037] Figure 4 A schematic diagram of the use of a secondary transform for each partition of a block of an image to be decoded according to an embodiment is shown;

[0038] Figure 5 A schematic diagram of non-zero transform domain regions located in partitions of blocks of an image according to an embodiment is shown;

[0039] Figure 6 A schematic diagram of the use of a third segmentation of a block of an image according to an embodiment is shown;

[0040] Figure 7 A schematic diagram of the use of a partition-specific intra prediction mode according to an embodiment is shown;

[0041] Figure 8 A schematic diagram of the use of a partition-specific angular intra prediction mode according to an embodiment is shown;

[0042] Figure 9 A block diagram of a method for decoding an image using a secondary transform for each partition of a block of an image to be decoded according to an embodiment is shown;

[0043] Figure 10 A block diagram of a method for encoding an image using a secondary transform for each partition of a block of an image to be encoded according to an embodiment is shown;

[0044] Figure 11 A block diagram of a method for decoding an image using a third segmentation for a block of an image according to an embodiment is shown;

[0045] Figure 12 A block diagram showing a method for encoding a picture using a third partition for blocks of the picture according to an embodiment;

[0046] Figure 13 A block diagram showing a method for decoding a picture using a partition - specific intra - prediction mode for blocks of the picture according to an embodiment;

[0047] Figure 14 A block diagram showing a method for encoding a picture using a partition - specific intra - prediction mode for blocks of the picture according to an embodiment; and

[0048] Figure 15 A coding unit syntax according to an embodiment is shown. DETAILED DESCRIPTION

[0049] Even if reference numerals appear in different figures, in the following description, the same or equivalent elements or elements having the same or equivalent functions are denoted by the same or equivalent reference numerals.

[0050] In the following description, numerous details are set forth to provide a more thorough explanation of embodiments of the present invention. However, it will be clear to those skilled in the art that the embodiments of the present invention can be practiced without these specific details. In other instances, well - known structures and devices are shown in block diagram form rather than specifically, to avoid obscuring the embodiments of the present invention. In addition, unless otherwise specifically indicated, the features of different embodiments described below can be combined with each other.

[0051] The following description of the drawings begins with the presentation of a description of an encoder and a decoder for block - based predictive coding, which is used to encode pictures of a video to form an example of an encoding framework in which embodiments of the present invention can be incorporated. Figures 1 to 3 The corresponding encoder and decoder are described. Hereinafter, with respect to Figures 4 to 8 the description of embodiments of the concepts of the present invention, together with how these concepts are separately incorporated into Figure 1 and Figure 2 the encoder and decoder of Figure 4 is presented, although the embodiments described using the subsequent Figure 1 and Figure 2 and the subsequent drawings can also be used to form encoders and decoders that operate without an encoding framework underlying the encoders and decoders of

[0052] Figure 1 A device (e.g., a video encoder) that exemplarily uses transform - based residual coding to predict - encode picture 12 into data stream 14 is shown. This device or encoder is denoted by reference numeral 10. Figure 2Shows the corresponding decoder 20, i.e., the device 20 configured to also predict the decoded picture 12' from the data stream 14 using transform-based residual decoding, where an apostrophe has been used to indicate that the picture 12' reconstructed by the decoder 20 deviates from the picture 12 initially encoded by the device 10 in terms of the coding loss introduced by the quantization of the prediction residual signal. Figure 1 and Figure 2 Exemplarily, transform-based prediction residual coding is used, but embodiments of the present application are not limited to this kind of prediction residual coding. As will be outlined below, the same applies to Figure 1 and Figure 2 the other details described.

[0053] The encoder 10 is configured to perform a spatial-to-spectral transformation on the prediction residual signal and encode the thus obtained prediction residual signal into the data stream 14. Similarly, the decoder 20 is configured to decode the prediction residual signal from the data stream 14 and perform a spectral-to-spatial transformation on the thus obtained prediction residual signal.

[0054] Internally, the encoder 10 may include a prediction residual signal former 22 that generates a prediction residual 24 to measure the deviation of the prediction signal 26 from the original signal (i.e., from the picture 12), where the prediction signal 26 may be interpreted as a linear combination of a set of one or more predictor blocks according to embodiments of the present invention. The prediction residual signal former 22 may be, for example, a subtractor that subtracts the prediction signal from the original signal (i.e., from the picture 12). The encoder 10 then further includes a transformer 28 that subjects the prediction residual signal 24 to a spatial-to-spectral transformation to obtain a spectral-domain prediction residual signal 24', which is then quantized by a quantizer 32 also included in the encoder 10. The thus quantized prediction residual signal 24'' is encoded into the bitstream 14. For this purpose, the encoder 10 may optionally include an entropy encoder 34 that performs entropy encoding on the prediction residual signal transformed and quantized into the data stream 14.

[0055] The prediction signal 26 is generated by the prediction stage 36 of the encoder 10 based on the prediction residual signal 24'' encoded into the data stream 14 and decodable from the data stream 14. For this purpose, as Figure 1As shown, the prediction stage 36 may internally include a dequantizer 38 that dequantizes the prediction residual signal 24″ to obtain a spectral domain prediction residual signal 24″′ corresponding to the signal 24′ except for the dequantization loss, and then includes an inverse transformer 40 that inverse-transforms (i.e., spectral to spatial transform) the subsequent prediction residual signal 24″′ to obtain a prediction residual signal 24″′′ corresponding to the original prediction residual signal 24 except for the dequantization loss. The combiner 42 of the prediction stage 36 then recombines the prediction signal 26 and the prediction residual signal 24″′′, for example, by addition, to obtain a reconstructed signal 46, i.e., the reconstruction of the original signal 12. The reconstructed signal 46 may correspond to the signal 12′. The prediction module 44 of the prediction stage 36 then generates the prediction signal 26 based on the signal 46 by using, for example, spatial prediction (i.e., intra-picture prediction) and / or temporal prediction (i.e., inter-picture prediction).

[0056] Similarly, as Figure 2 shown, the decoder 20 may internally include components corresponding to the prediction stage 36 and interconnected in a manner corresponding to the prediction stage 36. Specifically, the entropy decoder 50 of the decoder 20 may entropy-decode the quantized spectral domain prediction residual signal 24″ from the data stream, and then the dequantizer 52, the inverse transformer 54, the combiner 56, and the prediction module 58, which are interconnected and cooperate in the manner described above for the modules of the prediction stage 36, recover the reconstructed signal based on the prediction residual signal 24″, so that as Figure 2 shown, the output of the combiner 56 generates the reconstructed signal (i.e., the picture 12′).

[0057] Although not specifically described above, it is readily apparent that the encoder 10 may set some encoding parameters including, for example, prediction modes, motion parameters, etc., according to some optimization schemes (e.g., in a manner that optimizes a rate and distortion related criterion (i.e., the encoding cost)). For example, the encoder 10 and the decoder 20 and the corresponding modules 44, 58 may respectively support different prediction modes such as the intra-frame encoding mode and the inter-frame encoding mode. The granularity at which the encoder and the decoder switch between these prediction mode types may respectively correspond to the subdivision of the pictures 12 and 12′ into coding segments or coding blocks. For example, in units of these coding segments, the picture may be subdivided into blocks encoded intra-frame and blocks encoded inter-frame.

[0058] As described below regarding Figures 4 to 8More specifically outlined, an intra-coded block is predicted based on the spatial, already encoded / decoded neighborhood (e.g., the current template) of the corresponding block (e.g., the current block). Several intra-coding modes can exist and are selected for the corresponding intra-coding segments including directional or angular intra-coding modes, and according to these modes, the sample values of the neighborhood are extrapolated along a certain direction specific to the corresponding directional intra-coding mode to fill the corresponding segments. For example, the intra-coding mode can also include one or more other modes, such as: the DC coding mode, according to which the prediction of the corresponding intra-coded block assigns a DC value to all samples within the corresponding intra-coding segment; and / or the planar intra-coding mode, according to which the prediction of the corresponding block is approximated as or determined to be the spatial distribution of the sample values described by a two-dimensional linear function at the sample positions of the corresponding intra-coded block, driving the tilt and offset of the plane defined by the two-dimensional linear function based on adjacent samples.

[0059] In contrast, an inter-coded block can be predicted, for example, temporally. For an inter-coded block, a motion vector can be signaled within the data stream 14, which indicates the spatial offset of a portion of a previously encoded picture (e.g., a reference picture) of the video to which the picture 12 belongs, at which the previously encoded / decoded picture is sampled to obtain the prediction signal for the corresponding inter-coded block. This means that, in addition to the residual signal coding included in the data stream 14 (e.g., the entropy-coded transform coefficient levels representing the quantized spectral-domain prediction residual signal 24″), the data stream 14 may have encoded therein coding mode parameters for assigning coding modes to various blocks, prediction parameters for some blocks (e.g., the motion parameters of the inter-coding segments), and optionally other parameters (e.g., parameters for controlling and signaling the subdivision of the pictures 12 and 12′ into segments). The decoder 20 uses these parameters to subdivide the pictures in the same way as the encoder, assign the same prediction modes to the segments, and perform the same prediction to generate the same prediction signal.

[0060] Figure 3 Shows the relationship between (on the one hand) the reconstructed signal (i.e., the reconstructed picture 12′) and (on the other hand) the combination of the prediction residual signal 24″′′ signaled in the data stream 14 and the prediction signal 26. As already described above, this combination can be an addition. The prediction signal 26 is shown in Figure 3 as a subdivision of the picture area into intra-coded blocks illustratively indicated with hatched lines and inter-coded blocks illustratively indicated without hatched lines. This subdivision can be an arbitrary subdivision, such as a regular subdivision of the picture area into rows and columns of square or non-square blocks, or a multi-tree subdivision of the picture 12 from a root block into a plurality of leaf blocks of variable size, such as a quadtree subdivision, etc., where Figure 3shows its mixing, in which the picture region is first subdivided into rows and columns of root blocks, and then the root blocks are further subdivided into one or more leaf blocks according to a recursive multi-tree subdivision.

[0061] Similarly, the data stream 14 may have an intra-coding mode encoded therein for the intra-coded block 80, which assigns one of a number of supported intra-coding modes to the corresponding intra-coded block 80. For the inter-coded block 82, the data stream 14 may have one or more motion parameters encoded therein. In general, the inter-coded block 82 is not limited to being coded temporally. Alternatively, the inter-coded block 82 may be any block predicted based on a previously encoded portion outside of the current picture 12 itself, such as a previously encoded picture of the video to which the picture 12 belongs, or a picture of another view, or a lower layer in the case where the encoder and decoder are a scalable encoder and decoder, respectively.

[0062] Figure 3 The prediction residual signal 24″′′ in is also shown as a subdivision of the picture region into blocks 84. These blocks may be referred to as transform blocks in order to distinguish them from the coded blocks 80 and 82. In fact, Figure 3 shows that the encoder 10 and the decoder 20 may use two different subdivisions of the picture 12 and the picture 12′ into blocks, namely, one subdivision into coded blocks 80 and 82 respectively, and another subdivision into transform blocks 84. The two subdivisions may be the same, i.e., each coded block 80 and 82 may simultaneously form a transform block 84, but Figure 3 shows a situation where, for example, the subdivision into transform blocks 84 forms an extension of the subdivision into coded blocks 80 and 82, such that any boundary between two of the blocks 80 and 82 covers a boundary between two of the blocks 84, or alternatively stated, each block 80, 82 either coincides with one of the transform blocks 84 or coincides with a group of transform blocks 84. However, the subdivisions may also be determined or selected independently of each other, such that the transform blocks 84 may alternatively straddle the block boundaries between the blocks 80, 82. With respect to the subdivision into transform blocks 84, statements similar to those regarding the subdivision into blocks 80, 82 are thus true, i.e., the blocks 84 may be the result of a regular subdivision (arranged or not arranged into rows and columns) of the picture region into blocks, the result of a recursive multi-tree subdivision of the picture region, or a combination thereof or any other type of block partitioning method. Incidentally, it should be noted that the block residual signals 82 and 84 are not limited to square, rectangular or any other shape.

[0063] Figure 3 It is also shown that the combination of the prediction signal 26 and the prediction residual signal 24″′′ directly produces the reconstructed signal 12′. However, it should be noted that according to an alternative embodiment, more than one prediction signal 26 may be combined with the prediction residual signal 24″′′ to produce the picture 12′.

[0064] In Figure 3 , the transform block 84 shall have the following meaning. The transformers 28 and the inverse transformers 54 perform their transforms in units of these transform blocks 84. For example, many codecs use some DST (Discrete Sine Transform) or DCT (Discrete Cosine Transform) for all transform blocks 84. Some codecs allow skipping the transform, such that for some of the transform blocks 84, the prediction residual signal is directly encoded in the spatial domain. However, according to the embodiments described below regarding Figures 4 to 8 , the encoder 10 and the decoder 20 are configured in such a way that they support several transforms. For example, the transforms supported by the encoder 10 and the decoder 20 may include:

[0065] · DCT-II (or DCT-III), where DCT represents the Discrete Cosine Transform

[0066] · DST-IV, where DST represents the Discrete Sine Transform

[0067] · DCT-IV

[0068] · DST-VII

[0069] · Identity transform (IT)

[0070] Of course, although the transformer 28 will support all forward transform versions of these transforms, the decoder 20 or the inverse transformer 54 will support their corresponding backward or inverse versions:

[0071] · Inverse DCT-II (or Inverse DCT-III)

[0072] · Inverse DST-IV

[0073] · Inverse DCT-IV

[0074] · Inverse DST-VII

[0075] · Identity transform (IT)

[0076] The following description provides more details about the transforms that the encoder 10 and the decoder 20 may support. In any case, it should be noted that the set of supported transforms may include only one transform, such as a spectrum-to-space or space-to-spectrum transform, but it is also possible that the encoder or the decoder does not use any transform at all or does not use a transform for a single block 80, 82, 84.

[0077] As outlined above, Figures 1 to 3 has been presented as an example in which the inventive concept described below can be implemented to form a specific example of an encoder and a decoder according to the present application. In this regard, Figures 4 to 8 and Figure 1 and Figure 2The encoder and decoder can respectively represent possible implementations of the encoder and decoder described herein. However, Figure 1 and Figure 2 are merely examples. However, the encoder according to an embodiment of the present application can perform block-based encoding on Picture 12 using the concept of an encoder outlined in more detail with respect to Figures 4 to 8 and different from Figure 1 wherein the intra-coded block 80 is further subdivided into partitions, and / or the subdivision into partitions is performed by horizontal and vertical splitting of a single intra-coded block 80, and / or a first transform and a secondary transform are used for the partitions of a single intra-coded block 80, and / or respective intra-prediction modes are used for the partitions of a single intra-coded block 80. Similarly, the decoder according to an embodiment of the present application can perform block-based decoding on Picture 12' in the data stream 14 using the encoding concept further outlined with respect to Figures 4 to 8 but can be different from, for example, Figure 2 the decoder 20 in that the intra-coded block 80 is further subdivided into partitions, and / or the subdivision into partitions is performed by horizontal and vertical splitting of a single intra-coded block 80, and / or a first transform and a secondary transform are used for the partitions of a single intra-coded block 80, and / or respective intra-prediction modes are used for the partitions of a single intra-coded block 80.

[0078] According to one aspect, as Figure 4As shown, the data stream 112 into which the image 110 is encoded can convey the information 112a inserted into the data stream 112 by the encoder, and based on this information 112a, the decoder derives the assignment of the picture 110, at the granularity of the blocks into which the picture is subdivided, to the set 114 of prediction types. That is, the information 112a assigns each block of the picture 110 to one of the set 114 of prediction types. As described above, the set 114 of prediction types includes intra prediction 116 and inter prediction 118, such that each block is assigned to the associated prediction type in the set 114. For each intra prediction block 120, that is, for each block assigned to the intra prediction 116, the data stream 112 includes the information 112c inserted by the encoder, and based on this information 112c, the decoder derives, for the corresponding intra prediction block 120, the intra prediction mode associated with the corresponding intra prediction block 120 in the set 122 of intra prediction modes. For each predetermined intra prediction block 120, that is, for those predetermined intra prediction blocks 120 whose associated intra prediction mode is included in the subset 124 of predetermined intra prediction modes, the data stream 112 further includes the information 112d inserted into the data stream 112 by the encoder to enable the decoder to derive therefrom the partition 128 of the corresponding predetermined intra prediction block 120 into partitions 130. Note that not every intra prediction block needs to be a predetermined intra prediction block. Instead, as further described above and below, the information 112b for the intra prediction blocks conveyed in the data stream can determine whether a certain intra prediction block is a predetermined intra prediction block, i.e., an ISP block. Similarly, the subset of intra prediction modes does not need to be a suitable subset, but rather, all the intra prediction modes of the set 122 can also be members of the subset 124. The set 122 can include angular intra prediction modes 123a, planar intra prediction modes 123b, DC intra prediction modes 123c, and / or block-based intra prediction modes 123d. The subset 124 includes, for example, angular intra prediction modes 123a, planar intra prediction modes 123b, and / or DC intra prediction modes 123c.

[0079] Figure 4 Different possibilities of the partitioning are shown. Based on the partitioning 128 of the corresponding predetermined intra prediction block 120, intra prediction is performed on each partition 130 of the corresponding predetermined intra prediction block 120 in a manner depending on the predetermined intra prediction mode assigned to the corresponding predetermined intra prediction block, to obtain a partition-specific prediction signal. Some examples of the intra prediction modes are presented above, and further details are briefly elaborated below.

[0080] For each block, the data stream 112 also includes information 112e about the prediction residual of the corresponding block. For each intra-prediction block 120 within a predetermined frame, this information is transmitted in the data stream as follows. Specifically, for each partition 130 of the corresponding intra-prediction block 120 within a predetermined frame, the data stream 112 includes a partition-specific prediction residual signal 132, which is related to the spatial prediction residual signal 134 of the corresponding partition 130 via a predetermined transform T. For the corresponding intra-prediction block within a predetermined frame, the information 136 within the data stream 112 is inserted into the data stream by the encoder and derived from the data stream by the decoder, and the information 136 identifies the predetermined transform T in the set of transforms 138. The information 136 is derived, for example, by reading the transform syntax element 112f for the corresponding intra-prediction block 120 transmitted in the data stream 112. The transform syntax element 112f can be signaled globally for the block in the data stream 112 or for each partition 130 of the corresponding intra-prediction block 120. The information 136 that identifies the predetermined transform T in the set of transforms 138 can be derived partition-by-partition 130 or globally for the block 120. The set of transforms 138 includes at least a first transform T1 and a second transform T2, where the second transform T2 corresponds to the cascade of the primary transform T1 / Tp and a secondary transform Ts applied to a subset 140 of the transform coefficients of the primary transform T1 / Tp. The primary transform Tp can be the first transform T1. The first transform T1 and / or the primary transform Tp can be a separable transform such as DCT or DST, while the secondary transform can be a non-separable transform. Based on the information about the prediction residual and the prediction signal, each block can then be reconstructed by the decoder using the prediction type assigned to the corresponding block. That is, the inter-prediction block derives the prediction signal through inter-prediction, and for the intra-prediction block, the prediction signal is obtained through intra-prediction.

[0081] The transform syntax element 112f can be represented by a secondary transform flag and / or a secondary transform indication syntax element. The secondary transform flag can indicate whether the predetermined transform is the first transform T1 or the second transform T2, and the secondary transform indication syntax element can indicate that the secondary transform Ts is to be used in the case where the predetermined transform is the second transform T2.

[0082] According to an embodiment, the decoder / encoder is configured to decide, at different levels among the block level and the partition level, whether the predetermined transform T is the first transform T1 or the second transform T2, and to make a decision among different secondary transform candidates. Thus, the predetermined transform T can be the first transform T1 or the second transform T2, where the secondary transform Ts for the second transform T2 can be selected from different secondary transform candidates to determine the secondary transform Ts at different levels among the block level and the partition level.

[0083] Before continuing to describe the various possibilities of signaling transform information 136 for the ISP block, some other nodes will illustrate the possible implementations set forth above. For example, the above description briefly ignores that the intra prediction block does not necessarily have to be an ISP block. That is, frankly speaking, not every intra prediction block needs to be a predetermined intra prediction block. From the following brief discussion, this and other implementation details will become apparent.

[0084] Specifically, the decoder and encoder can support different intra prediction modes, which are summarized in Figure 4 set 122. There may be an angular intra prediction mode 123a, according to which, for the region to be intra predicted (e.g., the entire intra prediction block of a non-ISP block and the region of each individual partition 130 of the ISP block), the reference samples adjacent to the region are used to fill the predetermined block 120 to obtain the intra prediction signal for the block. Specifically, the reference samples that can be arranged along the boundary of the region (e.g., along the upper edge and left edge of the region) represent the picture content that is extrapolated or copied into the interior of the region along a predetermined direction. Before extrapolation or copying, the picture content represented by the adjacent samples can be subjected to interpolation filtering, or in other words, can be derived from the adjacent samples by means of interpolation filtering. The angular intra prediction modes 123a are different from each other in the intra prediction direction. Each angular intra prediction mode 123a can have an index associated therewith, and the association between the index and the angular intra prediction mode 123a can be such that: when the angular intra prediction modes 123a are sorted according to the associated mode index, the direction rotates monotonically clockwise or counterclockwise.

[0085] There can also be non-angular intra prediction modes, such as the planar intra prediction mode 123b, according to which, based on the aforementioned adjacent samples adjacent to the region to be intra predicted, a two-dimensional linear function defined by a horizontal slope, a vertical slope, and an offset is derived, and this linear function defines the predicted sample values within the region to be intra predicted (i.e., the region of the entire block 120 or a certain partition 130). The horizontal slope, vertical slope, and offset are derived based on the adjacent samples.

[0086] A specific non-angular intra prediction mode 123c (i.e., the DC mode) can be included in set 122. Here, a value (i.e., the quasi-DC value) is derived based on the adjacent samples, and this one DC value is attributed to all the samples within the region to be intra predicted (i.e., the region of the entire block 120 or a certain partition 130) to obtain the intra prediction signal. Although two examples of non-intra prediction modes are shown, there may be no example of non-intra prediction modes, one example, or more than two examples of non-intra prediction modes in set 122.

[0087] The intra prediction modes 123a-c form a subset 124 of the intra prediction modes supported by the encoder and the decoder. As Figure 4 shown in 123d, they can optionally compete, in a rate / distortion optimization sense, with the block-based intra prediction modes indicated using the reference symbol 123d. According to these block-based intra prediction modes 123d, a matrix-vector product between (on the one hand) a vector derived from neighboring samples and (on the other hand) a predetermined prediction matrix can be used to produce a prediction vector for predicting samples within the region to be intra predicted. Such block-based intra prediction modes 123d (if present) can be different from each other in the prediction matrix associated with the respective mode. Thus, in brief summary, the encoder and decoder according to the embodiments described herein can include the set 124 of intra prediction modes 123a to 123d.

[0088] The intra prediction block 120 can be encoded / decoded using intra prediction in the following manner. For example, the mode indication 112c can be implemented as follows. Specifically, in the case where block-based prediction modes 123d are supported, a set selection syntax element or information 112b (which can be named intra_mip_flag) can be conveyed in the data stream 120 to indicate whether any one of the subset 124 of the intra prediction modes, or any one of the modes 123d, is to be used to predict the block 120. The information 112b is Figure 4 additionally indicated relative to the information 112c. It should be noted that, from one perspective, the information 112b and the information 112c act together to indicate the intra prediction mode for the block 120 in the set 122, but since Figure 4 the block 120 in is assumed to be an ISP block, it is known that the mode of the block is in the set 124, and thus the information 112c is sufficient to indicate the mode of the block 120.

[0089] If information 112b indicates that any mode of subset 124 will be used to predict block 120, information 112c can be signaled as follows. A list of the most likely candidates in set 124 can be constructed / formed at the decoder and encoder based on the intra prediction modes of neighboring blocks that have been used for prediction and are adjacent to block 120. The neighboring blocks can be determined relative to the position of block 120 in a predetermined manner, for example by determining those neighboring blocks that cover some neighboring samples of block 120 (e.g., the samples above the top left sample of block 120), and the blocks that contain the samples to the left of the just mentioned top corner sample. Of course, this is only an example. This also applies to the number of neighboring blocks used for mode prediction, which is not limited to two for all embodiments. More than two or only one can be used. If any of these neighboring blocks is missing, the default intra prediction mode in set 124 can be used by default as an alternative to the intra prediction mode of the missing neighboring block. The same applies if any of the neighboring blocks has been encoded / decoded using an inter prediction mode (e.g., by motion compensated prediction). The construction of the list of the most likely modes in set 122 can be such that the list length (i.e., the number of the most likely modes therein) can be fixed by default. An index in the data stream can indicate one of the modes in the list to be used for block 120. Indexing is performed along the list order or sorting with a list index that is, for example, variable length encoded, such that the length of the index monotonically increases along the order 530. The list index will form part of information 112c. First, it is worth filling the most likely mode list only with the most likely modes in set 124 and placing the modes upstream relative to the modes with lower probability that are suitable for intra predicting block 120 along the list order. When deriving the modes of the list based on the modes of the neighboring blocks, any of the neighboring blocks may have been intra predicted using a block-based mode 123d. In that case, the mapping from the block-based mode 123d to the modes within set 124 can be used for list construction. If the set selection syntax element 112b indicates that a predetermined block 120 will be encoded by any mode in set 124, optionally, the MPM syntax element (which can be referred to as intra_luma_mpm_flag) can be part of information 112c, indicating whether the intra prediction mode to be used for the predetermined block 18 is within the MPM list, and if so, the data stream 112 includes the just mentioned list index (which can be referred to as intra_luma_mpm_idx) into the MPM list, which indicates the mode in the MPM list to be used for the predetermined block by indexing along the list order.However, if the pattern in set 124 is not within the MPM list, the data stream 112 includes another syntax element (which may be referred to as intra_luma_mpm_remainder) as part of information 112c for block 120, indicating which pattern in set 124 is to be used for block 120. This syntax element may indicate the pattern in a way that only differentiates those patterns in set 124 that are not included in the MPM list.

[0090] In another case, i.e., in the case where information 112b indicates that any one of the block-based intra prediction modes 123d is to be used to predict a predetermined block 120, block 120 can be encoded into and decoded from the data stream. To this end, an index can be used to index the selected mode among the block-based intra prediction modes 123d, or to indicate which one of the block-based intra prediction modes 123d is to be used. For this purpose, another MPM list construction can be used together with information indicating whether the selected mode among the modes 123d is within the MPM list (if so, the mode therein is the selected mode, and if not, the mode among the modes 123d is the mode of block 120).

[0091] In the case where block 120 is encoded using any pattern in subset 124, i.e., in the case where block 120 may be a predetermined intra prediction block, other syntax elements or other information can be included in the data stream, and the other syntax elements parameterize the intra prediction modes within set 124 in some way. Optionally, a syntax element (which may be referred to as intra lumaref idx) can parameterize or change the region where the above-mentioned reference samples around block 120 are located, based on which the patterns in set 124 are used for intra prediction of the interior of block 120, for example, in terms of the distance from the outer periphery of block 120. It can be part of information 112b or the reference symbol can be used in Figure 4shown in, and a syntax element that may be referred to as intra_subpartitions_mode_flag may indicate whether block 120 is an ISP block, i.e., it parameterizes whether the above-mentioned reference samples are used by the selected mode in set 124 to perform intra prediction globally or for the entire block for the interior of block 120, or whether intra prediction is performed on a per-partition 130 basis into which block 120 is subdivided, and the intra prediction is performed on these partitions 130 in turn such that the prediction residuals encoded into the data stream for one partition 130 can be used to construct / reconstruct new reference samples for intra prediction of subsequent partitions 130. According to an optional embodiment, the ISP coding option controlled by the just-mentioned ISP syntax element 112b may be available (and the corresponding syntax element may be present in the data stream) only if an optionally present syntax element that controls the position of the reference samples has a predetermined state corresponding to, for example, the region immediately adjacent to block 120 where the reference samples are located.

[0092] Partition 130 may be defined by subdividing block 120 along a predetermined direction, which may be, for example, horizontal, resulting in partition 130 being as tall as block 120, as indicated by 127b; or vertical, resulting in partition 130 being as wide as block 120, as indicated by 127a. Optionally, indicating one or more other partitions 127c and, for example, splitting into nXn partitions 130 is also feasible. If signaled that the split is active for block 120, i.e., if block 120 is the described ISP block, a syntax element that is part of information 112d may be present in the data stream, and this syntax element 112d may control the split direction used.

[0093] According to an embodiment, the split 128 of block 120 may be performed as described with respect to Figure 6 that which is described.

[0094] A description of the residual coding of the ISP block 120 will now be given. That is, the block 120 is signaled to be partitioned into partitions 130 according to the selected partition 126, and for each partition 130, the data stream 112 has encoded the partition-specific residual signal 112e therein, and the decoder decodes the residual signal from the data stream for the partition 130. As described, the decoder uses the spatial-domain residual signal 134 derived from the signal 132 in order to correct the prediction signal derived for the corresponding partition 130 using the intra prediction mode signaled for the ISP block 120, so that the next partition can perform intra prediction using reference samples adjacent to the next partition and partially residing in, for example, the previous partition. The partition-specific residual signal 132 in the transform domain is transmitted in the data stream. Information 112f is used in the data stream to signal in any of the ways described herein (e.g., on a per-partition basis) which transform T is the basis for the partition-specific signal 132, such that the transform T can be different between the partitions 130 of one ISP block 120. However, before describing different embodiments in this regard, a brief description will be given of how the partition-specific residual signal 132 is encoded into the data stream 112, i.e., what the information 112e in the data stream 112 looks like as information indicating the partition-specific residual signal 132.

[0095] Specifically, for each partition 130, the information 112e can include an encoded block flag (CBF), a last position (LP) syntax element, and the encoding of the transform coefficient levels of the transform coefficients forming the transform-domain signal 132 of the corresponding partition 130. Thus, for an ISP block 120 having K partitions 130, each partition 130 having a non-zero CBF can have K CBFs and one LP. The context used for encoding each CBF can depend on the value of the CBF of the previously encoded partition 130 along the order of prediction of the partitions within the same block 120. As the transform T for a certain partition 130, a 2-D transform can be used (except when one of the dimension sizes of the partition 130 is 1 (sample width), in which case a 1-D transform will be applied). The decoder is thus able to obtain the transform coefficient levels of the transform T for each partition 130 and also perform the inverse transform to obtain the prediction residual samples for each partition 130 in the spatial domain as shown in 134.

[0096] According to the ISP scheme, i.e., splitting into partitions 130, this scheme may be available for the current intra-coded block only when one or more conditions are met. One or more conditions may include, for example: the intra-coded block needs to be greater than a certain minimum size in terms of the number of samples of the block, and / or the intra-coded block may not be allowed to exceed a certain dimension size (at least in both the horizontal and vertical directions), so as not to cause an overly large transform size. More precisely, the ISP mode may be available only when the intra-coded block is less than or equal to the just-mentioned maximum transform-related size in at least one direction (i.e., the horizontal direction or the vertical direction). Therefore, the intra_subpartitions_mode_flag that signals the ISP mode as part of the information 112b of the intra-coded block may exist in the data stream only when the block meets the just-mentioned conditions. Otherwise, the decoder may infer that the intra-coded block is non-partitioned intra-coded. When the intra_subpartitions_mode_flag indicates that the intra-coded block 80 is an ISP-coded block, a partition dimension flag (which may be referred to as intra_subpartitions_split_flag) as part of the information 112d may also be signaled for the intra-coded block 80. However, the intra_subpartitions_mode_flag may not inevitably be explicitly signaled, but may be inferred in some cases to indicate a certain partition dimension. For example, when the width of the intra-coded block exceeds the aforementioned maximum transform size (but the height does not exceed the aforementioned maximum transform size), the partition dimension may have to be horizontal, and when the height of the block exceeds the just-mentioned maximum transform size (but the width does not exceed the maximum transform size), the partition dimension may have to be vertical. In both of these cases, the intra_subpartitions_split_flag will not be explicitly signaled in the data stream, but will be inferred by the decoder accordingly. In addition, the encoder and decoder may determine the number of partitions 130 into which the block 120 is divided depending on the size of the block 120. Thus, no signal needs to be spent in the data stream. For a small block size, this number may be 2, while the number of partitions 130 may be 4. The partition order for performing intra prediction on the partitions 130 and encoding the prediction residuals in the data stream may be along the partition direction from the leftmost partition to the farthest partition in the case of horizontal partitioning (as shown in 127b); in the case of vertical partitioning (as shown in 127a) may be along the partition direction from the uppermost partition to the farthest partition. Nor will any signal be spent for this.

[0097] As described above, the residual transformation 112e can be performed partition by partition 130. That is, each partition 130 can be separately transformed using a certain transformation T. As an intermediate note, it should be noted that in the case of a non-ISP intra-coded block, the number of transformations can also depend on the size of the block: if the non-ISP intra-coded block is smaller than the aforementioned maximum transformation size in both the horizontal and vertical directions, one transformation is used to encode the residual of the intra-coded block, i.e., the residual of the block undergoes exactly one transformation. Since the ISP block is transformed partition by partition, this may not occur in the case of the ISP block. In the case where the non-ISP intra-coded block exceeds the maximum transformation size in the horizontal direction, the non-ISP intra-coded block is horizontally divided into two halves or a corresponding number of transform blocks such that the two halves or transform blocks meet the maximum transformation size, and for each half / transform block, the residual of the block undergoes one transformation. This also applies to the case where the block exceeds the maximum transformation size in the vertical direction. If the block exceeds the maximum transformation size in both the horizontal and vertical directions, four or a corresponding number of transformations are used to transform the residual of the four quadrants of the block or the blocks in a regular two-dimensional subdivision of the block into a corresponding number of transform blocks. Of course, the processing of non-ISP intra-coded blocks deviates from the processing of ISP intra-coded blocks 120 in other ways. For example, in the case of non-ISP intra-coded blocks, no explicit signaling such as a split flag is used. According to an embodiment, non-ISP intra-coded blocks are intra-predicted block by block, while ISP blocks are intra-coded sequentially partition by partition. Other differences can be related to the coding of the transformation used to encode the prediction residual. For each transformation in the case of an ISP block (i.e., for each partition 130) and for each transformation in the case of a non-ISP intra-coded block, a coded block flag CBF such as tu_cbf_luma is transmitted, but for non-ISP intra-coded blocks, this flag may inevitably be coded for each transformation, and in the case where all previous CBFs for the previous transformation / partition are zero, for the transformation of the last partition 130 of the block 120, this flag can be inferred as 1.

[0098] Regarding the changes described above, some explanations should be made. For example, although the sequential intra coding of partitions that takes into account the prediction residuals of previous partitions when performing intra coding on the current partition has been described, the above embodiments can be modified to perform intra prediction for all or a subgroup of partitions based on reference samples located outside the ISP block (quasi-block), while the partition-based processing of the ISP block is only related to partition-based transform coding. Note that the changes just mentioned can be mixed to produce embodiments where the encoder and decoder use different ones of the above changes for different block sizes, such that, for example, there are blocks of the following sizes: intra prediction per partition is used for the block when performing ISP coding, and there are blocks of different sizes: intra prediction per block or per group of partitions is used for the block when performing ISP coding. Additionally, as often mentioned above, the signaling presented above is merely illustrative and can be implemented differently. Signaling via the use of an MPM list, for example, is merely illustrative and can be implemented without such an MPM list. Additionally, the signaling order and mutual adjustment between the syntax elements presented above can be changed.

[0099] The low-frequency non-separable transform (LFNST) is used below as the secondary transform Ts. However, it is obvious that other secondary transforms can also be used.

[0100] The low-frequency non-separable transform is a secondary transform that is only applied to the upper left corner of the primary transform coefficients, e.g., to the coefficient subset 140 of the primary transform Tp / T1. In VTM-6.0, there are various LFNST sets, which are selected according to the intra mode of the current block. Each LFNST set includes two different transforms. Therefore, the decoder has 3 options for selection according to the explicitly signaled element (0 -> do not use LFNST, 1 -> use the first LFNST, 2 -> use the second LFNST). In some cases, depending on certain conditions below, the LFNST cannot be applied to a block:

[0101] · Example 1: In a certain region 160 of block 120, the primary transform coefficients are not zero.

[0102] · Example 2: The transform coefficients obtained after applying the secondary transform violate certain conditions, such as the number of valid coefficients being greater than a certain threshold or the resulting last position syntax element being greater than a certain threshold.

[0103] In the foregoing examples and related cases, the decoder does not parse the signaling and infers that no secondary transform has been applied.

[0104] Currently in VTM-6.0 (intra sub-partition), the ISP cannot use LFNST. It is proposed to enable this algorithm on the blocks using ISP at the transform unit (TU) level. This means that if a block using ISP is split into n sub-partitions, each resulting sub-partition will use the LFNST algorithm in the same way as a normal block without using ISP, for example. In subsequent embodiments, the implicit signaling and explicit signaling of the optional selection between LFNST usage and secondary transform candidates, as well as the different possibilities of performing any of the just-mentioned decisions / signaling at the block level and partition level, are described separately.

[0105] Secondary transform T s Is applied at the sub-partition level. Thus, for example, there are syntax elements parsed for each sub-partition 130, i.e., the transform syntax element 112f, to decide whether the secondary transform T s Will be calculated, and which secondary transform to calculate in the affirmative case. Of course, if the secondary transform conditions are met, such as the number of valid coefficients being greater than a certain threshold and / or there being no non-zero coefficients outside a predetermined region, then this syntax element can be parsed (e.g., only).

[0106] As outlined above, currently in VTM-6.0 (intra sub-partition), the ISP cannot use LFNST. It is proposed herein to enable this algorithm on the blocks using ISP. This means that each sub-partition 130 of the block 120 using ISP will potentially be able to apply the secondary transform T s . This can be explicitly indicated to the decoder by having information 112f of the syntax element, but it can also be implicitly indicated by using parameters already present at the decoder or by a combination of implicit and explicit elements. In addition to the above examples, various examples are provided below.

[0107] In other words, the above different possibilities of combining ISP and LFNST are shown. Below, variants for signaling or changing the transform T for transform coding the prediction residual of the partition 130 are outlined hereinafter.

[0108] Explicitly at the sub-partition level

[0109] For each sub-partition 130, the secondary transform Ts can be explicitly signaled to the decoder. This version simply extrapolates the normal use of LFNST to each sub-partition 130 created by ISP.

[0110] That is, continue the description of transform coding the prediction residual 132 on a per-partition 130 basis. As described, the prediction residual 132 can be coded in the transform domain into the data stream 112. The transform domain is related to the spatial domain via the transform T. According to the variant just presented, the information 112f can indicate for each partition 130 whether the transform T is the first transform T1 or the primary transform Tp / T1, followed by a secondary transform Ts. In other words, for each partition, the transform syntax element 112f is read, which indicates whether the first transform T1 or the second transform T2 is used for the corresponding partition. The secondary transform Ts can be fixed by default for the block 120, or signaled once for the ISP block 120. Depending on the intra prediction mode for the block 120 in the set 124, two Ts can form a set of secondary transform candidates for the block 120, and one of them is selected for the block 120. For example, the primary transform T1 will be fixed by default for the ISP block 120. Note that in that case, when a decision is sent to apply the secondary transform Ts for the entire block 120, the decision only affects the partitions whose residual information 112e does not immediately indicate that the secondary transform may not be applied for the reasons indicated above, such as: if one of the following conditions applies to the residual transform coded by the information 112e for a certain partition, the LFNST is not available for that partition (otherwise it is available): 1) non-zero transform coefficients are outside a predetermined region 140 (as Figure 5 shown); 2) the number of non-zero coefficients exceeds a certain limit; 3) the last non-zero coefficient position along the scan path from the DC coefficient position to the opposite or highest frequency coefficient position (as indicated by one or more syntax elements (referred to herein as LP syntax elements) for the residual transform of the corresponding partition, quasi-serving as a measure of the expected count of non-zero coefficients within the transform) exceeds a certain threshold; or 4) if two or more of the conditions 1 to 3 are satisfied, the LFNST is not available; or 5) if each condition in a subset of the conditions 1 to 3 is satisfied, the LFNST is not available.

[0111] In this regard, what has been described above should be recalled. Specifically, for the partition 130 of the block 120, the CBF is transmitted in the data stream on a per-partition basis. The CBF can indicate whether the partition (e.g., transform unit) includes residual information, such as information generated by prediction. If the CBF indicates that the residual is zero, there is no need for a secondary transform flag 112f to exist for the corresponding partition 130. Additionally, as described above, the availability of the secondary transform option can depend on certain circumstances: for example, the above LP syntax elements serve as non-zero region indicators. It signals the non-zero transform domain region 160 for the corresponding partition 130, as Figure 5As shown, all non-zero coefficients of the partition-specific prediction residual signal 132 are placed only in the non-zero transform domain region 160. Correspondingly, only zero coefficients are located in this region. Depending on the extension and / or position of this non-zero transform domain region 160, the predetermined transform T must be equal to only the primary transform T1, or the secondary transform T2 is available (in this case the secondary transform flag 112f is transmitted by the encoder or read by the decoder). For example, the secondary transform Ts is available only when the non-zero transform domain region 160 does not exceed the transform coefficient portion of the transform T1 to which the secondary transform Ts is applied (i.e., Figure 4 or Figure 5 the portion 140 in). Therefore, the existence of the secondary transform flag 112f for a certain partition 130 is a partition-specific issue, and although there may be a secondary transform flag for one transform 130 of the block 120, for another partition 130 of the block 120, the secondary transform flag may not exist or may be default.

[0112] Depending on the extension and / or position of this non-zero transform domain region 160, the decoder can be configured to make a decision between: reading from the data stream 112 the secondary transform flag 112f for the corresponding partition 130 transmitted in the data stream 112, which indicates whether the predetermined transform T is the first transform T1 or the second transform T2; or inferring that the predetermined transform T is the first transform T1. If the non-zero transform domain region 160 is completely located within the region defined by the portion 140 of the transform coefficients of the transform T1 to which the secondary transform Ts is applied, the secondary transform flag 112f is read. Otherwise, it is inferred that the predetermined transform T is the first transform T1. Whether the secondary transform flag 112f indicates the first transform T1 or the second transform T2 as the predetermined transform can depend on one or more of the above conditions 1 to 5.

[0113] Alternatively or additionally, the decision between reading the secondary transform flag 112f and inferring that the predetermined transform T is the first transform T1 can depend on whether the non-zero transform domain region 160 only covers the coefficients 140 of the primary transform Tp / T1 to which the secondary transform Ts is applied by the cascaded application of the primary transform Tp / T1 and the secondary transform Ts, and / or on the number of non-zero coefficients within the non-zero transform domain region 160.

[0114] According to an embodiment, the decoder / encoder is configured to: if the dimensional size of the partition 130 is lower than a predetermined threshold, infer that the predetermined transform T is the first transform T1.

[0115] In addition, a certain flag for each partition 130 of block 120 (i.e., the secondary transform indication syntax element) may be present in the data stream, for which the secondary transform options are available and signaled to be applied, i.e., a flag for selecting the secondary transform Ts from the corresponding set of secondary transforms (i.e., the set including two possible secondary transforms). Instead of sending two flags for such a partition 130, a three-array syntax element may be used to indicate the transform for the corresponding partition, i.e., as the information 112f for that partition 130.

[0116] Before proceeding to the next option for indicating the transform T for the partition 130 of block 120, note the following additionally. Specifically, the coding of the transform coefficient levels for a certain partition 130 is limited to the non-zero part 160. When one-dimensionally scanning the transform coefficients of transform T1 from the highest spectral frequency position to the DC frequency position (e.g., the upper left corner of transform T1), the LP syntax element may indicate a part by indicating the first non-zero coefficient position. In the case of a two-dimensional T1 transform, this position may be indicated by x and y coordinates, or this position may be indicated by an index that measures the aforementioned non-zero position using the distance from the DC transform position along the one-dimensional scan order just mentioned in terms of coefficients. Also note that depending on whether the transform signal will be the combined transform T2 or the single transform T1, the transform coefficient levels for the partition 130 transmitted in the data stream 112 may be the transform coefficients of the first transform T1 or the coefficients of the second transform T2. Also note that if the transform T is signaled as a two-level transform, i.e., the second transform T2 following T1, the decoder may perform the inverse transform by sequentially performing the inverse transform of the second transform and then the inverse transform of transform T1, or performing the inverse transform of T2 combined in one inverse transform, to obtain the spatial domain residual signal 134 for the corresponding partition 130. In other words, the partition-specific prediction residual signal 132 may undergo an inverse transform T−1 that forms the inverse of the predetermined transform T to obtain the spatial domain residual signal 134.

[0117] Explicitly at the block level

[0118] The secondary transform Ts may be signaled to the decoder explicitly only once for the entire block 120. This means that the same secondary transform Ts will be applied to all sub-partitions 130 within block 120. If the secondary transform flag 112f indicates that the predetermined transform T is the second transform T2, the decoder / encoder may be configured to use the second transform T2 for all partitions 130.

[0119] This may imply that certain restrictions on the application of the secondary transform may be applied differently. For example, in this case, the aforementioned Example 2 may not be absolutely necessary, even though it may still be applied. Alternatively, they may also be applied as a whole group, or different thresholds etc. may be required.

[0120] In other words, it may include, in the data stream 112, a signal 112f for the entire block 120 once, which signals whether Ts is applied, i.e., whether T is T1 or T1 followed by Ts (i.e., T2), and if the latter is true, which Ts in the candidate set of secondary transforms is signaled. The availability limit of LFNST, i.e., the application of Ts, can be tested for each partition 130, where the block 120 globally signals that it only applies to those partitions 130 where LFNST is available. In the case where LFNST is not available for all partitions 130, the signaling 112f for the block 120 can be omitted, i.e., the encoder does not encode such an LFNST activation flag for the block 120 and the decoder infers that the flag is set to LFNST being deactivated. Even alternatively, the signaling 112f can be present for the block 120 only if it is available for all partitions 130, otherwise it is inferred that LFNST is not used. In the latter case, if the LFNST availability for all partitions 130 is applied and the block-level LFNST activation signaling indicates the activation of LFNST for the block 120, the Ts selective signaling will thus be encoded only for the block 120. The same or other criteria or other thresholds as discussed above in 1 to 5 can be used in the variant of signaling the block-level activation of LFNST only if LFNST is available for all partitions 130 of the block 120.

[0121] According to an embodiment, the decoder can be configured to decide between: reading from the data stream 112 a secondary transform flag 112f transmitted in the data stream 112 for the corresponding intra-predicted block 120 of a predetermined frame, which indicates whether the predetermined transform T is the first transform T1 or the second transform T2; or inferring that the predetermined transform T is the first transform T1. This decision can depend on, as Figure 5The relative position of the non-zero transform domain region 160 shown with respect to the coefficient 140 of the primary transform Tp / T1 to which the secondary transform Ts has been applied by cascading the primary transform Tp / T1 and the secondary transform Ts. If the non-zero transform domain region 160 is completely located within the region defined by the transform coefficients 140 of the primary transform Tp / T1 to which the secondary transform Ts has been applied, the secondary transform flag 112f is read. Otherwise, it is inferred that the predetermined transform T is the first transform T1. Alternatively or additionally, this decision may depend on whether the non-zero transform domain region 160 only covers the coefficient 140 of the primary transform Tp / T1 to which the secondary transform Ts has been applied by cascading the primary transform Tp / T1 and the secondary transform Ts, and / or on the number of non-zero coefficients within the non-zero transform domain region 160. Which of the first transform T1 or the second transform T2 is indicated as the predetermined transform by the secondary transform flag 112f may depend on one or more of the above conditions 1 to 5. In addition to the secondary transform flag 112f, a secondary transform indication syntax element may be signaled globally for the corresponding in-predicted frame prediction block 120 to indicate the exact secondary transform for all partitions 130.

[0122] The decoder may be configured to make a decision once for all partitions 130 of the in-predicted frame prediction block 120, where the above conditions on which the decision may depend may be checked for all partitions 130 of the in-predicted frame prediction block 120. According to an embodiment, by checking one or more of the following criteria for all partitions 130 of the in-predicted frame prediction block 120, and if one or more of the following criteria are satisfied for all partitions 130, it is decided to read the secondary transform flag (112f), thereby making a decision once for all partitions 130 of the in-predicted frame prediction block 120:

[0123] - No non-zero transform coefficient is located outside the predetermined region 140; and / or

[0124] - The number of non-zero coefficients exceeds a predetermined limit; and / or

[0125] - The last non-zero coefficient position along the scan path from the DC coefficient position to the highest frequency coefficient position indicated by the last position syntax element in the data stream exceeds another predetermined threshold.

[0126] As will be described in more detail in the following additional description, it may be sufficient for all partitions 130 of the in-predicted frame prediction block 120 to satisfy only some of the above criteria, and for at least one partition 130 of the in-predicted frame prediction block 120 to satisfy some criteria in order to read the secondary transform flag 112f.

[0127] According to an embodiment, the decoder / encoder is configured to: If the dimension size of the partition 130 is below a predetermined threshold, it is inferred that the predetermined transform T is the first transform T1.

[0128] Explicitly at the combined block level and sub - partition level

[0129] Explicit syntax elements can exist at both the block level and sub - partition level. For example, at the block level, it can be signaled (e.g., signaled by the secondary transform flag 112f) that all sub - partitions 130 will use or not use the secondary transform Ts. Then, if the secondary transform Ts will be used, each sub - partition 130 will explicitly signal which secondary transform will be applied (independently of each other), for example, through a secondary change indication syntax element. Of course, the global activation of the LFNST can only affect the sub - partitions 130 of block 120 whose residual information 112e meets the conditions for LFNST availability. If the conditions for LFNST availability are met for at least one sub - partition, the corresponding activation for only block 120 can be encoded into the data stream. And only for those sub - partitions, the syntax elements for explicitly selecting Ts will be encoded. Alternatively, there can be a signal 112f for block 120 only when it is available for all sub - partitions 130, otherwise it is inferred that LFNST is not used. That is, the signal for block - level LFNST activation will be signaled only when LFNST availability applies to all sub - partitions. Therefore, for each sub - partition, the Ts selection signal will be included in the data stream of block 120.

[0130] That is, as an alternative to the following scheme: signaling for each sub - partition 130 for which LFNST is available whether Ts is applied, and if so, signaling which one of the set of Ts candidates determined for the ISP block is applied as Ts. Alternatively, it can be that the primary transform T1 is fixed for all sub - partitions 130 of block 120, and the information 112f globally signals only once at the block level the decision on whether LFNST is applied to the sub - partitions 130 of block 120 for which LFSNT will be available from the perspective of the above other factors (position of non - zero parts and / or number of non - zero coefficients and / or position of the last non - zero coefficient). At the same time, if it is signaled that LFNST is applied to the sub - partitions 130 of block 120, the secondary transform Ts is changed for each such sub - partition 130 for which LFSNT is available by encoding the corresponding Ts selection syntax element for each such sub - partition.

[0131] In all cases of explicit signaling (e.g., explicitly at the sub - partition level and / or explicitly at the block level and / or explicitly at the combined block level and sub - partition level), the decoder / encoder can be configured to: select a subset of one or more candidate secondary transforms Ts from the set of candidate secondary transforms Ts in a manner that depends on the associated intra - prediction mode (i.e., the intra - prediction mode selected for block 120 from set 122 or subset 124). If the subset of one or more candidate secondary transforms Ts contains more than one candidate secondary transform Ts, then a secondary transform Ts is selected from the subset of one or more candidate secondary transforms Ts depending on the secondary transform indication syntax element for the corresponding predetermined intra - prediction block 120 transmitted in the data stream 112.

[0132] According to an embodiment, the set of candidate secondary transforms is disjoint from the secondary transforms Ts used by the decoder / encoder for other intra - prediction blocks that have been intra - predicted without partitioning.

[0133] Implicitly

[0134] When the coefficients are decoded, the decoder can use existing elements to obtain information 136 about the transform to be used, i.e., to determine whether a secondary transform is applied to each sub - partition 130. Examples of such parameters are sub - partition index, sub - partition size, block size, intra - mode, etc.

[0135] The decoder / encoder can be configured to: for each partition 130, depending on the sorting of the corresponding partition 130 along the order of intra - prediction 116 for the partition 130, the size of the corresponding partition 130, the size of the predetermined intra - prediction block 120, and / or the associated intra - prediction mode 122, determine whether the predetermined transform T is the first transform T1 or the second transform T2. The result of this determination can be signaled by the secondary transform flag 112f in the aforementioned explicit signaling. The exact secondary transform Ts can be selected depending on the associated intra - prediction mode 122, partition size, block 120 size, partition 130 processing order, and / or partitioning 128. This selection can be performed once for all partitions 130 of the predetermined intra - prediction block 120 or for each partition 130 of the predetermined intra - prediction block 120.

[0136] Implicitly / explicitly mixed

[0137] Any of the above examples or their related versions can be combined with implicit rules. For example, it can be explicitly determined whether to use a secondary transform, but the decision about which secondary transform must be used can be implicitly derived by using intra - mode, sub - partition size, block size, sub - partition index, ISP split type (horizontal or vertical), etc.

[0138] Additional notes

[0139] The following details and examples are added to the above description. For example, another example of the conditions for the availability of LFNST is added. More precisely, in the case where the use / activation of the secondary transform is explicitly signaled at the block level, it may be that the availability of LFNST (or more precisely, the transmission of the secondary transform flag 112f for block 120 to globally activate the secondary transform Ts for the block) can be defined by more than one condition a), b), c), etc. This has been described. For example, for all partitions 130 of a block, it may be required to satisfy more than one condition, such as more than one of the conditions 1 to 5 listed above. Alternatively, it may be required that more than one condition be true or satisfied in at least one of the partitions. However, it may also be that one or more conditions (e.g., a) and b)) should be true in all partitions, while one or more other conditions (e.g., c)) only need to be true in at least one partition.

[0140] For the case where a certain condition must be true or satisfied only in at least one partition 130, the LFNST signaled for the entire block 120 (if signaled as activated) can be applied to all partitions 130 within block 120, regardless of the individual availability conditions of each partition 130. Another possibility is that it can be applied only to those partitions 130 that satisfy the condition. If no partition 130 satisfies the availability condition, then the LFNST is not signaled (or parsed), and it is assumed that the LFNST is not used.

[0141] The decoder / encoder can be configured to, for example, perform the above decision globally for the block once between reading the secondary transform flag 112f and inferring that the predetermined transform T is the first transform T1 for all partitions 130 of a predetermined intra prediction block 120 in the following manner: check one or more of the following criteria a to c for all partitions 130, and if one or more of the following first criteria a to c are satisfied for all partitions 130, while one or more of the following second criteria a to c are satisfied for at least one partition, then decide to read the secondary transform flag 112f:

[0142] a) No non-zero transform coefficients are located outside the predetermined region 140; and / or

[0143] b) The last non-zero coefficient position along the scan path from the DC coefficient position to the highest frequency coefficient position indicated by the last position syntax element in the data stream exceeds another predetermined threshold; and / or

[0144] c) The number of non-zero coefficients exceeds a predetermined limit

[0145] According to an embodiment, the decoder / encoder is configured to determine to read the secondary transform flag 112f if one or more first criteria among criteria a to c are satisfied for all partitions 130, and / or if one or more second criteria among criteria a to c are satisfied for at least one partition 130. Thus, it is possible that the secondary transform flag 112f is read if one or more of conditions a to c are satisfied for all partitions 130, without at least one of the partitions 130 having to satisfy one or more additional second criteria. Alternatively, it is also possible that the secondary transform flag 112f for the entire block 120 is read if one or more of the second criteria are satisfied for at least one partition 130, without all partitions 130 having to satisfy at least one of the first criteria.

[0146] If the secondary transform flag 112f indicates that the predetermined transform T is the second transform T2, the second transform T2 is used for the partitions 130 that satisfy one or more second criteria, and the first transform T1 is used for the partitions 130 that do not satisfy at least one of the one or more second criteria.

[0147] Example 1:

[0148] Assume a block 120 having N partitions 130, and the availability of LFNST is defined by the following conditions

[0149] a) In all partitions 130, all coefficients outside the predetermined region 140 must be zero

[0150] b) At least one partition 130 must have a last significant position (in scan order) greater than a certain threshold.

[0151] In this example, condition a) must be true in all partitions 130 for LFNST to be available, but condition b) is only necessary in at least one partition.

[0152] According to an embodiment, one or more first criteria are that there are no non-zero transform coefficients outside the predetermined region 140, and one or more second criteria are that the last non-zero coefficient position (indicated by the last position syntax element in the data stream 112) along the scan path from the DC coefficient position to the highest frequency coefficient position exceeds another predetermined threshold.

[0153] Example 2:

[0154] Assume a block 120 having N partitions 130, and the availability of LFNST is defined by the following conditions

[0155] a) In all partitions 130, all coefficients outside the predetermined region 140 must be zero

[0156] b) In all partitions, the last valid position (in scan order) must be greater than a certain threshold.

[0157] In this example, a) and b) must be true in all partitions 130.

[0158] Example 3:

[0159] Suppose a block 120 with N partitions 130, and the availability of LFNST is defined by the following conditions

[0160] a) In all partitions 130, all coefficients outside the predetermined area 140 must be zero.

[0161] In this example, only the condition a) exists, and it must be true for all partitions 130 for LFNST to be available.

[0162] Regarding using a new kernel or an existing kernel

[0163] The sub - partitions 130 to which the secondary transform Ts is applied can share the same secondary transform as the non - ISP blocks, or they can have a dedicated secondary transform table only for the ISP sub - partitions 130. This can be applied to all cases, or may only be applied to a subset of them. For example, for sub - partitions of 1xN, 2xN, Nx2, and Nx1 (or any subset depending on the dimensional size of the sub - partition for this problem), or may depend on the intra - frame mode or sub - partition index or ISP split type (horizontal or vertical), or other parameters available at the decoder side. That is, compared with non - ISP blocks encoded with the same intra - frame coding mode associated with them, in the case where a certain block is encoded / decoded as an ISP (predetermined) block, non - overlapping Ts candidates can be used for this block.

[0164] There are differences between non - ISP blocks and ISP blocks regarding the use of LFNST.

[0165] From the perspective of the encoder, LFNST is the application of a "secondary" transform to "primary" transform coefficients. This is subject to the lfnst index, i.e., the transform syntax element 112f, which can take 3 different values (0 -> no LFNST applied, 1 -> apply LFNST1, 2 -> apply LFNST2). In other words, the transform syntax element 112f can indicate whether the second transform T2 is used for this block. If the transform syntax element 112f is in the first state (e.g., 0), it indicates the use of the first transform T1, while if the transform syntax element 112f is in the second state (e.g., 1) or the third state (e.g., 2), it indicates the use of the second transform T2. LFNST1 and LFNST2 depend on the intra - frame mode. That is, the basis functions for the transform vary with the intra - frame mode.

[0166] Furthermore, if certain availability conditions are not met, the value of the LFNST index can be implicitly set to 0. In other words, if the following availability conditions are not met, the decoder / encoder can infer the use of the first transformation T1 for non-ISP blocks:

[0167] · Condition 1: There must be no non-zero coefficients outside a predetermined range (which depends on the block size).

[0168] · Condition 2: There must be at least one non-zero coefficient that is not the DC coefficient (0, 0).

[0169] The use of LFNST in ISP blocks is the same as in non-ISP blocks, but with the following differences:

[0170] 1) LFNST is signaled globally only once for the entire block. In other words, the transform syntax element 112f is derived / signaled globally for the block.

[0171] 2) The availability of the LFNST index (i.e., the transform syntax element 112f) can depend on the following conditions. The conditions that must occur for the LFNST index to be resolvable (otherwise, it is assumed to be 0) are as follows:

[0172] a. The size of the sub-partition must be at least 4×4

[0173] b. Condition 1 (as described above) must be satisfied for all sub-partitions. Thus, if at least one sub-partition violates it, LFNST0 (no LFNST) will be used for all sub-partitions.

[0174] c. For the ISP case, Condition 2 (as described above) is ignored. In other words, for the availability of the transform syntax element 112f, it is not necessary for each partition to include a non-zero coefficient that is not the DC coefficient (0, 0).

[0175] 3) If the LFNST index is resolved, the following occurs:

[0176] a. The index is 0

[0177] i. No LFNST is used for any sub-partition

[0178] b. The index is 1

[0179] i. For each sub-partition, if the CBF (Coded Block Flag) of the sub-partition

[0180] is non-zero, then LFNST1, such as the first variant of the second transformation T2, is applied

[0181] c. The index is 2

[0182] i. For each sub - partition, if the CBF of the sub - partition is not zero, apply LFNST2, e.g., the second variant of the second transform T2

[0183] If the LFNST index is 1 or 2, the primary transform used for each sub - partition will be DCT2.

[0184] According to an embodiment, according to Figure 15 the coding unit syntax shown, parse the transform syntax element 112f, i.e., lfnst_idx.

[0185] According to an embodiment, the value of lfnst_idx 112f discussed above can be binarized, for example, using a truncated unary codeword of one or two binary values (bin), where the first binary value can indicate whether the predetermined transform is the first transform T1 or the second transform T2, and the second binary value can indicate which second transform T2 is to be used for block 120 or partition 130. The first binary value can represent the secondary transform flag, and the second binary value can represent the secondary transform indication flag, i.e., the secondary transform indication syntax element. The binarized lfnst_idx 112f (e.g., the first binary value and / or the second binary value) can be entropy - encoded into the data stream using context - adaptive binary arithmetic coding (CABAC). It is obvious that this is only an example, and lfnst - idx112f can be signaled in a different way in the data stream.

[0186] Figure 15The shown function transform_tree113 is recursive and loops over all sub - partitions 130. Therefore, lfnst_idx 112f is signaled only once at the end of block 120. It can be checked whether both the width and height of the sub - partitions 130 of the transform block 120 are at least 4. If both the width and height of each sub - partition 130 of the transform block 120 are at least 4, then lfnst_idx 112f can be signaled only in the data stream or derived by the decoder from the data stream. If the predetermined block is an ISP block, i.e., the intra - prediction block 120 is split into partitions 130 using split 128, then this condition can be violated only in the luminance component. In the case of ISP, i.e., for the predetermined block 120 that is split 128, condition 2 is ignored (see LfnstDcOnly 313). In other words, for the predetermined block 120 that is split 128, information 136 identifying the predetermined transform T can be derived from / signaled to the data stream without controlling whether there is at least one non - zero coefficient other than the DC coefficient (0, 0) in each partition 130. However, for each partition, it is checked whether the CBF (Coded Block Flag) of the corresponding partition is non - zero. Information 136 identifying the predetermined transform T can be derived from / signaled to the data stream only for partitions that include at least one non - zero transform coefficient. Note that in VVC, blocks using ISP always have at least one partition with a non - zero CBF.

[0187] The transformation process of the scalable transform coefficients is described below. The main transform selection for ISP may be affected by the lfnst_idx syntax element 112f. For the horizontal and vertical directions of block 120 or each sub - partition 130, the main transform is specified by trTypeHor and trTypeVer. The main transform is a separable transform.

[0188] The variable trTypeHor that specifies the horizontal transform kernel and the variable trTypeVer that specifies the vertical transform kernel are derived as follows:

[0189] - If one or more of the following conditions are true, then trTypeHor and trTypeVer are set equal to 0.

[0190] - cIdx is greater than 0

[0191] - IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT, and lfnst_idx is not equal to 0

[0192] - Otherwise, if implicitMtsEnabled is equal to 1, then the following is applied:

[0193] - If cu_sbt_flag is equal to 1, then depending on cu_sbt_horizontal_flag and cu_sbt_pos_flag, trTypeHor and trTypeVer are specified in Table 40.

[0194] - Otherwise (cu_sbt_flag is equal to 0), trTypeHor and trTypeVer are derived as follows:

[0195] trTypeHor = (nTbW >= 4 && nTbW <= 16)? 1 : 0

[0196] trTypeVer = (nTbH >= 4 && nTbH <= 16)? 1 : 0

[0197] - Otherwise, depending on mts_idx, trTypeHor and trTypeVer are specified in Table 39 of the VVC standard.

[0198] For the ISP case, if lfnst_idx > 0, then trTypeHor and trTypeVer are set to 0 (0 represents DCT-II). Thus, the in-frame prediction and the predetermined block 120 after partitioning will be used as the horizontal transform and the vertical transform with DCT-II as the primary transform. The primary transform and the first transform will be the same transform.

[0199] The flag that makes (DCT-II, DCT-II) the primary transform is called spsmts_enabled_flag (MTS means "multiple transform selection"). If sps_mts_enabled_flag is 0, then in all cases the primary transform is (DCT-II, DCT-II), which is completely independent of the fact of whether lfnst is used or not. In other words, the primary transform Tp is equal to the first transform T1. Thus, the decoder / encoder is configured to: derive / encode from / to the data stream the information 136 that identifies the predetermined transform T in the set 138 of transforms that includes the first transform T1 and the second transform T2, where the second transform T2 is equal to the concatenation of the first transform T1 and the secondary transform Ts applied to the subset 140 of the coefficients of the first transform T1. Thus, in this case, for the cases of lfnst_idx = 0 and lfnst_idx > 0, DCT-II is used as the primary transform.

[0200] The sps_mts_enabled_flag being equal to 1 specifies that the sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag exist in the SPS. The sps_mts_enabled_flag being equal to 0 specifies that the sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag do not exist in the SPS.

[0201] During the transformation of the scaling transform coefficients, setting the HLS flag to 0 prevents setting the values of trTypeHor and trTypeVer:

[0202] The variable implicitMtsEnabled is derived as follows:

[0203] - If the sps_mts_enabled_flag is equal to 1 and one or more of the following conditions are true, then implicitMtsEnabled is set to be equal to 1:

[0204] - IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT

[0205] - cu_sbt_flag is equal to 1 and Max(nTbW, nTbH) is less than or equal to 32

[0206] - sps_explicit_mts_intra_enabled_flag is equal to 0 and CuPredMode[0][xTbY][yTbY] is equal to MODE_INTRA, and

[0207] lfnst_idx[x0][y0] is equal to 0 and intra_mip_flag[x0][y0] is equal to 0

[0208] - Otherwise, implicitMtsEnabled is set to be equal to 0. The variable implicitMtsEnabled remains at the 0 value, which prevents the main transform from being anything other than DCT-II.

[0209] The variable trTypeHor that specifies the horizontal transform kernel and the variable trTypeVer that specifies the vertical transform kernel are derived as follows:

[0210] - If one or more of the following conditions are true, then trTypeHor and trTypeVer are set to be equal to 0.

[0211] - cIdx is greater than 0

[0212] - IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT and lfnst_idx is not equal to 0 (if lfnst_idx is not 0, this makes the main transform DCT-II. This is independent of implicitMtsEnabled. The main transform is the first transform)

[0213] - Otherwise, if implicitMtsEnabled is equal to 1, then the following is applied:

[0214] - If cu_sbt_flag is equal to 1, then depending on cu_sbt_horizontal_flag and cu_sbt_pos_flag, trTypeHor and trTypeVer are specified in Table 40.

[0215] - Otherwise (cu_sbt_flag is equal to 0), trTypeHor and trTypeVer are derived as follows:

[0216] trTypeHor = (nTbW >= 4 && nTbW <= 16)? 1 : 0

[0217] trTypeVer = (nTbH >= 4 && nTbH <= 16)? 1 : 0

[0218] - Otherwise, depending on mts_idx, trTypeHor and trTypeVer are specified in Table 39 of the VVC standard. The transform of the predetermined block 120 (i.e., trTypeHor and trTypeVer) is set to its default value (for ISP, mts_idx is always 0). Thus, trTypeHor and trTypeVer are set to 0, i.e., DCT-II.

[0219] Therefore, it is obvious that the main transforms of the first transform and the second transform can be the same transform. The decoder / encoder is configured to use the first transform as the main transform.

[0220] The second transform can be applied according to the following characteristics independently of whether the block 120 is split 128 (i.e., whether it is an ISP block):

[0221] Specifically, using 1fnst_idx 112f, the variable ApplyLfnstFlag is derived as follows:

[0222] - If treeType is equal to SINGLE_TREE, then the following is applied:

[0223] ApplyLfnstFlag = (lfnst_idx > 0 && cIdx == 0)? 1 : 0

[0224] - Otherwise, apply the following formula:

[0225] ApplyLfnstFlag = (lfnst_idx > 0)? 1 : 0

[0226] During the transformation process of the scaling transformation coefficient, ApplyLfhstFlag prepares variables for the transformation process, and ApplyLfnstFlag calls it:

[0227] When ApplyLfhstFlag is equal to 1, transform_skip_flag[xTbY][yTbY][cIdx] is equal to 0, and both nTbW and nTbH are greater than or equal to 4, apply the following:

[0228] - The variables predModeIntra, nLfnstOutSize, log2LfnstSize, nLfnstSize, and nonZeroSize are derived as follows:

[0229] predModeIntra = (cIdx == 0)?

[0230] IntraPredModeY[xTbY][yTbY] : IntraPredModeC[xTbY][yTbY] (1178)

[0232] nLfnstOutSize = (nTbW >= 8 && nTbH >= 8)? 48 : 16

[0233] log2LfnstSize = (nTbW >= 8 && nTbH >= 8)? 3 : 2

[0234] nLfnstSize = 1 << log2LfhstSize

[0235] nonZeroSize = ((nTbW == 4 && nTbH == 4) ||

[0236] (nTbW == 8 && nTbH == 8))? 8 : 16

[0237] - When intra_mip_flag[xTbY][yTbY] is equal to 1 and cIdx is equal to 0, predModeIntra is set to be equal to INTRA_PLANAR.

[0238] - When predModeIntra is equal to INTRA_LT_CCLM, INTRA_L_CCLM or INTRA_T_CCLM, predModeIntra is derived as follows:

[0239] - If

[0240] intra_mip_flag[xTbY + nTbW*SubWidthC / 2][yTbY + nTbH*

[0241] SubHeightC / 2] is equal to 1, then predModeIntra is set to be equal to

[0242] INTRA_PLANAR.

[0243] - Otherwise, if

[0244] - CuPredMode[0][xTbY + nTbW*SubWidthC / 2][yTbY + nTbH*SubHeightC / 2] is equal to MODE_IBC or MODE_PLT, then predModeIntra is set to be equal to INTRA_DC.

[0245] - Otherwise, predModeIntra is set to be equal to

[0246] - IntraPredModeY[xTbY + nTbW*SubWidthC / 2][yTbY + nTbH*SubHeightC / 2].

[0247] - Call the wide-angle intra prediction mode mapping process specified in Section 8.4.5.2.6 with predModeIntra, nTbW, nTbH, and cIdx as inputs and the modified predModeIntra as the output.

[0248] - The values of list u[x] are derived as follows, where x = 0..nonZeroSize - 1:

[0249] xC = DiagScanOrder[2][2][x][0]

[0250] yC = DiagScanOrder[2][2][x][1]

[0251] u[x] = d[xC][yC]

[0252] - Input the input length of the scaling transformation coefficient nonZeroSize, the transformation output length nTrS set to be equal to nLfnstOutSize, the list of scaling non-zero transformation coefficients u[x] (where x = 0..nonZeroSize - 1), and the intra prediction mode predModeIntra for LFNST set selection, and output a list v[x] (where x = 0..nLfnstOutSize - 1), and call the one-dimensional low-frequency non-separable transformation process specified in Section 8.7.4.2. (This calls the process that actually executes the lfnst transformation process.)

[0253] - The array d[x][y] is derived as follows, where x = 0..nLfnstSize - 1 and y = 0..nLfnstSize - 1:

[0254] - If predModeIntra is less than or equal to 34, then apply the following formula:

[0255] d[x][y] = (y < 4)? v[x+(y << log2LfnstSize)] :

[0256] ((x < 4)? v[32 + x + ((y - 4) << 2)] : d[x][y])

[0257] - Otherwise, apply the following formula:

[0258] d[x][y] = (x < 4)? v[y+(x << log2LfnstSize)] :

[0259] ((y < 4)? v[32 + y + ((x - 4) << 2)] : d[x][y])

[0260] According to the embodiment, regarding Figure 4 and / or Figure 5 the encoder / decoder described can include the features and / or functions described regarding Figures 6 to 8 described.

[0261] Use of four-way splitting (embodiment)

[0262] The current design of the ISP allows the use of two different splitting types:

[0263] · Horizontal splitting: The block is divided into n parts along the vertical dimension

[0264] · Vertical splitting: The block is divided into n parts along the horizontal dimension

[0265] In VTM-5.0, depending on the block size, n can be 2 or 4.

[0266] InFigure 6 In the illustrated embodiment, a third splitting type is introduced. Picture 110 is encoded into data stream 112. According to Figure 6 , the decoder / encoder is configured to: derive / encode Picture 110 from / to data stream 112 with respect to the assignment 112a to the set 114 of prediction types including intra prediction 116 and inter prediction 118 at the granularity of the blocks 120 into which Picture 110 is subdivided, such that each block 120 is assigned to an associated prediction type in the set 114 of prediction types. Further, the decoder / encoder is configured to: for each intra prediction block 120 assigned intra prediction 116, derive / encode the associated intra prediction mode 125 from / to data stream 112 in the set 122 of intra prediction modes.

[0267] For each predetermined intra prediction block 120 for which the associated intra prediction mode 125 is included in the predetermined subset 124 of predetermined intra prediction modes, the decoder / encoder is configured to: derive / encode the information 126 of the split 128 of the corresponding predetermined intra prediction block 120 to the partitions 130 in the set 127 of identification split modes, the set 127 of split modes including: a first split 127a, a second split 127b, and a third split 127c, wherein according to the first split 127a, the corresponding predetermined intra prediction block 120 is horizontally split such that the partition 130 of the corresponding predetermined intra prediction block 120 becomes as wide as the corresponding predetermined intra prediction block 120; according to the second split 127b, the corresponding predetermined intra prediction block 120 is vertically split such that the partition 130 of the corresponding predetermined intra prediction block 120 becomes as high as the corresponding predetermined intra prediction block 120; and according to the third split 127c, the corresponding predetermined intra prediction block 120 is horizontally and vertically split such that the partitions 130 of the corresponding predetermined intra prediction block 120 are arranged in partition rows and partition columns. Further, for each predetermined intra prediction block 120 for which the associated intra prediction mode 125 is included within the predetermined subset 124 of predetermined intra prediction modes, the decoder / encoder is configured to: perform intra prediction on each partition 130 of the corresponding predetermined intra prediction block 120 in a manner depending on the predetermined intra prediction mode 125 assigned to the corresponding predetermined intra prediction block 120.

[0268] The decoder / encoder is configured to, for each block 120, derive / encode information 112e of a prediction residual 132 regarding the corresponding block 120 from / to a data stream 112 by deriving / encoding a prediction residual 132 of each intra-prediction block within a predetermined frame, wherein the prediction residual of each intra-prediction block within a predetermined frame is derived / encoded by: for each partition 130 of the corresponding intra-prediction block 120 within a predetermined frame, deriving / encoding a partition-specific prediction residual signal related to a spatial prediction residual signal of the corresponding partition 130 of the corresponding intra-prediction block 120 from / to the data stream 112 via a predetermined transform T.

[0269] In addition, the decoder / encoder is configured to reconstruct each block 120 using the information of the prediction residual 132 regarding the corresponding block 120 and a prediction signal obtained using a prediction type 125 assigned to the corresponding block 120.

[0270] According to an embodiment, the corresponding intra-prediction block 120 is horizontally and vertically split in a third split 127c such that the number of partition rows is equal to the number of partition columns.

[0271] According to an embodiment, the decoder is configured to derive information 126 of a split 128 of the corresponding intra-prediction block 120 to partitions 130 in a set 127 of split patterns from the data stream 112 by reading a split indicator 112d transmitted in the data stream 112 for the corresponding intra-prediction block 120 within a predetermined frame and using the split indicator 112d (i.e., a split index) to identify the split 128 of the corresponding intra-prediction block 120 to partitions 130 in the set 127 of split patterns. The encoder may include a feature parallel to the decoder, wherein the encoder is configured to encode the information 126 into the data stream 112 by transmitting the split indicator 112d in the data stream 112.

[0272] According to an embodiment, the decoder is configured to read the split indicator 112d transmitted in the data stream 112 for an intra-prediction block 120 within a predetermined frame by: reading a first flag included in the split indicator 112d, the first criterion indicating whether the split 128 of the corresponding intra-prediction block 120 is the third split 127c. The encoder may include a feature parallel to the decoder, wherein the encoder is configured to transmit the split indicator 112d by transmitting the first flag. If the split 128 of the corresponding intra-prediction block 120 is not the third split 127c, the decoder / encoder is configured to read / transmit a second flag included in the split indicator 112d, the second flag indicating whether the split 128 of the corresponding intra-prediction block 120 is the first split 127a or the second split 127b.

[0273] According to an embodiment, regardingFigure 6 The described encoder / decoder may include features and / or functions described in one of the embodiments regarding Figure 4 , Figure 5 , Figure 7 and Figure 8 .

[0274] Preferred version example:

[0275] Each partition has the same size, and the preferred value of n can be 2 or 4. Therefore, there will be 4 or 16 partitions in the case of the same dimensional size.

[0276] Algorithm proposal:

[0277] It is proposed to include another splitting type, namely the third split 127c, i.e., the four-way split, which divides the block into n parts in both the horizontal and vertical dimensions. Therefore, the total number of sub-partitions will be n 2 . For example, for n = 2, there will be a total of 4 sub-partitions. This splitting type will require new syntax elements to indicate its use, or it can be implicitly determined by existing block parameters (such as block dimensional size or intra mode) on the decoder side. This split will work in the same way as the existing split types.

[0278] Use of local intra mode (embodiment)

[0279] In the ISP design of VTM-5.0, the intra mode is shared among all sub-partitions. Therefore, for example, it only needs to be signaled once for the entire block. This mode is defined as the global block mode.

[0280] In the Figure 7 shown embodiment, a local intra mode is introduced. Each partition 130 (e.g., 1300 to 130 3 ) can be assigned its own intra mode. Some partitions 130 of the predetermined intra prediction block 120 can be assigned the same intra mode as other partitions 130 of the predetermined intra prediction block 120. Alternatively, each partition 130 of the predetermined intra prediction block 120 is assigned a different intra mode from other partitions 130 of the predetermined intra prediction block 120.

[0281] Figure 7 Details of the decoder for decoding an image from the data stream 112 and the encoder for encoding an image into the data stream 112 are shown. Also not in Figure 7As shown, the decoder / encoder is configured to: derive / encode from / to the data stream 112 an assignment of pictures, in terms of the blocks into which the pictures are subdivided, to a set 114 of prediction types including intra prediction 116 and inter prediction 118, such that each block 120 is assigned to an associated prediction type in the set 114 of prediction types. This can be performed by a decoder / encoder similar to the decoder / encoder described with respect to Figures 4 to 6 The decoder / encoder described.

[0282] In addition, the decoder / encoder is configured to: for each intra prediction block 120 assigned intra prediction, derive / encode from / to the data stream 112 an associated intra prediction mode 125 in a set of intra prediction modes. The set of intra prediction modes can be equal to or similar to the set 122 described with respect to Figure 4 or Figure 6 described, and can be equal to or similar to a subset 124 of predetermined intra prediction modes described with respect to Figure 4 or Figure 6 described within a subset of predetermined intra prediction modes mentioned below.

[0283] For each predetermined intra prediction block 120 for which the associated intra prediction mode 125 is included within a predetermined subset 124 of predetermined intra prediction modes, the decoder / encoder is configured to: derive / encode from / to the data stream 112 information 126 regarding a split 128 of the corresponding predetermined intra prediction block 128 into partitions 130. The derivation or encoding of the split 128 can be performed as described with respect to Figure 4 and / or Figure 6 described. In addition, for each predetermined intra prediction block 120 for which the associated intra prediction mode 125 is included within a predetermined subset 124 of predetermined intra prediction modes, the decoder / encoder is configured to: derive / encode from / to the data stream 112 information 150 identifying a predetermined rule 156 in a set 154 of rules, using which, for each partition 130 of the corresponding predetermined intra prediction block 120, a partition - specific intra prediction mode 152 is determined from the predetermined subset 124 of predetermined intra prediction modes based on the associated intra prediction mode 125 of the corresponding predetermined intra prediction block 120. Intra prediction of the corresponding partition is performed using the partition - specific intra prediction mode 152 determined for each partition 130 of the corresponding predetermined intra prediction block 120. A first split - specific intra prediction mode 152 0 can be assigned to a first split 1300, a second split - specific intra prediction mode 152 1 can be assigned to a second split 130 1 and a third split - specific intra prediction mode 152 2 can be assigned to a third split 130 2, and a fourth partition specific intra prediction mode 152 3 can be assigned to the fourth partition 130 3 . Determining which partition specific intra prediction mode 152 for the partition 130 depends on a predetermined rule 156 and the associated intra prediction mode 125 of the corresponding predetermined intra prediction block 120.

[0284] In addition, the decoder / encoder is configured to: for each block 120, derive / encode information about the prediction residual of the corresponding block 120 from / into the data stream 112 by deriving the prediction residual of each predetermined intra prediction block, wherein the prediction residual of each predetermined intra prediction block 120 is derived by: for each partition 130 of the corresponding predetermined intra prediction block, deriving a partition specific prediction residual signal 132 related to the spatial prediction residual signal 134 of the corresponding partition 130 of the corresponding predetermined intra prediction block 120 from the data stream 112 via a predetermined transform T. The derivation of the prediction residual can be performed as Figure 4 or Figure 6 described in

[0285] The decoder / encoder is configured to reconstruct each block 120 using the information about the prediction residual of the corresponding block 120 and the prediction signal obtained by using the prediction type assigned to the corresponding block 120. The block 120 can be reconstructed by obtaining a partition specific prediction signal for each partition 130 using the partition specific intra prediction mode 152 determined for the corresponding partition 130. Intra prediction can be performed on the partitions 130 in sequence. The block 120 can be reconstructed by sequentially reconstructing each partition 130 of the block 120 using the partition specific prediction signal and the partition specific prediction residual of the corresponding partition 130.

[0286] According to an embodiment, the decoder / encoder is configured to: perform intra prediction on the partitions 130 of the corresponding predetermined intra prediction block 120 in sequence in a manner that depends on the predetermined intra prediction mode 125 assigned to the corresponding predetermined intra prediction block 120, using the correction of the intra prediction signal of the previous partition by the partition specific prediction residual signal 132 of the previous partition.

[0287] According to an embodiment, the decoder / encoder is configured to: derive / encode information 150 of a predetermined rule 156 in a set 154 of identification rules from / to a data stream 112 by reading a rule index 112g transmitted in the data stream 112 for the corresponding predetermined intra prediction block 120, or by predicting the rule index based on features of adjacent blocks, and for each partition 130 of the corresponding predetermined intra prediction block 120, use the predetermined rule 156 to determine a partition-specific intra prediction mode 152 from a predetermined subset 124 of predetermined intra prediction modes based on the associated intra prediction mode 125 of the corresponding predetermined intra prediction block. Thus, the information 150 of the predetermined rule 156 in the set 154 of identification rules may be explicitly signaled using the rule index 112g transmitted in the data stream 112, or may be implicitly obtained by predicting the rule index based on features of adjacent blocks adjacent to the predetermined block 120. A rule index may be used to identify the predetermined rule 156 in the set 154 of identification rules.

[0288] According to an embodiment, the decoder / encoder is configured such that: each rule of the set 154 of rules determines a partition-specific intra prediction mode 152 for a partition 130 of the corresponding predetermined intra prediction block 120 such that if the associated intra prediction mode 125 of the corresponding predetermined intra prediction block 120 is an angular mode 123a, then the partition-specific intra prediction mode 152 for the partition 130 of the corresponding predetermined intra prediction block 120 is also an angular mode 123a. Further, as Figure 8 shown, the average value of the intra prediction directions 170 of the partition-specific intra prediction modes 152 for the partition 130 of the corresponding predetermined intra prediction block 120 is equal to the intra prediction direction 172 of the associated intra prediction mode 125 of the corresponding predetermined intra prediction block 120. In the case where the angular intra prediction modes 123a are sorted according to the clockwise rotation of their associated intra prediction directions, Figure 8 the intra prediction direction 170 of the partition-specific intra prediction mode 152 according to the first rule 155 in the set 154 of rules is shown on the left, while 1 the intra prediction direction 170 of the partition-specific intra prediction mode 152 according to the second rule 155 in the set 154 of rules is shown on the right. Figure 8 In the case where the angular intra prediction modes 123a are sorted according to the clockwise rotation of their associated intra prediction directions, 2 the intra prediction direction 170 of the partition-specific intra prediction mode 152 according to the second rule 155 in the set 154 of rules is shown on the right.

[0289] As outlined above, the angular intra prediction modes 123a are different from each other in the intra prediction directions 170 / 172. Each angular intra prediction mode 123a may have an index associated therewith, where the association of the index with the angular intra prediction mode 123a may be such that: when the angular intra prediction modes 123a are sorted according to the associated mode index, the direction rotates monotonically clockwise or counterclockwise.

[0290] According to an embodiment, the decoder / encoder is configured such that the set of rules 154 includes one or more pairs of first variant rules (e.g., 155 1 ) and second variant rules (e.g., 155 2 ). The first variant rule (e.g., the first rule 155 1 ) determines the partition-specific intra prediction mode 152 of the partition 130 of the corresponding predetermined intra prediction block 120 in the following manner: when spatially traversing the partition 130 along a predetermined direction 174, the intra prediction direction 170 of the partition-specific intra prediction mode 152 of the partition 130 of the corresponding predetermined intra prediction block 120 deviates from the intra prediction direction 172 of the associated intra prediction mode 125 of the corresponding predetermined intra prediction block 120 by an angular deviation magnitude and an opposite angular offset sign, the angular deviation magnitude monotonically increases with the distance along the predetermined direction 174 to the middle of the corresponding predetermined intra prediction block 120, and the upstream of the middle has an opposite angular deviation sign compared to the downstream of the middle. The second variant rule (e.g., the second rule 155 2 ) determines the partition-specific intra prediction mode 152 of the partition 130 of the corresponding predetermined intra prediction block 120 in the following manner: for each partition 130 of the corresponding predetermined intra prediction block 120, the intra prediction direction 170 of the partition-specific intra prediction mode 152 of the corresponding partition 130 deviates from the intra prediction direction 172 of the associated intra prediction mode 125 of the corresponding predetermined intra prediction block 120 by an angular deviation magnitude and an angular deviation sign, the angular deviation magnitude being equal to the angular deviation magnitude of the deviation of the intra prediction direction 170 of the partition-specific intra prediction mode 152 of the corresponding partition 130 from the associated intra prediction mode 125 of the corresponding predetermined intra prediction block 120 according to the first variant rule (e.g., the first rule 155 1 ), and the angular deviation sign being opposite to the angular deviation sign of the deviation of the intra prediction direction 170 of the partition-specific intra prediction mode 152 of the corresponding partition 130 from the associated intra prediction mode 125 of the corresponding predetermined intra prediction block 120 according to the first variant rule. The first variant rule 155 1 and the second variant rule 155 2 define the partition-specific intra prediction mode 152 of each partition by the same angular deviation magnitude but opposite angular deviation signs.

[0291] In the above derivation or encoding of the segmentation 128, the information 126 may identify the segmentation 128 in the set of segmentation patterns 127. As Figure 4 or Figure 6As shown, the set 127 of splitting patterns may include: a first split 127a and a second split 127b. According to the first split 127a, the corresponding intra-prediction block 120 within a predetermined frame is horizontally split such that the partition 130 of the corresponding intra-prediction block 120 has the same width as the corresponding intra-prediction block 120. And according to the second split 127b, the corresponding intra-prediction block 120 is vertically split such that the partition 130 of the corresponding intra-prediction block 120 has the same height as the corresponding intra-prediction block 120. Optionally, the set 127 of splitting patterns includes a third split 127c. According to the third split 127c, the corresponding intra-prediction block 120 is horizontally and vertically split such that the partitions 130 of the corresponding intra-prediction block 120 are arranged in partition rows and partition columns. The predetermined direction 174 is vertical when the split 128 of the corresponding intra-prediction block 120 is the first split 127a, and horizontal when the split 128 of the corresponding intra-prediction block 120 is the second split 127b. Figure 7 and Figure 8 The split shown is performed according to the first split 127a.

[0292] According to an embodiment, the decoder / encoder is configured to determine the split 128 based on the dimensional size of the corresponding intra-prediction block 120 and according to the number of partitions 130. The split 128 depends on, for example, the height and width of the corresponding intra-prediction block 120. The split 128 may result in a number of partitions 130 exceeding two. The corresponding intra-prediction block 120 may be split into at least three partitions 130 by the split 128.

[0293] According to an embodiment, regarding Figure 7 and / or Figure 8 the encoder / decoder described may include the features and / or functions described regarding Figures 1 to 6 the description.

[0294] Algorithm proposal:

[0295] It is proposed to allow each sub-partition 130 to have its own intra-mode, i.e., a partition-specific intra-prediction mode 152, which does not necessarily have to be equal to the global block mode, i.e., the associated intra-prediction mode 125. In this new method, the global block mode 125 will still be parsed by the decoder, for example, but then each sub-partition 130 will have its own local mode 152. The local mode 152 may depend on the global block mode 125, the sub-partition index, explicitly transmitted syntax elements, adjacent intra-modes, etc. For example, there is a block 120 with 4 sub-partitions (e.g., partition 130 0 to 130 3 ) and having indices from 0 to 3 and a global intra-mode i. Then, the local intra-mode 152 of the sub-partition 130 may be as follows:

[0296] Local mode 0 = i-2

[0297] Local mode 1 = i-1

[0298] Local mode 2 = i + 1

[0299] Local mode 3 = i + 2

[0300] Example of preferred version:

[0301] If ISP is used, the decoder parses the flag (i.e., local intra prediction indication 112h) to decide if there is a local change of intra mode 125. The decoder / encoder may be configured to read / transmit the local intra prediction indication 112h, which indicates whether the partition-specific intra prediction mode 152 is used for prediction of each partition 130.

[0302] If there will be local variations, a second flag (i.e., rule index 112g) will be sent to indicate which variation model, i.e., rule 156, will be adopted. In the case where the local intra prediction indication 112h indicates the use of a partition-specific intra prediction mode 152, the decoder / encoder may be configured to: read / transmit the rule index 112g or predict the rule index based on the characteristics of neighboring blocks and use the rule index to identify a predetermined rule 156 from the set of rules 154. There are, for example, two possibilities:

[0303] Sub - partition Index Model 1 Model 2 0 i-3 i+3 1 i-1 i+1 2 i+1 i-1 3 i+3 i-3

[0304] Additional Notes:

[0305] Figure 4 or Figure 6 The picture 110 shown in FIG. Figures 1 to 3 The described picture 12 or 12' is associated. Figure 4 , Figure 6 and Figure 7 The data flow 112 shown in FIG. 1 may be related to Figure 1 and Figure 2 The described data flow 14 is associated. Figure 4 , Figure 6 , Figure 7 and Figure 8 The intra prediction block 120 shown in FIG. 1 may be related to Figure 3 The intra prediction block 80 described is associated.

[0306] Figure 9A method 1000 for decoding a picture is shown, which includes deriving 1100 from a data stream in which the picture is encoded an assignment, at a granularity of blocks into which the picture is subdivided, to a set of prediction types including intra prediction and inter prediction, such that each block is assigned to an associated prediction type in the set of prediction types. The method 1000 includes: for each intra prediction block assigned intra prediction, deriving from the data stream an associated intra prediction mode in a set of intra prediction modes. For each predetermined intra prediction block whose associated intra prediction mode is included in a predetermined subset of predetermined intra prediction modes, the method 1000 includes: deriving 1300 from the data stream information about a split of the corresponding predetermined intra prediction block into partitions; and performing intra prediction 1400 on each partition of the corresponding predetermined intra prediction block in a manner depending on the predetermined intra prediction mode assigned to the corresponding predetermined intra prediction block. Additionally, the method 1000 includes: for each block, deriving 1500 from the data stream information about a prediction residual of the corresponding block by deriving a prediction residual of each predetermined intra prediction block, wherein the prediction residual of each predetermined intra prediction block is derived by: for each partition of the corresponding predetermined intra prediction block, deriving 1510 from the data stream a partition-specific prediction residual signal associated with a spatial prediction residual signal of the corresponding partition of the corresponding predetermined intra prediction block via a predetermined transform. Additionally, the method 1000 includes: for each block, deriving 1500 from the data stream information about a prediction residual of the corresponding block by deriving a prediction residual of each predetermined intra prediction block, wherein the prediction residual of each predetermined intra prediction block is derived by: deriving 1520 from the data stream information identifying a predetermined transform in a set of transforms, the set of transforms including a first transform and a second transform, the second transform being equal to a cascade of a primary transform and a secondary transform applied to a subset of coefficients of the primary transform. Each block is reconstructed 1600 using the information about the prediction residual of the corresponding block and a prediction signal obtained using the prediction type assigned to the corresponding block.

[0307] As Figure 10As shown, method 2000 for encoding a picture in parallel with method 1000 for decoding includes: encoding 2100 the assignment of the picture, at the granularity of the blocks into which the picture is subdivided, to a set of prediction types including intra prediction and inter prediction, into the data stream into which the picture is encoded, such that each block is assigned to an associated prediction type in the set of prediction types. Method 2000 includes: encoding 2200, into the data stream, the associated intra prediction mode in the set of intra prediction modes for each intra prediction block to which intra prediction is assigned. For each predetermined intra prediction block whose associated intra prediction mode is included in a predetermined subset of predetermined intra prediction modes, method 2000 includes: encoding 1300 into the data stream the information regarding the partitioning of the corresponding predetermined intra prediction block; and performing intra prediction 2400 on each partition of the corresponding predetermined intra prediction block in a manner depending on the predetermined intra prediction mode assigned to the corresponding predetermined intra prediction block. Additionally, method 2000 includes: for each block, encoding into the data stream the information regarding the prediction residual of the corresponding block by encoding the prediction residual coding of each predetermined intra prediction block, wherein the prediction residual coding of each predetermined intra prediction block is encoded 2510 into the data stream by: for each partition of the corresponding predetermined intra prediction block, encoding the partition-specific prediction residual signal related to the spatial-domain prediction residual signal of the corresponding partition of the corresponding predetermined intra prediction block into the data stream via a predetermined transform. Additionally, method 2000 includes: for each block, encoding 2500 into the data stream the information regarding the prediction residual of the corresponding block by encoding the prediction residual of each predetermined intra prediction block, wherein the prediction residual of each predetermined intra prediction block is encoded by: encoding 2520 into the data stream the information of a predetermined transform in a set of transforms, the set of transforms including a first transform and a second transform, the second transform being equal to the concatenation of a primary transform and a secondary transform applied to a subset of the coefficients of the primary transform. For each block, the corresponding block is reconstructable using the prediction signal obtained using the information regarding the prediction residual of the corresponding block and the prediction type assigned to the corresponding block.

[0308] Figure 11A method 3000 for decoding a picture is provided, which includes deriving 1100 from a data stream in which the picture is encoded, an assignment of the picture, at a granularity of blocks into which the picture is subdivided, to a set of prediction types including intra prediction and inter prediction, such that each block is assigned to an associated prediction type in the set of prediction types. The method 3000 includes: for each intra prediction block assigned intra prediction, deriving from the data stream an associated intra prediction mode in a set of intra prediction modes. For each predetermined intra prediction block whose associated intra prediction mode is included in a predetermined subset of predetermined intra prediction modes, the method 3000 includes: deriving 1300 from the data stream information identifying a split of the respective predetermined intra prediction block to partitions in a set of split patterns, the set of split patterns including: a first split, a second split, and a third split, wherein according to the first split, the respective predetermined intra prediction block is horizontally split such that the partition of the respective predetermined intra prediction block becomes as wide as the respective predetermined intra prediction block; according to the second split, the respective predetermined intra prediction block is vertically split such that the partition of the respective predetermined intra prediction block becomes as high as the respective predetermined intra prediction block; and according to the third split, the respective predetermined intra prediction block is horizontally and vertically split such that the partitions of the respective predetermined intra prediction block are arranged in partition rows and partition columns. Further, for each predetermined intra prediction block whose associated intra prediction mode is included in a predetermined subset of predetermined intra prediction modes, the method 3000 includes: performing intra prediction 1400 on each partition of the respective predetermined intra prediction block in a manner depending on the predetermined intra prediction mode assigned to the respective predetermined intra prediction block. Additionally, the method 3000 includes: for each block, deriving 1500 from the data stream information regarding a prediction residual of the respective block by deriving a prediction residual of each predetermined intra prediction block, wherein the prediction residual of each predetermined intra prediction block is derived by: for each partition of the respective predetermined intra prediction block, deriving a partition-specific prediction residual signal related to a spatial domain prediction residual signal of the respective partition of the respective predetermined intra prediction block from the data stream via a predetermined transform. The method 3000 includes reconstructing 1600 each block using the information regarding the prediction residual of the respective block and a prediction signal obtained using the prediction type assigned to the respective block.

[0309] As Figure 12As shown, a method 4000 for encoding a picture in parallel with a method 3000 for decoding includes: encoding 2100 an assignment of the picture, at a granularity of blocks into which the picture is subdivided, to a set of prediction types including intra prediction and inter prediction, into a data stream into which the picture is encoded, such that each block is assigned to an associated prediction type in the set of prediction types. The method 4000 includes: encoding 2200, for each intra prediction block to which intra prediction is assigned, an associated intra prediction mode in a set of intra prediction modes into the data stream. For each predetermined intra prediction block whose associated intra prediction mode is included in a predetermined subset of predetermined intra prediction modes, the method 4000 includes: encoding 2300 into the data stream information identifying a partitioning of the respective predetermined intra prediction block to partitions, the set of partitioning modes including: a first partitioning, a second partitioning, and a third partitioning, wherein according to the first partitioning, the respective predetermined intra prediction block is horizontally split such that the partition of the respective predetermined intra prediction block becomes as wide as the respective predetermined intra prediction block; according to the second partitioning, the respective predetermined intra prediction block is vertically split such that the partition of the respective predetermined intra prediction block becomes as high as the respective predetermined intra prediction block; and according to the third partitioning, the respective predetermined intra prediction block is horizontally and vertically split such that the partitions of the respective predetermined intra prediction block are arranged in partition rows and partition columns. Further, for each predetermined intra prediction block whose associated intra prediction mode is included in a predetermined subset of predetermined intra prediction modes, the method 4000 includes: performing intra prediction 2400 on each partition of the respective predetermined intra prediction block in a manner depending on the predetermined intra prediction mode assigned to the respective predetermined intra prediction block. Additionally, the method 4000 includes: encoding 2500 into the data stream information regarding a prediction residual of the respective block, for each block, by encoding a prediction residual encoding of each predetermined intra prediction block, wherein the prediction residual encoding of each predetermined intra prediction block is encoded by: for each partition of the respective predetermined intra prediction block, encoding into the data stream a partition-specific prediction residual signal associated with a spatial-domain prediction residual signal of the respective partition of the respective predetermined intra prediction block via a predetermined transform. For each block, the respective block is reconstructable using the information regarding the prediction residual of the respective block and a prediction signal obtained using the prediction type assigned to the respective block.

[0310] Figure 13There is provided a method 5000 for decoding a picture, which includes deriving 1100 from a data stream in which the picture is encoded an assignment of the picture, at a granularity of blocks into which the picture is subdivided, to a set of prediction types including intra prediction and inter prediction, such that each block is assigned to an associated prediction type in the set of prediction types. The method 5000 includes: for each intra prediction block assigned with intra prediction, deriving 1200 from the data stream an associated intra prediction mode in a set of intra prediction modes. For each predetermined intra prediction block whose associated intra prediction mode is included in a predetermined subset of predetermined intra prediction modes, the method 5000 includes: deriving 1300 from the data stream information about a split of the corresponding predetermined intra prediction block into partitions; deriving 1350 from the data stream information identifying a predetermined rule in a set of rules, using the predetermined rule, for each partition of the corresponding predetermined intra prediction block, to determine a partition-specific intra prediction mode from the predetermined subset of predetermined intra prediction modes based on the associated intra prediction mode of the corresponding predetermined intra prediction block; and performing intra prediction 1400 on the corresponding partitions using the partition-specific intra prediction modes determined for each partition of the corresponding predetermined intra prediction block. Additionally, the method 5000 includes: for each block, deriving 1500 from the data stream information about a prediction residual of the corresponding block by deriving a prediction residual of each predetermined intra prediction block, wherein the prediction residual of each predetermined intra prediction block is derived by: for each partition of the corresponding predetermined intra prediction block, deriving from the data stream a partition-specific prediction residual signal related to a spatial prediction residual signal of the corresponding partition of the corresponding predetermined intra prediction block via a predetermined transform. The method 5000 includes reconstructing 1600 each block using the information about the prediction residual of the corresponding block and a prediction signal obtained using the prediction type assigned to the corresponding block.

[0311] As Figure 14As shown, a method 6000 for encoding a picture in parallel with a method 5000 for decoding includes: encoding 2100 an assignment of the picture at a granularity of blocks into which the picture is subdivided to a set of prediction types including intra prediction and inter prediction into a data stream into which the picture is encoded, such that each block is assigned to an associated prediction type in the set of prediction types. The method 6000 includes: for each intra prediction block to which intra prediction is assigned, encoding 2200 an associated intra prediction mode in a set of intra prediction modes into the data stream. For each predetermined intra prediction block whose associated intra prediction mode is included in a predetermined subset of predetermined intra prediction modes, the method 6000 includes: encoding 2300 information about a segmentation of the corresponding predetermined intra prediction block into partitions into the data stream; encoding 2350 information identifying a predetermined rule in a set of rules, using which, for each partition of the corresponding predetermined intra prediction block, a partition-specific intra prediction mode is determined from the predetermined subset of predetermined intra prediction modes based on the associated intra prediction mode of the corresponding predetermined intra prediction block; and performing intra prediction 2400 on each partition of the corresponding predetermined intra prediction block using the partition-specific intra prediction mode determined for the corresponding partition. Additionally, the method 6000 includes: for each block, encoding 2500 information about a prediction residual of the corresponding block into the data stream by encoding a prediction residual encoding of each predetermined intra prediction block, wherein the prediction residual encoding of each predetermined intra prediction block is encoded by: for each partition of the corresponding predetermined intra prediction block, encoding a partition-specific prediction residual signal related to a spatial-domain prediction residual signal of the corresponding partition of the corresponding predetermined intra prediction block into the data stream via a predetermined transform. For each block, the corresponding block can be reconstructed using the information about the prediction residual of the corresponding block and a prediction signal obtained using the prediction type assigned to the corresponding block.

[0312] Other embodiments:

[0313] Although some aspects have been described in the context of an apparatus, it will be clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of method steps also represent a description of the features of the corresponding block or item or of the respective apparatus. Some or all of the method steps may be performed by (or using) a hardware device, such as a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more of the most important method steps may be performed by such a device.

[0314] Depending on certain implementation requirements, embodiments of the present invention can be implemented in hardware or in software. Implementations can be carried out using a digital storage medium (e.g., a floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM, or flash memory) on which an electronically readable control signal is stored, which cooperates with (or is capable of cooperating with) a programmable computer system to perform the various methods. Thus, the digital storage medium can be computer-readable.

[0315] Some embodiments according to the present invention include a data carrier having an electronically readable control signal, which is capable of cooperating with a programmable computer system to perform one of the methods described herein.

[0316] Generally, embodiments of the present invention can be implemented as a computer program product having program code operable to perform one of the methods when the computer program product is run on a computer. The program code can be stored, for example, on a machine-readable carrier.

[0317] Other embodiments include a computer program stored on a machine-readable carrier for performing one of the methods described herein.

[0318] In other words, embodiments of the method of the present invention are thus computer programs having program code for performing one of the methods described herein when the computer program is run on a computer.

[0319] Thus, another embodiment of the method of the present invention is a data carrier (or digital storage medium or computer-readable medium) on which a computer program is recorded for performing one of the methods described herein. The data carrier, digital storage medium, or recording medium is generally tangible and / or non-transitory.

[0320] Thus, another embodiment of the method of the present invention is a data stream or signal sequence representing a computer program for performing one of the methods described herein. The data stream or signal sequence can be configured, for example, to be transmitted via a data communication connection (e.g., via the Internet).

[0321] Another embodiment includes a processing device, e.g., a computer or a programmable logic device, configured or adapted to perform one of the methods described herein.

[0322] Another embodiment includes a computer on which a computer program is installed for performing one of the methods described herein.

[0323] Another embodiment according to the present invention includes an apparatus or system configured to transmit a computer program to a receiver (e.g., electronically or optically) for performing one of the methods described herein. The receiver can be, for example, a computer, a mobile device, a storage device, etc. The apparatus or system can include, for example, a file server for transmitting the computer program to the receiver.

[0324] In some embodiments, a programmable logic device (e.g., a field programmable gate array) can be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array can cooperate with a microprocessor to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware device.

[0325] The apparatus described herein can be implemented using a hardware device, or using a computer, or using a combination of a hardware device and a computer.

[0326] The apparatus described herein or any component of the apparatus described herein can be implemented at least partially in hardware and / or software.

[0327] The methods described herein can be performed using a hardware device, or using a computer, or using a combination of a hardware device and a computer.

[0328] The methods described herein or any component of the apparatus described herein can be performed at least partially by hardware and / or by software.

[0329] The above embodiments are illustrative only of the principles of the present invention. It is to be understood that modifications and variations of the arrangements and details described herein will be apparent to other those skilled in the art. Therefore, it is intended to be limited only by the scope of the appended patent claims and not by the specific details given by the description and explanation of the embodiments herein.

Claims

1. A decoder for decoding a picture, comprising: a processor, and a memory storing a computer program which, when executed by the processor, causes the decoder to: derive from a data stream into which the picture is encoded an assignment of the picture, at a granularity of blocks into which the picture is subdivided, to a set of prediction types including intra prediction and inter prediction, such that each block is assigned to an associated prediction type in the set of prediction types, for each first intra prediction block to which the intra prediction is assigned, derive from the data stream an associated intra prediction mode in a set of intra prediction modes, and for each second intra prediction block whose associated intra prediction mode is included in a subset of the set of intra prediction modes: derive from the data stream information about a partitioning of the respective second intra prediction block, use the associated intra prediction mode assigned to the respective second intra prediction block to determine a partition - specific intra prediction mode for each partition of the respective second intra prediction block, for each partition of the respective second intra prediction block, derive from the data stream a partition - specific prediction residual signal, for each partition of the respective second intra prediction block, determine whether no non - zero transform coefficients are located outside a respective predetermined region, in response to a determination that no non - zero transform coefficients are located outside the respective predetermined region for each partition of the respective second intra prediction block, decode a secondary transform index for the respective second intra prediction block, wherein the secondary transform index indicates: not using any inverse secondary transform, or using a first inverse secondary transform, or using a second inverse secondary transform, for generating a partition - specific spatial residual for each partition of the respective second intra prediction block, and use the partition - specific spatial residual and the partition - specific intra prediction to reconstruct each partition of the respective second intra prediction block.

2. The decoder according to claim 1, configured to: in response to a determination that at least one non - zero transform coefficient is located outside the respective predetermined region for a partition of the respective second intra prediction block, not decode the secondary transform index and infer that no secondary transform is used for the respective second intra prediction block.

3. The decoder according to claim 1, wherein the first inverse secondary transform or the second inverse secondary transform is also indicated by the associated intra prediction mode assigned to the respective second intra prediction block.

4. A method for decoding a picture, the method comprising: deriving from a data stream into which the picture is encoded an assignment of the picture, at a granularity of blocks into which the picture is subdivided, to a set of prediction types including intra prediction and inter prediction, such that each block is assigned to an associated prediction type in the set of prediction types, for each first intra prediction block to which the intra prediction is assigned, deriving from the data stream an associated intra prediction mode in a set of intra prediction modes, and for each second intra prediction block whose associated intra prediction mode is included in a subset of the set of intra prediction modes: Derive information about the partitioning of the corresponding intra-prediction block into partitions from the data stream, Using the associated intra-prediction mode assigned to the corresponding intra-prediction block, determine the partition-specific intra-prediction mode for each partition of the corresponding intra-prediction block, For each partition of the corresponding intra-prediction block, derive a partition-specific prediction residual signal from the data stream, For each partition of the corresponding intra-prediction block, determine whether no non-zero transform coefficients are located outside a corresponding predetermined region, In response to the determination that no non-zero transform coefficients are located outside the corresponding predetermined region for each partition of the corresponding intra-prediction block, decode a secondary transform index for the corresponding intra-prediction block, wherein the secondary transform index indicates: not using any inverse secondary transform, or using a first inverse secondary transform, or using a second inverse secondary transform, for generating a partition-specific spatial residual for each partition of the corresponding intra-prediction block, and Use the partition-specific spatial residual and the partition-specific intra-prediction to reconstruct each partition of the corresponding intra-prediction block.

5. The method according to claim 4, further comprising: In response to the determination that there are non-zero transform coefficients located outside the corresponding predetermined region for at least one partition of the corresponding intra-prediction block, do not decode the secondary transform index and infer that no secondary transform is used for the corresponding intra-prediction block.

6. The method according to claim 4, wherein the first inverse secondary transform or the second inverse secondary transform is further indicated by the associated intra-prediction mode assigned to the corresponding intra-prediction block.

7. An encoder for encoding a picture, comprising: a processor, and a memory storing a computer program which, when executed by the processor, causes the encoder to: Encode the assignment of the picture, at the granularity of the blocks into which the picture is subdivided, to a set of prediction types including intra-prediction and inter-prediction, into the data stream into which the picture is encoded, such that each block is assigned to an associated prediction type in the set of prediction types, For each first intra-prediction block assigned the intra-prediction, encode the associated intra-prediction mode in the set of intra-prediction modes into the data stream, For each second intra-prediction block whose associated intra-prediction mode is included in a predetermined subset of the set of intra-prediction modes: Encode information about the partitioning of the corresponding intra-prediction block into partitions into the data stream, Using the associated intra-prediction mode assigned to the corresponding intra-prediction block, determine the partition-specific intra-prediction mode for each partition of the corresponding intra-prediction block, For each partition of the corresponding intra-prediction block, encode the partition-specific prediction residual signal into the data stream, For each partition of the corresponding intra-prediction block, determine whether no non-zero transform coefficients are located outside the corresponding predetermined region, and In response to a determination that there are no non-zero transform coefficients outside a corresponding predetermined region for each partition of the corresponding intra-prediction block in the second frame, encoding a secondary transform index for the corresponding intra-prediction block in the second frame, wherein the secondary transform index indicates: not using any inverse secondary transform, or using a first inverse secondary transform, or using a second inverse secondary transform, for generating a partition-specific spatial residual for each partition of the corresponding intra-prediction block in the second frame, wherein each partition of the corresponding intra-prediction block in the second frame can be reconstructed using the partition-specific spatial residual and the partition-specific intra-prediction.

8. A method for encoding a picture, comprising: encoding an assignment of the picture, at a granularity of blocks into which the picture is subdivided, to a set of prediction types including intra-prediction and inter-prediction, into a data stream into which the picture is encoded, such that each block is assigned to an associated prediction type in the set of prediction types, for each first intra-prediction block to which the intra-prediction is assigned, encoding an associated intra-prediction mode in a set of intra-prediction modes into the data stream, for each second intra-prediction block whose associated intra-prediction mode is included in a predetermined subset of the set of intra-prediction modes: encoding information about a partitioning of the corresponding second intra-prediction block into partitions into the data stream, using the associated intra-prediction mode assigned to the corresponding second intra-prediction block, determining a partition-specific intra-prediction mode for each partition of the corresponding second intra-prediction block, for each partition of the corresponding second intra-prediction block, encoding a partition-specific prediction residual signal into the data stream, for each partition of the corresponding second intra-prediction block, determining whether there are no non-zero transform coefficients outside a corresponding predetermined region, and in response to a determination that there are no non-zero transform coefficients outside a corresponding predetermined region for each partition of the corresponding second intra-prediction block, encoding a secondary transform index for the corresponding second intra-prediction block, wherein the secondary transform index indicates: not using any inverse secondary transform, or using a first inverse secondary transform, or using a second inverse secondary transform, for generating a partition-specific spatial residual for each partition of the corresponding second intra-prediction block, wherein each partition of the corresponding second intra-prediction block can be reconstructed using the partition-specific spatial residual and the partition-specific intra-prediction.

9. A computer program product comprising a computer program which, when executed by a processor, implements the method according to claim 4 or 8.

10. A method for transmitting a data stream, comprising: obtaining a data stream by the method according to claim 8, and transmitting the data stream.

Images