Efficient Implementation of Matrix-Based Intra-frame Prediction

Abstract

A device (54) for decoding a predetermined block (18) of a picture using intra-frame prediction is configured to read a mode index (200) from a data stream (12) using a binarized code (202). The mode index (200) points to one of a list (204) of matrix-based intra-frame prediction modes. The device is configured to predict samples (108) of the predetermined block (18) by calculating a matrix-vector product (206) between an input vector (102) derived from a reference sample (17) in a neighborhood of the predetermined block (18) and a prediction matrix (19) associated with the matrix-based intra-frame prediction mode (k) pointed to by the mode index (200), and associating components (210) of an output vector (208) obtained by the matrix-vector product (206) to sample positions (104) of the predetermined block. The list (204) of matrix-based intra-frame prediction modes consists of an even number of matrix-based intra-frame prediction modes. The list (204) of matrix-based intra-frame prediction modes consists of matrix-based intra-frame prediction mode pairs (212), and for each matrix-based intra-frame prediction mode pair (212), the prediction matrix (19) associated with a first matrix-based intra-frame prediction mode of the corresponding matrix-based intra-frame prediction mode pair (212) is equal to the prediction matrix (19) associated with a second matrix-based intra-frame prediction mode of the corresponding matrix-based intra-frame prediction mode pair (212). For each matrix-based intra-frame prediction mode pair (212), the device is configured such that if the matrix-based intra-frame prediction mode pointed to by the mode index (200) is the first matrix-based intra-frame prediction mode of the corresponding matrix-based intra-frame prediction mode pair (212), the association of the reference sample (17) in the neighborhood of the predetermined block (18) with the component (214) of the input vector (112) and the association of the sample position (104) of the predetermined block (18) with the component (210) of the output vector (208) are transposed relative to the association in the case where the matrix-based intra-frame prediction mode pointed to by the mode index (200) is the second matrix-based intra-frame prediction mode of the corresponding matrix-based intra-frame prediction mode pair (212). Furthermore, the apparatus is configured to read the mode index (200) from the data stream (12) using a binarized code (202) in such a way that for each matrix-based intra-frame prediction mode pair (212), the first matrix-based intra-frame prediction mode is assigned a first codeword and the second matrix-based intra-frame prediction mode is assigned a second codeword, and the two codewords are equal in length.

Description

Technical Field

[0001] Embodiments according to the present invention relate to apparatus and methods for encoding or decoding pictures or videos using efficient implementation of matrix-based intra prediction. Background Art

[0002] Currently, for each block of a picture on which MIP can be used, the number of MIP modes is an odd integer N = 2n + 1. Here, mode 0 may not be used. Transpose (transpose), while all other modes may be transposed: for i>n, MIP mode i is the transpose of MIP mode in. Mode 0 may not be transposed, since MIP modes are signaled by an MPM scheme with 3 MPMs (most probable modes) [1]. In this scheme, the total number of MIP modes is N=2 k +3, where k is an integer. However, at the 16th JVET conference in Gothenburg, the latter signaling scheme for the MIP mode was replaced by a truncated binary code [2]. Furthermore, MIP is excluded for blocks with an aspect ratio greater than or equal to 4.

[0003] Therefore, it is desirable to provide concepts for making picture coding and / or video coding more efficient to support matrix-based intra prediction. Additionally or alternatively, it is desirable to reduce the bitstream and thereby reduce signaling costs.

[0004] This is achieved by the subject matter of the independent claims of the present application.

[0005] Further embodiments according to the invention are defined by the subject matter of the dependent claims of the present application. Summary of the Invention

[0006] According to a first aspect of the present invention, the inventors of the present application have recognized that one problem encountered when attempting to use matrix-based intra prediction modes (MIP modes) for predicting samples of a predetermined block of a picture stems from the fact that the memory requirements for additionally storing an additional MIP matrix for the MIP mode, which cannot be used in a transposed manner, can be prohibitively expensive. According to the first aspect of the present application, this difficulty is overcome by using a list of matrix-based intra prediction modes, wherein all MIP modes in the list are applicable in a transposed manner, thereby reducing the number of MIP modes. The inventors have found that it is advantageous to group the MIP modes in the list of matrix-based intra prediction modes into matrix-based intra prediction mode pairs, wherein both MIP modes in the MIP mode pair are associated with the same prediction matrix, thereby reducing the number of required prediction matrices. For each MIP mode pair, the first MIP mode of the corresponding MIP mode pair is used in a transposed manner compared to the second MIP mode of the corresponding MIP mode pair. This is based on the idea that simply determining the MIP mode and whether the MIP mode is used in a transposed manner from the mode index is efficient. This special grouping of MIP modes makes it possible to determine the transposed condition of the MIP mode based on the parity of the MIP modes in the MIP mode list. Therefore, decoding / encoding an additional syntax element indicating whether the MIP mode is to be used in a transposed manner is unnecessary. Since the mode index indicates the prediction mode and whether this prediction mode is to be used in a transposed manner, bitstream and signaling costs can be reduced. In this way, signaling a mode and its transposed mode is equally expensive in terms of VLC length. No asymmetry is created. Construction of a most probable mode list (MPM list) is also unnecessary.

[0007] Therefore, according to a first aspect of the present application, an apparatus for decoding / encoding a predetermined block of a picture using intra prediction is configured to: read a mode index from a data stream / insert the mode index into a data stream using a binarized code. The mode index points to an intra prediction mode from a list of matrix-based intra prediction modes. The apparatus is configured to: predict samples of the predetermined block by calculating a matrix-vector product between an input vector derived from reference samples in a neighborhood of the predetermined block and a prediction matrix associated with the matrix-based intra prediction mode pointed to by the mode index, and associating components of the output vector obtained by the matrix-vector product to sample positions of the predetermined block. This prediction of the samples of the predetermined block may be affine linear weighted intra prediction (ALWIP). The input vector may be derived from reference samples (i.e., boundary values) by averaging the reference samples and / or by obtaining original reference samples (see, for example, the description under item 1.4). The reference samples may represent, for example, samples to the left of the predetermined block and samples above the predetermined block. The reference samples may be samples that have already been decoded / encoded by the apparatus. The list of matrix-based intra prediction modes consists of an even number of matrix-based intra prediction modes. The list of matrix-based intra prediction modes consists of pairs of matrix-based intra prediction modes, and for each matrix-based intra prediction mode pair, the prediction matrix associated with the first matrix-based intra prediction mode of the corresponding matrix-based intra prediction mode pair is equal to the prediction matrix associated with the second matrix-based intra prediction mode of the corresponding matrix-based intra prediction mode pair. In both matrix-based intra prediction modes of a matrix-based intra prediction mode pair, the same prediction matrix may be used to predict samples of a predetermined block. This means, for example, that the same prediction matrix is ​​used for modes 2k and 2k+1. For each matrix-based intra prediction mode pair, the apparatus is configured such that, if the matrix-based intra prediction mode pointed to by the mode index is the first matrix-based intra prediction mode of the corresponding matrix-based intra prediction mode pair (e.g., the mode with an odd mode index 2k+1), the associations of reference samples in the neighborhood of the predetermined block with components of the input vector and the associations of sample positions of the predetermined block with components of the output vector are transposed relative to the associations when the matrix-based intra prediction mode pointed to by the mode index is the second matrix-based intra prediction mode of the corresponding matrix-based intra prediction mode pair (e.g., the mode with an even mode index 2k). That is, if in the former case a component of the input vector is associated with position (x, y), where (0, 0) represents the top-left corner sample of the predetermined block, then in the latter case it is associated with (y, x). The same applies to the components of the output vector.The apparatus is configured to read a mode index from a data stream / insert the mode index into the data stream using a binarized code in such a manner that, for each matrix-based intra prediction mode pair, a first matrix-based intra prediction mode is assigned a first codeword and a second matrix-based intra prediction mode is assigned a second codeword, and the two codewords are equal in length. Thus, the mode index may indicate a prediction matrix used to predict samples of a predetermined block and may also indicate whether the prediction of the samples of the predetermined block is performed in a transposed manner.

[0008] The binarization code may be a variable length code including codewords of different lengths, or the binarization code may be a truncated binary code, in which case the number of matrix-based intra prediction modes is not a power of 2. The truncated binary code may have codewords of different lengths.

[0009] The apparatus is, for example, configured to read / insert a pattern index from / into a data stream using an equiprobable bypass mode of a context-adaptive binary arithmetic decoder / encoder.

[0010] According to an embodiment, the apparatus is configured to horizontally set the inter-sample distances of sample positions of the predetermined block and the inter-sample distances of reference samples in a neighborhood of the predetermined block according to a first ratio of the horizontal size of the predetermined block relative to a horizontal default size, and / or vertically set the inter-sample distances of sample positions of the predetermined block and the inter-sample distances of reference samples in a neighborhood of the predetermined block according to a second ratio of the vertical size of the predetermined block relative to a vertical default size. Thus, it is possible to use MIP modes not only for quadratic blocks but also for rectangular blocks. The apparatus may obtain prediction samples associated with the quadratic block by matrix-based intra prediction, for which reason the apparatus may be configured to set the inter-sample distances of the sample positions of the predicted samples to improve reconstruction of the rectangular block, wherein the apparatus may fill the spaces between the predicted samples by interpolation. Optionally, the apparatus is configured to use a list of matrix-based intra prediction modes for multiple block sizes. The apparatus may be configured to equally order the matrix-based intra prediction modes in the list of matrix-based intra prediction modes for the multiple block sizes. Alternatively, the order may be adapted, for example, to blocks that are wider than they are tall, or vice versa, or to quadratic blocks. The plurality of block sizes includes, for example, at least one block size corresponding to an aspect ratio greater than 4. The at least one block size may be associated with a predetermined block having 8×1 samples, 16×2 samples, 32×4 samples, 32×2 samples, 32×1 samples, 64×8 samples, 64×4 samples, 64×2 samples, 64×1 samples, or other blocks having an aspect ratio greater than 4.

[0011] According to an embodiment, the apparatus is configured to determine, based on the parity of the mode index, whether the matrix-based intra prediction mode pointed to by the mode index is a first matrix-based intra prediction mode of a corresponding matrix-based intra prediction mode pair or a second matrix-based intra prediction mode of the corresponding matrix-based intra prediction mode pair. The first matrix-based intra prediction mode may be associated with odd parity, and the second matrix-based intra prediction mode may be associated with even parity.

[0012] According to an embodiment, the apparatus is configured to index a prediction matrix of the plurality of prediction matrices using an integer part of the pattern index divided by 2.

[0013] According to an embodiment, the last bit of the mode index read from / inserted into the data stream indicates whether the prediction of the samples of the predetermined block is performed in a transposed manner. This bit may indicate whether the first matrix-based intra prediction mode or the second matrix-based intra prediction mode is to be used. This bit may be understood as the aforementioned codeword.

[0014] According to a second aspect of the invention, the inventors of the present application have recognized that one problem encountered when attempting to use matrix-based intra prediction modes (MIP modes) for predicting samples of a predetermined block of a picture stems from the fact that MIP modes are currently not allowed for blocks with an aspect ratio greater than 4, resulting in reduced flexibility when decoding / encoding pictures from / into a data stream. The inventors have found that it is possible to use MIP modes also for blocks with an aspect ratio greater than 4. This is based on the idea that this large rectangular block can be efficiently downsampled / upsampled to a quadratic block so that matrix-based intra prediction can predict samples of the block, and additional samples of the block can be interpolated / extrapolated based on the predicted samples. Thus, a high degree of flexibility in the block sizes that can be used for MIP modes is achieved without increasing signaling costs or reducing decoding efficiency.

[0015] Therefore, according to a second aspect of the present application, an apparatus for decoding / encoding predetermined blocks of a picture using intra prediction is configured to predict each of the predetermined intra prediction blocks of the picture by: reading a mode index from a data stream / inserting a mode index into a data stream, the mode index pointing to an intra prediction mode in a list of matrix-based intra prediction modes; and predicting samples of the corresponding predetermined intra prediction block by calculating a matrix-vector product between an input vector derived from reference samples in a neighborhood of the corresponding predetermined intra prediction block and a prediction matrix associated with the matrix-based intra prediction mode pointed to by the mode index, and associating components of the output vector obtained by the matrix-vector product to sample positions of the corresponding predetermined intra prediction block. In other words, for each of the predetermined intra prediction blocks, the apparatus is configured to predict the corresponding block using the prediction mode pointed to by the mode index, wherein the prediction mode represents matrix-based intra prediction, i.e., affine linear weighted intra prediction (ALWIP). The predetermined intra prediction blocks include blocks whose aspect ratio is greater than 4. The predetermined intra prediction block includes, for example, an 8×1 block, a 16×2 block, a 32×4 block, a 32×2 block, a 32×1 block, a 64×8 block, a 64×4 block, a 64×2 block, a 64×1 block, or other blocks having an aspect ratio greater than 4.

[0016] According to an embodiment, the apparatus is configured to set the inter-sample distances of sample positions of the corresponding predetermined intra-frame prediction block and the inter-sample distances of reference samples in a neighborhood of the corresponding predetermined intra-frame prediction block horizontally according to a first ratio of a horizontal size of the predetermined block to a horizontal default size and / or vertically according to a second ratio of a vertical size of the predetermined block to a vertical default size.

[0017] According to an embodiment, the apparatus is configured to use a list of matrix-based intra prediction modes for a plurality of block sizes. Optionally, the apparatus is configured to equally order the matrix-based intra prediction modes in the list of matrix-based intra prediction modes for the plurality of block sizes. Alternatively, the order may be adapted, for example, to blocks that are wider than they are tall, or vice versa, or to quadratic blocks.

[0018] According to a third aspect of the present invention, the inventors of the present application have recognized that one problem encountered when attempting to decode / encode transform flags for blocks of a picture from a data stream stems from the fact that the efficiency of coding syntax elements for MIP blocks may be reduced compared to blocks not predicted by a MIP mode. According to the first aspect of the present application, this difficulty is overcome by using separate contexts for context-based adaptive binary arithmetic coding (CABAC) of transform flags for blocks predicted using a MIP mode and blocks predicted without a MIP mode (i.e., using planar, angular, or DC mode). It is asserted that there is a statistical difference between the use of multiple transform selection (MTS) for MIP and non-MIP blocks, and this difference can be exploited by separating the corresponding contexts. In particular, because the MIP mode must have been learned using a specific residual transform (here, the default transform), other transforms are less likely to produce better results than heuristically designed modes such as DC, planar, and angular modes. Therefore, improved coding efficiency can be achieved by using separate contexts for MIP and non-MIP blocks.

[0019] Therefore, according to a third aspect of the present application, an apparatus for decoding / encoding predetermined blocks of a picture using intra prediction is configured to predict each of predetermined intra prediction blocks (e.g., matrix-based intra prediction blocks (MIP blocks)) of the picture by: reading a mode index from a data stream / inserting a mode index into a data stream, the mode index pointing to an intra prediction mode in a list of matrix-based intra prediction modes; and predicting samples of the corresponding predetermined intra prediction block by calculating a matrix-vector product between an input vector derived from reference samples in a neighborhood of the corresponding predetermined intra prediction block and a prediction matrix associated with the matrix-based intra prediction mode pointed to by the mode index, and associating components of an output vector obtained by the matrix-vector product to sample positions of the corresponding predetermined intra prediction block. The apparatus is, for example, configured to obtain a prediction signal for the corresponding predetermined intra prediction block through this prediction. The apparatus is configured to predict each of additional predetermined intra-frame prediction blocks (e.g., "normal" intra-frame blocks) of the picture using one of a set of normal intra-frame prediction modes including an angular prediction mode, a DC prediction mode, and a planar prediction mode to obtain a prediction signal. For a predetermined block among the predetermined intra-frame prediction blocks and the additional predetermined intra-frame prediction blocks, the apparatus is configured to decode from a data stream / encode into the data stream a transform flag (e.g., a multiple transform selection flag (MTS flag)), the transform flag indicating whether a prediction residual of the predetermined block is encoded into the data stream using a default transform or a different transform. The transform flag is encoded for both the MIP block and the normal intra-frame block. The apparatus is configured to decode from the data stream / encode into the data stream the transform flag using context-adaptive binary arithmetic coding using a probability model selected depending on whether the predetermined block is one of the predetermined intra-frame prediction blocks or one of the additional predetermined intra-frame prediction blocks. The apparatus is configured to decode a prediction residual of a predetermined block from a data stream / encode the prediction residual of the predetermined block into the data stream and retransform the prediction residual using an inverse transform to obtain a prediction residual signal, wherein if a transform flag indicates that the prediction residual of the predetermined block was encoded into the data stream using a default transform, the inverse transform reverses the default transform, and if the transform flag indicates that the prediction residual of the predetermined block was encoded into the data stream using a different transform, the inverse transform reverses the different transform. The encoder is configured to perform the retransformation in a prediction loop. The apparatus is configured to use the prediction residual signal to correct a prediction signal. This correction is performed, for example, for a predetermined intra-frame prediction block and a further predetermined intra-frame prediction block.

[0020] According to an embodiment, the default transform is DCT-II applied horizontally and vertically.

[0021] If the transform flag indicates that the prediction residual of the predetermined block is encoded into the data stream using a different transform, the apparatus is, for example, configured to decode from the data stream / encode into the data stream one or more further syntax elements indicating the different transform, the one or more further syntax elements indicating, for example, a different transform from a set of separable transforms, the transforms horizontally using a first one of DST-VII and DCT-DCT-VIII and vertically using a second one of DST-VII and DCT-DCT-VIII.

[0022] According to an embodiment, the apparatus is configured to: in decoding / encoding a transform flag from / into a data stream, if the predetermined block is one of predetermined intra-frame prediction blocks, use a first probability model as a probability model for decoding / encoding the transform flag and update the first probability model depending on the transform flag, and if the predetermined block is one of further predetermined intra-frame prediction blocks, use a second probability model as a probability model for decoding / encoding the transform flag and update the second probability model depending on the transform flag.

[0023] An embodiment relates to a method for decoding / encoding a predetermined block of a picture using intra prediction, comprising: reading / inserting a mode index from a data stream using a binarized code, the mode index pointing to an intra prediction mode in a list of matrix-based intra prediction modes; and predicting samples of the predetermined block by computing a matrix-vector product between an input vector derived from reference samples in a neighborhood of the predetermined block and a prediction matrix associated with the matrix-based intra prediction mode pointed to by the mode index, and associating components of an output vector obtained by the matrix-vector product to sample positions of the predetermined block. The list of matrix-based intra prediction modes consists of an even number of matrix-based intra prediction modes. The list of matrix-based intra prediction modes consists of pairs of matrix-based intra prediction modes, and for each pair of matrix-based intra prediction modes, a prediction matrix associated with a first matrix-based intra prediction mode of the corresponding matrix-based intra prediction mode pair is equal to a prediction matrix associated with a second matrix-based intra prediction mode of the corresponding matrix-based intra prediction mode pair. The method comprises: if the matrix-based intra-frame prediction mode pointed to by the mode index is the first matrix-based intra-frame prediction mode of the corresponding matrix-based intra-frame prediction mode pair, then the association of reference samples in the neighborhood of the predetermined block with the components of the input vector and the association of sample positions of the predetermined block with the components of the output vector are transposed relative to the association when the matrix-based intra-frame prediction mode pointed to by the mode index is the second matrix-based intra-frame prediction mode of the corresponding matrix-based intra-frame prediction mode pair. The method comprises: using a binarization code in the following manner to read the mode index from the data stream / insert the mode index into the data stream, so that for each matrix-based intra-frame prediction mode pair, the first matrix-based intra-frame prediction mode is assigned a first codeword and the second matrix-based intra-frame prediction mode is assigned a second codeword, and the two codewords are equal in length.

[0024] An embodiment relates to a method for decoding / encoding predetermined blocks of a picture using intra prediction, comprising predicting each of predetermined intra prediction blocks of the picture by: reading a mode index from a data stream / inserting a mode index into a data stream, the mode index pointing to an intra prediction mode from a list of matrix-based intra prediction modes; and predicting samples of the corresponding predetermined intra prediction block by computing a matrix-vector product between an input vector derived from reference samples in a neighborhood of the corresponding predetermined intra prediction block and a prediction matrix associated with the matrix-based intra prediction mode pointed to by the mode index, and associating components of an output vector obtained by the matrix-vector product to sample positions of the corresponding predetermined intra prediction block. The predetermined intra prediction block includes blocks whose aspect ratio is greater than 4.

[0025] An embodiment relates to a method for decoding / encoding predetermined blocks of a picture using intra prediction, comprising predicting each of predetermined intra prediction blocks (e.g., MIP blocks) of the picture by: reading a mode index from a data stream / inserting a mode index into a data stream, the mode index pointing to an intra prediction mode from a list of matrix-based intra prediction modes; and predicting samples of the corresponding predetermined intra prediction block by computing a matrix-vector product between an input vector derived from reference samples in a neighborhood of the corresponding predetermined intra prediction block and a prediction matrix associated with the matrix-based intra prediction mode pointed to by the mode index, and associating components of an output vector obtained by the matrix-vector product to sample positions of the corresponding predetermined intra prediction block. Furthermore, the method comprises predicting each of additional predetermined intra prediction blocks of the picture using one of a set of normal intra prediction modes including an angular prediction mode, a DC prediction mode, and a planar prediction mode to obtain a prediction signal. For a predetermined block among the predetermined intra-prediction blocks and the further predetermined intra-prediction blocks, the method comprises decoding from a data stream / encoding into the data stream a transform flag indicating whether a prediction residual for the predetermined block is encoded into the data stream using a default transform or a different transform; decoding from the data stream / encoding into the data stream a prediction residual for the predetermined block, retransforming the prediction residual using an inverse transform to obtain a prediction residual signal, the inverse transform reversing the default transform if the transform flag indicates that the prediction residual for the predetermined block is encoded into the data stream using the default transform, and reversing the different transform if the transform flag indicates that the prediction residual for the predetermined block is encoded into the data stream using a different transform; and correcting the prediction signal using the prediction residual signal. The method comprises decoding from the data stream / encoding into the data stream a transform flag using context-adaptive binary arithmetic coding using a probability model selected depending on whether the predetermined block is one of the predetermined intra-prediction blocks or one of the further predetermined intra-prediction blocks.

[0026] Embodiments relate to a data stream having pictures or videos encoded therein using the method for encoding described herein.

[0027] An embodiment relates to a computer program having a program code for performing the method described herein when the program runs on a computer. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings are not necessarily to scale; instead, emphasis is generally placed upon illustrating the principles of the invention. In the following description, various embodiments of the invention are described with reference to the following drawings, in which:

[0029] Figure 1An embodiment of encoding into a data stream is shown;

[0030] Figure 2 An embodiment of an encoder is shown;

[0031] Figure 3 An embodiment of the reconstruction of a picture is shown;

[0032] Figure 4 An embodiment of a decoder is shown;

[0033] Figure 5.1 shows the prediction of a block with a reduced sample value vector according to an embodiment;

[0034] Figure 5.2 shows prediction of a block using sample interpolation according to an embodiment;

[0035] Figure 5.3 shows the prediction of a block with a reduced sample value vector according to an embodiment, where only some boundary samples are averaged;

[0036] Figure 5.4 shows a prediction of a block with a reduced sample value vector according to an embodiment, wherein groups of four boundary samples are averaged;

[0037] Figure 6 shows matrix-based intra prediction of a predetermined block of a picture based on a mode index according to an embodiment;

[0038] Figure 7 shows the relationship between matrix-based intra prediction mode pairs and application of inter-sample distance settings according to an embodiment;

[0039] Figure 8 shows matrix-based intra prediction of a predetermined block having an aspect ratio greater than 4 according to an embodiment; and

[0040] Figure 9 The encoding of the transform flag using context-based adaptive binary arithmetic coding according to an embodiment is shown. DETAILED DESCRIPTION

[0041] In the following description, the same or equivalent elements or elements having the same or equivalent functions are denoted by the same or equivalent reference numerals even though they appear in different drawings.

[0042] In the following description, a number of details are set forth to provide a more comprehensive explanation of the embodiments of the present invention. However, it will be apparent to those skilled in the art that embodiments of the present invention may be practiced without these specific details. In other instances, to avoid obscuring the embodiments of the present invention, well-known structures and devices are shown in block diagram form rather than in detail. In addition, the features of the different embodiments described below may be combined with each other unless otherwise specifically noted.

[0043] In the following, various examples are described that can help achieve more efficient compression when using matrix-based intra prediction. For example, matrix-based intra prediction can be added to other heuristically designed intra prediction modes, or can be provided specifically.

[0044] To facilitate understanding of the following examples of the present application, the present description begins with the presentation of possible encoders and decoders fitting therein, into which the subsequently outlined examples of the present application may be built. Figure 1 An apparatus is shown for encoding a picture 10 block by block into a data stream 12. The apparatus is indicated by reference numeral 14 and can be a still picture encoder or a video encoder. In other words, when encoder 14 is configured to encode a video 16 including picture 10 into data stream 12, picture 10 can be the current picture in video 16, or encoder 14 can specifically encode picture 10 into data stream 12.

[0045] As mentioned, the encoder 14 performs encoding in a block-by-block manner or on a block basis. To this end, the encoder 14 subdivides the picture 10 into blocks, which the encoder 14 encodes into the data stream 12 in units of blocks. Possible examples of subdividing the picture 10 into blocks 18 are explained in more detail below. In general, the subdivision can end up with blocks 18 of a constant size, such as an array of blocks arranged in rows and columns, or with blocks 18 of different block sizes, such as by using a hierarchical multi-tree subdivision, where the multi-tree subdivision starts from the entire picture area of ​​the picture 10, or starts from a pre-partition of the picture 10 into an array of treeblocks, wherein these examples should not be regarded as excluding other possible ways of subdividing the picture 10 into blocks 18.

[0046] Furthermore, the encoder 14 is a predictive encoder that is configured to predictively encode the picture 10 into the data stream 12. For a certain block 18, this means that the encoder 14 determines a prediction signal for the block 18 and encodes a prediction residual (i.e., the prediction signal with a prediction error by which it deviates from the actual picture content within the block 18) into the data stream 12.

[0047] Encoder 14 can support different prediction modes to derive prediction signals for a particular block 18. In the following example, the important prediction mode is intra-prediction mode, according to which the interior of block 18 is spatially predicted from neighboring, already-encoded samples of picture 10. The encoding of picture 10 into data stream 12, and therefore the corresponding decoding process, can be based on a specific coding order 20 defined among blocks 18. For example, coding order 20 can traverse block 18 in a raster scan order, such as traversing row by row from top to bottom, where each row is traversed from left to right, for example. In the case of a hierarchical multi-tree subdivision, raster scan ordering can be applied within each hierarchical level, where a depth-first traversal order can be applied, i.e., according to coding order 20, leaf annotations within a block at a certain hierarchical level can precede blocks at the same hierarchical level with the same parent block. Depending on coding order 20, neighboring, already-encoded samples of block 18 can generally be located on one or more sides of block 18. For instance, in the case of the examples presented herein, the adjacent, already encoded samples of block 18 are positioned to the top and left of block 18 .

[0048] Intra-prediction modes may not be the only ones supported by encoder 14. For example, in the case where encoder 14 is a video encoder, encoder 14 may also support inter-prediction modes, according to which block 18 is temporally predicted from previously encoded pictures of video 16. Such inter-prediction modes may be motion-compensated prediction modes, according to which a motion vector is signaled for such block 18, the motion vector indicating the relative spatial offset of the portion from which the prediction signal for block 18 is to be derived as a copy. Additionally or alternatively, other non-intra-prediction modes may also be available, such as inter-view prediction modes in the case where encoder 14 is a multi-view encoder, or non-predictive modes, according to which the interior of block 18 is encoded as is, i.e., without any prediction.

[0049] Before starting with focusing the description of this application on the intra prediction mode, Figure 2 More specific examples of possible block-based encoders are described, namely possible implementations for the encoder 14, wherein the examples of fitting to Figure 1 and 2 Two corresponding examples of decoders for .

[0050] Figure 2 Shown Figure 1 A possible implementation of the encoder 14 is one in which the encoder is configured to use transform coding for encoding the prediction residual, but this is merely an example and the application is not limited to that kind of prediction residual coding. Figure 2, the encoder 14 includes a subtractor 22 configured to subtract the corresponding prediction signal 24 from the incoming signal (i.e., the picture 10, or the current block 18 on a block basis) to obtain a prediction residual signal 26, which is then encoded into the data stream 12 by a prediction residual encoder 28. The prediction residual encoder 28 consists of a lossy encoding stage 28a and a lossless encoding stage 28b. The lossy stage 28a receives the prediction residual signal 26 and includes a quantizer 30, which quantizes the samples of the prediction residual signal 26. As mentioned above, this example uses transform coding of the prediction residual signal 26, and therefore, the lossy encoding stage 28a includes a transform stage 32 connected between the subtractor 22 and the quantizer 30 to transform such spectrally decomposed prediction residual 26 using quantization performed by the quantizer 30 on the transform coefficients in which the residual signal 26 is present. The transform can be a DCT, a DST, an FFT, a Hadamard transform, etc. The transformed and quantized prediction residual signal 34 is then subjected to lossless encoding by a lossless encoding stage 28 b, which is an entropy encoder that entropy encodes the quantized prediction residual signal 34 into the data stream 12. The encoder 14 further comprises a prediction residual signal reconstruction stage 36 connected to the output of the quantizer 30 in order to reconstruct the prediction residual signal from the transformed and quantized prediction residual signal 34 in a manner that is also usable at the decoder, i.e. taking into account the coding losses of the quantizer 30. To this end, the prediction residual reconstruction stage 36 comprises a dequantizer 38 that performs the inverse of the quantization of the quantizer 30 and is followed by an inverse transformer 40 that performs the inverse of the transform performed by the transformer 32, such as the inverse of the spectral decomposition, such as the inverse of any of the specific transform examples mentioned above. The encoder 14 includes an adder 42 that adds the reconstructed prediction residual signal output by the inverse transformer 40 and the prediction signal 24 to output a reconstructed signal, i.e., reconstructed samples. This output is fed into a predictor 44 of the encoder 14, which then determines the prediction signal 24 based thereon. The predictor 44 supports the above-mentioned Figure 1 All forecast models discussed. Figure 2 It is also illustrated that in the case where the encoder 14 is a video encoder, the encoder 14 may also include an in-loop filter 46, which filters the fully reconstructed picture, which after having been filtered forms a reference picture for the predictor 44 with respect to the inter-frame prediction blocks.

[0051] As mentioned above, encoder 14 operates on a block basis. For the following description, the block basis of interest is the subdivision of picture 10 into blocks, for which an intra-prediction mode is selected from a set or multiple intra-prediction modes supported by predictor 44 or encoder 14, respectively, and the selected intra-prediction mode is independently performed. However, other types of blocks into which picture 10 is subdivided may also exist. For example, the aforementioned decision of whether picture 10 is inter-coded or intra-coded can be made at a granularity other than block 18, or at a block level. For example, the inter / intra mode decision can be made at the level of the coding blocks into which picture 10 is subdivided, with each coding block being subdivided into prediction blocks. Prediction blocks for which intra-prediction has been decided are each subdivided into an intra-prediction mode decision. To this end, for each of these prediction blocks, a decision is made as to which supported intra-prediction mode should be used for the corresponding prediction block. These prediction blocks will form the block 18 of interest here. Prediction blocks within coding blocks associated with inter-prediction will be treated differently by predictor 44. They will be inter-predicted from the reference picture by determining the motion vector and copying the prediction signal for this block from the position in the reference picture pointed to by the motion vector. Another block subdivision involves subdivision into transform blocks in units of the transforms performed by the transformer 32 and the inverse transformer 40. The transformed blocks may, for example, be the result of a further subdivision of the coding blocks. Naturally, the examples set out herein should not be considered limiting and other examples exist. For the sake of completeness only, it is noted that the subdivision into coding blocks may, for example, use a multi-tree subdivision and that the prediction blocks and / or transform blocks may also be obtained by further subdividing the coding blocks using a multi-tree subdivision.

[0052] exist Figure 3 The fitting to Figure 1 A decoder 54 or means for block-by-block decoding of the encoder 14. This decoder 54 performs the opposite of the encoder 14, i.e. it decodes the picture 10 from the data stream 12 in a block-by-block manner and supports multiple intra prediction modes for this purpose. For example, the decoder 54 may include a residual provider 156. Figure 1All other possibilities discussed are also valid for decoder 54. To this end, decoder 54 can be a still picture decoder or a video decoder, and all prediction modes and prediction possibilities are also supported by decoder 54. The difference between encoder 14 and decoder 54 lies primarily in the fact that encoder 14 selects or chooses coding decisions based on some optimization, such as, for example, to minimize some cost function that may depend on the coding rate and / or coding distortion. One of these coding options or coding parameters may involve selecting an intra-prediction mode for current block 18 among the available or supported intra-prediction modes. The selected intra-prediction mode can then be signaled by encoder 14 for current block 18 within data stream 12, with decoder 54 using this signaling for block 18 in data stream 12 to reselect. Similarly, the subdivision of picture 10 into block 18 can be optimized within encoder 14, and the corresponding subdivision information can be transmitted within data stream 12, with decoder 54 recovering the subdivision of picture 10 into block 18 based on this subdivision information. In summary, the decoder 54 may be a predictive decoder operating on a block basis, and in addition to the intra prediction mode, the decoder 54 may support other prediction modes, such as the inter prediction mode, in the case where the decoder 54 is a video decoder, for example. In decoding, the decoder 54 may also use the Figure 1 The coding order 20 discussed is followed, and since this coding order 20 is followed at both the encoder 14 and the decoder 54, the same adjacent samples are available for the current block 18 at both the encoder 14 and the decoder 54. Therefore, in order to avoid unnecessary repetition, the description of the mode of operation of the encoder 14 will also apply to the decoder 54 as far as the subdivision of the picture 10 into blocks is concerned, for example, as far as prediction is concerned and as far as the coding of the prediction residual is concerned. The difference lies in the fact that the encoder 14 selects some coding options or coding parameters and signals within the data stream 12 by optimization, or inserts the coding parameters into the data stream 12, which are then derived from the data stream 12 by the decoder 54 in order to re-perform the prediction, subdivision, etc.

[0053] Figure 4 Shown Figure 3 A possible implementation of the decoder 54 is to fit Figure 2 As shown in Figure 1 The implementation of the encoder 14 is implemented in this way. Figure 4 Many elements of the decoder 54 are related to Figure 2 The same as those appearing in the corresponding encoder, so in Figure 4 The same reference symbols provided with a prime are used in order to indicate these elements. In particular, the adder 42', the optional in-loop filter 46' and the predictor 44' are shown in the same manner as they are in the Figure 2is connected to the prediction loop in the same way as in the encoder of FIG. 4 . The reconstructed, i.e. dequantized and retransformed prediction residual signal applied to the adder 42 ′ is derived from a sequence of entropy decoders 56 which reverse the entropy coding of the entropy encoder 28 b and are followed by a residual signal reconstruction stage 36 ′ consisting of a dequantizer 38 ′ and an inverse transformer 40 ′, just as was the case on the encoding side. The output of the decoder is a reconstruction of the picture 10. The reconstruction of the picture 10 can be available directly at the output of the adder 42 ′ or, alternatively, at the output of the in-loop filter 46 ′. Some post filters can be arranged at the output of the decoder in order to subject the reconstruction of the picture 10 to some post filtering, thereby improving the picture quality, but in Figure 4 This option is not depicted in .

[0054] Likewise, about Figure 4 , above about Figure 2 The description proposed for Figure 4 will also be valid, except that only the encoder performs the optimization tasks and associated decisions about coding options. However, all descriptions of block subdivision, prediction, dequantization, and retransformation are for Figure 4 The decoder 54 is also effective.

[0055] The embodiments described below utilize so-called matrix-based intra prediction. The general concept will be outlined below. This concept is sometimes referred to below as ALWIP (affine linear weighted intra prediction), as an alternative synonym for MIP (matrix-based intra prediction).

[0056] In ALWIP or MIP mode, to predict samples of a rectangular block of width W and height H, affine linear weighted intra prediction (ALWIP) (or MIP) can take as input a row of H reconstructed neighboring boundary samples to the left of the block and a row of W reconstructed neighboring boundary samples above the block. If reconstructed samples are not available, they can be generated as done in conventional intra prediction. Figures 5.1 to 5.4 Matrix-based intra prediction of samples 104 of a predetermined block 18 is shown.

[0057] The generation of the prediction signal (e.g., the value for the complete block 18) may be based on at least some of the following three steps:

[0058] 1. Samples 102 may be extracted from the boundary samples 17 by averaging or downsampling (e.g., four samples in the case of W=H=4 and / or eight samples in other cases) (e.g., step 811). At step 811, it is possible to reduce the number of samples 17a and 17c adjacent to the block 18. As shown, for example, Figure 5.1At 100 in FIG. 1 , after subdividing the rows 17 c and columns 17 a into groups 110 of two samples each, each group 110 may maintain a single sample (e.g., an average of the samples in the group 110 or a simple selection among the samples in the group 110). Or as shown, for example, Figure 5.4 At 122 , the boundary samples may be grouped into groups 120 of four consecutive samples, where in this case each group 120 may also hold only one sample (eg, selected among four samples, or an average of four samples).

[0059] 2. A matrix-vector multiplication may be performed using the averaged samples (or samples remaining from downsampling) as input, followed by the addition of an offset. The result may be a reduced prediction signal over the subsampled set of samples in the original block.

[0060] (For example, step 812)

[0061] 3. The prediction signals at the remaining positions may be generated, for example, by upsampling (eg, by linear interpolation) the prediction signals from the subsampled set (eg, step 813).

[0062] Thanks to steps 1 (811) and / or 3 (813), the total number of multiplications required in the calculation of the matrix-vector product can be such that it is always less than or equal to 4*W*H. In addition, the averaging operation on the boundaries and the linear interpolation of the reduced prediction signal are performed by using additions and bit shifts alone. In other words, for example, for the ALWIP mode, a maximum of four multiplications are required per sample.

[0063] In some examples, the matrix (e.g., 17M) and offset vector (e.g., b) required to generate the prediction signal are k ) can be taken from a set of matrices (e.g., three sets), such as S0, S1, S2, which can be stored in (multiple) storage units of, for example, a decoder and an encoder.

[0064] In some examples, the set S0 may include n0 (eg, n0=16 or n0=18 or another number) matrices (e.g., consisting of), each of which may have 16 rows and 4 columns, and 18 offset vectors of size 16 each Based on execution Figure 5.1 This set of matrices and offset vectors is applied to blocks of size 4×4 18. Once the boundary vector has been reduced to P red =4 vector (such as for Figure 5.1 In step 811), it is possible to reduce the number of samples 102 to P. red= 4 samples are directly mapped into the Q = 16 samples of the 4×4 block 18 to be predicted.

[0065] In some examples, the set S1 may include n1 (e.g., n1 = 8 or n1 = 18 or another number) matrices (e.g., composed of), each of which may have 16 rows and 8 columns, and 18 offset vectors each of size 16 to perform the techniques according to Figure 5.2 or 5.3. The matrices and offset vectors of this set S1 can be used for blocks of sizes 4×8, 4×16, 4×32, 4×64, 16×4, 32×4, 64×4, 8×4, and 8×8. Additionally, it can also be used for blocks of size WxH (where max(W,H)>4 and min(W,H)=4), i.e., blocks of size 4x16 or 16x4, 4x32 or 32x4, and 4x64 or 64x4. The 16×8 matrix refers to a reduced version of the block 18 obtained as in Figure 5.2 and 5.3 , which is a 4×4 block.

[0066] Additionally or alternatively, the set S2 may include n2 (e.g., n2 = 6 or n2 = 18 or another number) matrices (e.g., composed of), each of which may have 64 rows and 8 columns, and 18 offset vectors of size 64 The 64×8 matrix refers to, for example, a reduced version of the block 18 obtained as in Figure 5.4 , which is an 8×8 block. The matrices and offset vectors of this set can be used for blocks of sizes 8×16, 8×32, 8×64, 16×8, 16×16, 16×32, 16×64, 32×8, 32×16, 32×32, 32×64, 64×8, 64×16, 64×32, 64×64.

[0067] The matrices and offset vectors of that set or parts of these matrices and offset vectors can be used for all other block shapes.

[0068] 1.1 Averaging or downsampling of the boundary

[0069] Here, features regarding step 811 are provided.

[0070] Boundary samples (17a, 17c) can be averaged and / or downsampled (e.g., from P samples to P red <P samples).

[0071] In the first step, the input boundaries bdry top (e.g., 17c) and bdry left (e.g., 17a) can be reduced to smaller boundaries and To obtain the reduced set 102. Here, in the case of a 4x4 block, and Both consist of 2 samples, and in the other case, both consist of 4 samples.

[0072] In the case of 4x4 blocks, it is possible to define:

[0073]

[0074] And similarly define therefore, is an average value obtained using, for example, a bit shift operation.

[0075] In all other cases (e.g., for blocks whose width or height is different from 4), if the block width W is given as W=4*2 k , then for 0≤i<4, define:

[0076]

[0077] And similarly define

[0078] In some cases, it is possible to downsample the boundary (e.g., by selecting a specific boundary sample from a set of boundary samples) to achieve a reduced number of samples. top [0] and bdry top [1] Choose And can be in bdry top [2] and bdry top [3] Choose Similarly defined It is also possible.

[0079] Two reduction boundaries and Can be cascaded to the reduced boundary vector bdry red (associated with reduced set 102), also indicated by 17P. Thus, for a block of shape 4×4 ( Figure 5.1 Example), the reduced boundary vector bdry red Can have size 4 (P red =4), and for all other shapes of blocks ( Figures 5.2-5.4 ), which may have a size of 8 (P red =8).

[0080] Here, if mode < 18 (or the number of matrices in the matrix set), it is possible to define:

[0081]

[0082] If mode ≥ 18, which corresponds to the transposed mode of mode -17, it is possible to define:

[0083]

[0084] Therefore, depending on the specific state (one state: mode < 18; another state: mode ≥ 18), it is possible to distribute the predicted values ​​of the output vector along different scanning orders (for example, a scanning order: Another scan order: ).

[0085] Other strategies may be implemented. In other examples, the pattern index "pattern" is not necessarily in the range of 0 to 35 (other ranges may be defined). Furthermore, each of the three sets S0, S1, S2 does not necessarily have 18 matrices (thus, instead of an expression such as pattern ≥ 18, it is possible that pattern ≥ n0, n1, n2, which is the number of matrices in each matrix set S0, S1, S2, respectively). Furthermore, the sets may each have a different number of matrices (for example, S0 may have 16 matrices, S1 may have 8 matrices, and S2 may have 6 matrices).

[0086] The mode and transposition information are not necessarily stored and / or transmitted as a combined mode index "mode": in some examples there is the possibility to signal it explicitly as a transposition flag and a matrix index (0-15 for S0, 0-7 for S1, and 0-5 for S2).

[0087] In some cases, the combination of the transpose flag and the matrix index can be interpreted as a set index or a pattern index. For example, there may be one bit that operates as a transpose flag, and some bits that indicate the matrix index (collectively referred to as a "set index" or a "pattern index").

[0088] 1.2 Generate reduced prediction signal through matrix-vector multiplication

[0089] Here, features regarding step 812 are provided.

[0090] Can be obtained from the reduced input vector bdry red (Boundary vector 17P) generates a reduced prediction signal pred red The latter signal can be a W red and height H red Here, W red and H red Can be defined as: W red =4,Hred =4; if max(W,H)≤8,

[0091] W red =min(W,8),H red =min(H,8); other cases.

[0092] Reduced prediction signal pred red It can be calculated by taking the matrix-vector product and adding the bias: pred red =A·bdry red +b.

[0093] Here, A is a matrix (e.g., prediction matrix 17M), which may have W red *H red rows, and 4 columns if W=H=4, and 8 columns in all other cases, and b can be of size W red *H red vector.

[0094] If W=H=4, then A may have 4 columns and 16 rows, and so in that case 4 multiplications may be required per sample to compute pred red In all other cases, A can have 8 columns, and it can be verified that in these cases, there are 8*W red *H red ≤4*W*H, i.e., also in these cases, at most 4 multiplications are required per sample to calculate pred red .

[0095] The matrix A and vector b can be taken from one of the sets S0, S1, S2 as follows. The index idx=idx(W,H) is defined as follows: if W=H=4, then set idx(W,H)=0, if max(W,H)=8, then set idx(W,H)=1, and in all other cases, set idx(W,H)=2. In addition, if mode<18, then m=mode, and otherwise, m=mode-17. Then, if idx≤1 or idx=2 and min(W,H)>4, then and In the case where idx = 2 and min(W,H) = 4, let A be obtained by leaving out The matrix generated by each row of W = 4 is corresponds to odd x-coordinates in the downsampled block, or in the case of H=4, the Corresponds to an odd y - coordinate in the down - sampling block. If the mode ≥ 18, the reduced prediction signal is replaced by its transposed signal. In alternative examples, different strategies can be executed. For example, use a smaller matrix of S1(idx = 1) with W red = 4 and H red = 4 instead of reducing the size of the larger matrix (“omitting”), i.e., such blocks are now assigned to S1 instead of S2.

[0096] Other strategies can be executed. In other examples, the mode index “mode” is not necessarily in the range from 0 to 35 (other ranges can be defined). Additionally, it is not necessary for each of the three sets S0, S1, S2 to have 18 matrices (thus, instead of an expression like mode < 18, it is possible to have mode < n0, n1, n2, which are the number of matrices in each matrix set S0, S1, S2 respectively). Additionally, the sets can each have a different number of matrices (e.g., it is possible that S0 has 16 matrices, S1 has 8 matrices, and S2 has 6 matrices).

[0097] 1.3 Linear interpolation for generating the final prediction signal

[0098] Here, features regarding step 812 are provided.

[0099] Interpolation of the subsampled prediction signal may require a second version of the average boundary on large blocks. That is, if min(W, H)>8 and W≥H, write W = 8 * 2 l , and for 0 ≤ i < 8, define:

[0100]

[0101] If min(W, H)>8 and H>W, define similarly

[0102] Additionally or alternatively, it is possible to have “hard down - sampling”, where equals

[0103]

[0104] Similarly, it can be defined

[0105] In the generation of pred red (for example, Figures 5.2-5.4 in step 813 of the example), at the sample positions omitted in the generation of the final prediction signal, the final prediction signal can be generated by linear interpolation from pred red . In some examples, if W = H = 4 (for example, Figure 5.1), then this linear interpolation may be unnecessary.

[0106] Linear interpolation can be given as follows (although other examples are possible). Assume W ≥ H. Then, if H > H red , you can execute pred red In that case, pred red We can extend one row toward the top as follows. If W=8, then pred red May have a width W red =4, and can be obtained by averaging the boundary signal Extend to the top, e.g., as defined above. If W>8, then pred red With width W red = 8, and it passes the average boundary signal Extends to the top, e.g. as defined above. For pred red The first line can be written as pred red [x][-1]. Then, the width W red And the height is 2*H red The signal on the block can be given as:

[0107] where 0≤x <W red , and 0≤y <H red The latter process can be performed k times until 2 k *H red =H. Therefore, if H=8 or H=16, it can be performed at most once. If H=32, it can be performed twice. If H=64, it can be performed three times. Next, the horizontal upsampling operation can be applied to the result of the vertical upsampling. The latter upsampling operation can use the complete boundary on the left of the prediction signal. Finally, if H>W, it can be performed similarly by first upsampling in the horizontal direction (if necessary) and then upsampling in the vertical direction.

[0108] This is an example of interpolation using reduced boundary samples for the first interpolation (horizontally or vertically) and the original boundary samples for the second interpolation (vertically or horizontally). Depending on the size of the block, only the second interpolation is required or no interpolation is required. If both horizontal and vertical interpolation are required, the order depends on the width and height of the block.

[0109] However, different techniques may be implemented: for example, the original boundary samples may be used for both the first and second interpolation, and the order may be fixed, eg first horizontal then vertical (in other cases first vertical then horizontal).

[0110] Therefore, the interpolation order (horizontal / vertical) and the use of downscaled / original boundary samples may vary.

[0111] 1.4 Example of the entire ALWIP process

[0112] Figures 5.1-5.4 The whole process of averaging, matrix-vector multiplication, and linear interpolation is illustrated for different shapes in . Note that the remaining shapes are considered to be in one of the depicted cases.

[0113] 1. Given a 4×4 block, ALWIP can be implemented by using Figure 5.1 The technique takes two averages along each axis of the boundary. The resulting four input samples enter the matrix-vector multiplication 19. The matrix is ​​taken from the set S0. After adding the offset, this can produce 16 final prediction samples. Linear interpolation is not necessary to generate the prediction signal. Therefore, a total of (4*16) / (4*4)=4 multiplications are performed per sample. See for example Figure 5.1 .

[0114] 2. Given an 8×8 block, ALWIP can take four averages along each axis of the boundary. By using Figure 5.2 The resulting eight input samples enter the matrix-vector multiplication 19. The matrix is ​​taken from the set S1. This produces 16 samples at odd positions of the prediction block. Therefore, a total of (8*16) / (8*8)=2 multiplications are performed per sample. After adding the offset, these samples can be interpolated vertically, for example, by using the top boundary, and horizontally, for example, by using the left boundary. See, for example Figure 5.2 .

[0115] 3. Given an 8×4 block, ALWIP can be implemented by using Figure 5.3 The technique takes four average values ​​along the horizontal axis of the boundary and four original boundary values ​​on the left boundary. The resulting eight input samples enter the matrix-vector multiplication 19. The matrix is ​​taken from set S1. This produces 16 samples at odd horizontal and vertical positions of the prediction block. Therefore, a total of (8*16) / (8*4)=4 multiplications are performed per sample. For example, after adding the offset, these samples are horizontally interpolated by using the left boundary. See, for example Figure 5.3 .

[0116] The transposed case is handled accordingly.

[0117] 4. Given a 16×16 block, ALWIP can take four averages along each axis of the boundary. By using Figure 5.4 The resulting eight input samples enter the matrix-vector multiplication 19. The matrix is ​​taken from the set S2. This produces 64 samples at odd positions in the prediction block. Therefore, a total of (8*64) / (16*16)=2 multiplications are performed per sample. For example, after adding the offset, these samples are interpolated vertically by using the top boundary and horizontally by using the left boundary. See, for example Figure 5.4 .

[0118] For larger shapes the procedure is probably essentially the same, and it's easy to check that the number of multiplications per sample is less than two.

[0119] For W×8 blocks, only horizontal interpolation is necessary because samples are given at odd horizontal and every vertical position. Therefore, in these cases, at most (8*64) / (16*8)=4 multiplications are performed per sample.

[0120] Finally, for W×4 blocks with W>8, let A k is a matrix generated by omitting each row corresponding to the odd entries along the horizontal axis of the downsampled block. Therefore, the output size can be 32, and again, only horizontal interpolation still needs to be performed. A maximum of (8*32) / (16*4)=4 multiplications can be performed per sample.

[0121] The transposed case can be handled accordingly.

[0122] According to the embodiments proposed below, MIP modes are applied in such a way that the use of MIPs is even more efficient than what was envisaged so far in current VVC versions. The embodiments relate to three different aspects, which can be applied separately from each other or combined in pairs or in full. Compared to current VVC implementations, it is first proposed that all MIP modes (i.e. also the 0th MIP mode) can be used in a transposed manner, where the parity of the mode determines the transposition condition. In doing so, the number of MIP modes for MipSizeId=0 can be reduced from 35 to 32, and the number of MIP modes for MipSizeId=1 can be reduced from 19 to 16. The MIP modes can then be encoded using a fixed-length code for small blocks. Secondly, it is proposed to remove the restriction that MIPs cannot be used on blocks whose aspect ratio is greater than or equal to 4. Additionally, it is proposed to introduce a separate context for the MTS index depending on the MIP flag.

[0123] Without the proposed context, the proposed method reportedly yields a -0.06% BD rate change compared to VTM-6.0 in the AI ​​configuration and a 0.04% BD rate saving compared to VTM-6.0 in the RA configuration. Encoder runtime is reported as 101% for the AI ​​configuration and 100% for the RA configuration. Decoder runtime is reported as 100% for the AI ​​configuration and 99% for the RA configuration. With the proposed context added, the proposed method reportedly yields a -0.09% BD rate change compared to VTM-6.0 in the AI ​​configuration and a -0.05% BD rate change compared to VTM-6.0 in the RA configuration. Encoder runtime is reported as 100% for the AI ​​configuration and 100% for the RA configuration. Decoder runtime is reported as 101% for the AI ​​configuration and 99% for the RA configuration.

[0124] It should be noted that Figures 6 to 9 The following description mainly describes features that can be used in a picture decoder or a video decoder (ie, a device for decoding). However, it is clear that any features described herein can also be used in the context of a picture encoder or a video encoder.

[0125] Figure 6 Means 54 are shown for decoding a predetermined block 18 of a picture using intra prediction.

[0126] The apparatus 54 is configured to read a mode index 200 from the data stream 12 using a binarized code 202, the mode index pointing to an intra-prediction mode in a list 204 of matrix-based intra-prediction modes. The list 204 of matrix-based intra-prediction modes includes an even number of matrix-based intra-prediction modes, wherein the matrix-based intra-prediction modes of the list 204 are grouped into matrix-based intra-prediction mode pairs 212. Each pair 212 includes a first matrix-based intra-prediction mode and a second matrix-based intra-prediction mode. The apparatus 54 is configured to read the mode index 200 from the data stream 12 using the binarized code 202 in such a manner that, for each matrix-based intra-prediction mode pair 212, the first matrix-based intra-prediction mode is assigned a first codeword and the second matrix-based intra-prediction mode is assigned a second codeword, and the two codewords are equal in length.

[0127] Optionally, the binarized code 202 is a variable length code that includes codewords of different lengths. Alternatively, the binarized code can be a truncated binary code, and the number of matrix-based intra-frame prediction modes is not a power of 2, so that the truncated binary code has codewords of different lengths. The matrix-based intra-frame prediction mode associated with the first matrix-based intra-frame prediction mode pair 212 can be assigned a codeword that is different in length from the codeword assigned to the matrix-based intra-frame prediction mode associated with the second matrix-based intra-frame prediction mode pair 212. However, the two codewords of the matrix-based intra-frame prediction mode pair 212 are equal in length.

[0128] According to an embodiment, the apparatus 54 may be configured to read the pattern index 200 from the data stream 12 using an equiprobable bypass mode of the context adaptive binary arithmetic decoder.

[0129] Similar to the device 54 for decoding a predetermined block 18 of a picture using intra-frame prediction (i.e., a decoder), the device for encoding a predetermined block 18 of a picture using intra-frame prediction (i.e., an encoder) can be configured to encode the mode index 200 into the data stream 12 using a binarization code 202 and, optionally, using an equiprobable bypass mode of a context-adaptive binary arithmetic encoder.

[0130] The decoder and the encoder are configured to predict samples 108 of the predetermined block 18 by computing a matrix-vector product 206 between an input vector 102 derived from reference samples 17 in a neighborhood of the predetermined block 18 and a prediction matrix 19 associated with the matrix-based intra prediction mode k pointed to by the mode index 200. The computation of the matrix-vector product 206 results in an output vector 208. Furthermore, the samples 108 of the predetermined block 18 are predicted by associating components 210 of the output vector 208 obtained by the matrix-vector product 206 to the sample positions 104 of the predetermined block 18. This prediction of the samples 108 of the predetermined block 18 may be as described with respect to Figures 5.1 to 5.4 Perform as described.

[0131] For each matrix-based intra prediction mode pair 212, the prediction matrix 19 associated with the first matrix-based intra prediction mode of the corresponding matrix-based intra prediction mode pair 212 is equal to the prediction matrix 19 associated with the second matrix-based intra prediction mode of the corresponding matrix-based intra prediction mode pair 212. Therefore, the same prediction matrix 19 is used for matrix-based intra prediction modes 2k and 2k+1. For each matrix-based intra-frame prediction mode pair 212, the encoder and the decoder are configured such that if the matrix-based intra-frame prediction mode pointed to by the mode index 200 is the first matrix-based intra-frame prediction mode of the corresponding matrix-based intra-frame prediction mode pair 212, for example, the mode with an odd mode index 2k+1, then the association of the reference sample 17 in the neighborhood of the predetermined block 18 with the component 214 of the input vector 112 and the association of the sample position 104 of the predetermined block 18 with the component 210 of the output vector 208 are transposed relative to the association when the matrix-based intra-frame prediction mode pointed to by the mode index 200 is the second matrix-based intra-frame prediction mode of the corresponding matrix-based intra-frame prediction mode pair 212 (for example, the mode with an even mode index 2k).

[0132] The decoder / encoder may be configured to determine, based on the parity of the mode index 200, whether the matrix-based intra prediction mode pointed to by the mode index 200 is the first matrix-based intra prediction mode of the corresponding matrix-based intra prediction mode pair or the second matrix-based intra prediction mode of the corresponding matrix-based intra prediction mode pair 212. The parity of the mode index 200 may indicate whether the input vector 102 and the output vector 208 are used in a transposed manner for predicting the samples 108 of the predetermined block 18. That is, if Figure 7 As shown in , if in the former case a certain component of the components 1 to n of the input vector 102 is associated with the position (x, y), where (0, 0) represents the top left corner sample AA of the predetermined block 18, then in the latter case it is associated with (y, x). The same applies to the components (AA, AB, AC, BA, CA, ...) of the output vector 208.

[0133] Each pair 212 consists of a first matrix-based intra prediction mode and a second matrix-based intra prediction mode, the modes being related to each other by the same prediction matrix 19 and differing only in whether the input vector 102 and the output vector 208 are transposed. Depending on the embodiment, the last significant bit (i.e., the last bin) of the binarized code 202 of the index 200, or alternatively the most significant bit (i.e., the first bin) of the binarized code 202 (such as a truncated binary code or a fixed-length binary code), may indicate whether the MIP mode is to be used in a transposed manner. Alternatively, or viewed from a different perspective, the mode index consists of a syntax element representing a sub-index onto a corresponding pair of mutually transposed MIP modes, which may also be binarized using a truncated binary code having a fixed number of bits and an additional syntax element representing a transposition flag, which indicates whether transposition is to be applied. In other words, the index 200 will consist of the first syntax element indicating the MIP mode pair and an additional flag indicating whether the matrix forming the basis of the MIP mode pair is to be applied in a transposed manner. Additional bits or flags may be decoded from / encoded into the data stream 12 before or after the sub-index syntax element is decoded from / encoded into the data stream 12.

[0134] According to an embodiment, the decoder / encoder is configured to index a prediction matrix 19 from the plurality of prediction matrices using the integer part of the mode index 200 divided by 2. This is based on the idea that the two matrix-based intra prediction modes 212 use the same prediction matrix 19 for predicting the samples 108 of the predetermined block 18, for which reason the prediction matrix 19 is already sufficiently indicated by pointing to the relevant pair 212 in the list 204 with the mode index 200.

[0135] like Figure 6 and 7As shown in FIG, the decoder / encoder may be configured to horizontally set 217 the inter-sample distances 216 of sample positions 104 of the predetermined block 18 and the inter-sample distances 218 of reference samples 17 in the neighborhood of the predetermined block 18 according to a first ratio of a horizontal dimension 220 of the predetermined block 18 to a horizontal default dimension, and / or vertically set 217 according to a second ratio of a vertical dimension 222 of the predetermined block 18 to the vertical default dimension. This enables the use of the matrix-based list 204 of intra prediction modes for multiple block sizes. The apparatus may fill the spaces between predicted samples by interpolation. The inter-sample distance setting 217 of the inter-sample distances 216 of sample positions 104 of the predetermined block 18 and the inter-sample distances 218 of reference samples 17 in the neighborhood of the predetermined block 18 enables an improved distribution of the predicted samples 108 in the predetermined block 18 and the reference samples 17 in the neighborhood of the predetermined block 18. As a result, the predicted samples may be equally distributed, thereby enabling improved interpolation of samples of the predetermined block 18.

[0136] According to an embodiment, the decoder / encoder is configured to sort the matrix-based intra prediction modes in the list 204 of matrix-based intra prediction modes equally for multiple block sizes. Alternatively, the order may be adapted to, for example, blocks that are wider than they are tall, or vice versa (i.e., blocks that are taller than they are wide), or to quadratic blocks. This ordering may increase coding efficiency and reduce bitrate because matrix-based intra prediction modes for common block sizes may be associated with short codewords, and matrix-based intra prediction modes for rare block sizes may be associated with longer codewords.

[0137] Optionally, the plurality of block sizes includes at least one block size corresponding to an aspect ratio greater than 4. The matrix-based intra prediction may be optimized such that the predetermined block 18 having an aspect ratio of the horizontal dimension 220 to the vertical dimension 222 is greater than 4. That is, the plurality of block sizes includes predetermined blocks having a horizontal dimension 220 that is at least four times the vertical dimension 222 and / or predetermined blocks having a vertical dimension 222 that is at least four times the horizontal dimension 220. Figure 7 Predetermined blocks 18 having a block size corresponding to an aspect ratio greater than 4 may be shown.

[0138] In the current VVC working draft, for each block 18 on which MIP may be used, the number of MIP modes is an odd integer N=2n+1. Here, mode 0 may not be transposed, while all other modes may be transposed: for i>n, MIP mode i is the transposition of MIP mode in. It is asserted that the restriction that mode 0 may not be transposed has originally been made due to the fact that in the first adopted version of MIP from the 15th JVET meeting in Geneva, the MIP modes were signaled by an MPM scheme with 3 MPMs (Most Probable Modes) [1]. In this scheme, the total number of MIP modes is N=2 k +3, where k is an integer. However, at the 16th JVET conference in Gothenburg, the latter signaling scheme for MIP modes was replaced by a truncated binary code [2]. Therefore, it is asserted that the number of MIP modes no longer needs to be odd, and thus the special treatment for the transposed 0th MIP mode can be removed.

[0139] Therefore, in clause 8.4.5.2.1 of the current VVC draft, for the MIP mode predModIntra, the transposed flag isTransposed that determines whether the mode is transposed is defined as:

[0140] transposed_flag=predModeIntra&1.

[0141] Therefore, it is proposed in clause 8.4.5.2.1 that the index modeId 200 which determines which matrix 19 is to be used for a given MIP mode predModIntra is defined as:

[0142] modeId=predModeIntra>>1.

[0143] In this way, signaling a mode and its transposed mode is equally expensive in terms of VLC length. No asymmetry is created. No MPM list construction is necessary.

[0144] Furthermore, for MipSizeId=0, it is proposed to reduce the number of MIP modes from 35 to 32, and for MipSizeId=1, it is proposed to reduce the number of MIP modes from 19 to 16. For MipSizeId=2, it is proposed to use 12 MIP modes instead of 11, since it is proposed that the 0th MIP mode can also be transposed. Therefore, in the proposed method, for MipSizeId=0, 16 MIP matrices 19 are required instead of 18, and for MipSizeId=1, 8 MIP matrices 19 are required instead of 10. As mentioned above, for MipSizeId=2, 6 MIP matrices 19 are required.

[0145] Next, in the current VVC working draft, MIPs are excluded for blocks 18 whose aspect ratio is greater than or equal to 4. It is asserted that this restriction is unnecessary since the current VVC MIPs work for these blocks 18 without any additional design changes. It is asserted that the restriction of MIPs on block shapes was made primarily to save encoder runtime and that standardization is therefore undesirable. Therefore, it is proposed to remove the latter restriction on the use of MIPs and allow MIPs for all block shapes, such as Figure 8 As shown in .

[0146] Figure 8 An apparatus 54 for decoding a predetermined block of a picture using intra prediction is shown, the apparatus 54 being configured to predict each of the predetermined intra prediction blocks 18 of the picture by reading a mode index 200 from a data stream 12, the mode index 200 pointing to an intra prediction mode from a list 204 of matrix-based intra prediction modes. The apparatus 54 is configured to predict samples 108 of the respective predetermined intra prediction block 18 by calculating a matrix-vector product 206 between an input vector 102 derived from a reference sample 17 in a neighborhood of the respective predetermined intra prediction block 18 and a prediction matrix 19 associated with the matrix-based intra prediction mode (k) pointed to by the mode index 200, and associating components of an output vector 208 obtained by the matrix-vector product 206 to sample positions 104 of the respective predetermined intra prediction block 18. The predetermined intra prediction block 18 comprises blocks whose aspect ratio is greater than 4.

[0147] Furthermore, the apparatus 54 may be configured to horizontally set the inter-sample distances 216 of the sample positions 104 of the corresponding predetermined intra prediction block and the inter-sample distances 218 of the reference samples 17 in the neighborhood of the corresponding predetermined intra prediction block 18 according to a first ratio of the horizontal size 220 of the predetermined block 18 to the horizontal default size and / or vertically set the inter-sample distances 216 of the sample positions 104 of the corresponding predetermined intra prediction block and the inter-sample distances 218 of the reference samples 17 in the neighborhood of the corresponding predetermined intra prediction block 18 according to a second ratio of the vertical size 222 of the predetermined block to the vertical default size. Optionally, the apparatus 54 is configured to use the list of matrix-based intra prediction modes for multiple block sizes. Depending on the embodiment, the apparatus may be configured to equally order the matrix-based intra prediction modes in the list 204 of matrix-based intra prediction modes for the multiple block sizes, or to adapt the order, for example, to blocks that are wider than they are tall, or vice versa, or to quadratic blocks.

[0148] The device for encoding a predetermined block of a picture using intra-frame prediction (encoder) can have the same or similar features as the device 54 for decoding a predetermined block of a picture using intra-frame prediction (decoder), wherein the encoder is configured to encode the mode index 200 into the data stream 12.

[0149] According to an embodiment, the mode index 200 is as follows Figure 6 is read / encoded as described.

[0150] Optionally, the decoder / encoder is configured to perform prediction of the samples 108 of the corresponding predetermined intra prediction block 18, as described with respect to Figures 5.1 to 7 As described in one of the.

[0151] according to Figure 9 In the embodiment shown in , if the corresponding coding unit (e.g., a predetermined intra-frame prediction block 18) is using MIP, what is proposed is to use a separate context for the MTS flag (i.e., the transform flag 300). The MTS flag 300 can indicate whether the residual transform for that block 18 should be the default transform, such as DCT-II applied horizontally and vertically, or if not, in which case subsequent syntax elements can optionally signal whether, for example, DST-VII or DCT-VIII is applied vertically and / or horizontally instead. What is asserted is that there is a statistical difference between the use of MTS for MIP and for non-MIP blocks, and this difference can be exploited by separating the corresponding context. In particular, because the MIP mode must have been learned using a specific residual transform (here, the default transform), other transforms are less likely to produce better results than heuristically designed modes such as DC, planar, and angular modes.

[0152] Figure 9 Means 54 are shown for decoding a predetermined block 18 of a picture using intra prediction.

[0153] The apparatus 54 is configured to predict 310 each of the predetermined intra-prediction blocks 18 (e.g., MIP blocks) of the picture by reading a mode index 200 from the data stream 12, the mode index 200 pointing to an intra-prediction mode from a list 204 of matrix-based intra-prediction modes; and predicting samples 108 of the respective predetermined intra-prediction block 18 by computing a matrix-vector product 206 between an input vector 102 derived from a reference sample 17 in a neighborhood of the respective predetermined intra-prediction block 18 and a prediction matrix 19 associated with the matrix-based intra-prediction mode (k) pointed to by the mode index 200, and associating components of an output vector 208 obtained by the matrix-vector product 206 to sample positions 104 of the respective predetermined intra-prediction block 18. This intra-prediction 310 of the predetermined intra-prediction block 18 may be as described with respect to Figures 5.1 to 8 The device 54 is configured to obtain a prediction signal 340 from this prediction 310 .

[0154] Furthermore, the apparatus 54 is configured to predict 310 each of further predetermined intra prediction blocks 18 (e.g., “normal” intra blocks) of the picture using one of a set of normal intra prediction modes including an angular prediction mode, a DC prediction mode, and a planar prediction mode to obtain a prediction signal 340 .

[0155] For a predetermined block 18 of the predetermined intra-prediction block and the further predetermined intra-prediction blocks, the apparatus 54 is configured to decode a transform flag 300, such as an MTS flag, from the data stream 12. This transform flag 300 may be encoded for both the predetermined intra-prediction block and the further predetermined intra-prediction blocks (i.e., MIP blocks and normal intra blocks). The transform flag 300 indicates whether the prediction residual of the predetermined block 18 is encoded into the data stream 12 using a default transform 320 or a different transform 322. The default transform 320 may be DCT-II applied horizontally and vertically, and the different transform 322 may be DST-VII or DCT-DCT-VIII applied horizontally and DST-VII or DCT-DCT-VIII applied vertically.

[0156] The apparatus 54 is configured to decode the transform flag 300 from the data stream 12 using context-adaptive binary arithmetic coding 350 using a probability model selected depending on whether the predetermined block is one of the predetermined intra-prediction blocks or one of the further predetermined intra-prediction blocks. Alternatively, the apparatus 54 is configured to use a first probability model as a probability model for decoding the transform flag 300 and to update the first probability model depending on the transform flag 300 if the predetermined block 18 is one of the predetermined intra-prediction blocks, and to use a second probability model as a probability model for decoding the transform flag and to update the second probability model depending on the transform flag 300 if the predetermined block 18 is one of the further predetermined intra-prediction blocks.

[0157] According to an embodiment, the apparatus 54 is configured to decode one or more further syntax elements indicating a different transform 322 from the data stream 12 if the transform flag 300 indicates that the prediction residual 302 of the predetermined block 18 was encoded into the data stream 12 using the different transform 322. The one or more further syntax elements may indicate a different transform 322 from a set of separable transforms, the transform using a first one of DST-VII and DCT-DCT-VIII horizontally and a second one of DST-VII and DCT-DCT-VIII vertically.

[0158] The apparatus 54 is configured to decode a prediction residual 302 for the predetermined block 18 from the data stream 12 and retransform the prediction residual 302 using an inverse transform that reverses the default transform 320 if the transform flag 300 indicates that the prediction residual 302 for the predetermined block 18 was encoded into the data stream 12 using the default transform 320, and to reverse the different transform 322 if the transform flag 300 indicates that the prediction residual 302 for the predetermined block 18 was encoded into the data stream 12 using the different transform 322, so as to obtain a prediction residual signal 330. The apparatus 54 is configured to use the prediction residual signal 330 to correct a prediction signal 340.

[0159] The means for encoding a predetermined block of a picture using intra prediction (encoder) may have the same or similar features as the means 54 for decoding a predetermined block of a picture using intra prediction (decoder), wherein the encoder is configured to perform re-transformation in the prediction loop.

[0160] According to an embodiment, the encoder is configured to encode the prediction residual 302 of the predetermined block 18 into the data stream 12 by: if the transform flag 300 indicates that the prediction residual 302 of the predetermined block 18 is to be encoded into the data stream 12 using a default transform 320, then transforming the prediction residual signal 330 using the default transform 320, and if the transform flag 300 indicates that the prediction residual 302 of the predetermined block 18 is to be encoded into the data stream 12 using a different transform 322, then transforming the prediction residual signal 330 using the different transform 322 to obtain a transformed prediction residual signal. In addition, the encoder is configured to encode the transformed prediction residual signal into the data stream 12 such that the prediction signal 340 is correctable by re-transforming the transformed prediction residual signal.

[0161] Additionally, the encoder is configured to encode the transform flag 300 into the data stream 12 using context adaptive binary arithmetic coding 350 using a probability model selected depending on whether the predetermined block 18 is one of the predetermined intra predicted blocks or one of the further predetermined intra predicted blocks.

[0162] The encoder and / or decoder may include information about Figures 5.1 to 8 One of the described features and / or functions.

[0163] The tables depicted below present experimental results under general test conditions [3] and test conditions for intra-frame CE [4]. In Tables 1 and 2, the results are reported for the proposed cleanup but without the addition of additional context for the MTS flag. In Tables 3 and 4, the results are reported for the proposed cleanup with the proposed addition of additional context for the MTS flag. In any test, no optimization was performed at the encoder compared to the VTM-6.0 anchor.

[0164] Table 1. Results of the incorporated MIP changes: Reference is VTM-6.0 anchor, Test is VTM-6.0 with the proposed cleanup and no additional context added, AI configuration

[0165] Y U V Coding Time Decoding time Category A1 -0,08％ -0,03％ -0,03％ 101％ 101％ Category A2 -0,06％ -0，04％ -0,06％ 101％ 101％ Category B -0,07％ -0,08％ -0,12％ 100％ 101％ Category C -0,03％ -0,18％ -0，01％ 101％ 97％ Category E -0,09％ 0,08％ 0,02％ 100％ 99％ overall -0,06％ -0,06％ -0,05％ 101％ 100％ Category D -0,02％ -0,04％ 0,07％ 101％ 99％ Category F -0,03％ -0,05％ -0,01％ 100％ 100％ .

[0166] Table 2. Results of the incorporated MIP changes: Reference is VTM-6.0 anchor, Test is VTM-6.0 with the proposed cleanup and no additional context added, RA configuration

[0167] Y U V Coding Time Decoding time Category A1 -0,03％ -0,11％ -0,12％ 100％ 100％ Category A2 -0,02％ 0,12％ -0,07％ 101％ 97％ Category B -0,05％ 0,03％ -0,12％ 101％ 100％ Category C -0,03％ 0,08％ -0,01％ 100％ 102％ Category E overall -0,04％ 0,03％ -0,08％ 100％ 99％ Category D -0,04％ -0,12％ -0,12％ 101％ 102％ Category F -0,01％ -0,05％ 0,00％ 102％ 102％ .

[0168] Table 3. Results of the incorporated MIP changes: Reference is VTM-6.0 anchor, Test is VTM-6.0 with the proposed cleanup and additional context added, AI configuration

[0169] Y U V Coding Time Decoding time Category A1 -0,12％ -0,16％ -0,10％ 96％ 99％ Category A2 -0,08％ -0,1％ -0,08％ 100％ 102％ Category B -0,08％ -0,07％ -0,12％ 103％ 102％ Category C -0,04％ -0,18％ -0,07％ 100％ 99％ Category E -0,13％ -0,07％ -0，05％ 101％ 105％ overall -0,09％ -0,11％ -0,08％ 100％ 101％ Category D -0,02％ 0,05％ -0,09％ 99％ 98％ Category F -0,03％ -0,1％ -0,12％ 100％ 101％ .

[0170] Table 4. Results of the incorporated MIP changes: Reference is VTM-6.0 anchor, Test is VTM-6.0 with the proposed cleanup and additional context added, RA configuration

[0171] Y U V Coding Time Decoding time Category A1 -0,05％ -0,05％ -0,08％ 100％ 99％ Category A2 -0,03％ -0,02％ 0,01％ 100％ 99％ Category B -0,07％ 0,01％ -0,04％ 101％ 102％ Category C -0,03％ 0,07％ -0,08％ 99％ 100％ Category E overall -0,05％ 0,01％ -0，05％ 100％ 99％ Category D -0,05％ -0,11％ 0,00％ 101％ 100％ Category F -0,06％ 0,06％ -0,03％ 100％ 100％

[0172] In this application, we propose the transposed use of MIP modes and the alignment of the number of MIP modes with the newly adopted signaling of MIP modes. Furthermore, we propose the removal of unnecessary restrictions on MIP usage. Finally, we propose the separation of contexts for MTS indexing depending on the MIP flag. Any or all of these changes may be advantageously adopted in the next working draft of VVC.

[0173] Implementation alternatives:

[0174] Although some aspects have been described in the context of devices, it should be clear that these aspects also represent the description of corresponding methods, wherein blocks or devices correspond to method steps or features of method steps. Similarly, the aspects described in the context of method steps also represent the description of corresponding blocks or items or features of corresponding devices. Some or all method steps can be performed by (or using) hardware devices, such as, for example, microprocessors, programmable computers, or electronic circuits. In some embodiments, one or more of the most important method steps can be performed by such devices.

[0175] Depending on certain implementation requirements, embodiments of the present invention may be implemented in hardware or software. The implementation may be performed using a digital storage medium (e.g., a floppy disk, DVD, Blu-ray disc, CD, ROM, PROM, EPROM, EEPROM, or FLASH memory) having stored thereon electronically readable control signals, the digital storage medium cooperating (or capable of cooperating) with a programmable computer system to cause the corresponding method to be performed. Thus, the digital storage medium may be computer-readable.

[0176] Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0177] Generally, embodiments of the present invention can be implemented as a computer program product with a program code, when the computer program product runs on a computer, the program code being operative for performing one of the methods. The program code may, for example, be stored on a machine-readable carrier.

[0178] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0179] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0180] A further embodiment of the inventive method is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, the digital storage medium, or the recorded medium are typically tangible and / or non-transitory.

[0181] Therefore, another embodiment of the inventive method is a data stream or signal sequence representing a computer program for executing one of the methods described herein. This data stream or signal sequence can, for example, be configured to be transmitted via a data communication connection (for example, via the Internet).

[0182] A further embodiment comprises a processing means, for example a computer or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0183] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0184] Another embodiment according to the present invention includes an apparatus or system configured to transfer (e.g., electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system may, for example, include a file server for transferring the computer program to the receiver.

[0185] In some embodiments, a programmable logic device (e.g., a field programmable gate array) can be used to perform some or all of the functions of the methods described herein. In some embodiments, the field programmable gate array can cooperate with a microprocessor to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware device.

[0186] The devices described herein may be implemented using hardware devices, or using computers, or using a combination of hardware devices and computers.

[0187] The apparatus described herein or any component of an apparatus described herein may be implemented at least partially in hardware and / or in software.

[0188] The methods described herein may be performed using a hardware device, or using a computer, or using a combination of a hardware device and a computer.

[0189] Any component of a method described herein or an apparatus described herein may be performed at least in part by hardware and / or by software.

[0190] The embodiments described above are merely illustrative of the principles of the present invention. It is to be understood that modifications and variations of the arrangements and details described herein will be apparent to others skilled in the art. Accordingly, it is intended that the present invention be limited only by the scope of the upcoming patent claims and not by the specific details presented through the description and explanation of the embodiments herein.

[0191] References

[0192] [1]B.Bross et al., "Versatile Video Coding (Draft 5)", JVET-N1001-v8, Geneva, Switzerland, March 2019

[0193] [2]B.Bross et al., "Versatile Video Coding (Draft 6)", JVET-O2001, Gothenburg, Sweden, July 2019

[0194] [3]F.Bossen et al., "JVET common test conditions and softwarereference configurations for SDR video", JVET-N1010, Geneva, Switzerland, March2019

[0195] [4] G. Van der Auwera et al., "Description of Core Experiment 3: IntraPrediction and Mode Coding", JVET-O2023, Gothenburg, Sweden, July 2019.

Claims

1. An apparatus (54) for decoding a predetermined block (18) of a picture using intra prediction, the apparatus being configured to: Each of the predetermined intra prediction blocks of the picture is predicted (310) by: reading a mode index (200) from a data stream (12), the mode index (200) pointing to an intra prediction mode in a list (204) of matrix-based intra prediction modes, predicting samples (108) of the corresponding predetermined intra prediction block (18) by computing a matrix-vector product (206) between an input vector (102) derived from reference samples (17) in a neighborhood of the corresponding predetermined intra prediction block (18) and a prediction matrix (19) associated with the matrix-based intra prediction mode (k) pointed to by the mode index (200), and associating components (210) of an output vector (208) obtained by the matrix-vector product (206) to sample positions (104) of the corresponding predetermined intra prediction block, predicting (310) each of the further predetermined intra prediction blocks of the picture using one of a set of normal intra prediction modes including an angular prediction mode, a DC prediction mode, and a planar prediction mode to obtain a prediction signal, For a predetermined block (18) among a predetermined intra-frame prediction block and another predetermined intra-frame prediction block, decoding a transform flag (300) from a data stream (12), the transform flag (300) indicating whether a prediction residual for a predetermined block (18) was encoded into the data stream (12) using a default transform (320) or a different transform (322), decoding a prediction residual (302) for the predetermined block (18) from the data stream (12), Retransforming the prediction residual using an inverse transform to obtain a prediction residual signal, the inverse transform reversing the default transform (320) if the transform flag (300) indicates that the prediction residual of the predetermined block (18) was encoded into the data stream (12) using a default transform (320), and reversing the different transform (322) if the transform flag (300) indicates that the prediction residual of the predetermined block (18) was encoded into the data stream (12) using a different transform (322), using the prediction residual signal to correct the prediction signal (340), The apparatus is configured to decode a transform flag (300) from a data stream (12) using context-adaptive binary arithmetic coding (350) using a probability model selected depending on whether the predetermined block (18) is one of the predetermined intra-frame prediction blocks or one of the further predetermined intra-frame prediction blocks.

2. The apparatus of claim 1, wherein the default transform (320) is DCT-II applied horizontally and vertically.

3. The apparatus according to claim 1 or claim 2, configured to: If the transform flag (300) indicates that the prediction residual of the predetermined block (18) was encoded into the data stream (12) using a different transform (322), one or more further syntax elements indicating the different transform (322) are decoded from the data stream (12).

4. The apparatus of claim 3 , wherein the one or more further syntax elements indicate different transforms ( 322 ) from a set of separable transforms that use a first one of DST-VII and DCT-DCT-VIII horizontally and a second one of DST-VII and DCT-DCT-VIII vertically.

5. The apparatus according to any one of the preceding claims 1, 2 and 4, configured to, in decoding a transform flag (300) from a data stream (12): If the predetermined block (18) is one of the predetermined intra-frame prediction blocks, the first probability model is used as the probability model for decoding the transform flag (300) and the first probability model is updated depending on the transform flag (300), and if the predetermined block (18) is one of the other predetermined intra-frame prediction blocks, the second probability model is used as the probability model for decoding the transform flag (300) and the second probability model is updated depending on the transform flag (300).

6. An apparatus for encoding a predetermined block (18) of a picture using intra prediction, the apparatus being configured to: Each of the predetermined intra prediction blocks of the picture is predicted as follows: inserting a mode index (200) into the data stream (12), the mode index (200) pointing to an intra-prediction mode in a list (204) of matrix-based intra-prediction modes, predicting samples (108) of the corresponding predetermined intra prediction block (18) by computing a matrix-vector product (206) between an input vector (102) derived from reference samples (17) in a neighborhood of the corresponding predetermined intra prediction block (18) and a prediction matrix (19) associated with the matrix-based intra prediction mode (k) pointed to by the mode index (200), and associating components (210) of an output vector (208) obtained by the matrix-vector product (206) to sample positions (104) of the corresponding predetermined intra prediction block, predicting each of the further predetermined intra prediction blocks of the picture using one of a set of normal intra prediction modes including an angular prediction mode, a DC prediction mode, and a planar prediction mode to obtain a prediction signal (340), For a predetermined block (18) among a predetermined intra-frame prediction block and another predetermined intra-frame prediction block, encoding a transform flag (300) into a data stream (12), the transform flag (300) indicating whether a prediction residual for a predetermined block (18) is encoded into the data stream (12) using a default transform (320) or a different transform (322), encoding a prediction residual (302) for the predetermined block (18) into the data stream (12), Retransforming the prediction residual using an inverse transform to obtain a prediction residual signal, the inverse transform reversing the default transform (320) if the transform flag (300) indicates that the prediction residual of the predetermined block (18) was encoded into the data stream (12) using a default transform (320), and reversing the different transform (322) if the transform flag (300) indicates that the prediction residual of the predetermined block (18) was encoded into the data stream (12) using a different transform (322), using the prediction residual signal to correct the prediction signal (340), The apparatus is configured to encode the transform flag (300) into the data stream (12) using context-adaptive binary arithmetic coding (350) using a probability model selected depending on whether the predetermined block (18) is one of the predetermined intra-frame prediction blocks or one of the further predetermined intra-frame prediction blocks.

7. The apparatus of claim 6, wherein the default transform (320) is DCT-II applied horizontally and vertically.

8. The apparatus according to claim 6 or claim 7, configured to: If the transform flag (300) indicates that the prediction residual of the predetermined block (18) was encoded into the data stream (12) using a different transform (322), one or more further syntax elements indicating the different transform (322) are encoded into the data stream (12).

9. The apparatus of claim 8, wherein the one or more further syntax elements indicate different transforms (322) from a set of separable transforms, the transforms using a first one of DST-VII and DCT-DCT-VIII horizontally and a second one of DST-VII and DCT-DCT-VIII vertically.

10. The apparatus according to any one of the preceding claims 6, 7 and 9, configured to, when encoding a transformation flag (300) into a data stream (12): If the predetermined block (18) is one of the predetermined intra-frame prediction blocks, the first probability model is used as the probability model for encoding the transform flag (300) and the first probability model is updated depending on the transform flag (300), and if the predetermined block (18) is one of the other predetermined intra-frame prediction blocks, the second probability model is used as the probability model for encoding the transform flag (300) and the second probability model is updated depending on the transform flag (300).

11. A method for decoding a predetermined block (18) of a picture using intra prediction, comprising: Each of the predetermined intra prediction blocks of the picture is predicted as follows: Reading a mode index (200) from a data stream, the mode index pointing to an intra prediction mode in a list (204) of matrix-based intra prediction modes, predicting samples (108) of the corresponding predetermined intra prediction block (18) by computing a matrix-vector product between an input vector (102) derived from reference samples (17) in a neighborhood of the corresponding predetermined intra prediction block (18) and a prediction matrix (19) associated with the matrix-based intra prediction mode (k) pointed to by the mode index, and associating components of the output vector obtained by the matrix-vector product to sample positions (104) of the corresponding predetermined intra prediction block, predicting each of the further predetermined intra prediction blocks of the picture using one of a set of normal intra prediction modes including an angular prediction mode, a DC prediction mode, and a planar prediction mode to obtain a prediction signal, For a predetermined block among a predetermined intra prediction block and another predetermined intra prediction block, decoding a transform flag (300) from a data stream, the transform flag (300) indicating whether a prediction residual for a predetermined block is encoded into the data stream using a default transform or a different transform, decoding a prediction residual (302) of a predetermined block from a data stream, retransforming the prediction residual using an inverse transform to obtain a prediction residual signal, wherein if the transform flag indicates that the prediction residual of the predetermined block was encoded into the data stream using a default transform, the inverse transform reverses the default transform, and if the transform flag indicates that the prediction residual of the predetermined block was encoded into the data stream using a different transform, the inverse transform reverses the different transform, Use the prediction residual signal to correct the prediction signal, Therein the method comprises decoding a transform flag from a data stream using context adaptive binary arithmetic coding using a probability model selected depending on whether the predetermined block is one of predetermined intra predicted blocks or one of further predetermined intra predicted blocks.

12. A method for encoding a predetermined block (18) of a picture using intra prediction, comprising: Each of the predetermined intra prediction blocks of the picture is predicted as follows: inserting a mode index (200) into the data stream, the mode index pointing to an intra prediction mode in a list (204) of matrix-based intra prediction modes, predicting samples (108) of the corresponding predetermined intra prediction block (18) by computing a matrix-vector product between an input vector (102) derived from reference samples (17) in a neighborhood of the corresponding predetermined intra prediction block (18) and a prediction matrix (19) associated with the matrix-based intra prediction mode (k) pointed to by the mode index, and associating components of the output vector obtained by the matrix-vector product to sample positions (104) of the corresponding predetermined intra prediction block, predicting each of the further predetermined intra prediction blocks of the picture using one of a set of normal intra prediction modes including an angular prediction mode, a DC prediction mode, and a planar prediction mode to obtain a prediction signal, For a predetermined block among a predetermined intra prediction block and another predetermined intra prediction block, encoding a transform flag (300) into the data stream, the transform flag (300) indicating whether a prediction residual for a predetermined block is encoded into the data stream using a default transform or a different transform, encoding the prediction residual (302) of the predetermined block into a data stream, retransforming the prediction residual using an inverse transform to obtain a prediction residual signal, wherein if the transform flag indicates that the prediction residual of the predetermined block was encoded into the data stream using a default transform, the inverse transform reverses the default transform, and if the transform flag indicates that the prediction residual of the predetermined block was encoded into the data stream using a different transform, the inverse transform reverses the different transform, Use the prediction residual signal to correct the prediction signal, Therein the method comprises encoding a transform flag into a data stream using context adaptive binary arithmetic coding using a probability model selected depending on whether the predetermined block is one of predetermined intra predicted blocks or one of further predetermined intra predicted blocks.

13. A computer program product having a program code, said program code being operable to perform the method of one of claims 11 or 12 when the computer program product is run on a computer.

Images