Method for decoding / encoding an image, video decoder / encoder, and non-transitory digital storage medium

Abstract

A block-based decoder is configured to: divide a picture (10) having more than one color component (101, 102) and having a color sampling format into blocks using a partitioning scheme, sample each color component (101, 102) equivalently according to the color sampling format, and divide the picture (10) equivalently for each color component (101, 102) according to the partitioning scheme. Additionally, the block-based decoder is configured to: decode the first color component (101) of the picture (10) block by block by selecting, for each of the first color component blocks (18’ 11 to 18’ 1n ) of the picture (10) for intra prediction, one of a first set (508) of intra prediction modes. The first set (508) includes matrix-based intra prediction modes (5101 to 510 m ), and according to each intra prediction mode, to predict inside the block (18) by: deriving a sample value vector (514) from reference samples (17) adjacent to the inside of the block (18), calculating a matrix-vector product (512) between the sample value vector (514) and a prediction matrix (516) associated with the corresponding matrix-based intra prediction mode (5101 to 510 m ) to obtain a prediction vector (518), and predicting samples in the inside of the block (18) based on the prediction vector (518). Additionally, the block-based decoder is configured to: intra predict a predetermined second color component block (182) of the picture by using a matrix-based intra prediction mode selected for the first color component block (181) of intra prediction at the same position, and decode the second color component (102) of the picture (10) block by block.

Description

Technical Field

[0001] Embodiments in accordance with the present invention relate to an apparatus and method for encoding or decoding pictures or video using matrix-based intra prediction (MIP) for all channels in the case of 4:4:4 chroma format and a single tree. Background Art

[0002] In the current VTM, MIP is only used for the luminance component [3]. If the intra mode on a chroma intra block is the direct mode (DM), and if the intra mode of the luminance block at the same position is the MIP mode, then the chroma block must use the planar mode to generate an intra prediction signal. The main reason for asserting this treatment of the DM mode in the case of MIP is that for the 4:2:0 case or the case of a dual tree, the luminance block at the same position may have a different shape from the chroma block. Therefore, since the MIP mode cannot be applied to all block shapes, the MIP mode of the luminance block at the same position may not be applicable to the chroma block in this case.

[0003] Accordingly, there is a desire to provide concepts for more efficiently presenting picture encoding and / or video encoding to support matrix-based intra prediction for all channels. Additionally or alternatively, there is a desire to reduce the bitstream and thus reduce the signalization cost.

[0004] This is achieved by the subject matter of the independent claims of the present application.

[0005] Other embodiments in accordance with the present invention are defined by the subject matter of the dependent claims of the present application. Summary of the Invention

[0006] According to a first aspect of the present invention, the inventors of the present application have recognized that one problem encountered when attempting to use a matrix-based intra prediction mode (MIP-mode) to predict samples of a predetermined block of a picture stems from the fact that it is currently not possible to use MIP for all color components of a picture. According to a first aspect of the present application, for example, in the case where two color components sharing the same position in a picture are equally sampled and equally divided into blocks, this difficulty is overcome by using the same MIP mode for the intra prediction blocks of these two color components (i.e., the intra prediction blocks located at the same position). The inventors have found that it is advantageous to use the same MIP mode for intra prediction blocks located at the same position of two or more color components of a picture. This is based on the idea that strong correlation of the predicted signal across all color components of a picture is beneficial, and if the same intra prediction mode is used for two or more color components of an intra prediction block of a picture, this correlation can be increased. Since the same MIP mode is used for blocks located at the same position of different color components of the same picture, the bitstream can be reduced and thus the signaling cost can be reduced. In addition, the coding complexity can be reduced.

[0007] Thus, according to a first aspect of the present application, a block-based decoder / encoder is configured to: divide a picture having more than one color component and a color sampling format into blocks using a partitioning scheme, wherein each color component is equally sampled according to the color sampling format, e.g., the first color component of the picture has the same sampling as all other color components of the picture, e.g., all color components of the picture have the same spatial resolution, and wherein the picture is equally divided for each color component according to the partitioning scheme, e.g., the first color component of the picture is divided in the same way as all other color components of the picture, e.g., all color components have the same spatial resolution. The picture may consist of, for example, a luminance component and two chrominance components in a YUV, YPbPr, and / or YCbCr color space, where the luminance component and the two chrominance components represent more than one color component. For example, in an RGB color space, the picture may also consist of a red component, a green component, and a blue component representing more than one color component. The picture consists, for example, of a first color component, a second color component, and an optional third color component. Obviously, the described block-based decoder / encoder can also be used for pictures having other color components and pictures having different numbers of color components. For example, the picture is equally divided for each color component by dividing the first color component into first color component blocks, by dividing the second color component into second color component blocks, and optionally by dividing the third color component into third color components. It can be determined at the granularity of the color component blocks or in units of the color component blocks whether the color components of the picture are inter-frame predicted (i.e., inter-frame encoded) or intra-frame predicted (i.e., intra-frame encoded). The block-based decoder / encoder is configured to: decode / encode the first color component of the picture into / from a data stream in units of blocks (i.e., in units of the first color component blocks) by selecting, for each intra-frame predicted first color component block of the picture (i.e., for each of the first color component blocks associated with intra-frame prediction), an intra-frame prediction mode from a first set of intra-frame prediction modes. The first color component blocks associated with inter-frame prediction, i.e., the inter-frame predicted first color component blocks, will be processed differently. The first set of intra-frame modes includes matrix-based intra-frame prediction modes, wherein according to each intra-frame prediction mode, the samples inside the block are predicted by: deriving a vector of sample values in the reference samples adjacent to the inside of the block, calculating the matrix-vector product between the vector of sample values and a prediction matrix associated with the corresponding matrix-based intra-frame prediction mode to obtain a prediction vector, and predicting the samples inside the block based on the prediction vector. Additionally, the block-based decoder / encoder is configured to: intra-frame predict a predetermined second color component block of the picture using the matrix-based intra-frame prediction mode selected for the intra-frame predicted first color component block located at the same position, to decode / encode the second color component of the picture in units of blocks.The first color component of the picture may be the luminance component of the picture, and the second color component of the picture may be the chrominance component of the picture.

[0008] According to an embodiment, the number of components of the prediction vector is less than the number of samples in the block interior, and the block-based decoder / encoder is configured to: predict the samples based on the prediction vector by interpolating the samples in the block interior based on the components of the prediction vector assigned to the support sample positions in the block interior. This is based on the idea that such a prediction vector can be obtained by reducing the total number of multiplications required in the calculation of the matrix-vector product, and thus the complexity and signaling cost of the decoder / encoder can be reduced.

[0009] According to an embodiment, the block-based decoder / encoder is configured to: select one of a first option and a second option for each of the second color component blocks for intra prediction of the picture (i.e., for each of the second color component blocks associated with intra prediction). In the first option, the intra prediction mode for the corresponding second color component block for intra prediction is derived based on the intra prediction mode selected for the first color component block for intra prediction at the same position, such that when the intra prediction mode selected for the first color component block for intra prediction at the same position is one of the matrix-based intra prediction modes, the intra prediction mode for the corresponding second color component block for intra prediction is equal to the intra prediction mode selected for the first color component block for intra prediction at the same position. In the second option, the intra prediction mode for the corresponding second color component block for intra prediction is selected based on the intra mode index for the corresponding second color component block for intra prediction signaled / present in the data stream. For example, the decoder / encoder may be configured to: select the first option when the second color component block for intra prediction indicates a direct mode or a residual coding color transform mode. Otherwise, by selecting the second option, for example, the mode signaled in the data stream is used for the prediction of the second color component block for intra prediction.

[0010] According to an embodiment, the block-based decoder / encoder is configured such that: a matrix-based intra prediction mode included in a first set of intra prediction modes is selected depending on a block size of a first color component block of the intra prediction, so as to be a subset of matrix-based intra prediction modes in a set of disjoint subsets of matrix-based intra prediction modes. For example, each subset of matrix-based intra prediction modes may be associated with a specific block size. For example, only matrix-based intra prediction modes of the selected subset are included in the first set of intra prediction modes. For different block sizes, the first set of intra prediction modes (from which an intra prediction mode for a first color component block of the intra prediction is selected) may be different. Accordingly, the decoder / encoder is configured to preselect an intra prediction mode that may be suitable for a first color component block of the intra prediction based on the block size of the first color component block of the intra prediction. This may reduce the complexity and signaling cost of the decoder / encoder.

[0011] According to an embodiment, a prediction matrix associated with a set of disjoint subsets of matrix-based intra prediction modes is machine-learned, prediction matrices included in a subset of matrix-based intra prediction modes have equal sizes, and prediction matrices included in two subsets of matrix-based intra prediction modes selected for different block sizes have different sizes. Each subset may group multiple matrix-based intra prediction modes associated with prediction matrices of the same size.

[0012] According to an embodiment, the block-based decoder / encoder is configured such that: a prediction matrix associated with a matrix-based intra prediction mode included in a first set of intra prediction modes has an equal size and is machine-learned.

[0013] According to an embodiment, the block-based decoder / encoder is configured such that: in addition to matrix-based intra prediction modes included in a first set of intra prediction modes, the intra prediction modes included in the first set of intra prediction modes include a DC mode, a planar mode, and a directional mode. In addition to matrix-based intra prediction modes, the first set of intra prediction modes may include, for example, a DC mode and / or a planar mode and / or a directional mode.

[0014] According to an embodiment, a block-based decoder / encoder is configured to select a partitioning scheme from a set of partitioning schemes. The set of partitioning schemes includes other partitioning schemes according to which a picture is partitioned for a first color component using first partitioning information present / signaled in the data stream, and the picture is partitioned for a second color component using second partitioning information present / signaled in the data stream and separate from the first partitioning information. Thus, the set of partitioning schemes includes, for example: a partitioning scheme according to which the picture is partitioned equivalently for each color component; and other partitioning schemes according to which the picture is partitioned for the first color component in a different way than for the second color component.

[0015] According to an embodiment, the block-based decoder / encoder is configured to: select one of a first option and a second option for each of the second color component blocks for intra prediction of a picture, e.g., as already indicated above. In the first option, based on the intra prediction mode selected for the first color component block for intra prediction at the same position, an intra prediction mode for the second color component block for the corresponding intra prediction is derived such that in the case where the intra prediction mode selected for the first color component block for intra prediction at the same position is one of the matrix-based intra prediction modes, the intra prediction mode for the second color component block for the corresponding intra prediction is equal to the intra prediction mode selected for the first color component block for intra prediction at the same position. In the second option, an intra prediction mode for the second color component block for the corresponding intra prediction is selected based on the intra mode index for the second color component block for the corresponding intra prediction present / signaled in the data stream. In addition, the block-based decoder / encoder is configured to: partition another picture having a color sampling format with more than one color component into other blocks using another partitioning scheme (i.e., e.g., as described above, partitioning another picture differently for the first color component than for the second color component), where each color component is sampled equivalently according to the color sampling format. The first color component of the other picture may be partitioned into other first color component blocks, and the second color component of the other picture may be partitioned into other second color component blocks. The decoder / encoder is configured to, e.g., decode / encode the first color component of the other picture in units of other blocks by selecting one intra prediction mode from a first set of intra prediction modes for each of the other first color component blocks for intra prediction of the other picture. Additionally, the decoder / encoder may be configured to: select one of the first option and the second option for each of the other second color component blocks for intra prediction of the other picture. In the first option, based on the intra prediction mode selected for the other first color component block at the same position, an intra prediction mode for the other second color component block for the corresponding intra prediction is derived such that in the case where the intra prediction mode selected for the other first color component block at the same position is one of the matrix-based intra prediction modes, the intra prediction mode for the other second color component block for the corresponding intra prediction is equal to the planar intra prediction mode. In the second option, an intra prediction mode for the other second color component block for the corresponding intra prediction is selected based on the intra mode index for the other second color component block for the corresponding intra prediction present / signaled in the data stream. As already outlined above, the first option, e.g., indicates that the prediction mode for the second color component block for intra prediction is directly derived from the prediction mode of the first color component block for intra prediction at the same position. Such derivation may depend on the partitioning scheme selected for the picture.Thus, compared with other pictures, the implementation of the first option for this picture can be different because this picture is partitioned using a partitioning scheme that equally divides the picture for each color component, while other pictures are partitioned using other partitioning schemes. According to the other partitioning scheme, the picture is partitioned for the first color component using the first partitioning information present / signaled in the data stream, and the picture is partitioned for the second color component using the second partitioning information separate from the first partitioning information present / signaled in the data stream.

[0016] According to an embodiment, the block-based decoder / encoder is configured to: select one of a first option and a second option for each of the second color component blocks of the intra prediction of a picture, e.g., as already indicated above. In the first option, based on the intra prediction mode selected for the first color component block of the intra prediction at the same location, an intra prediction mode for the second color component block of the corresponding intra prediction is derived such that in the case where the intra prediction mode selected for the first color component block of the intra prediction at the same location is one of the matrix-based intra prediction modes, the intra prediction mode for the second color component block of the corresponding intra prediction is equal to the intra prediction mode selected for the first color component block of the intra prediction at the same location. In the second option, based on the intra mode index for the second color component block of the corresponding intra prediction present / signaled in the data stream, an intra prediction mode for the second color component block of the corresponding intra prediction is selected. Further, the block-based decoder / encoder is configured to partition yet other pictures having more than one color component and having different color sampling formats, wherein the more than one color component is sampled differently according to the different color sampling formats, and to decode / encode the first color component of the yet other pictures in units of yet other blocks by selecting one intra prediction mode from a first set of intra prediction modes for each of the yet other first color component blocks of the intra prediction of the yet other pictures. The block-based decoder / encoder may be configured to: select one of the first option and the second option for each of the yet other second color component blocks of the intra prediction of the yet other pictures. In the first option, based on the intra prediction mode selected for the yet other first color component block at the same location, an intra prediction mode for the yet other second color component block of the corresponding intra prediction is derived such that in the case where the intra prediction mode selected for the yet other first color component block at the same location is one of the matrix-based intra prediction modes, the intra prediction mode for the yet other second color component block of the corresponding intra prediction is equal to the planar intra prediction mode. In the second option, based on the intra mode index for the yet other second color component block of the corresponding intra prediction present / signaled in the data stream, an intra prediction mode for the yet other second color component block of the corresponding intra prediction is selected. As already outlined above, the first option indicates, e.g., that the prediction mode for the second color component block of the intra prediction is directly derived from the prediction mode of the first color component block of the intra prediction at the same location. Such derivation may depend on the color sampling format of the picture. Thus, for this picture, the implementation of the first option may be different compared to other pictures because this picture has a color sampling format according to which each color component is sampled equally, while other pictures have different color sampling formats according to which more than one color component is sampled differently.

[0017] According to an embodiment, a block-based decoder / encoder is configured to: select a partitioning scheme from a set of partitioning schemes, the set of partitioning schemes including other partitioning schemes, according to which other partitioning schemes, a picture is partitioned for a first color component using first partitioning information in a data stream, and the picture is partitioned for a second color component using second partitioning information that is present / signaled in the data stream and separate from the first partitioning information. The block-based decoder / encoder is configured to: partition yet other pictures having more than one color component using the partitioning scheme or other partitioning schemes.

[0018] According to an embodiment, a block-based decoder / encoder is configured to: if a residual coding color transform mode is signaled in the data stream as being deactivated for a second color component block for corresponding intra prediction, perform a selection from a first option and a second option depending on what is present / signaled in the data stream for the second color component block for the corresponding intra prediction; and if a residual coding color transform mode is signaled in the data stream as being activated for a second color component block for corresponding intra prediction, perform a selection from the first option and the second option by inferring that the first option is to be selected.

[0019] An embodiment relates to a method for block-based decoding / encoding, the method including partitioning a picture having more than one color component and having a color sampling format into blocks using a partitioning scheme, where each color component is sampled equivalently according to the color sampling format, and the picture is partitioned equivalently for each color component according to the partitioning scheme. Further, the method includes: decoding / encoding a first color component of the picture on a block-by-block basis by selecting, for each of first color component blocks for intra prediction of the picture, one intra prediction mode from a first set of intra prediction modes, the first set including matrix-based intra prediction modes, where according to each intra prediction mode, samples inside the block are predicted by: deriving a vector of sample values from reference samples adjacent to the inside of the block, calculating a matrix-vector product between the vector of sample values and a prediction matrix associated with the corresponding matrix-based intra prediction mode to obtain a prediction vector, and predicting samples inside the block based on the prediction vector. Additionally, the method includes: decoding / encoding a second color component of the picture on a block-by-block basis by performing intra prediction on a predetermined second color component block of the picture using a matrix-based intra prediction mode selected for a first color component block for intra prediction located at the same position.

[0020] The above method is based on the same considerations as the above encoder / decoder. Thus, the method can be accomplished with all the features and functions that have also been described with respect to the encoder and / or decoder.

[0021] An embodiment relates to a data stream having a picture or video encoded therein using the method for encoding described herein.

[0022] The embodiments relate to a computer program having program code for performing the methods described herein when run on a computer. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings are not necessarily to scale, but rather typically emphasize illustrating the principles of the invention. In the following description, various embodiments of the invention are described with reference to the following drawings, in which:

[0024] Figure 1 An embodiment encoded into a data stream is shown;

[0025] Figure 2 An embodiment of an encoder is shown;

[0026] Figure 3 An embodiment of the reconstruction of a picture is shown;

[0027] Figure 4 An embodiment of a decoder is shown;

[0028] Figure 5 A schematic diagram of prediction of a block for encoding and / or decoding according to an embodiment is shown;

[0029] Figure 6 A matrix operation for block prediction for encoding and / or decoding according to an embodiment is shown;

[0030] Figure 7.1 Block prediction with a reduced sample value vector according to an embodiment is shown;

[0031] Figure 7.2 Block prediction using sample interpolation according to an embodiment is shown;

[0032] Figure 7.3 Block prediction with a reduced sample value vector according to an embodiment, where only some of the boundary samples are averaged;

[0033] Figure 7.4 Block prediction with a reduced sample value vector according to an embodiment, where the grouping of four boundary samples is averaged;

[0034] Figure 8 A matrix operation performed by a device according to an embodiment is shown;

[0035] Figures 9a to 9c A detailed matrix operation performed by a device according to an embodiment is shown;

[0036] Figure 10 A detailed matrix operation performed by a device using offset and scaling parameters according to an embodiment is shown;

[0037] Figure 11 illustrates detailed matrix operations performed by a device using offset and scaling parameters according to different embodiments;

[0038] Figure 12 illustrates an embodiment of prediction of a predetermined second color component block of a picture; and

[0039] Figure 13 illustrates embodiments of different predictions of a predetermined second color component block of a picture. DETAILED DESCRIPTION

[0040] Even if reference numerals appear in different figures, in the following description, the same or equivalent elements or elements having the same or equivalent functions are denoted by the same or equivalent reference numerals.

[0041] In the following description, numerous details are set forth to provide a more thorough explanation of embodiments of the present invention. However, it will be clear to those skilled in the art that the embodiments of the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form rather than specifically, to avoid obscuring embodiments of the present invention. In addition, unless otherwise specifically indicated, the features of the different embodiments described below may be combined with each other.

[0042] 1 Introduction

[0043] Next, different inventive examples, embodiments, and aspects will be described. At least some of these examples, embodiments, and aspects relate in particular to methods and / or devices for video coding and / or performing block-based prediction, such as using a linear transform or an affine transform with adjacent sample reduction, and / or for optimizing video delivery (e.g., broadcast, streaming, file playback, etc.), such as for video applications and / or virtual reality applications.

[0044] In addition, the examples, embodiments, and aspects may relate to High Efficiency Video Coding (HEVC) or successors. In addition, other embodiments, examples, and aspects will be defined by the appended claims.

[0045] It should be noted that any embodiment, example, and aspect defined by the claims may be supplemented by any details (features and functions) described in the following sections.

[0046] In addition, the embodiments, examples, and aspects described in the following sections may be used alone and may also be supplemented by any feature in another section or any feature included in the claims.

[0047] In addition, it should be noted that the individuals, examples, embodiments, and aspects described herein can be used alone or in combination. Therefore, details can be added to each of the individual aspects without adding details to another of the examples, embodiments, and aspects.

[0048] It should also be noted that the present disclosure explicitly or implicitly describes features of decoding and / or encoding systems and / or methods.

[0049] In addition, the method-related features and functions disclosed herein can also be used in a device. Moreover, any features and functions regarding a device disclosed herein can also be used in a corresponding method. In other words, the methods disclosed herein can be supplemented by any features and functions described regarding a device.

[0050] In addition, as will be described in the "Implementation Alternatives" section, any features and functions described herein can be implemented in hardware or software, or using a combination of hardware and software.

[0051] In addition, in some examples, embodiments, or aspects, any features described in parentheses ("(...) " or "[...]") can be considered optional.

[0052] 2 Encoder, Decoder

[0053] Hereinafter, various examples are described that can help achieve more efficient compression when using block-based prediction. Some examples achieve high compression efficiency by using a set of intra prediction modes. The latter can be added to other intra prediction modes designed heuristically, for example, or can be provided exclusively. And there are other examples that use the two characteristics just discussed. However, as a variant of these embodiments, intra prediction can be transformed into inter prediction by using reference samples in another picture.

[0054] To facilitate the understanding of the following examples of the present application, the description begins with a presentation suitable for possible encoders and decoders into which the subsequent overview examples of the present application can be incorporated. Figure 1 A device for encoding picture 10 block by block into data stream 12 is shown. The device is indicated by reference numeral 14 and can be a still picture encoder or a video encoder. In other words, when encoder 14 is configured to encode video 16 including picture 10 into data stream 12, or encoder 14 can exclusively encode picture 10 into data stream 12, picture 10 can be the current picture in video 16.

[0055] As described above, the encoder 14 performs encoding in a block-by-block or block-based manner. To this end, the encoder 14 subdivides the picture 10 into blocks, and the encoder 14 encodes the picture 10 into the data stream 12 in units of blocks. Possible subdivision examples of subdividing the picture 10 into blocks 18 are elaborated in more detail below. Generally, the subdivision can ultimately become blocks 18 of a constant size such as an array of blocks arranged in rows and columns, or blocks 18 of different block sizes, for example by using a hierarchical multi-tree subdivision that starts from the entire picture area of the picture 10 or starts from a pre-partitioning of the picture 10 into an array of tree blocks. These examples should not be considered as excluding other possible ways of subdividing the picture 10 into blocks 18.

[0056] Furthermore, the encoder 14 is a predictive encoder configured to predictively encode the picture 10 into the data stream 12. For a certain block 18, this means that the encoder 14 determines the prediction signal of the block 18 and encodes the prediction residual (i.e., the prediction error by which the prediction signal deviates from the actual picture content within the block 18) into the data stream 12.

[0057] The encoder 14 can support different prediction modes to derive the prediction signal for a certain block 18. A prediction mode that is important in the following examples is the intra prediction mode, according to which the inside of the block 18 is predicted spatially from adjacent, already encoded samples of the picture 10. Encoding the picture 10 into the data stream 12 and thus the corresponding decoding process can be based on a certain encoding order 20 defined between the blocks 18. For example, the encoding order 20 can traverse the blocks 18 in a raster scan order such as row by row from top to bottom and, for example, column by column from left to right. In the case of a hierarchical multi-tree-based subdivision, the raster scan sorting can be applied within each level, where a depth-first traversal order can be applied, i.e., the leaf nodes within a block of a certain level can be before the blocks of the same level that have the same parent block according to the encoding order 20. Depending on the encoding order 20, the adjacent, already encoded samples of the block 18 can generally be located on one or more sides of the block 18. In the case of the examples presented herein, for example, the adjacent, already encoded samples of the block 18 are located on the top side and the left side of the block 18.

[0058] The intra prediction mode may not be the only mode supported by the encoder 14. In the case where the encoder 14 is a video encoder, for example, the encoder 14 may also support an inter prediction mode, according to which the block 18 is predicted in time from a previously encoded picture of the video 16. Such an inter prediction mode may be a motion compensated prediction mode, according to which, for such a block 18, a motion vector is signaled, which indicates the relative spatial offset of the part from which the prediction signal for the block 18 is to be derived as a copy. Additionally or alternatively, other non-intra prediction modes may also be available, such as an inter prediction mode in the case where the encoder 14 is a multi-view encoder, or a non-prediction mode, according to which the inside of the block 18 is encoded as is, i.e., without any prediction.

[0059] Before focusing the description of the present application on the intra prediction mode, as described with respect to Figure 2 a more specific example of a possible block-based encoder (i.e., a possible implementation of the encoder 14), and then presenting two corresponding examples of a decoder that are respectively suitable for Figure 1 and Figure 2 it.

[0060] Figure 2 Shows Figure 1 a possible implementation of the encoder 14, i.e., an implementation in which the encoder is configured to use transform coding to encode the prediction residuals, although this is almost an example and the present application is not limited to such prediction residual coding. According to Figure 2, the encoder 14 includes a subtractor 22 configured to subtract a corresponding prediction signal 24 from an input signal (i.e., picture 10 or the current block 18 based on a block) to obtain a prediction residual signal 26, which is then encoded by a prediction residual encoder 28 into the data stream 12. The prediction residual encoder 28 consists of a lossy coding stage 28a and a lossless coding stage 28b. The lossy stage 28a receives the prediction residual signal 26 and includes a quantizer 30 that quantizes the samples of the prediction residual signal 26. As already described above, this example uses transform coding of the prediction residual signal 26, and thus, the lossy coding stage 28a includes a transform stage 32 connected between the subtractor 22 and the quantizer 30 to transform such spectrally decomposed prediction residual 26, where quantization by the quantizer 30 occurs on the transformed coefficients of the residual signal 26. The transform can be a DCT, DST, FFT, Hadamard transform, etc. The transformed and quantized prediction residual signal 34 is then losslessly encoded by the lossless coding stage 28b, which is an entropy encoder that entropy encodes the quantized prediction residual signal 34 into the data stream 12. The encoder 14 further includes a prediction residual signal reconstruction stage 36 connected to the output of the quantizer 30 to reconstruct the prediction residual signal from the transformed and quantized prediction residual signal 34 in a manner available to the decoder (i.e., taking into account the coding loss in the quantizer 30). To this end, the prediction residual reconstruction stage 36 includes a dequantizer 38 that performs the inverse operation of the quantization by the quantizer 30, followed by an inverse transformer 40 that performs an inverse transform relative to the transform performed by the transformer 32, such as an inverse operation of spectral decomposition, such as the inverse operation of any of the above specific transform examples. The encoder 14 includes an adder 42 that adds the reconstructed prediction residual signal output by the inverse transformer 40 to the prediction signal 24 to output a reconstructed signal, i.e., reconstructed samples. This output is fed into a predictor 44 of the encoder 14, which then determines the prediction signal 24 based on this. The predictor 44 supports all prediction modes discussed above regarding Figure 1 All prediction modes that have been discussed. Figure 2 It is also shown that in the case where the encoder 14 is a video encoder, the encoder 14 may further include a loop filter 46 that fully filters the reconstructed pictures, which, after being filtered, form reference pictures for the predictor 44 for inter-frame prediction blocks.

[0061] As already described above, the encoder 14 operates on a block basis. For the following description, the block basis of interest is the basis on which picture 10 is subdivided into blocks, for which an intra prediction mode is selected from one or more intra prediction modes supported by predictor 44 or encoder 14, respectively, and the selected intra prediction mode is performed individually. However, there may also be other kinds of blocks into which picture 10 is subdivided. For example, the above decision as to whether picture 10 is inter-coded or intra-coded can be made at a granularity or unit of blocks that deviate from block 18. For example, the inter / intra mode decision can be made at the level of coding blocks into which picture 10 is subdivided, and each coding block is further subdivided into prediction blocks. In the case of a coding block for which intra prediction has been decided, the prediction blocks are each further subdivided for an intra prediction mode decision. To this end, for each of these prediction blocks, it is decided which of the supported intra prediction modes should be used for the respective prediction block. These prediction blocks will form block 18 of interest here. The prediction blocks within a coding block associated with inter prediction will be treated differently by predictor 44. They will perform inter prediction from a reference picture by determining a motion vector and copying the prediction signal of the block from the position in the reference picture pointed to by the motion vector. Another kind of block subdivision involves subdivision into transform blocks, and the transform by transformer 32 and inverse transform by inverse transformer 40 are performed on a per transform block basis. The transform blocks can be, for example, the result of further subdividing coding blocks. Of course, the examples set forth herein should not be considered restrictive and there are other examples. For the sake of completeness only, it should be noted that the subdivision into coding blocks can, for example, use a quadtree subdivision, and coding blocks can also be further subdivided using a quadtree subdivision to obtain prediction blocks and / or transform blocks.

[0062] Figure 3 is depicted as suitable Figure 1 for decoder 54 or apparatus for block-by-block decoding of encoder 14. This decoder 54 behaves in a manner opposite to encoder 14, i.e., it decodes picture 10 from data stream 12 in a block-by-block manner and supports multiple intra prediction modes for this purpose. For example, decoder 54 can include a residual provider 156. As described above with respect to Figure 1All other possibilities discussed are also valid for decoder 54. To this end, decoder 54 can be a still picture decoder or a video decoder, and decoder 54 also supports all prediction modes and prediction possibilities. The difference between encoder 14 and decoder 54 mainly lies in that encoder 14 selects or picks encoding decisions according to some optimizations, such as minimizing some cost functions that can depend on the coding rate and / or coding distortion. One of these coding options or coding parameters can involve selecting the intra prediction mode to be used for the current block 18 among the available or supported intra prediction modes. The selected intra prediction mode can then be signaled by encoder 14 for the current block 18 in data stream 12, and decoder 54 uses this signaling in data stream 12 to redo the selection for block 18. Similarly, the subdivision of picture 10 into blocks 18 can be optimized within encoder 14, and the corresponding subdivision information can be transmitted in data stream 12, and decoder 54 subdivides picture 10 into blocks 18 based on this subdivision information. In summary, decoder 54 can be a prediction decoder based on block operations, and in addition to the intra prediction mode, decoder 54 can support other prediction modes, such as the inter prediction mode in the case where decoder 54 is a video decoder, for example. In decoding, decoder 54 can also use the coding order 20 discussed with respect to Figure 1 Since the coding order 20 is observed at both encoder 14 and decoder 54, the same neighboring samples are available for the current block 18 at both encoder 14 and decoder 54. Therefore, to avoid unnecessary repetition, the description of the operating mode of encoder 14 should also apply to decoder 54 in terms of the subdivision of picture 10 into blocks, such as in terms of prediction and in terms of the coding of prediction residuals. The difference is that encoder 14 selects some coding options or coding parameters through optimization, signals the coding parameters in data stream 12 or inserts the coding parameters into data stream 12, and then decoder 54 derives the coding parameters from data stream 12 in order to re - perform prediction, subdivision, etc.

[0063] Figure 4 is shown Figure 3 a possible implementation of decoder 54, namely an implementation suitable for Figure 1 encoder 14, as Figure 2 shown. Since Figure 4 many elements of encoder 54 are the same as the elements that appear in the corresponding encoder of Figure 2 the same reference numerals with an apostrophe are used in Figure 4 to indicate these elements. Specifically, adder 42’, optional loop filter 46’ and predictor 44’ are in the same way as they are in Figure 2is connected to the prediction loop in the same way as in the encoder. The reconstructed (i.e., dequantized and inverse-transformed) prediction residual signal applied to the adder 42’ is derived from the sequence of the entropy decoder 56 and the subsequent residual signal reconstruction stage 36’. The entropy decoder 56 performs the inverse operation of the entropy encoding of the entropy encoder 28b. The residual signal reconstruction stage 36’ consists of a dequantizer 38’ and an inverse-transformer 40’, just as in the case of the encoding side. The output of the decoder is Figure 10 the reconstruction. The reconstruction of Picture 10 can be directly available at the output of the adder 42’ or alternatively at the output of the loop filter 46’. Some post-filters can be arranged at the output of the decoder in order to perform a certain post-filtering on the reconstruction of Picture 10 to improve the picture quality, but Figure 4 this option is not described in

[0064] Again, regarding Figure 4 , except that only the encoder performs the optimization tasks and the relevant decisions regarding the encoding options, the above description regarding Figure 2 proposed should also be valid for Figure 4 . However, all the descriptions regarding block subdivision, prediction, dequantization, and inverse transformation are also valid for the decoder 54 of Figure 4 .

[0065] 3ALWIP (Affine Linear Weighted Intra Predictor)

[0066] Some non-limiting examples regarding ALWIP are discussed herein, although ALWIP does not always need to embody the techniques discussed here.

[0067] This application particularly relates to an improved block-based prediction mode concept for per-block picture encoding that can be used in video codecs such as HEVC or any successor of HEVC. The prediction mode can be an intra prediction mode, but theoretically the concepts described herein can also be transferred to an inter prediction mode where the reference samples are part of another picture.

[0068] A block-based prediction concept is sought that allows for an efficient implementation (e.g., a hardware-friendly implementation).

[0069] This object is achieved by the subject matter of the independent claims of this application.

[0070] Intra prediction modes are widely used in picture coding and video coding. In video coding, intra prediction modes compete with other prediction modes, such as inter prediction modes like motion-compensated prediction modes. In an intra prediction mode, the current block is predicted based on neighboring samples, which are samples that have been encoded on the encoder side and decoded on the decoder side. The neighboring sample values are extrapolated into the current block to form a prediction signal for the current block, where the prediction residual is transmitted in the data stream for the current block. The better the prediction signal, the lower the prediction residual, and thus, the fewer bits required to encode the prediction residual.

[0071] To be effective, several aspects should be considered to form an effective framework for intra prediction in a per-block picture coding environment. For example, the larger the number of intra prediction modes supported by the codec, the greater the auxiliary information rate consumption to signal this choice to the decoder. On the other hand, the set of supported intra prediction modes should be able to provide a good prediction signal, i.e., a prediction signal that results in a low prediction residual.

[0072] Hereinafter, as a comparative example or a basic example, a device (encoder or decoder) for decoding pictures block by block from a data stream is disclosed, which supports at least one intra prediction mode, according to which a prediction signal for a block of a predetermined size of a picture is determined by applying a first template of samples adjacent to the current block to an affine linear predictor, which will be referred to hereinafter as an affine linear weighted intra predictor (ALWIP).

[0073] The device may have at least one of the following attributes (which may also apply to a method or another technology, e.g., implemented in a non-transitory storage unit storing instructions which, when executed by a processor, cause the processor to implement the method and / or act as the device):

[0074] 3.1 The predictor can be complementary to other predictors

[0075] The intra prediction modes that can form the subject of an improved implementation described further below can be complementary to other intra prediction modes of the codec. Thus, they can be complementary to the DC prediction mode, the planar prediction mode, or the angular prediction mode defined in the HEVC codec (the corresponding JEM reference software). The latter three intra prediction modes of the intra prediction modes should be referred to hereinafter as traditional intra prediction modes. Thus, for a given block in the intra mode, a flag needs to be parsed by the decoder, which indicates whether one of the intra prediction modes supported by the device is to be used.

[0076] 3.2 More than one proposed prediction mode

[0077] The apparatus may include more than one ALWIP mode. Thus, in the case where the decoder knows one of the ALWIP modes supported by the apparatus to be used, the decoder needs to parse additional information indicating which one of the ALWIP modes supported by the apparatus is to be used.

[0078] The signaling of the supported modes may have the following characteristics: Encoding of some ALWIP modes may require less binary data than other ALWIP modes. Which of these modes require less binary data and which require more binary data may either depend on information extractable from the already decoded bitstream or may be pre-fixed.

[0079] 4 Some aspects

[0080] Figure 2 A decoder 54 for decoding a picture from a data stream 12 is shown. The decoder 54 may be configured to decode a predetermined block 18 of the picture. Specifically, the predictor 44 may be configured to map a set of P neighboring samples adjacent to the predetermined block 18 to a set of Q predicted values for the samples of the predetermined block using a linear transformation or an affine linear transformation [e.g., ALWIP].

[0081] As Figure 5 shown, the predetermined block 18 includes Q values to be predicted (which will be "predicted values" at the end of the operation). If the block 18 has M rows and N columns, then Q = M·N. The Q values of the block 18 may be in the spatial domain (e.g., pixels) or in the transform domain (e.g., DCT, discrete wavelet transform, etc.). The Q values of the block 18 may be predicted based on P values taken from neighboring blocks 17a to 17c that are typically adjacent to the block 18. The P values of the neighboring blocks 17a to 17c may be in the position closest to the block 18 (e.g., adjacent). The P values of the neighboring blocks 17a to 17c have been processed and predicted. The P values are indicated as the values in the parts 17’a to 17’c to distinguish them from the blocks to which they belong (in some examples, 17’b is not used).

[0082] As Figure 6As shown, in order to perform prediction, it is possible to operate using the following: a first vector 17P having P entries (each entry associated with a specific position in adjacent parts 17'a to 17'c), a second vector 18Q having Q entries (each entry associated with a specific position in block 18), and a mapping matrix 17M (each row associated with a specific position in block 18 and each column associated with a specific position in adjacent parts 17'a to 17'c). The mapping matrix 17M thus predicts the P values of adjacent parts 17'a to 17'c as the values of block 18 according to a predetermined pattern. The entries in the mapping matrix 17M can thus be understood as weighting factors. In the following paragraphs, we will use labels 17a to 17c instead of 17'a to 17'c to refer to the adjacent parts of the boundary.

[0083] In the art, several traditional patterns are known, such as the DC pattern, the planar pattern, and 65 directional prediction patterns. For example, 67 patterns may be known.

[0084] However, it has been noted that different patterns can also be used, here referred to as linear transforms or affine linear transforms. A linear transform or affine linear transform includes P·Q weighting factors, where at least 1 / 4P·Q of the weighting factors are non-zero weight values, and for each of the Q predicted values, the non-zero weight values include a series of P weighting factors associated with each predicted value. When arranged one below the other in raster scan order in the samples of a predetermined block, the series forms an omnidirectional non-linear envelope.

[0085] It is possible to map the following: the P positions of adjacent values 17'a to 17'c (templates), the Q positions of adjacent samples 17'a to 17'c, and at the values of the P*Q weighting factors in matrix 17M. A plane is an example of an envelope for the series used for DC transform (it is the plane for DC transform). The envelope is clearly planar and is thus excluded by the definition of linear transform or affine linear transform (ALWIP). Another example is a matrix that produces an angular pattern: the envelope will be excluded from the ALWIP definition, and frankly, the envelope looks like a hill sloping from top to bottom along a direction in the P / Q plane. The planar pattern and the 65 directional prediction patterns will have different envelopes, which will be linear in at least one direction (i.e., all directions for exemplary DC and the hill direction for angular patterns).

[0086] Conversely, the envelope of a linear or affine transformation will not be omnidirectionally linear. It has been understood that in some cases, this type of transformation can be optimal for performing the prediction of block 18. It has been noted that preferably, at least 1 / 4 of the weighting factors are different from zero (i.e., at least 25% of the P*Q weighting factors are different from 0).

[0087] According to any regular mapping rule, the weighting factors can be uncorrelated with each other. Thus, the matrix 17M can be such that the values of its entries do not have an obviously recognizable relationship. For example, the weighting factors cannot be described by any analytical or differential function.

[0088] In an example, the ALWIP transformation is such that: the average of the maximum values of the cross-correlation between a first series of weighting factors associated with a corresponding predicted value and a second series of weighting factors (or the reverse version of the latter series, whichever results in a higher maximum value) associated with predicted values other than that corresponding predicted value, can be lower than a predetermined threshold (e.g., 0.2 or 0.3 or 0.35 or 0.1, e.g., a threshold within the range between 0.05 and 0.035). For example, for each pair of rows (i1, i2) of the ALWIP matrix 17M, the cross-correlation can be calculated by multiplying the P values of the i1-th row by the P values of the i2-th row. For each obtained cross-correlation, the maximum value can be obtained. Thus, the mean (average value) of the entire matrix 17M can be obtained (i.e., averaging the maximum values of the cross-correlations in all combinations). Thereafter, the threshold can be, for example, 0.2 or 0.3 or 0.35 or 0.1, e.g., a threshold within the range between 0.05 and 0.035.

[0089] The P adjacent samples of blocks 17a to 17c can be located along a one-dimensional path that extends along the boundary of a predetermined block 18 (e.g., 18c, 18a). For each of the Q predicted values of the predetermined block 18, the series of P weighting factors associated with the corresponding predicted value can be sorted in a manner that traverses the one-dimensional path in a predetermined direction (e.g., from left to right, from top to bottom, etc.).

[0090] In an example, the ALWIP matrix 17M can be non-diagonal or non-block diagonal.

[0091] An example of the ALWIP matrix 17M for predicting a 4x4 block 18 from 4 already predicted adjacent samples can be:

[0092] {

[0093] {37,59,77,28},

[0094] {32,92,85,25},

[0095] {31,69,100,24},

[0096] {33, 36, 106, 29},

[0097] {24, 49, 104, 48},

[0098] {24, 21, 94, 59},

[0099] {29, 0, 80, 72},

[0100] {35, 2, 66, 84},

[0101] {32, 13, 35, 99},

[0102] {39, 11, 34, 103},

[0103] {45, 21, 34, 106},

[0104] {51, 24, 40, 105},

[0105] {50, 28, 43, 101},

[0106] {56, 32, 49, 101},

[0107] {61, 31, 53, 102},

[0108] {61, 32, 54, 100}

[0109] }。

[0110] (Here, {37, 59, 77, 28} is the first row; {32, 92, 85, 25} is the second row; and {61, 32, 54, 100} is the 16th row of matrix 17M.) Matrix 17M has a size of 16x4 and includes 64 weighting factors (as a result of 16 * 4 = 64). This is because matrix 17M has a size QxP, where Q = M * N, which is the number of samples of block 18 (block 18 is a 4x4 block) to be predicted, and P is the number of samples of the predicted samples. Here, M = 4, N = 4, Q = 16 (as a result of M * N = 4 * 4 = 16), and P = 4. The matrix is non-diagonal and non-block diagonal and is not described by a specific rule.

[0111] It can be seen that less than 1 / 4 of the weighting factors are 0 (in the case of the matrix shown above, one of the 64 weighting factors is 0). When these values are arranged one below the other in raster scan order, the envelope formed by these values forms an omnidirectional non-linear envelope.

[0112] Even though the above description has been mainly discussed with reference to a decoder (e.g., decoder 54), the same operations may be performed at an encoder (e.g., encoder 14).

[0113] In some examples, for each block size (in a set of block sizes), the ALWIP transforms of the intra prediction modes within a second set of intra prediction modes for the corresponding block size are different from each other. Additionally or alternatively, the cardinality of the second set of intra prediction modes for the block sizes in the set of block sizes may coincide, but the associated linear transforms or affine linear transforms of the intra prediction modes within the second set of intra prediction modes for different block sizes may not be convertible to each other by scaling.

[0114] In some examples, the ALWIP transforms may be defined in such a way that they "have no sharing" with traditional transforms (e.g., the ALWIP transforms may "have no" sharing with the corresponding traditional transforms even though they have been mapped via one of the above mappings).

[0115] In an example, the ALWIP mode is used for both the luminance component and the chrominance component, but in other examples, the ALWIP mode is used for the luminance component but not for the chrominance component.

[0116] 5 Affine Linear Weighted Intra Prediction Mode with Encoder Acceleration (e.g., Test CE3-1.2.1)

[0117] 5.1 Description of the Method or Apparatus

[0118] The Affine Linear Weighted Intra Prediction (ALWIP) mode tested in CE3-1.2.1 may be the same as that proposed in JVET-L0199 under Test CE3-2.2.2 except for the following changes:

[0119] · Coordination with multi-reference line (MRL) intra prediction, especially encoder estimation and signaling, i.e., MRL is not combined with ALWIP, and the transmission of the MRL index is limited to non-ALWIP blocks.

[0120] · Subsampling is now mandatory for all blocks with W×H≥32×32 (previously optional for 32×32); thus, additional tests at the encoder and for sending the subsampling flag have been removed.

[0121] · ALWIP for 64×N and N×64 blocks (N≤32) has been added by downsampling to 32×N and N×32 respectively and applying the corresponding ALWIP mode.

[0122] In addition, Test CE3-1.2.1 includes the following encoder optimizations for ALWIP:

[0123] ·Combined mode estimation: The traditional mode and the ALWIP mode use a shared Hadamard candidate list for full RD estimation, i.e., ALWIP mode candidates are added to the same list as traditional (and MRL) mode candidates based on the Hadamard cost.

[0124] ·For the combined mode list, fast intra EMT and fast intra PB are supported, with additional optimizations to reduce the number of full RD checks.

[0125] ·Following the same method as the traditional mode, only the MPMs of the available left and above blocks are added to this list for full RD estimation for ALWIP.

[0126] 5.2 Complexity evaluation

[0127] In Test CE3-1.2.1, excluding the computation of calling the discrete cosine transform, each sample requires at most 12 multiplications to generate the prediction signal. Additionally, a total of 136,492 parameters are required, each parameter being 16 bits. This corresponds to 0.273 megabytes of memory.

[0128] 5.3 Experimental results

[0129] The tests are evaluated according to the common test conditions JVET-J1010 0, for the intra-only (AI) and random access (RA) configurations, with the VTM software version 3.0.1. The corresponding simulations are carried out on an Intel Xeon cluster (E5-2697Av4, AVX2 enabled, Intel Turbo Boost Technology turned off) with a Linux OS and a GCC 7.2.1 compiler.

[0130] Table 1 Results of CE3-1.2.1 for VTM AI configuration

[0131]

[0132]

[0133] Table 2 Results of CE3-1.2.1 for VTM RA configuration

[0134]

[0135] 5.4 Complexity reduction for affine linear weighted intra prediction (e.g., Test CE3-1.2.2)

[0136] The technique tested in CE2 is related to the "affine linear intra prediction" described in JVET-L0199[1], but it is simplified in terms of memory requirements and computational complexity:

[0137] · There may be only three different prediction matrices (e.g., S0, S1, S2, see below) and bias vectors (e.g., for providing offset values) that cover all block shapes. Thus, the number of parameters is reduced to 14,400 10-bit values, which is less than the memory stored in a 128×128 CTU.

[0138] · The input size and output size of the predictor are further reduced. In addition, instead of transforming the boundaries via DCT, averaging or downsampling can be performed on the boundary samples, and the generation of the prediction signal can use linear interpolation instead of the inverse DCT. Thus, at most four multiplications per sample are required to generate the prediction signal.

[0139] 6. Example

[0140] How to perform some predictions with ALWIP prediction is discussed here (e.g., as Figure 6 shown).

[0141] In principle, referring to Figure 6 , to obtain the Q = M*N values of the MxN block 18 to be predicted, the Q*P samples of the QxP ALWIP prediction matrix 17M should be multiplied by the P samples of the Px1 adjacent vector 17P. Thus, generally, to obtain each of the Q = M*N values of the MxN block 18 to be predicted, at least P = M + N value multiplications are required.

[0142] These multiplications have a very adverse impact. The dimension P of the boundary vector 17P usually depends on the number M + N of boundary samples (binary data or pixels) 17a, 17c adjacent (e.g., neighboring) to the M×N block 18 to be predicted. This means that if the size of the block 18 to be predicted is large, the number M + N of boundary pixels (17a, 17c) is correspondingly large, thus increasing the dimension P = M + N of the Px1 boundary vector 17P and the length of each row of the QxP ALWIP prediction matrix 17M, and thus also increasing the number of necessary multiplications (generally, Q = M*N = W*H, where W (width) is another symbol for N and H (height) is another symbol for M; in the case where the boundary vector consists of only one row of samples and / or one column of samples, P is P = M + N = H + W).

[0143] This problem is usually exacerbated by the fact that in a microprocessor-based system (or other digital processing system), multiplication is generally a power-consuming operation. It can be imagined that a large number of multiplications on a very large number of samples of a large number of blocks will result in a waste of computing power, which is generally undesirable.

[0144] Therefore, preferably, the number of multiplications Q*P required to predict the MxN block 18 is reduced.

[0145] It has been understood that by intelligently selecting alternative operations that are easier to process instead of multiplication, it is possible to somehow reduce the computational power required for each intra-prediction of each block 18 to be predicted.

[0146] Specifically, referring to Figures 7.1 to 7.4 , it has been understood that an encoder or a decoder can use multiple adjacent samples (e.g., 17a, 17c) to predict a predetermined block (e.g., 18) of a picture by performing the following operations:

[0147] Reducing (e.g., in step 811) (e.g., by averaging or downsampling) multiple adjacent samples (e.g., 17a, 17c) to obtain a set of reduced sample values that is lower in the number of samples compared to the multiple adjacent samples,

[0148] Performing (e.g., in step 812) a linear transformation or an affine linear transformation on the set of reduced sample values to obtain the predicted values of the predetermined samples of the predetermined block.

[0149] In some cases, the decoder or the encoder can also derive the predicted values of other samples of the predetermined block based on the predicted values of the predetermined samples and the predicted values of the multiple adjacent samples, for example, by interpolation. Thus, an upsampling strategy can be obtained.

[0150] In an example, it is possible to perform (e.g., in step 811) some averaging on the samples of the boundary 17 in order to obtain a set of reduced samples 102 with a reduced number of samples (at least one of the samples in the reduced number of samples 102 can be the average of two samples of the original boundary samples or selected from the original boundary samples) ( Figures 7.1 to 7.4 ). For example, if the original boundary has P = M + N samples, the set of reduced samples can have P red = M red + N red samples, where M red < M and N red < N, and at least one of them satisfies such that P red < P. Therefore, the boundary vector 17P actually used for prediction (e.g., in step 812b) will not have Px1 entries, but will have P red x1 entries, where P red < P. Similarly, the ALWIP prediction matrix 17M selected for prediction will not have a QxP size, but will have a reduced number of matrix elements, QxP red (or Q red xP red , see below) size, at least because P red < P (by means of M red < M and N red < N, and at least one of them).

[0151] In some examples (e.g., Figure 7.2 , Figure 7.3 ), if the block obtained by ALWIP (at step 812) is a block having dimensions of M′ red ×N′ red , where M′ red <M and / or N′ red <N (i.e., the samples directly predicted by ALWIP are fewer in number than the samples of block 18 to be actually predicted). Thus, set Q red = M′ red *N′ red , which will obtain the ALWIP prediction by using Q red *P red multiplications instead of Q*P red multiplications (where Q red *P red <Q*P red <Q*P). This multiplication will predict the reduced block, which has dimensions of M′ red ×N′ red . Nevertheless, it will be possible to perform (e.g., at subsequent step 813) upsampling from the reduced M′ red ×N′ red prediction block to the final MxN prediction block (e.g., obtained by interpolation).

[0152] Although matrix multiplication involves a reduced number of multiplications (Q red *P red or Q*P red ), both the initial reduction (e.g., averaging or downsampling) and the final transformation (e.g., interpolation) can be performed by reducing (or even avoiding) multiplications, so these techniques can be advantageous. For example, downsampling, averaging, and / or interpolation can be performed (e.g., at steps 811 and / or 813) by employing computationally less power - demanding binary operations such as addition and shift.

[0153] In addition, addition is a very easy operation that can be easily performed without a large amount of computational work.

[0154] The shift operation can be used, for example, to average two boundary samples and / or to interpolate two samples (support values) of the reduced prediction block (or taken from the boundary) to obtain the final prediction block. (For interpolation, two sample values are required. Inside the block, we always have two predetermined values, but to interpolate samples along the left and upper boundaries of the block, we only have one predetermined value, as Figure 7.2 shown, so we use the boundary samples as support values for interpolation.)

[0155] A two - step process can be used, for example:

[0156] First, sum the values of the two samples;

[0157] Then, halve the value of the sum (e.g., by a right shift).

[0158] Alternatively, it is possible to:

[0159] Halve each of the samples (e.g., by a left shift);

[0160] Then, sum the values of the two halved samples.

[0161] When sampling is performed currently (e.g., at step 811), since only one sample needs to be selected from a set of samples (e.g., samples adjacent to each other), easier operations can be performed.

[0162] Therefore, techniques for reducing the number of multiplications to be performed can now be defined. Some of these techniques can be based in particular on at least one of the following principles:

[0163] Even if the block 18 to be actually predicted has dimensions MxN, the block can be reduced (in at least one of the two dimensions) and have reduced dimensions Q red xP red of the ALWIP matrix (where Q red = M′ red * N′ red , P red = N red + M red , where M red < M and / or N′ red < N, and / or M red < M and / or N red < N) can be applied. Thus, the boundary vector 17P will have dimensions P red x1, meaning there will be only P red < P multiplications (where P red = M red + N red and P = M + N).

[0164] P red x1 boundary vector 17P can be easily obtained from the original boundary 17, for example:

[0165] By downsampling (e.g., by only selecting some of the samples of the boundary); and / or

[0166] By averaging multiple samples of the boundary (which can be easily obtained by addition and shift without multiplication).

[0167] Additionally or alternatively, instead of predicting all Q = M * N values of the block 18 to be predicted by multiplication, it is possible to predict only a reduced block of reduced size (e.g., Q red = M′ red * N′ red , where M′ red < M and / or Nv red < N). The remaining samples of the block 18 to be predicted will be obtained by interpolation, for example, using Q red samples as support values for the remaining Q - Q red values to be predicted.

[0168] According to Figure 7.1 the example shown, the 4x4 block 18 to be predicted (M = 4, N = 4, Q = M * N = 16) and the neighborhoods 17 of the sample 17a (vertical column with 4 predicted samples) and the sample 17c (horizontal row with 4 predicted samples) have been predicted in the previous iteration (the neighborhoods 17a and 17c can be jointly indicated by 17). A priori, by using Figure 5 the equation shown, the prediction matrix 17M should be a QxP = 16x8 matrix (by means of Q = M * N = 4 * 4 and P = M + N = 4 + 4 = 8), and the boundary vector 17P should have a size of 8x1 (by means of P = 8). However, this would result in the need to perform 8 multiplications for each of the 16 samples of the 4x4 block 18 to be predicted, resulting in a total of 16 * 8 = 128 multiplications to be performed. (Note that the average number of multiplications per sample is a good assessment of the computational complexity. For traditional intra prediction, each sample requires four multiplications, and this increases the computational effort involved. Therefore, it is possible to use this as an upper limit for ALWIP to ensure that the complexity is reasonable and does not exceed the complexity of traditional intra prediction.)

[0169] Nevertheless, it has been understood that by using the present technique, it is possible to reduce the number of samples 17a and 17c adjacent to the block 18 to be predicted from P to P red < P at step 811. Specifically, it has been understood that it is possible to average adjacent boundary samples (17a, 17c) (e.g., in Figure 7.1at 100 locations in) to obtain a reduced boundary 102 with two horizontal rows and two vertical columns, so the blocks used as block 18 are 2x2 blocks (the reduced boundary is formed by the average). Alternatively, it is possible to perform downsampling, so two samples are selected for row 17c and two samples are selected for column 17a. Thus, the horizontal row 17c, instead of having four original samples, is processed to have two samples (e.g., averaged samples), and the vertical column 17a, which originally had four samples, is processed to have two samples (e.g., averaged samples). It can also be understood that after subdividing row 17c and column 17a into groups 110 each having two samples, one single sample is retained (e.g., the average of the samples in group 110 or a simple selection between the samples in group 110). Thus, by means of a set 102 having only four samples (M red = 2, N red = 2, P red = M red + N red = 4, where P red < P), a so-called set 102 of reduced sample values is obtained.

[0170] It can be understood that it is possible to perform operations (such as averaging or downsampling 100) without performing too many multiplications at the processor level: the averaging or downsampling 100 performed at step 811 can be obtained simply by direct and computationally power - free operations such as addition and shift.

[0171] It has been understood that at this point, it is possible (e.g., using a prediction matrix such as Figure 5 of matrix 17M) to perform a linear transformation or an affine - linear (ALWIP) transformation 19 on the set of reduced sample values 102. In this case, the ALWIP transformation 19 directly maps the four samples 102 to the sample values 104 of block 18. Interpolation is not required in the current case.

[0172] In this case, the ALWIP matrix 17M has dimensions QxP red = 16x4, which follows from the fact that all Q = 16 samples of the block 18 to be predicted are obtained directly by ALWIP multiplication (no interpolation is required).

[0173] Thus, at step 812a, a suitable ALWIP matrix 17M with dimensions QxP red is selected. This selection can be at least partially based on, for example, signaling from the data stream 12. The selected ALWIP matrix 17M can also be indicated by A k where k can be understood as an index, which can be signaled in the data stream 12 (in some cases, the matrix is also indicated as See below). The selection can be performed according to the following scheme: for each dimension (e.g., the height / width pair of block 18 to be predicted), select an ALWIP matrix 17M from one of, for example, three matrix sets S0, S1, S2 (each of the three sets S0, S1, S2 can group multiple ALWIP matrices 17M of the same dimension, and the ALWIP matrix to be selected for prediction will be one of them).

[0174] In step 812b, perform the multiplication of the selected QxP red ALWIP matrix 17M (also denoted as A k ) with the P red x1 boundary vector 17P.

[0175] In step 812c, an offset value (e.g., b k ) can be added to all the obtained values 104 of the vector 18Q obtained, for example, through ALWIP. The offset value (b k or also denoted in some cases with

[0176] See below) can be associated with a specifically selected ALWIP matrix (A k ) and can be based on an index (e.g., which can be signaled in the data stream 12).

[0177] Therefore, here is again a comparison between the present technique and not using the present technique:

[0178] Without using the present technique:

[0179] Block 18 to be predicted, which has dimensions M = 4, N = 4;

[0180] Q = M * N = 4 * 4 = 16 values to be predicted;

[0181] P = M + N = 4 + 4 = 8 boundary samples

[0182] For each of the Q = 16 values to be predicted, P = 8 multiplications, for a total of P * Q = 8 * 16 = 128 multiplications;

[0183] When using the present technique, we have:

[0184] Block 18 to be predicted, which has dimensions M = 4, N = 4;

[0185] Ultimately, Q = M * N = 4 * 4 = 16 values to be predicted;

[0186] Reduced dimension of the boundary vector: P red = M red + N red= 2 + 2 = 4;

[0187] For each of the Q = 16 values to be predicted by ALWIP, P red = 4 multiplications,

[0188] In total P red * Q = 4 * 16 = 64 multiplications (half of 128!)

[0189] The ratio between the number of multiplications and the number of final values to be obtained, and is Pred * Q / Q = 4, i.e., half of P = 8 multiplications for each sample to be predicted!

[0190] It can be understood that by relying on operations that are direct and not computationally power - demanding such as taking an average (and in that case, adding and / or shifting and / or downsampling), it is possible to obtain appropriate values in step 812.

[0191] Reference Figure 7.2 , where the block 18 to be predicted here is an 8x8 block of 64 samples (M = 8, N = 8). Here, a priori, the prediction matrix 17M should have dimensions QxP = 64x16 (Q = 64, by virtue of Q = M * N = 8 * 8 = 64, M = 8 and N = 8, and by virtue of P = M + N = 8 + 8 = 16). Thus, a priori, for each of the Q = 64 samples of the 8x8 block 18 to be predicted, P = 16 multiplications will be required, amounting to 64 * 16 = 1024 multiplications for the entire 8x8 block 18!

[0192] However, as Figure 7.2 seen, a method 820 can be provided according to which, instead of using all 16 samples of the boundary, only 8 values are used (e.g., 4 values in the horizontal boundary row 17c between the original samples of the boundary and 4 values in the vertical boundary column 17a). From the boundary row 17c, 4 samples can be used instead of 8 samples (e.g., they can be the average of every two and / or one sample selected from two samples). Thus, the boundary vector is not a Px1 = 16x1 vector, but only a P red x1 = 8x1 vector (P red = M red + N red = 4 + 4). It has been understood that it is possible to select or average (e.g., every two) the samples of the horizontal row 17c and the samples of the vertical column 17a to have only P red = 8 boundary values instead of having the original P = 16 samples, thus forming a reduced sample value set 102. This reduced set 102 will allow obtaining a reduced version of the block 18 that has Q red = M red * N red= 4 * 4 = 16 samples (instead of Q = M * N = 8 * 8 = 64). It is possible to apply the ALWIP matrix to predict a block with dimensions M red xN red = 4x4. The reduced version of block 18 includes the samples indicated in gray in Figure 7.2 scheme 106: The samples indicated by the gray squares (including sample 118' and sample 118") form a 4x4 reduced block, which has Q red = 16 values obtained in step 812. The 4x4 reduced block has been obtained by applying the linear transformation 19 in step 812. After obtaining the values of the 4x4 reduced block, it is possible to obtain the values of the remaining samples (the samples indicated by the white samples in scheme 106), for example, by interpolation.

[0193] Relative to Figure 7.1 method 810, method 820 may additionally include step 813: Derive, for example, by interpolation, the predicted values of the remaining Q - Q red = 64 - 16 = 48 samples (white squares) of the MxN = 8x8 block 18 to be predicted. The remaining Q - Q red = 64 - 16 = 48 samples can be obtained from Q red = 16 samples obtained directly by interpolation (for example, interpolation can also utilize the values of the boundary samples). As can be seen from Figure 7.2 although samples 118' and 118" have been obtained in step 812 (as shown by the gray squares), sample 108' (indicated by a square in the middle of samples 118' and 118") is obtained by interpolation between samples 118' and 118" in step 813. It has been understood that interpolation can also be obtained by operations similar to those for averaging, such as shifting and adding. Thus, in Figure 7.2 value 108' can generally be determined as the intermediate value (which can be the average value) between the value of sample 118' and the value of sample 118".

[0194] By performing interpolation, in step 813, the final version of the MxN = 8x8 block 18 can also be obtained based on the multiple sample values indicated in 104.

[0195] Therefore, the comparison between using this technique and not using it is:

[0196] Without using this technique:

[0197] Block 18 to be predicted, which has dimensions M = 8, N = 8, and

[0198] Q = M * N = 8 * 8 = 64 samples to be predicted in block 18;

[0199] In boundary 17, P = M + N = 8 + 8 = 16 samples;

[0200] For each of the Q = 64 values to be predicted, there are P = 16 multiplications, for a total of P * Q = 16 * 64 = 1028 multiplications

[0201] The ratio between the number of multiplications and the number of final values to be obtained is P * Q / Q = 16

[0202] In the case of using this technology:

[0203] The block 18 to be predicted has dimensions M = 8, N = 8;

[0204] The final Q = M * N = 8 * 8 = 64 values to be predicted;

[0205] But Q is to be used red xP red ALWIP matrix, where P red = M red + N red , Q red = M red * N red , M red = 4, N red = 4

[0206] P in the boundary red = M red + N red = 4 + 4 = 8 samples, where P red < P

[0207] For each of the Q = 16 values of the 4x4 reduced block (formed by the gray squares in scenario 106) to be predicted red = 8 multiplications for P red

[0208] For a total of P red * Q red = 8 * 16 = 128 multiplications (much less than 1024!)

[0209] The ratio between the number of multiplications and the number of final values to be obtained is P red * Q red / Q = 128 / 64 = 2 (much less than the 16 obtained in the case of not using this technology!)

[0210] Therefore, the power requirement of the technology presented here is 8 times less than that of the prior technology.

[0211] Figure 7.3Another example is shown (which may be based on method 820), where the block 18 to be predicted is a rectangular 4x8 block (M = 8, N = 4), where Q = 4 * 8 = 32 samples to be predicted. The boundary 17 is formed by a horizontal row 17c with N = 8 samples and a vertical column 17a with M = 4 samples. Thus, a priori, the boundary vector 17P will have dimensions Px1 = 12x1, and the predicted ALWIP matrix should be a QxP = 32x12 matrix, so Q * P = 32 * 12 = 384 multiplications are required.

[0212] However, for example, at least 8 samples of the horizontal row 17c can be averaged or downsampled to obtain a reduced horizontal row with only 4 samples (e.g., the averaged samples). In some examples, the vertical column 17a will remain as it is (e.g., not averaged). Overall, the reduced boundary will have dimensions P red = 8, where P red < P. Thus, the boundary vector 17P will have dimensions P red x1 = 8x1. The ALWIP prediction matrix 17M will be a matrix with dimensions M * N red * P red = 4 * 4 * 8 = 64. The 4x4 reduced block (formed by the gray columns in scheme 107) directly obtained when performing step 812 will have dimensions Q red = M * N red = 4 * 4 = 16 samples (instead of Q = 4 * 8 = 32 of the original 4x8 block 18 to be predicted). Once the reduced 4x4 block is obtained via ALWIP, it is possible to add an offset value b k (step 812c) and perform interpolation in step 813. As can be seen from Figure 7.3 step 813 in, the reduced 4x4 block is expanded to the 4x8 block 18, where the values 108' not obtained in step 812 are obtained by interpolating the values 118' and 118" (gray squares) obtained in step 812 in step 813.

[0213] Therefore, the comparison between using this technique and not using it is:

[0214] Without using this technique:

[0215] The block 18 to be predicted, with dimensions M = 4, N = 8

[0216] Q = M * N = 4 * 8 = 32 values to be predicted;

[0217] P = M + N = 4 + 8 = 12 samples in the boundary;

[0218] For each of the Q = 32 values to be predicted, there are P = 12 multiplications, for a total of P * Q = 12 * 32 = 384 multiplications

[0219] The ratio between the number of multiplications and the number of final values to be obtained is P * Q / Q = 12

[0220] In the case of using this technology:

[0221] Block 18 to be predicted, with dimensions M = 4 and N = 8 for this block

[0222] Finally, Q = M * N = 4 * 8 = 32 values to be predicted;

[0223] But Q can be used red xP red = 16x8 ALSIP matrix, where M = 4, N red = 4, Q red = M * N red = 16, P red = M + N red = 4 + 4 = 8

[0224] P in the boundary red = M + N red = 4 + 4 = 8 samples, where P red < P

[0225] For each of the Q for the reduced block to be predicted red = 16 values, Pred = 8 multiplications,

[0226] In total Q red *P red = 16 * 8 = 16 * 8 = 128 multiplications (less than 384!)

[0227] The ratio between the number of multiplications and the number of final values to be obtained is P red *Q red / Q = 128 / 32 = 4 (much less than 12 obtained in the case of not using this technology!)

[0228] Therefore, using this technology, the computational effort is reduced to one-third

[0229] Figure 7.4 Shows the case of block 18 to be predicted with dimensions MxN = 16x16 and finally having Q = M * N = 16 * 16 = 256 values to be predicted. This block has P = M + N = 16 + 16 = 32 boundary samples. This will result in a prediction matrix of size QxP = 256x32, which means 256 * 32 = 8192 multiplications!

[0230] However, by applying method 820, the number of boundary samples can be reduced (e.g., by averaging or downsampling) at step 811, e.g., from 32 to 8: For example, for each group 120 of four consecutive samples in row 17a, a single sample is retained (e.g., selected from the four samples, or the average of the samples). Additionally, for each group of four consecutive samples in column 17c, a single sample is retained (e.g., selected from the four samples, or the average of the samples).

[0231] Here, the ALWIP matrix 17M is Q red xP red = 64x8 matrix: This is because it has been chosen that P red = 8 (by using 8 averaged samples from 32 samples at the boundary or 8 samples selected from 32 samples at the boundary) and the reduced block to be predicted at step 812 is an 8x8 block (in scenario 109, the grey square is 64).

[0232] Thus, once the 64 samples of the reduced 8x8 block are obtained at step 812, it is possible to derive the remaining Q - Q red = 256 - 64 = 192 values 104 of the block 18 to be predicted at step 813.

[0233] In this case, to perform interpolation, all samples of the boundary column 17a have been chosen to be used and only alternating samples in the boundary row 17c are used. Other selections can be made.

[0234] While in the case of using this method, the ratio between the number of multiplications and the number of values finally obtained is Q red *P red / Q = 8 * 64 / 256 = 2, which is much less than 32 multiplications per value in the case of not using this technique!

[0235] The comparison between using this technique and not using it is as follows:

[0236] In the case of not using this technique:

[0237] For the block 18 to be predicted, the dimensions of the block are M = 16, N = 16

[0238] Q = M * N = 16 * 16 = 256 values to be predicted;

[0239] P = M + N = 16 * 16 = 32 samples in the boundary;

[0240] For each of the Q = 256 values to be predicted, P = 32 multiplications, for a total of P * Q = 32 * 256 = 8192 multiplications;

[0241] The ratio between the number of multiplications and the number of final values to be obtained is P*Q / Q = 32 in the case of using this technology:

[0242] The block 18 to be predicted, with the size of the block being M = 16 and N = 16

[0243] The final Q = M*N = 16*16 = 256 values to be predicted;

[0244] But the Q to be used red xP red = 64x8 ALWIP matrix, where M red = 4, N red = 4, the Q to be predicted by ALWIP red = 8*8 = 64 samples,

[0245] P red = M red +N red = 4+4 = 8

[0246] P in the boundary red = M red +N red = 4+4 = 8 samples, where P red <P

[0247] For each of the Q for the reduced block to be predicted red = 64 values, Pred = 8 multiplications,

[0248] In total Q red *P red = 64*4 = 16*8 = 256 multiplications (less than 8192!)

[0249] The ratio between the number of multiplications and the number of final values to be obtained is P red *Q red / Q = 8*64 / 256 = 2 (much smaller than the 32 obtained in the case of not using this technology!

[0250] Much smaller).

[0251] Therefore, the computing power required by this technology is 16 times less than that of traditional technologies!

[0252] Therefore, it is possible to use multiple adjacent samples (17) to predict a predetermined block (18) of an image by:

[0253] Reducing (100, 813) multiple adjacent samples (17) to obtain a set of reduced sample values (102) that is lower in the number of samples compared to the multiple adjacent samples,

[0254] Perform a linear transformation (812) or an affine linear transformation (19, 17M) on the reduced sample value set to obtain the predicted values of the predetermined samples (104, 118’, 188”) of the predetermined block (18).

[0255] Specifically, it is possible to perform reduction (100, 813) by downsampling a plurality of adjacent samples to obtain a reduced sample value set (102) with a lower number of samples compared to the plurality of adjacent samples (17).

[0256] Alternatively, it is possible to perform reduction (100, 813) by averaging a plurality of adjacent samples to obtain a reduced sample value set (102) with a lower number of samples compared to the plurality of adjacent samples (17).

[0257] In addition, it is possible to derive (813) the predicted values of other samples (108, 108’) of the predetermined block (18) by interpolating based on the predicted values of the predetermined samples (104, 118’, 118”) and a plurality of adjacent samples (17).

[0258] A plurality of adjacent samples (17a, 17c) can extend one-dimensionally along both sides of the predetermined block (18) (e.g., to the right and downwards in Figures 7.1 to 7.4 ). The predetermined samples (e.g., the predetermined samples obtained by ALWIP in step 812) can also be arranged in rows and columns, and along at least one of the rows and columns, the predetermined samples can be located at every nth position starting from the samples (112) adjacent to both sides of the predetermined block 18 of the predetermined sample 112.

[0259] Based on a plurality of adjacent samples (17), for each of at least one of the rows and columns, it is possible to determine the support value (118) of an adjacent position (118) among the plurality of adjacent positions, which is aligned with the corresponding one of at least one of the rows and columns. It is also possible to derive the predicted values 118 of other samples (108, 108’) of the predetermined block (18) by interpolation based on the predicted values of the predetermined samples (104, 118’, 118”) and the support values of the adjacent samples (118) aligned with at least one of the rows and columns.

[0260] The predetermined samples (104) can be located at every nth position along the row starting from the samples (112) adjacent to both sides of the predetermined block 18, and the predetermined samples are located at every mth position along the column starting from the samples adjacent to both sides of the predetermined block (18) of the predetermined sample (112), where n, m > 1. In some cases, n = m (e.g., in Figure 7.2 and Figure 7.3Among them, samples 104, 118', 118" directly obtained at 812 by ALWIP and indicated by gray squares are alternated with samples 108, 108' subsequently obtained in step 813 along rows and columns).

[0261] Along at least one of the row (17c) and the column (17a), it is possible to perform the determination of the support value, for example, by downsampling or averaging (122) a group (120) of adjacent samples including the adjacent sample (118) for which the corresponding support value is determined among a plurality of adjacent samples for each support value. Therefore, in Figure 7.4 In step 813, it is possible to obtain the value of sample 119 by using the value of the predetermined sample 118''' (previously obtained in step 812) and the value of the adjacent sample 118 as the support value.

[0262] The plurality of adjacent samples can extend one-dimensionally along both sides of a predetermined block (18). It is possible to perform the reduction (811) by grouping the plurality of adjacent samples (17) into one or more groups (110) of consecutive adjacent samples and performing downsampling or averaging on each group of one adjacent sample or a plurality of adjacent samples having two or more adjacent samples (110).

[0263] In the example, the linear transformation or the affine linear transformation can include P red *Q red or P red *Q weighting factors, where P red is the number of sample values (102) within the set of reduced sample values, and Q red or Q is the number of predetermined samples within the predetermined block (18). At least 1 / 4P red *Q red or 1 / 4P red *Q weighting factors are non-zero weight values. P red *Q red or P red *Q weighting factors can include a series of P red weighting factors related to each of the Q or Q red predetermined samples, where when the series is arranged one below the other in the raster scan order among the predetermined samples of the predetermined block (18), it forms an omnidirectional non-linear envelope. P red *Q or P red *Q redThe weighted factors can be independent of each other by any regular mapping rule. The average of the maximum value of the cross-correlation between the first series of weighted factors associated with the corresponding predetermined sample and the second series of weighted factors (or the reverse version of the latter series, whichever results in a higher maximum value) associated with the predetermined samples other than the corresponding predetermined sample is lower than a predetermined threshold. The predetermined threshold can be 0.3 [or 0.2 or 0.1 in some cases]. P red Adjacent samples (17) can be positioned along a one-dimensional path that extends along both sides of a predetermined block (18), and for Q or Q red For the predetermined samples, a series of P red weighted factors associated with the corresponding predetermined sample are sorted in a manner that traverses the one-dimensional path in a predetermined direction.

[0264] 6.1 Description of the method and apparatus

[0265] To predict the samples of a rectangular block having a width W (also denoted by N) and a height H (also denoted by M), affine linear weighted intra prediction (ALWIP) can take as input a row of H reconstructed adjacent boundary samples on the left side of the block and a row of W reconstructed adjacent boundary samples above the block. If the reconstructed samples are not available, they can be generated as in conventional intra prediction.

[0266] The generation of the prediction signal (e.g., the value of the complete block 18) can be based on at least some of the following three steps:

[0267] 1. Among the boundary samples 17, samples 102 (e.g., four samples in the case of W = H = 4, and / or eight samples in other cases) can be extracted by performing averaging or downsampling (e.g., step 811).

[0268] 2. The averaged samples (or the remaining samples after downsampling) can be used as input to perform matrix-vector multiplication and then perform the addition of an offset. The result can be a reduced prediction signal on the subsampled set of samples in the original block (e.g., step 812).

[0269] 3. The prediction signal at the remaining positions can be generated, for example, by upsampling according to the prediction signal on the subsampled set, e.g., by linear interpolation (e.g., step 813).

[0270] Due to step 1. (811) and / or step 3. (813), the total number of multiplications required to calculate the matrix quantity product can be such that the total number is always less than or equal to 4 * W * H. In addition, the averaging operation on the boundary and the linear interpolation of the reduced prediction signal are performed only by using addition and shift. In other words, in the example, each sample in the ALWIP mode requires at most four multiplications.

[0271] In some examples, the matrices (e.g., 17M) and offset vectors (e.g., b k ) required to generate the prediction signal can be taken from a set of matrices (e.g., three sets) that can be stored in the storage units of the decoder and encoder, such as S0, S1, S2.

[0272] In some examples, set S0 may include (e.g., contain) n0 (e.g., n0 = 16 or n0 = 18 or another number) matrices Each of these matrices can have 16 rows and 4 columns and 18 offset vectors of size 16 each to perform the techniques according to Figure 7.1 . The matrices and offset vectors of this set are used for a 4×4 block 18. Once the boundary vectors have been reduced to P red = 4 vectors (as in Figure 7.1 step 811), it is possible to directly map the P red = 4 samples of the reduced sample set 102 to the Q = 16 samples of the 4x4 block 18 to be predicted.

[0273] In some examples, set S1 may include (e.g., contain) n1 (e.g., n1 = 8 or n1 = 18 or another number) matrices Each of these matrices can have 16 rows and 8 columns and 18 offset vectors of size 16 each to perform the techniques according to Figure 7.2 or 7.3. The matrices and offset vectors of set S1 can be used for blocks of sizes 4×8, 4x16, 4x32, 4x64, 16x4, 32x4, 64x4, 8×4, and 8×8. Additionally, it can be used for blocks of size WxH, where max(W,H) > 4 and min(W,H) = 4, i.e., for blocks of size 4x16 or 16x4, 4x32 or 32x4, and 4x64 or 64x4. The 16x8 matrix refers to a reduced version of block 18, which is a 4x4 block, as obtained in Figure 7.2 and Figure 7.3 .

[0274] Additionally or alternatively, set S2 may include (e.g., contain) n2 (e.g., n2 = 6 or n2 = 18 or another number) matrices Each of these matrices can have 64 rows and 8 columns and 18 offset vectors of size 64 The 64x8 matrix refers to a reduced version, which is an 8x8 block, e.g., as in Figure 7.4Obtained as described above. The matrices and offset vectors of this set can be used for blocks of sizes 8×16, 8×32, 8×64, 16×8, 16×16, 16×32, 16×64, 32×8, 32×16, 32×32, 32×64, 64×8, 64×16, 64×32, 64×64.

[0275] The matrices and offset vectors of this set or a part of these matrices and offset vectors can be used for all other block shapes.

[0276] 6.2 Averaging or downsampling of boundaries

[0277] Here, features regarding step 811 are provided.

[0278] As described above, boundary samples (17a, 17c) can be averaged and / or downsampled (e.g., from P samples to P red <P samples).

[0279] In the first step, the input boundaries bdry top (e.g., 17c) and bdry left (e.g., 17a) can be reduced to smaller boundaries and to obtain the reduced set 102. Here, and both consist of 2 samples in the case of 4x4 blocks and 4 samples in other cases.

[0280] In the case of 4x4 blocks, it is possible to define

[0281]

[0282] and similarly define Therefore, is the average value obtained, for example, using shift operations.

[0283] In all other cases (e.g., for blocks with a width or height not equal to 4), if the block width W is given as W = 4 * 2 k , then for 0 ≤ i < 4, it is defined as

[0284]

[0285] and similarly defined

[0286] In still other cases, it is possible to downsample the boundaries (e.g., by selecting a specific boundary sample from a set of boundary samples) to obtain a reduced number of samples. For example, can be selected from bdrytop [0] and bdry top [1], and can be selected from bdry top [2] and bdry top [3]. Similarly, two reduced boundaries

[0287] can be defined and can be cascaded to the reduced boundary vector bdry red (associated with the reduced set 102), also indicated by 17P. The reduced boundary vector bdry red for a block of shape 4×4 ( Figure 7.1 example) can thus be of size 4 (P red = 4), while for blocks of all other shapes ( Figures 7.2 to 7.4 example) can be of size 8 (P red = 8).

[0288] Here, if mode < 18 (or the number of matrices in the matrix set), then

[0289]

[0290] if mode ≥ 18, which corresponds to the transposed mode of mode - 17, then

[0291]

[0292] Thus, depending on a specific state (one state: mode < 18; another state: mode ≥ 18), the predicted values of the output vector can be assigned along different scan orders (e.g., one scan order: another scan order: ).

[0293] Other strategies can be executed. In other examples, the mode index'mode' is not necessarily in the range of 0 to 35 (other ranges can be defined). Additionally, each of the three sets S0, S1, S2 does not have to have 18 matrices (thus, instead of an expression like mode ≥ 18, one can use mode ≥ n0, n 1、 n2, which are the number of matrices in each set of matrices S0, S1, S2 respectively). Additionally, each set can have a different number of matrices (e.g., it can be: S0 has 16 matrices, S1 has 8 matrices, and S2 has 6 matrices).

[0294] The mode and transpose information need not be stored and / or transmitted as a combined mode index 'pattern': in some examples, it is possible to explicitly signal as a transpose flag and matrix index (0 - 15 for S0, 0 - 7 for S1, and 0 - 5 for S2).

[0295] In some cases, the combination of transpose flag and matrix index can be interpreted as a set index. For example, there can be one bit serving as the transpose flag and some bits indicating the matrix index, collectively referred to as the "set index".

[0296] 6.3 Generating a reduced prediction signal through matrix - vector multiplication

[0297] Here, features regarding step 812 are provided.

[0298] In the reduced input vector bdry red (boundary vector 17P), a reduced prediction signal pred red can be generated. The latter signal can be a signal on a down - sampled block with width W red and height H red Here, W red and H red can be defined as:

[0299] W red = 4, H red = 4; if max(W, H) ≤ 8,

[0300] Otherwise, W red = min(W, 8), H red = min(H, 8).

[0301] The reduced prediction signal pred can be calculated by computing a matrix - vector product and adding an offset red :

[0302] pred red = A · bdry red + b.

[0303] Here, A is a matrix (e.g., prediction matrix 17M), which may have W red * H red rows and 4 columns if W = H = 4, and 8 columns in all other cases, and b is a vector of size W red * H red .

[0304] If W = H = 4, A can have 4 columns and 16 rows, and thus, in this case, each sample may require 4 multiplications to compute pred red。In all other cases, A may have 8 columns, and it can be verified that in these cases it has 8*W red *H red ≤4*W*H, that is, also in these cases, each sample requires at most 4 multiplications to calculate pred red 。

[0305] The matrix A and the vector b can be taken from one of the following sets S0, S1, S2. The index idx = idx(W,H) can be defined as follows: If W = H = 4, set idx(W,H) = 0, if max(W,H) = 8, set idx(W,H) = 1, and in all other cases set idx(W,H) = 2. Additionally, if mode < 18, m can be made equal to mode, otherwise m = mode - 17. Then, if idx ≤ 1 or idx = 2 and min(W,H) > 4, then it can be made such that and In the case where idx = 2 and min(W,H) = 4, make A the matrix produced by omitting each row of, which in the case of W = 4 corresponds to the odd x - coordinates in the downsampling block, or in the case of H = 4 corresponds to the odd y - coordinates in the downsampling block. If mode ≥ 18, replace the reduced prediction signal with its transposed signal. In an alternative example, a different strategy can be executed. For example, instead of reducing the size of the larger matrix ("omitting"), a smaller matrix S1 (idx = 1) with W red = 4 and H red = 4 can be used. That is, such a block is now assigned to S1 instead of S2.

[0306] Other strategies can be executed. In other examples, the mode index'mode' is not necessarily in the range from 0 to 35 (other ranges can be defined). Additionally, each of the three sets S0, S1, S2 does not have to have 18 matrices (thus, instead of an expression like mode < 18, it is possible to use mode < n 0、 n 1、 n2, which are the number of matrices in each set of matrices S0, S1, S2 respectively). Additionally, each set can have a different number of matrices (for example, it can be: S0 has 16 matrices, S1 has 8 matrices, and S2 has 6 matrices).

[0307] 6.4 Linear interpolation produces the final prediction signal

[0308] Here, the features regarding step 812 are provided.

[0309] Interpolation of the subsampled prediction signal may require a second version of the border to be averaged on large blocks. That is, if min(W,H)>8 and W≥H, then write W = 8*2 l , and for 0 ≤ i < 8, define

[0310]

[0311] If min(W,H)>8 and H>W, then define similarly

[0312] Additionally or alternatively, it is possible to have "hard downsampling", where equals

[0313]

[0314] Furthermore, it can be defined similarly

[0315] At the sample positions omitted when generating pred red , the final prediction signal can be generated by linear interpolation from pred red (e.g., step 813 in the example of Figures 7.2 to 7.4 ). In some examples, if W = H = 4 (e.g., the example of Figure 7.1 ), this linear interpolation may be unnecessary.

[0316] The linear interpolation can be given as follows (nevertheless, other examples are possible). Assume W≥H. Then, if H>H red , then vertical upsampling of pred red can be performed. In this case, pred red can be extended one row to the top as follows. If W = 8, pred red can have width W red = 4 and can be extended to the top by the border signal averaged , e.g., as defined above. If W>8, then pred red has width W red = 8 and it is extended to the top by the border signal averaged , e.g., as defined above. For the first row of pred red , it can be written as pred red [x][-1]. Then, the signal red on the block with width W red and height 2*H can be given as follows

[0317]

[0318] where 0 ≤ x < W red and 0 ≤ y < H red , the latter process can be carried out k times until 2 k *H red = H. Therefore, if H = 8 or H = 16, it can be carried out at most once. If H = 32, it can be carried out twice. If H = 64, it can be carried out three times. Next, a horizontal upsampling operation can be applied to the result of the vertical upsampling. The subsequent upsampling operations can use all the boundaries on the left side of the prediction signal. Finally, if H > W, a similar process can be carried out by first upsampling in the horizontal direction (if necessary) and then in the vertical direction.

[0319] This is an example of interpolation using reduced boundary samples for the first interpolation (horizontally or vertically) and original boundary samples for the second interpolation (vertically or horizontally). Depending on the block size, only the second interpolation may be required or no interpolation may be needed. If both horizontal and vertical interpolations are required, the order depends on the width and height of the block.

[0320] However, different techniques can be implemented: for example, the original boundary samples can be used for both the first and second interpolations, and the order can be fixed, such as first horizontally and then vertically (in other cases, first vertically and then horizontally).

[0321] Therefore, the interpolation order (horizontal / vertical) and the use of reduced boundary samples / original boundary samples can be changed.

[0322] 6.5 Description of an example of the entire ALWIP process

[0323] For Figures 7.1 to 7.4 the different shapes in, the entire processes of averaging, matrix-vector multiplication, and linear interpolation are shown. Note that the remaining shapes will be treated as one of the depicted cases.

[0324] 1. Given a 4×4 block, ALWIP can take two averages along each axis of the boundary by using Figure 7.1 the techniques of. The four resulting input samples enter the matrix-vector multiplication. The matrix is taken from the set S0. After adding the offset, this can produce 16 final prediction samples. No linear interpolation is required to generate the prediction signal. Therefore, each sample performs a total of (4*16) / (4*4) = 4 multiplications. See, for example Figure 7.1 .

[0325] 2. Given an 8×8 block, ALWIP can take four averages along each axis of the boundary. By using Figure 7.2The eight input samples generated by the technology enter the matrix-vector multiplication. The matrix is taken from the set S1. This produces 16 samples at the odd positions of the prediction block. Therefore, each sample performs a total of (8 * 16) / (8 * 8) = 2 multiplications. After adding the offset, these samples can be interpolated, for example, vertically interpolated by using the top boundary, and, for example, horizontally interpolated by using the left boundary. See, for example Figure 7.2 .

[0326] 3. Given an 8×4 block, ALWIP can take four averages along the horizontal axis of the boundary by using Figure 7.3 the technology and take four original boundary values on the left boundary. The eight input samples generated enter the matrix-vector multiplication. The matrix is taken from the set S1. This produces 16 samples at the odd horizontal positions and each vertical position of the prediction block. Therefore, each sample performs a total of (8 * 16) / (8 * 4) = 4 multiplications. For example, after adding the offset, these samples will be horizontally interpolated by using the left boundary. See, for example Figure 7.3 .

[0327] The transposed case is handled accordingly.

[0328] 4. Given a 16×16 block, ALWIP can take four averages along each axis of the boundary. By using Figure 7.2 the technology, the eight input samples generated enter the matrix-vector multiplication. The matrix is taken from the set S2. This produces 64 samples at the odd positions of the prediction block. Therefore, each sample performs a total of (8 * 64) / (16 * 16) = 2 multiplications. After adding the offset, these samples are vertically interpolated by using the top boundary and horizontally interpolated by using the left boundary. See, for example Figure 7.2 . See, for example Figure 7.4 .

[0329] For larger shapes, the process may be basically the same, and it is easy to check whether the number of multiplications per sample is less than two.

[0330] For W×8 blocks, since the samples are given at the odd horizontal positions and each vertical position, only horizontal interpolation is required. Therefore, in these cases, each sample performs a maximum of (8 * 64) / (16 * 8) = 4 multiplications.

[0331] Finally, for W×4 blocks where W > 8, let A k be the matrix generated by omitting each row corresponding to the odd entries along the horizontal axis of the downsampled block. Therefore, the output size can be 32, and again, only horizontal interpolation remains to be performed. Each sample can perform a maximum of (8 * 32) / (16 * 4) = 4 multiplications.

[0332] The transposed case can be processed accordingly.

[0333] 6.6 Number of Parameters Required and Complexity Evaluation

[0334] The parameters required for all possible proposed intra prediction modes can be included by matrices and offset vectors belonging to sets S0, S1, S2. All matrix coefficients and offset vectors can be stored as 10-bit values. Thus, according to the above description, the proposed method may require a total of 14,400 parameters, each with a precision of 10 bits. This corresponds to 0.018 megabytes of memory. It should be noted that currently, a CTU with a size of 128×128 in the standard 4:2:0 chroma subsampling consists of 24,576 values, each value being 10 bits. Therefore, the memory requirement of the proposed intra prediction tool does not exceed that of the current picture reference tool adopted in the previous meeting. In addition, it should be noted that due to the PDPC tool or the 4-tap interpolation filter for the angular prediction mode with fractional angular positions, the traditional intra prediction mode requires four multiplications per sample. Therefore, in terms of operation complexity, the proposed method does not exceed the traditional intra prediction mode.

[0335] 6.7 Signaling of the Proposed Intra Prediction Modes

[0336] For example, for a luminance block, 35 ALWIP modes are proposed (other numbers of modes can be used). For each coding unit (CU) of the intra mode, a flag indicating whether the ALWIP mode is applied to the corresponding prediction unit (PU) is sent in the bitstream. The signaling of the latter index can be coordinated with the MRL in the same way as in the first CE test. If the ALWIP mode is to be applied, an MPM list with 3 MPMS can be used to signal the index predmode of the ALWIP mode.

[0337] Here, the intra modes of the above and left PUs can be used as follows to perform the derivation of the MPM. There can be tables, such as three fixed tables map_angular_to_alwip idx , idx ∈ {0, 1, 2}, which can assign the ALWIP mode to each traditional intra prediction mode predmode Angular

[0338] predmode ALWIP = map_angular_to_alwip idx [predmode Angular .

[0339] For each PU with width W and height H, it is defined and indexed

[0340] idx(PU) = idx(W,H) ∈ {0,1,2}

[0341] which indicates from which of the three sets the ALWIP parameters are to be obtained, as described in Section 4 above. If the above prediction unit PU above is available, belongs to the same CTU as the current PU and is in the intra mode, if idx(PU) = idx(PU above ) and if ALWIP is applied to the PU above above, in the ALWIP mode then cause

[0342]

[0343] If the above PU is available, belongs to the same CTU as the current PU and is in the intra mode, and if the traditional intra prediction mode is applied to the above PU, then cause

[0344]

[0345] In all other cases, cause

[0346]

[0347] which means that the mode is not available. In the same way but without restricting that the left PU needs to belong to the same CTU as the current PU, the mode

[0348]

[0349] Finally, three fixed default lists list idx are provided, idx ∈ {0,1,2}, each of which contains three different ALWIP modes. In the default list list idx(PU) and the modes and , three different MPMs are constructed by replacing -1 with the default value and eliminating duplicates.

[0350] 6.8 Derivation of the adjusted MPM list for traditional luminance and chrominance intra prediction modes

[0351] The proposed ALWIP mode can be coordinated with the MPM-based coding of traditional intra prediction modes as follows. The luminance and chrominance MPM list derivation process of traditional intra prediction modes can use the fixed table map_lwip_to_angular idx , idx ∈ {0,1,2}, to map the ALWIP mode predmode LWIP on a given PU to one of the traditional intra prediction modes

[0352] predmode Angular = map_lwip_to_angular idx(PU) [predmode LWIP 。

[0353] For luminance MPM list export, whenever an adjacent luminance block using the ALWIP mode predmode LWIP is encountered, the block can be processed as if it were using the conventional intra prediction mode predmode Angular . For chrominance MPM list export, whenever the current luminance block uses the LWIP mode, the ALWIP mode can be converted to the conventional intra prediction mode using the same mapping.

[0354] 7 Efficient embodiments

[0355] Let us briefly summarize the above examples as they may form the basis for other extended embodiments described below.

[0356] To predict the predetermined block 18 of Picture 10, the use of multiple adjacent samples 17a, 17c is employed.

[0357] Reduction 100 has been performed by averaging multiple adjacent samples to obtain a set of reduced sample values 102 that are lower in the number of samples compared to the multiple adjacent samples. This reduction is optional in the embodiments herein and results in a so-called sample value vector described below. The set of reduced sample values is linearly transformed or affinely linearly transformed 19 to obtain the predicted value of the predetermined sample 104 of the predetermined block. This transformation is later indicated using matrix A and offset vector b, which have been obtained by machine learning (ML) and should be efficiently implemented.

[0358] By interpolation, the predicted value of other samples 108 of the predetermined block is derived based on the predicted values of the predetermined sample and multiple adjacent samples. It should be noted that, in theory, the result of the affine / linear transformation can be associated with the non-full pixel sample positions of block 18 such that all samples of block 18 can be obtained by interpolation according to alternative embodiments. It may also be that interpolation is not needed at all.

[0359] Multiple adjacent samples can extend one-dimensionally along both sides of a predetermined block. The predetermined samples are arranged in rows and columns and along at least one of the rows and columns. The predetermined samples can be located at every nth position starting from the samples (112) adjacent to both sides of the predetermined block among the predetermined samples. Based on the multiple adjacent samples, for each of at least one of the rows and columns, a support value of one adjacent position (118) among the multiple adjacent positions can be determined. The adjacent position (118) is aligned with the corresponding one of at least one of the rows and columns. And through interpolation, the predicted values of other samples 108 of the predetermined block can be derived based on the predicted values of the predetermined samples and the support values of the adjacent samples aligned with at least one of the rows and columns. The predetermined samples can be located at every nth position along the row starting from the samples 112 adjacent to both sides of the predetermined block 18 among the predetermined samples, and the predetermined samples can be located at every mth position along the column starting from the samples 112 adjacent to both sides of the predetermined block among the predetermined samples, where n, m > 1. It is possible that n = m. Along at least one of the rows and columns, the determination of the support value can be done by downsampling or averaging (122) a group 120 of adjacent samples among the multiple adjacent samples that includes the adjacent sample 118 for which each support value is determined. The multiple adjacent samples can extend one-dimensionally along both sides of the predetermined block, and the reduction can be performed by grouping the multiple adjacent samples into one or more groups 110 of consecutive adjacent samples and performing averaging on each of the one or more groups of adjacent samples that have more than two adjacent samples.

[0360] For the predetermined block, the prediction residual can be transmitted in the data stream. The prediction residual can be derived from the data stream at the decoder and used, along with the predicted values of the predetermined samples, to reconstruct the predetermined block. At the encoder, the prediction residual is encoded into the data stream at the encoder.

[0361] The picture can be subdivided into multiple blocks of different block sizes, and the multiple blocks include the predetermined block. Then, a linear transformation or an affine linear transformation of the block 18 can be selected depending on the width W and height H of the block, such that the linear transformation or affine linear transformation selected for the predetermined block is selected from: a first set of linear transformations or affine linear transformations, provided that the width W and height H of the predetermined block are within a first set of width / height pairs; and a second set of linear transformations or affine linear transformations, provided that the width W and height H of the predetermined block are within a second set of width / height pairs that do not intersect with the first set of width / height pairs. Also, as will become clear later, the affine / linear transformation is represented by other parameters, namely the weights C and optionally the offset and scale parameters.

[0362] The decoder and encoder can be configured to subdivide a picture into a plurality of blocks of different block sizes, the blocks including a predetermined block, and to select a linear transformation or an affine linear transformation depending on the width W and height H of the predetermined block, such that the linear transformation or affine linear transformation selected for the predetermined block is selected from:

[0363] a first set of linear transformations or affine linear transformations, provided that the width W and height H of the predetermined block are within a first set of width / height pairs,

[0364] a second set of linear transformations or affine linear transformations, provided that the width W and height H of the predetermined block are within a second set of width / height pairs that is disjoint from the first set of width / height pairs, and

[0365] a third set of linear transformations or affine linear transformations, provided that the width W and height H of the predetermined block are within a third set of one or more width / height pairs that is disjoint from the first set of width / height pairs and the second set of width / height pairs.

[0366] The third set of one or more width / height pairs includes only one width / height pair W’, H’, and each linear transformation or affine linear transformation in the first set of linear transformations or affine linear transformations is used to transform N’ sample values into W’*H’ predicted values of an array of sample positions of W’xH’.

[0367] Each of the first set and the second set of width / height pairs may include a first width / height pair W p , H p , where W p is not equal to H p , and a second width / height pair W q , H q , where H q =W p and W q =H p .

[0368] Each of the first set and the second set of width / height pairs may additionally include a third width / height pair W p , H p , where W p is equal to H p and H p >H q .

[0369] For a predetermined block, a set index may be transmitted in the data stream, which index indicates which linear transformation or affine linear transformation is to be selected for block 18 from a predetermined set of linear transformations or affine linear transformations.

[0370] Multiple adjacent samples can extend one-dimensionally along both sides of a predetermined block, and reduction can be performed in the following manner: for a first subset of multiple adjacent samples adjacent to a first side of the predetermined block, group the first subset into a first group 110 of one or more consecutive adjacent samples; and for a second subset of multiple adjacent samples adjacent to a second side of the predetermined block, group the second subset into a second group 110 of one or more consecutive adjacent samples; and average each group in the first group and the second group of one or more adjacent samples that has more than two adjacent samples to obtain a first sample value from the first group and a second sample value from the second group. Then, a linear or affine linear transformation can be selected depending on a set index in a predetermined set of linear or affine linear transformations, such that two different states of the set index result in the selection of one of the linear transformation or the affine linear transformation in the predetermined set of linear or affine linear transformations. In the case where the set index adopts the first state of the two different states, a predetermined linear transformation or affine linear transformation can be performed on the reduced set of sample values to generate an output vector of predicted values in the form of a first vector, and distribute the predicted values of the output vector to predetermined samples of the predetermined block along a first scan order. And in the case where the set index adopts the second state of the two different states, in the form of a second vector, the first vector is different from the second vector, such that a component filled with one of the first sample values in the first vector is filled with one of the second sample values in the second vector, and a component filled with one of the second sample values in the first vector is filled with one of the first sample values in the second vector, so as to generate an output vector of predicted values and distribute the predicted values of the output vector to predetermined samples of the predetermined block transposed relative to the first scan order along a second scan order.

[0371] Each linear transformation or affine linear transformation within a first set of linear or affine linear transformations can be used to transform N1 sample values into w1*h1 predicted values for an array of w1xh1 sample positions, and each linear transformation or affine linear transformation within a second set of linear or affine linear transformations is used for w at sample positions 2xThe h2 array transforms N2 sample values into w2*h2 prediction values, where for a first predetermined width / height pair in a first set of width / height pairs, w1 can exceed the width of the first predetermined width / height pair, or h1 can exceed the height of the first predetermined width / height pair, and for a second predetermined width in the first set of width / height pairs, not only can w1 not exceed the width of the second predetermined width / height pair, but h1 also cannot exceed the height of the second predetermined width / height pair. Then reduction (100) can be performed by averaging a plurality of adjacent samples to obtain a set of reduced sample values (102) such that if a predetermined block has a first predetermined width / height pair and if a predetermined block has a second predetermined width / height pair, the set of reduced sample values 102 has N1 sample values, and if a predetermined block has a first predetermined width / height pair, a selected linear transform or an affine linear transform on the set of reduced sample values can be performed by using only a first sub-part of the selected linear transform or affine linear transform, which is related to sub-sampling of the w1xh1 array of sample positions along the width dimension if w1 exceeds the width of a width / height pair, or which is related to sub-sampling of the w1xh1 array of sample positions along the height dimension if h1 exceeds the height of a width / height pair; and if a predetermined block has a second predetermined width / height pair, a selected linear transform or an affine linear transform on the set of reduced sample values can be performed by using the selected linear transform or affine linear transform completely.

[0372] Each linear transform or affine linear transform within a first set of linear transforms or affine linear transforms can be used to transform N1 sample values into w1*h1 prediction values for a w1xh1 array of sample positions, where w1 = h1, and each linear transform or affine linear transform within the first set of linear transforms or affine linear transforms is used for a w 2x The h2 array transforms N2 sample values into w2*h2 prediction values, where w2 = h2.

[0373] All of the above embodiments are illustrative only, as they can form the basis of the embodiments described below. That is, the above concepts and details are applied to understand the following embodiments and serve as a repository for possible extensions and modifications of the embodiments described below. Specifically, many of the above details are optional, such as the averaging of adjacent samples, the fact that adjacent samples are used as reference samples, etc.

[0374] More generally, the embodiments described herein assume that a prediction signal on a rectangular block is generated from already reconstructed samples. For example, an intra-frame prediction signal on a rectangular block is generated from adjacent, already reconstructed samples to the left and above the block. The generation of the prediction signal is based on the following steps.

[0375] 1. In a reference sample, now referred to as a boundary sample (but not excluding the possibility that this description transfers to reference samples located elsewhere), samples can be extracted by taking an average. Here, the average is taken over the boundary samples both to the left and above the block or over the boundary samples on only one of the two sides. If the average is not taken over one side, the samples on that side remain unchanged.

[0376] 2. Perform a matrix-vector multiplication, optionally followed by adding an offset, where, if the average is applied only on the left side, the input vector for the matrix-vector multiplication is the concatenation of the averaged boundary samples on the left side of the block and the original boundary samples above the block; or if the average is applied only on the upper side, the input vector for the matrix-vector multiplication is the concatenation of the original boundary samples on the left side of the block and the averaged boundary samples above the block; or if the average is applied on both sides of the block, the input vector for the matrix-vector multiplication is the concatenation of the averaged boundary samples on the left side of the block and the averaged boundary samples above the block. Similarly, there will be alternatives, such as an alternative of not using averaging at all.

[0377] 3. The result of the matrix-vector multiplication and the optional offset addition can optionally be a reduced prediction signal on a subsampled set of samples in the original block. The prediction signals at the remaining positions can be generated from the prediction signals on the subsampled set by linear interpolation.

[0378] The calculation of the matrix-vector product in step 2 is preferably carried out in integer arithmetic. Thus, if x = (x1,…,x n ) represents the input to the matrix-vector product, i.e., x represents the concatenation of the (averaged) boundary samples on the left side and above the block, then in x, the (reduced) prediction signal calculated in step 2 should be calculated using only shifts, addition of an offset vector, and multiplication of integers. Ideally, the prediction signal in step 2 will be given as Ax + b, where b is an offset vector that may be zero, and where A is derived by some machine learning-based training algorithm. However, such a training algorithm typically only produces the matrix A = A float . Thus, there is the problem of specifying integer operations in the foregoing sense such that the expression A float x is well approximated using these integer operations. Here, it is important to mention that it is not necessary to choose these integer operations such that they approximate the expression A float x under the assumption that the vector x is uniformly distributed, but typically the expression A float x is considered where the input vector x for which the approximation is to be made is the (averaged) boundary samples from a natural video signal, where some correlations between the components x i of x can be expected.

[0379] Figure 8An improved ALWIP prediction is shown. Samples of a predetermined block can be predicted based on a first matrix-vector product between a matrix A1100 derived by a machine learning-based training algorithm and a sample value vector 400. Optionally, an offset b1110 can be added. To achieve an integer approximation or a fixed-point approximation of the first matrix-vector product, the sample value vector can be subjected to a reversible linear transformation 403 to determine another vector 402. A second matrix-vector product between another matrix B1200 and another vector 402 can be equal to the result of the first matrix-vector product.

[0380] Due to the characteristics of another vector 402, the second matrix-vector product can be an integer approximated by a matrix-vector product 404 between a predetermined prediction matrix C405 and another vector 402 plus another offset 408. Another vector 402 and another offset 408 can consist of integers or fixed-point values. All components of another offset are the same, for example. The predetermined prediction matrix 405 can be a quantized matrix or a matrix to be quantized. The result of the matrix-vector product 404 between the predetermined prediction matrix 405 and another vector 402 can be understood as a prediction vector 406.

[0381] More details about this integer approximation are provided below.

[0382] Possible solution according to Example I: Subtracting and adding the mean

[0383] Expression A available for the above situation float A possible combination for the integer approximation of x is to replace the i0 component of x (i.e., the sample value vector 400) with the mean of the components of x, mean(x) (i.e., the predetermined value 1400) (i.e., the predetermined component 1500), and subtract this mean from all other components. In other words, as Figure 9a shown, the reversible linear transformation 403 is defined such that: the predetermined component 1500 of another vector 402 becomes a, and each of the other components of another vector 402 except the predetermined component 1500 is equal to the corresponding component of the sample value vector minus a, where a is the predetermined value 1400, which is, for example, the average of the components of the sample value vector 400, such as the arithmetic mean or the weighted average. This operation on the input is given by the reversible transformation T403, which has an obvious integer implementation, especially if the dimension n of x is a power of 2.

[0384] Since A float =(A float T -1 )T, if such a transformation is performed on the input x, an integer approximation of the matrix-vector product By must be found, where B=(A float T -1) and y = Tx. Since the matrix-vector product A float x represents a prediction of a rectangular block (i.e., a predetermined block), and since x is included by the (e.g., averaged) boundary samples of the block, it should be expected that in the case where all sample values of x are equal, i.e., for all i, x i = mean(x), each sample value in the prediction signal A float x should be close to or exactly equal to mean(x). This means that it should be expected that the i0-th column of B (i.e., the column corresponding to the predetermined component) is very close to or equal to a column consisting only of 1s. Thus, if M(i0), i.e., the integer matrix 1300, is a matrix whose i0-th row consists of 1s and all its other rows are zeros, written as By = Cy + M(i0)y, where C = B - M(i0), it should be expected that the i0-th row of C (i.e., the predetermined prediction matrix 405) has rather small entries or is zero, as Figure 9b shown. Furthermore, since the components of x are correlated, it can be expected that for each i ≠ i0, the i-th component y i = x i - mean(x) generally has an absolute value much smaller than the i-th component of x. Since the matrix M(i0) is an integer matrix, if an integer approximation of Cy is given, an integer approximation of By is achieved, and by the above discussion, corresponding to A float x, it can be expected that the quantization error resulting from quantizing each entry of C in a suitable manner should only slightly affect the error in the quantization result of the obtained By.

[0385] In another possible combination of the integer approximation of the expression A float x, the i0-th component of x remains unchanged, and the same value is subtracted from all other components i.e., for each i ≠ i0, and

[0386] In other words, the predetermined value 1400 can be the component in the sample value vector 400 corresponding to the predetermined component 1500.

[0387] Alternatively, the predetermined value 1400 is a default value, or a value signaled in the data stream into which the picture is encoded.

[0388] According to an embodiment, a device is configured to include a plurality of reversible linear transforms 403, each reversible linear transform being associated with a component of another vector 402. Further, the device is configured to, for example, select a predetermined component 1500 from the components of the sample value vector 400, and use the reversible linear transform 403 associated with the predetermined component 1500 among the plurality of reversible linear transforms as the predetermined reversible linear transform. This is because, for example, the different positions of the i0-th row (i.e., the row of the reversible linear transform 403 corresponding to the predetermined component) depend on the position of the predetermined component in that other vector. If, for example, the first component (i.e., y1) of the other vector 402 is the predetermined component, the i0-th row will replace the first row of the reversible linear transform.

[0389] As Figure 9b shown, the matrix components 414 of the predetermined prediction matrix C 405 within the column 412 (i.e., the i0-th column) of the predetermined prediction matrix 405 are all zero, for example, and the matrix component 414 corresponds to the predetermined component 1500 of the other vector 402. In this case, the device is configured to, for example: calculate the matrix-vector product 404 by performing a multiplication, which is performed by calculating the matrix-vector product 407 between the reduced prediction matrix C'405 generated by removing the column 412 from the predetermined prediction matrix C 405 and another vector 410 generated by removing the predetermined component 1500 from the other vector 402, as Figure 9c shown. Therefore, the prediction vector 406 can be calculated with fewer multiplications.

[0390] As Figure 8 、 Figure 9b and Figure 9c shown, a device can be configured to: when predicting the samples of a predetermined block based on the prediction vector 406, for each component of the prediction vector 406, calculate the sum of the corresponding component and a (i.e., the predetermined value 1400). The sum can be represented by the sum of the prediction vector 406 and the vector 409, where all components of the vector 409 are equal to the predetermined value 1400, as Figure 8 and Figure 9c shown. Alternatively, the sum can be represented by the sum of the prediction vector 406 and the matrix-vector product 1310 between the integer matrix M 1300 and another vector 402, as Figure 9b shown, where the matrix components of the integer matrix 1300 are 1 in a column (i.e., the i0-th column) corresponding to the predetermined component 1500 of the other vector 402, and all other components are zero, for example.

[0391] The result of the sum of the predetermined prediction matrix 405 and the integer matrix 1300 is equal to or approximately equal to, for example, Figure 8 another matrix 1200 shown in

[0392] In other words, another matrix B 1200 generated by summing each matrix component of the predetermined prediction matrix C 405 within the column 412 (i.e., the i0-th column) of the predetermined prediction matrix 405 corresponding to the predetermined component 1500 of another vector 402 with 1 (i.e., matrix B multiplied by the invertible linear transformation 403) corresponds to a quantized version of, for example, the machine learning prediction matrix A 1100, as Figure 8 , Figure 9a and Figure 9b shown. The summation of each matrix component of the predetermined prediction matrix C 405 within the i0-th column 412 with 1 may correspond to the summation of the predetermined prediction matrix 405 and the integer matrix 1300, as Figure 9b shown. As Figure 8 shown, the machine learning prediction matrix A 1100 may be equal to the result of multiplying another matrix 1200 by the invertible linear transformation 403. This is because A·x = BT·yT -1 . The predetermined prediction matrix 405 is, for example, a quantization matrix, an integer matrix, and / or a fixed-point matrix, whereby a quantized version of the machine learning prediction matrix A 1100 can be achieved.

[0393] Matrix multiplication using only integer arithmetic

[0394] For low-complexity implementations (in terms of the complexity of adding and multiplying scalar values, as well as in terms of the storage required for the entries of the matrices involved), it is desirable to perform the matrix multiplication 404 using only integer arithmetic.

[0395] To compute an approximation of z = Cy, i.e.,

[0396]

[0397] According to an embodiment, using only integer arithmetic, the real-valued C i,j must be mapped to integer values This can be done, for example, by uniform scalar quantization or by considering specific correlations between the values y i The integer values represent, for example, fixed-point numbers, each of which can be stored with a fixed number of bits n_bits, e.g., n_bits = 8.

[0398] The matrix-vector product 404 with a matrix of size m×n (i.e., the predetermined prediction matrix 405) can be performed as shown in this pseudocode, where <<, >> are arithmetic binary left shift and right shift operations, and +, -, and * operate only on integer values. (1)

[0400]

[0401] Here, the array C, i.e., the predetermined prediction matrix 405, stores fixed-point numbers as integers, for example. The final addition of final_offset and the right shift operation of right_shift_result reduce the precision by rounding to obtain the required fixed-point format for the output.

[0402] To allow for an increased range of real values representable by the integers in C, two additional matrices offset i,j and scale i,j are used, as shown in the embodiments of Figure 10 and Figure 11 such that each coefficient b j of y in the following matrix-vector product i,j

[0403]

[0404] is given by

[0405]

[0406] The values offset i,j and scale i,j themselves are integer values. For example, these integers can represent fixed-point numbers, each of which can be stored with a fixed number of bits (e.g., 8 bits) or, for example, the same number of bits n_bits as the number of bits used to store the value .

[0407] In other words, a device is configured to use prediction parameters (e.g., integer values and the values offset i,j and scale i,j ) to represent the predetermined prediction matrix 405 and to calculate the matrix-vector product 404 by performing multiplication and summation on the components of another vector 402 and the prediction parameters and the resulting intermediate results, where the absolute value of the prediction parameters can be represented by an n-bit fixed-point number representation, where n is equal to or less than 14, or alternatively equal to or less than 10, or alternatively equal to or less than 8. For example, the components of another vector 402 are multiplied by the prediction parameters to produce a product as an intermediate result, which is then either summed or forms an addend in the summation.

[0408] According to an embodiment, the prediction parameters include weights, each of which is associated with a corresponding matrix component of the prediction matrix. In other words, the predetermined prediction matrix is, for example, replaced or represented by the prediction parameters. The weights are, for example, integers and / or fixed-point values.

[0409] According to an embodiment, the prediction parameters further include one or more scaling factors, such as the value scale i,j, each of the scaling factors is associated with one or more corresponding matrix components of a predetermined prediction matrix 405 to scale the weights associated with one or more corresponding matrix components of the predetermined prediction matrix 405, e.g., integer values Additionally or alternatively, the prediction parameter includes one or more offsets, e.g., the value offset i,j , each of the offsets is associated with one or more corresponding matrix components of a predetermined prediction matrix 405 to offset the weights associated with one or more corresponding matrix components of the predetermined prediction matrix 405, e.g., integer values

[0410] To reduce the storage amount required for offset i,j and scale i,j , their values can be chosen to be constant for a particular set of indices i, j. For example, their entries can be constant for each column, or they can be constant for each row, or they can be constant for all i, j, as Figure 10 shown.

[0411] For example, in a preferred embodiment, offset i,j and scale i,j are constant for all values of the matrix for one prediction mode, as Figure 11 shown. Thus, when there are K prediction modes, where k = 0..K - 1, only a single value o k and a single value s k are needed to compute the prediction for mode k.

[0412] In the case of offset representation o k and scaling representation s k , the calculation in (1) can be modified to: (2)

[0414]

[0415] Extended embodiments resulting from the above solution

[0416] The above solution means the following embodiments:

[0417] 1. A prediction method as in Part I, where in step 2 of Part I, the following operations are performed on the integer approximation of the matrix-vector product involved: In the (averaged) boundary samples x = (x1,..., x n ), for a fixed i0 (where 1 ≤ i0 ≤ n), compute the vector y = (y1,..., y n ), where y i = x i-mean(x) (for i ≠ i0) and where and where mean(x) represents the mean of x. The vector y is then used as the input to an integer implementation of the matrix-vector product Cy such that the (downsampled) prediction signal pred from step 2 of Part I is given by pred = Cy +

[0418] meanpred(x). In this equation, meanpred(x) represents the signal that is equal to mean(x) for each sample position in the domain of the (downsampled) prediction signal. (See for example Figure 9b )

[0419] 2. A prediction method as in Part I, where in step 2 of Part I

[0420] the following operations are performed on the integer approximation of the matrix-vector product involved: In the (averaged) boundary samples x = (x1, …, x n ), for a fixed i0 where 1 ≤ i0 ≤ n, the vector y = (y1, …, y n-1 ) is computed, where for i < i0, y i = x i - mean(x) and where for i ≥ i0, y i = x i+1 - mean(x), and where mean(x) represents the mean of x. The vector y is then used as the input to an integer implementation of the matrix-vector product Cy such that the (downsampled) prediction signal pred from step 2 of Part I is given by pred = Cy + meanpred(x). In this equation, meanpred(x) represents the signal that is equal to mean(x) for each sample position in the domain of the (downsampled) prediction signal. (See for example Figure 9c )

[0421] 3. A prediction method as in Part I, where the integer implementation of the matrix-vector product Cy is given by using the coefficients in the matrix-vector product z j = ∑ j b i,j * y j . (See for example

[0422] ) Figure 10 )

[0423] 4. A prediction method as in Part I, where step 2 uses one of K matrices such that multiple prediction modes can be computed, each prediction mode using a different matrix

[0424] where k = 0…K-1, where the matrix-vector product C k The integer implementation of y is achieved by using the matrix-vector product z j = ∑ j b i,j *y j of the coefficients in to give. (See for example Figure 11 )

[0425] That is, according to an embodiment of the present application, the encoder and decoder operate as follows to predict a predetermined block 18 of picture 10, in conjunction with Figures 7.1 to 7.4 See one of Figure 8 . For prediction, multiple reference samples are used. As outlined above, the embodiments of the present application will not be limited to intra-frame coding, and thus, the reference samples will not be limited to neighboring samples, that is, samples of neighboring blocks 18 of picture 10. Specifically, the reference samples will not be limited to samples arranged along the outer edges of block 18, such as samples adjacent to the outer edges of neighboring blocks. However, this case is of course an embodiment of the present application.

[0426] To perform the prediction, a sample value vector 400 is formed from reference samples such as reference samples 17a and 17c. The possible formation has been described above. The formation may involve averaging, thus reducing the number of samples 102 or the number of components of vector 400 compared to the reference samples 17 contributing to the formation. As described above, the formation may also depend in some way on the size or dimensions of block 18, such as its width and height.

[0427] An affine transformation or a linear transformation should be performed on this vector 400 to obtain the prediction of block 18. Different nomenclatures have been used above. Using the most recent nomenclature, the aim is to perform the prediction by applying vector 400 to matrix A by utilizing a matrix-vector product in the summation with offset vector b. The offset vector b is optional. The affine transformation or linear transformation determined by A, or A and b, can be determined by the encoder and decoder, or more precisely, the affine transformation or linear transformation is determined by the encoder and decoder for prediction based on the size and dimensions of block 18 as described above.

[0428] However, in order to achieve the computational efficiency improvements outlined above or to make predictions more effective in terms of implementation, affine or linear transformations have been quantized, and the encoder and decoder or their predictors use C and T as mentioned above to represent and perform linear or affine transformations, where C and T applied in the above manner represent a quantized version of the affine transformation. Specifically, instead of directly applying the vector 400 to the matrix A, the predictor in the encoder and decoder applies the vector 402 generated from the sample value vector 400 to the matrix A in a manner that maps it via a predetermined invertible linear transformation T. The transformation T used here can be the same as long as the vector 400 has the same size, i.e., is independent of the size of the block (i.e., width and height), or at least the same for different affine / linear transformations. Above, the vector 402 is represented as y. To perform the affine / linear transformation determined by machine learning, the exact matrix would be B. However, the prediction in the encoder and decoder does not exactly perform B, but rather is done through its approximate or quantized version. Specifically, this representation is done by appropriately representing C in the manner outlined above, where C + M represents the quantized version of B.

[0429] Accordingly, the prediction in the encoder and decoder is further carried out by computing the matrix-vector product 404 between the vector 402 and a predetermined prediction matrix C appropriately represented and stored at the encoder and decoder in the above manner. The vector 406 generated from this matrix-vector product is then used to predict the samples 104 of the block 18. As mentioned above, for prediction, as indicated at 408, each component of the vector 406 can be summed with the parameter a in order to compensate for the corresponding definition of C. Deriving the prediction of the block 18 based on the vector 406 can also involve an optional summation of the vector 406 with an offset vector b. As mentioned above, each component of the vector 406, and accordingly each component of the summation of the vector 406, the vector of all a's indicated at 408, and the optional vector b, may directly correspond to the samples 104 of the block 18 and thus indicate the predicted values of the samples. It is also possible that only a subset of the samples 104 of the block are predicted in this way, while the remaining samples of the block 18, such as 108, are derived by interpolation.

[0430] As mentioned above, there are different embodiments for setting a. For example, it could be the arithmetic mean of the components of the vector 400. For this case, see, for example, FIG. 9. The invertible linear transformation T can be as Figure 9a indicated. i0 respectively indicates predetermined components of the sample value vector and the vector 402, which are replaced by a. However, as indicated above, there are other possibilities. However, in terms of the representation of C, it has also been indicated above that it can be implemented in different ways. For example, the matrix-vector product 404 can become the actual computation of a smaller matrix-vector product with a lower dimension in its actual computation. See, for example, Figure 9c. Specifically, as indicated above, due to the definition of C, the entire i0-th column 412 of C becomes 0, such that the actual calculation of the product 404 can be performed with a reduced version of the vector 402, which is generated from the vector 402 by omitting the component (i.e., multiplying the reduced vector 410 by a reduced matrix C' generated from C by omitting the i0-th column 412).

[0431] The weights of C or the weights of C' (i.e., the components of the matrix) can be represented and stored in fixed-point representation. However, as described above, these weights 414 can also be stored in a manner related to different scaling and / or offset. The scaling and offset can be defined for the entire matrix C, i.e., equal for all weights 414 of the matrix C or matrix C', or can be defined in such a way that the weights 414 of the same column or the same row of the matrix C and matrix C' are respectively constant or equal. In this regard, Figure 10 it is shown that the calculation of the matrix-vector product (i.e., the result of the product) can actually be performed slightly differently, i.e., for example, by shifting the multiplication with scaling towards the vector 402 or vector 404, thereby reducing the number of multiplications that have to be further performed. Figure 11 It shows the case of using one scaling and one offset for all weights 414 of C or C', as performed in the above calculation (2).

[0432] 8. Embodiment of using block-based intra prediction mode for all channels in special cases

[0433] All of the above descriptions should be regarded as optional implementation details of the embodiments described herein. Note that hereinafter, the term matrix-based intra prediction (MIP) is used to represent an intra prediction mode, e.g., a block-based intra prediction mode, which can embody or be equivalent to the mode indicated by the above ALWIP (e.g., as described with respect to Figures 5 to 11 ).

[0434] In this document, for 4:4:4 chroma format and a single tree, it is proposed that for a chroma intra block where the chroma intra mode is a direct mode (DM) mode and the luma intra mode is a MIP mode, this MIP mode is used to generate an intra chroma prediction signal.

[0435] In the direct mode, the chroma intra prediction mode is derived from the luma intra prediction mode, for example. For example, the chroma intra prediction mode is equal to the luma intra prediction mode. For the luma intra prediction mode being a MIP mode, there are exceptions as follows: where the chroma intra prediction mode is a MIP mode only in the case of 4:4:4 color sampling format and in the case of a single tree, while in all other cases, the chroma intra prediction mode is a planar mode. The 4:4:4 color sampling format represents a color sampling format where each color component is sampled equally.

[0436] 8.1 Description of the proposed method

[0437] In the current VTM, MIP is only used for the luminance component [3]. If, within a chrominance frame block, the intra mode is the direct mode (DM), and if the intra mode of the luminance block at the same position is the MIP mode, then the chrominance block must use the planar mode to generate the intra prediction signal. It can be asserted that the main reason for this processing of the DM mode in the case of MIP is that: for the 4:2:0 case or the dual-tree case, the luminance block at the same position can have a different shape from the chrominance block. Therefore, since the MIP mode cannot be applied to all block shapes, in this case, the MIP mode of the luminance block at the same position may not be applicable to the chrominance block.

[0438] On the other hand, it can be asserted that for the case where the chrominance format is 4:4:4 and a single tree is used, the above inapplicability of the luminance MIP mode to the chrominance component will never hold. Therefore, it is proposed that in this case, if the chrominance intra mode within the intra block is the DM mode and if the luminance intra mode is the MIP mode, then this MIP mode is used to generate the chrominance intra prediction signal.

[0439] It can be asserted that the proposed change is relevant, especially if ACT (Adaptive Color Transformation) is enabled, because for the case of using ACT on a given block, the chrominance mode is inferred to be the DM mode. It can be asserted that, especially for the ACT case, the strong correlation of the prediction signal across all channels is beneficial, and if the same intra prediction mode is used for all three components of the block, this correlation can be increased. It can be asserted that the experimental results reported below support this view, which lies in the fact that the proposed change has a significant impact on the camera-captured content in RGB format, which is asserted to be the intersection of the cases where MIP and ACT are mainly designed.

[0440] 8.2 Embodiment of the proposed method

[0441] Figure 12 An embodiment of the prediction of a predetermined second color component block 182 of Picture 10 is shown. A block-based decoder, a block-based encoder, and / or a device for predicting Picture 10 can be configured to perform this prediction.

[0442] According to the embodiment, Picture 10 with more than one color component 101, 102 and a color sampling format 11 is divided into blocks (e.g., divided into first color component blocks 18' 11 to 18' 1n and divided into second color component blocks 18' 21 to 18' 2n) Each color component 101, 102, etc. is equally sampled according to the color sampling format 11, and the picture 10 is equally divided for each color component 101, 102 according to the partitioning scheme 11'. According to the color sampling format 11, each color component 101, 102 is, for example, equally sampled, and according to the partitioning scheme 11', each color component, for example, 101, 102 is divided in the same way. Thus, for example, according to the partitioning scheme 11', the first color component block 18' 11 to 18' 1n has the same geometric features as the second color component block 18' 21 to 18' 2n and, according to the color sampling format 11, the first color component block 18' 11 to 18' 1n has the same sampling as the second color component block 18' 21 to 18' 2n The sampling is indicated by the sampling points in a predetermined second color component block 182 and the first color component block 181 of in - frame prediction located at the same position.

[0443] For each of the first color component blocks 18' 11 to 18' 1n in the in - frame prediction of the picture 10, one intra - prediction mode in the first set 508 of intra - prediction modes is selected to decode and / or encode the first color component 101 of the picture 10 in units of blocks. The first set 508 includes matrix - based intra - prediction modes (MIP modes) 5101 to 510 m , according to each of the matrix - based intra - prediction modes, to predict the inside of the block 18 as follows: a sample value vector 514 is derived from the reference samples 17 adjacent to the inside of the block 18, a matrix - vector product 512 between the sample value vector 514 and the prediction matrices 516 associated with the respective matrix - based intra - prediction modes 5101 to 510 m is calculated to obtain a prediction vector 514, and the samples inside the block 18 are predicted based on the prediction vector 514. The sample value vector 514 has, for example, the features and / or functions as described for the sample value vector 400 in Figure 6 or as described for another vector 402 in Figures 8 to 11 . In other words, the matrix - based intra - prediction 5101 to 510 m is performed, for example, as described in one of Figures 5 to 11 .

[0444] Optionally, the number of components of the prediction vector 518 is less than the number of samples in the intra-block 18. In this case, the samples in the intra-block 18 are predicted based on the prediction vector 518 by interpolating the samples based on the components of the prediction vector 518 assigned to the positions of the support samples in the intra-block 18.

[0445] Optionally, the matrix-based intra prediction modes 5101 to 510 included in the first set 508 of intra prediction modes are selected depending on the block size of the first color component block 181 of the corresponding intra prediction m , so as to be a subset of the matrix-based intra prediction modes in a set of disjoint subsets of the matrix-based intra prediction modes. The prediction matrix 516 associated with the set of disjoint subsets of the matrix-based intra prediction modes is, for example, machine-learned. The prediction matrices 516 included by a subset of the matrix-based intra prediction modes have equal sizes, and the prediction matrices 516 included by two subsets of the matrix-based intra prediction modes selected for different block sizes have different sizes from each other. That is, the prediction matrices included by one subset have equal sizes, but the prediction matrices of different subsets have different sizes.

[0446] Optionally, the prediction matrix 516 associated with the matrix-based intra prediction modes 5101 to 510 included in the first set 508 of intra prediction modes m has equal sizes and is machine-learned.

[0447] Optionally, in addition to the matrix-based intra prediction modes 5101 to 510 included in the first set 508 of intra prediction modes m , the intra prediction modes included in the first set 508 of intra prediction modes include the DC mode 506, the planar mode 504, and the directional modes 5001 to 500 l .

[0448] The predetermined second color component block 182 of the picture 10 is intra-predicted by using the matrix-based intra prediction mode selected for the first color component block 181 of the intra prediction at the same position, and the second color component 102 of the picture 10 is decoded and / or encoded in units of blocks. Therefore, the decoder and / or encoder needs to be able to adopt the MIP modes 5101 to 510 for at least one intra-coded second / third color component block m . For example, this is performed when the initial prediction mode of the predetermined second color component block 182 is the direct mode.

[0449] In direct mode, the intra prediction mode of a predetermined second color component block 182 is derived, for example, from the intra prediction mode of an intra predicted first color component block 181 located at the same position. For example, the intra prediction mode of the predetermined second color component block 182 is equal to the intra prediction mode of the intra predicted first color component block 181 located at the same position. In other words, when the intra prediction mode of the intra predicted first color component block 181 located at the same position is one of MIP modes 5101 to 510 m the predetermined second color component block 182 is also predicted by the same MIP modes 5101 to 510 m According to an embodiment, if no index indicating a prediction mode is signaled in the data stream for the predetermined second color component block 182, it can be inferred as direct mode. Otherwise, the mode signaled in the data stream is used for predicting the predetermined second color component block 182.

[0450] In other words, for each of the second color component blocks 18’

[0451] to 18’ 21 of picture 10, one of a first option and a second option can be selected. According to the first option, based on the intra prediction mode selected for the first color component blocks 18’ 2n to 18’ 11 located at the same position, the intra prediction mode for the corresponding second color component blocks 18’ 1n to 18’ 21 is derived such that when the intra prediction mode selected for the intra predicted first color component blocks 18’ 2n to 18’ 11 located at the same position is one of the matrix-based intra prediction modes 5101 to 510 1n the intra prediction mode for the corresponding second color component blocks 18’ m to 18’ 21 is equal to the prediction mode selected for the intra predicted first color component blocks 18’ 2n to 18’ 11 located at the same position. When the intra prediction mode selected for the intra predicted first color component blocks 18’ 1n to 18’ 11 located at the same position is an intra prediction mode other than the matrix-based intra prediction modes 5101 to 510 1n such as a planar intra prediction mode 504, a DC intra prediction mode 506, and / or an oriented intra prediction mode 5001 to 500 m l 21 ​​to 18’ 2n The intra prediction mode for 2n can also be equal to the intra prediction mode for the first color component block 18’ located at the same position. 11 to 18’ 1n The first option can represent the direct mode. According to the second option, based on the intra mode index present in the data stream for the corresponding second color component block 18’ 21 to 18’ 2n The intra prediction mode for the corresponding second color component block 18’ is selected. 21 to 18’ 2n Optionally, in addition to the other intra mode indices present in the data stream for the first color component block 18’ for intra prediction located at the same position 11 to 18’ 1n The intra mode index is present in the data stream for selection from the first set 508 of intra prediction modes. The intra mode index present in the data stream for the corresponding second color component block 18’ 21 to 18’ 2n Indicates, for example, the prediction mode to be selected from the first set 508 of intra prediction modes for predicting the corresponding second color component block 18’ 21 to 18’ 2n .

[0452] According to an embodiment, an apparatus for predicting a predetermined block of a picture, a block-based decoder, and / or a block-based encoder are configured to: in the case of a 4:4:4 color sampling format 11 of the picture 10 and in the case where the prediction mode of the predetermined second color component block 182 is the direct mode, use the same MIP modes 5101 to 510 for the predetermined second color component block 182 as for the first color component block 181 for intra prediction located at the same position m , where the first color component block 181 for intra prediction located at the same position and the predetermined second color component block 182 represent predetermined blocks associated with different color components (i.e., the first color component and the second color component). The first color component block 181 for intra prediction located at the same position and the predetermined second color component block 182 include, for example, the same geometric features and the same spatial positioning in the picture 10. In other words, the first color component coding tree (e.g., the luminance coding tree) is equal to the second color component coding tree (e.g., the chrominance coding tree). A single tree is used, for example. The picture 10 is equivalently partitioned for each color component 101, 102. The partitioning scheme 11’ defines, for example, the single tree processing of the picture 10.

[0453] According to an embodiment, double-tree processing of picture 10 is also possible. The double-tree processing may be associated with another partitioning scheme 11', according to which picture 10 is partitioned for a first color component 101 using first partitioning information in a data stream, and picture 10 is partitioned for a second color component 102 using second partitioning information that exists in the data stream and is separate from the first partitioning information.

[0454] In the case of a double-tree or non-single-tree, an apparatus for predicting a predetermined block of a picture, a block-based decoder, and / or a block-based encoder are configured, for example, such that when a prediction mode of a first color component block 181 of intra prediction at the same position is one of MIP modes 5101 to 510 m and a prediction mode of a predetermined second color component block 182 is a direct mode, a planar intra prediction mode 504 is used. However, when the prediction mode of the first color component block 181 of intra prediction at the same position is not an MIP mode, for example, when the prediction mode of the first color component block 181 of intra prediction at the same position is a DC intra prediction mode 506, a planar intra prediction mode 504, or an angular intra prediction mode 5001 to 5001 (e.g., an angular intra prediction mode), and the prediction mode of the predetermined second color component block 182 is a direct mode, the prediction mode of the predetermined second color component block 182 is equal to the prediction mode of the first color component block 181 of intra prediction at the same position. This also applies to the case of using a single tree but not a 4:4:4 color sampling format 11, for example, the case of using a 4:1:1 color sampling format 11, a 4:2:2 color sampling format 11, or a 4:2:0 color sampling format 11.

[0455] The color sampling format 11 can be represented by x:y:z, where the first number x, for example, refers to the size of the color component block 18', and the following two numbers y and z both refer to the second color component sample and / or the third color component sample. They, namely y and z, are both related to the first number and respectively define horizontal sampling and vertical sampling. A 4:4:4 signal is not compressed (so it is not subsampled) and completely transmits both the first color component data and the other color component data, such as the second color component sample and / or the third color component sample, that is, the chrominance samples. In a four-by-two array of pixels, 4:2:2 has half the chrominance samples of 4:4:4, while 4:2:0 and 4:1:1 have one-quarter of the chrominance information available. The 4:2:2 signal will have a sampling rate that is half in the horizontal direction but will remain fully sampled in the vertical direction. On the other hand, 4:2:0 will only sample the colors of half of the pixels on the first row and will completely ignore the second row samples, while 4:1:1 will only sample the colors of one pixel on the first row of the samples and one pixel on the second row, that is, the 4:1:1 signal will have a sampling rate that is one-quarter in the horizontal direction but will remain fully sampled in the vertical direction.

[0456] According to an embodiment, the apparatus for predicting a predetermined block of a picture, the block-based decoder, and / or the block-based encoder are configured to: when the prediction mode of the first color component block 181 of intra prediction located at the same position is not one of the MIP modes 5101 to 510 m , for example, is the DC intra prediction mode 506, the planar intra prediction mode 504, or the directional intra prediction modes 5001 to 5001, and when the prediction mode of the predetermined second color component block 182 is the direct mode, independent of the color sampling format 11 and / or independent of the partitioning scheme for each color component of the picture 10, that is, using a single tree or a double tree, use the prediction mode of the first color component block 181 of intra prediction located at the same position as the prediction mode of the predetermined second color component block.

[0457] In the DC intra prediction mode 506, for example, a value (quasi-DC value) is derived based on the neighboring samples 17 that are spatially adjacent to the predetermined block 18 (for example, the first color component block 181 of intra prediction located at the same position and / or the predetermined second color component block 182), and this one DC value is attributed to all the samples of the predetermined block 18 to obtain the intra prediction signal.

[0458] In planar intra prediction mode 504, for example, a two-dimensional linear function defined by a horizontal slope, a vertical slope, and an offset is derived based on neighboring samples 17 that are spatially adjacent to a predetermined block 18 (e.g., the first color component block 181 of intra prediction located at the same position and / or a predetermined second color component block 182), where the linear function defines the predicted sample values of the predetermined block 18.

[0459] In directional intra prediction modes 5001 to 5001 (e.g., angular intra prediction modes), reference samples 17 adjacent to a predetermined block 18 (e.g., the first color component block 181 of intra prediction located at the same position and / or a predetermined second color component block 182) are used to fill the predetermined block 18 in order to obtain the intra prediction signal of the predetermined block 18. Specifically, the reference samples 17 that can be arranged along the boundary of the predetermined block 18 (e.g., along the upper edge and the left - hand edge of the predetermined block 18) represent picture content that is extrapolated or copied into the interior of the predetermined block 18 along a predetermined direction 502. Before extrapolation or copying, the picture content represented by the neighboring samples 17 can be subjected to interpolation filtering, or in other words, can be derived from the neighboring samples 17 by means of interpolation filtering. The angular intra prediction modes 5001 to 5001 differ from each other in the intra prediction direction 502. Each of the angular intra prediction modes 5001 to 5001 can have an associated index, where the association of the index with the angular intra prediction modes 5001 to 5001 can cause the direction 502 to rotate monotonically clockwise or counter - clockwise when the angular intra prediction modes are sorted according to the associated mode index.

[0460] Figure 13 Predictions of the predetermined second color component block 182 of picture 10 according to different partitioning schemes and color sampling formats are shown in more detail.

[0461] Predictions are shown for picture 10a, other pictures 10b, and still other pictures 10c, where different conditions are set for different pictures.

[0462] According to an embodiment, a device, such as a block - based decoder, a block - based encoder, and / or a device for predicting pictures, includes or has access to a set of partitioning schemes 11' to select and / or obtain a partitioning scheme for picture 10. The set of partitioning schemes 11' includes: partitioning scheme 11'1, i.e., the first partitioning scheme, according to which picture 10 is equally partitioned for each color component 101, 102; and optionally, other partitioning schemes 11'2, according to which, using the first partitioning information 11' in the data stream 12 2a picture 10 is partitioned for the first color component 101, and using the first partitioning information 11' present in the data stream 12 2aSeparate second partitioning information 11’ 2b The picture 10 is partitioned with respect to the second color component 102. The first partitioning scheme 11’1 may represent a single-tree processing of the picture 10, while other partitioning schemes 11’2 may represent a dual-tree processing of the picture 10.

[0463] In other partitioning schemes 11’2, the first color component 101 is partitioned in a manner different from the second color component 102. One of the color components 101, 102 may be subdivided, while the other color component 101, 102 may be roughly partitioned. The second color component blocks 18’ 21 to 18’ 24 The boundaries of 11 to 18’ 116 It is also possible that the boundaries of 21 to 18’ 24 do not lie at the same position as the boundaries of the first color component blocks 18’ 11 to 18’ 116 inside the blocks, such as 10 2alternative or as shown in the partitioning of other pictures 10b.

[0464] For the picture 10a, the same prediction as described with respect to Figure 12 may be applied, where the first partitioning scheme 11’1 is selected from the set of partitioning schemes 11’, and where a color sampling format that samples each color component 10a1, 10a2, etc. equivalently is used.

[0465] Figure 13 The apparatus in is configured, for example, to select one of the first option and the second option for each of the second color component blocks for intra prediction of the picture 10a. According to the first option, based on the intra prediction mode selected for the first color component block 18a1 for intra prediction at the same position, an intra prediction mode for the corresponding second color component block (e.g., for a predetermined second color component block 18a2) for intra prediction is derived such that when the intra prediction mode selected for the first color component block 18a1 for intra prediction at the same position is one of the matrix-based intra prediction modes 5101 to 510 m the intra prediction mode of the second color component block 18a2 for the corresponding intra prediction is equal to the prediction mode selected for the first color component block 18a1 for intra prediction at the same position, as Figure 12As shown. According to the second option, an intra prediction mode for the second color component block 18a2 for corresponding intra prediction is selected based on the intra mode index 509 of the second color component block 18a2 for corresponding intra prediction present in the data stream 12. The intra mode index 509 is present in the data stream 12, for example, in addition to the other intra mode indices 507 of the first color component block 18a1 for intra prediction located at the same position, for selection from the first set of intra prediction modes.

[0466] Optionally, the apparatus is configured to: use another partitioning scheme 11'2 to partition more than one color component 10b1, 10b2 and another picture 10b having a color sampling format into other blocks, and sample each color component equivalently according to the color sampling format, so that the first color component 10b1 has the same color sampling as the second color component 10b2, but different block sizes. As Figure 13 shown, a predetermined other second color component block 18b2 of the other picture 10b has, for example, a quarter of the size of the other first color component block 18b1 located at the same position of the other picture 10b. For example, the other picture 10b is sampled in a 4:4:4 color sampling format.

[0467] In the case of using another partitioning scheme 11'2, the other first color component block 18b1 located at the same position of the predetermined other second color component block 18b2 is determined based on, for example, the pixels located in both the block 18b1 and 18b2. According to an embodiment, the pixel is located at the upper left corner, upper right corner, lower left corner, lower right corner, and / or the middle of the predetermined other second color component block 18b2.

[0468] The other first color component 10b1 of the other picture 10b is decoded on a block-by-block basis by selecting one intra prediction mode from the first set of intra prediction modes for each of the other first color component blocks 18b1 for intra prediction of the other picture 10b.

[0469] For each of the other second color component blocks for intra prediction of the other picture 10b, one of the first option and the second option can be selected. According to the first option, an intra prediction mode for the corresponding other second color component block (e.g., the predetermined other second color component block 18b2) for intra prediction is derived based on the intra prediction mode selected for the other first color component block 18b1 located at the same position, such that when the intra prediction mode selected for the other first color component block 18b1 located at the same position is based on the matrix intra prediction modes 5101 to 510 mIn the case of one of them, the intra prediction mode of the other second color component block 18b2 for the corresponding intra prediction is equal to the planar intra prediction mode 504. When the intra prediction mode selected for the other first color component block 18b1 at the same position is not one of the matrix-based intra prediction modes 5101 to 510 m One (for example, the intra prediction mode is the planar intra prediction mode 504, the DC intra prediction mode 506, or the directional intra prediction modes 5001 to 5001), the prediction mode of the other second color component block 18b2 for the corresponding intra prediction is equal to the intra prediction mode selected for the other first color component block 18b1 at the same position. According to the second option, the intra prediction mode of the other second color component block 18b2 for the corresponding intra prediction is selected based on the intra mode index 509 of the other second color component block 18b2 for the corresponding intra prediction present in the data stream 12. In addition to the other intra mode index 5072 of the other first color component block 18b1 for the intra prediction at the same position present in the data stream 12, the intra mode index 5092 is present in the data stream 12, for example, for selection from the first set of intra prediction modes.

[0470] Optionally, the apparatus is configured to: partition more than one color component 101, 102 and yet other pictures 10c having different color sampling formats, and sample the more than one color component 101, 102 differently according to the different color sampling formats. As Figure 13 shown, the first color component 10c1 is sampled in a different manner from the second color component 10c2. The sampling is indicated by sampling points. According to Figure 13 the illustrated embodiment, yet other pictures are sampled according to the 4:2:1 color sampling format. However, it is obvious that other color sampling formats can also be used in addition to the 4:4:4 color sampling format. The use of the first partitioning scheme 11’1 is shown once, and the use of the other partitioning scheme 11’2 is shown once. Independent of the partitioning scheme 11’, the following prediction of yet other pictures 10c can be performed.

[0471] The first color component 10c1 of yet other pictures 10c is decoded block by block by selecting one intra prediction mode from the first set 508 of intra prediction modes for each of the yet other first color component blocks for the intra prediction of yet other pictures 10c.

[0472] In addition, for example, the apparatus is configured to select one of a first option and a second option for each of still other second color component blocks for intra prediction of still other picture 10c. According to the first option, an intra prediction mode for a still other second color component block 18c2 for the corresponding intra prediction is derived based on the intra prediction mode selected for a still other first color component block 18c1 located at the same position, such that when the intra prediction mode selected for the still other first color component block 18c1 located at the same position is one of matrix-based intra prediction modes 5101 to 510 m and one, the intra prediction mode for the still other second color component block 18c2 for the corresponding intra prediction is equal to the planar intra prediction mode 504. When the intra prediction mode selected for the still other first color component block 18c1 located at the same position is not one of matrix-based intra prediction modes 5101 to 510 m and one (e.g., the intra prediction mode is the planar intra prediction mode 504, the DC intra prediction mode 506, or the directional intra prediction modes 5001 to 5001), the prediction mode for the still other second color component block 18c2 for the corresponding intra prediction is equal to the intra prediction mode selected for the other first color component block 18c1 located at the same position. According to the second option, an intra prediction mode for a still other second color component block 18c2 for the corresponding intra prediction is selected based on an intra mode index 5093 for the still other second color component block 18c2 for the corresponding intra prediction present in the data stream. Optionally, in addition to the still other intra mode index 5073 for the still other first color component block 18c1 for intra prediction located at the same position present in the data stream 12, the intra mode index 5093 is present in the data stream 12 for selection from a first set of intra prediction modes.

[0473] According to an embodiment, if the residual coding color transform mode is signaled in the data stream 12 as being deactivated for the corresponding intra-predicted second color component block 18a2, the corresponding other intra-predicted second color component blocks 18b2, and / or the corresponding still other intra-predicted second color component blocks 18c2, e.g., ACT (Adaptive Color Transform) is deactivated, the selection from the first and second options depends on the signaling present in the data stream 12 for the blocks 18a2, 18b2, and / or 18c2. If the residual coding color transform mode is signaled in the data stream 12 as being activated for the blocks 18a2, 18b2, and / or 18c2, e.g., ACT (Adaptive Color Transform) is activated, then no signaling is required. In this case, i.e., the residual coding color transform mode is signaled as being activated for the blocks 18a2, 18b2, and / or 18c2, e.g., ACT (Adaptive Color Transform) is activated, the device may be configured to infer that the first option will be selected.

[0474] For the case of using ACT on a given block, the chroma mode is inferred as the DM mode, i.e., the first option. It can be asserted that, particularly for the ACT case, the strong correlation of the prediction signal across all channels is beneficial, and this correlation can be increased if the same intra-prediction mode is used for all three components of the block. It can be asserted that the experimental results reported below support this view, since the proposed changes have a significant impact on the camera-captured content in RGB format, which is asserted to be the intersection of the cases mainly targeted by the design of MIP and ACT.

[0475] 8.3 Experimental Results

[0476] In this section, the experimental results are reported for 4:4:4 according to common test conditions. In Tables 1 and 2, the results for the proposed changes are reported for the AI and RA configurations, respectively, compared to the VTM-7.0 anchor. The corresponding simulations are performed on an Intel Xeon cluster (E5-2697Av4, AVX2 enabled, Intel Turbo Boost Technology (turboboost) disabled) with a Linux OS and GCC 7.2.1 compiler. The results for RGB and YUV are reported according to the CE-8 conditions. The single tree is always enabled because for the single tree being disabled, the proposed changes do not change the bitstream.

[0477] Table 1. RGB Results of the Proposed Changes: The reference is the VTM-8.0 anchor, and the test is VTM-8.0 with the proposed changes and the AI configuration. The single tree is enabled in both the anchor and the test.

[0478]

[0479]

[0480] Table 2, RGB results of the proposed changes: The reference is the VTM-8.0 anchor, and the test is VTM-8.0 with the proposed changes and RA configuration. Single-tree is enabled in both the anchor and the test.

[0481]

[0482] Table 3, YUV results of the proposed changes: The reference is the VTM-8.0 anchor, and the test is VTM-8.0 with the proposed changes and AI configuration. Single-tree is enabled in both the anchor and the test.

[0483]

[0484]

[0485] Table 4, YUV results of the proposed changes: The reference is the VTM-8.0 anchor, and the test is VTM-8.0 with the proposed changes and RA configuration. Single-tree is enabled in both the anchor and the test.

[0486] Y U V enc time dec time TGM 1080 0,04％ 0,03％ -0,01％ 100％ 103％ TGM 720 -0,03％ 0,06％ -0,08％ 100％ 98％ animation -0,07％ -0,15％ -0,16％ 99％ 99％ mixed -0,14％ -0,15％ -0,26％ 100％ 102％ camera-captured -0,08％ -0,13％ -0,09％ 99％ 101％ overall -0,05％ -0,05％ -0,11％ 100％ 100％ TGM 1080 0,04％ 0,03％ -0,01％ 100％ 103％ TGM 720 -0,03％ 0,06％ -0,08％ 100％ 98％

[0487] 8.4 Conclusions

[0488] In this document, enabling MIP for all three channels in the case of 4:4:4 content and single-tree is proposed. For this case, if the MIP mode is used for the luminance component on intra blocks and if the chroma intra mode is the DM mode, it is proposed to use this MIP mode to generate the chroma intra prediction signal. Adopting the techniques described in this document in the next working draft of VVC is proposed.

[0489] 9. References

[0490] [1] P. Helle et al., “Non-linear weighted intra prediction”, JVET-L0199, Macao, China, October 2018.

[0491] [2] F. Bossen, J. Boyce, K. Suehring, X. Li, V. Seregin, “JVET common test conditions and software reference configurations for SDR video”, JVET-K1010, Ljubljana, SI, July 2018.

[0492] [3]B.Bross,J.Chen,S.Liu,Y.-K.Wang,Versatile Video Coding(Draft 7),Document JVET-P2001,Version 14,Geneva,Switzerland,October 2019。

[0493] […]for the common test conditions of 4:4:4.

[0494] Other embodiments and examples

[0495] Generally, an example can be implemented as a computer program product having program instructions that are operable to perform one of the methods when the computer program product is run on a computer. The program instructions can be stored, for example, on a machine-readable medium.

[0496] Other examples include a computer program stored on a machine-readable carrier for performing one of the methods described herein.

[0497] In other words, an example of a method is thus a computer program having program instructions for performing one of the methods described herein when the computer program is run on a computer.

[0498] Thus, another example of a method is a data carrier medium (or digital storage medium or computer-readable medium) on which a computer program is recorded for performing one of the methods described herein. The data carrier medium, digital storage medium or recording medium is tangible and / or non-transitory, rather than intangible and transitory signals.

[0499] Thus, another example of a method is a data stream or signal sequence representing a computer program for performing one of the methods described herein. The data stream or signal sequence can be transmitted, for example, via a data communication connection (e.g., via the Internet).

[0500] Another example includes a processing device, such as a computer or a programmable logic device, for performing one of the methods described herein.

[0501] Another example includes a computer on which a computer program is installed for performing one of the methods described herein.

[0502] Another example includes a device or system for transmitting a computer program to a recipient (e.g., electronically or optically) for performing one of the methods described herein. The recipient can be, for example, a computer, a mobile device, a memory device, etc. The device or system can include, for example, a file server for transmitting the computer program to the recipient.

[0503] In some examples, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some examples, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. Generally, these methods may be performed by any suitable hardware device.

[0504] The above examples are illustrative only of the principles disclosed above. It is to be understood that modifications and variations of the arrangements and details described herein will be apparent. Accordingly, it is intended to be limited by the scope of the appended claims rather than by the specific details given by way of description and explanation of the examples herein.

[0505] Even if reference numerals appear in different figures, in the following description, the same or equivalent elements or elements having the same or equivalent functions are denoted by the same or equivalent reference numerals.

Claims

1. A method for reconstructing a picture (10) having more than one color component (101, 102), the method comprising: Dividing a first color component of the picture (10) into coding tree blocks; By selecting an intra prediction mode from a first set (508) of intra prediction modes, a first block of a first color component (101) is decoded using matrix-based intra prediction, the first set (508) including a plurality of matrix-based intra prediction modes (5101 to 510 m ), wherein according to each matrix-based intra prediction mode, an internal block (18) is predicted by: deriving a sample value vector (514) from reference samples (17) adjacent to the internal block (18), calculating a matrix-vector product (512) between the sample value vector (514) and a prediction matrix (516) associated with the corresponding matrix-based intra prediction mode (5101 to 510 m ), so as to obtain a prediction vector (518), and predicting samples in the internal block (18) based on the prediction vector (518). Decoding a second block of the second color component (102) of the picture (10) that is in the same position as a first block of the first color component (101) using direct mode, which means that the intra prediction mode of the second color component block is derived based on the intra prediction mode selected for the first block of the first color component. The method further includes: determining whether to use a 4:4:4 color sampling format in which color components are equally sampled, and also determining whether to use a single tree to divide the first color component and the second color component into coding tree blocks, and in the case of determining to use both 4:4:4 color sampling and a single tree, using a matrix-based intra prediction mode selected for the first block of the first color component to decode the second color component block, and if it is determined not to use one or both of 4:4:4 color sampling and a single tree, using a planar intra prediction mode to decode the second color component block.

2. The method according to claim 1, wherein, Select the matrix-based intra prediction modes (5101 to 510 m ) included in the first set (508) of intra prediction modes according to the block size of the first color component block depending on the corresponding intra prediction, so as to be a subset of the matrix-based intra prediction modes (5101 to 510 m ) among the sets of disjoint subsets of the matrix-based intra prediction modes (5101 to 510 m ).

3. The method according to claim 2, wherein, The prediction matrix associated with a set of disjoint subsets of matrix-based intra prediction modes (5101 to 510 m ) is machine-learned. The prediction matrices included in a subset of the matrix-based intra prediction modes have equal sizes, and the prediction matrices included in two subsets of the matrix-based intra prediction modes selected for different block sizes have different sizes from each other.

4. The method according to claim 1, wherein The prediction matrices associated with the matrix-based intra prediction modes (5101 to 510 m ) included in the first set (508) of intra prediction modes have equal sizes and are machine-learned.

5. A video decoder comprising at least one processor and a memory, the memory comprising instructions that, when executed by the at least one processor, cause the video decoder to: Execute the method according to any one of claims 1 to 4.

6. A method for encoding a picture (10) having more than one color component (101, 102), the method comprising: Dividing a first color component of the picture (10) into coding tree blocks; Encoding a first block of a first color component (101) using matrix-based intra prediction by selecting an intra prediction mode from a first set (508) of intra prediction modes, the first set (508) including a plurality of matrix-based intra prediction modes (5101 to 510 m ), and for each matrix-based intra prediction mode, predicting an internal block (18) by: deriving a sample value vector (514) from reference samples (17) adjacent to the internal block (18), calculating a matrix-vector product (512) between the sample value vector (514) and a prediction matrix (516) associated with the corresponding matrix-based intra prediction mode (5101 to 510 m ) to obtain a prediction vector (518), and predicting samples in the internal block (18) based on the prediction vector (518); Encoding a second block of the second color component (102) of the picture (10) that is in the same position as a first block of the first color component (101) using direct mode, which means that the intra prediction mode of the second color component block is derived based on the intra prediction mode selected for the first block of the first color component. The method further includes: determining whether to use a 4:4:4 color sampling format in which color components are equally sampled, and also determining whether to use a single tree to divide the first color component and the second color component into coding tree blocks, and in the case of determining to use both 4:4:4 color sampling and a single tree, using a matrix-based intra prediction mode selected for the first block of the first color component to encode the second color component block, and if it is determined not to use one or both of 4:4:4 color sampling and a single tree, using a planar intra prediction mode to encode the second color component block.

7. The method according to claim 6, wherein, Select the matrix-based intra prediction modes (5101 to 510 m ) included in the first set (508) of intra prediction modes according to the block size of the first color component block depending on the corresponding intra prediction, so as to be a subset of the matrix-based intra prediction modes (5101 to 510 m ) among the sets of disjoint subsets of the matrix-based intra prediction modes (5101 to 510 m ).

8. The method according to claim 7, wherein, associated with a set of disjoint subsets of matrix-based intra prediction modes (5101 to 510 m ) is machine-learned, the prediction matrices included by a subset of matrix-based intra prediction modes have equal sizes, and the prediction matrices included by two subsets of matrix-based intra prediction modes selected for different block sizes have different sizes from each other.

9. The method according to claim 6, wherein The prediction matrices associated with the matrix-based intra prediction modes (5101 to 510 m ) included in the first set (508) of intra prediction modes have equal sizes and are machine-learned.

10. A video encoder comprising at least one processor and a memory, the memory comprising instructions that, when executed by the at least one processor, cause the video encoder to: Execute the method according to any one of claims 6 to 9.

11. A non-transitory digital storage medium having a computer program stored thereon, the computer program, when run on a computer, performing the method according to any one of claims 1 to 4 or 6 to 9.

Images