Matrix-based prediction for all channels in 4:4:4 chroma format and single tree case

Abstract

A block-based decoder is disclosed that is configured to partition one or more color components (101, 102) and a picture (10) of a color sampling format into blocks using a partitioning scheme (11'1) in which the picture (10) is equally partitioned for each color component (101, 102) as each color component (101, 102) is equally sampled. Additionally, for each intra-predicted first color component block (18'11-18'1n) of the picture (10), the block-based decoder decodes the first color component (101) of the picture (10) in blocks by selecting one of a first set (508) of intra-predicted modes. The first set (508) includes a matrix-based intra-prediction mode (5101 to 510m) which derives a sample value vector (514) among reference samples (17), neighbors block inners (18), calculates a matrix-vector product (512) between the sample value vector (514) and a prediction matrix (516) associated with individual matrix-based intra-prediction modes (5101 to 510m) to obtain a prediction vector (518), and predicts a sample of block inners (18) based on the prediction vector (518), each according to which block inners (18) are predicted. Additionally, the block-based decoder is configured to decode the second color component (102) of the picture (10) in blocks by intra-predicting the predetermined second color component block (182) of the picture using a matrix-based intra-predicting mode selected for the co-located intra-predicted first color component block (181).

Description

Technology Field

[0001] Embodiments according to the present invention relate to an apparatus and method for encoding or decoding a picture or video using matrix-based intra-prediction for all channels in a 4:4:4 chroma format and a single tree case. Background Technology

[0002] Currently, in the VVC Test Model (VTM), Matrix-based Prediction (MIP) is used only for the lumina component [3]. When the intra mode of a chroma intra-block is Direct Mode (DM) and the intra mode of a co-located lumina block is MIP Mode, the chroma block must generate an intra-prediction signal using Planar Mode. It is argued that the main reason for treating the DM mode in this way in the case of MIP is that in the case of 4:2:0 or dual trees, the co-located lumina block may have a different shape from the chroma block. Therefore, since MIP Mode cannot be applied to all block shapes, the MIP Mode of the co-located lumina block may not be applied to the chroma block in that case.

[0003] Therefore, it is desired to provide a concept for rendering picture coding and / or video coding that is more efficient in supporting matrix-based intra-prediction for all channels. Additionally, or alternatively, it is desired to reduce signaling costs by reducing the bitstream.

[0004] This is achieved by the subject matter of the independent claim of the present application.

[0005] Further embodiments according to the present invention are defined by the subject matter of the dependent claims of this application.

[0006] According to the first aspect of the present invention, the inventors of the present application realized that one problem faced when attempting to use a matrix-based intra-prediction mode (MIP mode) to predict samples of a predetermined block of a picture is due to the fact that it is not currently possible to use MIP for all color components of the picture. According to the first aspect of the present application, this difficulty is overcome by using the same MIP mode for the intra-prediction blocks of two color components that share the same location in the picture, i.e., coexisting intra-prediction blocks, for example, when two color components are sampled identically and partitioned identically into blocks. The inventors found that it is advantageous to use the same MIP mode for coexisting intra-prediction blocks of two or more color components of the picture. This is based on the idea that a strong correlation of prediction signals for all color components of the picture is beneficial, and that this correlation can be increased when the same intra-prediction mode is used for two or more color components of the intra-prediction blocks of the picture. The bitstream and associated signalization costs can be reduced by using the same MIP mode for coexisting blocks of different color components within the same picture. Additionally, coding complexity can be reduced.

[0007] Accordingly, according to the first aspect of the present application, a block-based decoder / encoder partitions a picture of one or more color components and a color sampling format into blocks using a partitioning scheme in which the picture is partitioned equally for each color component, for example, the first color component of the picture has the same sampling as all other color components of the picture, for example, all color components of the picture have the same spatial resolution.

[0008] A picture may be composed of one luminance component and two chroma components, for example, in YUV, YPbPr, and / or YCbCr color spaces, wherein the luminance component and the two chroma components represent one or more color components. For example, in RGB color spaces, a picture may be composed of a red component, a green component, and a blue component, representing one or more color components. A picture may be composed of, for example, a first color component, a second color component, and optionally a third color component. It is evident that the described block-based decoder / encoder can also be used for pictures having different color components and pictures having different numbers of color components. For example, a picture is partitioned equally for each color component by partitioning the first color component into a first color component block, partitioning the second color component into a second color component block, and optionally partitioning the third color component into a third color component. The determination of whether the color components of a picture are inter-predicted, i.e., inter-coded, or intra-predicted, i.e. intra-coded, can be made at the granularity level or at the color component block level.

[0009] A block-based decoder / encoder is configured to decode / encode the first color component of a picture from / to a data stream in blocks, i.e., in units of first color component blocks, by selecting one from a first set of intra-prediction modes for each intra-prediction first color component block of the picture, i.e., for each intra-prediction first color component block. The first color component blocks associated with inter-prediction, i.e., the inter-prediction first color component blocks, may be processed differently. A first set of intra-prediction modes includes a matrix-based intra-prediction mode that derives a sample value vector among reference samples, neighbors a block inner, performs a matrix-vector product between a sample value vector and a prediction matrix associated with an individual matrix-based intra-prediction mode to obtain a prediction vector, and predicts the block inner by predicting the samples of the block inner based on the prediction vector. Additionally, the block-based decoder / encoder is configured to decode / encode the second color component of a picture in units of the blocks by intra-predicting a predetermined second color component block of the picture using a selected matrix-based intra-prediction mode for a co-located intra-predicted first color component block. The first color component of the picture may be the luminance component of the picture, and the second color component of the picture may be the chroma component of the picture.

[0010] According to one embodiment, the number of components of the prediction vector is less than the number of samples in the block inner, and the block-based decoder / encoder is configured to predict samples in the block inner by interpolating samples based on the components of the prediction vector assigned to the supporting sample positions of the block inner based on the prediction vector. This allows the total number of multiplications required in the matrix-vector multiplication operation to be reduced by obtaining such a prediction vector, and for this reason, the complexity and signaling cost of the decoder / encoder can be reduced.

[0011] According to one embodiment, a block-based decoder / encoder is configured to select one of a first option and a second option for each of the intra-predicted second color component blocks of a picture, that is, for each second color component associated with intra-predicting. In the first option, if the intra-predicted mode selected for the coexisting intra-predicted first color component blocks is any of the matrix-based intra-predicted modes, the intra-predicted mode for the individual intra-predicted second color component blocks is derived based on the intra-predicted mode selected for the coexisting intra-predicted first color component blocks (181) in such a way that the intra-predicted mode for the individual intra-predicted second color component blocks is identical to the intra-predicted mode selected for the coexisting intra-predicted first color component blocks. In the second option, an intra-prediction mode for an individual intra-prediction second color component block is selected based on an intra-mode index present in the data stream for the individual intra-prediction second color component block. For example, the decoder / encoder may be configured to select the first option if a direct mode or a residual coding color conversion mode is indicated for the intra-prediction second color component block. Otherwise, by selecting the second option, for example, a mode signaled in the data stream is used for the prediction of the intra-prediction second color component block.

[0012] According to one embodiment, a block-based decoder / encoder is configured such that, depending on the block dimension of an individual intra-predicted first color component block, a matrix-based intra-predicted mode composed of a first set of intra-predicted modes is selected to be one subset of matrix-based intra-predicted modes among a collection of mutually disjoint subsets of matrix-based intra-predicted modes. For example, each subset of matrix-based intra-predicted modes may be associated with any block dimension. For example, only the matrix-based intra-predicted modes of the selected subset may be composed of the first set of intra-predicted modes. The first set of intra-predicted modes may differ from other block dimensions among the selected intra-predicted modes for an individual intra-predicted first color component block. Therefore, the decoder / encoder is configured to preselect an intra-predicted mode, which can be adapted to individual intra-predicted first color component blocks based on the block dimensions of the intra-predicted first color component blocks. This can reduce decoder / encoder complexity and signaling costs.

[0013] According to one embodiment, a prediction matrix associated with a set of mutually disjoint subsets of matrix-based prediction modes is machine-learned, and a prediction matrix included by one subset of matrix-based intra-prediction modes has the same size, and a prediction matrix included by two subsets of matrix-based intra-prediction modes selected for different block sizes has different sizes. Each subset can group multiple matrix-based intra-prediction modes associated with prediction matrices of the same dimension.

[0014] According to one embodiment, the block-based decoder / encoder is configured such that the prediction matrix associated with the matrix-based intra-prediction mode included by the first set of intra-prediction modes has the same size as the machine learning matrix.

[0015] According to one embodiment, a block-based decoder / encoder is configured such that, except for a matrix-based intra-prediction mode included by a first set of intra-prediction modes, the matrix-based intra-prediction mode included by the first set of intra-prediction modes is configured to include a DC mode, a planar mode, and a directional mode. The first set of intra-prediction modes additionally includes, for example, a DC mode and / or a planar mode and / or a directional mode for the matrix-based prediction mode.

[0016] According to one embodiment, a block-based decoder / encoder is configured to select a partitioning scheme from a set of partitioning schemes. The set of partitioning schemes includes a further partitioning scheme in which a picture is partitioned for a first color component using first partitioning information of a data stream, and the picture is partitioned for a second color component using second partitioning information that exists in / signaled in a data stream separated from the first partitioning information. Accordingly, the set of partitioning schemes includes, for example, a partitioning scheme in which the picture is partitioned equally for each color component, and a further partitioning scheme in which the picture is partitioned differently for the first color component than for the second color component.

[0017] According to one embodiment, a block-based decoder / encoder is configured to select either a first option or a second option for each of the intra-predicted second color component blocks of a picture, i.e., as described above. In the first option, if the intra-predicted mode selected for the coexisting intra-predicted first color component blocks is any of the matrix-based intra-predicted modes, the individual intra-predicted second color component blocks are derived based on the intra-predicted mode selected for the coexisting intra-predicted first color component blocks in such a way that the intra-predicted mode for the individual intra-predicted second color component blocks is identical to the intra-predicted mode selected for the coexisting intra-predicted first color component blocks. In the second option, an intra-predicted mode for an individual intra-predicted second color component block is selected based on an intra-mode index present in / signaled in the data stream for the individual intra-predicted second color component block. Additionally, the block-based decoder / encoder partitions one or more color components and further pictures of color sampling formats into further blocks using an additional partitioning scheme, as each color component is sampled equally, for example, the further pictures are partitioned for the first color component differently from the second color component as mentioned above. The first color component of the further pictures may be partitioned into a second color component further block. The decoder / encoder is configured to decode / encode the first color component of the further pictures in units of the further blocks by selecting one of the first set of intra-predicted modes for, for example, the intra-predicted first color component further blocks of the further pictures.Additionally, the decoder / encoder may be configured to select one of a first option and a second option for each of the intra-predicted second color component addition blocks of the additional picture. In the first option, if the intra-predicted mode selected for the coexisting intra-predicted first color component addition blocks is any of the matrix-based intra-predicted modes, the intra-predicted mode for the individual intra-predicted second color component addition blocks is derived based on the intra-predicted mode selected for the coexisting first color component addition blocks in such a way that the intra-predicted mode for the individual intra-predicted second color component addition blocks is identical to the planar intra-predicted mode. In the second option, the intra-predicted mode for the individual intra-predicted second color component addition blocks is selected based on the intra-mode index present in the data stream for the individual intra-predicted second color component addition blocks. As outlined above, the first option, for example, the prediction mode for the intra-predicted second color component block, is directly derived from the prediction mode of the coexisting intra-predicted first color component block. This derivation may depend on the partitioning scheme selected for the picture. Accordingly, the implementation of the first option may differ from that of the additional picture when compared to the picture, because the picture is partitioned using a partitioning scheme corresponding to the picture being partitioned equally for each color component, the additional picture is partitioned using an additional partitioning scheme corresponding to the picture being partitioned for the first color component using first partitioning information existing / signaled in the data stream, and the picture is partitioned for the second color component using second partitioning information existing / signaled in the data stream separated from the first partitioning information.

[0018] According to one embodiment, a block-based decoder / encoder is configured to select one of a first option and a second option for each of the intra-predicted second color component blocks of a picture, i.e., as described above. In the first option, if the intra-predicted mode selected for the coexisting intra-predicted first color component blocks is any of the matrix-based intra-predicted modes, an intra-predicted mode for the individual intra-predicted second color component blocks is derived based on the intra-predicted mode selected for the coexisting intra-predicted first color component blocks in such a way that the intra-predicted mode for the individual intra-predicted second color component blocks is identical to the intra-predicted mode selected for the coexisting intra-predicted first color component blocks. In the second option, an intra-prediction mode for an individual intra-prediction second color component block is selected based on an intra-mode index present in the data stream for the individual intra-prediction second color component block. Additionally, the block-based decoder / encoder is configured to partition one or more color components and an even addition picture of different color sampling formats according to one or more color components being sampled differently, and to select one of a first set of intra-prediction modes for each of the intra-prediction first color component addition blocks of the even addition picture, thereby decoding / encoding the first color component of the even addition picture on an even addition block basis. The block-based decoder / encoder may be configured to select either the first option or the second option for each of the intra-prediction second color component addition blocks (18c2) of the even addition picture. In the first option, the intra-prediction mode selected for the coexisting intra-prediction first color component even addition block (18c1) is a matrix-based intra-prediction mode (5101-510 mIn any of the cases, an intra-predicted mode for an individual intra-predicted second color component even addition block (18c2) is derived based on an intra-predicted mode selected for a coexisting first color component even addition block (18c1), in such a way that the intra-predicted mode for an individual intra-predicted second color component even addition block (18c2) is identical to the planar intra-predicted mode. In the second option, an intra-predicted mode for an individual intra-predicted second color component even addition block is selected based on an intra-mode index present in / signaled in the data stream for the individual intra-predicted second color component even addition block. As already outlined above, in the first option, for example, the prediction mode for the intra-predicted second color component block is directly derived from the prediction mode of the coexisting intra-predicted first color component block. This derivation may depend on the color sampling format of the picture. Accordingly, the implementation of the first option may differ from the picture when compared with an additional picture, because the picture includes a color sampling format in which each color component is sampled identically, and the additional picture includes different color sampling formats in which one or more color components are sampled differently.

[0019] According to one embodiment, a block-based decoder / encoder is configured to select a partitioning scheme from a sheet of partitioning schemes, and the set of partitioning schemes includes additional partitioning schemes in which a picture is partitioned for a first color component using first partitioning information of a data stream and partitioned for a second color component using second partitioning information present in / signaled in a data stream separated from the first partitioning information. The block-based decoder / encoder is configured to partition an even additional picture of one or more color components using the partitioning scheme or the additional partitioning scheme.

[0020] According to one embodiment, the block-based decoder / encoder is configured to make a selection of either the first or the second option by inferring that the first option is selected when the residual coding color transformation mode is signaled to be deactivated for an individual intra-predicted second color component block of the data stream, according to the signaling present in the data stream for the individual intra-predicted second color component block, and when the residual coding color transformation mode is signaled to be activated for an individual intra-predicted second color component block (182) of the data stream.

[0021] One embodiment relates to a block-based decoding / encoding method, and the method comprises the step of partitioning one or more color components and a picture of a color sampling format into blocks using a partitioning scheme in which a picture (10) is equally partitioned for each color component as each color component is equally sampled. Additionally, the method comprises the steps of, for each intra-predicted first color component block of a picture, selecting one of a first set of intra-prediction modes—the first set includes matrix-based intra-prediction modes corresponding to each of which a block inner is predicted by deriving a sample value vector among reference samples—neighboring the block inners, computing a matrix-vector product between a sample value vector and a prediction matrix associated with an individual matrix-based intra-prediction mode to obtain a prediction vector, and predicting samples of the block inners based on the prediction vector to decode / encode the first color component of the picture block by block. Additionally, the method comprises the step of decoding / encoding the second color component of the picture block by block by intra-predicting a predetermined second color component block of the picture using the selected matrix-based intra-prediction mode for a coexisting intra-predicted first color component block.

[0022] The method described above is based on the same considerations as the encoder / decoder described above. Through this, the method can be completed for all the features and functionalities described for the encoder / decoder.

[0023] One embodiment relates to a data stream having a picture or video encoded using the encoding method described herein.

[0024] One embodiment relates to a computer program comprising program code for performing the method described herein when run on a computer. Brief explanation of the drawing

[0026] The drawings are not necessarily scaled in importance but generally focus on explaining the principles of the invention. In the following description, various embodiments of the invention are described with reference to the following drawings. FIG. 1 illustrates an example of encoding into a data stream. FIG. 2 illustrates an embodiment of an encoder. FIG. 3 illustrates an example of picture reconstruction. FIG. 4 illustrates an example of a decoder. FIG. 5 illustrates a schematic diagram of block prediction for encoding and / or decoding according to one embodiment. FIG. 6 illustrates the matrix operation of block prediction for encoding and / or decoding according to one embodiment. FIG. 7a illustrates a block prediction having a reduced sample value vector according to one embodiment. FIG. 7b illustrates block prediction using sample interpolation according to one embodiment. FIG. 7c illustrates a block prediction having a reduced sample value vector in which only a few boundary samples are averaged according to one embodiment. FIG. 7d illustrates a block prediction having a reduced sample value vector in which four boundary samples are averaged according to one embodiment. FIG. 8 illustrates a matrix operation performed by a device according to one embodiment. FIGS. 9a to 9c illustrate detailed matrix operations performed by the device according to one embodiment. FIG. 10 illustrates a detailed matrix operation performed by a device using offset and scaling parameters according to one embodiment. FIG. 11 illustrates a detailed matrix operation performed by the device using offset and scaling parameters according to another embodiment. FIG. 12 illustrates an example of the prediction of a predetermined second color component of a picture. FIG. 13 illustrates an example of another prediction of a predetermined second color component of a picture. Specific details for implementing the invention

[0027] Identical or equivalent elements or elements having the same or equivalent function are indicated in the following description by the same or equivalent reference number, even if they appear in different drawings.

[0028] In the following description, a number of details are presented to provide a more complete description of the embodiments of the present invention. However, it will be apparent to those skilled in the art that embodiments of the present invention may be practiced without these specific details. In other examples, well-known structures and devices are depicted in block diagram form without detail to avoid obscuring embodiments of the present invention. Additionally, features of different embodiments described below may be combined with one another unless specifically stated otherwise.

[0029] 1. Introduction

[0030] In the following, different examples, embodiments, and aspects of the invention are described. At least some of these examples, embodiments, and aspects are referred to as methods and / or apparatus for performing block-based prediction using linear or affine transformations, e.g., video coding and / or e.g., neighboring sample reduction, and for optimizing video delivery (e.g., broadcasting, streaming, file playback, etc.) for video applications and / or virtual reality applications.

[0031] Additionally, examples, embodiments, and aspects may be referenced to High Efficiency Video Coding (HEVC) or subsequent work. Furthermore, additional embodiments, examples, and aspects will be defined by the appended claims.

[0032] It should be noted that any embodiments, examples, and aspects defined by the claims may be supplemented by any details (features and functions) described in the following chapter.

[0033] Additionally, the embodiments, examples, and aspects described in the following chapter may be used individually and may be supplemented by any features of other chapters or any features included in the claims.

[0034] Additionally, it should be noted that the individual examples, embodiments, and aspects described in this specification may be used individually or in combination. Accordingly, details may be added to each of the individual aspects without adding details to any other of the examples, embodiments, and aspects.

[0035] Additionally, it should be noted that the present disclosure explicitly or implicitly describes features of a decoding and / or encoding system and / or method.

[0036] Furthermore, features and functions disclosed herein in relation to the method may also be utilized in the device. Additionally, any features and functions disclosed herein in relation to the device may also be utilized in the corresponding method. In other words, the method disclosed herein may be complemented by any features and functions described in relation to the device.

[0037] Additionally, any of the features and functions described herein may be implemented in hardware or software, or a combination of hardware and software, as described in the "Implementation Alternatives" section.

[0038] Additionally, any of the functions described in parentheses ((...)" or "[...]") may be considered optional in some examples, embodiments, or aspects.

[0039] 2. Encoder, Decoder

[0040] In the following, various examples are described that can help achieve more effective compression when using block-based prediction. Some examples achieve high compression efficiency by consuming a set of intra-prediction modes. The latter may be, for example, added to other heuristically designed intra-prediction modes, or, for example, provided exclusively. Other examples also utilize both of the two specializations discussed just before. However, as a minor variation of these embodiments, intra-prediction may be switched to inter-prediction instead by using reference samples from other pictures.

[0041] To facilitate understanding of the following examples of the present application, the description begins with a presentation of possible encoders and decoders in which the subsequently outlined examples of the present application can be constructed. FIG. 1 illustrates an apparatus for encoding an image (10) into a data stream (12) in blocks. This apparatus is indicated by reference numeral (14) and may be a still picture encoder or a video encoder. In other words, when the encoder (14) is configured to encode a video (16) containing a picture (10) into a data stream (12), the picture (10) may be the current picture of the video (16), or the encoder (14) may exclusively encode the picture (10) into the data stream (12).

[0042] As mentioned, the encoder (14) performs encoding in a block-wise or block-based manner. To this end, the encoder (14) subdivides the picture (10) into blocks, and a block is a unit in which the encoder (14) encodes the picture (10) into a data stream (12). An example of how the picture (10) can be subdivided into blocks (18) is described in more detail below. Generally, subdivision can result in blocks (18) of different block sizes, such as a block (18) of a fixed size, such as an array of blocks arranged in rows and columns, or using hierarchical multitree subdivision, such as starting multitree subdivision from the entire picture area of ​​the picture (10) or from pre-partitioning the picture (10) into an array of tree blocks, and this example should not be treated as excluding other possible methods of subdividing the picture (10) into blocks (18).

[0043] Additionally, the encoder (14) is a predictive encoder configured to predictively encode the picture (10) into a data stream (12). For a specific block (18), this means that the encoder (14) determines a predictive signal for the block (18) and encodes the predictive residual, that is, the predictive error where the predictive signal deviates from the actual picture content within the block (18), into the data stream (12).

[0044] The encoder (14) may support different prediction modes to induce a prediction signal for a specific block (18). In the following example, the important prediction mode is an intra prediction mode in which the inner of the block (18) is predicted spatially. The encoding of the picture (10) into a data stream (12) and the corresponding decoding procedure may be based on a specific coding order (20) defined between the blocks (18). For example, the coding order (20) may traverse the blocks (18) according to a raster scan order, such as traversing each row from left to right, or from top to bottom with respect to the row. In the case of hierarchical multi-tree based segmentation, the raster scan order may be applied within each layer level, where a depth-first traversal order is applied, that is, leaf nodes within a block of a specific layer level may precede blocks of the same layer that have the same parent block according to the coding order (20). Depending on the coding order (20), neighboring and already encoded samples of block (18) are primarily located on one or more sides of block (18). In the example presented here, for instance, neighboring and already encoded samples of block (18) are located on the top and left sides of block (18).

[0045] The intra-prediction mode may not be the only one supported by the encoder (14). If the encoder (14) is a video encoder, for example, the encoder (14) may support an inter-prediction mode as the block (18) is temporarily predicted from a picture of a previously encoded video (16). This inter-prediction mode may be a motion-compensated prediction mode in which a motion vector is signaled for such a block (18) that represents a relative spatial offset of a portion from which the prediction signal of the block (18) is derived as a copy. Additionally or alternatively, other non-intra-prediction modes may also be available, such as an inter-prediction mode when the encoder (14) is a multi-view encoder, or a non-prediction mode in which the inner of the block (18) is coded as is, i.e., without any prediction.

[0046] Before starting the description of the present application by focusing on the intra-prediction mode, two corresponding examples for decoder fitting are provided in FIG. 1 and FIG. 2, respectively, for a more specific example of a possible block-based encoder, namely a possible implementation of the encoder (14) described in relation to FIG. 2.

[0047] FIG. 2 illustrates a possible implementation of the encoder (14) of FIG. 1, namely an implementation in which the encoder is configured to use transform coding to encode a predicted residual, but this is close to an example and the application is not limited to such a type of predicted residual coding. According to FIG. 2, the encoder (14) includes a subtractor (22) configured to subtract a corresponding predicted signal (24) from the current block (18) based on an inbound signal, i.e., a picture (10) or block, so as to obtain a predicted residual signal (26) that is then encoded by the predicted residual encoder (28) into the data stream (12). The predicted residual encoder (28) is composed of a lossy encoding stage (28a) and a lossless encoding stage (28b). The loss stage (28a) receives a predicted residual signal (26) and includes a quantizer (30) that quantizes samples of the predicted residual signal (26). As already mentioned above, the present example utilizes a transform coding of the predicted residual signal (26), and accordingly, the loss encoding stage (28a) includes a transform stage (32) connected between the subtractor (22) and the quantum regression (30) to transform the spectrally decomposed predicted residual (26) with the quantization of the quantizer (30) occurring on the transformed coefficients representing the residual signal (26). The transform may be a DCT, DST, FFT, Hadamard transform, etc. The converted and quantized predicted residual signal (34) is losslessly encoded by a lossless encoding stage (28b), which is an entropy coder that entropies the quantized predicted residual signal (34) into a data stream (12).The encoder (14) further includes a prediction residual signal reconstruction stage (36) connected to the output of the quantizer (30) to reconstruct the prediction residual signal (34) from the converted and quantized prediction residual signal (34) in a manner that can also be used in the decoder, i.e., the quantizer (30), by considering the coding loss. To this end, the prediction residual reconstruction stage (36) includes an inverse quantizer (38) that performs the inverse of the quantization of the quantizer (30), followed by an inverse transformer (40) that performs the inverse of the transformation, such as the inverse of the spectral decomposition, such as the inverse of the special transformation examples mentioned above performed in the transformer (32). The encoder (14) includes an adder (42) that adds the reconstructed prediction residual signal output by the inverse transformer (40) and the prediction signal (24) so ​​as to output the reconstructed signal, i.e., the reconstructed sample. This output is supplied to the predictor (44) of the encoder (14), and the encoder (14) determines the prediction signal (24) based thereon. It is the predictor (44) that supports all prediction modes already discussed above in relation to FIG. 1. FIG. 2 shows that if the encoder (14) is a video encoder, the encoder (14) may also include an in-loop filter (46) that filters the fully reconstructed picture, which forms a reference picture for the predictor (44) for the inter-predicted block after filtering.

[0048] As already mentioned above, the encoder (14) operates on a block basis. For further explanation, block bases of interest are one in which the picture (10) is selected from a set or multiple intra-prediction modes that are individually supported by the predictor (44) or the encoder (14), and the selected intra-prediction mode is subdivided into blocks that are performed individually. There may also be other types of blocks in which the picture (10) is subdivided. For example, the determination of whether the picture (10) is inter-coded or intra-coded, as mentioned above, may be performed at the granularity level or at the block level deviating from the block (18). For example, the inter / intra mode determination may be performed at the coding block level in which the picture (10) is subdivided, and each coding block is subdivided into prediction blocks. Prediction blocks having encoding blocks determined to have intra-prediction used are each subdivided for an intra-prediction mode decision. To this end, for each of these prediction blocks, it must be determined whether a supported intra-prediction mode should be used for the individual prediction block. These prediction blocks will form the blocks of interest (18) here. Prediction blocks within a coding block associated with inter-prediction will be processed differently by the predictor (44). They will be inter-predicted from the reference picture by determining the motion vector and copying the prediction signal for this block from the position within the reference picture specified by the motion vector. Other block subdivisions belong to subdivisions into transformation blocks in units where transformations by the transformer (32) and the inverse transformer (40) are performed. The transformed blocks may be, for example, the result of further subdivision of the coding blocks. Naturally, the examples described herein should not be treated as limited, and other examples exist.It should be noted that subdivision into coding blocks may be performed using, for example, multi-tree subdivision only for completeness, and that prediction blocks and / or transformation blocks may also be obtained by further subdividing the coding blocks using multi-tree subdivision.

[0049] A decoder (54) or device for block-wise decoding suitable for the encoder (14) of FIG. 1 is illustrated in FIG. 3. The decoder (54) decodes from the data stream (12) picture (10) in a block-wise manner, in the opposite of the encoder (14), and supports a plurality of intra-prediction modes for this purpose. The decoder (54) may include, for example, a residual provider (156). All other possibilities discussed above in relation to FIG. 1 are also valid for the decoder (54). For this purpose, the decoder (54) may be a still picture decoder or a video decoder, and all prediction modes and prediction possibilities are also supported by the decoder (54). The difference between the encoder (14) and the decoder (54) lies primarily in the fact that the encoder (14) chooses or selects coding decisions based on some optimization, such as minimizing some cost function that may depend, for example, on the coding rate and / or coding distortion. One of these coding options or coding parameters may be involved in the selection of an intra-prediction mode to be used for the current block (18) among available or supported intra-prediction modes. The selected intra-prediction mode may then be signaled by the encoder (14) for the current block (18) within the data stream (12), and the decoder (54) re-executes the selection using this signaling within the data stream (12). In this way, subdividing the picture (10) into blocks (18) may be subject to optimization within the encoder (14), and the corresponding subdivision information may be transmitted to a decoder (54) that restores the subdivision of the picture (10) into blocks (18) based on the subdivision information. The corresponding subdivision information may be transmitted within the data stream (12) and to the decoder (54) within the data stream (12).To summarize the above, the decoder (54) may be a block-based prediction decoder in addition to the intra prediction mode, and the decoder (54) may support other prediction modes, such as the inter prediction mode, for example, if the decoder (54) is a video decoder. In decoding, the decoder (54) may also use the coding order (20) discussed for FIG. 1, and since this coding order (20) is observed in both the encoder (14) and the decoder (54), the same neighboring samples are available for the current block (18) in both the encoder (14) and the decoder (54). Accordingly, to avoid unnecessary repetition, the description of the operation mode of the encoder (14) must also apply to the decoder (54), for example, as far as prediction is concerned with the subdivision of the picture (10) into blocks, for example, as far as prediction is concerned with the coding of the prediction residual. The difference lies in the fact that the encoder (14), by optimization, selects some coding options or coding parameters and signals the coding parameters derived from the data stream (12) into the data stream (12) or inserts them into the data stream (12) so that the decoder (54) re-executes prediction, segmentation, etc.

[0050] FIG. 4 illustrates a possible implementation of the decoder (54) of FIG. 3, that is, one that fits the implementation of the encoder (14) of FIG. 1 as shown in FIG. 2. Since many elements of the encoder (54) are identical to those shown in the corresponding encoder of FIG. 2, the same reference number is used in FIG. 4 to indicate that it is one of these elements, provided with an apostrophe. In particular, the adder (42'), optional in-loop filter (46'), and predictor (44') are connected to the prediction loop in the same way as in the encoder of FIG. 2. The reconstructed, i.e., dequantized and retransformed predicted residual signal applied to the adder (42') is derived by a sequence of entropy decoders (56) that perform the inverse of the entropy encoding of the entropy encoder (28b), followed by a residual signal reconstruction stage (36') consisting of a dequantizer (38') and a detransformer (40'), identical to the case on the encoding side. The output of the decoder is a reconstruction of the picture (10). The reconstruction of the picture (10) may be available directly from the output of the adder (42'), or alternatively, from the output of the in-loop filter (46'). Several post-filters may be placed at the output of the decoder so that the reconstruction of the picture (10) is subject to some post-filtering for picture quality improvement.

[0051] Again, regarding FIG. 4, the descriptions taken above for FIG. 2 are also valid for FIG. 4, except that only the encoder performs optimization work and related decisions regarding coding options. However, all descriptions regarding block-segmentation, prediction, inverse quantization, and re-transformation are also valid for the decoder (54) of FIG. 4.

[0052] 3. ALWIP (Affine Linear Weighted Intra Predictor)

[0053] Although not always necessary for the implementation of the technology discussed here, some non-limiting examples of ALWIP are discussed here.

[0054] This application relates to an improved block-based prediction mode concept for block-unit picture coding, such as that available in video codecs like HEVC or any successor to HEVC. The prediction mode may be an intra-prediction mode, but theoretically, the concept described herein may also be transferred to an inter-prediction mode where a reference sample is part of another picture.

[0055] A block-based prediction concept is pursued that allows for efficient implementation, such as hardware-friendly implementation.

[0056] This objective is achieved by the subject of the independent claim of the present application.

[0057] Intra-prediction mode is widely used in photo and video coding. In video coding, intra-prediction mode competes with other prediction modes, such as motion-compensated prediction mode and inter-prediction mode. In intra-prediction mode, the current block is predicted based on neighboring samples—that is, samples already encoded on the encoder side and already decoded on the decoder side. The neighboring sample values ​​are extrapolated to the current block to form a prediction signal for the current block, along with the prediction residual transmitted in the data stream for the current block. The better the prediction signal, the lower the prediction residual, and consequently, fewer bits are required to code the prediction residual.

[0058] To be effective, several aspects must be considered to form an effective framework for intra prediction in a block-based picture coding environment. For example, the more intra prediction modes supported by a codec, the greater the consumption of side information rates for signaling selection to the decoder. On the other hand, the set of supported intra prediction modes must be able to provide a good prediction signal, that is, a prediction signal that results in low prediction residuals.

[0059] Next, as a comparative example or basic example, an apparatus (encoder or decoder) for decoding a picture from a data stream in blocks is disclosed, and the apparatus supports at least one intra-prediction mode in which, according to the intra-prediction mode, an intra-prediction signal for a block of a predetermined size is determined by applying a first template of a sample adjacent to the current block to an affine linear predictor, which will be called ALWIP (Affine Linear Weighted Intra Predictor) in the sequel.

[0060] The device may have at least one of the following properties (the same may be implemented in a non-transient storage unit that stores instructions that, when executed by a method or other technique, e.g., a processor, cause the processor to implement the method and / or act as a device):

[0061] 3.1 A predictor can be complementary to other predictors.

[0062] Intra-prediction modes that can form the subject of implementation improvements further explained below may be complementary to other intra-prediction modes of the codec. Thus, these are DC-, Plana-, or AN-ANGUL f3.2 one or more proposed prediction modes defined in the HEVC codec resp. JEM reference software.

[0063] The device may include one or more ALWIP modes. Therefore, in the case where the decoder knows that one of the ALWIP modes supported by the device will be used, the decoder needs to parse additional information indicating which of the ALWIP modes supported by the device will be used.

[0064] The signalization of supported modes may have the property that the coding of some ALWIP modes may require fewer bins than other ALWIP modes. Which of these modes requires fewer bins and which requires more bins may depend on information that can be extracted from an already decoded bitstream or fixed in advance.

[0065] 4. Some aspects

[0066] FIG. 2 illustrates a decoder (54) for decoding a picture from a data stream (12). The decoder (54) may be configured to decode a predetermined block (18) of the picture. In particular, the predictor (44) may be configured to map a set of P neighbor samples adjacent to a predetermined block (18) to a set of Q predicted values ​​for samples of a predetermined block using a linear or affine linear transformation [e.g., ALWIP].

[0067] As illustrated in FIG. 5, a predetermined block (18) contains a Q value to be predicted (which will be the "predicted value" at the end of the operation). If it has an untransformed row and N columns, Q = M·N. The Q value of the block (18) may be in a spatial domain (e.g., pixels) or a transformation domain (e.g., DCT, Discrete Wavelet Transform, etc.). The Q value of the block (18) may generally be predicted based on a P value taken from neighboring blocks (17a-17c) adjacent to the block (18). The P values ​​of the neighboring blocks (17a-17c) may be located closest (i.e., adjacent) to the block (18). The P values ​​of the neighboring blocks (17a-17c) have already been processed and predicted. To distinguish between the blocks and portions being transformed (in some examples, 17'b is not used), the value is indicated as a value in the portion (17'a-17'c).

[0068] As illustrated in FIG. 6, to perform a prediction, it is possible to operate with a first vector (17P) having P entries (each entry is associated with a specific location in the neighboring portion (17'a-17'c)), a second vector (18Q) having Q entries (each entry is associated with a specific location in the block (18)), and a mapping matrix (17M) (each row is associated with a specific location in the block (18) and each column is associated with a specific location in the neighboring portion (17'a-17'c)). Thus, the mapping matrix (17M) performs a prediction of the value of P of the neighboring portion (17'a-17'c) to the value of the block (18) according to a predetermined mode. Thus, the entries of the mapping matrix (17M) can be understood as weighting factors. In the following passage, we will refer to neighboring portions of the boundary using 17a-17c instead of 17'a-17'c.

[0069] In this technical field, several existing modes are known, such as DC mode, planar mode, and 65 directional prediction modes. For example, 67 modes may be known.

[0070] However, here, other modes called linear or affine linear transformations may be used. The linear or affine linear transformation includes P·Q weighting factors, of which at least 1 / 4 P·Q weighting factors are non-zero weighting values, and for each Q prediction value, a series of P weighting factors associated with individual prediction values. This series forms an envelope that is omnidirectly nonlinear when arranged under another according to the raster scan order among samples of a predetermined block.

[0071] It is possible to map the P position of the neighbor value (17'a-17'c) (template), the Q position of the neighbor sample (17'a-17'c), and the value of the P·Q weighting factor of the matrix (17M). The plane is an example of an envelope of a series for DC transformation (plane for DC transformation). Since the envelope is obviously planar, it is excluded by the definition of a linear or affine linear transformation (ALWIP). Another example is a matrix that causes the emulation of an angular mode: the envelope is excluded from the ALWIP definition and, frankly speaking, can look like a hill running diagonally from top to bottom along the direction of the P / Q plane. The planar mode and the 65 directional prediction mode will have different envelopes that may be linear in at least one direction, namely all directions for the exemplified DC, and hill directions for the angular mode, for example.

[0072] Conversely, the envelope of a linear or affine transformation is not omnidirectionally linear. It is understood that this type of transformation may be optimal for making predictions for blocks (18) in some situations. It is noted that it is desirable for at least 1 / 4 of the weighting factors to be different from 0 (i.e., at least 25% of the P*Q weighting factors to be different from 0).

[0073] Weighting factors may not be related to each other according to general mapping rules. Therefore, the matrix (17M) may not have an obvious recognizable relationship between the values ​​of its entries. For example, weighting factors cannot be explained by analytical or differential functions.

[0074] In the example, the ALWIP transformation may result in a mean of maxima of the cross-correlation between a first series of weighting factors associated with individual predicted values ​​and a second series of weighting factors associated with predicted values ​​other than individual predicted values, or the inverse version of the latter series, which may be lower than a predetermined threshold (e.g., 0.2 or 0.3 or 0.35 or 0.1, for example, a threshold in the range between 0.05 and 0.035), which leads to a higher maximum value. For example, for each pair (i1, i2) of rows in the ALWIP matrix (17M), the cross-correlation can be calculated by multiplying the P-value of the i1th row by the P-value of the i2th row. For each obtained cross-correlation, a maximum value can be obtained. Thus, an average for the entire matrix (17M) can be obtained (i.e., the maximum values ​​of the cross-correlations across all combinations are averaged). After that, the threshold value may be, for example, 0.2 or 0.3 or 0.35 or 0.1, or a threshold value in the range of, for example, 0.05 to 0.035.

[0075] P neighbor samples of blocks (17a-17c) may be located along a 1-dimensional path extending along the border (e.g., 18c, 18a) of a predetermined block (18). For each of Q predicted values ​​of the predetermined block (18), a series of P weighted factors associated with the individual predicted value may be ordered in a manner traversing the 1-dimensional path in a predetermined direction (e.g., from left to right, from top to bottom, etc.).

[0076] In the example, the ALWIP matrix (17M) may be non-diagonal or non-block diagonal.

[0077] An example of an ALWIP matrix (17M) for predicting a 4x4 block (18) from four already predicted neighbor samples is:

[0078] {

[0079] { 37, 59, 77, 28},

[0080] { 32, 92, 85, 25},

[0081] { 31, 69, 100, 24},

[0082] { 33, 36, 106, 29},

[0083] { 24, 49, 104, 48},

[0084] { 24, 21, 94, 59},

[0085] { 29, 0, 80, 72},

[0086] { 35, 2, 66, 84},

[0087] { 32, 13, 35, 99},

[0088] { 39, 11, 34, 103},

[0089] { 45, 21, 34, 106},

[0090] { 51, 24, 40, 105},

[0091] { 50, 28, 43, 101},

[0092] { 56, 32, 49, 101},

[0093] { 61, 31, 53, 102},

[0094] { 61, 32, 54, 100}

[0095] }

[0096] It could be.

[0097] (Here, { 37, 59, 77, 28} is the first row, { 32, 92, 85, 25} is the second row, and { 61, 32, 54, 100} is the 16th row of matrix (17M).) Matrix (17M) has dimensions of 16x4 and contains 64 weighted factors (resulting in 16*4=64). This is because the matrix (17M) has QxP dimensions, where Q=M*N is the number of samples in the block (18) to be predicted (block (18) is a 4x4 block), and P is the number of samples in the already predicted sample. Here, M=4, N=4, Q=16 (result of M*N=4*4=16), and P=4. The matrix is ​​non-diagonal and non-block diagonal and is not explained by any specific rule.

[0098] As can be seen, less than 1 / 4 of the weighting factors are 0 (in the case of the matrix shown above, one of the 64 weighting factors is 0). The envelope formed by these values ​​forms a forward-non-linear envelope when arranged below the other according to the raster scan order.

[0099] Even though the above description is discussed primarily with reference to a decoder (e.g., decoder (54)), the same content can be performed in an encoder (e.g., encoder (14)).

[0100] In some examples, for each block size (in a set of block sizes), the ALWIP transforms of the intra-prediction modes within the second set of intra-prediction modes for the individual block sizes are different. Additionally, or alternatively, the cardinality of the second set of intra-prediction modes for the block sizes may match, but the associated linear or affine linear transforms of the intra-prediction modes within the second set of intra-prediction modes for different block sizes may be made non-transferable for each by scaling.

[0101] In some examples, the ALWIP transformation can be defined as having "nothing in common" with existing transformations (e.g., the ALWIP transformation may have "nothing in common" with the corresponding existing transformation even if it is mapped through one of the mappings above).

[0102] In one example, ALWIP mode is used for both the Luma and Chroma components, but in another example, ALWIP mode is used for the Luma component but not for the Chroma component.

[0103] 5. Affine linear weighted intra prediction mode with encoder speedup (e.g., test CE3-1.2.1)

[0104] 5.1 Description of Method or Device

[0105] The ALWIP (Affine Linear Weighted Intra Prediction) mode tested in CE3-1.2.1 may be identical to that proposed in JVET-L0199 in CE3-2.2.2, except for the following changes.

[0106] ● MRL (multiple reference line) intra-prediction, particularly encoder estimation and signaling harmony, i.e., MRL, is not combined with ALWIP, and MRL index transmission is limited to non-ALWIP blocks.

[0107] ● Subsampling is now mandatory for all blocks W×H ≥ 32×32 (previously optional for 32×32). Therefore, additional testing and subsampling flag transmission in the encoder have been removed.

[0108] ● ALWIP for 64×N and N×64 blocks (N ≤ 32) was added by downsampling to 32×N and N×32, respectively, and applying the corresponding ALWIP mode.

[0109] In addition, Test CE3-1.2.1 includes the following encoder optimizations for ALWIP.

[0110] ● Combined Mode Estimation: The existing and ALWIP modes use a shared Hadamard candidate list for the total RD estimation. That is, ALWIP mode candidates are added to the same list as the existing (and MRL) mode candidates based on Hadamard costs.

[0111] ● EMT Intra Fast and PB Intra Fast are supported for the combined mode list, along with additional optimizations to reduce the total number of RD checks.

[0112] ● Following the same approach as in existing modes, only the MPMs of the left and top blocks available for full RD estimation for ALWIP are added to the list.

[0113] 5.2 Complexity Assessment

[0114] In test CE3-1.2.1, excluding the calculation that calls the discrete cosine transform, up to 12 multiplications per sample were required to generate the prediction signal. Additionally, a total of 136,492 parameters, each 16 bits, were required. This corresponds to 0.273 megabytes of memory.

[0115] 5.3 Experimental Results

[0116] Test evaluations were performed according to the common test conditions JVET-J1010[2] for intra-only (AI) and random access (RA) configurations of VTM software version 3.0.1. Corresponding simulations were performed on an Intel Xeon cluster (E5-2697A v4, AVX2 on, Turbo Boost off) with Linux OS and GCC 7.2.1 compiler installed.

[0117] Table 1. Results of CE3-1.2.1 for VTM AI configuration

[0118]

[0119] Table 2. Results of CE3-1.2.1 for VTM RA configuration

[0120]

[0121] 5.4 Affine Linear Weighted Intra Prediction with Reduced Complexity (e.g., test CE3-1.2.2)

[0122] The technology tested in CE2 is related to "Affine Linear Intra Predictions" described in JVET-L0199[1], but simplifies it in terms of memory requirements and computational complexity.

[0123] ● There may only be three different sets of prediction matrices covering all block shapes (e.g., S0, S1, S2, see below) and bias vectors (e.g., for providing offset values). As a result, the number of parameters is reduced to 14,400 10-bit values, which is less memory than that stored in 128×128 CTUs.

[0124] ● The input and output sizes of the predictor are further reduced. Additionally, instead of transforming the boundary through the DCT, averaging or downsampling is performed on the boundary samples, and linear interpolation can be used instead of the inverse DCT to generate the prediction signal. Consequently, up to 4 multiplications per sample may be required to generate the prediction signal.

[0125] 6. Yes

[0126] Here, we describe how to perform some predictions (e.g., shown in Figure 6) using ALWIP predictions.

[0127] In principle, referring to FIG. 6, to obtain the Q=M*N value of the MxN block (18) to be predicted, the multiplication of the Q*P sample of the QxP ALWIP prediction matrix (17M) and the Px1 neighbor vector (17P) must be performed. Thus, generally, to obtain each Q=M*N value of the MxN block (18) to be predicted, at least the multiplication of the P=M+N value is required.

[0128] These multiplications have an extremely undesirable effect. The dimension (P) of the boundary vector (17P) generally depends on the number of boundary samples (bins or pixels) (17a, 17c) (17a, 17c) adjacent (e.g., adjacent) to the MxN block (18) to be predicted. This means that if the size of the block (18) to be predicted is large, the number of boundary pixels (17a, 17c) M+N is correspondingly large, and thus the dimension P=M+N of the Px1 boundary vector (17P), the length of each row of the QxP ALWIP prediction matrix (17M), and the number of multiplications required accordingly (generally Q=M*N=W*H, where W (width) is N and H (height) is another sign of M, and if the boundary vector consists of only one row and / or one column of samples, P is P=M+N=H+W)) increase.

[0129] This problem is generally exacerbated by the fact that multiplication in microprocessor-based systems (or other digital processing systems) is typically a power-consuming operation. One can imagine that a large number of multiplications performed on an extremely large number of samples across a large number of blocks leads to a generally unwanted waste of computational power.

[0130] Therefore, it would be desirable to reduce the number of odds Q*P required to predict MxN blocks (18).

[0131] It was understood that by intelligently selecting an operation that is easier to process and serves as an alternative to the odds, it is possible to somehow reduce the computational power required for each intra prediction of each block (18) to be predicted.

[0132] In particular, with reference to FIGS. 7a through 7d, it has been understood that an encoder or decoder can predict a predetermined block (e.g., 18) of a picture using a plurality of neighbor samples (e.g., 17a, 17c) as follows.

[0133] Reduce multiple neighbor samples (e.g., 17a, 17c) (e.g., by averaging or downsampling) to obtain a reduced set of sample values ​​with lower sample numbers compared to multiple neighbor samples (e.g., in step 811).

[0134] To obtain predicted values ​​for a predetermined sample of a predetermined block, apply a linear or affine linear transformation to a reduced set of sample values ​​(e.g., in step 812).

[0135] In some cases, the decoder or encoder may derive a predicted value for additional samples of a predetermined block based on the predicted values ​​for a predetermined sample and multiple neighboring samples, for example, by interpolation. Thus, an upsampling strategy can be obtained.

[0136] In the example, it is possible to perform some averages on the samples of the boundary (17) to arrive at a reduced set (102) (Figs. 7a to 7d) of samples having a reduced number of samples (at least one of the samples having a reduced number of samples (102) may be the average of two samples of the original boundary samples or a selection of the original boundary samples). For example, if the original boundary has P=M+N samples, the reduced set of samples is Mred <M 및 N red <N 중 적어도 하나와 함께 P red =M red +N red Can have, and P red <P가 될 수 있다. 따라서, 예측을 위해 실제로 사용될 바운더리 벡터(17P)(예를 들어, 단계 812b에서)는 P×1 엔트리를 갖지 않고 P red <P를 갖는 P red It will have a ×1 entry. Similarly, the ALWIP prediction matrix (17M) selected for prediction will not have a Q×P dimension and will have a reduced number of elements of the matrix. red (or Q red ×P red , see below) has dimensions, which is at least P red <P(M red <M 및 N red d <N 중 적어도 어느 하나에 의해)이기 때문이다.

[0137] In some examples (e.g., FIG. 7b, 7c), the block obtained by ALWIP (in step 812) is M' red <M 및 / 또는 N' red <N (즉, ALWIP에 의해 직접 예측된 샘플은 실제로 예측될 블록(18)의 샘플보다 수가 적음)인, M' red ×N' red In the case of a reduced block of size, it is possible to further reduce the number of multiplications. Therefore, Qred=M' red ×N' red If set to this, this is Q×P red Instead of odds, Q red ×P red Odds (Q red ×P red < Q×P red <Q×P)을 이용하여 ALWIP 예측을 얻도록 한다. 이 승산은 디멘전 M' red ×N' red A reduced block will be predicted. Nevertheless, the reduced M' red×N' red It will be possible to perform upsampling from the prediction block to the final M×N prediction block (e.g., obtained by interpolation) (e.g., in the subsequent step 813).

[0138] This technique is a matrix odds reduced number of odds (Q red *P red or Q*P red It may be advantageous that both the initial reduction (e.g., averaging or downsampling) and the final transformation (e.g., interpolation) are performed by reducing (or even avoiding) the multiplication. For example, downsampling, averaging, and / or interpolation can be performed by adopting non-computational-power-demanding binary operations such as adding and shifting (e.g., in steps 811 and / or 813).

[0139] In addition, addition is a very easy task that can be performed easily without much computational effort.

[0140] This shifting operation can be used, for example, to average two boundary samples and / or to interpolate two samples (supporting values) of the reduced prediction block (or taken from the boundary) to obtain the final prediction block. (Two sample values ​​are required for interpolation. Within the block, we always have two predetermined values, but since there is only one predetermined value as in Fig. 7b to interpolate samples along the left and top boundaries of the block, we use one boundary sample as a supporting value for interpolation.)

[0141] The following two-step procedure can be used.

[0142] First, sum the values ​​of the two samples;

[0143] Then, divide the sum in half (e.g., right shift).

[0144] Alternatively, the following can be done:

[0145] Divide each sample in half (e.g., shift left);

[0146] Then, sum the values ​​of the samples divided into two halves.

[0147] When downsampling (e.g., at step 811), a much easier operation can be performed because it is only necessary to select the amount of one sample from a sample group (e.g., adjacent samples).

[0148] Therefore, it is now possible to define techniques for reducing the number of odds to be taken. Some of these techniques may be based specifically on one or more of the following principles:

[0149] Even in the case where the size of the predicted block (18) is actually MxN, the block can be reduced (in at least one of the two dimensions) and the reduced size Q red xP red ALWIP matrix (M' having red <M 및 / 또는 N' red <N 및 / 또는 M red <M 및 / 또는 N red <N 인 Q red =M' red *N' red , P red =N red +M red (In) can be applied. Therefore, the boundary vector (17P) has a magnitude P red It will have ×1, and this is P red <P 승산(P red = M red +N red and only implies (imply) having P=M+N.

[0150] P red ×1 The boundary vector (17P) can be easily obtained from the original boundary (17). Example:

[0151] Downsampling (e.g., selecting only some samples of the boundary); and / or

[0152] Averaging multiple samples of the boundary (without multiplication, easily obtained through addition and shifting).

[0153] Additionally or alternatively, instead of predicting by multiplying all Q=M*N values ​​of the block (18) to be predicted, it is possible to predict only the reduced block with reduced dimensions (e.g., M' red <M 및 / 또는 N' red <N을 갖는 Q red =M' red *N' red ). The remaining samples of the block (18) to be predicted are interpolated, e.g., the remaining QQ to be predicted red Q as a supporting value for the value red It will be obtained by the method of using the sample.

[0154] According to the example illustrated in FIG. 7a, a 4x4 block (18) (M=4, N=4, Q=M*N=16) is predicted, and the neighbors (17) (vertical column with 4 already predicted samples) and 17c (horizontal row with 4 already predicted samples) of sample (17a) have already been predicted in a previous iteration (neighbors (17a and 17c) can be collectively denoted as 17). A priori, by using the equation illustrated in FIG. 5, the prediction matrix (17M) must be a QxP=16x8 matrix (by Q=M*N=4*4 and P=M+N=4+4=8), and the boundary vector (17P) must have an 8x1 dimension (by P=8). However, this would necessitate performing 8 multiplications for each of the 16 samples of the predicted 4x4 block (18), thus requiring a total of 16*8=128 multiplications (it should be noted that the average number of multiplications per sample is a good assessment tool for evaluating computational complexity. In the case of conventional intra-prediction, 4 multiplications per sample are required, which increases the associated computational effort. Therefore, this can be used as an upper limit criterion for ALWIP to ensure that the complexity is reasonable and does not exceed the complexity of conventional intra-prediction).

[0155] Nevertheless, by using the present technique, the number of neighboring samples (17a, 17c) to be predicted in step (811) in block (18) is P to P red<P로 감소시키는 것이 가능함을 이해하였다. 특히, 2개의 가로 행과 2개의 세로 열을 가짐으로써 2x2 블록인 블록(18)으로서 동작(감소된 바운더리는 평균값에 의해 형성됨)하는 감소된 바운더리(102)를 얻도록, 서로 인접한(adjacent to) 바운더리 샘플(17a, 17c)을 평균화(예: 도 7a의 100)하는게 가능하다는 것이 이해되었다. 대안적으로, 다운샘플링을 수행하여 행(17c)에 대해 2개의 샘플을 선택하고 열(17a)에 대해 2개의 샘플을 선택하는 것이 가능하다. 따라서, 가로 행(17c)은, 4개의 원 샘플 대신에 닌 2개의 샘플(예: 평균 샘플)을 가지는 것으로 처리되고, 세로 열(17a)은 4개의 원 샘플 대신에 2개의 샘플(예: 평균 샘플)을 가지는 것으로 처리된다. 행(17c)과 열(17a)을 각각 2개의 샘플의 그룹(110)으로 세분화(subdivide)한 후, 하나의 단일 샘플이 유지(예: 그룹(110)의 샘플의 평균 또는 그룹(110)의 샘플 중에서 단순 선택)되는 것을 이해하는 것도 가능하다. 따라서, 샘플 값의, 소위 감소된 세트(102)는 단지 4개의 샘플(M red =2, N red =2, P red =M red +N red =4, P red <P)을 갖는 세트(102)에 의해 얻어진다.

[0156] It has been understood that it is possible to perform operations (e.g., averaging or downsampling (100)) without performing too many multiplications at the processor level: the averaging or downsampling (100) performed in step (811) can be achieved by straightforward and computationally non-power-consuming operations such as addition and shifting.

[0157] At this point, it is understood that a reduced set of sample values ​​(102) can be applied to a linear or affine linear (ALWIP) transformation (19) (e.g., using a prediction matrix such as the matrix (17M) of FIG. 5). In this case, the ALWIP transformation (19) directly maps four samples (102) to the sample values ​​(104) of block (18). In this case, interpolation is unnecessary.

[0158] In this case, the ALWIP matrix (17M) is dimension QxP red =16x4: This follows the fact that all Q=16 samples of the block (18) to be predicted are obtained directly by ALWIP multiplication (no interpolation needed).

[0159] Therefore, in step (812a), dimension QxP red A suitable ALWIP matrix (17M) having is selected. The selection may be based, for example, at least partially on signaling from the data stream (12). The selected ALWIP matrix (17M) is also A k It can be represented as, where k can be understood as an index that can be signaled in the data stream (12) (in some cases, the matrix It is also indicated as (see below). The selection can be performed according to the following scheme. For each dimension (e.g., height / width pair of block (18) to be predicted), the ALWIP matrix (17M) is selected from one of three sets, for example, matrices S0, S1, S2 (each of the three sets S0, S1, S2 can be grouped into multiple ALWIP matrices (17M) of the same dimension, and any one of them will be the ALWIP matrix to be selected for prediction).

[0160] In step (812b), the selected QxP red ALWIP Matrix (17M)(A k Also indicated as) and P redMultiplication between x1 boundary vectors (17P) is performed.

[0161] In step (812c), the offset value (e.g., b k ) can be added to all obtained values ​​(104) of the vector (18Q) obtained, for example, by ALWIP. Offset value (b k Or in some cases Indicated by (see below) a specific selected ALWIP matrix (A k It can be associated with ) and based on an index (e.g., this can be signaled in the data stream (12)).

[0162] Therefore, the comparison between using this technology and not using this technology is resumed here:

[0163] Without current technology:

[0164] Predicted block (18), block having dimensions M=4, N=4;

[0165] Q=M*N=4*4=16 values ​​are predicted.

[0166] P=M+N=4+4=8 Boundary Sample

[0167] P=8 multiplier for each predicted Q=16 value,

[0168] Total P*Q = 8*16 = 128 odds

[0169] With current technology, we:

[0170] Predicted block (18), block having dimensions M=4, N=4;

[0171] Q=M*N=4*4=16 values ​​are predicted at the end.

[0172] Reduced dimension of the boundary vector: P red =M red +N red =2+2=4;

[0173] Pred=4 multiplier for each of the Q=16 values ​​predicted by ALWIP,

[0174] Total Pred Q=4*16=64 odds (half of 128!)

[0175] The ratio between the number of odds and the number of final values ​​to be obtained is P red Q / Q=4, meaning it is half the odds of P=8 for each sample to be predicted!

[0176] As can be understood, it is possible to obtain an appropriate value in step (812) by relying on simple and computationally non-power-requiring operations such as averaging (and, depending on the case, addition and / or shift and / or downsampling).

[0177] Referring to FIG. 7b, the block to be predicted (18) is an 8x8 block of 64 samples (M=8, N=8). Here, a priori, the prediction matrix (17M) must have a size QxP=64x16 (Q=64 because Q=M*N=8*8=64, and M=8 and N=8 because P=M+N=8+8=16). Therefore, a priori, a multiplication of P=16 is required for each of the Q=64 samples of the 8x8 block to be predicted (18) to arrive at a multiplication of 64*16=1024 for the entire 8x8 block (18)!

[0178] However, as can be seen in FIG. 7b, instead of using all 16 samples of the boundary, only 8 values ​​are used (e.g., 4 from the horizontal boundary row (17c) and 4 from the vertical boundary column (17a) between the original samples of the boundary). From the boundary row (17c), 4 samples may be used instead of 8 (e.g., a 2x2 mean and / or a selection of one of 2 samples). Therefore, the boundary vector is not a Px1=16x1 vector, but P red x1=8x1 vector(P red =M red +N red =4+4) only. Instead of the original P=16 samples, only P forms a reduced set (102) of sample values. redIt was understood that it is possible to select or average samples of the horizontal row (17c) and vertical column (17a) to have only boundary values ​​of =8. This reduced set (102) will allow obtaining a reduced version of the block (18), and the reduced version is Q red =M red *N red It has 4*4=16 samples (instead of Q=M*N=8*8=64). The size is M red xN red It is possible to apply an ALWIP matrix to predict a block of =4x4. A reduced version of the block (18) includes samples marked in gray in the scheme (106) of FIG. 7b: the samples marked with gray squares (including samples (118' and 118")) are Q obtained in the application step (812). red A 4x4 reduced block having a value of =16 is formed. The 4x4 reduced block is obtained by applying a linear transformation (19) in the application step (812). After obtaining the value of the 4x4 reduced block, it is possible to obtain the value of the remaining sample (the sample indicated as a white sample in method (106)), for example, by interpolation.

[0179] In relation to the method (810) of FIG. 7, this method (820) is, for example, the remaining QQ of the MxN=8x8 block (18) to be predicted by interpolation. red =64-16=48 may additionally include a step (813) of deriving a predicted value for a sample (white square). The remaining QQ red =64-16=48 The sample is directly obtained Q red=16 It can be obtained by interpolation from samples (interpolation can also use, for example, the values ​​of boundary samples). As can be seen in FIG. 7b, samples (118' and 118") are obtained in step (812) (indicated by gray squares), while sample 108' (intermediate between samples 118' and 118") is obtained by interpolation between samples 118' and 118") in step (813). It is understood that interpolation can also be obtained by operations similar to those for averaging, such as shifting and addition. Thus, in FIG. 7b, the value (108') can generally be determined as the intermediate value (which may be the average) between the value of sample (118') and the value of sample (118").

[0180] By performing interpolation, it is also possible to reach the final version of the MxN=8x8 block (18) based on the multiple sample values ​​shown in 104 in step (813).

[0181] Therefore, the comparison between using current technology and not using it is:

[0182] Without current technology:

[0183] Predicted block (18) having dimensions M=8, N=8;

[0184] P=M+N=8+8=16 samples of the boundary (17);

[0185] P=16 odds for each of the predicted Q=64 values,

[0186] The total number of probabilities of P*Q = 16*64 = 1028

[0187] The ratio between the odds and the number of final values ​​obtained is P*Q / Q=16

[0188] With current technology:

[0189] Predicted block (18) having dimensions M=8, N=8

[0190] The predicted Q=M*N=8*8=64 value at the end;

[0191] However, P red =M red +N red , Q red =M red *N red , M red =4, N red = 4 people Q red xP red The ALWIP matrix can be used.

[0192] P red <P인, P red =M red +N red Sample of the boundary of =4+4=8

[0193] Predicted Q of a 4x4 reduced block red =16 P for each value red =8 multiplication (formed as a gray square in method (106)),

[0194] Total P red *Q red =8*16=128 Total odds (much less than 1024!)

[0195] The ratio between the number of multiplications and the number of final values ​​to be obtained is P red *Q red / Q=128 / 64=2 (much smaller than 16 obtained without current technology!).

[0196] Therefore, the technology presented here requires 8 times less power than previous technology.

[0197] FIG. 7c illustrates another example (which may be based on method (820)) where the block to be predicted (18) is a rectangular 4x8 block (M=8, N=4) having Q=4*8=32 samples to be predicted. The boundary (17) is formed by a horizontal row (17c) having N=8 samples and a vertical column (17a) having M=4 samples. Thus, a priori, the boundary vector (17P) has dimensions Px1=12x1, whereas the predicted ALWIP matrix must be a QxP=32x12 matrix, which gives rise to a requirement for Q*P=32*12=384 multiplication.

[0198] However, it is possible to average or downsample at least 8 samples of the horizontal row (17c) to obtain, for example, a reduced horizontal row having only 4 samples (e.g., averaged samples). In some examples, the vertical column (17a) will remain unchanged (e.g., without averaging). The overall reduced boundary is P red <P를 갖는 디멘전 P red =8. Therefore, the boundary vector (17P) is dimension P red x1 will have 8x1. The ALWIP prediction matrix (17M) is a dimension M*N red *P red It will be a matrix having =4*4*8=64. The 4x4 reduced block obtained directly in the application step (812) (formed by the gray column of method (107)) is of size Q red =M*N red =4*4=16 samples (Q=4*8=32 instead of the original 4x8 block (18) to predict). When the reduced 4x4 block is acquired by ALWIP, the offset value b k The values ​​can be added (step (812c)) and interpolated in step (813). As can be seen in step (813) of FIG. 7c, the reduced 4x4 block is expanded into a 4x8 block (18), and the value (108') not obtained in step (812) is obtained in step (813) by interpolating the value (118' and 118") (gray square) obtained in step (812).

[0199] Therefore, the comparison between using current technology and not using it is:

[0200] Without current technology:

[0201] Block to be predicted (18) with dimensions M=4, N=8

[0202] Predicted Q=M*N=4*8=32 values;

[0203] Boundary P=M+N=4+8=12 samples;

[0204] P=12 odds for each predicted Q=32 value,

[0205] Total odds of P*Q = 12*32 = 384

[0206] The ratio between the odds and the number of final values ​​obtained is P*Q / Q=12

[0207] With current technology:

[0208] Block to be predicted (18) with dimensions M=4, N=8

[0209] Q=M*N=4*8=32 values ​​predicted at the end

[0210] However, M=4, N red =4, Q red =M*N red =16, P red =M+N red Q = 4+4=8 red xP red The ALWIP matrix can be used.

[0211] P red <P인, P red =M+N red Sample of the boundary of =4+4=8

[0212] Predicted reduced block Q red =16 P for each value red =8 odds,

[0213] Total Q red *P red *=16*8=128 Total odds (less than 384!)

[0214] The ratio between the number of odds and the number of final values ​​to be obtained is P red *Q red / Q=128 / 32=4 (much smaller than 12 obtained without current technology!).

[0215] Therefore, using current technology reduces computational effort by one-third.

[0216] FIG. 7d illustrates a case of a predicted block (18) having a value Q=M*N=16*16=256 predicted at the interval, with MxN=16x16 dimensions and P=M+N=16+16=32 boundary samples. This derives a prediction matrix having dimensions QxP=256x32, which implies 256*32=8192 odds!

[0217] However, by applying method (820), in step (811), it is possible to reduce the number of boundary samples (e.g. by averaging or downsampling) from, for example, 32 to 8: for example, for every group (120) of four consecutive samples in row (17a), one single sample (e.g., selected from four samples or the average of the samples) remains. Also, for every group of four consecutive samples in column (17c), one single sample (e.g., selected from four samples or the average of the samples) remains.

[0218] Here, the ALWIP matrix (17M) is Q red xP red = is a 64x8 matrix: This is P red From the fact that =8 (using the average of 8 samples or selected samples out of 32 samples of the boundary) was selected, the reduced block to be predicted in step (812) is an 8x8 block (the gray square in method (109) is 64).

[0219] Accordingly, when 64 samples of the reduced 8x8 block are obtained in step (812), the remaining QQ of the block (18) to be predicted in step (813) red =256-64=192 It is possible to predict the value (104).

[0220] In this case, to perform interpolation, it was chosen to use only all samples in the boundary column (17a) and replacement samples in the boundary row (17c). Other choices may be made.

[0221] The ratio between the odds of winning and the final values ​​obtained using the current method is Q red *P red / Q=8*64 / 256=2, which is much less than the 32 multiplications for each value without this technique!

[0222] The comparison between using current technology and not using it is:

[0223] Without current technology:

[0224] Predicted block (18) having dimensions M=16, N=16

[0225] Predicted Q=M*N=16*16=256 values;

[0226] P=M+N=16*16=32 samples at the boundary;

[0227] P=32 odds for each predicted Q=256 value,

[0228] Total odds of P*Q = 32*256 = 8192;

[0229] The ratio between the odds and the number of final values ​​obtained is P*Q / Q=32

[0230] With current technology:

[0231] Predicted block (18) having dimensions M=16, N=16

[0232] The predicted value of Q=M*N=16*16=256 at the end;

[0233] However, M red =4, N red =4, Q predicted by ALWIP red =8*8=64 samples, P red =M red +N red Q = 4+4=8 red xP red A 64x8 ALWIP matrix is ​​used,

[0234] P red <P인 바운더리에서 P red =M red +N red =4+4=8 samples

[0235] Predicted reduced block Q red =64 P for each value red =8 odds,

[0236] Total Q red *P red =64*4=256 Odds of multiplication (less than 8192!)

[0237] The ratio between the number of odds and the number of final values ​​to be obtained is P red *Q red / Q=8*64 / 256=2(Much smaller than 32 obtained without current technology!).

[0238] Therefore, the computational power required by this technology is 16 times less than that of existing technology!

[0239] Therefore, it is possible to predict a specific block (18) of a picture using multiple neighbor samples (17).

[0240] A step (100, 813) of reducing a plurality of neighbor samples to obtain a reduced set (102) of sample values ​​that is lower in sample number compared to a plurality of neighbor samples (17),

[0241] It includes the step (812) of applying a reduced set of sample values ​​(102) to a linear or affine linear transformation (19, 17M) to obtain predicted values ​​for a predetermined sample (104, 118', 188") of a predetermined block (18).

[0242] In particular, it is possible to perform a reduction (100, 813) by downsampling multiple neighbor samples to obtain a reduced set (102) of sample values ​​with fewer samples than the number of neighbor samples (17).

[0243] Alternatively, it is possible to perform a reduction (100, 813) by averaging multiple neighbor samples to obtain a reduced set (102) of sample values ​​with fewer samples than the number of neighbor samples (17).

[0244] Additionally, by interpolation, it is possible to derive (813) a predicted value for an additional sample (108, 108') of a predetermined block (18) based on the predicted values ​​of a predetermined sample (104, 118', 118) and a plurality of neighbor samples (17).

[0245] Multiple neighboring samples (17a, 17c) may be extended one-dimensionally along two sides of a predetermined block (18) (e.g., toward the right and downward in FIG. 7a to 7d). The predetermined samples (e.g., obtained by ALWIP in step (812)) may be arranged in rows and columns, and along at least one of the rows and columns, the predetermined samples may be located at every n-th position from the predetermined samples (112) adjacent to the two sides of the predetermined block (18).

[0246] It is also possible to determine a supporting value (118) for one of a plurality of neighboring positions (118) aligned for each of at least one of a row and a column, based on a plurality of neighboring samples (17). It is also possible to derive a predicted value (118) for additional samples (108, 108') of a predetermined block (18) by interpolation, based on a predetermined sample (104, 118', 118") and a supporting value for a neighboring sample (118) aligned for at least one of a row and a column.

[0247] A predetermined sample (104) may be positioned every nth from a sample (112) adjacent to two sides of a predetermined block (18) along a row, and a predetermined sample may be positioned every mth from a sample (112) of a predetermined sample adjacent to two sides of a predetermined block (18) along a column, where n, m > 1. In some cases, n = m (e.g., in FIG. 7b and 7c, samples are obtained directly by ALWIP in 812, and samples (104, 118', 118") indicated by gray squares are alternated with samples (108, 108') subsequently obtained in step (813) along rows and columns.

[0248] It is possible to perform the operation of determining a supporting value by downsampling or averaging a group of neighbor samples (120) within a plurality of neighbor samples, including neighbor samples (119) for determining an individual supporting value, along at least one of row (17c) and column (17a), for example, for each supporting value. In step (813) of FIG. 7d, the value of sample (119) can be obtained by using the value of a predetermined sample (118', 118") (previously obtained in step (812)) and the neighbor sample (118) as supporting values.

[0249] Multiple neighbor samples may be extended one-dimensionally along both sides of a predetermined block (18). A reduction (811) may be performed by grouping multiple neighbor samples (17) into one or more consecutive groups of neighbor samples (110) and performing downsampling or averaging on each of the one or more groups of neighbor samples (110) having two or more neighbor samples.

[0250] In the example, the linear or affine linear transformation is P red *Q red or P red *Q may include a weighting factor, and Pred is the number of sample values ​​(102) in the reduced set of sample values, and Q red Or Q is the number of predetermined samples within a predetermined block (18). At least 1 / 4 P red *Q red or 1 / 4 P red *Q weighting factors are non-zero weights (weighting values). P red *Q red or P red The Q-weighted factor is Q or Q red For each predetermined sample, P associated with the individual predetermined sample red It may include a series of weighting factors, wherein the series forms an omnidirectionally non-linear envelope when one of the predetermined samples of a predetermined block (18) is arranged below another according to the raster scan order. P red *Q or P red *Q red Weighting factors may be unrelated to one another through general mapping rules. The mean of the maximum cross-correlation between the series of first weighting factors associated with each predetermined sample and the series of second weighting factors associated with a predetermined sample other than an individual predetermined sample, or the reversed version of the latter series, or whatever leads to a higher maximum, is lower than a predetermined threshold. The predetermined threshold may be 0.3 [or 0.2 or 0.1 in some cases]. P red Neighbor samples (17) can be located along a one-dimensional path extending along two sides of a predetermined block (18), and Q or Q red For each predetermined sample, P associated with the predetermined individual sample red A series of weighted factors can be ordered in a manner that crosses a one-dimensional path in a predetermined direction.

[0251] 6.1 Description of Method and Device

[0252] To predict samples of a square block of width W (also denoted as N) and height H (also denoted as M), Affine-Leading Weighted Intra Prediction (ALWIP) may take as input one line of reconstructed neighbor boundary samples H to the left of the block and one line of reconstructed neighbor boundary samples W above the block. If reconstructed samples are not available, they may be generated in the same way as performed in conventional intra prediction.

[0253] The generation of a prediction signal (e.g., the value of a complete block (18)) may be based on at least some of the following three steps:

[0254] 1. Among the boundary samples (17), samples (102) (e.g., 4 samples in the case of W=H=4 and / or 8 samples in other cases) can be extracted by averaging or downsampling (e.g., step (811).

[0255] 2. Matrix vector multiplication followed by the addition of offsets can be performed using the average sample (or the sample remaining from downsampling) as input. The result may be a reduced prediction signal for a set of samples subsampled from the original block (e.g., step (812)).

[0256] 3. In the remaining locations, the prediction signal can be generated from a set of prediction signals subsampled by, for example, linear interpolation (e.g., step (813)), by, for example, upsampling.

[0257] Thanks to steps 1.(811) and / or 3.(813), the total number of multiplications required for the calculation of the matrix-vector product can always be less than or equal to 4*W*H. Additionally, the averaging operation on the boundary and the linear interpolation of the reduced prediction signal are performed using only addition and bit shifting. That is, in the example, ALWIP mode requires a maximum of 4 multiplications per sample.

[0258] In some examples, the matrix (e.g., 17M) and offset vector (e.g., b) required to generate the prediction signal k ) can be taken from S0, S1, S2 of a matrix stored in a set (e.g., 3 sets), for example, the storage unit(s) of a decoder and an encoder.

[0259] In some examples, set S0 is a matrix n0 (e.g., n0=16 or n0=18 or other number) that may have 16 rows and 4 columns, respectively, to perform the technique according to FIG. 7a. Wow, 18 offset vectors, each with a size of 16 It may include (e.g., may be configured). The matrix and offset vector of this set are used in a block (18) of size 4×4. The boundary vector is P red When reduced to a =4 vector (as in step (811) of FIG. 7a), P of the reduced set of samples (102) red It is possible to map the 4 samples directly to the Q=16 samples of the 4×4 block (18).

[0260] In some examples, set S1 is a matrix n1 (e.g., n1=8 or n1=18 or another number) that may have 16 rows and 8 columns, respectively, to perform the technique according to FIG. 7b or 7c. Wow, 18 offset vectors of size 16 It may include (e.g., may be configured). The matrix and offset vector of this set S1 may be used for blocks of size 4x8, 4x16, 4x32, 4x64, 16x4, 32x4, 64x4, 8x4, and 8x8. Additionally, it may be used for blocks of size WxH where max(W,H)>4 and min(W,H)=4, i.e., blocks of size 4x16 or 16x4, 4x32 or 32x4, and 4x64 or 64x4. The 16x8 matrix refers to a reduced version of block (18), which is a 4x4 block, as obtained in FIG. 7b and 7c.

[0261] Additionally or alternatively, set S2 is an n2 matrix (e.g., n2=6 or n2=18 or other numbers) that can have 64 rows and 8 columns. Wow, 18 offset vectors of size 64 It may include (e.g., may be configured). The 64x8 matrix refers to a reduced version of the block (18), which is an 8x8 block, as obtained in FIG. 7d. The matrix and offset vector of this set may be used for blocks of sizes 8×16, 8×32, 8×64, 16×8, 16×16, 16×32, 16×64, 32×8, 32×16, 32×32, 32×64, 64×8, 64×16, 64×32, 64×64.

[0262] The matrix and offset vector of that set, or parts of such matrix and offset vector, can be used for all other block shapes.

[0263] 6.2 Averaging or downsampling of the boundary

[0264] As explained above, boundary samples (171, 17c) can be averaged or / and downsampled (e.g., P samples P red <P 샘플로)

[0265] In the first step, the input boundary bdry top (e.g., 17th century) and bdry left (e.g., 17a) is a smaller boundary to reach a reduced set (102) and It can be reduced to. Here, and All consist of 2 samples in the 4x4 block case, and all consist of 4 samples in the other case.

[0266] In the case of a 4x4 block, it is possible to define the following, and

[0267]

[0268] and It is also possible to define it similarly. Therefore, , , , is, for example, an average value obtained using bit-shifting operations.

[0269] In all other cases (e.g., for blocks with a width or height different from 4), the block width W is W=4*2 k In the case where it is given as 0 For i < 4, it is defined as follows, and

[0270]

[0271] and It is also defined similarly.

[0272] In other cases, it is possible to downsample the boundary to reach a reduced number of samples (e.g., by selecting one specific boundary sample from a group of boundary samples). For example, silver and It can be selected from among, Is and It can be selected from among them. It is also possible to define it similarly.

[0273] 2 reduced boundaries and is a reduced boundary vector It can be concatenated with (in association with the reduced set (102)) and is also denoted as 17P. Reduced boundary vector is, therefore, for a 4×4 shaped block (example in FIG. 7a), size 4(P red =4) and all other shapes of blocks (for the examples in Figs. 7b-7d, size 8(P red =8) It can be.

[0274] Here, if mode < 18 (or the number of matrices in the matrix set), the following can be defined.

[0275]

[0276] If mode ≥ 18 corresponds to the transposed mode of mode-17, the following can be defined.

[0277]

[0278] Therefore, depending on a specific state (one state: mode<18; another state: mode≥18), the predicted value of the output vector is given in a different scan order (e.g., one scan order: ; Different scan order: It is possible to distribute according to ).

[0279] Other strategies may be implemented. In other examples, the mode index 'mode' is not necessarily in the range between 0 and 35 (other ranges may be defined). Also, the three sets S0, S1, and S2 do not necessarily have to each contain 18 matrices (thus, instead of an expression like mode≥18, mode≥n0, n1, n2, which are the number of matrices for each set of matrices S0, S1, and S2, respectively, are possible). Furthermore, the sets may each have a different number of matrices (for example, S0 may have 16 matrices, S1 may have 8 matrices, and S2 may have 6 matrices).

[0280] Mode and transposed information are not necessarily stored and / or transmitted as a single combined mode index 'mode'. In some examples, there is a probability that it is explicitly signaled as a transposed flag and matrix index (0-15 for S0, 0-7 for S1, 0-5 for S2).

[0281] In some cases, a combination of a transpose flag and a matrix index can be interpreted as a set index. For example, a single bit may function as a transpose flag, and some bits representing the matrix index may be collectively referred to as a "set index."

[0282] 6.3 Generation of Reduced Prediction Signals by Matrix Vector Multiplication

[0283] Here, features regarding step (812) are provided.

[0284] Reduced input vector One of the boundary vectors (17P) is a reduced prediction signal It can generate. The latter signal has a width W red and height H red It can be a signal for the downsampled block of. Here, W red and H redIt can be defined as follows.

[0285]

[0286] Reduced prediction signal It can be computed by calculating the matrix vector product and adding the offset:

[0287]

[0288] Here, A is W in all other cases if W=H=4 and 8 columns. red * H red It is a matrix that can have rows and 4 columns (e.g., a prediction matrix (17M)), and b is of size W red * H red 를 It is a vector that can have.

[0289] If W=H=4, A can have 4 columns and 16 rows, so To calculate, 4 multiplications per sample may be required. In all other cases, A can have 8 columns, and in one of these cases, one is 8 * W red * H red If ≤4*W*H, that is, even in this case To calculate, at most 4 multiplications per sample are required.

[0290] Matrix A and vector b can be taken from one of sets S0, S1, or S2 as follows. One is an index set by idx(W,H)=0 if W=H=4, idx(W,H)=1 if max(W,H)=8, and idx(W,H)=2 for all other cases. Defines . Also, one can set m=mode (put) if mode<18, and otherwise set m=mode-17. Then, if idx≤1 or idx=2 and min(W,H)>4, one, and It can be set as follows. If idx=2 and min(W,H)=4, one corresponds to A, which corresponds to an odd x-coordinate in the downsampled block when W=4, or to an odd y-coordinate in the downsampled block when H=4. It is defined as the matrix resulting from omitting all rows of. If mode ≥ 18, one replaces the reduced predicted signal with its transposed signal. In alternative examples, a different strategy may be performed. For example, instead of reducing the size of a larger matrix ("leave out"), W red =4 and H red The smaller matrix of S1 (idx=1) with =4 is used. That is, this block is now assigned to S1 instead of S2.

[0291] Other strategies may be implemented. In another example, the mode index 'mode' is not necessarily in the range between 0 and 35 (another range may be defined). Also, each of the three sets S0, S1, and S2 does not need to have 18 matrices (therefore, instead of an expression like mode<18, mode, which is the number of matrices for each set of matrices S0, S1, and S2, <n0, n1, n2가 가능하다). 또한, 세트마다 서로 다른 수의 매트릭스를 가질 수 있다(예를 들어, S0은 16개의 매트릭스를 갖고 S1은 8개의 매트릭스를 갖고, S2는 6개의 매트릭스를 가질 수 있다).

[0292] 6.4 Linear Interpolation Generating the Final Prediction Signal

[0293] Here, features regarding step (812) are provided.

[0294] Interpolation of prediction signals subsampled from a large block may require a second version of the averaged boundary. In other words, if min(W,H) > 8 and W ≥ H, then one is W = 8 * 2 l Write, and for 0≤i<8,

[0295]

[0296] Defines.

[0297] If min(W,H)>8 and H>W, then one Defines it similarly.

[0298] Additionally or alternatively, it is possible to have "hard downsampling," and here Is,

[0299]

[0300] It is the same as.

[0301] also, It can also be defined similarly.

[0302] At sample locations missing from the generation, the final predicted signal is This can be achieved by linear interpolation from ㄹ (e.g., step (813) of the examples in FIG. 7b to 7d). Such linear interpolation may be unnecessary in some examples if W = H = 4 (e.g., the example in FIG. 7a).

[0303] This linear interpolation can be given as follows (although other examples are possible). Assume W≥H. Then, H>H red If, Vertical upsampling of can be performed. In this case, It can be extended one line upward as follows. If W=8, is width W red =4, and the boundary signal averaged upwards It can be extended to that extent, as defined above, for example. If W > 8, is width W red Boundary signal averaged upwards with =8 It can be extended to that extent, for example, as defined above. One is For the first line of e You can use . Next, width W red and height 2*H red Signal on a block having It is given as follows.

[0304]

[0305] Here, 0≤x< W red and 0≤y <H red It is. The latter process is 2 k *H red It can be performed k times until =H. Therefore, if H=8 or H=16, it can be performed at most once. If H=32, it can be performed twice. If H=64, it can be performed three times. Next, a horizontal upsampling operation can be applied to the result of vertical upsampling. The latter upsampling operation can use the entire boundary to the left of the prediction signal. Finally, if H>W, it can proceed similarly by first upsampling in the horizontal direction (if necessary) and then upsampling in the vertical direction.

[0306] This is an example of interpolation using reduced boundary samples for the first interpolation (horizontal or vertical) and original boundary samples for the second interpolation (vertical or horizontal). Depending on the block size, only the second interpolation may be required, or no interpolation may be needed at all. If both horizontal and vertical interpolation are required, the order depends on the width and height of the blocks.

[0307] However, other techniques can be implemented. For example, a single boundary sample can be used for both the first and second interpolation, and the order can be fixed, e.g., first horizontal then vertical (in other cases, first vertical then horizontal).

[0308] Therefore, the interpolation order (horizontal / vertical) and the use of reduced / circular boundary samples may vary.

[0309] 6.5 Example of the Full ALWIP Process

[0310] The entire process of averaging, matrix-vector-multiplication, and linear interpolation is described for different shapes in FIGS. 7a through 7d. The remaining shapes are handled as in one of the depicted cases.

[0311] 1. Given a 4×4 block, ALWIP can take the average of two values ​​along each axis of the boundary using the technique of Fig. 7a. The resulting four input samples enter matrix-vector-multiplication. The matrix is ​​taken from set S0. After adding the offset, this yields 16 final prediction samples. Linear interpolation is not required to generate the prediction signal. Therefore, a total of (4*16) / (4*4)=4 multiplications are performed per sample. For example, see Fig. 7a.

[0312] 2. Given an 8×8 block, ALWIP can take the average of 4 values ​​along each axis of the boundary. The resulting 8 input samples are entered into matrix-vector-multiplication using the technique of Fig. 7b. The matrix is ​​taken from set S1. This yields 16 samples on the odd positions of the prediction block. Therefore, the total per sample ( The multiplication is performed. After adding the offset, these samples can be interpolated, for example, vertically using the upper boundary and, for example, vertically using the left boundary. For example, see Fig. 7b.

[0313] 3. Given an 8×4 block, ALWIP can take four averages along the horizontal axis of the boundary and four original boundary values ​​on the left boundary using the technique of Fig. 7c. The resulting 8 input samples enter the matrix-vector-multiplication. The matrix is ​​taken from set S1. This yields 16 samples on the odd horizontal and each vertical position of the prediction block. Therefore, the total per sample ( Multiplication is performed. After adding the offset, these samples can be horizontally interpolated, for example, using the left boundary. For example, see Fig. 7c.

[0314] The transposed case is handled accordingly.

[0315] 4. Given a 16×16 block, ALWIP can take the average of four values ​​along each axis of the boundary. The resulting 8 input samples enter the matrix-vector-multiplication using the technique of Fig. 7b. The matrix is ​​taken from set S2. This yields 64 samples on the odd positions of the prediction block. Therefore, the total per sample ( Multiplication is performed. After adding the offset, these samples are interpolated, for example, vertically using the upper boundary and horizontally using the left boundary. For example, see Fig. 7d.

[0316] For larger shapes, the process can be essentially the same, and it is easy to check if the number of multipliers per sample is less than 2.

[0317] For W×8 blocks, since samples are given at odd horizontal and vertical positions, only horizontal interpolation is required. Therefore, in this case, a maximum of (8*64) / (16*8)=4 multiplications are performed per sample.

[0318] Finally, for a W×4 block where W>8, A k Let be a matrix resulting from omitting all rows corresponding to odd entries along the horizontal axis of the downsampled block. Thus, the output size is 32, and only horizontal interpolation is performed again. A maximum of (8*32) / (16*4)=4 multiplications can be performed per sample.

[0319] Cases of transposition can be processed accordingly.

[0320] 6.6 Assessment of the Number and Complexity of Required Parameters

[0321] The parameters required for all proposed intra prediction modes can be composed of matrices and offset vectors belonging to sets S0, S1, and S2. All matrix coefficients and offset vectors can be stored as 10-bit values. Therefore, according to the above description, the proposed method may require a total of 14,400 parameters, each with 10-bit precision. This corresponds to 0,018 megabytes of memory. It should be noted that in the current standard 4:2:0 chroma-subsampling, a 128×128 CTU consists of 24,576 values, each 10-bit. Therefore, the memory requirements of the proposed intra prediction tool do not exceed the memory requirements of the current picture referencing tool adopted at the last meeting. In addition, it is pointed out that existing intra prediction modes require 4 multiplications per sample due to the PDPC tool or the 4-tap interpolation filter for angular prediction modes with fractional angle positions. Therefore, in terms of computational complexity, the proposed method does not exceed existing intra prediction modes.

[0322] 6.7 Signalization of the Proposed Intra Prediction Mode

[0323] For a luma block, for example, 35 ALWIP modes are proposed (another number of modes may be used). For each coding unit (CU) of the intra mode, a flag indicating whether the ALWIP mode is applied to the corresponding prediction unit (PU) is transmitted as a bitstream. The signaling of the latter index can be harmonized with the MRL in the same way as the first CE test. If the ALWIP mode is applied, the ALWIP mode index predmode can be signaled using an MPM-list with 3 MPMS.

[0324] Here, the derivation of the MPM can be performed as follows using the intra-modes of the above and left PUs. Tables, e.g., three fixed tables , There may be, and these are each existing intra prediction modes ALWIP mode

[0325]

[0326] It can be assigned to.

[0327] For each PU of width W and height H, one defines the following index:

[0328]

[0329] As in Section 4 above, the index indicates which of the three sets the ALWIP parameters will be taken from. The above forecast unit PU above If it is available, belongs to the same CTU as the current PU, and is in intra mode, and ALWIP is ALWIP-mode PU having above If this applies, one is,

[0330]

[0331] Place it as.

[0332] If the above PU is available, belongs to the same CTU as the current PU, and is in intra mode, and the existing intra prediction mode If applied to the above PU, one is,

[0333]

[0334] Place it as.

[0335] In all other cases, one is,

[0336]

[0337] Place this, which means that this mode is not available. In the same way, but without the restriction that the left PU needs to belong to the same CTU as the current PU, one mode

[0338]

[0339] It derives.

[0340] Finally, 3 fixed default lists is provided, and each list contains three distinct ALWIP modes. Default list and mode and Among them, one constructs three characteristic MPMs by replacing -1 with the default value and removing repetition.

[0341] 6.8 Adapted MPM-List Derivation for Existing Luma and Chroma Intra-Predictive Modes

[0342] The proposed ALWIP mode can be harmonized with the MPM-based coding of the existing intra-prediction mode as follows. The luminance and chroma MPM-list derivation process for the existing intra-prediction mode is the ALWIP mode for a given PU. to the existing intra-prediction mode

[0343]

[0344] A fixed table mapping to one of them You can use .

[0345] ALWIP mode for Luma MPM-list induction Whenever it encounters a neighboring luma block using, this block is the existing intra-prediction mode It can be processed as if using . For chroma MPM-list derivation, whenever the current luma block uses ALWIP-mode, the same mapping can be used to translate ALWIP-mode into the existing intra-predicted mode.

[0346] 7. Efficiently implemented embodiments

[0347] Let us briefly summarize the above example, which can form the basis for further extending the embodiments described below.

[0348] To predict a predetermined block (18) of a picture (10), multiple neighbor samples (17a,c) are used.

[0349] The reduction (100) of multiple neighbor samples by averaging is performed to obtain a reduced set (102) of sample values ​​that is smaller in number compared to the multiple neighbor samples. This reduction is optional in the embodiments of this specification and yields the so-called sample value vector mentioned below. The reduced set of sample values ​​is subject to a linear or affine linear transformation (19) to obtain predicted values ​​for a predetermined sample (104) of a predetermined block. This transformation is represented using a matrix A and an offset vector b, which are later obtained by machine learning (ML) and must be performed efficiently.

[0350] By interpolation, the predicted value for an additional sample (108) of a predetermined block is derived based on the predicted values ​​for the predetermined sample and a plurality of neighboring samples. Theoretically, the result of the affine / linear transformation can be associated with the non-full-pel sample location of the block (18), thereby allowing all samples of the block (18) to be obtained by interpolation according to an alternative embodiment. Interpolation may not be necessary at all.

[0351] Multiple neighbor samples may be extended one-dimensionally along two sides of a predetermined block, and the predetermined samples may be arranged along at least one of the rows and columns, wherein the predetermined samples may be located at every nth position from the sample (112) of the predetermined samples adjacent to the two sides of the predetermined block. Based on the multiple neighbor samples, for each of at least one of the rows and columns, a supporting value for one of the multiple neighbor positions (118) may be determined, which is aligned with at least one of the individual rows and columns, and by interpolation, a predicted value for an additional sample (108) of the predetermined block is derived based on the predicted value for the predetermined sample and the supporting value for the neighbor samples aligned with at least one of the rows and columns. A predetermined sample may be located at every n-th position from a sample (112) of a predetermined sample adjacent to two sides of a predetermined block along a row, and a predetermined sample may be located at every m-th position from a sample (112) of a predetermined sample adjacent to two sides of a predetermined block along a column, where n, m > 1. n = m. Along at least one of a row and a column, the determination of a supporting value may be performed by averaging (122) a group of neighbor samples (120) within a plurality of neighbor samples, which includes neighbor samples (118) for which an individual supporting value is determined for each supporting value. A plurality of neighbor samples may be extended one-dimensionally along two sides of a predetermined block, and reduction may be performed by grouping a plurality of neighbor samples into one or more consecutive groups of neighbor samples (110) and performing averaging on each of the one or more groups of neighbor samples having one or more neighbor samples.

[0352] For a predetermined block, the predicted residual can be transmitted in the data stream. In the decoder, it can be derived therefrom, and the predetermined block can be reconstructed using the predicted residual and predicted value for the predetermined samples. In the encoder, the predicted residual is encoded into the data stream in the encoder.

[0353] The picture may be subdivided into multiple blocks of different block sizes that constitute a predetermined block. Then, a linear or affine linear transformation for the block (18) may be selected depending on the width W and height H of the predetermined block, thereby the linear or affine linear transformation selected for the predetermined block is selected from a first set of linear or affine linear transformations as long as the width W and height H of the predetermined block are within a first set of width / height pairs, and from a second set of linear or affine linear transformations as long as the width W and height H of the predetermined block are within a second set of width / height pairs—the second set of width / height pairs is disjoint with respect to the first set of width / height pairs. Again, it will later become clear that the affine / linear transformation is expressed by other parameters, namely the weight of C, and optionally by methods such as offset and scale parameters.

[0354] The decoder and encoder are configured to subdivide a picture into multiple blocks of different block sizes containing a predetermined block, and to select a linear or affine linear transformation depending on the width W and height H of the predetermined block, thereby selecting the linear or affine linear transformation for the predetermined block from among the following.

[0355] As long as the predetermined block width W and height H are within the first set of width / height pairs, the first set of linear or affine linear transformations,

[0356] Insofar as the predetermined block width W and height H are within a second set of width / height pairs separated from a first set of width / height pairs, a second set of linear or affine linear transformations, and

[0357] A third set of linear or affine linear transformations, wherein the width W and height H of a predetermined block are within a third set of one or more width / height pairs separated from the first and second sets of width / height pairs.

[0358] A third set of one or more width / height pairs comprises only one width / height pair W', H', and each linear or affine linear transformation in the first set of linear or affine linear transformations is intended to transform N' sample values ​​into W'*H' predicted values ​​for the W'xH' array.

[0359] Each of the first and second sets of width / height pairs is W p Ga H p First width / height pair W that is not equal to p ,H p and H q =W p and W q =H p In 2nd width / height pair W q ,H q It may include.

[0360] Each of the first and second sets of width / height pairs is a third width / height pair (W p , H p It may additionally include ), and W p is H p Same as and H p > H q am.

[0361] For a predetermined block, a set index indicating which linear or affine linear transformation is selected for the block (18) from a predetermined set of linear or affine linear transformations can be transmitted in the data stream.

[0362] A plurality of neighbor samples can be expanded one-dimensionally along two sides of a predetermined block, and reduction involves grouping a first subset of a plurality of neighbor samples adjacent to a first side of a predetermined block into a first group (110) of one or more consecutive neighbor samples, and for a second subset of a plurality of neighbor samples adjacent to two sides of a predetermined block, grouping a second subset into a second group (110) of one or more consecutive neighbor samples, and performing averaging on each of the first and second groups of one or more neighbor samples having two or more neighbor samples to obtain a first sample value from the first group and a second sample value for the second group.Then, a linear or affine linear transformation is selected depending on a set index among a predetermined set of linear or affine linear transformations, thereby causing two different states of the set index to select either one of the linear or affine linear transformations of the predetermined set of linear or affine linear transformations, assuming that the first of the two different states in the form of a first vector yields an output vector of predicted values ​​and distributes the predicted values ​​of the output vector into predetermined samples of a predetermined block according to a first scan order, and assuming that the second of the two different states in the form of a second vector, a reduced set of sample values ​​may be subject to a predetermined linear or affine linear transformation, and the first and second vectors are different such that a component filled by any one of the first sample values ​​of the first vector is filled by any one of the second sample values ​​of the second vector, and a component filled by any one of the second sample values ​​of the first vector is filled by any one of the first sample values ​​of the second vector, thereby yielding an output vector of predicted values, According to the second scan order, the predicted value of the output vector is distributed to a predetermined sample of a predetermined block transposed for the first scan order.

[0363] Each of the linear or affine linear transformations in the first set of linear or affine linear transformations is for transforming N1 sample values ​​into w1*h1 predicted values ​​for a w1xh1 array at a sample location, and each of the linear or affine linear transformations in the first set of linear or affine linear transformations is for transforming N2 sample values ​​into w2*h2 predicted values ​​for a w2xh2 array at a sample location, wherein for a first predetermined one of the first set of width / height pairs, w1 may exceed the width of the first predetermined width / height pair and h1 may exceed the height of the first predetermined width / height pair, and for a second predetermined one of the first set of width / height pairs, w1 may not exceed the width of the second predetermined width / height pair and h1 may not exceed the height of the second predetermined width / height pair. By averaging a plurality of neighbor samples to obtain a reduced set (102) of sample values, the reduction (100) can be performed such that when a predetermined block has a first predetermined width / height pair and a predetermined block has a second predetermined width / height pair, the reduced set (102) of sample values ​​has N1 sample values, and applying the reduced set of sample values ​​to a selected linear or affine linear transformation can be performed by using only the first sub-position of the selected linear or affine linear transformation associated with the subsampling of the w1xh1 array of sample positions when w1 exceeds the width of one width / height pair or h1 along the height dimension exceeds the height of one width / height pair or when a predetermined block has a first predetermined width / height pair, and by completely using the selected preceding or affine linear transformation when a predetermined block has the size of a second predetermined width / height pair.

[0364] Each of the linear or affine linear transformations in the first set of linear or affine linear transformations may be for transforming N1 sample values ​​into w1*h1 predicted values ​​for a w1xh1 array at a sample location where w1=h1, and each of the linear or affine linear transformations in the first set of linear or affine linear transformations is for transforming N2 sample values ​​into w2*h2 predicted values ​​for a w2xh2 array at a sample location where w2=h2.

[0365] All embodiments described above are merely illustrative in that they may form the basis for the embodiments described below. That is, the concepts and details above will aid in understanding the following embodiments and should serve as a resource for possible extensions and modifications of the embodiments described below. In particular, many of the details described above, such as the averaging of neighbor samples and the fact that neighbor samples are used as reference samples, are optional.

[0366] More generally, the embodiments described herein assume that the prediction signal of a rectangular block is generated from already reconstructed samples, such as the intra prediction signal of the rectangular block is generated from already reconstructed samples adjacent to the left and above the block. The generation of the prediction signal is based on the following steps.

[0367] 1. Among the reference samples currently referred to as boundary samples, however, without excluding the possibility of transmitting a description to a reference sample at a different location, samples may be extracted by averaging. Here, averaging is performed on both the left and upper q-boundary samples of the block, or on only one of the boundary samples on one side. If averaging is not performed on one side, the samples on that side are retained without change.

[0368] 2. Matrix vector multiplication is performed, optionally followed by an addition of an offset, wherein the input vector for the matrix vector multiplication may be any one of the following: the connection of the averaged boundary sample on the left side of the block and the original boundary sample on the top side of the block when averaging is applied only to the left side; the connection of the original boundary sample on the left side of the block and the averaged boundary sample on the top side of the block when averaging is applied only to the top side; or the connection of the averaged boundary sample on the left side of the block and the averaged boundary sample on the top side of the block when averaging is applied to both sides of the block. Again, alternatives such as not using averaging at all will exist.

[0369] 3. The result of matrix vector multiplication and optional offset addition may optionally be a reduced prediction signal for a subsampled set of samples of the original block. The prediction signal at the remaining locations may be generated from the prediction signal on the subsampled set by linear interpolation.

[0370] In Step 2, the calculation of matrix vector multiplication should preferably be performed using integer arithmetic. Therefore, x=(x1, ..., x n If ) represents the input for matrix vector multiplication, that is, if x represents the connection of block left and upper (averaged) boundary samples, then among x, the (reduced) prediction signal calculated in Step 2 must be calculated using only bit shifts, addition of offset vectors, and integer multiplication. Ideally, the prediction signal in Step 2 is given by Ax+b, where b is an offset vector that can be zero, and A is derived by some machine learning-based training algorithm. However, these training algorithms are primarily used for matrices A=A given in floating-point precision. floatIt only generates. Therefore, one faces a problem in specifying integer operations for the purpose mentioned above, and Equation A float x is ensured to be well approximated using these integer operations. Here, expression A, which assumes a uniform distribution of vector x, float These integer operations are not necessarily chosen to approximate x, and typically expression A float The input vector x for approximating x is the component x of x i It is important to mention that we consider (averaged) boundary samples of natural video signals from which some correlation can be expected.

[0371] FIG. 8 illustrates an improved ALWIP prediction. Samples of a predetermined block can be predicted based on a first matrix-vector product between a matrix A (1100) and a sample value vector (400) derived by some machine-learning-based training algorithm. Optionally, an offset b (1110) may be added. To achieve an integer approximation or fixed-point approximation of this first matrix-vector product, the sample value vector may undergo an invertible linear transformation (403) to determine an additional vector (402). A second matrix-vector product between an additional matrix B (1200) and an additional vector (402) may be the same as the result of the first matrix-vector product.

[0372] Due to the characteristics of the additional vector (402), the second matrix-vector product can be integer-approximated by the matrix-vector product (404) between the predetermined prediction matrix C (405) and the additional vector (402) plus the additional offset (408). The additional vector (402) and the additional offset (408) can be composed of integer or fixed-point values. For example, all components of the additional offset are identical. The predetermined prediction matrix (405) can be a quantized matrix or a matrix to be quantized. The result of the matrix-vector product (404) between the predetermined prediction matrix (405) and the additional vector (402) can be understood as the prediction vector (406).

[0373] Further details regarding this integer approximation are provided below.

[0374] Possible solutions according to Example I: Subtraction and addition of mean values

[0375] Equation A that can be used in the above scenario float One possible incorporation of the integer approximation of x is the i0th component x of x i0That is, x, that is, a predetermined component (1500) of the sample value vector (400), is replaced with the mean value mean (x), that is, a predetermined value (1400) of component x, and this mean value is subtracted from all other components. That is, as illustrated in FIG. 9a, the invertible linear transformation (403) is defined such that the predetermined component (1500) of the additional vector (402) becomes a, and each other component of the additional vector (402), excluding the predetermined component (1500), is equal to the value obtained by subtracting a from the corresponding component of the sample value vector, where a is a predetermined value (1400), which is, for example, an average such as an arithmetic mean or a weighted mean of the components of the sample value vector (400). These operations on these inputs are given by an invertible transformation T (403) with an obvious integer implementation, especially when the dimension n of x is a power of 2.

[0376] Therefore, if one performs this transformation on the input x, one performs the matrix-vector multiplication by - here and we need to find the integer approximation of y=Tx. Matrix-vector product A float x represents a rectangular block, that is, a prediction of a predetermined block, and since x consists of boundary samples (e.g., averaged) of that block, one is when all sample values ​​of x are the same, that is, for all i, x iIn the case where =mean(x), each sample value of the prediction signal Afloatx should be expected to be close to or exactly the same as mean(x). This means that one should expect the i0th column of B, i.e., the column corresponding to the predetermined component, to be very similar to or identical to a column consisting only of one. Thus, if M(i0), i.e., the integer matrix (1300), is a matrix in which the i0th column is 1s and all other columns are 0s, recording By=Cy+M(i0)y as C=BM(i0), then the i0th column of C, i.e., the predetermined prediction matrix (405), is a somewhat small number of entries or 0s, as illustrated in FIG. 9b. Furthermore, since the components of x are correlated, for each i≠i0, the i-th component y of y i =x i -mean(x) can often be expected to have an absolute value much smaller than the i-th component of x. Since matrix M(i0) is an integer matrix, given the integer approximation of Cy, the integer approximation of By is achieved, and according to the above arguments, the quantization error resulting from one quantizing each entry of C in an appropriate manner is A float We can expect that the result of x's By resp. should have a marginal effect on the error in quantization.

[0377] Formula A float In other possible integrations of integer approximations of x, the i0th component x of x i0 is maintained unchanged (unaltered), and the same value x i0 is subtracted from all other components. That is, for each i≠i0, y i0 =x i0 and y i =x i -x i0That is, the predetermined value (1400) may be a component of the sample value vector (400) corresponding to the predetermined component (1500).

[0378] Alternatively, a predetermined value (1400) is a default value or a value signaled in the data stream where the picture is coded.

[0379] For example, it is advantageous for a predetermined value (1400) to have a small deviation from the predicted values ​​of the samples of a predetermined block.

[0380] According to one embodiment, the device is configured to include a plurality of reversible linear transformations (403), each of which is associated with one component of an additional vector (402). Additionally, the device is configured to select a predetermined component (1500) from among the components of a sample value vector (400), for example, and to use the reversible linear transformation (403) as a predetermined reversible linear transformation among a plurality of reversible linear transformations associated with the predetermined component (1500). This depends on the position of the predetermined component in the additional vector due to different positions of the row of the reversible linear transformation (403) corresponding to the predetermined vector, for example, the i0th row. If, for example, the first component, i.e., y1 of the additional vector (402), is a predetermined component, and The row replaces the first row of the invertible linear transformation.

[0381] As illustrated in FIG. 9b, the matrix components (414) of the predetermined prediction matrix C (405) within the column (412) of the predetermined prediction matrix (405) corresponding to the predetermined component (1500) of the additional vector (402), i.e., the i0th column, are, for example, all zero. In this case, the device is configured to calculate the matrix-vector product (404) by performing a multiplication by calculating the matrix-vector product (407) between the reduced prediction matrix C' (405) resulting from the predetermined prediction matrix C (405) by removing the row (412) as illustrated in FIG. 9c, for example, and the even additional vector (410) resulting from the additional vector (402) by removing the predetermined component (1500). Thus, the prediction vector (406) can be calculated with a small multiplication.

[0382] As illustrated in FIGS. 8, 9b and 9c, the device may be configured to calculate the sum of an individual component and a, i.e., a predetermined value (1400), for each component of the prediction vector (406) when predicting samples of a predetermined block based on the prediction vector (406). This summation may be expressed as the sum of the vector (409) in which all components of the prediction vector (406) and the vector (409) are equal to the predetermined value (1400), as illustrated in FIGS. 8 and 9c. Alternatively, the summation can be expressed as the sum of the prediction vector (406) and the matrix-vector product (1310) between the integer matrix M (1300) and the additional vector (402), as illustrated in FIG. 9b, wherein the matrix component of the integer matrix (1300) is a column of the integer matrix (1300) corresponding to a predetermined component (1500) of the additional vector (402), i.e. It is 1 within the column, and all other components are, for example, 0.

[0383] The sum of the predetermined prediction matrix (405) and the integer matrix (1300) is equal to or approximates, for example, the additional matrix (1200) shown in FIG. 8.

[0384] In other words, a column (412) of a predetermined prediction matrix (405) corresponding to a predetermined component (1500) of an additional vector (402) having 1 times (i.e., matrix B) of the invertible linear transformation (403), i.e., A matrix, i.e., an additional matrix B (1200), which is the result of summing each matrix component of a predetermined prediction matrix C (405) within a column, corresponds to a quantized version of the machine learning prediction matrix A (1100), as illustrated in, for example, FIG. 8, FIG. 9a, and FIG. 9b, having 1 The sum of each matrix component of the predetermined prediction matrix C (405) within the column (412) may correspond to the sum of the predetermined prediction matrix (405) and the integer matrix (1300), as illustrated in FIG. 9b. As illustrated in FIG. 8, the machine learning prediction matrix A (1100) may be equivalent to an additional vector (1200) multiplied by an invertible linear transformation (403). This It is attributed to. The predetermined prediction matrix (405) is, for example, a quantized matrix, an integer matrix, and / or a fixed-point matrix, through which a quantized version of the machine learning prediction matrix A (1100) can be realized.

[0385] Matrix multiplication using only integer operations

[0386] For low-complexity implementations (in terms of the complexity of adding and multiplying scalar values ​​and the storage required for the entries of the partaking matrix), it is desirable to perform matrix multiplication (404) using only integer arithmetic.

[0387] To calculate the approximate value of z=Cy, that is

[0388]

[0389] According to the embodiment, using only operations on integers, a real number value C i,j is an integer value It must be mapped to. This is, for example, uniform scalar quantization or value y i It can be performed by considering specific correlations between them. Integer values ​​represent fixed-point numbers that can be stored, for example, as a fixed number of bits n_bits, for example, n_bits=8.

[0390] A matrix vector product (404) having a matrix, i.e., a predetermined prediction matrix (405) of size mxn, can be performed as shown in this pseudo code. Here, << and >> are arithmetic binary left- and right-shift operations, and +, - and * are performed only on integer values.

[0391] (1)

[0392] final_offset = 1 << (right_shift_result - 1);

[0393] for i in 0 m-1

[0394] {

[0395] accumulator = 0

[0396] for j in 0 ,,, n-1

[0397] {

[0398] accumulator: = accumulator + y[j]*C[i,j]

[0399] }

[0400] z[i] = (accumulator + final_offset) >> right_shift_result;

[0401] }

[0402] Here, array C, i.e., a predetermined prediction matrix (405), stores fixed-point numbers, for example, as integers. The final addition of final_offset and the right-shift operation with right_shift_result reduce precision by rounding to obtain the fixed-point format required in the output.

[0403] To allow for an increased range of actual values ​​that can be represented as integers in C, two additional matrices, as shown in the embodiments of FIGS. 10 and 11 offset i,j and scale i,j can be used, and through this, y in the matrix-vector product j Each coefficient b i,j

[0404] i=

[0405] go

[0406]

[0407] It is given by.

[0408] value offset i,j and scale i,j is an integer value itself. For example, these integers are fixed-point modification values, for example, a fixed number of bits n_bits, for example 8 bits, or for example, the value Represents fixed-point numbers that can be stored as n_bits, each with the same number of bits used to store them.

[0409] In other words, the device, for example, predictive parameters, e.g., integer values and value offset i,j and scale i,jIt is implemented to represent a predetermined prediction matrix (405) using and to calculate a matrix-vector product (404) by performing multiplication and addition on the components of the addition vector (402), the prediction parameters, and the intermediate results derived therefrom, wherein the absolute value of the prediction parameters can be represented by an n-bit fixed-point number having n equal to or lower than 14, or alternatively 10, or alternatively 8. For example, the components of the addition vector (402) are multiplied with the prediction parameters to yield a product as an intermediate result that is then subject to summation or forms the addends of the summation.

[0410] According to one embodiment, the prediction parameters each include a weight associated with a corresponding matrix component of the prediction matrix. That is, a predetermined prediction matrix is ​​replaced or represented, for example, by the prediction parameters. The weights are, for example, integer and / or fixed-point values.

[0411] According to one embodiment, the prediction parameter is one or more scaling factors, for example, a value scale i,j It further includes, and each value is a weight associated with one or more corresponding matrix components of a predetermined prediction matrix (405), for example, an integer value associated with one or more corresponding matrix components of a predetermined prediction matrix (405). It is associated with one or more corresponding matrix components of a predetermined prediction matrix (405) for scaling. Additionally or alternatively, the prediction parameter is, for example, one or more offsets, e.g., value offset. i,j It includes, and each value is a weight, for example, a value associated with one or more corresponding matrix components of a predetermined prediction matrix (405). It is associated with one or more corresponding matrix components of a predetermined prediction matrix (405) to offset.

[0412] offset i,j and scale i,j To reduce the amount of storage required for this, their values ​​can be selected to be constant for a specific set of indices i,j. For example, their entries can be constant for each column, or they can be constant for each row, or they can be constant for all i,j as shown in Fig. 10.

[0413] For example, in one preferred embodiment, offset i,j and scale i,j is constant for all values ​​of the matrix of a single prediction mode, as illustrated in Fig. 11. Therefore, if there exist K prediction modes with k=0 ... k-1, only a single value o k and single value s k It is required to calculate the prediction for mode k.

[0414] The offset is o k Lo, the scale is s k It is expressed as such, and the calculation in (1) can be modified as follows.

[0415] (2)

[0416] final_offset = 0;

[0417] for i in 0 ... n-1

[0418] {

[0419] final_offset: = final_offset - y[i];

[0420] }

[0421] final_offset *= final_offset * offset * scale;

[0422] final_offset += 1 << (right_shift_result - 1);

[0423] for i in 0 ... m-1

[0424] {

[0425] accumulator = 0

[0426] for j in 0 ... n-1

[0427] {

[0428] accumulator: = accumulator + y[j]*C[i,j]

[0429] }

[0430] z[i] = (accumulator*scale + final_offset) >> right_shift_result;

[0431] }

[0432] Extended embodiments derived from that solution

[0433] The above solution implies the following embodiments:

[0434] 1. In the prediction method of Section I, for the integer approximation of the matrix vector product involved in Step 2 of Section I, the following is performed. For a fixed i0 such that 1 ≤ i0 ≤ n, the (averaged) boundary samples x = (x1, ..., x n Among ), vector y=(y1,..., y n ) is calculated, and here, about y i =x i -mean(x) and, And, mean(x) represents the mean value of x. The vector y then functions as an input for the matrix vector product Cy (of integer realizations), thereby obtaining the (downsampled) prediction signal from Step 2 of Section I. pred is It is given as. In this equation, is for each sample location within the domain of the (downsampled) prediction signal It indicates that it is the same signal as. (For example, refer to Fig. 9b)

[0435] 2. In the prediction method of Section I, for the integer approximation of the matrix vector product involved in Step 2 of Section I, the following is performed. For a fixed i0 such that 1 ≤ i0 ≤ n, the (averaged) boundary samples x = (x1, ..., x n Among ), vector y=(y1,..., y n-1 ) is calculated, and here, about y i =x i -mean(x) and, About And, mean(x) represents the mean value of x. The vector y then functions as an input for the matrix vector product Cy (of integer realizations), thereby obtaining the (downsampled) prediction signal from Step 2 of Section I. pred is It is given as. In this equation, is for each sample location within the domain of the (downsampled) prediction signal It indicates that it is the same signal as. (For example, refer to Fig. 9c)

[0436] 3. In the prediction method of Section I, the integer realization of the matrix-vector product cy is the matrix-vector product The coefficient of It is given by using. (See, for example, Fig. 10)

[0437] 4. In the prediction method of Section I, Step 2 uses one of K matrices, thereby allowing multiple prediction modes, each with different matrices It can be calculated using (K=0 ... K-1), where the matrix-vector product The integer realization of is the matrix-vector product The coefficient of It is given by using. (See, for example, Fig. 11)

[0438] That is, according to an embodiment of the present application, the encoder and decoder operate as follows to predict a predetermined block (18) of a picture (10) (refer to FIG. 8 in combination with any one of FIG. 7a to 7d). For prediction, a plurality of reference samples are used. As outlined above, the embodiment of the present application is not limited to intra-coding, and thus the reference samples are not limited to neighboring samples, i.e., samples of the picture (10) adjacent to the block (18). In particular, the reference samples are not limited to those arranged along the outer edge of the block (18), such as samples touching the outer edge of the block. However, such an environment is certainly an embodiment of the present application.

[0439] To perform a prediction, a sample value vector (400) is formed from reference samples such as reference samples (17a, 17c). Possible formations are described above. The formation may involve averaging, thereby reducing the number of samples (102) or the number of components of the vector (400) compared with the reference samples (17) contributing to the formation. The formation may also depend to some extent on the dimensions or size of the block (18), such as width and height, as described above.

[0440] It is this vector (400) that must be the subject of an affine or linear transformation to obtain a prediction of block (18). Different nomenclatures have been used above. Using the most recent one, the purpose is to perform a prediction by applying the vector (400) to matrix A through matrix vector multiplication to perform a summation with offset vector b. The offset vector b is optional. The affine or linear transformation determined by A or A and b can be determined by an encoder and a decoder, or more precisely, for the prediction based on the size and dimensions of block (18) as already described above.

[0441] However, to achieve the computational efficiency improvements described above or to make predictions more effective in terms of implementation, the affine or linear transformation is quantized, and the encoder and decoder or their predictors use the aforementioned C and T to represent and perform a linear or affine transformation having C and T, which are applied in the manner described above and represent a quantized version of the affine transformation. In particular, instead of applying the vector (400) directly to matrix A, the predictors of the encoder and decoder apply the vector (402) generated from the sample value vector (400) by subjecting it to a mapping through a predetermined invertible linear transformation T. The transformation T used here is the same as long as the vector (400) has the same size, that is, it does not depend on the dimensions of the block, i.e., width and height, or at least is the same for other affine / linear transformations. Above, the vector (402) is denoted as y. The exact matrix to perform the affine / linear transformation determined by machine learning would have been B. However, instead of performing B exactly, predictions in the encoder and decoder are performed through approximations or quantized versions thereof. In particular, the representation proceeds by appropriately expressing C in the manner described above, as C + M representing the quantized version of B.

[0442] Accordingly, prediction in the encoder and decoder is further performed by calculating a matrix-vector product (404) between a predetermined prediction matrix C and a vector (402) that are appropriately represented and stored in the encoder and decoder in the manner described above. The vector (406) resulting from this matrix-vector product is used to predict samples (104) of the block (18). As described above, for prediction, each component of the vector (406) may be applied to a summation with a parameter a, such as that indicated in 408, to compensate for the corresponding definition of C. An optional summation of the vector (406) with an offset vector b may also be involved in deriving the prediction of the block (18) based on the vector (406). As described above, each component of the vector (406), and thus each component of the sum of the vector (406), all vectors of a indicated in 408, and optional vector b can directly correspond to the samples (104) of the block (18) and thus represent the predicted values ​​of the samples. Additionally, only a subset of the samples (104) of the block is predicted in such a way, and the remaining samples of the block (18), such as the block (108), can be derived by interpolation.

[0443] As described above, there are various embodiments for setting a. For example, it may be the arithmetic mean of the components of vector (400). In that case, refer to FIG. 9, for example. The invertible linear transformation T may be as shown in FIG. 9a. i0 represents the sample value vector and the predetermined components of vector (402), respectively, which are replaced by a. However, as mentioned above, other possibilities exist. However, it was also noted above that, as far as the representation of C is concerned, the same thing can be implemented differently. For example, the matrix-vector product (404) may, in actual calculation, result in the actual calculation of a smaller matrix-vector product with lower dimensionality (see FIG. 9c, e.g.). In particular, as noted above, due to the definition of C, its entire The column (412) obtains 0, so the actual calculation of the product (404) is the component The omission of, that is By omitting the column (412), the reduced matrix C' resulting from C and this reduced vector (410) can be done by multiplying it by the reduced version of the vector (402) resulting from the vector (402).

[0444] The weights of C or the weights of C', i.e., the components of this matrix, can be represented and stored as fixed-point number representations. However, these weights (414) can also be stored in a manner associated with different scales and / or offsets, as described above. The scale and offset can be defined for the entire matrix C, that is, they can be the same for all weights (414) of matrix C or matrix C', or they can be defined individually in a constant or the same manner for all weights (414) in the same row or same column of matrix C or matrix C'. FIG. 10 illustrates that, in this respect, the calculation of the matrix-vector product can actually be performed slightly differently, i.e., by shifting the multiplication for the scale toward the vector (402 or 404), thereby reducing the number of additional multiplications that need to be performed. FIG. 11 illustrates the case where one scale and one offset are used for all weights (414) of C or C', as in the calculation (2) above.

[0445] 8. Examples using block-based intra prediction for all channels in special cases

[0446] All of the above descriptions should be considered as optional implementation details for the embodiments described below. In the following, the term matrix-based intra prediction (MIP) is used to denote an intra prediction mode, e.g., a block-based intra prediction mode, which may be implemented or identical to that represented by ALWIP, as described above, for example with respect to FIGS. 5 through 11.

[0447] In this document, it is proposed that for a 4:4:4 chroma format and a single tree, a chroma intra prediction signal is generated on a chroma intra-block where the chroma intra mode is direct mode (DM) mode and the luma intra mode is MOP mode using this MIP mode.

[0448] In direct mode, the chroma intra prediction mode is derived, for example, from the lumina intra prediction mode. For example, the chroma intra prediction mode is identical to the lumina intra prediction mode. An exception is when the lumina intra prediction mode is an MIP mode, where the chroma intra prediction mode is an MIP mode only in the case of the 4:4:4 color sampling format and the single tree, and in all other cases, the chroma-intra prediction mode is a planar mode. The 4:4:4 color sampling format represents a color sampling format in which each color component is sampled equally.

[0449] 8.1 Description of the Proposed Method

[0450] Currently, in VTM, MIP is used only for the luminance component [3]. If the intra-mode of the chroma intra-block is Direct Mode (DM) and the intra-mode of the co-located luminance block is MIP Mode, then the chroma block must use Planar Mode to generate the intra-predicted signal. It is argued that the main reason for processing these DM modes in the case of MIP is that, in the case of 4:2:0 or dual-tree, the co-located luminance block may have a different shape from the chroma block. Therefore, since MIP Mode cannot be applied to all block shapes, the MIP Mode of the co-located luminance block may not be applied to the chroma block in that case.

[0451] Meanwhile, it is claimed that when the chroma format is 4:4:4 and a single tree is used, the above-described non-applicability of the luminance MIP mode to the chroma component is not maintained. Therefore, in this case, when the chroma-intra mode in the intra block is DM mode and the luminance intra mode is MIP mode, it is proposed that the chroma intra-predicted signal be generated in this MIP mode.

[0452] It is argued that this proposed change is particularly relevant when ACT (adaptive color transform) is enabled, as the chroma mode is inferred as DM mode when ACT is used in a given block. Specifically for ACT, it is argued that strong correlation of the prediction signal across all channels is advantageous, and that this correlation can be increased when the same intra prediction mode is used for all three components of the block. The experimental results reported below support this view, in that the proposed change has a significant impact on camera capture content in RGB format, which is claimed to be the intersection of MIP and ACT where they are primarily designed.

[0453] 8.2 Examples of the proposed method

[0454] FIG. 12 illustrates an example of the prediction of a predetermined second color component block (182) of a picture (10). A block-based decoder, a block-based encoder and / or a device for predicting the picture (10) may be configured to perform such prediction.

[0455] According to an embodiment, a picture (10) of one or more color components (101, 102) and a color sampling format (11) is partitioned using a partitioning method (11') such that the picture (10) is equally partitioned for each color component (101, 102). Each color component (101, 102) is formed into a block, for example, a first color component block (18'). 11 -18' 1n ) and the second color component block (18' 21 -18' 2n) is sampled evenly. According to the color sampling format (11), each color component (101, 102) is sampled evenly, for example, and according to the partitioning scheme (11'), each color component (101, 102) is partitioned in the same way, for example. Thus, for example, according to the partitioning scheme (11'), the first color component block (18' 11 -18' 1n ) is the second color component block (18' 21 -18' 2n Having the same geometric features as ) and according to the color sampling format (11), the first color component block (18' 11 -18' 1n ) is the second color component block (18' 21 -18' 2n It has the same sampling as ). The sampling is indicated by sample points of a predetermined second color component block (182) and a coexisting intra-predicted first color component block (181).

[0456] The first color component (101) of the picture (10) is an intra-predicted first color component block (18') of the picture (10). 11 -18' 1n For each, one of the first set (508) of intra-prediction modes is selected and decoded and / or encoded in block units, and the first set (508) is a matrix-based intra-prediction mode (MIP mode) (5101 to 510) according to each of which is predicted by the block inner (18) deriving a sample value vector (514) from a reference sample (17). m Sample value vector (514) and individual matrix-based intra-prediction modes (5101 to 510) including ) and neighboring to the block inner (18) to obtain a prediction vector (514). mCompute the matrix-vector product (512) between the prediction matrix (516) associated with ) and predict samples within the block inner (18) based on the prediction vector (514). The sample value vector (514) has features and / or functionalities described, for example, for the sample value vector (400) of FIG. 6 or for the additional vector (402) of FIG. 8 through 11. That is, matrix-based intra prediction (5101 through 510 m ) is performed as described, for example, with respect to one of FIGS. 5 to 11.

[0457] Optionally, the number of components of the prediction vector (518) is less than the number of samples of the inner block (18). In this case, the samples of the block inner (18) can be predicted based on the prediction vector (518) by interpolating the samples based on the components of the prediction vector (518) assigned to the supporting sample positions of the block inner (18).

[0458] Optionally, matrix-based intra prediction modes (5101 to 510) included in the first set (508) of intra prediction modes m) is selected according to the block dimensions of individual intra-predicted first color component blocks (181) and becomes one subset of the matrix-based intra-predicted mode among a set of mutually disjoint subsets of the matrix-based intra-predicted mode. The prediction matrix (516) associated with the set of mutually disjoint subsets of the matrix-based intra-predicted mode is machine-learned, for example, and the prediction matrix (516) included in one subset of the matrix-based intra-predicted mode has the same size, and the prediction matrix (516) included in two subsets of the matrix-based intra-predicted mode selected for different block sizes has different sizes. That is, the prediction matrix included in one subset has the same size, but the prediction matrix included in different subsets has different sizes.

[0459] Optionally, matrix-based intra prediction modes (5101 to 510) included in the first set (508) of intra prediction modes m The prediction matrix (516) associated with ) has mutually equal size and is machine-learned.

[0460] Optionally, matrix-based intra prediction modes (5101 to 510) included in the first set (508) of intra prediction modes m The intra prediction modes included in the first set (508) of intra prediction modes, excluding ), are DC mode (506), Plenar mode (504), and directional mode (5001 to 500 I Includes ).

[0461] The second color component (102) of the picture (10) is decoded and / or encoded in blocks by intra-predicting the predetermined second color component block (182) of the picture (10) using a matrix-based intra-predicting mode selected for a co-located intra-predicted first color component block (181). Accordingly, the decoder and / or encoder uses an MIP mode (5101 to 510) for at least one intra-coded second / third color component block. m It is necessary to adopt ). This is done, for example, in the case of the initial prediction mode of the predetermined second color component block (182), which is direct mode.

[0462] In direct mode, the intra prediction mode of a predetermined second color component block (182) is derived, for example, from the intra prediction mode of a coexisting intra prediction first color component block (181). For example, the predetermined second color component block (182) is identical to the intra prediction mode of the coexisting intra prediction first color component block (181). That is, the intra prediction mode of the coexisting intra prediction first color component block (181) is MIP mode (5101 to 510 m In the case of ), the predetermined second color component block (182) also has the same MIP mode (5101 to 510 m It is predicted by ).

[0463] According to one embodiment, if there is no index in the data stream indicating that the prediction mode has been signaled for a predetermined second color component block (182), the direct mode may be inferred. Otherwise, the mode signaled in the data stream is used for the prediction of the predetermined second color component block (182).

[0464] That is, the second color component block (18') of the picture (10). 21 -18'2n For each of ), one of the first option and the second option may be selected. According to the first option, each second color component block (18' 21 -18' 2n The intra prediction mode for ) is a coexisting intra-predicted first color component block (18' 11 -18' 1n A coexisting intra-predicted first color component block (18') derived based on an intra-predicted mode selected for ) 11 -18' 1n The intra prediction mode selected for ) is a matrix-based intra prediction mode (5101 to 510 m In the case of any of the above, an individual second color component block (18' 21 -18' 2n The intra-prediction mode for ) is a coexisting intra-predicted first color component block (18' 11 -18' 1n It becomes identical to the selected intra prediction mode for ). coexisting intra-predicted first color component block (18' 11 -18' 1n The intra prediction mode selected for ) is a matrix-based intra prediction mode (5101 to 510 m In the case of any of the prediction modes other than ), individual second color component blocks (18' 21 -18' 2n In addition to the intra-prediction mode for ), the planar intra-prediction mode (504), the DC intra-prediction mode (506) and / or the directional intra-prediction mode (5001 to 500 I As with ), coexisting intra-predicted first color component block (18' 11 -18' 1n It may be the same as the intra prediction mode selected for ). The first option can be expressed in direct mode. According to the second option, individual second color component blocks (18' 21 -18'2n The intra-prediction mode for ) is an individual second color component block (18' 21 -18' 2n It is selected based on an intra-mode index existing in the data stream for ). Optionally, an intra-mode index exists within the data stream, and for additional selection among the first set (508) of intra-predicted modes, an additional intra-mode index coexists with an intra-predicted first color component block (18'). 11 -18' 1n It exists within the data stream for ). Individual second color component block (18' 21 -18' 2n The intramode index existing within the data stream for ) is, for example, an individual second color component block (18' 21 -18' 2n Displays a prediction mode among the first set (508) of intra-prediction modes selected for the prediction of ).

[0465] According to one embodiment, an apparatus for predicting a predetermined block of a picture, a block-based decoder and / or a block-based encoder, wherein, when the picture (10) is in a 4:4:4 color sampling format (11) and the prediction mode of a predetermined second color component block (182) is a direct mode, for a coexisting intra-predicted first color component block (181), for a predetermined second color component block (182), the same MIP mode (5101 to 510) mIt is configured to use ), wherein the coexisting intra-predicted first color component block (181) and the predetermined second color component block (182) represent different color components, namely the first color component and the second color component, and the predetermined blocks associated with them. The coexisting intra-predicted first color component block (181) and the predetermined second color component block (182) include the same geometrical features and the same spatial location, for example, in the picture (10). That is, the first color component coding tree, e.g., the luminance coding tree, is identical to the second color component coding tree, e.g., the chroma coding tree. For example, a single tree is used. The picture (10) is partitioned evenly for each color component (101, 102). The partitioning scheme (11') defines, for example, a single tree treatment of the picture (10).

[0466] According to one embodiment, dual tree processing of the picture (10) is also possible. Dual tree processing may be associated with an additional partitioning scheme (11') as the picture (10) is partitioned for the first color component (101) using first partitioning information of the data stream, and the picture (10) is partitioned for the second color component (102) using second partitioning information existing in a data stream separate from the first partitioning information.

[0467] Where it is not a dual tree or a single tree, the picture’s predetermined block prediction device, block-based decoder and / or block-based encoder, for example, the prediction mode of a coexisting intra-predicted first color component block (181) is an MIP mode (5101 to 510 m) and if the prediction mode of the predetermined second color component block (182) is direct mode, it is configured to use the planar intra prediction mode (504). However, if the prediction mode of the coexisting intra-predicted first color component block (181) is MIP mode (5101 to 510 m ) not, for example, the prediction mode of the coexisting intra-predicted first color component block (181) is a direct intra-predicted mode (506), a planar intra-predicted mode (504), or a directional intra-predicted mode (5001-500 i ) and when the prediction mode of the predetermined second color component block (182) is direct mode, the prediction mode of the predetermined second color component block (182) is the same as the prediction mode of the coexisting intra-predicted first color component block (181). This can be applied when using a single tree but not the 4:4:4 color sampling format (11), for example, using the 4:1:1 color sampling format (11), the 4:2:2 color sampling format (11), or the 4:2:0 color sampling format (11).

[0468] The color sampling format (11) can be represented as x:y:z, where the first number x represents, for example, the size of a color component block (18'), and the following two numbers y and z both represent second and / or third color component samples. They, i.e., y and z, both relate to the first number and define horizontal and vertical sampling, respectively. Since the 4:4:4 signal is uncompressed (not subsampled), it transports the first color component and additional color component data, for example, the second and / or third color components, i.e., chroma samples, in their entirety. In a 4x2 pixel array, 4:2:2 has half of the chroma samples of 4:4:4, while 4:2:0 and 4:1:1 have one-quarter of the available chroma information. The 4:2:2 signal has half the sampling rate horizontally but maintains full sampling vertically. On the other hand, 4:2:0 samples only the color of half the pixels in the first row and completely ignores the second row of samples, and 4:1:1 samples only the color of one pixel in the first row, that is, the 4:1:1 signal has a sampling rate of 1 / 4 horizontally but maintains full sampling vertically.

[0469] According to one embodiment, a predetermined block prediction device, a block-based decoder and / or a block-based encoder of a picture predicts the prediction mode of a coexisting intra-predicted first color component block (181), wherein each color component, i.e., the prediction mode of a coexisting intra-predicted first color component block (181), is an MIP mode (5101 to 510 m ) not, for example, direct intra-prediction mode (506), planar intra-prediction mode (504) or directional intra-prediction mode (5001-500 i) and when the prediction mode of the predetermined second color component block (182) is direct mode, it is configured to be used as the prediction mode of the predetermined second color component block (182) independently of the partitioning scheme of the sampling format (11) and / or picture (10), i.e., using a single tree or multi-tree color.

[0470] In the DC intra-prediction mode (506), for example, a single value such as a DC value is derived based on neighbor samples (17) spatially adjacent to a predetermined block (18), for example, a coexisting intra-prediction first color component block (181) and / or a predetermined second color component block (182), and this single DC value is attributed to all samples of the predetermined block (18) to obtain an intra-prediction signal.

[0471] In the planar intra prediction mode (504), for example, a two-dimensional linear function defined by a horizontal slope, a vertical slope, and an offset is derived based on neighbor samples (17) spatially adjacent to a predetermined block (18), for example, a coexisting intra-predicted first color component block (181) and / or a predetermined second color component block (182), and this linear function defines the predicted sample values ​​of the predetermined block (18).

[0472] Directional Intra Prediction Mode (5001-500 l), for example, in an angular intra-predicted mode, reference samples (17) spatially adjacent to a predetermined block (18), for example, a coexisting intra-predicted first color component block (181) and / or a predetermined second color component block (182), are used to fill the predetermined block (18) to obtain an intra-predicted signal for the predetermined block (18). In particular, reference samples (17) arranged along the boundaries of the predetermined block (18), such as the upper and left hand edges of the predetermined block (18), represent picture content that is extrapolated or copied along a predetermined direction (502) toward the inner of the predetermined block (18). Prior to extrapolation or copying, the picture content represented by neighbor samples (17) is subject to interpolation filtering, or, in other words, can be derived from neighbor samples (17) through interpolation filtering. Angular intra-prediction mode (5001-500 l ) differ from each other in the intra-prediction direction (502). Each angular intra-prediction mode (5001-500) l ) can have an index associated with the following, where Angular intra prediction mode(5001-500 l The association of the index for ) is, according to the associated mode index, the Angular intra prediction mode (5001-500 l When ordering ), the direction (502) can be made to rotate monotonically in a clockwise or counterclockwise direction.

[0473] FIG. 13 illustrates in more detail the prediction of a predetermined second color component block (182) of a picture (10) depending on different partitioning schemes and color sampling formats.

[0474] Predictions for picture (10a), additional picture (10b), and even additional picture (10c) are shown, where different conditions are set for different pictures.

[0475] According to one embodiment, a device, e.g., a block-based decoder, a block-based encoder and / or a picture prediction device, comprises a set of partitioning schemes (11') or has access to such a set of partitioning schemes to select and / or obtain a partitioning scheme for a picture (10). The set of partitioning schemes (11') comprises a partitioning scheme (11'1), i.e., a first partitioning scheme, and accordingly, the picture (10) is divided according to each color component (101, 102) and optionally an additional partitioning scheme (11'2), and the picture (10) has first partitioning information (11') within a data stream (12). 2a The first color component (101) is partitioned using ), the picture (10) exists within the data stream (12), and the first partitioning information (11' 2a Second partitioning information (11') separated from ) 2b The second color component (102) is partitioned using ). The first partitioning scheme 11'1) can represent a single tree processing of the picture (10), and the additional partitioning scheme (11'2) can represent a double tree processing of the picture.

[0476] In the additional partitioning scheme (11'2), the first color component (101) is partitioned differently from the second color component (102). One of the color components (101, 102) may be finely partitioned, and the other color component (101, 102) may be roughly partitioned. The second color component block (18' 21 -18' 24 The boundaries (boarders) of ) are the first color component block (18' 11 -18' 116It is also possible that it is not located at the same position as the boundary of ). The second color component block (18' 21 -18' 24 The boundary of ) is, 10 2alternative Or, as illustrated in the partitioning of the additional picture (10b), the first color component block (18' 11 -18' 116 It can pass through the inner part of the block.

[0477] For the picture (10a), the same prediction as described for FIG. 12 may be applied, wherein the first partitioning scheme (11'1) is selected from the set (11') of the partitioning schemes, and the color sampling format is each color component (10 a1 , 10 a2 It is used depending on whether ) is sampled equally.

[0478] The device of FIG. 13 is configured to select either a first option or a second option, for example, for each intra-predicted second color component block of a picture (10a). According to the first option, an intra-predicted mode for each intra-predicted second color component block, for example, a predetermined second color component block (18 a2 The intra-prediction mode for ) is a coexisting intra-predicted first color component block (18 a1 Based on the intra-prediction mode selected for ), coexisting intra-predicted first color component blocks (18 a1 The intra-prediction mode selected for ) is a matrix-based intra-prediction mode (5101-510) as shown in FIG. 12. m In the case of any of the above, an individual second color component block (18 a2 An intra-predicted first color component block (18) in which an intra-predicted mode for ) coexists a1It is derived in a manner identical to the selected intra-prediction mode for ). According to the second option, each intra-predicted second color component block (18 a2 The intra prediction mode for ) is each intra-predicted second color component block (18 a2 ) is selected based on an intra-mode index (509) existing within the data stream (12). The intra-mode index (509) is, for example, a coexisting first intra-predicted color component block (18) for selection from a first set of intra-predicted modes. a1 In addition to the additional intramode index (507) existing within the data stream (12) for ), it exists within the data stream (12).

[0479] Optionally, the device, depending on whether each color component is sampled equally, has one or more color components (10 b1 , 10 b2 The additional picture (10b) of the color sampling format is configured to be partitioned into additional blocks using an additional partitioning scheme (11'2). Thus, the first color component (10b1) has the same color sampling as the second color component (10b2) but has different block sizes. As shown in FIG. 13, the predetermined second color component additional block (18b2) of the additional picture (10b) has, for example, 1 / 4 the size of the co-located first color component additional block (18b1) of the additional picture (10b). The additional picture (10b) is sampled, for example, in a 4:4:4 color sampling format.

[0480] When using an additional partitioning scheme (11'2), the first color component addition block (18b1) coexisting with the predetermined second color component addition block (18b2) is determined, for example, based on pixels located in the two blocks (18b1 and 18b2). According to one embodiment, these pixels are located in the top-left corner, top-right corner, bottom-left corner, bottom-right corner and / or in the middle of the predetermined second color component addition block (18b2).

[0481] The first color component (10b1) of the additional picture (10b) is decoded in units of additional blocks by selecting one of the first sets of intra-predicted modes for each of the intra-predicted first color component additional blocks of the additional picture (10b).

[0482] For each intra-predicted second color component addition block of an additional picture (10b), one of the first option and the second option may be selected. According to the first option, the intra-predicted mode for each intra-predicted second color component addition block, e.g., a predetermined second color component addition block (18b2), is derived based on the intra-predicted mode selected for a co-located first color component addition block (18b1), wherein the intra-predicted mode selected for the co-located first color component addition block (18b1) is a matrix-based intra-predicted mode (5101-510 m In the case of any of the above, the intra-prediction mode for each intra-predicted second color component addition block (18b2) is derived in such a way that it is identical to the planar intra-prediction mode (504). For the coexisting first color component addition block (18b1), the selected intra-prediction mode is a matrix-based intra-prediction mode (5101-510 mIf it is not any of the following, for example, the intra-prediction mode is a planar intra-prediction mode (504), a DC intra-prediction mode (506), or a directional intra-prediction mode (5001-500 I In the case of ), the intra-prediction mode for each intra-prediction second color component addition block (18b2) is the same as the intra-prediction mode selected for the coexisting first color component addition block (18b1). According to the second option, the intra-prediction mode for each intra-prediction second color component addition block (18b2) is selected based on an intra-mode index (5092) existing within the data stream (12) for each intra-prediction second color component addition block (18b2). The intra-mode index (5092) is, for example, a coexisting intra-prediction first color component addition block (18) for selection from a first set of intra-prediction modes. b1 In addition to the additional intramode index (5072) existing within the data stream (12) for ), it exists within the data stream (12).

[0483] Optionally, the device is configured to partition one or more color components and an even additional picture (10c) of different color sampling formats according to how one or more color components are sampled differently. As illustrated in FIG. 13, the first color component (10c1) is sampled differently from the second color component (10c2). Sampling is indicated by sample points. According to the embodiment illustrated in FIG. 13, the even additional picture is sampled according to the 4:2:1 color sampling format. However, it is evident that other color sampling formats, excluding the 4:4:4 color sampling format, may also be used. This is illustrated once for the use of the first partitioning scheme (11'1) and once for the use of the additional partitioning scheme (11'2). Independent of the partitioning scheme, the prediction of the even additional picture (10c) below may be performed.

[0484] The first color component (10c1) of the even addition picture (10c) is decoded in units of even addition blocks by selecting one of the first sets of intra prediction modes for each of the intra-predicted first color component even addition blocks of the even addition picture (10c).

[0485] Additionally, the device may be configured to select one of a first option and a second option for each of the intra-predicted second color component even-add blocks of, for example, an even-add picture (10c). According to the first option, each intra-predicted second color component even-add block (18c2) is derived based on an intra-prediction mode selected for a co-located first color component even-add block (18c1), wherein the intra-prediction mode selected for the co-located first color component even-add block (18c1) is a matrix-based intra-prediction mode (5101-510 mIn the case of any of the above, the intra prediction mode for each intra-predicted second color component even addition block (18c2) is derived in such a way that it is identical to the planar intra prediction mode (504). The selected intra prediction mode for the coexisting first color component even addition block (18c1) is a matrix-based intra prediction mode (5101-510 m If it is not any of the following, for example, the intra-prediction mode is a planar intra-prediction mode (504), a DC intra-prediction mode (506), or a directional intra-prediction mode (5001-500 I In the case of ), the intra-prediction mode for each intra-predicted second color component even addition block (18c2) is the same as the intra-prediction mode selected for the coexisting first color component even addition block (18c1). According to the second option, the intra-prediction mode for each intra-predicted second color component even addition block (18c2) is selected based on the intra-mode index (5093) existing in the data stream (12) for each intra-predicted second color component even addition block (18c2). Optionally, the intra-mode index (5093) exists in the data stream (12) in addition to the even addition intra-mode index (5073) existing in the data stream (12) for the coexisting intra-predicted first color component even addition block (18c1) for selection from the first set of intra-prediction modes.

[0486] According to one embodiment, the selection between the first and second options depends on the signaling present in the data stream (12) for individual intra-predicted second color component blocks (18a2), ​​individual intra-predicted second color component addition blocks (18b2) and / or individual intra-predicted second color component even addition blocks (18c2), whereby the residual coding color transformation mode is signaled to be disabled for blocks (18a2, 18b2 and / or 18c2) within the data stream (12), for example, where the ACT (adaptive color transform) is disabled. If a residual coding color transformation is signaled to be enabled for blocks (18a2, 18b2 and / or 18c2) within the data stream (12), for example, if an ACT (adaptive color transform) is enabled, signaling is not required and the device can be configured to infer that the first option has been selected.

[0487] When ACT is used in a given block, for example, the chroma mode is inferred as DM mode, i.e., the second option. In particular, for ACT, the strong correlation of the prediction signal across all channels is advantageous, and it is claimed that this correlation can be increased when the same intra prediction mode is used for all three components of the block. The experimental results reported below are claimed to support this view in that the proposed changes have a significant impact on camera capture content in RGB format, which is claimed to be located at the intersection of the cases where MIP and ACT are primarily designed.

[0488] 8.3 Experimental Results

[0489] In this section, experimental results are reported under general test conditions for 4:4:4. Tables 1 and 2 report the results for the proposed changes, compared with the VTM-7.0 anchor for AI and RA configurations, respectively. Corresponding simulations were performed on an Intel Xeon cluster (E5-2697A v4, AVX2 enabled, Turbo Boost disabled) with a Linux OS and GCC 7.2.1 compiler installed. Results are reported for RGB and YUV under CE-8 conditions. Since the proposed change for Single-Tree OFF does not alter the bitstream, Single-Tree is always enabled.

[0490] RGB-Results for the Proposed Change: The reference is VTM-8.0 Anchor, and the test is VTM-8.0 with the proposed change and AI configuration. A single tree is enabled on the Anchor and the experiment is performed.

[0491]

[0492] RGB-Results for Proposed Changes: The reference is VTM-8.0 Anchor, and the test is VTM-8.0 with the proposed changes, RA configuration. A single tree is enabled on Anchor and tested.

[0493]

[0494] YUV-Results for the Proposed Change: The reference is VTM-8.0 Anchor, and the test is VTM-8.0 with the proposed change and AI configuration. A single tree is enabled in Anchor and tested.

[0495]

[0496] YUV-Results for the Proposed Change: The reference is VTM-8.0 Anchor, and the test is VTM-8.0 with the proposed change, RA configuration. A single tree is enabled in Anchor and tested.

[0497]

[0498] 8.4 Conclusion

[0499] In this document, it is proposed to enable MIP for all three channels for 4:4:4 content and a single tree. In this case, if the luminance component in the intra-block is in MIP mode and the chroma intra-mode is in DM mode, it is proposed to generate the chroma intra-prediction signal in this MIP mode. It is proposed to adopt the technology described in this document in the next working draft of VVC.

[0500] 9. References

[0501] [1] P. Helle et al., “Non-linear weighted intra prediction” JVET-L0199, Macao, China, October 2018.

[0502] [2] F. Bossen, J. Boyce, K. Suehring, X. Li, V. Seregin, "JVET common test conditions and software reference configurations for SDR video", JVET-K1010, Ljubljana, SI, July 2018.

[0503] [3] B. Bross, J. Chen, S. Liu, Y.-K. Wang, Verstatile Video Coding (Draft 7), Document JVET-P2001, Version 14 , Geneva, Switzerland, October 2019

[0504] [...] Common test conditions for 4:4:4

[0505] Additional Examples and Examples

[0506] Generally, examples can be implemented as a computer program product having program instructions, and the program instructions operate to perform one of the methods when the computer program product is executed on a computer. The program instructions can be stored, for example, on a machine-readable medium.

[0507] Another example includes a computer program for performing one of the methods described herein, stored in a machine-readable carrier.

[0508] That is, an example of a method is a computer program that has program instructions to perform one of the methods described herein when the computer program is executed on a computer.

[0509] Therefore, an additional example of the method is a data carrier medium (or digital storage medium, or computer-readable medium) on which a computer program for performing one of the methods described herein is recorded. The data carrier medium, digital storage medium, or recording medium is tangible and / or non-transient rather than an intangible transient signal.

[0510] Accordingly, an additional example of the method is a sequence of signals representing a data stream or a computer program for performing one of the methods described herein. The data stream or the sequence of signals may be transmitted, for example, through a data communication connection, for example, over the Internet.

[0511] Additional examples include processing means, e.g., a computer, or a programmable logic device that performs one of the methods described herein.

[0512] Additional examples include a computer with a computer program installed to perform one of the methods described herein.

[0513] Additional examples include a device or system that transmits a computer program for performing one of the methods described herein to a receiver (e.g., electronically or optically). The receiver may be, for example, a computer, a mobile device, a memory device, etc. The device or system may include, for example, a file server for transmitting the computer program to the receiver.

[0514] In some examples, a programmable logic device (e.g., an FPGA) may be used to perform some or all of the functions of the method described herein. In some examples, the FPGA may work with a microprocessor to perform one of the methods described herein. Generally, the method may be performed by any suitable hardware device.

[0515] The examples described above are merely for illustrating the principles described above. It should be understood that modifications and variations to the arrangements and details described in this specification will be apparent. Accordingly, it is intended to be limited by the scope of the following claims, rather than by the description of the embodiments and specific details presented in the description of this specification.

[0516] Identical or equivalent elements or elements having the same or equivalent functions are indicated in the following description by the same or equivalent reference numbers, even if they appear in different drawings.

Claims

Claim 1 A method for reconstructing a picture (10) of one or more color components (101, 102), comprising the steps of: partitioning a first color component of the picture (10) into coding tree blocks; decoding a first block of the first color component (101) using matrix-based intra prediction by selecting one of a first set (508) of intra-prediction modes, wherein the first set (508) derives a sample value vector (514) among reference samples (17), neighbors an inner block (18), and obtains a prediction vector (518) using a sample value vector (514) and an individual matrix-based intra prediction mode (5101-510). m Matrix-based intra-prediction modes (5101 to 510) according to each of which the inner block (18) is predicted by computing a matrix-vector product (512) between the prediction matrix (516) associated with ) and predicting samples of the inner block (18) based on the prediction vector (518). m Including -; a step of decoding a second block of a second color component (102) of the picture (10) that co-locates with the first block of the first color component (101) using a direct mode, meaning that the intra prediction mode of the second color component block is derived based on the intra prediction mode selected for the first block of the first color component; a step of determining whether a 4:4:4 color sampling format in which the color components are sampled identically is used, and also determining whether a single tree is used to partition the first and second color components into coding tree blocks; A method comprising the step of decoding the second color component block using the matrix-based intra prediction mode selected for the first block of the first color component when both the 4;4;4 color sampling format and the single tree are used, and decoding the second color component using the planar intra prediction mode when either or both of the 4;4;4 color sampling format and the single tree are not used. Claim 2 In claim 1, the matrix-based intra-predicted mode (5101-510) included by the first set (508) of intra-predicted modes, depending on the block dimensions of the individual intra-predicted first color component blocks. m ) is selected as matrix-based intra prediction mode (5101-510 m Among the sets of mutually disjoint subsets of ), matrix-based intra prediction modes (5101-510 m A method configured to become one subset of ). Claim 3 In paragraph 2, matrix-based intra-prediction mode (5101-510 m A prediction matrix associated with a set of mutually disjoint subsets of ) is machine-learned, the prediction matrix included by one subset of matrix-based intra prediction modes has mutually equal size, and the prediction matrix included by two subsets of matrix-based intra prediction modes selected for different block sizes has mutually different sizes, method. Claim 4 In claim 1, a matrix-based intra-prediction mode (5101-510) included by a first set of intra-prediction modes m A method in which the prediction matrices associated with ) have mutually equal sizes and are machine-learned. Claim 5 In claim 1, a matrix-based intra-prediction mode (5101 to 510) included by the first set of intra-prediction modes m A method in which, excluding ), the matrix-based intra-prediction mode included by the first set of intra-prediction modes comprises a DC mode, a planar mode, and a directional mode. Claim 6 A video decoder comprising one or more processors and a memory, wherein the memory comprises, when executed by the one or more processors, an instruction that causes the video decoder to perform the method of any one of claims 1 to 5. Claim 7 A method for encoding a picture (10) of one or more color components (101, 102), comprising: partitioning a first color component of the picture (10) into coding tree blocks; and encoding a first block of the first color component (101) using matrix-based intra prediction by selecting one of a first set (508) of intra-prediction modes, wherein the first set (508) derives a sample value vector (514) among reference samples (17), neighbors an inner block (18), and obtains a prediction vector (518) using a sample value vector (514) and an individual matrix-based intra prediction mode (5101-510). m Matrix-based intra-prediction modes (5101 to 510) according to each of which the inner block (18) is predicted by computing a matrix-vector product (512) between the prediction matrix (516) associated with ) and predicting samples of the inner block (18) based on the prediction vector (518). m A method comprising: a step of encoding a second block of a second color component (102) of a picture (10) that co-locates with a first block of a first color component (101) using a direct mode, wherein the intra prediction mode of the second color component block is derived based on the intra prediction mode selected for the first block of the first color component; a step of determining whether a 4:4:4 color sampling format in which the color components are sampled identically is used, and also determining whether a single tree is used to partition the first and second color components into coding tree blocks; and a step of encoding the second color component block using the matrix-based intra prediction mode selected for the first block of the first color component when both the 4:4:4 color sampling format and the single tree are used, and encoding the second color component using a planar intra prediction mode when either or both of the 4:4:4 color sampling format and the single tree are not used. Claim 8 In claim 7, the matrix-based intra-predicted mode (5101-510) included by the first set (508) of intra-predicted modes, depending on the block dimensions of the individual intra-predicted first color component blocks. m ) is selected as matrix-based intra prediction mode (5101-510 m Among the sets of mutually disjoint subsets of ), matrix-based intra prediction modes (5101-510 m A method configured to become one subset of ). Claim 9 In paragraph 8, matrix-based intra-prediction mode (5101-510 m A prediction matrix associated with a set of mutually disjoint subsets of ) is machine-learned, the prediction matrix included by one subset of matrix-based intra prediction modes has mutually equal size, and the prediction matrix included by two subsets of matrix-based intra prediction modes selected for different block sizes has mutually different sizes, method. Claim 10 In paragraph 7, a matrix-based intra-prediction mode (5101-510) included by the first set of intra-prediction modes m A method in which the prediction matrices associated with ) have mutually equal sizes and are machine-learned. Claim 11 In paragraph 7, a matrix-based intra-prediction mode (5101 to 510) included by the first set of intra-prediction modes m A method in which, excluding ), the matrix-based intra-prediction mode included by the first set of intra-prediction modes comprises a DC mode, a planar mode, and a directional mode. Claim 12 A video encoder comprising one or more processors and memory, wherein the memory comprises, when executed by the one or more processors, an instruction that causes the video encoder to perform the method of any one of claims 7 to 11. Claim 13 A non-transient digital storage medium, wherein a computer program to be executed is stored, and when said computer program is driven by a computer, the non-transient digital storage medium executes the method of any one of claims 1 to 5 or claims 7 to 11. Claim 14 delete Claim 15 delete Claim 16 delete Claim 17 delete Claim 18 delete Claim 19 delete Claim 20 delete Claim 21 delete Claim 22 delete Claim 23 delete Claim 24 delete Claim 25 delete Claim 26 delete Claim 27 delete Claim 28 delete

Images