Matrix-based intra prediction with mode-global setting

Abstract

To provide a method for encoding blocks of a picture using intra prediction.SOLUTION: A method in a decoder 54 encodes into a data stream 12 a mode index 200 indicating one of a plurality of matrix-based intra-prediction modes associated with 6, 8, or 16 different matrices on the basis of the size of the block 18, derives the input vector 102 based on downsampling of reference samples 17 neighboring the block, determines the matrix based on the size of the block and the mode index, calculates each output for each component of a matrix-vector product 206 between the input vector and the determined matrix by performing a right shift by a number of bits independent of the matrix-based intra-prediction mode indicated by the mode index, and uses each output for each component of the matrix-vector product to predict each sample of the block.SELECTED DRAWING: Figure 6.1

Description

[Technical Field]

[0001] Embodiments according to the invention relate to matrix-based intra prediction with mode-global settings for picture and video encoding / decoding. [Background technology]

[0002] Typical block-based image or video codecs usually operate by predictive coding. Thus, when a receiver of a coded image or video signal generates its signal for a given block from information already available from the coded data, the receiver constructs a prediction signal. This prediction signal serves as a first approximation of the signal for that block. In a second step, a prediction residual is decoded from the bitstream and added to the prediction signal. A better prediction signal reduces the number of bits required to transmit the prediction residual. Thus, the quality of the prediction signal has a significant impact on the efficiency of the overall codec.

[0003] Typically, there are two ways to generate a prediction signal: the first way, which is only used in video codecs, is inter-prediction, where the prediction signal is generated from reconstructed samples, which belong to a frame different from the current frame; the second way is intra-prediction, where the prediction signal is generated from reconstructed samples, which belong to the same frame and are typically spatially adjacent to a given block.

[0004] In classical codecs, intra prediction is performed using either angular prediction mode or DC and planar modes. Angular prediction mode replicates reconstructed samples to the left and top of the block along a specific direction defined by an angle parameter, and for fractional angular positions, an interpolation filter is used. DC mode generates a prediction signal as the average sample value of neighboring samples to the left and top of the block. Finally, planar mode generates a prediction signal as a linear combination of predictions along the horizontal and vertical directions. Optionally, post-filtering of the prediction signal or pre-flattening of reference samples may be applied to any of the above-mentioned prediction techniques.

[0005] Unlike the classical intra-prediction methods described above, matrix-based intra-prediction (MIP) has been introduced as a new technique for generating intra-prediction signals. It is part of the latest draft of the evolving Versatile Video Coding (VVC) standard [1]. MIP can be viewed as a low-complexity variant of the more general data-driven, neural network-based intra-prediction modes. Each MIP mode generates an intra-prediction signal by multiplexing a predefined matrix, depending on the prediction mode, with downsampled versions of the top and left boundary samples, and then upsampling the result. For further details, see the section review of matrix-based intra-prediction.

[0006] An important property of MIP is that the matrices used for various MIP modes are determined via a training algorithm that uses a large set of training data. The training algorithm attempts to find matrices such that they minimize a predefined loss function on the training data. Here, a stochastic gradient descent approach is used, where the matrix entries are iteratively updated. Such an approach for determining the matrix entries requires calculations in floating-point arithmetic, and thus the resulting matrix entries are given as floating-point numbers. Thus, after training, for each MIP mode i, a reduced predicted signal is calculated for MIP mode i in floating-point.

number

number

number

[0007] On the other hand, for application in the final standard, each matrix-vector multiplication (1) needs to be approximated by the rules specified in integer arithmetic. This means that for each MIP mode i, the reduced prediction signal pred red The calculation of

number

[0008] where Ai -c i is A i From every entry in c i represents the matrix that results when subtracting v from w. Finally, if v and w are vectors, and w has integral entries, then

number

number

number

[0009] Therefore, equation (2) is a variable input vector r red A matrix A with integral entries that reasonably approximates equation (1) for i Otherwise, it is desired to obtain the matrix A i The MIP prediction mode, which requires performing the matrix-vector product in equation (2) using

number

[0047] This can deviate significantly from the trained MIP mode, which uses a matrix-vector product with

[0048] (see equation (1)). Thus, the data-driven approach to intra prediction that lags behind MIP is violated. Summary of the Invention

[0010] Therefore, it is desirable to provide a concept for more efficient rendering of picture coding and / or video coding to support matrix-based intra prediction, and in addition or alternatively, to reduce bitstreams and thus signaling costs.

[0011] This is achieved by the subject matter of the independent claims of the present application.

[0012] Further embodiments according to the invention are defined by the subject matter of the dependent claims of the present application.

[0013] According to a first aspect of the present invention, the inventors of the present application have recognized that one problem faced when attempting to use a matrix-based intra-prediction mode (MIP mode) for predicting samples of a predetermined block of a picture arises from the fact that the matrix-vector product, i.e., matrix-vector multiplication, performed in the MIP mode needs to be approximated by integer arithmetic, which can result in a large deviation between the approximated matrix-vector product and the non-approximated matrix-vector product, i.e., the "true" matrix-vector product. In the following, the approximated matrix-vector product can be understood as the matrix-vector product of the respective MIP mode, since for each MIP mode, only this approximated matrix-vector product is calculated, and not the "true" matrix-vector product, to determine a prediction signal of the predetermined block. According to the first aspect of the present application, this problem is overcome by implementing a constraint on the calculation of the prediction signal by the matrix-vector product. The inventors have discovered that it is beneficial to represent all entries of a prediction matrix associated with a MIP mode using a fixed-point representation of a predetermined bit depth, and to apply the same predetermined bit depth to all matrix-based intra-prediction modes, for example, at least to those associated with the same block size, but optionally to prediction matrices of all block sizes. If all prediction matrix entries have a common, fixed, predetermined bit depth, this enables efficient implementation of matrix-vector products, since it is possible to use a specific multiplier to match the predetermined bit depth and share the specific multiplier across all MIP modes for matrix-vector product calculations. Furthermore, if all entries of all prediction matrices can be stored in a fixed-point representation, i.e., with fixed precision, efficient memory management is enabled when dealing with prediction matrices.In addition, the inventors have discovered that it is beneficial to calculate, for each MIP mode, the matrix-vector product between the input vector and the prediction matrix associated with each MIP mode by performing a right shift by an equal number of bits for all MIP modes, e.g., at least one associated with the same block size, but optionally, perhaps for all block size prediction matrices. This is based on the idea that a fixed right shift for all MIP modes allows for an efficient implementation of the shift in the matrix-vector product, since if the shift value does not depend on the MIP mode, a table lookup is saved and a single fixed shift operation can be implemented for the MIP, which is advantageous for a compact SIMD implementation of the matrix-vector product and reduces the case-dependent implementation of the matrix-vector product in hardware implementations.

[0014] According to a first aspect of the present application, an apparatus for decoding a predetermined block of a picture using intra prediction is configured to read a mode index from a data stream, and an apparatus for encoding the predetermined block of a picture using intra prediction is configured to insert the mode index into the data stream. For example, the encoding apparatus, i.e., the encoder, may select this mode, and optionally a further mode of an inter prediction mode, by rate-distortion optimization from a list of modes. The mode index indicates one from a list of matrix-based intra prediction modes. Additionally, the apparatus, i.e., the decoding apparatus and / or the encoding apparatus, is configured to predict samples of the predetermined block by calculating a matrix-vector product between an input vector derived from reference samples neighboring the predetermined block and a prediction matrix associated with the matrix-based intra prediction mode indicated by the mode index, and associating components of the output vector obtained by the matrix-vector product with sample positions of the predetermined block. For each matrix-based intra-prediction mode, all entries of the prediction matrix associated with the respective matrix-based intra-prediction mode are represented by a fixed-point representation of a predetermined bit depth, the predetermined bit depth being equal for the matrix-based intra-prediction mode, e.g., equal for at least one associated with the same block size, but optionally, possibly equal for matrices of all block sizes. Additionally, the apparatus is configured to, for each matrix-based intra-prediction mode, calculate a matrix-vector product between the input vector and the prediction matrix associated with the respective matrix-based intra-prediction mode by performing, for each component of the output vector, a right shift in the number of bits that is equal for the matrix-based intra-prediction mode, e.g., equal for at least one associated with the same block size, but optionally, possibly equal for matrices of all block sizes.

[0015] Depending on the embodiment, the number of matrix-based intra-prediction modes in the list of matrix-based intra-prediction modes may be 12, 16, or 32.

[0016] According to an embodiment, the decoding device and / or the encoding device is configured such that the matrix-based intra-prediction modes in the list of matrix-based intra-prediction modes have 6, 8, or 16 different matrices associated therewith. For example, there may be different lists for mutually exclusive block size sets, one with 6 different matrices associated with 12 modes for block sizes in a first block size set, one with 8 different matrices associated with 16 modes for smaller block sizes in a second block size set, and one with 16 different matrices associated with 32 modes for even smaller block sizes in a third block size set.

[0017] According to an embodiment, the decoding device and / or the encoding device is configured to calculate a matrix-vector product between an input vector and a prediction matrix associated with each matrix-based intra-prediction mode in a fixed-point arithmetic operation involving applying a right shift to an intermediate result obtained by the matrix-vector product for each component of the output vector, for example, the intermediate result being obtained by the matrix-vector product between the input vector and the prediction matrix, or the matrix-vector product between the input vector and the prediction matrix offset by a positive integer, for example, the positive integer being subtracted / added from / to every entry of the prediction matrix resulting in an intermediate matrix, and the intermediate result being obtained by the matrix-vector product between the input vector and the intermediate matrix.

[0018] According to an embodiment, the decoding device and / or the encoding device is configured to offset, for each matrix-based intra-prediction mode, all entries of the prediction matrix associated with the respective matrix-based intra-prediction mode, e.g., by addition or subtraction, before calculating the matrix-vector product, by an offset value that is equal for the matrix-based intra-prediction mode, e.g., equal for at least one associated with the same block size, but optionally, possibly equal for matrices of all block sizes. This is based on the idea that a fixed offset value for all MIP modes allows for an efficient implementation of the matrix-vector product by saving table lookups.

[0019] According to an embodiment, the decoding device and / or the encoding device is configured to store, for each matrix-based intra-prediction mode, a fixed-point representation at a predetermined bit depth for each entry of the prediction matrix associated with the respective matrix-based intra-prediction mode.

[0020] According to an embodiment, the decoding device and / or the encoding device is configured to decode / encode pictures at 10-bit resolution, store, for each matrix-based intra-prediction mode, the order of the entries of the prediction matrix associated with the respective matrix-based intra-prediction mode at 7-bit precision, and use 6 bits as the number of bits for right shifting.

[0021] According to an embodiment, the decoding device and / or the encoding device is configured to store, for each matrix-based intra-prediction mode, the entries of the prediction matrix associated with each matrix-based intra-prediction mode in an 8-bit signed digit representation. Alternatively, in the case where the entries of the prediction matrix associated with each matrix-based intra-prediction mode are entries of the same sign, the decoding device and / or the encoding device is configured to offset, for each matrix-based intra-prediction mode, all entries of the prediction matrix associated with each matrix-based intra-prediction mode by an offset value equal to the matrix-based intra-prediction mode, for example, by addition or subtraction, before calculating the matrix-vector product, and for each matrix-based intra-prediction mode, all entries of the prediction matrix associated with each matrix-based intra-prediction mode can be represented by a signed 8-bit representation. Therefore, according to this alternative, since there is no need to indicate the sign, only a 7-bit digit is stored for each matrix entry.

[0022] According to an embodiment, the decoding device and / or the encoding device is configured to calculate a matrix-vector product between an input vector and a prediction matrix associated with each matrix-based intra-prediction mode in a fixed-point arithmetic operation involving applying a right shift to an intermediate result obtained by the matrix-vector product for each component of the output vector and expressed in a bit-precision twice as high as the bit-precision in which the entries of the prediction matrix associated with the matrix-based intra-prediction mode are stored. For example, the matrix-vector product thus calculated, e.g., a prediction signal obtained by offsetting the prediction matrix by a positive integer resulting in an intermediate matrix, calculating the matrix-vector product between the input vector and the intermediate matrix resulting in the intermediate result, and performing a right shift on the intermediate result, is expressed in a bit-precision twice as high as the bit-precision in which the entries of the prediction matrix associated with the matrix-based intra-prediction mode are stored.

[0023] According to an embodiment, the list of matrix-based intra-prediction modes includes one or more pairs of matrix-based intra-prediction modes. It will be noted that the list of matrix-based intra-prediction modes may not consist exclusively of such pairs of modes; rather, other modes may also be present that are applied exclusively using either the transposed or non-transposed option. For each pair of matrix-based intra-prediction modes, the prediction matrix associated with the first matrix-based intra-prediction mode of each pair of matrix-based intra-prediction modes is equal to the prediction matrix associated with the second matrix-based intra-prediction mode of each pair of matrix-based intra-prediction modes. The device, i.e., the decoding device and / or the encoding device, is configured such that when the matrix-based intra-prediction mode indicated by the mode index is the first matrix-based intra-prediction mode of each pair of matrix-based intra-prediction modes, for example, a mode with an odd mode index, the association of the reference sample adjacent to the predetermined block with the component of the input vector and the association of the sample position of the predetermined block with the component of the output vector are transposed relative to the association in the case where the matrix-based intra-prediction mode indicated by the mode index is the second matrix-based intra-prediction mode of each pair of matrix-based intra-prediction modes, for example, a mode with an even mode index. That is, if a specific component of the input vector is associated with a position (x, y) having (0, 0) representing the upper left corner sample of the predetermined block in the former case, it is associated with (x, y) in the latter case. The same applies to the component of the output vector.

[0024] According to an embodiment, the decoding device and / or the encoding device is configured to use a list of matrix-based intra prediction modes for multiple block dimensions.

[0025] According to an embodiment, the decoding device and / or the encoding device is configured to predict, based on the output vector or based on the output vector and reference samples neighboring the predetermined block, by upsampling and / or interpolation, samples of the predetermined block that are offset from the sample positions to which the components of the output vector are associated.

[0026] According to an embodiment, the decoding device and / or the encoding device is configured to derive the input vector from reference samples neighboring a predetermined block by downsampling and / or pooling.

[0027] According to an embodiment, the reference samples neighboring the predetermined block include a first reference sample above the predetermined block and a second reference sample to the left of the predetermined block, and the apparatus, i.e., the decoding apparatus and / or the encoding apparatus, is configured to derive an input vector from the reference samples neighboring the predetermined block by downsampling and / or pooling, deriving a first intermediate component from the first reference sample, deriving a second intermediate component from the second reference sample by downsampling and / or pooling, concatenating the first intermediate component and the second intermediate component to derive a preliminary input vector, and forming an input vector from the preliminary input vector.

[0028] According to an embodiment, the decoding / encoding device is configured to decode / encode a picture at B-bit resolution. The device performs a 2-bit step from the first component of the preliminary input vector to obtain the first component of the input vector. B-1and forming the input vector from the preliminary input vector by subtracting the first component of the preliminary input vector from the further component of the preliminary input vector to obtain a further component of the input vector, or subtracting the first component of the preliminary input vector from the further component of the preliminary input vector so that the input vector is formed from the further components. Additionally, the apparatus is configured to correct the output vector by component-wise addition of the first component of the preliminary input vector.

[0029] According to an embodiment, the entries of the prediction matrices of the matrix-based intra-prediction modes in the list of matrix-based intra-prediction modes correspond to the entries in Table 2 shown below, although it will be noted that perhaps a different shift value may be chosen to list the values ​​in the table, and perhaps the values ​​in the table may be expressed in a different scale.

[0030] According to an embodiment, the decoding device and / or the encoding device is configured to use the trained prediction matrix selected for the predetermined block to predict samples of the predetermined block by calculating a matrix-vector product between an input vector derived from reference samples neighboring the predetermined block and the trained prediction matrix associated with the matrix-based intra-prediction mode selected for the predetermined block, and associating elements of the output vector obtained by the matrix-vector product with sample positions of the predetermined block. The trained prediction matrix is ​​trained by an apparatus for training a prediction matrix, for example, by an apparatus according to the second aspect.

[0031] According to a second aspect of the present invention, the inventors of the present application have recognized that one problem encountered when attempting to use a matrix-based intra-prediction mode (MIP mode) for predicting samples of a predetermined block of a picture arises from the fact that the matrix-vector products, i.e., matrix-vector multiplications performed in MIP mode, need to be approximated by integer arithmetic and that the trained prediction matrix for such matrix-vector multiplications is obtained in floating-point precision. According to the second aspect of the present application, this problem is overcome by already implementing constraints on the calculation of prediction signals by matrix-vector products in the training of a prediction matrix for such matrix-vector products. The inventors have found it beneficial to optimize the entries of the prediction matrix associated with MIP mode by using a cost function that depends on a prediction distortion measure associated with setting the entries of the prediction matrix to intermediate values ​​whose representative values ​​are mapped using a differentiable function. This approach makes it possible to limit the range of all prediction matrix entries and avoid the possibility that some entries may not be updated during training. During training, entries are represented in floating-point representation, and intermediate values ​​are then quantized to a fixed-point representation with a predetermined bit depth equal to all MIP modes. Because the entries of all prediction matrices have a common, fixed, predetermined bit depth, this enables efficient implementation of matrix-vector products. Such prediction matrices allow video or picture encoders / decoders to use specific multipliers tailored to the predetermined bit depth and share the specific multipliers across all MIP modes for matrix-vector product calculations. Furthermore, efficient memory management is enabled when dealing with prediction matrices if all entries of all prediction matrices can be stored in a fixed-point representation, i.e., with fixed precision.

[0032] Therefore, according to a second aspect of the present application, there is provided an apparatus for training prediction matrices of a list of matrix-based intra-prediction modes, one of which is to be selected for the predetermined block, to predict samples of the predetermined block by calculating a matrix-vector product between an input vector derived from reference samples neighboring the predetermined block and one of the prediction matrices associated with the matrix-based intra-prediction mode selected for the predetermined block, and associating elements of an output vector obtained by the matrix-vector product with sample positions of the predetermined block. The apparatus is configured to train the prediction matrix of the list of matrix-based intra-prediction modes, for example, by using a training set of known (e.g., original) samples and their corresponding neighborhoods, by optimizing representative values ​​for entries of the prediction matrix of the list of matrix-based intra-prediction modes, expressed in floating-point representation, using a steepest descent approach and a cost function dependent on a prediction distortion measure associated with setting the entries of the prediction matrix to intermediate values ​​to which the representative values ​​are mapped using a differentiable function. The prediction distortion measure defines, for example, the increasing cost with decreasing quality of prediction resulting from applying a differentiable function to a prediction matrix under training, meaning that the differentiable function is applied to every entry of the prediction matrix under training, the domain and codomain of the differentiable function are defined by a floating-point representation, the image of the differentiable function has a predetermined dynamic range, and the differentiable function is equivalent for matrix-based intra prediction modes.Additionally, the apparatus may, for example, after training, quantize intermediate values ​​to a fixed-point representation, so that, for each matrix-based intra-prediction mode, the prediction matrix associated with the respective matrix-based intra-prediction mode has all entries represented by the fixed-point representation of a predetermined bit depth, so that the predetermined bit depth is equal for the matrix-based intra-prediction mode, so that, for each matrix-based intra-prediction mode, the matrix-vector product between the input vector and the prediction matrix associated with the respective matrix-based intra-prediction mode is computable by performing a right shift, for each component of the output vector, by a number of bits equal for the matrix-based intra-prediction mode.

[0033] According to an embodiment, the differentiable function, i.e., the clipping function, has a slope of 1 at the origin of the image, is strictly monotonically increasing, and has horizontal asymptote at the upper and lower image limits. The horizontal asymptote at the upper and lower image limits of the differentiable function may define a predetermined dynamic range.

[0034] According to an embodiment, the differentiable function is

number

[0035] According to an embodiment, the differentiable function, i.e., the clipping function, is parameterized by a shift parameter with respect to the shift of the image in the co-domain. The apparatus is configured to optimize the shift parameter using a steepest descent approach (which may, but need not be), and to derive, for each matrix-based intra-prediction mode, from the shift parameter an equal offset value for the matrix-based intra-prediction mode, to be used to offset, for example by addition or subtraction, all entries of the prediction matrix associated with the respective matrix-based intra-prediction mode prior to computation of the matrix-vector product.

[0036] An embodiment relates to a method for decoding a predetermined block of a picture using intra prediction, the method including: reading a mode index from a data stream, the mode index indicating one from a list of matrix-based intra prediction modes; and predicting samples of the predetermined block by calculating a matrix-vector product between an input vector derived from reference samples neighboring the predetermined block and a prediction matrix associated with the matrix-based intra prediction mode indicated by the mode index, and associating elements of the output vector obtained by the matrix-vector product with sample positions of the predetermined block. For each matrix-based intra prediction mode, all entries of the prediction matrix associated with the respective matrix-based intra prediction mode are represented by a fixed-point representation of a predetermined bit depth, the predetermined bit depth being equal for the matrix-based intra prediction modes, e.g., equal for at least one associated with the same block size, but optionally, possibly equal for matrices of all block sizes. Additionally, the method includes, for each matrix-based intra-prediction mode, calculating, for each component of the output vector, a matrix-vector product between the input vector and the prediction matrix associated with the respective matrix-based intra-prediction mode by performing a right shift in a number of bits that is equal for the matrix-based intra-prediction mode, e.g., equal for at least one associated with the same block size, but optionally, possibly equal for matrices of all block sizes.

[0037] An embodiment relates to a method for encoding a predetermined block of a picture using intra prediction, the method including inserting a mode index into a data stream, the mode index indicating one from a list of matrix-based intra prediction modes, where the mode may be selected by rate-distortion optimization from the list of matrix-based intra prediction modes and optionally from further modes such as inter prediction modes. The method further includes predicting samples of the predetermined block by calculating a matrix-vector product between an input vector derived from reference samples neighboring the predetermined block and a prediction matrix associated with the matrix-based intra prediction mode indicated by the mode index, and associating elements of the output vector obtained by the matrix-vector product with sample positions of the predetermined block. For each matrix-based intra prediction mode, all entries of the prediction matrix associated with the respective matrix-based intra prediction mode are represented by a fixed-point representation of a predetermined bit depth, the predetermined bit depth being equal for the matrix-based intra prediction modes, e.g., equal for at least one associated with the same block size, but optionally, possibly equal for matrices of all block sizes. Additionally, the method includes, for each matrix-based intra-prediction mode, calculating, for each component of the output vector, a matrix-vector product between the input vector and the prediction matrix associated with the respective matrix-based intra-prediction mode by performing a right shift in a number of bits that is equal for the matrix-based intra-prediction mode, e.g., equal for at least one associated with the same block size, but optionally, possibly equal for matrices of all block sizes.

[0038] The method as described above is based on the same considerations as the encoder / decoder described above, and the method may by completed with all the features and functionality also described with respect to the encoder / decoder.

[0039] An embodiment relates to a method for training prediction matrices of a list of matrix-based intra-prediction modes, one of which is to be selected for a predetermined block, to predict samples of the predetermined block by calculating a matrix-vector product between an input vector derived from reference samples neighboring the predetermined block and one of the prediction matrices associated with the matrix-based intra-prediction mode selected for the predetermined block, and associating elements of an output vector obtained by the matrix-vector product with sample positions of the predetermined block. The method includes training the prediction matrix of the list of matrix-based intra-prediction modes, for example, by using a training set of predetermined blocks of known (original) samples and their corresponding neighborhoods, by optimizing, using a steepest descent approach, representative values ​​for entries of the prediction matrix of the list of matrix-based intra-prediction modes, expressed in floating-point representation, using a cost function dependent on a prediction distortion measure associated with setting the entries of the prediction matrix to intermediate values ​​to which the representative values ​​are mapped using a differentiable function equal to the matrix-based intra-prediction mode, whose domain and codomain are defined by a floating-point representation and whose image has a predetermined dynamic range. Additionally, the method may include, for example, after training, quantizing intermediate values ​​to a fixed-point representation, so that, for each matrix-based intra-prediction mode, a prediction matrix associated with the respective matrix-based intra-prediction mode has all entries represented by the fixed-point representation of a predetermined bit depth, so that the predetermined bit depth is equal for the matrix-based intra-prediction mode, so that, for each matrix-based intra-prediction mode, a matrix-vector product between the input vector and the prediction matrix associated with the respective matrix-based intra-prediction mode is computable by performing a right shift, for each component of the output vector, by a number of bits equal for the matrix-based intra-prediction mode.

[0040] The method as described above is based on the same considerations as the above-described apparatus for training prediction matrices, and the method may, however, be completed with all the features and functionality also described with respect to the apparatus for training prediction matrices.

[0041] The embodiments relate to data streams having pictures or video encoded therein using the encoding methods described herein.

[0042] An embodiment also relates to a computer program having a program code for performing the methods described herein, when the computer program runs on a computer.

[0043] The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention.In the following drawings, various embodiments of the invention are described with reference to the following drawings: [Brief explanation of the drawings]

[0044] [Figure 1] FIG. 1 illustrates an embodiment of encoding into a data stream. [Figure 2] FIG. 1 illustrates an embodiment of an encoder. [Figure 3] FIG. 1 illustrates an embodiment of picture reconstruction. [Figure 4] FIG. 1 illustrates an embodiment of a decoder. [Figure 5.1] 10 illustrates prediction of a block with a reduced sample value vector according to an embodiment; [Figure 5.2] 10 illustrates block prediction using sample interpolation according to an embodiment. [Figure 5.3] 10A and 10B illustrate prediction of a block with a reduced sample value vector in which only some boundary samples are averaged, according to an embodiment. [Figure 5.4]10A and 10B illustrate prediction of a block with a reduced sample value vector in which groups of four boundary samples are averaged, according to an embodiment. [Figure 6.1] FIG. 10 is a diagram illustrating matrix-based intra prediction of a predetermined block of a picture based on a mode index. [Figure 6.2] 10A and 10B are diagrams illustrating the relationship between matrix-based intra-prediction mode pairs and the application of inter-sample distance settings. [Figure 7] 1 illustrates an apparatus for decoding using MIP mode for prediction according to an embodiment. [Figure 8-1] 1 illustrates an apparatus for training a prediction matrix, according to an embodiment. [Figure 8-2] 1 illustrates an apparatus for training a prediction matrix, according to an embodiment. [Figure 9] FIG. 1 illustrates an example differentiable function. DETAILED DESCRIPTION OF THE INVENTION

[0045] Equal or equivalent elements or elements with equal or equivalent functionality are represented in the following description by equal or equivalent reference signs, even if they occur in different figures.

[0046] In the following description, numerous details are set forth to provide a more thorough explanation of embodiments of the present invention. However, it will be apparent to those skilled in the art that embodiments of the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in less detailed block diagram form to avoid obscuring the embodiments of the present invention. In addition, features of different embodiments described later in this specification may be combined with each other unless specifically stated otherwise.

[0047] Below, various examples are described that can assist in achieving more efficient compression when using matrix-based intra prediction, which may be provided in addition to, or exclusively with, other empirically designed intra prediction modes, for example.

[0048] To facilitate understanding of the following embodiments, the description begins with a presentation of possible compatible encoders and decoders onto which the above-outlined embodiments of the present application can be built. Figure 1 shows an apparatus for block-wise encoding a picture 10 into a data stream 12. The apparatus is indicated using reference numeral 14 and may be a still picture encoder or a video encoder. In other words, picture 10 may be the current picture from a video 16 when encoder 14 is configured to encode video 16 containing picture 10 into data stream 12, or encoder 14 may exclusively encode picture 10 into data stream 12.

[0049] As mentioned, the encoder 14 performs encoding in a block-wise or block-based manner. To this end, the encoder 14 subdivides the picture 10 into blocks, which are the units at which the encoder 14 encodes the picture 10 into the data stream 12. Examples of possible subdivisions of the picture 10 into blocks 18 are presented in more detail below. In general, the subdivision can end up in blocks 18 of a fixed size, such as an array of blocks arranged in rows and columns, or blocks 18 of different block sizes, such as by using hierarchical multi-tree subdivision by starting from the entire picture area of ​​the picture 10 or from a pre-partition of the picture 10 into an array of tree blocks; these examples should not be considered as excluding other possible ways of subdividing the picture 10 into blocks 18.

[0050] Furthermore, encoder 14 is a predictive encoder configured to predictively encode picture 10 into data stream 12. For a particular block 18, this means that encoder 14 determines a prediction signal for block 18 and encodes into data stream 12 the prediction residual, i.e., the prediction error, whereby the prediction signal deviates from the actual picture content in block 18.

[0051] The encoder 14 can support different prediction modes to derive a prediction signal for a particular block 18. In the following example, the key prediction mode is an intra-prediction mode, according to which the interior of the block 18 is spatially predicted from neighboring, already-encoded samples of the picture 10. The encoding of the picture 10 into the data stream 12, and the corresponding decoding procedure therefor, may be based on a particular coding order 20 defined among the blocks 18. For example, the coding order 20 may traverse the blocks 18 in a raster scan order, such as traversing each row from left to right and row-wise from top to bottom. In the case of hierarchical multi-tree subdivision, a raster scan ordering may be applied within each hierarchical level, or a depth-first traversal order may be applied, i.e., leaf nodes within a block at a particular hierarchical level may precede blocks at the same hierarchical level that have the same parent block, according to the coding order 20. Depending on the coding order 20, neighboring, already-encoded samples of the block 18 may typically be located on one or more sides of the block 18. In the case of the example presented here, for example, the already coded samples neighbouring block 18 are located above block 18 and to the left of block 18 .

[0052] Intra-prediction modes need not be the only modes supported by encoder 14. In cases where encoder 14 is, for example, a video encoder, encoder 14 may also support intra-prediction modes according to which of blocks 18 are temporally predicted from previously encoded pictures of video 16. Such intra-prediction modes may be motion-compensated prediction modes according to which of motion vectors are signaled for such blocks 18, indicating the relative spatial offset of the portion from which the prediction signal of block 18 will be derived as a replica. Additionally or alternatively, in cases where encoder 14 is a multiview encoder, other non-intra-prediction modes may also be available, such as inter-view prediction modes or non-predictive modes according to which of the interior portions of blocks 18 are coded as is, without any prediction.

[0053] Before starting to focus the present application's description on intra-prediction modes, further specific examples of possible block-based encoders, i.e., possible implementations of encoder 14, will be described with reference to FIG. 2, and then two corresponding examples of decoders compatible with FIGS. 1 and 2, respectively, will be presented.

[0054] While Figure 2 illustrates a possible implementation of the encoder 14 of Figure 1, i.e., an implementation in which the encoder is configured to use transform coding to encode the prediction residual, this is largely by way of example, and the present application is not limited to that classification of prediction residual coding. According to Figure 2, the encoder 14 includes a subtractor 22 configured to subtract a corresponding prediction signal 24 from an inbound signal, i.e., picture 10, or, on a block-by-block basis, from a current block 18, to obtain a prediction residual signal 26, which is then encoded into the data stream 12 by a prediction residual encoder 28. The prediction residual encoder 28 includes a lossy encoding stage 28a and a lossless encoding stage 28b. The lossy stage 28a receives the prediction residual signal 26 and includes a quantizer 30 that quantizes samples of the prediction residual signal 26. As already mentioned above, this embodiment uses transform coding of the prediction residual signal 26, and accordingly, the lossy encoding stage 28a includes a transform stage 32 connected between the subtractor 22 and the quantizer 30 to transform such a spectrally decomposed prediction residual 26 by quantization by the quantizer 30 performed on the transformed coefficients representing the residual signal 26. The transform may be a DCT, DST, FFT, or Hadamard transform, etc. The transformed and quantized prediction residual signal 34 is then subjected to lossless coding by the lossless encoding stage 28b, which is an entropy coder that entropy codes the quantized prediction residual signal 34 into the data stream 12. The encoder 14 further includes a prediction residual signal reconstruction stage 36 connected to the output of the quantizer 30 to reconstruct the prediction residual signal from the transformed and quantized prediction residual signal 34 in a manner that is also usable by the decoder, i.e., taking into account the coding loss in the quantizer 30. To this end, the prediction residual reconstruction stage 36 includes a dequantizer 38 that performs the inverse of the quantization of the quantizer 30, followed by an inverse transformer 40 that performs an inverse transform to the transform performed by the transformer 32, such as the inverse of the spectral decomposition, such as the inverse of any of the specific transform embodiments mentioned above.The encoder 14 includes an adder 42 that adds the reconstructed prediction residual signal as output by the inverse transformer 40 and the prediction signal 24 to output a reconstructed signal, i.e., a reconstructed sample. This output is fed to a predictor 44 of the encoder 14, which then determines the prediction signal 24 based thereon. It is a predictor 44 that supports all prediction modes already discussed above with respect to Figure 1. Figure 2 also illustrates that, in the case where the encoder 14 is a video encoder, the encoder 14 may also include an in-loop filter 46 that filters the fully reconstructed picture, which, after filtering, forms the reference picture for the predictor 44 for inter-predicted blocks.

[0055] As already mentioned above, the encoder 14 operates on a block basis. For the purposes of the following description, a block-based operation of interest is an operation of subdividing the picture 10 into blocks for which an intra-prediction mode is selected from a set or plurality of intra-prediction modes supported by the predictor 44 or the encoder 14, respectively, and the selected intra-prediction mode is individually implemented. However, other classifications of blocks into which the picture 10 is subdivided may also exist. For example, the above-mentioned decision of whether the picture 10 is inter-coded or intra-coded may be made at a block granularity or unit deviating from the block 18. For example, the inter / intra mode decision may be performed at the level of coding blocks into which the picture 10 is subdivided, and each coding block is subdivided into prediction blocks. The prediction blocks resulting from encoding a block for which it has been determined that intra-prediction is to be used are each subdivided into intra-prediction mode decisions. For this purpose, a decision is made for each of those prediction blocks as to which supported intra-prediction mode should be used for the respective prediction block. These predictive blocks form block 18, which is the block of interest here. Predictive blocks within coding blocks associated with inter prediction are treated differently by predictor 44. They are inter predicted from a reference picture by determining a motion vector and replicating the prediction for this block from the location in the reference picture pointed to by the motion vector. Another block subdivision involves subdivision into transform blocks in the units where transformation by transformer 32 and inverse transformer 40 is performed. Transformed blocks may, for example, be the result of further subdivision of a coding block. In nature, the examples shown herein should not be considered limiting, and other examples may exist.For the sake of completeness only, it will be noted that the subdivision into coding blocks may, for example, use multi-tree subdivision, and that prediction blocks and / or transform blocks can also be obtained by further subdivision of the coding blocks using multi-tree subdivision.

[0056] A decoder 54 or device for block-wise decoding compatible with the encoder 14 of FIG. 1 is described in FIG. 3. This decoder 54 performs the opposite of the encoder 14, i.e., it decodes the picture 10 from the data stream 12 in a block-wise manner and, to this end, supports multiple intra-prediction modes. The decoder 54 may, for example, include a residual provider 156. All other possibilities discussed above with respect to FIG. 1 are also valid for the decoder 54. To this end, the decoder 54 may be a still picture decoder or a video decoder, and all prediction modes and predictability are also supported by the decoder 54. The difference between the encoder 14 and the decoder 54 exists primarily in the fact that the encoder 14 chooses or selects coding decisions according to some optimization, such as to minimize some cost function, which may depend on the coding rate and / or coding distortion. One of those coding options or coding parameters may involve selecting the intra-prediction mode to be used for the current block 18 among the available or supported intra-prediction modes. The selected intra-prediction mode may then be signaled by encoder 14 for the current block 18 in data stream 12, with decoder 54 again making the selection using this signaling in data stream 12 for block 18. Similarly, the subdivision of picture 10 into blocks 18 may undergo optimization in encoder 14, and corresponding subdivision information may be conveyed in data stream 12 by decoder 54, which recovers the subdivision of picture 10 into blocks 18 based on the subdivision information. To summarize the above, decoder 54 may be a predictive decoder that operates on a block basis and in addition to intra-prediction modes, decoder 54 may support other prediction modes, such as inter-prediction modes, for example, in the case where decoder 54 is a video decoder.1, and since this coding order 20 is respected in both the encoder 14 and the decoder 54, the same neighboring samples are available for the current block 18 in both the encoder 14 and the decoder 54. Therefore, in order to avoid unnecessary repetitions, the descriptions of the modes of operation of the encoder 14, as far as they relate to the subdivision of the picture 10 into blocks, e.g., as far as they relate to prediction and as far as they relate to the coding of prediction residuals, may also apply to the decoder 54. The difference lies in the fact that the encoder chooses some coding options or coding parameters by optimization and signals the coding parameters in or inserts the coding parameters into the data stream 12, which are then derived from the data stream 12 by the decoder 54 to perform the prediction, subdivision, etc. again.

[0057] Figure 4 shows a possible implementation of the decoder 54 of Figure 3, i.e., an implementation that matches the implementation of the encoder 14 of Figure 1 as shown in Figure 2. Because many elements of the encoder 54 of Figure 4 are identical to those in the corresponding encoder of Figure 2, the same reference numerals, provided by apostrophes, are used in Figure 4 to indicate those elements. In particular, the adder 42', the optional in-loop filter 46', and the predictor 44' are connected in the prediction loop in the same manner as they are in the encoder of Figure 2. The reconstructed, i.e., quantized and retransformed, prediction residual signal applied to the adder 42' is derived by a sequence of an entropy decoder 56, which reverses the entropy coding of the entropy encoder 28b, followed by a residual signal reconstruction stage 36' consisting of a dequantizer 38' and an inverse transformer 40', just as in the encoding case. The output of the decoder is a reconstruction of the picture 10. The reconstruction of picture 10 may be available directly at the output of adder 42', or alternatively at the output of in-loop filter 46'. To subject the reconstruction of picture 10 to some post-filtering to improve picture quality, some post-filtering may also be arranged at the output of the decoder, but this option is not described in Figure 4.

[0058] Again, with respect to Figure 4, the explanations brought out above with respect to Figure 2 should also be valid for Figure 4, except that the encoder only performs optimization tasks and related decisions regarding coding options. However, all explanations regarding block subdivision, prediction, dequantization, and retransformation are also valid for the decoder 54 of Figure 4.

[0059] The embodiments described herein utilize so-called matrix-based intra prediction. The overall concept is outlined below.

[0060] A review of matrix-based intra prediction In order to keep this application self-contained, this section describes the main steps of the current matrix-based intra prediction (MIP) method contained in Working Draft 7 of Versatile Video Coding [1]. For further details, it refers to [1].

[0061] Matrix-based intra prediction (MIP) is a method for generating an intra prediction signal for a rectangular block of width W and height H. The input for the MIP prediction process is the reconstructed samples r of one row above the block. top and the reconstructed sample r in one column to the left of the block left , the MIP mode-index i, and the reconstructed sample r constructed from information about whether the MIP mode is to be interpolated or not. Then, the MIP prediction signal is generated using the following three steps: 1. Depends on W and H,

number

number

number

number

number

number

number

number

number

number

[0062] Presentation of an example of an implementation aspect The overall concept has been outlined above. To explain the use of those modes in more detail again, as an alternative synonym for MIP (matrix - based intra prediction), the concept may be referred to as ALWIP (affine - linear weighted intra prediction) below.

[0063] The overall process of populating the input vector based on proximity and calculating matrix - vector multiplication and linear interpolation is illustrated for different block shapes in the subsequent FIGS. 5.1 - 5.4. Note that one of the cases where the remaining shapes are described is treated as being in it. 1. Assuming a 4x4 block, ALWIP (or MIP) can take two averages along each axis of the boundary, see Figure 5.1. As an alternative to averaging, every second sample in the neighborhood is taken, or more generally and precisely, every component of the input vector for matrix-vector multiplication 19 is taken exactly from one neighboring sample. The resulting four input samples enter the matrix-vector multiplication. The matrix is ​​taken from set S0, which is a set of matrices for nearby block sizes. After adding an offset, this can yield 16 final predicted samples. Linear interpolation is not required to generate the predicted signal. Thus, a total of (4x16) / (4x4) = 4 multiplications are performed per sample. See, for example, Figure 5.1, which illustrates ALWIP for a 4x4 block. The exact calculation has been explained above. 2. Assuming an 8x8 block, ALWIP can take four averages along each axis of the boundary, see Figure 5.2. The resulting eight input samples enter matrix-vector multiplication 19. The matrix is ​​taken from set S1. This can result in 16 samples on the odd positions of the predicted block. Thus, a total of (8x16) / (8x8)=2 multiplications are performed per sample. After adding the offset, the samples are vertically interpolated by using the reduced top boundary. Horizontal interpolation follows, using the original left boundary. For example, see Figure 5.2, which illustrates ALWIP for an 8x8 block. 3. Assuming an 8x4 block, ALWIP can take the four averages along the horizontal axis of the boundary and the four original boundary values ​​on the left boundary, see Figure 5.3. The resulting eight input samples enter a matrix-vector multiplication. The matrix is ​​taken from set S1. This can result in 16 samples on odd horizontal positions and each vertical position of the prediction block. Thus, a total of (8x16) / (8x4)=4 multiplications are performed per sample. After adding the offset, the samples are horizontally interpolated by using the original left boundary. For example, see Figure 5.3, which illustrates ALWIP for an 8x4 block. Therefore, the interpolated case is treated. 4. Assuming a 16x16 block, ALWIP can take four averages along each axis of the boundary. The resulting eight input samples enter a matrix-vector multiplication. The matrix is ​​taken from set S2. This can result in 64 samples on the odd positions of the predicted block. Thus, a total of (8x64) / (16x16)=2 multiplications are performed per sample. After adding the offset, the samples are vertically interpolated by using the eight averages of the top boundary. Horizontal interpolation follows, using the original left boundary. For example, see Figure 5.4, which illustrates ALWIP for a 16x16 block. For larger shapes, the procedure can be essentially identical, and it is easy to check that the number of multiplications per sample is less than two. For Wx8 blocks, only horizontal interpolation is needed since samples are provided at odd horizontal positions and at every vertical position, so a maximum of (8x64) / (16x8) = 4 multiplications per sample are performed in those cases. Finally, for W×4 blocks with W>8, A kLet ∑ m = 16 × 4 m be the matrix resulting from excluding any rows corresponding to odd entries along the horizontal axis of the downsampled block. Thus, the output size may be 32, and again only horizontal interpolation remains to be performed. A maximum of (8 × 32) / (16 × 4) = 4 multiplications per sample may be performed. Therefore, the interpolated case may be handled, which is illustrated in the subsequent figures.

[0064] FIG. 6.1 shows an apparatus 54 for decoding a predetermined block 18 of a picture using intra prediction.

[0065] The device 54 is configured to read a mode index 200 from the data stream 12 using a binarization code 202, where the mode index points to one from a list 204 of matrix-based intra-prediction modes. The list 204 of matrix-based intra-prediction modes consists of even-numbered matrix-based intra-prediction modes, and the matrix-based intra-prediction modes of the list 204 are grouped into matrix-based intra-prediction mode pairs 212. Each pair 212 consists of a first matrix-based intra-prediction mode and a second matrix-based intra-prediction mode. For each matrix-based intra-prediction mode pair 212, the first matrix-based intra-prediction mode is assigned a first codeword, and the second matrix-based intra-prediction mode is assigned a second codeword, with both codewords being equal in length. The device 54 is configured to read the mode index 200 from the data stream 12 using the binarization code 202 in such a manner.

[0066] Optionally, the binarization code 202 is a variable-length code, and the variable-length code includes codewords of different lengths. Alternatively, the binarization code may be a truncated binary code, and the number of matrix-based intra-prediction modes is not a power of two, so that the truncated binary code has codewords of different lengths. The matrix-based intra-prediction mode associated with the first pair of matrix-based intra-prediction modes 212 may be assigned codewords of different lengths as the codewords assigned to the matrix-based intra-prediction mode associated with the second pair of matrix-based intra-prediction modes 212. However, both codewords of the matrix-based intra-prediction mode pair 212 are equal in length.

[0067] According to an embodiment, device 54 may be configured to read mode index 200 from data stream 12 using an equal probability bypass mode of a context-adaptive binary arithmetic decoder.

[0068] Similarly, for an apparatus 54 (i.e., a decoder) that decodes a predetermined block 18 of a picture using intra prediction, an apparatus (i.e., an encoder) that encodes a predetermined block 18 of a picture using intra prediction may be configured to encode a mode index 200 into a data stream 12 using a binarization code 202, optionally using an equal probability bypass mode of a context-adaptive binary arithmetic encoder.

[0069] The decoder and encoder are configured to predict samples 108 of a predetermined block 18 by computing a matrix-vector product 206 between an input vector 102 derived from reference samples 17 neighboring the predetermined block 18 and a prediction matrix 19 associated with a matrix-based intra-prediction mode k indicated by a mode index 200. Computing the matrix-vector product 206 results in an output vector 208. Furthermore, the samples 108 of the predetermined block 18 are predicted by associating components 210 of the output vector 208 obtained by the matrix-vector product 206 with sample positions 104 of the predetermined block 18. This prediction of samples 108 of the predetermined block 18 may be performed as described with respect to Figures 5.1 to 5.4.

[0070] For each pair of matrix-based intra-prediction modes 212, the prediction matrix 19 associated with the first matrix-based intra-prediction mode of each pair of matrix-based intra-prediction modes 212 is equal to the prediction matrix 19 associated with the second matrix-based intra-prediction mode of each pair of matrix-based intra-prediction modes 212. Thus, the same prediction matrix 19 is used for matrix-based intra-prediction modes 2k and 2k+1. For each pair 212 of matrix-based intra-prediction modes, the encoder and decoder are configured such that when the matrix-based intra-prediction mode indicated by the mode index 200 is the first matrix-based intra-prediction mode of the respective pair 212 of matrix-based intra-prediction modes, e.g., the mode with odd mode index 2k+1, the association of the reference sample 17 adjacent to the predetermined block 18 with the component 214 of the input vector 112 and the association of the sample position 104 of the predetermined block 18 with the component 210 of the output vector 20 are transposed relative to the association in the case when the matrix-based intra-prediction mode indicated by the mode index 200 is the second matrix-based intra-prediction mode of the respective pair 212 of matrix-based intra-prediction modes, e.g., the mode with even mode index 2k.

[0071] The decoder / encoder may be configured to determine, based on the parity of the mode index 200, whether the matrix-based intra-prediction mode indicated by the mode index 200 is the first matrix-based intra-prediction mode of a respective pair of matrix-based intra-prediction modes or the second matrix-based intra-prediction mode of a respective pair 212 of matrix-based intra-prediction modes. The parity of the mode index 200 may indicate whether the input vector 102 and the output vector 208 are used in a transposed manner for predicting the samples 108 of the predetermined block 18. That is, as shown in Figure 6.2, if a particular component of components 1 to n of the input vector 102 is associated with a position (x,y) having (0,0) representing the upper-left corner sample AA of the predetermined block 18 in the former case, it is associated with (x,y) in the latter case. The same applies to the components (AA, AB, AC, BA, CA, ...) of the output vector 208.

[0072] Each pair 212 consists of a first matrix-based intra-prediction mode and a second matrix-based intra-prediction mode that are related to each other by the same prediction matrix 19 and that differ from each other only in terms of whether the input vector 102 and the output vector 208 are transposed. This is beneficial because only the mode index 200 is needed in the data stream 12 to indicate the matrix-based intra-prediction mode and whether the matrix-based intra-prediction mode is used in transposed fashion. For the matrix-vector product 206, no additional index or flag is needed to indicate that the input vector 102 and the output vector 208 will be used in transposed fashion.

[0073] According to an embodiment, the decoder / encoder is configured to index a prediction matrix 19 from a plurality of prediction matrices using the integer part of the mode index 200 divided by 2. This is based on the idea that both matrix-based intra-prediction modes of a pair 212 use the same prediction matrix 19 for predicting samples 108 of a predetermined block 18, since the prediction matrix 19 is already fully indicated by pointing to the associated pair 212 in the list 204 via the mode index 200.

[0074] As shown in Figures 6.1 and 6.2, the decoder / encoder may be configured to set (217) the intersample distance 216 of the sample locations 104 of the predetermined block 18 and the intersample distance 218 of the reference samples 17 neighboring the predetermined block 18 horizontally according to a first ratio of a horizontal dimension 220 of the predetermined block 18 to a horizontal default dimension, and / or vertically according to a second ratio of a vertical dimension 222 of the predetermined block 18 to a vertical default dimension. This enables the use of a list 204 of matrix-based intra-prediction modes for multiple block dimensions. The device may fill spaces between predicted samples by interpolation. The intersample distance setting 217 of the intersample distance 216 of the sample locations 104 of the predetermined block 18 and the intersample distance 218 of the reference samples 17 neighboring the predetermined block 18 enables improved distribution of the predicted samples 108 within the predetermined block 18 and the reference samples 17 neighboring the predetermined block 18. Thus, the predicted samples can be equally distributed, allowing for improved interpolation of the samples of the predetermined block 18 .

[0075] According to an embodiment, the decoder / encoder is configured to order the matrix-based intra-prediction modes in the list of matrix-based intra-prediction modes 204 equally for multiple block dimensions. Alternatively, the order may be adapted, for example, for blocks that are wider than they are tall, or vice versa, i.e., taller than they are wide, or two-dimensional. This ordering may increase coding efficiency and reduce bitstream size, since matrix-based intra-prediction modes for common block dimensions may be associated with short codewords and matrix-based intra-prediction modes for rare block dimensions may be associated with longer codewords.

[0076] Optionally, the plurality of block dimensions includes at least one block dimension corresponding to an aspect ratio greater than 4. Matrix-based intra prediction may be optimized such that the predetermined blocks 18 have an aspect ratio of the horizontal dimension 220 to the vertical dimension 222 greater than 4. That is, the plurality of block dimensions includes predetermined blocks having a horizontal dimension 220 at least four times larger than the vertical dimension 222 and / or predetermined blocks having a vertical dimension 222 at least four times larger than the horizontal dimension 220. Figure 6.2 shows predetermined blocks 18 having block dimensions corresponding to an aspect ratio greater than 4.

[0077] According to the embodiment proposed below, the MIP mode is applied in a manner that makes the use of MIPs more efficient compared to the use envisaged so far in current VVC versions.

[0078] The embodiments below largely illustrate features and functionality with a decoder in mind. However, it will be apparent that the same or similar features and functionality may be provided by the encoder, e.g., the decoding performed by the decoder may correspond to the encoding by the encoder. Furthermore, the encoder may include the same features as described with respect to the decoder in a feedback loop, e.g., prediction stage 36.

[0079] FIG. 7 shows an apparatus 54 for decoding a predetermined block 18 of a picture using intra prediction.

[0080] The device 54 is configured to read a mode index 200 from the data stream 12. The mode index 200 is a matrix-based intra-prediction mode index 2051-2055. n , i.e., points to one of the matrix-based intra-prediction modes 2051 to 2055 in the list of matrix-based intra-prediction modes 204. n The number n is, for example, 12, 16, or 32. In the embodiment, the MIP modes 2051 to 205 n While the present discussion focuses on intra-prediction using , it will be apparent that mode index 200 can also be used to indicate additional modes, such as additional intra-prediction and / or inter-prediction modes. Mode index 200 may be inserted into data stream 12 by device 14 that encodes predetermined block 18 of a picture using intra-prediction.

[0081] According to an embodiment, the matrix-based intra-prediction modes 2051 to 2055 in the list of matrix-based intra-prediction modes 204 n are associated with 6, 8, or 16 different prediction matrices 19 .

[0082] According to an embodiment, the list 204 of matrix-based intra-prediction modes includes MIP modes for multiple block dimensions.

[0083] According to an embodiment, there may be two or more lists 204 of MIP modes, which differ from one another in terms of the block sizes of the predetermined blocks 18 with which the MIP modes are associated. MIP modes associated with the same or similar block sizes may be included in the same list of the two or more lists 204 of MIP modes. For example, there may be different lists for mutually exclusive block size sets, one having six different matrices associated with 12 modes for block sizes in a first block size set, one having eight different matrices associated with 16 modes for smaller block sizes in a second block size set, and one having 16 different matrices associated with 32 modes for even smaller block sizes in a third block size set. This is merely an example, and it will be apparent that a different number of lists 204 are possible, and that each list may include MIP modes associated with a block size set different from the one described above.

[0084] The device 54 is configured to predict samples 108 of a predetermined block 18 by calculating a matrix-vector product 206 between an input vector 102 derived from reference samples 17 adjacent to the predetermined block 18 and a prediction matrix 19 associated with a matrix-based intra-prediction mode 205 indicated by a mode index 200, and associating components 210 of an output vector 208 obtained by the matrix-vector product 206 with sample positions 104 of the predetermined block 18.

[0085] Matrix-based intra prediction modes 2051-205 nFor each MIP mode, all entries of the prediction matrix 19 associated with each matrix-based intra-prediction mode 205 are represented by a fixed-point representation 190 of a predetermined bit depth 192. The fixed-point representation 190 may have a fractional part 194 having n bits, an integer part 196 having m bits, and an optional sign bit 198. The predetermined bit depth 192 is used to represent all matrix-based intra-prediction modes 2051-205 of the list of MIP modes 204. n , as shown in FIG. 7, is equal to the following comparison constraint 1. In the case where there are two or more lists 204 of MIP modes, the predetermined bit depth 192 for each list of the two or more lists 204 of MIP modes is, for example, equal to all matrix-based intra prediction modes 2051 to 2055 of each list of MIP modes. n In other words, the predetermined bit depth 192 is at least the same for MIP modes associated with, for example, the same set of block sizes.

[0086] According to an embodiment, the device 54 selects matrix-based intra-prediction modes 2051-205. n For each matrix-based intra-prediction mode 205, the prediction matrix 19 is configured to store a fixed-point representation at a predetermined bit depth for each entry of the prediction matrix 19 associated with the respective matrix-based intra-prediction mode 205. This is illustrated, for example, in the following table examples, see Lists 1 to 4.

[0087] As shown in FIG. 7, matrix-based intra prediction modes 2051 to 205 n For each component 210 of the output vector, the device 54 selects all matrix-based intra-prediction modes 2051-2055 of the list of MIP modes 204. nThe method is configured to calculate a matrix-vector product 206 between the input vector 102 and the prediction matrix 19 associated with each matrix-based intra-prediction mode 205 by performing a right shift 209 with a number of bits 211 equal to x bits for each of the input vector 102 and the prediction matrix 19 associated with each matrix-based intra-prediction mode 205. The number of bits 211 for the right shift is, for example, x bits. In the case where there are two or more lists 204 of MIP modes, the number of bits 211 for each list of the two or more lists 204 of MIP modes is, for example, x bits for all of the matrix-based intra-prediction modes 2051-205 of the respective lists of MIP modes. n In other words, the number of bits 211 is at least the same for MIP modes associated with the same block size set. The right shift 209 is, for example, indicated by >> in the above-described equation (2), and is compared with the number of bits 211 d i and the device 54 performs matrix-based intra-prediction modes 2051-205. n The constraint that the number of bits 211 is equal to 211 is applied, and the following comparison constraint 2 is applied.

[0088] Matrix A i , i.e., the following constraints on the prediction matrix 19 and the parameter c in Eq. i and d i is desirable. 1. The range of matrix entries used for MIP is fixed, e.g., the entries of the prediction matrix 19 are represented by a fixed-point representation 190 of a predetermined bit depth 192. Thus, for example, a predefined non-negative integer μ 1,low , μ 1,up and μ 2,low , μ 2,up As a result, 2051 to 205 of MIP mode i n For each matrix A i , i.e., matrix entry a of the prediction matrix 19 k,l For each

number

number

number

number

number

number

number

[0089] Optionally, the following constraints may also be desirable: 3. Value c i , i.e., the offset to the entry of the prediction matrix 19 is 2051 to 205 for MIP mode i. n Therefore, there exists a positive integer c, and as a result, MIP mode i2051-205 n For each

number

[0090] The reason why we impose these constraints is as follows: All matrices (A i -c i ) have a common fixed bit depth, i.e., a predetermined bit depth 192, a specific multiplier adapted to that bit depth may be used for the calculation of the matrix-vector product 206, and all MIP modes 2051-205 n , the constraint 1 is the matrix-vector multiplication 206 (A i -c i ) allows for an efficient implementation of all matrices A i can be stored in a fixed precision, i.e., if the entries of the prediction matrix 19 are represented in a fixed point representation 190, then the matrix A i This allows for efficient memory management when dealing with all matrices A ican be stored in 8-bit precision, i.e., in one byte. In other words, the entries of the prediction matrix 19 may be represented in a fixed-point representation 190 having a predetermined bit-depth 192 which is 8 bits.

[0091] Constraint 2 allows for an efficient implementation of the shift in equation (2) since a single fixed shift operation can be implemented for the MIP if the shift value, i.e., the number of bits for the right shift 211, does not depend on the MIP mode i, saving a table lookup and benefiting from a compact SIMD implementation of equation (2), reducing the case-dependent implementation of equation (2) in hardware implementations, i.e., the matrix-vector product 206, which approximates the matrix-vector product by the prediction matrix in floating-point precision. Here, a particularly important example is that the value 6 is used as the fixed shift, i.e., the number of bits 211. The reason is that for 10-bit content, clipping to the 10-bit range reduces the pred red Therefore, before downshifting by 6, i.e., before performing the right shift 209 by the number of bits 211, which is 6 in equation (2), the term

number

number

[0092] According to an embodiment, the device 54 may, for example, calculate the intermediate result 108′ obtained by the matrix-vector product 206 for each component 210 of the output vector 208, e.g.,

number

[0093] According to an embodiment, the device decodes pictures at 10-bit resolution and uses matrix-based intra prediction modes 2051-2055. n For each matrix-based intra-prediction mode, the system is configured to store the order of the entries of the prediction matrix 19 associated with the respective matrix-based intra-prediction mode in 7-bit precision and use 6 bits as the number of bits 211 .

[0094] Similar to constraint 2), constraint 3) allows for a more efficient implementation of equation (2), again saving table lookups.

[0095] The problem that this application intends to solve is that it is not obvious how Constraint 2 or Constraint 3 can be satisfied together with Constraint 1 so that Equation (2) serves as an approximation for Equation (1).

[0096] For example, by constraint 1, each MIP matrix A i Constraint 2 requires that all matrix entries of i It will be assumed that needs to be used for all MIP modes i in equation (2). Also, for simplicity, c i = 0, which means that equation (2) is a floating-point matrix that is the result of the training algorithm for MIP.

number

number

number

number

number

number

number

number

[0097] On the other hand, a priori, the output is a MIP matrix

number

number

number

number

number

number

number

[0098] In fact, the trained matrix is ​​based on the MIP mode used in the current VVC draft [1].

number

number

[0099] Finally, the matrix that exists late in the MIP mode of the current VVC draft [1]

number

number

number

number

number

number

[0100] It is further contemplated that device 54 may include features and / or functionality as described with respect to Figures 6.1 and 6.2, such as matrix-based intra-prediction modes 2051-2055. n 6.1 includes one or more pairs 212 of matrix-based intra-prediction modes. It should be noted that list 204 may not consist exclusively of such pairs 212 of MIP modes as they are depicted in list 204 in FIG. 6.1; rather, there may also be other MIP modes that are applied exclusively using either the transposed or non-transposed option. nFor each pair 212 of matrix-based intra-prediction modes, the prediction matrix 19 associated with the first matrix-based intra-prediction mode of the respective pair 212 of matrix-based intra-prediction modes is equal to the prediction matrix 19 associated with the second matrix-based intra-prediction mode of the respective pair 212 of matrix-based intra-prediction modes, e.g., the same matrix 19 is used for modes 2k and 2k+1. The apparatus is configured such that, when the matrix-based intra-prediction mode 205 pointed to by the mode index 200 is the first matrix-based intra-prediction mode of the respective pair 212 of matrix-based intra-prediction modes, the association of the reference samples 17 neighboring the predetermined block with the components 214 of the input vector 112 and the association of the sample positions 104 of the predetermined block 18 with the components 210 of the output vector 208 are transposed relative to the association in the case when the matrix-based intra-prediction mode 205 pointed to by the mode index 200 is the second matrix-based intra-prediction mode of the respective pair 212 of matrix-based intra-prediction modes. That is, if a particular component of the input vector 102 is associated with a position (x,y) with (0,0) representing the upper left corner sample of the predetermined block 18 in the former case, it is associated with (x,y) in the latter case. The same applies to the components of the output vector 208. See the description of Figures 6.1 and 6.2 for further details.

[0101] It is further contemplated that device 54 may include features and / or functionality, for example, as described with respect to Figures 5.1-5.4.

[0102] According to an embodiment, for example as shown in one of Figures 5.1 to 5.4, the device 54 is configured to predict, by upsampling and / or interpolation, a sample of the predetermined block 18 that is offset from the sample position associated with a component 210 of the output vector 208 based on the output vector 208 or based on the output vector 208 and reference samples adjacent to the predetermined block 18.

[0103] According to an embodiment, for example as shown in one of Figures 5.1 to 5.4, the device 54 is configured to derive the input vector 102 from reference samples 17 adjacent to a predetermined block 18 by downsampling and / or pooling.

[0104] According to an embodiment, the reference samples 17 adjacent to the predetermined block 18 include a first reference sample 17c above the predetermined block 18 and a second reference sample 17a to the left of the predetermined block 18. The device 54 is configured to derive the input vector 102 from the reference samples 17 adjacent to the predetermined block 18 by deriving a first intermediate component from the first reference sample 17c by downsampling and / or pooling, deriving a second intermediate component from the second reference sample 17a by downsampling and / or pooling, concatenating the first intermediate component and the second intermediate component by downsampling and / or pooling to derive a preliminary input vector, and forming an input vector from the preliminary input vector.

[0105] According to an embodiment, the device 54 is configured to decode a picture at B-bit resolution. The device 54 performs a 2-bit subtraction from the first component of the preliminary input vector to obtain the first component of the input vector 102. B-1and by subtracting the first component of the preliminary input vector from the further component of the preliminary input vector to obtain a further component of the input vector 102. Alternatively, unit 54 may be configured to form input vector 102 from the preliminary input vector by subtracting the first component of the preliminary input vector from the further component of the preliminary input vector, such that input vector 102 is formed from the further component. In addition, unit 54 is configured to correct output vector 208 by component-wise addition of the first component of the preliminary input vector.

[0106] According to an embodiment, the matrix-based intra-prediction modes 2051 to 2055 in the list of matrix-based intra-prediction modes 204 n The entries of the prediction matrix correspond to the entries in Table 2 below, see, for example, Listing 2. However, perhaps a different shift value, i.e., a different number of bits 211, could be chosen to list the values ​​in the table, and perhaps the values ​​in the table could be expressed in a different scale.

[0107] The following embodiments focus on data-driven training of matrix-based intra-prediction modes with predefined fixed coefficient ranges and predefined fixed shifts, and their application in codecs.

[0108] The solution presented in this invention for the problem of obtaining a fixed bit depth, a fixed shift, and a fixed offset is to apply Constraint 1, Constraint 2, and Constraint 3 to the training of the MIP prediction mode, i.e., the matrix

number

[0109] The simplest way to do this is to use constraint 2. d Multiply each matrix by , then add the offset c from constraint 3 (if necessary), then clip the result to the desired range of constraint 1, then subtract the offset c, and finally 2 d However, this is not feasible because the clipping function has zero gradient outside the clipping range, so in such an approach, any weights that are outside the clipping range at any point in the stochastic gradient descent process will not be updated further.

[0110] Figure 8 shows the matrix-based intra prediction modes 2051 to 205 n List of 204 training prediction matrices 19 i 3 shows an embodiment of an apparatus 310 for matrix-based intra-prediction modes 2051-205 n , an input vector 102 derived from reference samples 17 adjacent to the predetermined block 18 and a matrix-based intra-prediction mode 205 selected for the predetermined block 18. i and the associated prediction matrix 19 i and associating the components 210 of the output vector 208 obtained by the matrix-vector product 206 with the sample locations 104 of the predetermined block 18, thereby predicting the samples 108 of the predetermined block 18. i will be selected.

[0111] The device 310 calculates the matrix-based intra prediction modes 2051-2055 using a steepest descent approach 322. n List of 204 prediction matrices 19 i The prediction matrix 19 is then trained (320). iis trained 320, for example, by using a training set of predetermined blocks 18 of known original samples and their corresponding neighborhoods 17. i is a differentiable function, f(x), where the representative values ​​are mapped to intermediate values ​​in the prediction matrix 19. i matrix-based intra-prediction modes 2051-205, expressed in floating-point representation, using a cost function 324 that depends on a prediction distortion measure 326 associated with setting the entries of n Representative values ​​for the entries of the prediction matrix 191 in the list 204, comparison

number

number

number

number

number

number

number

number

[0112] The domain of the differentiable function 328, e.g., the x-axis 302 in FIG. 9, and the co-domain, e.g., the y-axis in FIG. 9, are defined by a floating-point representation, the image 300 of the differentiable function 328 has a predetermined dynamic range, and the differentiable function 328 is used for matrix-based intra-prediction modes 2051-2055. n The predetermined dynamic range is defined, for example, by max(image 300) / min(image 300). According to an embodiment, max(image 300) is α+δ and min(image 300) is −α+δ, see equation (7). It should be noted that FIG. 9 only shows a graph of an example differentiable function f(x) 328.

[0113] Additionally, the device 310 may, for example, after training 320, quantize intermediate values ​​to a fixed point representation 190, resulting in matrix-based intra prediction modes 2051-2055. n For each matrix-based intra prediction mode 205 i and the associated prediction matrix 19 i has all entries represented by a fixed point representation 190 of a predetermined bit depth 192, so that the predetermined bit depth 192 is used for matrix-based intra prediction modes 2051-205. nAs a result, the matrix-based intra prediction modes 2051 to 205 n For each input vector 102 and the respective matrix-based intra prediction mode 205 i and the associated prediction matrix 19 are multiplied by the matrix-vector product 206 to produce matrix-based intra prediction modes 2051-205 for each component of the output vector 208. n For example, the fixed point representation of b y+x The portion not shifted out to b x+1 See equation (7) below, which is configured to be computable by performing a right shift 209 on the number of bits 211 meaning that α is sufficient to represent α. It will be noted that Figure 8 shows that the fixed point representation 190 is an (x+y+1)-bit signed digit representation. However, it is also possible for the fixed point representation 190 to have an (x+y)-bit digit representation, for example in cases where all intermediate values ​​have the same sign, and for example all intermediate values ​​can have positive values.

[0114] Therefore, as a solution, in the present invention, the clipping operation is approximated by a smoothing function, e.g., a differentiable function 328. More precisely, constraints 1, 2 and, optionally, 3 determine ranges for the unscaled matrix entries, i.e., representative values ​​for the entries of the prediction matrix, comparison

number

number

number

number

number

[0115] According to an embodiment, the differentiable function 328 has a slope of 1 at the origin, is strictly monotonically increasing, and has horizontal asymptote at the upper and lower limits of the image 300 .

[0116] According to an embodiment, the differentiable function 328 is parameterized by a shift parameter, e.g., δ, with respect to the shift of the image 300 within the codomain 304. Additionally, the device 310 may be configured to subject the shift parameter to optimization using a steepest descent approach 322. Furthermore, the device 310 may be configured to derive, for each matrix-based intra prediction mode, an offset value, c, from the shift parameter to be used to offset, e.g., by addition or subtraction, all entries of the prediction matrix 19 associated with the respective matrix-based intra prediction mode prior to calculation of the matrix-vector product 206. The derived offset value c is then used to offset all entries of the prediction matrix 19 associated with the respective matrix-based intra prediction mode, e.g., for each matrix-based intra prediction mode 2051-2055. n is equal to

[0117] In essence, the invention of this application provides a MIP realization having a portion given by equation (2) for which constraints 1 and 2 are satisfied, or for which constraints 1, 2, and 3 are satisfied, for which in both cases a floating-point matrix ∇ ...

number

[0118] The following embodiment describes an example of a stored representation of a prediction matrix.

[0119] The following listing shows the floating-point matrices resulting from training using the techniques provided in this application for the MIP mode used for mipSizeId=2, [1]:

number

[0120] The matrix coefficients shown in Listing 1 meet the requirements set out in the previous section. To illustrate this, Listing 2 shows the matrix coefficients after multiplication by 26. The range of those coefficients deviates from the final range by a fixed offset of 32. Thus, according to an example, the following values ​​are the matrix entries stored in fixed point representation: The matrix is ​​a 7x64 matrix (7 component input vectors and 64 component output vectors). There are 6 matrices for 6 modes. Depending on the embodiment, the entries may deviate from the values ​​shown below. For example, for the purposes of example only, 2 6 When multiplication by is chosen and a different factor is chosen accordingly, the matrix entries may look different as they are shown below. [Table 2-1] [Table 2-2] [Table 2-3] [Table 2-4] [Table 2-5] [Table 2-6] [Table 2-7] [Table 2-8] [Table 2-9] [Table 2-10] [Table 2-11]

[0121] Now, a fixed offset of 32 is added to those matrix coefficients, and the set of matrices results with all coefficients greater than or equal to -0.5, and therefore rounded to non-negative values. This set is shown in Listing 3. [Table 3-1] [Table 3-2] [Table 3-3] [Table 3-4] [Table 3-5] [Table 3-6] [Table 3-7] [Table 3-8] [Table 3-9] [Table 3-10] [Table 3-11]

[0122] Finally, the matrix coefficients are rounded to integer precision, i.e., intermediate values ​​are quantized 330 to a fixed-point representation 190. Because the smallest coefficient in the set is −0.5, the resulting integer coefficients are non-negative and therefore may be represented by unsigned integers. The largest coefficient in the matrix is ​​127.5. These coefficients would normally be rounded to 128, which introduces the same absolute rounding error, hence the resulting integer coefficients from the unsigned 7-bit range shown in Listing 4. Additionally, because the smallest rounded coefficient is 0 and the largest rounded coefficient is 127, the resulting integer coefficients fully utilize this range. [Table 4-1] [Table 4-2] [Table 4-3] [Table 4-4] [Table 4-5] [Table 4-6] [Table 4-7] [Table 4-8] [Table 4-9] [Table 4-10] [Table 4-11]

[0123] While some aspects have been described in the context of an apparatus, it is clear that those aspects also represent descriptions of corresponding methods, with blocks or devices corresponding to method steps or features of method steps. Similarly, aspects described in the context of a method step also represent descriptions of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps may be performed by (or using) a hardware apparatus such as a microprocessor, a programmable computer, or electronic circuitry. In some embodiments, one or more of the most significant method steps may be performed by such an apparatus.

[0124] The inventive data stream may be stored on a digital storage medium or transmitted over a transmission medium, such as a wireless or wired transmission medium, such as the Internet.

[0125] Depending on specific implementation requirements, embodiments of the invention may be implemented in hardware or software. Implementations may be performed using a digital storage medium, such as a floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM, or flash memory, having electronically readable control signals stored thereon that cooperate (or are capable of cooperating) with a programmable computer system so that the respective methods are performed. Thus, the digital storage medium may be computer-readable.

[0126] Some embodiments according to the invention include a data carrier having electronically readable control signals capable of cooperating with a programmable computer system to cause one of the methods described herein to be performed.

[0127] Generally, embodiments of the present invention may be implemented as a computer program product having program code operable to perform one of the methods when the computer program product is run on a computer, which may for example be stored on a machine-readable carrier.

[0128] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0129] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0130] A further embodiment of the inventive method is therefore a data carrier (or digital storage medium or computer readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, digital storage medium or recorded medium is typically tangible and / or non-transitory.

[0131] A further embodiment of the inventive method is, therefore, a data stream or sequence of signals representing the computer program for performing one of the methods described herein, the data stream or sequence of signals being for example adapted to be transmitted via a data communications connection, for example via the Internet.

[0132] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0133] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0134] Further embodiments according to the invention include an apparatus or system configured to transfer (e.g., electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, or a memory device. The apparatus or system may, for example, include a file server for transferring the computer program to the receiver.

[0135] In some embodiments, programmable logic circuitry (e.g., a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware apparatus.

[0136] The devices described herein may be implemented using a hardware device, or using a computer, or using a combination of a hardware device and a computer.

[0137] The apparatus described herein, or any components of the apparatus described herein, may be implemented at least partially in hardware and / or software.

[0138] The methods described herein may be performed using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0139] The methods described herein, or any components of the apparatus described herein, may be implemented at least in part in hardware and / or software.

[0140] The above-described embodiments are merely illustrative of the principles of the present invention. It will be understood that modifications and variations of the arrangements and details described herein will be apparent to those skilled in the art. It is therefore intended to be limited only by the scope of the appended claims, and not by the specific details presented in the description and discussion of the embodiments herein.

[0141] References [1]B.Bross et al.,Versatile Video Coding(Draft 7),Document JVET-P2001,Geneva,October 2019

Claims

1. 1. A method for encoding a block of a picture using intra prediction, comprising: The method comprises: encoding a mode index into the data stream indicating one matrix-based intra-prediction mode among a plurality of matrix-based intra-prediction modes associated with 6, 8, or 16 different matrices based on a size of the block; deriving an input vector based on downsampling of reference samples neighboring said block; determining a matrix based on the size of the block and the mode index; calculating, for each component of a matrix-vector product between the input vector and the determined matrix, a respective output calculated by performing a right shift by a number of bits independent of the matrix-based intra-prediction mode indicated by the mode index; for each element of the matrix-vector product, using each of the outputs to predict each sample of the block; Including, method.

2. The number of bits is 6. The method of claim 1.

3. each matrix-based intra prediction mode of the plurality applies a respective matrix having all entries represented by a respective 7-bit fixed-point representation; The method of claim 1.

4. further comprising encoding the picture at 10-bit resolution. The method of claim 1.

5. the reference samples adjacent to the block include samples above the block and samples to the left of the block; The method of claim 1.

6. Reference samples above and adjacent to the block are downsampled to form a first vector; Reference samples adjacent to the block to the left of the block are downsampled to form a second vector; the input vector is derived based on a concatenation of the first vector and the second vector. The method of claim 5.

7. 1. A method for decoding blocks of a picture using intra prediction, comprising: The method comprises: decoding, from the data stream, a mode index indicating one matrix-based intra-prediction mode from a plurality of matrix-based intra-prediction modes associated with 6, 8, or 16 different matrices based on a size of the block; deriving an input vector based on downsampling of reference samples neighboring said block; determining a matrix based on the size of the block and the mode index; calculating, for each component of a matrix-vector product between the input vector and the determined matrix, a respective output calculated by performing a right shift by a number of bits independent of the matrix-based intra-prediction mode indicated by the mode index; for each element of the matrix-vector product, using each of the outputs to predict each sample of the block; Including, method.

8. The number of bits is 6. The method of claim 7.

9. each matrix-based intra prediction mode of the plurality applies a respective matrix having all entries represented by a respective 7-bit fixed-point representation; The method of claim 7.

10. further comprising decoding the picture at 10-bit resolution. The method of claim 7.

11. the reference samples adjacent to the block include samples above the block and samples to the left of the block; The method of claim 7.

12. Reference samples above and adjacent to the block are downsampled to form a first vector; Reference samples adjacent to the block to the left of the block are downsampled to form a second vector; the input vector is derived based on a concatenation of the first vector and the second vector. The method of claim 11.

13. comprising at least one processor configured to carry out the method according to any one of claims 1 to 12, Video coding device.

14. comprising instructions which, when executed by at least one processor, perform the method of any one of claims 1 to 12, Non-transitory computer-readable medium.

Images