Method and apparatus for matrix-based intra prediction (MIP)

Abstract

A MIP method (700) for predicting samples of a current block. The method includes storing (s702) a set of MIP weight matrices. The method also includes determining (s704) the width W and height H of the current block. The method further includes setting (s706) the mipSizeId variable to 1 as a result of i) determining W = 4 and H = 16 or ii) determining W = 16 and H = 4. The method further includes storing (s708) a value predModeIntra that specifies the MIP prediction mode of the current block. The method further includes determining (s710) a modeId value based on the values of predModeIntra and the mipSizeId variable. And the method further includes selecting (s712) a MIP weight matrix to be used for the current block from the set of MIP weight matrices, where the selection is based on the values of modeId and the mipSizeId variable.

Description

Technical Field

[0001] The present disclosure relates to video coding and decoding. Background Art

[0002] 1. HEVC and VVC

[0003] High Efficiency Video Coding (HEVC) is a block-based video codec standardized by ITU-T and MPEG, which utilizes both temporal prediction and spatial prediction. Spatial prediction is achieved using intra prediction within the current picture. Temporal prediction is achieved using unidirectional (P) or bidirectional (B) inter prediction at the block level from previously decoded reference pictures. In the encoder, the difference between the original pixel data and the predicted pixel data (referred to as the residual) is transformed into the frequency domain, quantized, and then entropy coded, and then transmitted together with the necessary prediction parameters such as prediction mode and motion vectors (which are also entropy coded). The decoder performs entropy decoding, inverse quantization, and inverse transformation to obtain the residual, and then adds the residual to the intra or inter prediction to reconstruct the picture.

[0004] MPEG and ITU-T are studying the successor to HEVC in the Joint Video Exploration Team (JVET). The name of this video codec under development is Versatile Video Coding (VVC). At the time of writing, the current version of the VVC draft specification is "Versatile Video Coding (Draft 6)", JVET-O2001-vE. When VVC is mentioned in this document, it refers to Draft 6 of the VVC specification.

[0005] 2. Components

[0006] A video sequence consists of a series of pictures, where each picture consists of one or more components. Each component can be described as a two-dimensional rectangular array of sample values. Typically, the pictures in a video sequence consist of three components: a luminance component (Y), where the sample values are luminance values; and two chrominance components (Cb) and (Cr), where the sample values are chrominance values. Typically, the dimension of the chrominance components is 1 / 2 of the luminance component in each dimension. For example, the size of the luminance component of an HD picture will be 1920×1080, and the chrominance components will each have a size of 960×540. Components are sometimes referred to as color components. In this document, we describe methods useful for encoding and decoding video sequences. However, it should be understood that the described techniques can also be used for encoding and decoding still images.

[0007] 3. Blocks and Units

[0008] A block is a two-dimensional array of samples. In video coding, each component is split into one or more blocks, and the coded video bitstream is a sequence of blocks.

[0009] In video coding, typically, a picture is split into units that cover a specific area. Each unit consists of all the blocks that make up that specific area, and each block belongs entirely to only one unit. Coding units (CUs) in HEVC and VVC are examples of such units. A coding tree unit (CTU) is a logical unit that can be split into several CUs.

[0010] In HEVC, CUs are square, i.e., they have a size of N×N luminance samples, where N can have a value of 64, 32, 16, or 8. In the current H.266 test model Versatile Video Coding (VVC), CUs can also be rectangular, i.e., have a size of N×M luminance samples, where N is different from M.

[0011] 4. Intra prediction

[0012] There are two types of sample prediction: intra prediction and inter prediction. Intra prediction predicts a block based on the spatial extrapolation of samples from previously decoded blocks in the same (current) picture. It can also be used in image compression, i.e., the compression of still images where only one picture is to be compressed / decompressed. Inter prediction predicts a block by using samples from previously decoded pictures.

[0013] 5. Intra direction prediction

[0014] Intra direction prediction is utilized in HEVC and VVC. In HEVC, there are a total of 35 modes and 33 angular modes. In VVC, there are a total of 67 modes and 65 angular modes. The remaining two modes, "Planar" and "DC", are non-angular modes. Mode index 0 is used for the Planar mode, while mode index 1 is used for the DC mode. The angular prediction mode indices vary from 2 to 34 for HEVC and from 2 to 66 for VVC.

[0015] Intra direction prediction is used for all components in a video sequence, such as the luminance component Y and the two chrominance components Cb and Cr.

[0016] 6. Matrix-based intra prediction

[0017] Matrix-based intra prediction (MIP) is a coding tool that is included in the current version of the VVC draft. To predict the samples of a current block of width W and height H, MIP takes as input a column of H reconstructed neighboring boundary samples to the left of the current block and a row of W reconstructed neighboring samples above the current block. The predicted samples of the current block are derived based on the following three steps:

[0018] (Step 1) For both adjacent row and column boundaries, two or four samples are extracted by averaging the samples of each boundary (bdry 上 and bdry 左 ) using an averaging method that depends on the current block size. The averaged boundary samples extracted are named the reduced boundary bdry red .

[0019] (Step 2) Matrix-vector multiplication is performed using the averaged boundary samples extracted as input. The output is a reduced prediction signal consisting of a set of predicted sample values, where each predicted sample corresponds to a position in the current block, and where the set of positions is a subset of all positions of the current block. The output reduced prediction signal is named pred red .

[0020] (Step 3) The predicted sample values for the remaining positions in the current block that are not in the set of positions are generated by linear interpolation based on the reduced prediction signal, where the linear interpolation is a single-step linear interpolation in each direction (vertical and horizontal). The prediction signal consists of all the predicted sample values of the block. The order of interpolation depends on the relative width (W) and height (H) of the block.

[0021] If H > W, horizontal linear interpolation is first applied by using the reduced left boundary samples, which are named bdry red 左 or bdry redII 左 depending on the current block size. After horizontal linear interpolation, vertical linear interpolation is applied by using the original upper boundary bdry 上 . Depending on the block size, horizontal and / or vertical linear interpolation may not be performed on the block.

[0022] If H ≤ W, vertical linear interpolation is first applied by using the reduced upper boundary samples, which are named bdry red 上 or bdry redII 上 depending on the current block size. After vertical linear interpolation, horizontal linear interpolation is applied by using the original left boundary bdry 左 . Depending on the block size, horizontal and / or vertical linear interpolation may not be performed on the block.

[0023] For a 4×4 block, bdry red contains 4 samples, which result from averaging every two samples of each boundary. pred red is of size 4×4, which is the same as the current block. Thus, horizontal and vertical linear interpolation may be skipped. Figure 4AShows an example of the MIP process for a 4×4 block.

[0024] Given an 8×4 block, bdry red contains 8 samples that are derived from the original left boundary and averaging every two samples of the upper boundary. pred red is of size 4×4. The prediction signal at the remaining positions is generated by using the original left boundary bdry 左 according to horizontal linear interpolation. Figure 4B Shows an example of the MIP process for an 8×4 block.

[0025] Given a W×4 block, where W≥16, bdry red contains 8 samples that are derived from the original left boundary and averaging every W / 4 samples of the upper boundary. pred red is of size 8×4. The prediction signal at the remaining positions is generated by using the original left boundary bdry 左 according to horizontal linear interpolation.

[0026] Given a 4×8 block, bdry red contains 8 samples that are derived from averaging every two samples of the left boundary and the original upper boundary. pred red is of size 4×4. The prediction signal at the remaining positions is generated by using the original upper boundary bdry 上 according to vertical linear interpolation.

[0027] Given a 4×H block, where H≥16, bdry red contains 8 samples that are derived from averaging every H / 4 samples of the left boundary and the original upper boundary. pred red is of size 4×8. The prediction signal at the remaining positions is generated by using the original upper boundary bdry 上 according to vertical linear interpolation. Figure 5A Shows an example of the MIP process for a 4×16 block.

[0028] Given an 8×8 block, bdry red contains 8 samples that are derived from averaging every two samples of each boundary. pred red is of size 4×4. The prediction signal at the remaining positions is first generated by using the simplified upper boundary bdry red 上 according to vertical linear interpolation, and second by using the original left boundary bdry 左 according to horizontal linear interpolation. Figure 5B Shows an example of the MIP process for an 8×8 block.

[0029] Given a W×8 block, where W≥16, bdry red contains 8 samples that are derived from averaging every two samples of the left boundary and averaging every W / 4 samples of the upper boundary. pred red is of size 8×8. The prediction signal at the remaining positions is generated by using the original left boundary bdry 左 and performing horizontal linear interpolation. Figure 6A Shows an example of the MIP process for a 16×8 block.

[0030] Given an 8×H block, where H≥16, bdry red contains 8 samples that are derived from averaging every H / 4 samples of the left boundary and averaging every two samples of the upper boundary. pred red is of size 8×8. The prediction signal at the remaining positions is generated by using the original upper boundary bdry 上 and performing vertical linear interpolation.

[0031] Given a W×H block, where W≥16 and H≥16, bdry red contains 8 samples that are derived as follows:

[0032] (1) For H≤W, first, bdry redII 上 contains 8 samples that are derived by averaging every W / 8 samples of the upper boundary. Second, bdry red contains 8 samples that are derived from averaging every H / 4 samples of the left boundary and averaging every two samples of bdry redII 上 .

[0033] (2) For H>W, first, bdry redII 左 contains 8 samples that are derived by averaging every H / 8 samples of the left boundary. Second, bdry red contains 8 samples that are derived from averaging every two samples of bdry redII 左 and every W / 4 samples of the upper boundary.

[0034] pred red is of size 8×8. The prediction signal at the remaining positions is generated by using linear interpolation as follows:

[0035] (1) For H≤W, first, by using the simplified upper boundary samples bdry redII 上 derived by averaging every W / 8 samples of the upper boundaryVertical linear interpolation, and secondly, by using the original left boundary bdry 左 Horizontal linear interpolation.

[0036] (2) For H > W, first, by using the simplified left boundary samples bdry derived by averaging every H / 8 samples of the upper boundary redII 左 Horizontal linear interpolation, and secondly, by using the original upper boundary bdry 上 Vertical linear interpolation.

[0037] Figure 6B Shows an example of the MIP process for a 16×16 block.

[0038] In the current version of VVC, MIP is only applicable to the luminance component.

[0039] In the current version of VVC, given a W×H block, when the W / H or H / W ratio is equal to or less than 4, MIP can be applied to the current block. In other words, for blocks with dimensions of 4×32, 32×4, 4×64, 64×4, 8×64, or 64×8, MIP is disabled.

[0040] 7. mipSizeId mipSizeId is a variable used to determine the number of input samples for the simplified boundary, the number of output samples for the simplified prediction, and the MIP weight matrix to be used for the current block.

[0041] In the current version of VVC, mipSizeId is determined according to the size of the current block. Given a W×H block, mipSizeId is determined as follows: If both W and H are equal to 4, then mipSizeId is set to be equal to 0; if both W and H are not equal to 4, but both W and H are less than or equal to 8, then mipSizeId is set to 1; otherwise, mipSizeId is set to 2.

[0042] 8. MIP Weight Matrix and MIP Prediction Mode

[0043] The MIP prediction mode is used in the derivation process for selecting the MIP weight matrix from the matrix lookup table. The MIP prediction mode can be determined according to the decoded bitstream. In the current version of VVC, the number of MIP prediction modes (or simply "MIP modes") is specified by mipSizeId as follows: If mipSizeId is equal to 0, then the number of MIP prediction modes numModes is 35; if mipSizeId is equal to 1, then numModes is 19; and if mipSizeId is equal to 2, then numModes is 11. This is illustrated in Table 8-4 of VVC, which is reproduced below as Table 0:

[0044] Table 0

[0045] mipSizeId numModes boundarySize predW predH predC 0 35 2 4 4 4 1 19 4 4 4 4 2 11 4 Min(nTbW, 8) Min(nTbH, 8) 8

[0046] In the current version of VVC, when the value of W / H is greater than 4 or the value of H / W is greater than 4 (i.e., if one dimension is more than 4 times larger than the other dimension, MIP is not allowed and numModes is zero), MIP is not allowed for the current block.

[0047] The MIP weight matrix to be used for MIP prediction is derived from the matrix lookup table using the MIP size Id value mipSizeId and the mode Id value modeId. For example, when mipSizeId is equal to 0 and modeId is equal to 0, the MIP weight matrix size M×N (where M is equal to 16 and N is equal to 4) is derived as:

[0048] Table 1

[0049]

[0050] modeId is determined based on the determined MIP prediction mode (denoted as predModeIntra) and numModes. Specifically, according to VVC:

[0051] modeId = predModeIntra - (isTransposed? numModes / 2 : 0)

[0052] where:

[0053] isTransposed = (predModeIntra > (numModes / 2))? TRUE : FALSE.

[0054] The MIP weight matrix is a two-dimensional matrix. The size of the MIP weight matrix (or simply "MIP matrix") can be expressed as M×N, where N is equal to bdry red (the number of input samples of MIP INPUT), and M is equal to or greater than pred red (the number of output samples of MIP OUTPUT). The multiplication of the MIP weight matrix and the matrix of the input vector produces a vector of M samples, which are spatially located in a square matrix of size predC, where M = predC×predC.

[0055] MIP OUTPUT is a two-dimensional matrix with dimensions of predW×predH. The size of MIP OUTPUT can be expressed as predW×predH.

[0056] The sizes of MIP INPUT and MIP OUTPUT depend on mipSizeId. Table 2 shows the sizes of MIP INPUT, MIP OUTPUT, and the MIP weight matrix for each mipSizeId:

[0057] Table 2

[0058] The sizes of MIP INPUT, MIP OUTPUT, and the MIP weight matrix for each mipSizeId SUMMARY OF THE INVENTION

[0059] There are certain challenges. For example, in the current version of VVC, the MIP process for 16×4 or 4×16 blocks is inconsistent with the MIP process for other blocks. The handling of matrix multiplication with the MIP weight matrix mWeight[M][N] (where M is greater than MIP OUTPUT) increases the complexity of the MIP process.

[0060] The present disclosure simplifies the MIP process such that for all MIP decoded blocks, the selected MIP weight matrix mWeight[M][N] to be used for matrix multiplication is such that M is equal to the size of MIP OUTPUT. In other words, the size of MIP OUTPUT is determined to be equal to the size of one dimension (M) of the selected MIP weight matrix. This simplification eliminates the handling of matrix multiplication with a MIP weight matrix (mWeight[M][N]) where M is greater than the size of MIP OUTPUT. This simplification makes the matrix multiplication process consistent for all MIP decoded blocks. Thus, one advantage is a reduction in the decoding complexity of the MIP process. This is accomplished by eliminating the handling of matrix multiplication with a MIP weight matrix (where M is greater than the size of MIP OUTPUT). This simplification makes the matrix multiplication process consistent for all MIP decoded blocks.

[0061] Thus, in one aspect, there is provided a MIP method for predicting samples of a current block. In one embodiment, the method includes storing a set of MIP weight matrices. The method further includes determining a width W and a height H of the current block. The method further includes setting the mipSizeId variable to 1 as a result of i) determining that W = 4 and H = 16 or ii) determining that W = 16 and H = 4. The method further includes storing a value predModeIntra that specifies the MIP prediction mode of the current block. The method further includes determining a modeId value based on the values of predModeIntra and the mipSizeId variable. And the method further includes selecting an MIP weight matrix to be used for the current block from the set of MIP weight matrices, wherein the selection is based on the values of modeId and the mipSizeId variable. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 FIG. illustrates a system according to an embodiment.

[0063] Figure 2 is a schematic block diagram of a video encoder according to one embodiment.

[0064] Figure 3 is a schematic block diagram of a video decoder according to one embodiment.

[0065] Figure 4A FIG. shows an example of the MIP process for a 4×4 block.

[0066] Figure 4B FIG. shows an example of the MIP process for an 8×4 block.

[0067] Figure 5A FIG. shows an example of the MIP process for a 4×16 block.

[0068] Figure 5B FIG. shows an example of the MIP process for an 8×8 block.

[0069] Figure 6A FIG. shows an example of the MIP process for a 16×8 block.

[0070] Figure 6B FIG. shows an example of the MIP process for a 16×16 block.

[0071] Figure 7 is a flowchart illustrating a process according to an embodiment.

[0072] Figure 8 is a flowchart illustrating a process according to an embodiment.

[0073] Figure 9 is a block diagram of a device according to an embodiment.

[0074] Figure 10 It is a block diagram of a device according to an embodiment. Detailed implementation

[0075] The term "sample" can be interpreted as "sample value". For example, the statement "Derive X from Y samples" can be interpreted as "Derive X from Y sample values". Similarly, the statement "X samples are derived from Y" can be interpreted as "X sample values are derived from Y". The term "MIP INPUT" can be interpreted as "extracted simplified boundary bdry red , which is used as the input for matrix multiplication". The term "MIP OUTPUT" can be interpreted as "simplified prediction signal pred red , which is the output of matrix multiplication".

[0076] Figure 1 FIG. illustrates a system 100 according to an example embodiment. The system 100 includes an encoder 202 that communicates with a decoder 204 via a network 110 (e.g., the Internet or other network). Deblocking can be performed in both the encoder 202 and the decoder 204. The embodiments described herein can be used in a video encoder 102 or a video decoder 104.

[0077] Figure 2FIG. 0 is a schematic block diagram of a video encoder 102 according to an embodiment. Motion estimation is performed to predict a current pixel block by using a motion estimator 250 from a pixel block already provided in the same frame or a previous frame. In the case of inter-frame prediction, the result of the motion estimation is a motion or displacement vector associated with a reference block. The motion vector can be used by a motion compensator 250 to output an inter-frame prediction of the pixel block. An intra-frame predictor 249 calculates an intra-frame prediction of the current pixel block. The outputs from the motion estimator / compensator 250 and the intra-frame predictor 249 are input into a selector 251 that selects an intra-frame prediction or an inter-frame prediction for the current pixel block. The output from the selector 251 is input into an error calculator in the form of an adder 241 that also receives the pixel values of the current pixel block. The adder 241 calculates and outputs a residual as the difference in pixel values between the pixel block and its prediction. The error is transformed in a transformer 242, such as by a discrete cosine transform, and quantized by a quantizer 243, followed by encoding in an encoder 244, such as by an entropy encoder. In inter-frame encoding, the estimated motion vector is also taken to the encoder 244 to generate an encoded representation of the current pixel block. The transformed and quantized residual of the current pixel block is also provided to an inverse quantizer 245 and an inverse transformer 246 to retrieve the original residual. The error is added by an adder 247 to the block prediction output from the motion compensator 250 or the intra-frame predictor 249 to create a reference pixel block that can be used in the prediction and encoding of the next pixel block. This new reference block is first processed by a deblocking filter 200. The processed new reference block is then temporarily stored in a frame buffer 248 where it is available to the intra-frame predictor 249 and the motion estimator / compensator 250.

[0078] Figure 3is a block diagram of a video decoder 104 according to some embodiments. The decoder 104 includes a decoder 361, such as an entropy decoder, for decoding an encoded representation of a pixel block to obtain a set of quantized and transformed residuals. These residuals are dequantized by an inverse quantizer 362 and inverse-transformed by an inverse transformer 363 to provide a set of residuals. These residuals are added by an adder 364 to the pixel values of a reference pixel block. The reference block is determined by a motion estimator / compensator 367 or an intra predictor 366, depending on whether inter prediction or intra prediction is being performed. A selector 368 is thus interconnected with the adder 364, the motion estimator / compensator 367, and the intra predictor 366. The resulting decoded pixel block output from the adder 364 is input to a deblocking filter 300. The filtered pixel block is output from the decoder 104 and may be further temporarily provided to a frame buffer 365 to be used as a reference pixel block for subsequent pixel blocks to be decoded. The frame buffer 365 is thus connected to the motion estimator / compensator 367 to make the stored pixel blocks available to the motion estimator / compensator 367. The output from the adder 364 may also be input to the intra predictor 366 to be used as an unfiltered reference pixel block.

[0079] In the current version of VVC, the size of the MIP weight matrix is M×N, where M specifies the number of matrix multiplication samples. M is equal to or greater than the size of MIP OUTPUT.

[0080] As shown in Table 2, when mipSizeId is equal to 0 or 1, then M is equal to the size of MIP OUTPUT. (M = predC×predC = predW×predH = 16).

[0081] Also as shown in Table 2, when mipSizeId is equal to 2 and min(W,H) is equal to or greater than 8, M is equal to the size of MIP OUTPUT. (M = predC×predC = predW×predH = 64).

[0082] As further shown in Table 2, when mipSizeId is equal to 2 and min(W,H) is less than 8, M is greater than the size of MIP OUTPUT.

[0083] Specifically, given a W×H block, M is greater than the MIP size of a 16×4 or 4×16 block, and M is equal to the MIP size of all other W×H blocks.

[0084] When M is greater than the size of MIP OUTPUT, a subset of the matrix multiplication output is used as MIP OUTPUT, and the remainder of the matrix multiplication output is discarded.

[0085] In the current version of VVC, given a 16×4 or 4×16 block, with mipSizeId equal to 2, the size of the MIP OUTPUT is 32, and the size of the MIP weight matrix is M×N, where M is equal to 64 and N is equal to 8. The size of the matrix multiplication output is 64, and the subset of the output with odd indices is used as the MIP OUTPUT for the current block, while the remaining part of the output with even indices is discarded.

[0086] In other words, given a 16×4 or 4×16 block, the MIP OUTPUT is calculated as follows:

[0087] where k ranges from 0 to Varying this, M and N specify the size of the MIP weight matrix mWeight[M][N], where M is equal to 64 and N is equal to 8.

[0088] Compared with the matrix multiplication of a block with mWeight[M][N] where M is equal to the size of the MIP OUTPUT, the MIPOUTPUT is calculated as follows:

[0089] where k ranges from 0 to Varying

[0090] where M and N specify the size of mWeight[M][N].

[0091] As pointed out in the Summary of the Invention section, in the current version of VVC, the MIP process for 16×4 or 4×16 blocks is not consistent with the MIP process for other blocks. The handling of matrix multiplication with a MIP weight matrix mWeight[M][N] (where M is greater than the MIP OUTPUT) increases the complexity of the MIP process.

[0092] The present disclosure provides a process 700 for video encoding or decoding of current intra prediction blocks, which reduces the MIP process complexity. Process 700 is preferably applicable to blocks decoded by MIP.

[0093] Process 700 includes step s702, which includes storing a set of MIP weight matrices. An example set of weight matrices is identified in Section 8.4.5.2.3 in VVC and is reproduced in the appendix section of this document. The set of MIP weight matrices can be divided into three different groups (or subsets), where each group of matrices is associated with a different mipSizeId value. For example, each of the three different mipSizeId values identifies one of the groups.

[0094] Step s704 includes determining the width (W) and height (H) of the current block. W and H are determined from the syntax elements in the decoded bitstream.

[0095] Step s706 includes setting the value of variable mipSizeId based on W and H. In one embodiment, as a result of determining that WxH is greater than T1 (e.g., 16) but less than or equal to T2 (e.g., 64), mipSizeId is set to be equal to 1. In some embodiments, if WxH equals 16, mipSizeId is set to 0, and if WxH is greater than 64, mipSizeId is set to 2. In some embodiments, the following pseudocode can be used to set mipSizeId:

[0096]

[0097] In some embodiments, determining whether WxH is less than or equal to T2 (e.g., 64) includes determining whether WxH is less than (2×T2) (e.g., 128).

[0098] Step s708 includes determining the MIP prediction mode of the current block and storing the value predModeIntra that specifies the determined MIP prediction mode.

[0099] Step s710 includes determining the modeId value to be used with the mipSizeId value to select a matrix from a set of matrices based on predModeIntra and mipSizeId.

[0100] Step s712 includes selecting the MIP weight matrix to be used for the current block from a set of MIP weight matrices, where the selection is based on modeId and mipSizeId. For example, a MIP weight matrix lookup table that maps different mipSizeId-modeId pairs to different MIP weight matrices can be used to select the MIP weight matrix corresponding to the mipSizeId-modeId pair.

[0101] Process 700 selects the MIP weight matrix to be applied to the current block such that the number of samples of the MIP OUTPUT is equal to the number of samples of the matrix multiplication output of the MIP weight matrix and the MIP INPUT, e.g., the one-dimensional size (M) of the selected MIP weight matrix mWeight[M][N].

[0102] In some embodiments, the MIP weight matrix lookup table is the same as the lookup table in the current version of VVC.

[0103] In other words, the MIP weight matrix to be used for the current block is one of the following matrices:

[0104]

[0105] The MIP OUTPUT size is equal to predW × predH. The MIP OUTPUT size is equal to the size of one dimension (M) of the selected MIP weight matrix mWeight[M][N], where M is equal to predC × predC. Since the output of MIP matrix multiplication produces a square matrix. predW and predH are derived as:

[0106]

[0107] In both the horizontal and vertical dimensions, the MIP OUTPUT is equal to or smaller than the current block. Given a W × H block, predW is equal to or smaller than W, and predH is equal to or smaller than H.

[0108] predW = predH = predC ≤ minimum(W, H) (5-2-2)

[0109] The MIP weight matrix is derived from the mipSizeId and the MIP prediction mode. The mipSizeId is determined based on the width and height of the current block.

[0110] In this embodiment, Equation 5-2-2 is a criterion for determining the mipSizeId of the current block:

[0111] mipSizeId M predC minimum(W, H) 0 16 4 4 1 16 4 4 2 64 8 8

[0112] Given a W × 4 or 4 × H block, where W is equal to or greater than 4 and H is equal to or greater than 4, according to Equation 5-2-2, M is equal to or smaller than 16. Therefore, either mipSizeId 0 or mipSizeId 1 can be set for the current block.

[0113] Given a W × 8 or 8 × H block, where W is greater than or equal to 8 and H is greater than or equal to 8, according to Equation 5-2-2, M is less than or equal to 64. Therefore, either mipSizeId 0 or mipSizeId 1 or mipSizeId 2 can be set for the current block.

[0114] An example of determining the mipSizeId is as follows:

[0115]

[0116] Considering that the computational complexity (total number of multiplications / current block size) should be less than a threshold. Given a 4 × 4 block, the mipSizeId is set to 0, where the matrix selected from mipSizeId 0 gives the minimum computational complexity. In the current version of VVC, the total number of multiplications required in the calculation of matrix-vector products is always less than or equal to (4 × W × H). In other words, for the blocks decoded by MIP, at most four multiplications are required for each sample.

[0117] Given a 16×4 or 4×16 block, mipSizeId is set to 1, while in the current version of VVC, mipSizeId is set to 2.

[0118] The processes described herein can be applied in an encoder and / or decoder of a video or image coding system. For example, a decoder may perform all or a subset of the following steps of process 800 (see Figure 8 ) to decode an intra prediction block in a picture of a coded video bitstream.

[0119] Step s802 includes deriving, by decoding a syntax element in the bitstream, that the size of the current block is a width value W and a height value H.

[0120] Step s804 includes determining, based on an element in the coded bitstream, that the current block is an intra prediction block and that the current block is a matrix-based intra prediction (MIP) block.

[0121] Step s806 includes determining the MIP prediction mode of the current block according to the decoded elements in the bitstream.

[0122] Step s808 includes deriving a MIP size id value mipSizeId from the width W and the height H. For example, in one embodiment, step s808 includes calculating A = W×H, and then determining whether A is less than 32. If A is less than 32 (or equal to or less than 16), then mipSizeId is set to be equal to 0, otherwise it is determined whether A is less than 128 (or equal to or less than 64). If A is less than 128, then mipSizeId is set to 1; otherwise, mipSizeId is set to 2.

[0123] Step s810 includes determining a mode id value modeId based on the determined MIP prediction mode and mipSizeId.

[0124] Step s812 includes selecting a MIP weight matrix for the current block, where the matrix is selected based on the value of modeId and the value of mipSizeId. For example, a MIP weight matrix lookup table that maps different mipSizeId-modeId pairs to different MIP weight matrices can be used to select the MIP weight matrix corresponding to the mipSizeId-modeId pair.

[0125] Step s814 includes determining the original boundary sample values of the current block. The original boundary samples are W samples from the nearest adjacent samples above the current block and H samples from the nearest adjacent samples to the left of the current block.

[0126] Step s816 includes determining the size of the simplified boundary bdry based on the mipSizeId value of the current block. red

[0127] Step s818 includes determining the size of the simplified prediction signal pred based on the mipSizeId value of the current block. red

[0128] Step s819 includes deriving the simplified boundary bdry from the original boundary samples. red

[0129] Step s820 includes deriving the simplified prediction signal pred through matrix multiplication of the MIP weight matrix and the simplified boundary bdry. red red temp

[0130] Step s822 includes deriving the simplified prediction signal pred by using sample value clipping for each sample of pred. red temp red

[0131] Step s824 includes determining whether to i) apply vertical linear interpolation to the simplified prediction signal pred based on the width W and height H of the current block, and ii) apply horizontal linear interpolation to the simplified prediction signal pred based on the width W and height H of the current block. If it is decided to apply both vertical and horizontal linear interpolation, process 800 proceeds to step s826. red red

[0132] Step s826 includes determining whether to apply vertical linear interpolation before horizontal linear interpolation or horizontal linear interpolation before vertical linear interpolation based on the width W and height H of the current block.

[0133] If it is decided to apply vertical linear interpolation first, process 800 proceeds to step s828, otherwise it proceeds to step 832.

[0134] Step s828 includes determining the size of the simplified upper boundary bdry of vertical linear interpolation based on the width W and height H of the current block. Step s830 includes deriving the simplified upper boundary bdry from the original upper boundary samples. redII 上 redII 上

[0135] Step s832 includes determining the size of the simplified left boundary bdry of horizontal linear interpolation based on the width W and height H of the current block. redII ​​​​​​​​​​​左 The size. Step s834 includes deriving a simplified left boundary bdry from the original left boundary samples redII 左 .

[0136] Step s836 includes deriving the MIP prediction block pred by generating sample values at the remaining positions using linear interpolation.

[0137] Step s838 includes decoding the current block by using the derived MIP prediction block.

[0138] Examples of changes to the current VVC draft text

[0139] The following are the changes proposed to the current VVC draft text (refer to JVET-O2001-vE) for the MIP process for one embodiment. The text between the characters “[[” and “]]” is to be deleted, and the underlined text is to be added.

[0140]

[0141]

[0142]

[0143]

[0144]

[0145]

[0146] In some embodiments, the MIP weight matrix lookup table is different from the current version of the VVC. In this embodiment, different sets of mipSizeId values are associated with the numModes values. In one example, increasing the number of MIP prediction modes will increase the number of MIP weight matrices in the lookup table, which can improve the prediction accuracy. Decreasing the number of MIP prediction modes will reduce the number of matrices in the lookup table, which can reduce the memory usage of the codec.

[0147] Another example is that when 4x16 and 16x4 also belong to Group 1 and Group 2 does not have 4x16 and 16x4, the matrix element values can be updated by retraining the MIP matrix model for Group 1. The MIP matrix when MipSizeId is equal to 1 in the current VVC is trained by using block shapes 8×4, 4×8, and 8×8. When MipSizeId is equal to 1, a more appropriate way (or more consistent training) can be performed by deriving the matrix using block shapes 8x4, 4x8, 8x8, 4x16, and 16x4 during the matrix training process.

[0148] In another embodiment, MIP is not applicable to 4×16 and 16×4 blocks, e.g., blocks where the sum of the dimensions equals 20.

[0149] In another embodiment, for a block where W = 16 and H = 4, the mipSizeId is determined in a conventional manner (i.e., mipSizeId is set to be equal to 2), but for a block where W = 4 and H = 16, the mipSizeId is determined as explained with respect to step s706 (i.e., mipSizeId is set to be equal to 1).

[0150] In another embodiment, for a block where W = 4 and H = 16, the mipSizeId is determined in a conventional manner (i.e., mipSizeId is set to be equal to 2), but for a block where W = 16 and H = 4, the mipSizeId is determined as explained with respect to step s706 (i.e., mipSizeId is set to be equal to 1).

[0151] Figure 9 is a block diagram of a device 900 for implementing the video encoder 102 or the video decoder 104 according to some embodiments. As Figure 9As shown, device 900 may include: a processing circuit (PC) 902, which may include one or more processors (P) 955 (e.g., general-purpose microprocessors and / or one or more other processors such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), etc.), which may be co-located in a single housing or a single data center or may be geographically distributed (i.e., device 900 may be a distributed computing device); a network interface 948, which includes a transmitter (Tx) 945 and a receiver (Rx) 947 for enabling device 900 to transmit data to and receive data from other nodes, the other nodes being connected to a network 110 (e.g., an Internet Protocol (IP) network), the network 110 being (directly or indirectly) connected to network interface 948 (e.g., network interface 948 may be wirelessly connected to network 110, in which case network interface 948 is connected to an antenna arrangement); and a local storage unit (also referred to as a “data storage system”) 908, which may include one or more non-volatile storage devices and / or one or more volatile storage devices. In embodiments in which PC 902 includes a programmable processor, a computer program product (CPP) 941 may be provided. CPP 941 includes a computer-readable medium (CRM) 942 storing a computer program (CP) 943, CP 943 including computer-readable instructions (CRI) 944. CRM 942 may be a non-transitory computer-readable medium such as a magnetic medium (e.g., a hard disk), an optical medium, a memory device (e.g., a random-access memory, a flash memory), etc. In some embodiments, CRI 944 of computer program 943 is configured such that when executed by PC 902, the CRI causes device 900 to perform the steps described herein (e.g., the steps described herein with reference to the flowcharts). In other embodiments, device 900 may be configured to perform the steps described herein without code. That is, for example, PC 902 may consist of only one or more ASICs. Thus, the features of the embodiments described herein may be implemented in hardware and / or software.

[0152] Figure 10 Functional units of a device 1000 according to an embodiment are illustrated.

[0153] Overview of various embodiments

[0154] A1. A matrix-based intra prediction (MIP) method for predicting samples of a current block, the method comprising: storing a set of MIP weight matrices; determining the width (W) and height (H) of the current block; setting the mipSizeId variable to a certain value based on W and H, wherein setting the mipSizeId to a certain value includes, as a result of a determination condition being true, setting the mipSizeId to 1, wherein the determination condition being true includes determining that WxH is greater than T1 (e.g., 16) but less than or equal to T2 (e.g., 64); storing a value predModeIntra that specifies the MIP prediction mode of the current block; determining a modeId value based on predModeIntra and mipSizeId; and selecting an MIP weight matrix to be used for the current block from the set of MIP weight matrices, wherein the selection is based on modeId and mipSizeId.

[0155] A2. The method of embodiment A1, further comprising: storing a look-up table that maps different mipSizeId-modeId pairs to different MIP weight matrices, wherein selecting the MIP weight matrix includes using the look-up table to select the MIP weight matrix.

[0156] A3. The method of embodiment A1 or A2, wherein determining that WxH is greater than T1 (e.g., 16) but less than or equal to T2 (e.g., 64) includes: i) determining that W = 16 and H = 4 or ii) determining that W = 4 and H = 16.

[0157] A4. The method of any one of embodiments A1 - A3, wherein setting the mipSizeId to a certain value includes, as a result of i) determining that W = 4 and H = 16 or ii) determining that W = 16 and H = 4, setting the mipSizeId to 1.

[0158] A5. The method of any one of embodiments A1 - A4, wherein setting the mipSizeId to a certain value includes, as a result of determining that WxH is greater than T2, setting the mipSizeId to 2.

[0159] A6. The method of any one of embodiments A1 - A5, further comprising: determining the original boundary sample values of the current block, wherein the original boundary samples are W samples from the nearest adjacent samples above the current block and H samples from the nearest adjacent samples to the left of the current block.

[0160] A7. The method of embodiment A6, further comprising: determining the size of a simplified boundary bdry based on the mipSizeId value of the current block; determining a simplified prediction signal pred based on the mipSizeId value of the current block. red of the size; determining a simplified prediction signal pred based on the mipSizeId value of the current block. redthe size; deriving the simplified boundary bdry from the original boundary samples red ; deriving the simplified prediction signal pred through matrix multiplication of the selected MIP weight matrix and the simplified boundary bdry red red temp ; and deriving the simplified prediction signal pred by using sample value clipping for each sample of pred red temp red .

[0161] A8. The method of embodiment A7, further comprising: determining whether i) to apply vertical linear interpolation to the simplified prediction signal pred based on the width W and the height H of the current block, and ii) to apply horizontal linear interpolation to the simplified prediction signal pred based on the width W and the height H of the current block. red red

[0162] A9. The method of embodiment A8, further comprising: as a result of determining to apply both vertical and horizontal linear interpolation, determining whether to apply vertical linear interpolation before horizontal linear interpolation or to apply horizontal linear interpolation before vertical linear interpolation based on the width W and the height H of the current block.

[0163] A10. The method of embodiment A9, further comprising: as a result of determining to apply vertical linear interpolation before horizontal linear interpolation, determining the size of the simplified upper boundary bdry of the vertical linear interpolation based on the width W and the height H of the current block. redII 上

[0164] A11. The method of embodiment A10, further comprising: deriving the simplified upper boundary bdry from the original upper boundary samples redII 上 .

[0165] A12. The method of embodiment A9, further comprising: as a result of determining to apply horizontal linear interpolation before vertical linear interpolation, determining the size of the simplified left boundary bdry of the horizontal linear interpolation based on the width W and the height H of the current block. redII 左

[0166] A13. The method of embodiment A12, further comprising: deriving the simplified left boundary bdry from the original left boundary samples redII 左 .

[0167] ​​​​​​The method of Embodiment A11 or A13 further includes: deriving a MIP prediction block pred by generating sample values at remaining positions using linear interpolation; and decoding a current block by using the derived MIP prediction block.

[0168] B1. A matrix-based intra prediction MIP method (700) for predicting samples of a current block, the method includes: storing (s702) a set of MIP weight matrices; determining (s704) the width W and height H of the current block; as a result of i) determining W = 4 and H = 16 or ii) determining W = 16 and H = 4, setting (s706) the mipSizeId variable to 1; storing (s708) a value predModeIntra that specifies the MIP prediction mode of the current block; determining (s710) a modeId value based on the values of predModeIntra and the mipSizeId variable; and selecting (s712) a MIP weight matrix to be used for the current block from the set of MIP weight matrices, where the selection is based on the values of modeId and the mipSizeId variable.

[0169] B2. The method of claim B1 further includes: storing a look-up table that maps different mipSizeId-modeId pairs to different MIP weight matrices, where selecting the MIP weight matrix includes using the look-up table to select the MIP weight matrix.

[0170] B3. The method of claim B1 or B2 further includes: determining the original boundary sample values of the current block, where the original boundary samples are W samples from the nearest adjacent samples above the current block and H samples from the nearest adjacent samples to the left of the current block.

[0171] B4. The method of claim B3 further includes: determining the size of a simplified boundary bdry based on the mipSizeId value of the current block; determining the size of a simplified prediction signal pred based on the mipSizeId value of the current block; deriving the simplified boundary bdry from the original boundary samples; deriving the simplified prediction signal pred through matrix multiplication of the selected MIP weight matrix and the simplified boundary bdry; and deriving the simplified prediction signal pred by using sample value clipping for each sample of pred. red ; based on the mipSizeId value of the current block to determine the size of the simplified prediction signal pred red ; derive the simplified boundary bdry from the original boundary samples red ; through matrix multiplication of the selected MIP weight matrix and the simplified boundary bdry red to derive the simplified prediction signal pred red temp ; and by using sample value clipping for each sample of pred red temp to derive the simplified prediction signal pred red .

[0172] B5. The method of claim B4, further comprising: determining whether i) to apply vertical linear interpolation to the simplified prediction signal pred based on the width W and the height H of the current block, and ii) to apply horizontal linear interpolation to the simplified prediction signal pred based on the width W and the height H of the current block. red B6. The method of claim B5, further comprising: as a result of determining to apply both vertical and horizontal linear interpolation, determining whether to apply vertical linear interpolation before horizontal linear interpolation or to apply horizontal linear interpolation before vertical linear interpolation based on the width W and the height H of the current block. red B7. The method of claim B6, further comprising: as a result of determining to apply vertical linear interpolation before horizontal linear interpolation, determining the size of the simplified upper boundary bdry of the vertical linear interpolation based on the width W and the height H of the current block.

[0173] B8. The method of claim B7, further comprising: deriving the simplified upper boundary bdry from the original upper boundary samples.

[0174] B9. The method of claim B6, further comprising: as a result of determining to apply horizontal linear interpolation before vertical linear interpolation, determining the size of the simplified left boundary bdry of the horizontal linear interpolation based on the width W and the height H of the current block. redII 上 B10. The method of claim B9, further comprising: deriving the simplified left boundary bdry from the original left boundary samples.

[0175] B11. The method of claim B8 or B10, further comprising: deriving the MIP prediction block pred by generating sample values at the remaining positions using linear interpolation; and decoding the current block using the derived MIP prediction block. redII 上 B12. The method of any one of claims B1 - B11, further comprising: determining the size of the simplified prediction signal pred based on the mipSizeId value of the current block.

[0176] B13. The method of claim B12, further comprising: determining the size of the simplified prediction signal pred based on the width W and the height H of the current block. redII 左 B14. The method of claim B13, further comprising: deriving the simplified prediction signal pred from the original prediction signal samples.

[0177] B15. The method of claim B14, further comprising: generating a sample value at a remaining position using linear interpolation to derive the MIP prediction block pred; and decoding the current block using the derived MIP prediction block. redII 左 B16. The method of any one of claims B1 - B15, further comprising: determining whether to apply vertical linear interpolation to the simplified prediction signal pred based on the width W and the height H of the current block, and determining whether to apply horizontal linear interpolation to the simplified prediction signal pred based on the width W and the height H of the current block.

[0178] B17. The method of claim B16, further comprising: as a result of determining to apply both vertical and horizontal linear interpolation, determining whether to apply vertical linear interpolation before horizontal linear interpolation or to apply horizontal linear interpolation before vertical linear interpolation based on the width W and the height H of the current block.

[0179] B18. The method of claim B17, further comprising: as a result of determining to apply vertical linear interpolation before horizontal linear interpolation, determining the size of the simplified upper boundary bdry of the vertical linear interpolation based on the width W and the height H of the current block. red B19. The method of claim B18, further comprising: deriving the simplified upper boundary bdry from the original upper boundary samples.

[0180] The method of claim B12, further comprising: deriving a simplified prediction signal pred through matrix multiplication of a selected MIP weight matrix and a simplified boundary bdry red ; and deriving a simplified prediction signal pred by using sample value clipping for each sample of pred red temp ; and by using sample value clipping for each sample of pred red temp to derive a simplified prediction signal pred red .

[0181] C1. A computer program (943) comprising instructions (944) which, when executed by a processing circuit (902), cause the processing circuit (902) to perform the method of any one of the above embodiments A1 - A14 or B1 - B13.

[0182] C2. A carrier containing the computer program of embodiment C1, wherein the carrier is one of the following: an electronic signal, an optical signal, a radio signal, and a computer-readable storage medium (942).

[0183] D1. A device (900) adapted to perform the method of any one of embodiments A1 - A14.

[0184] E1. A device (1000, see Figure 10 ) for matrix-based intra prediction MIP, the device comprising: a matrix storage module (1002) for storing a set of MIP weight matrices; a size determination module (1004) for determining the width (W) and height (H) of a current block; a mipSizeId setting module (1006) for setting a mipSizeId variable to a certain value based on W and H, wherein setting the mipSizeId to a certain value includes setting the mipSizeId to 1 as a result of a determination condition being true, and the determination condition being true includes determining that WxH is greater than T1 (e.g., 16) but less than or equal to T2 (e.g., 64); a mode value storage module (1008) for storing a value predModeIntra that specifies the MIP prediction mode of the current block; a modeId determination module (1010) for determining a modeId value based on predModeIntra and mipSizeId; and a selection module (1012) for selecting an MIP weight matrix to be used for the current block from the set of MIP weight matrices, wherein the selection is based on modeId and mipSizeId.

[0185] Advantages

[0186] As pointed out above, the advantage of the embodiment is to reduce the decoding complexity of the MIP process, and this is accomplished by eliminating the handling of matrix multiplications with the MIP weight matrix mWeight[M][N] (where M is greater than the size of the MIP OUTPUT). This simplification makes the matrix multiplication process consistent for all MIP decoded blocks. Examples in VVC have been implemented using the VTM-6.0 reference VVC software. Compared with the current version of VVC, given a 16×4 or 4×16 block, mipSizeId is set to 1, where in the current version of VCC, for this case, mipSizeId is set to 2. Compared with VTM6.0, the proposed method has a negligible impact on decoding efficiency. The BD-rate results are as follows:

[0187]

[0188] Abbreviations

[0189] ALF Adaptive Loop Filter

[0190] APS Adaptation Parameter Set

[0191] AUD Access Unit Delimiter

[0192] BLA Broken Link Access

[0193] CRA Complete Random Access

[0194] CVS Decoded Video Sequence

[0195] CVSS CVS Start

[0196] CU Coding Unit

[0197] DPS Decoder Parameter Set

[0198] GRA Gradual Random Access

[0199] HEVC High Efficiency Video Coding

[0200] IDR Instantaneous Decoding Refresh

[0201] IRAP Intra Random Access Point

[0202] JVET Joint Video Exploration Team

[0203] LMCS Luma Mapping and Chroma Scaling

[0204] MPEG Moving Picture Experts Group

[0205] NAL Network Abstraction Layer

[0206] PES Packetized Elementary Stream

[0207] PPS Picture Parameter Set

[0208] RADL Random Access Decodable Leader

[0209] RASL Random Access Skip Leader

[0210] SPS Sequence Parameter Set

[0211] VCL Video Coding Layer

[0212] VPS Video Parameter Set

[0213] VVC Versatile Video Coding

[0214] SEI Supplementary Enhancement Information

[0215] Although the present document (including the appendices) describes various embodiments, it should be understood that these embodiments are given by way of example only and not by way of limitation. Thus, the breadth and scope of the present disclosure should not be limited by any of the exemplary embodiments described above. Moreover, unless otherwise indicated herein or otherwise clearly contradicted by the context, any combination of the above elements in all possible variations thereof is covered by the present disclosure.

[0216] In addition, although the processes described above and illustrated in the figures are shown as a series of steps, this is done for illustrative purposes only. Thus, it is contemplated that some steps may be added, some steps may be omitted, the order of the steps may be rearranged, and some steps may be executed in parallel.

[0217] Appendix

[0218] 8.4.5.2.3 MIP Weight Matrix Derivation Process

[0219] The inputs to this process are: variable mipSizeId and variable modeId.

[0220] The output of this process is the MIP weight matrix mWeight[x][y].

[0221] Derive the MIP weight matrix mWeight[x][y] depending on mipSizeId and modeId as follows:

[0222] If mipSizeId is equal to 0 and modeId is equal to 0, then the following applies:

[0223]

[0224] Otherwise, if mipSizeId is equal to 0 and modeId is equal to 1, then the following applies:

[0225]

[0226] Otherwise, if mipSizeId is equal to 0 and modeId is equal to 2, then the following applies:

[0227]

[0228] Otherwise, if mipSizeId is equal to 0 and modeId is equal to 3, then the following applies:

[0229]

[0230] Otherwise, if mipSizeId is equal to 0 and modeId is equal to 4, then the following applies:

[0231]

[0232]

[0233] Otherwise, if mipSizeId is equal to 0 and modeId is equal to 5, then the following applies:

[0234]

[0235] - Otherwise, if mipSizeId is equal to 0 and modeId is equal to 6, then the following applies:

[0236]

[0237] - Otherwise, if mipSizeId is equal to 0 and modeId is equal to 7, then the following applies:

[0238]

[0239] Otherwise, if mipSizeId is equal to 0 and modeId is equal to 8, then the following applies:

[0240]

[0241] Otherwise, if mipSizeId is equal to 0 and modeId is equal to 9, then the following applies:

[0242]

[0243] Otherwise, if mipSizeId is equal to 0 and modeId is equal to 10, then the following applies:

[0244]

[0245] - Otherwise, if mipSizeId is equal to 0 and modeId is equal to 11, then the following applies:

[0246]

[0247] - Otherwise, if mipSizeId is equal to 0 and modeId is equal to 12, then the following applies:

[0248]

[0249] Otherwise, if mipSizeId is equal to 0 and modeId is equal to 13, then the following applies:

[0250]

[0251]

[0252] Otherwise, if mipSizeId is equal to 0 and modeId is equal to 14, then the following applies:

[0253]

[0254] Otherwise, if mipSizeId is equal to 0 and modeId is equal to 15, then the following applies:

[0255]

[0256] Otherwise, if mipSizeId is equal to 0 and modeId is equal to 16, then the following applies:

[0257]

[0258] Otherwise, if mipSizeId is equal to 0 and modeId is equal to 17, then the following applies:

[0259]

[0260] Otherwise, if mipSizeId is equal to 1 and modeId is equal to 0, then the following applies:

[0261]

[0262] Otherwise, if mipSizeId is equal to 1 and modeId is equal to 1, then the following applies:

[0263]

[0264] Otherwise, if mipSizeId is equal to 1 and modeId is equal to 2, then the following applies:

[0265]

[0266] Otherwise, if mipSizeId is equal to 1 and modeId is equal to 3, then the following applies:

[0267]

[0268]

[0269] Otherwise, if mipSizeId is equal to 1 and modeId is equal to 4, then the following applies:

[0270]

[0271] Otherwise, if mipSizeId is equal to 1 and modeId is equal to 5, then the following applies:

[0272]

[0273] Otherwise, if mipSizeId is equal to 1 and modeId is equal to 6, then the following applies:

[0274]

[0275]

[0276] Otherwise, if mipSizeId is equal to 1 and modeId is equal to 7, then the following applies:

[0277]

[0278] Otherwise, if mipSizeId is equal to 1 and modeId is equal to 8, then the following applies:

[0279]

[0280] Otherwise, if mipSizeId is equal to 1 and modeId is equal to 9, then the following applies:

[0281]

[0282]

[0283] Otherwise, if mipSizeId is equal to 2 and modeId is equal to 0, then the following applies:

[0284]

[0285]

[0286] Otherwise, if mipSizeId is equal to 2 and modeId is equal to 1, then the following applies:

[0287]

[0288]

[0289] Otherwise, if mipSizeId is equal to 2 and modeId is equal to 2, then the following applies:

[0290]

[0291]

[0292] Otherwise, if mipSizeId is equal to 2 and modeId is equal to 3, then the following applies:

[0293]

[0294]

[0295] Otherwise, if mipSizeId is equal to 2 and modeId is equal to 4, then the following applies:

[0296]

[0297]

[0298] Otherwise (mipSizeId is equal to 2 and modeId is equal to 5), the following applies:

[0299]

[0300]

Claims

1. A matrix-based intra prediction (MIP) method for predicting samples of a current block, the method comprises: storing a set of MIP weight matrices; determining a width W and a height H of the current block; setting a mipSizeId variable to 1 as a result of i) determining W = 4 and H = 16 or ii) determining W = 16 and H = 4; storing a value predModeIntra that specifies an MIP prediction mode of the current block; determining a mode Id modeId value based on the values of predModeIntra and the mipSizeId variable; and selecting an MIP weight matrix to be used for the current block from the set of MIP weight matrices, wherein the selection is based on the values of modeId and the mipSizeId variable.

2. The method according to claim 1, further comprises: storing a look-up table that maps different mipSizeId-modeId pairs to different MIP weight matrices, wherein selecting the MIP weight matrix comprises using the look-up table to select the MIP weight matrix.

3. The method according to any one of claims 1-2, further comprises: determining original boundary sample values of the current block, wherein the original boundary samples are W samples from the nearest adjacent samples above the current block and H samples from the nearest adjacent samples to the left of the current block.

4. The method according to claim 3, further comprises: Determine the size of the simplified boundary bdry based on the mipSizeId value of the current block red ; Determine the size of the simplified prediction signal pred based on the mipSizeId value of the current block red ; Derive the simplified boundary bdry from the original boundary samples red ; Derive a simplified prediction signal pred through matrix multiplication of the selected MIP weight matrix and the simplified boundary bdry red red temp ;​ and By using sample value clipping for each sample of the said pred red temp the simplified prediction signal pred is derived red .

5. The method according to claim 4, further comprises: Determine whether i) vertical linear interpolation is applied to the simplified prediction signal pred based on the width W and height H of the current block, and ii) horizontal linear interpolation is applied to the simplified prediction signal pred based on the width W and height H of the current block. red Apply vertical linear interpolation, and ii) based on the width W and height H of the current block to the simplified prediction signal pred red Apply horizontal linear interpolation.

6. The method according to claim 5, further comprises: as a result of determining that both vertical and horizontal linear interpolations are applied, determining whether to apply vertical linear interpolation before horizontal linear interpolation or horizontal linear interpolation before vertical linear interpolation based on the width W and height H of the current block.

7. The method according to claim 6, further comprises: As a result of determining to apply horizontal linear interpolation before vertical linear interpolation, a simplified left boundary bdry of the horizontal linear interpolation is determined based on the width W and height H of the current block. redII 左 The size of.

8. The method according to claim 7, further comprises: Derive the simplified left boundary bdry from the original left boundary samples redII 左 .

9. The method according to claim 8, further comprises: deriving an MIP prediction block pred by generating sample values at remaining positions using linear interpolation; and decoding the current block by using the derived MIP prediction block.

10. The method according to any one of claims 1-2, further comprises: Determine the simplified prediction signal pred based on the mipSizeId value of the current block red for the size.

11. The method according to claim 10, further comprises: Derive a simplified prediction signal pred through matrix multiplication of the selected MIP weight matrix and the simplified boundary bdry red red temp ;​ and By using sample value clipping for each sample of the said pred red temp to derive the simplified prediction signal pred red .

12. A computer program product comprising instructions that, when executed by a processing circuit, cause the processing circuit to perform the method according to any one of claims 1-11 above.

13. A computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a processing circuit, cause the processing circuit to perform the method according to any one of claims 1-11 above.

14. A device for predicting samples of a current block, the device comprising a processing circuit adapted to: store a set of MIP weight matrices; determine a width W and a height H of the current block; As a result of i) determining that W = 4 and H = 16 or ii) determining that W = 16 and H = 4, set the mipSizeId variable to 1; Store the value predModeIntra that specifies the MIP prediction mode of the current block; Determine the mode Id modeId value based on the value of predModeIntra and the mipSizeId variable; and Select, from the set of MIP weight matrices, the MIP weight matrix to be used for the current block, where the selection is based on the values of modeId and the mipSizeId variable.

15. The apparatus according to claim 14, wherein, the apparatus is further configured to perform the method according to any one of claims 2-11.

Images