Method and apparatus for intra predictive coding of video data

Abstract

The invention relates to a method and an apparatus for intra prediction coding of video data. According to one aspect of the present invention, an apparatus for decoding video data comprises: a decoding unit that decodes, from a bitstream, a syntax element indicating an intra prediction type of a current block of video data; the intra prediction unit generates a prediction block of the current block by selectively performing matrix-based intra prediction or regular intra prediction based on the intra prediction type of the current block indicated by the syntax element.

Description

[0001] This application is a divisional application of the PCT patent application entered into China with Chinese patent application number 202080048546.8, invention name “Method and device for intra-frame prediction encoding of video data”, and application date July 8, 2020.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims priority to Patent Application No. 10-2019-0082130 filed in Korea on July 8, 2019, Patent Application No. 10-2019-0102494 filed in Korea on August 21, 2019, Patent Application No. 10-2019-0102495 filed in Korea on August 21, 2019, and Patent Application No. 10-2019-0123492 filed in Korea on October 6, 2019, the entire contents of which are incorporated herein by reference. Technical Field

[0004] The present invention relates to encoding and decoding of video data. Background Art

[0005] Since the amount of video data is usually larger than that of voice data or still image data, storing or transmitting the video data without compression processing requires a large amount of hardware resources (including memory).

[0006] Accordingly, when storing or transmitting video data, an encoder is usually used to compress the video data for storage or transmission. Then, a decoder receives the compressed video data, decompresses and reproduces the video data. Compression technologies used for video include H.264 / AVC and High Efficiency Video Coding (HEVC), which improves the coding efficiency of H.264 / AVC by about 40%.

[0007] However, for video data, the image size, resolution and frame rate are gradually increasing, and accordingly, the amount of data to be encoded is also increasing. Therefore, a new compression technology with better encoding efficiency and higher image quality than the existing compression technology is needed. Summary of the invention

[0008] Technical issues

[0009] The present invention discloses an improved technique for intra-frame predictive coding of blocks of video data.

[0010] Technical Solution

[0011] According to one aspect of the present invention, a method for decoding video data includes: decoding a syntax element indicating an intra-prediction type of a current block of video data from a bitstream, the intra-prediction type including matrix based intra-prediction (MIP) and regular intra-prediction; and generating a prediction block of the current block by selectively performing MIP or regular intra-prediction based on the intra-prediction type of the current block indicated by the syntax element.

[0012] When generating a prediction block of a current block by performing MIP, the method further includes: decoding a syntax element indicating a MIP mode of the current block from a bitstream, the syntax element being represented as a truncated binary code specifying one of a plurality of MIP prediction modes allowed for a width and a height of the current block; deriving an input boundary vector based on the width and the height of the current block using neighboring samples adjacent to the current block; generating predicted samples of the current block based on a matrix-vector multiplication between the input boundary vector and a matrix predefined for the MIP mode; and deriving a prediction block of the current block based on the predicted samples.

[0013] When generating a prediction block of a current block by performing conventional intra prediction, the method further includes: deriving a most probable mode (MPM) candidate based on a conventional intra prediction mode of a neighboring block adjacent to the current block, and configuring an MPM list of the current block; and deriving a conventional intra prediction mode of the current block based on the MPM list. When the intra prediction type of the neighboring block is MIP, the conventional intra prediction mode of the neighboring block is set to (regarded as) a planar (PLANAR) mode.

[0014] According to another aspect of the present invention, an apparatus for decoding video data includes: a decoder and an intra-frame predictor, wherein the decoder is configured to decode a syntax element indicating an intra-frame prediction type of a current block of video data from a bitstream, wherein the intra-frame prediction type includes a matrix-based intra-frame prediction (MIP) and a conventional intra-frame prediction; and the intra-frame predictor is configured to generate a prediction block of the current block by selectively performing MIP or conventional intra-frame prediction based on the intra-frame prediction type of the current block indicated by the syntax element.

[0015] When generating a prediction block of a current block by performing MIP, the intra predictor is configured to: decode a syntax element indicating a MIP mode of the current block from a bitstream, the syntax element being represented as a truncated binary code specifying one of a plurality of MIP prediction modes allowed for a width and a height of the current block; derive an input boundary vector based on the width and the height of the current block using neighboring samples adjacent to the current block; generate predicted samples of the current block based on a matrix-vector multiplication between the input boundary vector and a matrix predefined for the MIP mode; and derive a prediction block of the current block based on the predicted samples.

[0016] When generating a prediction block of a current block by performing conventional intra prediction, the intra predictor is configured to: derive a most probable mode (MPM) candidate based on a conventional intra prediction mode of a neighboring block adjacent to the current block, and configure an MPM list of the current block; and derive a conventional intra prediction mode of the current block based on the MPM list. When deriving the MPM candidate, when the intra prediction type of the neighboring block is MIP, the intra predictor sets (regards) the conventional intra prediction mode of the neighboring block to a planar (PLANAR) mode. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is an exemplary block diagram of a video encoding device capable of implementing the technology of the present invention.

[0018] Figure 2 A block partition structure using a QTBTTT structure is exemplarily shown.

[0019] Figure 3a A plurality of intra prediction modes are exemplarily shown.

[0020] Figure 3b A plurality of intra prediction modes including a wide-angle intra prediction mode are exemplarily shown.

[0021] Figure 4 is an exemplary block diagram of a video decoding device capable of implementing the techniques of the present invention.

[0022] Figure 5 is a conceptual diagram showing the main process of the MIP technology that can be used in the technology of the present invention.

[0023] Figure 6 Smoothing filtering and interpolation filtering to construct reference samples are shown.

[0024] Figures 7a to 7c is a conceptual diagram illustrating an exemplary method of constructing a boundary vector input to a matrix-vector multiplication operation using left neighboring samples.

[0025] Figures 8a to 8cis a conceptual diagram illustrating an exemplary method of constructing a boundary vector input to a matrix-vector multiplication operation using upper neighboring samples.

[0026] Fig. 9 is a flowchart illustrating a method of decoding video data according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] Hereinafter, some embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be noted that when reference numerals are added to the constituent elements in the various drawings, the same reference numerals also represent the same elements, although the elements are shown in different drawings. Further, in the following description of the present invention, the detailed description of the known functions and configurations incorporated herein will be omitted to avoid blurring the subject matter of the present invention.

[0028] Figure 1 is an exemplary block diagram of a video encoding device capable of implementing the technology of the present invention. Figure 1 A video encoding apparatus and elements of the apparatus are described below.

[0029] The video encoding device includes: an image divider 110, a predictor 120, a subtractor 130, a transformer 140, a quantizer 145, a rearrangement unit 150, an entropy encoder 155, an inverse quantizer 160, an inverse transformer 165, an adder 170, a filtering unit 180 and a memory 190.

[0030] Each element of the video encoding device may be implemented in hardware or software, or a combination of hardware and software. The functions of each element may be implemented as software, and a microprocessor may be implemented to execute the software functions corresponding to each element.

[0031] A video includes multiple images. Each image is divided into multiple regions, and encoding is performed on each region. For example, an image is divided into one or more tiles (tile) or / and slices (slice). Here, one or more tiles can be defined as tile groups. Each tile or slice is divided into one or more coding tree units (coding treeunit, CTU). Each CTU is divided into one or more coding units (coding unit, CU) through a tree structure. The information applied to each CU is encoded as the syntax of the CU, and the information commonly applied to the CU included in a CTU is encoded as the syntax of the CTU. In addition, the information commonly applied to all blocks in a slice is encoded as the syntax of the slice header, and the information applied to all blocks constituting the image is encoded in the picture parameter set (Picture Parameter Set, PPS) or the picture header. In addition, the information commonly referenced by multiple images is encoded in the sequence parameter set (Sequence Parameter Set, SPS). In addition, the information commonly referenced by one or more SPSs is encoded in the video parameter set (Video Parameter Set, VPS). Information commonly applied to one tile or tile group may be encoded as the syntax of a tile header or tile group header.

[0032] The image divider 110 is configured to determine the size of a coding tree unit (CTU). Information on the size of the CTU (CTU size) is encoded as a syntax of an SPS or a PPS and transmitted to a video decoding device.

[0033] The image splitter 110 is configured to split each image constituting a video into a plurality of CTUs having a predetermined size, and then recursively split the CTUs using a tree structure. In the tree structure, a leaf node is used as a coding unit (CU), which is a basic unit of coding.

[0034] The tree structure can be a quadtree (QuadTree, QT), a binary tree (BinaryTree, BT), a ternary tree (TernaryTree, TT), or a structure formed by a combination of two or more QT structures, BT structures and TT structures, wherein the quadtree (QT) is a node (or parent node) that is divided into four slave nodes (or child nodes) of the same size, the binary tree (BT) is a node that is divided into two slave nodes, and the ternary tree (TT) is a node that is divided into three slave nodes at a ratio of 1:2:1. For example, a quadtree plus binary tree (QuadTree plus BinaryTree, QTBT) structure can be used, or a quadtree plus binary tree ternary tree (QuadTree plus BinaryTree TernaryTree, QTBTTT) structure can be used. Here, BTTT can be collectively referred to as a multiple-type tree (MTT).

[0035] Figure 2 The QTBTTT segmentation tree structure is shown as an example. Figure 2 As shown, the CTU can be first split into a QT structure. The QT splitting can be repeated until the size of the split block reaches the minimum block size MinQTSize of the leaf node allowed in the QT. The first flag (QT_split_flag) indicating whether each node of the QT structure is split into four nodes of the lower layer is encoded by the entropy encoder 155 and notified to the video decoding device with a signal. When the leaf node of the QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in the BT, it can be further split into one or more BT structures or TT structures. The BT structure and / or the TT structure can have multiple splitting directions. For example, there can be two directions, namely, the direction of splitting the blocks of the nodes horizontally and the direction of splitting the blocks vertically. As Figure 2As shown, when the MTT splitting starts, the second flag (mtt_split_flag) indicating whether the node is split, the flag indicating the splitting direction (vertical or horizontal) in the case of splitting, and / or the flag indicating the splitting type (binary or trifurcated) are encoded by the entropy encoder 155, and the video decoding device is notified of the same by a signal. Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node is split into 4 nodes of the lower layer, the CU split flag (split_cu_flag) indicating whether the node is split may be encoded. When the value of the CU split flag (split_cu_flag) indicates that the splitting is not performed, the block of the node becomes a leaf node in the split tree structure and is used as a coding unit (CU), which is a basic unit of encoding. When the value of the CU split flag (split_cu_flag) indicates that the splitting is performed, the video encoding device starts encoding the flags from the first flag in the above-mentioned manner.

[0036] When QTBT is used as another example of a tree structure, there may be two types of segmentation, namely, a type of horizontally segmenting a block into two blocks of the same size (i.e., symmetrical horizontal segmentation) and a type of vertically segmenting a block into two blocks of the same size (i.e., symmetrical vertical segmentation). A segmentation flag (split_flag) indicating whether each node of the BT structure is segmented into blocks of the lower layer and segmentation type information indicating the segmentation type are encoded by the entropy encoder 155 and transmitted to the video decoding device. There may be an additional type of segmenting a block of a node into two asymmetric blocks. An asymmetric segmentation type may include a type of segmenting a block into two rectangular blocks at a size ratio of 1:3, or a type of segmenting a block of a node diagonally.

[0037] A CU may have various sizes according to the QTBT or QTBTTT partitioning of a CTU. Hereinafter, a block corresponding to a CU to be encoded or decoded (i.e., a leaf node of a QTBTTT) is referred to as a "current block". When QTBTTT partitioning is adopted, the shape of the current block may be a square or a rectangle.

[0038] The predictor 120 is configured to predict a current block to generate a predicted block. The predictor 120 includes an intra predictor 122 and an inter predictor 124.

[0039] Typically, each current block in an image can be predictively encoded separately. Typically, prediction of the current block is performed using an intra-frame prediction technique (which uses data from an image including the current block) or an inter-frame prediction technique (which uses data from an image encoded before the image including the current block). Inter-frame prediction includes both unidirectional prediction and bidirectional prediction.

[0040] The intra-frame predictor 122 is configured to predict pixels in the current block using pixels (reference pixels) located around the current block in the current image including the current block. There are multiple intra-frame prediction modes according to the prediction direction. For example, Figure 3a and Figure 3b As shown, multiple intra prediction modes may include 2 non-directional modes and 65 directional modes, wherein the 2 non-directional modes include a planar mode and a DC mode. The neighboring pixels and equations to be used are defined differently for each prediction mode. The following table lists the intra prediction mode numbers and their names.

[0041] Table 1

[0042] Intra prediction mode Related Name 0 INTRA_PLANAR 1 INTRA_DC 2……66 INTRA_ANGULAR2……INTRA_ANGULAR66

[0043] In order to perform effective directional prediction for the current block of rectangular shape, the additional use can be made of Figure 3b The dotted arrows in the figure indicate the directional modes (intra-prediction modes 67 to 80 and -1 to -14). These modes may be referred to as "wide angle intra-prediction modes". Figure 3b In the figure, the arrows indicate the corresponding reference samples used for prediction, rather than indicating the prediction direction. The prediction direction is opposite to the direction indicated by the arrow. The wide-angle intra prediction mode is a mode in which prediction is performed in a direction opposite to a specific direction mode without additional bit transmission when the current block has a rectangular shape. In this case, in the wide-angle intra prediction mode, some wide-angle intra prediction modes that can be used for the current block can be determined based on the ratio of the width to the height of the rectangular current block. For example, when the height of the rectangular shape of the current block is less than its width, a wide-angle intra prediction mode (intra prediction modes 67 to 80) with an angle less than 45 degrees can be utilized. When the height of the rectangular shape of the current block is greater than its width, a wide-angle intra prediction mode (intra prediction modes -1 to -14) with an angle greater than -135 degrees can be utilized.

[0044] The intra-frame predictor 122 may determine an intra-frame prediction mode to be used when encoding the current block. In some examples, the intra-frame predictor 122 may encode the current block using several intra-frame prediction modes and select an appropriate intra-frame prediction mode to be used from the tested modes. For example, the intra-frame predictor 122 may calculate a rate-distortion value using a rate-distortion analysis of several tested intra-frame prediction modes and may select an intra-frame prediction mode having the best rate-distortion characteristic from the tested modes.

[0045] The intra-frame predictor 122 is configured to select an intra-frame prediction mode from a plurality of intra-frame prediction modes and predict the current block using adjacent pixels (reference pixels) and equations determined according to the selected intra-frame prediction mode. Information about the selected intra-frame prediction mode is encoded by the entropy encoder 155 and transmitted to the video decoding device.

[0046] In addition, the intra predictor 122 can generate a prediction block of the current block using matrix-based intra prediction (MIP), which will be described later. The intra predictor 122 generates a prediction block of the current block using a boundary vector derived from samples reconstructed on the left side of the current block and samples reconstructed above the current block, a predefined matrix, and an offset vector.

[0047] The inter-frame predictor 124 is configured to generate a prediction block for the current block by motion compensation. The inter-frame predictor 124 can search for a block that is most similar to the current block in a reference image that has been encoded and decoded earlier than the current image, and use the searched block to generate a prediction block for the current block. Then, the inter-frame predictor is configured to generate a motion vector corresponding to the displacement between the current block in the current image and the prediction block in the reference image. Typically, motion estimation is performed on the luminance (luma) component, and the motion vector calculated based on the luminance component is used for both the luminance component and the chrominance component. The motion information including information about the reference image and information about the motion vector used to predict the current block is encoded by the entropy encoder 155 and transmitted to the video decoding device.

[0048] The subtractor 130 is configured to subtract the prediction block generated by the intra predictor 122 or the inter predictor 124 from the current block to generate a residual block.

[0049] The transformer 140 may divide the residual block into one or more transform blocks and apply a transform to one or more transform blocks, thereby transforming the residual values ​​of the transform block from the pixel domain to the frequency domain. In the frequency domain, the transform block is referred to as a coefficient block containing one or more transform coefficient values. A two-dimensional transform kernel may be used for the transform, while a one-dimensional transform kernel may be used for horizontal transform and vertical transform, respectively. The transform kernel may be based on a discrete cosine transform (DCT), a discrete sine transform (DST), etc.

[0050] The transformer 140 may transform the residual signal in the residual block using the entire size of the residual block as a transform unit. In addition, the transformer 140 may partition the residual block into two sub-blocks in the horizontal direction or the vertical direction, and may transform only one of the two sub-blocks. Accordingly, the size of the transform block may be different from the size of the residual block (and thus the size of the prediction block). Non-zero residual sample values ​​may not exist, or may be very rare in the untransformed sub-block. The residual samples of the untransformed sub-block are not signaled, and the residual samples of the untransformed sub-block may be regarded as "0" by the video decoding device. There may be multiple partition types according to the partition direction and the partition ratio. The transformer 140 may provide information about the encoding mode (or transform mode) of the residual block to the entropy encoder 155 (e.g., information indicating whether to transform the residual block or the residual sub-block, information indicating the partition type selected to partition the residual block into sub-blocks, and information identifying the sub-blocks to be transformed). The entropy encoder 155 may encode information about the encoding mode (or transform mode) of the residual block.

[0051] The quantizer 145 is configured to quantize the transform coefficients output from the transformer 140, and output the quantized transform coefficients to the entropy encoder 155. For some blocks or frames, the quantizer 145 may directly quantize the relevant residual block without transformation.

[0052] The rearrangement unit 150 can reorganize the coefficient values ​​of the quantized residual values. The rearrangement unit 150 can change the 2D coefficient array into a 1D coefficient sequence by coefficient scanning. For example, the rearrangement unit 150 can scan the coefficients from the DC coefficient to the coefficients in the high frequency region using a zig-zag scan or a diagonal scan to output a 1D coefficient sequence. According to the size of the transform unit and the intra-frame prediction mode, a vertical scan or a horizontal scan can be used instead of a zig-zag scan, wherein the vertical scan scans the two-dimensional array of coefficients in the column direction, and the horizontal scan scans the coefficients in the two-dimensional block shape in the row direction. That is, the scanning mode to be used can be determined among the zig-zag scan, the diagonal scan, the vertical scan, and the horizontal scan according to the size of the transform unit and the intra-frame prediction mode.

[0053] The entropy encoder 155 is configured to encode the one-dimensional quantized transform coefficient output from the rearrangement unit 150 using various encoding techniques such as Context-based Adaptive Binary Arithmetic Code (CABAC) and exponential Golomb to generate a bitstream.

[0054] The entropy encoder 155 may encode information related to block partitioning (e.g., CTU size, CU partition flag, QT partition flag, MTT partition type, and MTT partition direction) so that the video decoding device may partition the block in the same manner as the video encoding device. In addition, the entropy encoder 155 may encode information about a prediction type indicating whether the current block is encoded by intra-frame prediction or inter-frame prediction, and encode intra-frame prediction information (i.e., information about an intra-frame prediction mode) or inter-frame prediction information (i.e., information about a reference image index and a motion vector) according to the prediction type.

[0055] The inverse quantizer 160 is configured to inversely quantize the quantized transform coefficient output from the quantizer 145 to generate a transform coefficient. The inverse transformer 165 is configured to transform the transform coefficient output from the inverse quantizer 160 from the frequency domain to the spatial domain and reconstruct a residual block.

[0056] The adder 170 is configured to add the reconstructed residual block and the prediction block generated by the predictor 120 to reconstruct the current block. Pixels in the reconstructed current block are used as reference pixels when performing intra prediction of a subsequent block.

[0057] The filtering unit 180 is configured to filter the reconstructed pixels to reduce blocking artifacts, ringing artifacts, and blurring artifacts caused by block-based prediction and transform / quantization. The filtering unit 180 may include a deblocking filter 182 and a sample adaptive offset (SAO) filter 184.

[0058] The deblocking filter 182 is configured to filter the boundary between the reconstructed blocks to remove the block artifacts caused by block-by-block encoding / decoding, and the SAO filter 184 is configured to perform additional filtering on the deblocking filtered video. The SAO filter 184 is a filter for compensating for the difference between the reconstructed pixels and the original pixels caused by lossy coding.

[0059] The reconstructed blocks filtered by the deblocking filter 182 and the SAO filter 184 are stored in the memory 190. Once all blocks in one image are reconstructed, the reconstructed image can be used as a reference image for inter-frame prediction of blocks in subsequent images to be encoded.

[0060] Figure 4 is an exemplary functional block diagram of a video decoding device capable of implementing the technology of the present invention. Figure 4 A video decoding apparatus and elements of the apparatus are described below.

[0061] The video decoding apparatus may include an entropy decoder 410 , a rearrangement unit 415 , an inverse quantizer 420 , an inverse transformer 430 , a predictor 440 , an adder 450 , a filtering unit 460 , and a memory 470 .

[0062] Similar to Figure 1 In the video encoding device of the present invention, each element of the video decoding device can be implemented by hardware, software or a combination of hardware and software. In addition, the function of each element can be implemented by software, and the microprocessor can be implemented to execute the software function corresponding to each element.

[0063] The entropy decoder 410 is configured to determine a current block to be decoded by decoding a bitstream generated by a video encoding device and extracting information related to block partitioning, and extract prediction information required to reconstruct the current block and information about a residual signal, etc.

[0064] The entropy decoder 410 is configured to extract information about the size of the CTU from a sequence parameter set (SPS) or a picture parameter set (PPS), determine the size of the CTU, and divide the image into CTUs of the determined size. Then, the decoder is configured to determine the CTU as the highest level (i.e., the root node) of the tree structure and extract the partitioning information about the CTU to partition the CTU using the tree structure.

[0065] For example, when the CTU is segmented using the QTBTTT structure, the first flag (QT_split_flag) related to the segmentation of the QT is extracted to segment each node into four nodes of the sublayer. For the node corresponding to the leaf node of the QT, the second flag (MTT_split_flag) related to the segmentation of the MTT and the information about the segmentation direction (vertical / horizontal) and / or the segmentation type (binary / trifurcated) are extracted to segment the corresponding leaf node with the MTT structure. Thus, each node below the leaf node of the QT is recursively segmented with the BT or TT structure.

[0066] As another example, when a CTU is split using a QTBTTT structure, a CU split flag (split_cu_flag) indicating whether the CU is split may be extracted. When the corresponding block is split, a first flag (QT_split_flag) may be extracted. In a split operation, after zero or more recursive QT splits, zero or more recursive MTT splits may occur for each node. For example, a CTU may directly undergo MTT splitting without undergoing QT splitting, or may only undergo QT splitting multiple times.

[0067] As another example, when the CTU is segmented using the QTBT structure, the first flag (QT_split_flag) related to the QT segmentation is extracted, and each node is segmented into four nodes of the lower layer. Then, the segmentation flag (split_flag) indicating whether the node corresponding to the leaf node of the QT is further segmented with the BT and the segmentation direction information are extracted.

[0068] Once the current block to be decoded is determined by partitioning through the tree structure, the entropy decoder 410 is configured to extract information about the prediction type indicating whether the current block is intra-predicted or inter-predicted. When the prediction type information indicates intra-prediction, the entropy decoder 410 is configured to extract syntax elements for intra-prediction information (intra-prediction mode) for the current block. When the prediction type information indicates inter-prediction, the entropy decoder 410 is configured to extract syntax elements for inter-prediction information, that is, information indicating a motion vector and a reference image referenced by the motion vector.

[0069] The entropy decoder 410 is configured to extract information about the encoding mode of the residual block from the bitstream (e.g., information about whether to encode the residual block or only encode a sub-block of the residual block, information indicating a partition type selected to partition the residual block into sub-blocks, information identifying the encoded residual sub-block, a quantization parameter, etc.). The entropy decoder 410 is also configured to extract information about the quantized transform coefficient of the current block as the information about the residual signal.

[0070] The rearrangement unit 415 may change a sequence of one-dimensional quantized transform coefficients entropy-decoded by the entropy decoder 410 into a 2-dimensional coefficient array (ie, block) in the reverse order of coefficient scanning performed by the video encoding apparatus.

[0071] The inverse quantizer 420 is configured to inversely quantize the quantized transform coefficients. The inverse transformer 430 is configured to inversely transform the inversely quantized transform coefficients from the frequency domain to the spatial domain based on the information about the encoding mode of the residual block to reconstruct the residual signal, thereby generating a reconstructed residual block of the current block.

[0072] When the information about the encoding mode of the residual block indicates that the residual block of the current block has been encoded by the video encoding device, the inverse transformer 430 uses the size of the current block (and therefore the size of the residual block to be reconstructed) as the transform unit of the inverse quantized transform coefficient to perform an inverse transform, thereby generating a reconstructed residual block of the current block.

[0073] When the information about the encoding mode of the residual block indicates that only one subblock of the residual block has been encoded by the video encoding device, the inverse transformer 430 uses the size of the transformed subblock as the transform unit of the inverse quantized transform coefficient to perform inverse transform, thereby reconstructing the residual signal of the transformed subblock, and fills the residual signal of the untransformed subblock with the value "0" to generate a reconstructed residual block of the current block.

[0074] The predictor 440 may include an intra predictor 442 and an inter predictor 444. The intra predictor 442 is activated when the prediction type of the current block is intra prediction, and the inter predictor 444 is activated when the prediction type of the current block is inter prediction.

[0075] The intra-frame predictor 442 is configured to determine the intra-frame prediction mode of the current block in multiple intra-frame prediction modes based on the syntax elements of the intra-frame prediction mode extracted from the entropy decoder 410, and predict the current block using reference pixels around the current block according to the intra-frame prediction mode. In addition, the intra-frame predictor 442 can generate a prediction block of the current block using matrix-based intra-frame prediction (MIP), which will be described later. The intra-frame predictor 422 can generate a prediction block of the current block using a boundary vector derived from a sample reconstructed on the left side of the current block and a sample reconstructed above the current block, as well as a predefined matrix and an offset vector.

[0076] The inter predictor 444 is configured to determine a motion vector of a current block and a reference image referred to by the motion vector using a syntax element of the intra prediction mode extracted by the entropy decoder 410 , and predict the current block based on the motion vector and the reference image.

[0077] The adder 450 is configured to reconstruct the current block by adding the residual block output from the inverse transformer to the prediction block output from the inter-frame predictor or the intra-frame predictor. When intra-predicting a subsequent block to be decoded, the pixels in the reconstructed current block are used as reference pixels.

[0078] The filtering unit 460 may include a deblocking filter 462 and an SAO filter 464. The deblocking filter 462 is configured to perform deblocking filtering on the boundaries between the reconstructed blocks to remove block artifacts caused by block-by-block decoding. The SAO filter 464 may perform additional filtering on the reconstructed blocks after deblocking filtering the corresponding offsets to compensate for the difference between the reconstructed pixels and the original pixels caused by lossy coding. The reconstructed blocks filtered by the deblocking filter 462 and the SAO filter 464 are stored in the memory 470. When all blocks in one image are reconstructed, the reconstructed image is used as a reference image for inter-frame prediction of blocks in subsequent images to be encoded.

[0079] The techniques of the present invention generally relate to intra-frame prediction coding. The following description focuses primarily on decoding techniques, that is, the operation of a video decoder. Since the encoding techniques are in contrast to the decoding techniques, which are described in detail, the encoding techniques are briefly described.

[0080] In the discussion of the next generation video coding standard, namely Versatile Video Coding (VVC), several new coding tools are introduced, which can achieve better coding performance than High Efficiency Video Coding (HEVC).

[0081] Unlike HEVC, which can only use the nearest reference sample line in intra prediction, VVC can use two additional reference lines, which is called multiple reference line (MRL) intra prediction. Additional reference lines can only be used in MPM mode, not non-MPM mode. When the video encoder performs prediction for each of the directional modes, the encoder can select one reference line from the three reference lines based on the RD cost. The index of the selected reference line (mrl_idx) is signaled separately and transmitted to the video decoder.

[0082] ISP (Intra Sub-Partitions) is a coding tool that divides a CU into multiple sub-blocks of the same size in the vertical or horizontal direction according to the size of the CU, and performs prediction of each sub-block in the same intra prediction mode. The reconstructed sample value of each sub-block can be used to predict the next sub-block, and the reconstructed sample value of each sub-block is iteratively processed for each sub-block. The minimum block size suitable for ISP is 4×8 or 8×4. When the size of the block is 4×8 or 8×4, it is partitioned into two sub-blocks. When the size of the block is larger than this size, it can be partitioned into four sub-blocks. When the MRL index of the block is not 0, ISP is not used.

[0083] Unlike conventional in-loop filtering that is directly applied to reconstructed video images, LMCS (Luma Mapping with Chroma Scaling) adjusts the codeword distribution of the dynamic range of the video signal to enable efficient prediction and quantization, thereby improving coding performance and image quality. LMCS includes luminance component mapping and chroma component scaling.

[0084] Luma mapping refers to in-loop mapping in which the codewords for the dynamic range of the input luma signal are reallocated to codewords that can improve the encoding performance. Luma signal mapping can be performed by forward mapping and backward mapping corresponding thereto. Forward mapping divides the existing dynamic range into 16 equal segments, and then reallocates the codewords of the input video through a linear model for each segment. Chroma scaling modifies the chroma signal according to the correlation between the luma signal and the corresponding chroma signal.

[0085] Matrix-based intra prediction (MIP) is a new intra prediction technique introduced in VTM 5.0. The original idea was to use a neural network-based intra prediction technique, that is, to use a multi-layer neural network to predict the current PU pixel value based on neighboring reconstructed pixels. However, due to the high complexity of the prediction method using neural networks, an intra prediction technique based on affine linear transformation using pre-trained matrices was introduced.

[0086] To predict a rectangular block PU with width W and height H, MIP takes as input H reconstructed samples on the left side of the block and W reconstructed samples above the block. The final predicted pixels are obtained by averaging, matrix-vector multiplication, linear interpolation, etc.

[0087] The sizes of blocks to which MIP is applied are classified into the following three categories.

[0088]

[0089] According to idx(W, H), the number of MIP modes (numModes), boundary size (boundarySize) and prediction block size (predW, predH, predC) are defined as follows. In the following table, MipSizeId=idx(W, H).

[0090] Table 2

[0091] MipSizeId numModes boundarySize pred pred predC 0 35 2 4 4 4 1 19 4 4 4 4 2 11 4 Min(nTbW.8) Min(nTbH.8) 8

[0092] Figure 5 is a conceptual diagram showing the main process of the MIP technology that can be used in the technology of the present invention.

[0093] (1) Averaging

[0094] The main purpose of this process is to normalize the reference samples. Depending on the block size and shape (width and height) (i.e., MipSizeId), 4 or 8 samples are obtained. When the width and height of the current block are both 4 (i.e., W=H=4), a total of 4 samples are obtained, including 2 on the left and 2 above (boundarySize=2). In another case, a total of 8 samples are obtained, including 4 on the left and 4 above (boundarySize=4).

[0095] like Figure 5 As shown, the upper adjacent sample is bdry top The left adjacent sample is represented by bdry left Indicates. top and bdry left Perform averaging separately to obtain the downsampled sample set and Averaging is a downsampling process as follows.

[0096]

[0097] In the above equation, bDwn represents the downsampled scaling value (nTbs / boundarySize), and refS represents the original reference sample. For the left neighboring sample, the calculated redS is stored as For the upper adjacent sample, the calculated redS is stored as

[0098] Concatenate the downsampled reference samples into a vector of length 4 or 8. The reduced boundary vector bdry is input to the vector-matrix multiplication. red is defined as follows. For example, when W = H = 4 and the MIP mode is less than 18, and When W = H = 4 and the MIP mode is greater than or equal to 18, and The order of splicing and In the following equations, "mode" represents the MIP mode.

[0099]

[0100] (2) Matrix-vector multiplication

[0101] In this process, the downsampled prediction signal pred of the current block is generated from the reduced boundary vector red .pred red is the sum of the matrix-vector product and the offset, and can be calculated as follows.

[0102] pred red =A.bdry red +b

[0103] pred red The size is W red ×H red . Define W according to the size and shape of the current block red and H red , as shown below. The matrix A has red ×H red The number of rows is the same as that of W, and 4 columns when W = H = 4, and 8 columns otherwise. The offset vector b is of size W red ×H red Vector.

[0104]

[0105] For each category of coding block size, a set S of matrices A and offset vectors b that can be used for coding blocks is predefined 0 , S 1 and S 2 The index (0, 1, 2) of the set S is selected according to the above MipSizeId (ie, idx(W, H)), and the MIP mode applied to the current block is selected from the set S. 0 , S 1 and S 2 An extraction matrix A and an offset vector b.

[0106] Set S 0 From 18 matrices A 0 and 18 16-dimensional offset vectors b 0 and for a 4×4 block, each matrix A 0 There are 16 rows and 4 columns. Set S 1 From 10 matrices A 1 and 10 16-dimensional offset vectors b 1 and for blocks of size 4×8, 8×4, and 8×8, each matrix A 1 has 16 rows and 8 columns. Finally, the set S 2 The 6 matrices A 2 and 6 64-dimensional offset vectors b 2 , and for all other block shapes, each matrix A 2 It has 64 rows and 8 columns.

[0107] (3) Pixel interpolation

[0108] Interpolation is an upsampling process. As mentioned above, pred redis the downsampled prediction signal of the original block. In this case, the downsampled prediction block with sizes predW and predH is defined as follows.

[0109] pred red [x][y], where x = 0...predW1, y = 0...predH-1

[0110] A prediction block with an original block size (nTbW, nTbH) generated by linearly interpolating the prediction signal at the remaining positions in each direction is defined as follows.

[0111] predSamples[x][y], where x=0...nTbW-1, y=0...nTbH-1

[0112] According to the horizontal upsampling scaling factor upHor (=nTbW / predW) and the vertical upsampling scaling factor upVer (=nTbH / predH), the following is described from pred red Fill part or all of predSamples.

[0113] predSamples[(x+1)×upHor-1][(y+1)×upVer-1]=pred red [x][y]

[0114] When upHor=1, pred red All horizontal positions of predSamples in are filled. When upVer=1, pred red All vertical positions of predSamples in are filled.

[0115] Then, bilinear interpolation is used to fill the remaining blank samples of predSamples. The horizontal interpolation and vertical interpolation are upsampling processes. For the interpolation of the left and top samples in predSamples, the downsampled samples are The value assigned to predSamples[x][-1] and the original reference sample on the left is assigned to the value of predSamples[-1][y]. The interpolation order is determined according to the size of the current block. That is, interpolation is first performed in the direction of the short dimension. Subsequently, interpolation is performed in the direction of the long dimension.

[0116] (4) Signaling the MIP intra prediction mode

[0117] For each coding unit (CU) that is intra-prediction encoded, a flag indicating whether a matrix-based intra prediction mode (i.e., MIP mode) is applied is transmitted. In VVC Draft 5, in order to signal the MIP mode, similar to the conventional intra prediction mode (hereinafter referred to as "normal intra prediction mode") that is different from the matrix-based intra prediction, an MPM list consisting of 3 MPMs is used. For example, intra_mip_mpm_flag, intra_mip_mpm_idx, and intra_mip_mpm_remainder are used to signal the MIP mode. Intra_mip_mpm_idx is encoded using a truncated binary code, and intra_mip_mpm_remainder is encoded using a fixed length code.

[0118] Depending on the size of the coding block (CU), up to 35 MIP modes can be supported. For example, for a CU with max(W, H) ≤ 8 and W×H < 32, 35 modes are available. In addition, 19 prediction modes and 11 prediction modes are used for CUs with max(W, H) = 8 and max(W, H) > 8, respectively. In addition, a pair of modes (two modes) can share matrices and offset vectors to reduce memory requirements. The specific shared mode calculation is as follows. For example, for a 4×4 coding block, mode 19 uses the transposed matrix of the matrix assigned to mode 2.

[0119]

[0120] When there is a block to which MIP is applied near a block to which a conventional intra prediction mode is applied instead of MIP (hereinafter referred to as a "conventional block"), the mapping table defined between the MIP mode and the conventional mode can be used for MPM derivation of the conventional block. The mapping table is used to derive a conventional mode of similar characteristics from the MIP mode of the adjacent block to which MIP is applied. The conventional mode derived in this way is used for MPM derivation of the conventional block. Similarly, even when MIP is applied to a collocated luminance block used in chrominance DM derivation, the mapping table can be used to derive the conventional mode of the collocated luminance block, and the derived conventional mode is used for chrominance DM derivation. The following equation uses the mapping tables of Tables 3 and 4 to express the mapping between the conventional mode and the MIP mode.

[0121] predmode MIP =map_regular_to_mip idx [predmode regular ]

[0122] predmode regular =map_mip_to_regular idx[predmode MIP Table 3

[0123]

[0124] Table 4

[0125]

[0126] As mentioned above, in VVC Draft 5, when predicting a block based on MIP, the reconstructed neighboring samples on the left side of the block and the reconstructed neighboring samples above the block are always used as reference samples. In the case of blocks whose texture has directional characteristics, this method may deteriorate the prediction performance. In addition, in VVC Draft 5, in order to interpolate the top sample in predSamples, the downsampled sample set is assigned to the value of predSamples[x][-1], such as Figure 5 As shown, the interpolation process is more complicated than necessary. In addition, in VVC draft 5, where each MPM list is used to signal MIP mode and normal mode, the implementation may be very complicated due to many checks and conditions, such as mapping requirements between MIP mode and normal mode.

[0127] In view of the above, the present invention proposes several improved technologies that can reduce the implementation complexity of the MIP mode and improve the prediction performance.

[0128] Reference samples using smoothing filtering

[0129] In typical intra-frame prediction coding, a smoothing filter, a Gaussian interpolation filter, and a cubic interpolation filter may be selectively used to obtain reference samples of a directional pattern. Figure 6 Smoothing filtering and interpolation filtering are shown. During the reference sample filtering process, the intra-frame prediction modes are divided into three groups. Group A consists of horizontal prediction modes and vertical prediction modes, Group B consists of diagonal modes in multiples of 45 degrees, and Group C consists of other direction modes. No filter is applied to Group A. The diagonal modes belonging to Group B refer to pixels at integer positions, so there is no need to apply an interpolation filter, and only a [1, 2, 1] / 4 smoothing filter is applied to the reference sample. For Group C, the [1, 2, 1] / 4 smoothing filter is not applied, but a 4-tap Gaussian interpolation filter or a 4-tap cubic interpolation filter is applied according to the conditions to obtain reference samples at decimal positions.

[0130] Smoothing filtering for reference samples can be performed as follows. In the following, the reference samples before filtering are denoted as refUnfilt[x][y], and the reference samples after filtering are denoted as refFilt[x][y]. Here, filtering is performed using a [1, 2, 1] / 4 smoothing filter. refH and refW are the number of reference samples on the left and the number of reference samples on the top, respectively.

[0131] - Filter the upper left reference sample

[0132] refFilt[-1][-1]=(refUnfilt[-1][0]+2×refUnfilt[-1][-1]+

[0133] refUnfilt[0][-1]+2)>>2

[0134] - Filter the left reference samples

[0135] refFilt[-1][y]=(refUnfilt[-1][y+1]+2×refUnfilt[-1][y]+

[0136] refUnfilt[-1][y-1]+2)>>2{for y=0..refH-2}

[0137] refFilt[-1][refH-1]=refUnfilt[-1][refH-1]

[0138] - Filter the upper reference sample

[0139] refFilt[x][-1]=(refUnfilt[x-1][-1]+2×refUnfilt[x][-1]+

[0140] refUnfilt[x+1][-1]+2)>>2{for x=0..refW-2}

[0141] refFilt[refW-1][-1]=refUnfilt[refW-1][-1]

[0142] As shown in the above equation, filtering can be performed between adjacent reference samples located on the same reference line. However, filtering can also be performed between reference samples located on different reference lines. For example, the filtered reference sample can be obtained by calculating the average of two samples located at MRL indexes 0 and 1.

[0143] As described above, when MIP is used, the boundary samples of the adjacent decoded blocks located above and to the left of the coding block may constitute reference samples. In this case, the boundary samples on which filtering is not performed are used as reference samples. Considering that the MIP mode is often used to predict and reconstruct low-frequency components, it may be advantageous to use sample values ​​from which high-frequency components are removed by filtering the reference samples in the MIP prediction process.

[0144] According to one aspect of the present invention, as in the directional mode of conventional intra prediction, reference samples can be filtered and then used in MIP mode. Filtered reference samples and unfiltered reference samples can be adaptively used according to the prediction mode of the block, the size of the block and the MIP mode type.

[0145] Filtering of reference samples may be performed when all of the following conditions are true. Therefore, if any of the following conditions are not met, the value of refUnfil is copied to refFilt without filtering. In some cases, when intra sub-partitioning (ISP) is applied to a coding block, if the width or height of the sub-block of the partition is greater than or equal to 16, filtering may be performed on the boundary of the corresponding edge.

[0146] -MRL index is 0.

[0147] -The product of nTbW and nTbH is greater than 32.

[0148] - CIdx is 0 (ie, a luma sample).

[0149] - IntraSubPartitionsSplitType is ISP_NO_SPLIT (ie, ISP is not applied).

[0150] - RefFilterFlag is 1 (i.e., the intra prediction mode is one of 0, 14, 12, 10, 6, 2, 34, 66, 72, 76, 78, 80, or one of the MIP modes).

[0151] It can be determined whether to adaptively perform filtering on reference pixels according to the size of the block to which MIP is applied. As described above, the size of the block to which MIP is applied is divided into three categories, and it can be determined whether to filter the reference pixels according to idx (W, H). For example, when Idx is less than 2 (i.e., a block with a size of 4×4, 4×8, or 8×4), refUnfilt can be used, and when Idx is 2, refFilt can be used, or vice versa. As another example, when Idx is less than 1, refUnfilt is used, and when Idx is greater than or equal to 1, refFilt is used, or vice versa. As another example, when the size of the block is 4×4, 4×8, or 8×4, refUnFilt can be used, otherwise refFilt can be used, or vice versa. As another example, when the width of the block or the height of the block is equal to 4, refFilt may not be used. As another example, when any one of the width and height of the block is equal to 4, refFilt may not be used.

[0152] It can be determined whether to adaptively perform filtering on reference pixels based on the shape of the block to which the MIP is applied. For example, refFilt can be used only when the width and height of the coding block are the same. As another example, refFilt can be used only when the width and height of the coding block are different. As another example, the width and height of the coding block can be compared, and refFilt can be used only for the boundaries of the blocks with longer sides. Alternatively, refFilt can be used only for the boundaries of blocks with shorter lengths. As another example, refFilt can be used only for the boundaries of blocks whose width or height of the block is 16 or more (or 32 or more).

[0153] In some embodiments, the reference sample set bdry around the MIP-coded block may be configured as follows left and bdry top . For example, the reference sample set may be configured from luma samples mapped after LMCS (luminance mapping with chroma scaling) around the MIP-coded block, or may be configured from luma samples mapped before LMCS around the MIP-coded block. As another example, the reference sample set may be configured from luma samples of neighboring blocks before performing block boundary filtering for conventional intra prediction, or may be configured from luma samples of neighboring blocks after block boundary filtering. As another example, the reference sample set may be configured by the same process as that for performing conventional intra prediction around the MIP-coded block.

[0154] Generate input boundary vector

[0155] As described above, in predicting a coding block based on MIP, the reconstructed adjacent sample bdry on the left side of the coding block can be usedleft and the reconstructed adjacent samples bdry above the coded block top However, in the case of coding blocks whose textures have directional characteristics, this method may deteriorate the prediction performance.

[0156] Accordingly, selectively use the left With bdry top It may be advantageous to use a set of neighboring samples of the input boundary vector between the pixels of the current coding block to reflect the directional characteristics that the texture of the block may have. For example, when the pixels of the current coding block have horizontal characteristics, the left neighboring samples can be used to generate the prediction signal. Similarly, when the pixels of the current coding block have vertical characteristics, the upper neighboring samples can be used to generate the prediction signal. The set of neighboring samples to be used to generate the input boundary vector can be determined differently according to the MIP mode applied to the coding block.

[0157] In the following, for simplicity, several methods of determining a boundary vector and determining a predicted sample according to the boundary vector will be described for the case of using left neighboring samples and the case of using upper neighboring samples.

[0158] A. Use the left adjacent sample to construct the boundary vector

[0159] Figures 7a to 7c is a conceptual diagram illustrating an exemplary method of constructing a boundary vector to be input to a matrix-vector multiplication using left-side neighboring samples.

[0160] As an example, Figure 7a As shown, the size (height) of the current coding block and the boundary vector bdry red When the sizes are the same, the left adjacent sample set bdry can be used left To fill the border vector bdry of the same size red For example, each of the left neighboring samples may be included in the entries of the boundary vector.

[0161] As another example, Figure 7b As shown, we can use the adjacent sample set bdry from the left left The obtained downsampled sample set To fill the border vector bdry red For example, we can calculate the average bdry of each pair of samples. left To obtain

[0162] As another example, Figure 7c As shown, the boundary vector bdry can be filled by calculating the average value of the two pixels in each row using the two left columns adjacent to the coding block. red .

[0163] Depending on the size of the encoding block, bdry can be used adaptively left and For example, as shown in the following equation, when H ≤ 8, using bdry left Otherwise, you can use

[0164]

[0165] In some embodiments, in the case of left Generate downsampled sample set When downsampling is performed, it can be performed in different ways according to the characteristics of the video content. For example, in screen content coding (SCC), the video content can have the same pixel value or intensity value in a specific area, or can have gradually increasing / decreasing pixel values ​​or intensity values. In this case, in order to generate a downsampled sample set By only with bdry left The items corresponding to the even positions (or odd positions) of are sampled instead of being averaged from bdry left Generate downsampled sample set Might be useful for reducing the complexity of downsampling.

[0166] For example, as shown in the following equation, when bdry left has H entries from [0] to [H-1] and thus generates a If H≤8, you can use bdry left Otherwise, it can be generated by taking the entries corresponding to the even positions including 0

[0167]

[0168] In addition, by using bdry left The value of the item at the first position in is filled All items of can further reduce the complexity of downsampling.

[0169]

[0170] These simplified downsampling and downsampling by averaging operation can be selected according to the characteristics of the video content. For example, when the intra-block copy (IBC) of the coding tool suitable for screen content coding is applied to the coding block located on the left side of the current block of MIP coding, the simplified downsampling adoption item corresponding to the even position (or odd position) can be used for the coding block. As another example, when the transform skip mode (transform skipmode) often used in screen content coding is applied to the coding block located on the left side of the current block of MIP coding, the simplified downsampling adoption item corresponding to the even position (or odd position) can be used for the coding block. As another example, the downsampling by averaging operation can be completely replaced by the simplified downsampling adoption item corresponding to the even position (or odd position).

[0171] B. Construct using the boundary vector of the adjacent sample above

[0172] Figures 8a to 8c is a conceptual diagram illustrating an exemplary method of constructing a boundary vector to be input to a matrix-vector multiplication operation using upper neighboring samples.

[0173] As an example, Figure 8a As shown, the size (width) of the current coding block and the boundary vector bdry red When the sizes are the same, the upper adjacent sample set can be used to fill the boundary vector bdry of the same size red For example, each of the upper neighboring samples may be included in the entries of the boundary vector.

[0174] As another example, Figure 8b As shown, we can use the adjacent sample set bdry from above top The obtained downsampled sample set To fill the border vector bdry red For example, we can calculate the average bdry of each pair of samples. top To obtain

[0175] As another example, Figure 8c As shown, the boundary vector bdry can be filled by calculating the average value of the two pixels in each column using the two upper rows adjacent to the coding block. red .

[0176] Depending on the size of the encoding block, bdry can be used adaptively top and For example, as shown in the following equation, when W ≤ 8, bdry can be used top Otherwise, you can use

[0177]

[0178] As mentioned above, in order to generate the downsampled sample set By only with bdry top The items corresponding to the even positions (or odd positions) of are sampled instead of being averaged from bdry top Generate downsampled sample set Useful in reducing the complexity of downsampling.

[0179] For example, as shown in the following equation, when bdry top has W entries from [0] to [W-1] and thus generates a If W≤8, you can use bdry top Otherwise, it can be generated by taking the entries corresponding to the even positions including 0

[0180]

[0181] In addition, by using bdry top The value of the item at the first position in is filled All items of can further reduce the complexity of downsampling.

[0182]

[0183] These simplified downsampling and downsampling by averaging operation can be selected according to the characteristics of the video content. For example, when the intra-block copy (IBC) of the coding tool suitable for screen content coding is applied to the coding block located on the left (or above) of the current block of MIP coding, the simplified downsampling adoption item corresponding to the even position (or odd position) can be used for the coding block. As another example, when the transform skip mode frequently used in screen content coding is applied to the coding block located on the left (or above) of the current block of MIP coding, the simplified downsampling adoption item corresponding to the even position (or odd position) can be used for the coding block. As another example, the downsampling by averaging operation can be completely replaced by the simplified downsampling adoption item corresponding to the even position (or odd position).

[0184] (2) Matrix-vector multiplication

[0185] According to the MIP technique described in VVC draft 5, a boundary vector bdry of length 4 or 8 is obtained from the left adjacent sample set and the above adjacent sample set. red . Set the boundary vector bdry redInput to the vector-matrix multiplication operation. As opposed to applying the vector-matrix multiplication directly to the boundary vector bdry red , from the boundary vector bdry before applying the vector-matrix multiplication red Removing the DC component and adding the DC component after applying the vector-matrix multiplication may be more advantageous in terms of computation and hardware complexity. According to this method, all entries of the weight matrix used for the vector-matrix multiplication can be expressed as unsigned integers. That is, before applying the vector-matrix multiplication to the boundary vector bdry red Before, make the boundary vector bdry red It may be advantageous to include items in with a mean of zero or to convert them to values ​​close to zero.

[0186] As an example, before applying the vector-matrix multiplication, one can start with bdry red Subtract the value of bdry from each item red As another example, before applying the vector-matrix multiplication, you can get the pixel value from bdry red Subtract bdry from each item red As another example, before applying the vector-matrix multiplication, we can get the average value of red Subtract bdry from each item red The pixel value of the first item.

[0187] As an exemplary embodiment, the video encoder and the video decoder may calculate the boundary vector bdry red The average value of the vector matrix multiplication is applied to the vector bdry from the boundary vector red The input boundary vector to be input into the vector-matrix multiplication operation. red It can be defined as follows. Here, p_avr is the boundary vector bdry red The bitDepth represents the brightness bit depth.

[0188] When MipSizeId(=idx(W,H)) is 0 or 1,

[0189] input red [0]=p_avr-(1<<(bitDepth-1))

[0190] input red [j]=bdry red [j]-p_avr,j=1,…,size(bdry red )-1

[0191] When MipSizeId(=idx(W,H)) is 2,

[0192] input red [j]=bdry red [j+1]-p_avr,j=0,…,size(bdry red )-2

[0193] As another exemplary embodiment, in order to avoid calculating the average value, the boundary vector bdry may be used red In this case, the input boundary vector to be input into the vector-matrix multiplication operation is red It can be defined as follows.

[0194] When MipSizeId(=idx(W,H)) is 0 or 1,

[0195] input red [j]=bdry red [j]-bdry red [0], j = 1, ..., size (bdry red )-1

[0196] input red [0] = bdry red [0]-(1<<(bitDepth-1))

[0197] When MipSizeId(=idx(W,H)) is 2,

[0198] input red [j]=bdry red [j+1]-bdry red [0],j=0,…,size(bdry red )-2

[0199] That is, when MipSizeId is 0 or 1, based on half of the maximum value that can be represented by the bit depth ("1<<(bitDepth-1)") and the boundary vector bdry red The input boundary vector input is obtained by the difference between the first terms of red The first term, and based on the boundary vector bdry red Subtract the first term from each term to obtain the input boundary vector input red When MipSizeId = 2, the differential vector input red The length of is 7, and accordingly, the set S for this case 2 The weight matrix A 2Each has 64 rows and 7 columns (in VVC draft 5, the weight matrix A 2 Each has 64 rows and 8 columns).

[0200] In addition, by using the weight matrix (A 0 , A 1 , A 2 ) minus the offset vector to obtain the matrix mWeight[x][y], which can be obtained from pred red =A·bdr yre d+b removes the offset vector b, but will obtain a slightly erroneous predicted value. According to the improved method, the predicted sample set pred red [x][y] can be calculated as follows.

[0201]

[0202] here, and inSize is input red The size of [j].

[0203] Linear interpolation

[0204] When pred red When the number of entries in is less than the number of samples in the prediction block, interpolation is required. The downsampled prediction block with sizes predW and predH is defined as follows.

[0205] pred red [x][y], where x = 0...predW-1, y = 0...predH-1

[0206] A prediction block having a size of (nTbW, nTbH) corresponding to a coding block, in which prediction signals at remaining positions are to be generated by linear interpolation in each direction, is defined as follows.

[0207] predSamples[x][y], where x=0...nTbW-1, y=0...nTbH-1

[0208] According to the horizontal upsampling scaling factor upHor (=nTbW / predW) and the vertical upsampling scaling factor upVer (=nTbH / predH), the following is described from pred red Fill part or all of predSamples.

[0209] predSamples[(x+1)×upHor-1][(y+1)×upVer-1]=pred red [x][y]

[0210] When upHor=1, from pred red Fill all the horizontal positions of predSamples. When upVer=1, red Fill all positions in the vertical direction of predSamples.

[0211] Thereafter, the remaining blank samples of predSamples are filled by bilinear interpolation. Interpolation in the horizontal direction and interpolation in the vertical direction are upsampling processes. Interpolation can be performed in a fixed order regardless of the size of the coding block. For example, interpolation can be performed first in the horizontal direction of the coding block and then in the vertical direction. In this case, cropping can be performed before upsampling so that pred red The value of [x][y] or predSamples[x][y] is between 0 and 2 bitDepth -1.2 bitDepth -1 is the maximum value that can be expressed in a bit depth.

[0212] For interpolation, the pred of adjacent blocks red And reference pixels can be referenced as follows. For example, the original reference sample above can be assigned to the value of predSamples[x][-1], and the original reference sample on the left can be assigned to the value of predSamples[-1][y]. As another example, the luma samples in the neighboring blocks before or after the LMCS can be assigned to the predSamples[x][-1] position and the predSamples[-1][y] position. As yet another example, the luma samples in the neighboring blocks before or after the block boundary filtering for intra-frame prediction around the coding block can be assigned to the predSamples[x][-1] position and the predSamples[-1][y] position.

[0213] mip_transpose_flag

[0214] As mentioned above, VVC draft 5 supports up to 35 MIP modes depending on the size and shape of the CU. For example, for a CU with max(W, H) ≤ 8 and W×H < 32, 35 modes are available. For each CU with max(W, H) = 8 and max(W, H) > 8, 19 modes and 11 modes are used, respectively. In addition, a pair of modes (two modes) can share matrices and offset vectors to reduce memory requirements. For example, for a 4×4 coding block, mode 19 uses the transposed matrix of the matrix assigned to mode 2. In addition, by cascading in an order determined by the MIP mode and the size of the block and Generates the boundary vector bdry to be input to the vector-matrix multiplicationred .

[0215] An improved method can be used that can achieve a substantially equivalent level of coding efficiency while reducing complexity. According to another aspect of the present invention, instead of using the transpose of a matrix used by another mode in one mode, a method configured to change the concatenated boundary vector bdry can be used. red of and The new method of ordering the boundary vector bdry red The video encoder may signal a syntax element (mip_transpose_flag) indicating that the vector-matrix multiplication for each mode is to be performed. and Concatenate to form the boundary vector bdry for each mode red When the syntax element (mip_transpose_flag) indicates and When sequentially cascaded, the prediction matrix obtained by the matrix-vector multiplication operation can also be transposed. According to this method, the number of available MIP modes can be reduced by half compared with the conventional method, and the encoding complexity in terms of boundary vector generation and vector-matrix multiplication can be reduced.

[0216] Signaling MIP mode

[0217] For a coding unit (CU) encoded in an intra prediction mode, a flag indicating whether the intra prediction type is matrix-based intra prediction (MIP) may be signaled. When MIP is applied to the current CU, a syntax element indicating the MIP mode used in the current CU among multiple available MIP modes may be additionally signaled.

[0218] Unlike conventional intra prediction modes (ie, conventional intra prediction modes), the MPM list may not be used to signal the MIP mode. Instead, for example, a syntax element (eg, intra_mip_mode) may be used that indicates the MIP mode used in the current CU among multiple MIP modes and may be encoded with a truncated binary code.

[0219] The following provides a portion of an exemplary coding unit syntax proposed based on VVC draft 5. In the following syntax, the grey elements are used to provide understanding.

[0220] Table 5

[0221]

[0222] When intra_mip_flag[x0][y0] is 1, it indicates that the intra prediction type of the current block is MIP. When intra_mip_flag[x0][y0] is 0, it indicates that the intra prediction type of the current block is regular intra prediction, not MIP. When intra_mip_flag[x0][y0] is not present, it can be inferred to be equal to 0. intra_mip_mode[x0][y0] represents the MIP mode for the current block in MIP, and is expressed as a truncated binary code.

[0223] MPM (Most Probable Mode)

[0224] In the traditional method, intra-frame prediction coding using the Most Probable Mode (MPM) can be used. For example, in HEVC, a list of three MPMs is configured from the intra-frame prediction modes of the left block and the upper block. The disadvantage of this method is that more modes (intra-frame modes other than MPM) belong to non-MPMs that need to be encoded with more bits. Several methods have been proposed to expand the number of MPMs to 3 or more items (for example, 6 MPM modes). However, configuring such an MPM list with more items may require more checks and conditions, which may make the implementation more complicated.

[0225] In order to keep the configuration of the MPM list low in complexity, the MPM list including six MPM candidates can be configured using the intra-frame prediction modes of the left neighboring blocks and the upper neighboring blocks adjacent to the current block. The MPM candidates may include a default intra-frame prediction mode (e.g., a plane mode), an intra-frame prediction mode of a neighboring block, and an intra-frame prediction mode derived from the intra-frame prediction mode of a neighboring block. When the intra-frame prediction mode of a neighboring block is not used (e.g., when an inter-frame prediction is performed on a neighboring block, or when the neighboring block is located in a different slice or another tile), the intra-frame prediction mode of the neighboring block can be set to a plane mode.

[0226] According to the type of intra prediction mode of the left block mode (Left mode) and the upper block mode (Above mode), it is roughly divided into 4 cases. When the left mode and the upper mode are different from each other and the two modes are directional modes, they can be further divided according to the difference between the left mode and the upper mode to generate the MPM list. In the following table, Max refers to the larger mode between the left mode and the upper mode, and MIN refers to the smaller mode between the left mode and the upper mode.

[0227] Table 6

[0228]

[0229] The video encoder may signal a 1-bit flag (e.g., mpm_flag) indicating whether the intra prediction mode of the current block corresponds to an MPM. Typically, when the intra prediction mode of the current block corresponds to an MPM, an MPM index indicating one of the six MPMs is additionally signaled. Note that in Table 6, the plane mode is always included in the MPM list. That is, the six MPMs can be divided into plane MPMs and five non-plane MPMs. Therefore, when the intra prediction mode of the current block is an MPM mode, the encoder explicitly signals whether the intra prediction mode of the current block is a plane mode (e.g., using a 1-bit flag), and when the intra prediction mode of the current block is the same as one of the other five non-plane MPMs, it is effective to additionally signal an MPM index indicating one of the other five non-plane MPMs. When the intra prediction mode of the current block does not correspond to any MPM, a truncated binary code may be used to encode a syntax element indicating one of the remaining 61 non-MPMs other than the six MPMs.

[0230] A. Remove the mapping table between MIP mode and normal mode

[0231] In VVC draft 5, the MPM list is used to signal the MIP mode and the normal mode, and a mapping table between the MIP mode and the normal mode is required to configure the MIP list. Due to the characteristics of the MIP technology including averaging operations and interpolation operations, the residual signal of the block to which the MIP is applied may have a low-frequency component that dominates the transform domain. The characteristics of the residual signal may be similar to the residual signal of the block to which the planar mode or DC mode is applied. Therefore, when deriving the MPM list of a block encoded in a conventional intra-frame prediction mode, the use of a mapping table between the MIP mode and the normal mode can be avoided from the perspective of the similarity of the residual signal.

[0232] According to one aspect of the present invention, in the case of deriving an MPM list for a block encoded in a conventional intra prediction mode (i.e., a conventional block), when encoding a neighboring block in a MIP mode, the intra prediction mode of the neighboring block may be regarded as a plane mode (or a DC mode). For example, when the MIP mode is applied to a neighboring block, the plane mode (or the DC mode) may be added to the MPM list in place of the MIP mode of the neighboring block. Thus, the need for the encoder and the decoder to store a mapping table between the MIP mode and the conventional mode in a memory is eliminated.

[0233] Similarly, if MIP is applied to juxtaposed luma blocks, the intra-frame prediction mode of the luma block may be considered as a plane mode (or DC mode) even when chroma DM (direct mode) is derived, instead of using a mapping table between the MIP mode and the conventional mode. The video decoder parses the syntax element that specifies the intra-frame prediction mode for the chroma block, and the syntax element may indicate that the intra-frame prediction mode of the chroma block adopts the intra-frame prediction mode of the juxtaposed luma block. In this case, when MIP is applied to juxtaposed luma blocks, the intra-frame prediction mode of the luma block may be considered as a plane mode (or DC mode). That is, when MIP is applied to juxtaposed luma blocks in a chroma direct mode (DM), it may be determined that the intra-frame prediction mode of the chroma block is a plane mode (or DC mode).

[0234] B. Merge MIP mode into regular intra mode

[0235] In VVC Draft 5, MIP is regarded as a separate intra prediction type different from conventional intra prediction, and intra_mip_flag is used to signal whether MIP is used at the CU level. When MIP is used, the MIP mode selected for the current CU is encoded. When MIP is not used, the conventional intra prediction mode selected for the current CU is encoded. In the following, an alternative method that can improve the signaling efficiency of the intra prediction mode is proposed. The proposed method is based on replacing one of the conventional modes (for example, a planar mode, a DC mode, or a directional mode) with a MIP mode, or adding a MIP mode as one of the conventional modes. When there is possible redundancy between the MIP mode and a certain conventional mode, it may be useful to replace one of the conventional modes with the MIP mode. In this case, the proposed method can improve the signaling efficiency of the intra prediction mode.

[0236] A description will be exemplarily given of a method of decoding an intra prediction mode of a coding block in a case where the DC mode among the above-mentioned 67 modes is replaced with the MIP mode.

[0237] The video decoder can use the MPM list to determine the intra_predictiton_mode[x0][y0] of the coding block. The video decoder can decode the MPM-related syntax elements to determine intra_predictiton_mode[x0][y0]. When the value of intra_predictiton_mode[x0][y0] is one of {0, 2, ..., 66}, the intra-frame prediction type of the coding block is set to normal intra-frame prediction. When the value of intra_predictiton_mode[x0][y0] is 1 (which is originally the mode index of INTRA_DC), the video decoder sets the intra-frame prediction type of the coding block to MIP, and decodes the syntax element indicating the MIP prediction mode used to encode the coding block.

[0238] An intra-frame prediction mode can be used for the left block adjacent to the current coding block to configure an MPM list including, for example, 6 MPM candidates. Depending on whether the mode of the left block (left mode (Left)) and the mode of the upper block (above mode (Above)) are directional modes, it is roughly divided into 4 cases. When the left mode and the above mode are different from each other, and both modes are directional modes, 2 more cases can be included according to the difference between the left mode and the above mode to generate the MPM list. In Table 7 below, Max refers to the larger mode between the left mode and the above mode, and MIN refers to the smaller mode between the left mode and the above mode.

[0239] Table 7

[0240]

[0241] Note that in Table 7, the MIP mode is always included in the MPM list. That is, the six MPMs can be divided into one MIP and five non-MIP MPMs. Therefore, for the encoder, 1) when the intra-frame prediction mode of the current block is the MIP mode, first use a signal to notify whether the intra-frame prediction mode of the current block is the MIP mode (for example, using a 1-bit flag), and 2) when the intra-frame prediction mode of the current block is not the MIP mode, it is effective to additionally signal the MPM index indicating one of the other five non-MIP MPMs. When the intra-frame prediction mode of the current block is not any MPM, the syntax element indicating one of the remaining 61 non-MPMs other than the six MPMs can be encoded using a truncated binary code.

[0242] Although the conventional intra prediction mode can be applied with multiple reference lines (MRL) and intra sub-partitioning (ISP) of VVC, the MIP mode is only available when the MRL index is 0 (that is, the reference sample of the first row is used) and when ISP is not applied. However, MIP mode can be available regardless of whether MRL and ISP are applied.

[0243] A portion of an exemplary intra prediction mode related syntax is provided below. In the syntax below, the grey elements are used to provide understanding.

[0244] Table 8

[0245]

[0246] The intra prediction mode of the coding block of the luma component may be signaled using syntax elements including intra_luma_mpm_flag, intra_luma_not_MIP_flag, intra_luma_mpm_idx, intra_mip_mode, and intra_luma_mpm_remainder.

[0247] intra_luma_mpm_flag indicates whether the intra prediction mode of the coding block is MPM mode. When intra_luma_mpm_flag does not exist, it is inferred that intra_luma_mpm_flag is equal to 1. intra_luma_not_MIP_flag indicates whether the intra prediction mode of the coding block is MIP mode. When intra_luma_not_MIP_flag is 1, it indicates that the intra prediction mode of the coding block is not MIP mode (that is, it is a regular intra mode). intra_mip_mode can specify one MIP mode among multiple MIP modes that can be used for the size of the current coding block, and can be encoded with a truncated binary code. When intra_luma_mpm_flag does not exist, it is inferred that intra_luma_mpm_flag is equal to 1.

[0248] intra_luma_mpm_idx specifies one MPM mode in the five non-MIP MPMs that is the same as the intra prediction mode of the coding block. intra_luma_mpm_remainder specifies one non-MPM mode in the non-MPM that is the same as the intra prediction mode of the coding block. intra_luma_mpm_remainder may be encoded using truncated binary code.

[0249] Referring to Table 8, when intra_luma_mpm_flag is 1 and MRL INDEX (index) is 0 (ie, intra_luma_ref_idx=0), the video decoder parses intra_luma_not_MIP_flag. When intra_luma_not_MIP_flag is 0, the intra prediction type of the coding block is MIP, so the video decoder decodes intra_mip_mode to identify the MIP mode used in the coding block. When intra_luma_not_MIP_flag is 1, the video decoder configures an MPM list consisting of 5 non-MIP MPMs and decodes intra_luma_mpm_idx indicating MPM INDEX. When intra_luma_mpm_flag is 0, intra_luma_mpm_remainder is decoded.

[0250] Fig. 9 is a flow chart showing a method of decoding video data with some of the above-mentioned improvements according to an embodiment of the present invention.

[0251] The video decoder may decode a syntax element indicating an intra prediction type of a current block of video data from a bitstream (S910). The intra prediction type includes matrix-based intra prediction (MIP) and conventional intra prediction. The syntax element may be a truncated binary code specified as one of a plurality of MIP modes allowed for the size and shape of the current block.

[0252] The video decoder may generate a prediction block of the current block by selectively performing MIP or normal intra prediction based on the intra prediction type of the current block.

[0253] When generating a prediction block of a current block by performing conventional intra prediction, the video decoder may perform the following operations (S920 to S940). The video decoder may configure an MPM list of the current block by deriving a most probable mode (MPM) candidate based on a conventional intra prediction mode of a neighboring block adjacent to the current block (S920), and derive a conventional intra prediction mode of the current block based on the MPM list (S930). In deriving MPM candidates based on the conventional intra prediction mode of the neighboring blocks, when the intra prediction type of the neighboring blocks is matrix-based intra prediction, the video decoder may set the conventional intra prediction mode of the neighboring blocks to (regard as) a plane mode. The decoder may generate a prediction block of the current block based on the conventional intra prediction mode of the current block (S940).

[0254] When generating a prediction block of a current block by performing matrix-based intra prediction, the video decoder may perform the following operations (S921 to S951). In order to determine the matrix-based intra prediction mode of the current block, the video decoder may decode a syntax element indicating the matrix-based intra prediction mode of the current block from a bitstream (S921). The video decoder may derive an input boundary vector based on the width and height of the current block using adjacent samples adjacent to the current block (S931), and may generate a predicted sample of the current block based on a matrix predefined by the matrix-based intra prediction mode of the current block and the input boundary vector by matrix-vector multiplication (S941). The video decoder may derive a prediction block of the current block by performing cropping and linear interpolation based on the predicted samples (S951).

[0255] In order to derive the input boundary vector using adjacent samples adjacent to the current block, the video decoder generates an initial boundary vector filled with the values ​​of adjacent samples adjacent to the current block or the downsampled values ​​from the adjacent samples according to the width and height of the current block, and the DC component can be removed from the initial boundary vector to generate the input boundary vector to which the matrix-vector multiplication is applied. For example, removing the DC component from the initial boundary vector can be subtracting the value of the first item from each item in the initial boundary vector, or including subtracting the value of the first item from each item in the initial boundary vector. The first item in the input boundary vector can be obtained based on the difference between half of the maximum value that can be expressed in bit depth and the first item of the initial boundary vector, and subsequent items of the input boundary vector can be obtained based on subtracting the value of the first item from each item in the initial boundary vector.

[0256] To generate the initial boundary vector, the video decoder may decode a syntax element from a bitstream, the syntax element indicating a concatenation order of a first item of the initial boundary vector derived from left-neighboring samples adjacent to the current block and a second item of the initial boundary vector derived from upper-neighboring samples adjacent to the current block. The video decoder may generate the initial boundary vector by concatenating the first item and the second item according to the concatenation order.

[0257] In order to derive a prediction block of a current block based on the predicted samples, the video decoder may assign the predicted samples to positions in the prediction block, and perform horizontal interpolation and vertical interpolation on the predicted samples, left adjacent samples adjacent to the current block, and upper adjacent samples adjacent to the current block to generate predicted sample values ​​of positions in the prediction block to which the predicted samples are not assigned. The horizontal interpolation may be performed before the vertical interpolation.

[0258] When the current block is a luminance block composed of luminance components and MIP is applied to the luminance block, if the intra-frame prediction mode of the luminance block is to be used as the intra-frame prediction mode of the chrominance block corresponding to the luminance block, the intra-frame prediction mode of the chrominance block can be set to planar mode.

[0259] It should be understood that the exemplary embodiments described above can be implemented in many different ways. The functions or methods described in one or more samples can be implemented as hardware, software, or any combination thereof. It should be understood that the functional components described herein have been labeled as "units" to further emphasize their implementation independence.

[0260] The various functions or methods described in the present invention can be implemented with instructions stored in a non-volatile recording medium, which can be read and executed by one or more processors. Non-volatile recording media include, for example, all types of recording devices in which data is stored in a form readable by a computer system. For example, non-volatile recording media include storage media such as erasable programmable read-only memory (EPROM), flash drives, optical drives, magnetic hard drives, and solid-state drives (SSD).

[0261] Although exemplary embodiments of the present invention have been described for illustrative purposes, it will be appreciated by those skilled in the art that various modifications and changes are possible without departing from the spirit and scope of the present invention. For the sake of brevity and clarity, exemplary embodiments have been described. Accordingly, it will be appreciated by those of ordinary skill that the scope of the embodiments is not limited by the embodiments explicitly described above, but is included in the claims and their equivalents.

Claims

1. A device for decoding video data, the device comprising at least one processor, wherein the at least one processor is configured to: decoding, from a bitstream, a syntax element indicating an intra prediction type for a current block of video data, the intra prediction type being indicated by a matrix-based intra prediction MIP and a conventional intra prediction; as well as generating a prediction block of the current block by selectively performing MIP or normal intra prediction based on the intra prediction type of the current block indicated by the syntax element, in, When performing MIP, a prediction block of a current block is generated by the following processing, the processor comprising: decoding from a bitstream a syntax element indicating a MIP mode for a current block, the syntax element represented as a truncated binary code specifying one of a plurality of MIP prediction modes allowed for a width and a height of the current block; Based on the width and height of the current block, the input boundary vector is derived using neighboring samples adjacent to the current block; generating predicted samples of the current block based on a matrix-vector multiplication between an input boundary vector and a matrix predefined for the MIP mode; and Based on the predicted samples, the predicted block of the current block is derived. Among them, using adjacent samples adjacent to the current block to derive the input boundary vector includes: generating an initial boundary vector filled with neighboring samples adjacent to the current block or downsampled values ​​from the neighboring samples according to the width and height of the current block; and Generate the input boundary vectors to which the matrix-vector multiplication is applied from the initial boundary vectors, The generating of the initial boundary vector includes: decoding a syntax element from a bitstream, the syntax element indicating a concatenated order of a first term of an initial boundary vector derived from left neighboring samples adjacent to a current block and a second term of an initial boundary vector derived from top neighboring samples adjacent to the current block; and The first term and the second term are concatenated according to the concatenation order to generate an initial boundary vector.

2. The apparatus for decoding video data according to claim 1, wherein: Generating the input boundary vector involves: The first entry of the initial bounds vector is subtracted from each subsequent entry of the initial bounds vector.

3. The apparatus for decoding video data according to claim 1, wherein: Generating the input boundary vector involves: obtaining a first term of the input boundary vector based on a difference between half of the maximum value allowed to be expressed in the bit depth and the first term of the initial boundary vector, and Subsequent entries of the input boundary vector are obtained based on subtracting the first entry of the initial boundary vector from each subsequent entry of the initial boundary vector.

4. The apparatus for decoding video data according to claim 1, wherein: The prediction block of the current block is derived based on the predicted samples including: assigning predicted samples to positions in the prediction block; and Predicted sample values ​​of positions to which predicted samples are not allocated in the prediction block are generated by performing horizontal interpolation and vertical interpolation on the predicted samples, left neighboring samples adjacent to the current block, and upper neighboring samples adjacent to the current block.

5. The apparatus for decoding video data according to claim 4, wherein: Horizontal interpolation is performed before vertical interpolation.

6. The apparatus for decoding video data according to claim 4, wherein: Before performing horizontal and vertical interpolation, the predicted samples are cropped so that the predicted samples are located between 0 and 2. bitDepth -1, where bitDepth represents the brightness bit depth.

7. The apparatus for decoding video data according to claim 1, wherein: The current block is a luminance block consisting of luminance components, Among them, when MIP is performed on a luminance block and the intra prediction mode of the luminance block is used as the intra prediction mode of a chrominance block corresponding to the luminance block, the intra prediction mode of the chrominance block is set to the planar mode.

8. The apparatus for decoding video data according to claim 1, wherein: When conventional intra prediction is performed, a prediction block of a current block is generated through the following process, which includes: deriving a most probable mode (MPM) candidate based on a conventional intra prediction mode of each of neighboring blocks adjacent to the current block, thereby configuring an MPM list of the current block; and Based on the MPM list, the conventional intra prediction mode of the current block is derived. Among them, when the intra prediction type of the neighboring block is MIP, the normal intra prediction mode of the neighboring block is regarded as the planar mode.

9. An apparatus for encoding video data, the apparatus comprising at least one processor, the at least one processor being configured to: encoding a syntax element indicating a type of intra prediction for a current block of video data, the intra prediction type indicated by matrix-based intra prediction MIP and conventional intra prediction; generating a prediction block of the current block by selectively performing MIP or normal intra prediction based on the intra prediction type of the current block indicated by the syntax element, encoding a residual block, which is the difference between the current block and the predicted block, in, When performing MIP, the processor generates a prediction block for the current block by the following process, the process comprising: encoding a syntax element indicating a MIP mode for a current block, the syntax element represented as a truncated binary code specifying one of a plurality of MIP prediction modes allowed for a width and a height of the current block; Based on the width and height of the current block, the input boundary vector is derived using neighboring samples adjacent to the current block; generating predicted samples of the current block based on a matrix-vector multiplication between an input boundary vector and a matrix predefined for the MIP mode; and Based on the predicted samples, the predicted block of the current block is derived. Among them, using adjacent samples adjacent to the current block to derive the input boundary vector includes: generating an initial boundary vector filled with neighboring samples adjacent to the current block or downsampled values ​​from the neighboring samples according to the width and height of the current block; and Generate the input boundary vectors to which the matrix-vector multiplication is applied from the initial boundary vectors, The generating of the initial boundary vector includes: encoding a syntax element indicating a concatenation order of a first term of an initial boundary vector derived from left neighboring samples adjacent to the current block and a second term of an initial boundary vector derived from top neighboring samples adjacent to the current block; and The first term and the second term are concatenated according to the concatenation order to generate an initial boundary vector.

10. An apparatus for transmitting a bitstream comprising encoded video data, the apparatus comprising at least one processor, the at least one processor being configured to: generating a bitstream by encoding a current block using intra prediction; Sending a bitstream to a video decoding device, in, The processor is configured to generate a bitstream by the following process, the process comprising: encoding a syntax element indicating a type of intra prediction for a current block of video data, the intra prediction type indicated by matrix-based intra prediction MIP and conventional intra prediction; generating a prediction block of the current block by selectively performing MIP or normal intra prediction based on the intra prediction type of the current block indicated by the syntax element, encoding a residual block, which is the difference between the current block and the predicted block, Among them, generating a prediction block of the current block by executing MIP includes: encoding a syntax element indicating a MIP mode for a current block, the syntax element represented as a truncated binary code specifying one of a plurality of MIP prediction modes allowed for a width and a height of the current block; Based on the width and height of the current block, the input boundary vector is derived using neighboring samples adjacent to the current block; generating predicted samples of the current block based on a matrix-vector multiplication between an input boundary vector and a matrix predefined for the MIP mode; and Based on the predicted samples, the predicted block of the current block is derived. Among them, using adjacent samples adjacent to the current block to derive the input boundary vector includes: generating an initial boundary vector filled with neighboring samples adjacent to the current block or downsampled values ​​from the neighboring samples according to the width and height of the current block; and Generate the input boundary vectors to which the matrix-vector multiplication is applied from the initial boundary vectors, The generating of the initial boundary vector includes: encoding a syntax element indicating a concatenation order of a first term of an initial boundary vector derived from left neighboring samples adjacent to the current block and a second term of an initial boundary vector derived from top neighboring samples adjacent to the current block; and The first term and the second term are concatenated according to the concatenation order to generate an initial boundary vector.

Images