Method and apparatus for intra predictive coding of video data

Abstract

The invention relates to a method and an apparatus for intra prediction coding of video data. According to one aspect of the present invention, an apparatus for decoding video data comprises a decoding unit for decoding a syntax element indicating an intra prediction type of a current block of video data from a bitstream, and an intra prediction unit, and an intra prediction unit for generating a prediction block of the current block by selectively performing matrix-based intra prediction or regular intra prediction based on the intra prediction type of the current block indicated by the syntax element.

Description

[0001] This application is a divisional application of the PCT patent application entered into China with Chinese patent application number 202080046562.3, invention name “Method and device for intra-frame prediction encoding of video data”, and application date June 24, 2020.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims priority to Patent Application No. 10-2019-0075163 filed in Korea on June 24, 2019 and Patent Application No. 10-2019-0080749 filed in Korea on July 4, 2019, which are hereby incorporated by reference in their entirety. Technical Field

[0004] The present invention relates to encoding and decoding of video data. Background Art

[0005] Since the amount of video data is larger than that of voice data or still image data, storing or transmitting video data without compression requires a large amount of hardware resources (including memory). Accordingly, when storing or transmitting video data, an encoder is usually used to compress the video data for storage or transmission.

[0006] Then, the decoder receives the compressed video data, decompresses and reproduces the video data. Compression technologies used for such videos include H.264 / AVC and High Efficiency Video Coding (HEVC), which improves coding efficiency by about 40% over H.264 / AVC.

[0007] However, image size, resolution and frame rate are gradually increasing, and accordingly, the amount of data to be encoded is also increasing. Therefore, a new compression technology with better encoding efficiency and higher image quality than existing compression technologies is needed. Summary of the invention

[0008] The present invention proposes an improved technique for intra-frame predictive coding of blocks of video data.

[0009] According to one aspect of the present invention, a method for decoding video data is provided, comprising: decoding a syntax element indicating an intra-prediction type of a current block of video data from a bitstream, the intra-prediction type being indicated by matrix based intra-prediction (MIP) and conventional intra-prediction; and generating a prediction block of the current block by selectively performing MIP or conventional intra-prediction based on the intra-prediction type of the current block indicated by the syntax element.

[0010] As part of generating a prediction block of a current block by performing conventional intra prediction, the method includes: deriving a most probable mode (MPM) candidate based on the conventional intra prediction mode of each of the neighboring blocks adjacent to the current block, and configuring an MPM list of the current block; and deriving the conventional intra prediction mode of the current block based on the MPM list. When the intra prediction type of the neighboring block is MIP, the conventional intra prediction mode of the neighboring block is set to a planar (PLANAR) mode.

[0011] As part of generating a prediction block for a current block by performing conventional intra prediction, the method includes: decoding a syntax element indicating a MIP mode for the current block from a bitstream; deriving a boundary vector using neighboring samples adjacent to the current block based on a size and shape of the current block; generating predicted samples for the current block based on a matrix-vector multiplication between the boundary vector and a matrix predefined for the MIP mode; and deriving a prediction block for the current block based on the predicted samples.

[0012] According to another aspect of the present invention, there is provided an apparatus for decoding video data, comprising: a decoder and an intra-frame predictor, wherein the decoder is configured to decode a syntax element indicating an intra-frame prediction type of a current block of video data from a bitstream, wherein the intra-frame prediction type is indicated by a matrix-based intra-frame prediction (MIP) and a conventional intra-frame prediction; and the intra-frame predictor is configured to generate a prediction block of the current block by selectively performing MIP or conventional intra-frame prediction based on the intra-frame prediction type of the current block indicated by the syntax element.

[0013] As part of generating a prediction block of a current block by performing conventional intra prediction, the intra predictor is configured to: derive a most probable mode (MPM) candidate based on a conventional intra prediction mode of each of neighboring blocks adjacent to the current block, and configure an MPM list of the current block; and derive a conventional intra prediction mode of the current block based on the MPM list. When deriving the MPM candidate, when the intra prediction type of the neighboring block is MIP, the intra predictor sets (regards) the conventional intra prediction mode of the neighboring block to a planar mode.

[0014] As part of generating a prediction block for a current block by performing conventional intra prediction, the intra predictor may: decode from a bitstream a syntax element indicating a MIP mode for the current block; derive a boundary vector using neighboring samples adjacent to the current block based on a size and shape of the current block; generate predicted samples for the current block based on a matrix-vector multiplication between the boundary vector and a matrix predefined for the MIP mode; and derive a prediction block for the current block based on the predicted samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is an exemplary block diagram of a video encoding device capable of implementing the techniques of the present invention.

[0016] Figure 2 A block partition structure using a QTBTTT structure is exemplarily shown.

[0017] Figure 3a A plurality of intra prediction modes are exemplarily shown.

[0018] Figure 3b A plurality of intra prediction modes including a wide-angle intra prediction mode are exemplarily shown.

[0019] Figure 4 is an exemplary block diagram of a video decoding device capable of implementing the techniques of the present invention.

[0020] Figure 5 is a conceptual diagram showing the main process of the MIP technology that can be used in the technology of the present invention.

[0021] Figure 6 is a flowchart illustrating an exemplary process of predicting a block based on MIP according to an embodiment of the present invention.

[0022] Figures 7a to 7c is a conceptual diagram illustrating an exemplary method of constructing a boundary vector input to a matrix-vector multiplication operation using left neighboring samples.

[0023] Figures 8a to 8c is a conceptual diagram illustrating an exemplary method of constructing a boundary vector input to a matrix-vector multiplication operation using upper neighboring samples.

[0024] Fig. 9 The left neighboring samples, the boundary vector and the predicted samples predicted thereby are shown.

[0025] Fig.10 The upper neighboring samples, the boundary vector and the predicted samples predicted thereby are shown.

[0026] Fig.11 is a flowchart illustrating a method of decoding video data according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] Hereinafter, some embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be noted that when reference numerals are added to the constituent elements in the various drawings, the same reference numerals also represent the same elements, although the elements are shown in different drawings. In addition, in the following description of the present invention, the detailed description of the known functions and configurations incorporated herein will be omitted to avoid blurring the subject matter of the present invention.

[0028] Figure 1 is an exemplary block diagram of a video encoding device capable of implementing the technology of the present invention. Figure 1 A video encoding apparatus and elements of the apparatus are described below.

[0029] The video encoding device includes: an image divider 110, a predictor 120, a subtractor 130, a transformer 140, a quantizer 145, a rearrangement unit 150, an entropy encoder 155, an inverse quantizer 160, an inverse transformer 165, an adder 170, a filtering unit 180 and a memory 190.

[0030] Each element of the video encoding device may be implemented in hardware or software, or a combination of hardware and software. The functions of each element may be implemented as software, and a microprocessor may be implemented to implement the software functions corresponding to each element.

[0031] A video includes multiple images. Each image is divided into multiple regions, and encoding is performed on each region. For example, an image is divided into one or more tiles (tile) or / and slices (slice). In particular, one or more tiles can be defined as a tile group. Each tile or slice is divided into one or more coding tree units (coding treeunit, CTU). Each CTU is divided into one or more coding units (coding unit, CU) through a tree structure. The information applied to each CU is encoded as the syntax of the CU, and the information commonly applied to the CU included in a CTU is encoded as the syntax of the CTU. In addition, the information commonly applied to all blocks in a slice is encoded as the syntax of the slice header, and the information applied to all blocks constituting the image is encoded in the picture parameter set (Picture Parameter Set, PPS) or the picture header. In addition, the information commonly referenced by multiple images is encoded in the sequence parameter set (Sequence Parameter Set, SPS). In addition, the information commonly referenced by one or more SPSs is encoded in the video parameter set (Video Parameter Set, VPS). Information commonly applied to one tile or tile group may be encoded as the syntax of a tile header or tile group header.

[0032] The image divider 110 is configured to determine the size of a coding tree unit (CTU). Information on the size of the CTU (CTU size) is encoded as a syntax of an SPS or a PPS and transmitted to a video decoding device.

[0033] The image splitter 110 is configured to split each image constituting a video into a plurality of CTUs having a predetermined size, and then recursively split the CTUs using a tree structure. In the tree structure, a leaf node is used as a coding unit (CU), which is a basic unit of coding.

[0034] The tree structure can be a quadtree (QuadTree, QT), a binary tree (BinaryTree, BT), a ternary tree (TernaryTree, TT), or a structure formed by a combination of two or more QT structures, BT structures and TT structures, wherein the quadtree (QT) is a node (or parent node) that is divided into four slave nodes (or child nodes) of the same size, the binary tree (BT) is a node that is divided into two slave nodes, and the ternary tree (TT) is a node that is divided into three slave nodes at a ratio of 1:2:1. For example, a quadtree plus binary tree (QuadTree plus BinaryTree, QTBT) structure can be used, or a quadtree plus binary tree ternary tree (QuadTree plus BinaryTree TernaryTree, QTBTTT) structure can be used. Here, BTTT can be collectively referred to as a multiple-type tree (MTT).

[0035] Figure 2 The QTBTTT segmentation tree structure is shown as an example. Figure 2 As shown, the CTU can be first split into a QT structure. The QT splitting can be repeated until the size of the split block reaches the minimum block size MinQTSize of the leaf node allowed in the QT. The first flag (QT_split_flag) indicating whether each node of the QT structure is split into four nodes of the lower layer is encoded by the entropy encoder 155 and notified to the video decoding device with a signal. When the leaf node of the QT is smaller than the maximum block size (MaxBTSize) of the root node allowed in the BT, it can be further split into one or more BT structures or TT structures. The BT structure and / or the TT structure can have multiple splitting directions. For example, there can be two directions, namely, the direction of splitting the blocks of the nodes horizontally and the direction of splitting the blocks vertically. As Figure 2As shown, when the MTT splitting starts, the second flag (mtt_split_flag) indicating whether the node is split, the flag indicating the splitting direction (vertical or horizontal) in the case of splitting, and / or the flag indicating the splitting type (binary or trifurcated) are encoded by the entropy encoder 155, and the video decoding device is notified of the same by a signal. Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node is split into 4 nodes of the lower layer, the CU split flag (split_cu_flag) indicating whether the node is split may be encoded. When the value of the CU split flag (split_cu_flag) indicates that the splitting is not performed, the block of the node becomes a leaf node in the split tree structure and is used as a coding unit (CU), which is a basic unit of encoding. When the value of the CU split flag (split_cu_flag) indicates that the splitting is performed, the video encoding device starts encoding the flags from the first flag in the above-mentioned manner.

[0036] When QTBT is used as another example of a tree structure, there may be two types of segmentation, namely, a type of horizontally segmenting a block into two blocks of the same size (i.e., symmetrical horizontal segmentation) and a type of vertically segmenting a block into two blocks of the same size (i.e., symmetrical vertical segmentation). A segmentation flag (split_flag) indicating whether each node of the BT structure is segmented into blocks of the lower layer and segmentation type information indicating the segmentation type are encoded by the entropy encoder 155 and transmitted to the video decoding device. There may be an additional type of segmenting a block of a node into two asymmetric blocks. An asymmetric segmentation type may include a type of segmenting a block into two rectangular blocks at a size ratio of 1:3, or a type of segmenting a block of a node diagonally.

[0037] A CU may have various sizes according to the QTBT or QTBTTT partitioning of a CTU. Hereinafter, a block corresponding to a CU to be encoded or decoded (i.e., a leaf node of a QTBTTT) is referred to as a "current block". When QTBTTT partitioning is adopted, the shape of the current block may be a square or a rectangle.

[0038] The predictor 120 predicts the current block to generate a predicted block. The predictor 120 includes an intra predictor 122 and an inter predictor 124.

[0039] Typically, each current block in an image can be predictively encoded separately. Typically, prediction of the current block is performed using an intra-frame prediction technique (which uses data from an image including the current block) or an inter-frame prediction technique (which uses data from an image encoded before the image including the current block). Inter-frame prediction includes both unidirectional prediction and bidirectional prediction.

[0040] The intra-frame predictor 122 is configured to predict pixels in the current block using pixels (reference pixels) located around the current block in the current image including the current block. There are multiple intra-frame prediction modes according to the prediction direction. For example, Figure 3a and Figure 3b As shown, multiple intra prediction modes may include 2 non-directional modes and 65 directional modes, wherein the 2 non-directional modes include a planar mode and a DC mode. The neighboring pixels and equations to be used are defined differently for each prediction mode. The following table lists the intra prediction mode numbers and their names.

[0041] Table 1

[0042] Intra prediction mode Related Name 0 INTRA_PLANAR 1 INTRA_DC 2…66 INTRA_ANGULAR2…INTRA_ANGULAR66

[0043] In order to perform effective directional prediction for the current block of rectangular shape, the Figure 3b The dotted arrows in the figure indicate the directional modes (intra-prediction modes 67 to 80 and -1 to -14). These modes may be referred to as "wide angle intra-prediction modes". Figure 3b , the arrows indicate the corresponding reference samples used for prediction, rather than indicating the prediction direction. The prediction direction is opposite to the direction indicated by the arrow. The wide-angle intra prediction mode is a mode in which prediction is performed in a direction opposite to a specific direction mode without additional bit transmission when the current block has a rectangular shape. Specifically, in the wide-angle intra prediction mode, some wide-angle intra prediction modes that can be used for the current block can be determined based on the ratio of the width to the height of the rectangular current block. For example, when the height of the rectangular shape of the current block is less than its width, a wide-angle intra prediction mode (intra prediction modes 67 to 80) with an angle less than about 45 degrees can be utilized. When the height of the rectangular shape of the current block is greater than its width, a wide-angle intra prediction mode (intra prediction modes -1 to -14) with an angle greater than about -135 degrees can be utilized.

[0044] The intra-frame predictor 122 can be configured to determine the intra-frame prediction mode to be used when encoding the current block. In some examples, the intra-frame predictor 122 can be configured to encode the current block using several intra-frame prediction modes, and select the appropriate intra-frame prediction mode to be used from the tested modes. For example, the intra-frame predictor 122 can be configured to calculate the rate-distortion value using a rate-distortion analysis of several tested intra-frame prediction modes, and can be configured to select the intra-frame prediction mode with the best rate-distortion characteristics from the tested modes.

[0045] The intra-frame predictor 122 is configured to select an intra-frame prediction mode from a plurality of intra-frame prediction modes and predict the current block using adjacent pixels (reference pixels) and equations determined according to the selected intra-frame prediction mode. Information about the selected intra-frame prediction mode is encoded by the entropy encoder 155 and transmitted to the video decoding device.

[0046] In addition, the intra predictor 122 can be configured to generate a prediction block of the current block using a matrix based intra prediction (Matrix based Intra-Prediction, MIP), which will be described later. The intra predictor 122 is configured to generate a prediction block of the current block using a boundary vector derived from samples reconstructed on the left side of the current block and samples reconstructed above the current block, a predefined matrix, and an offset vector.

[0047] In addition, the inter-frame predictor 124 is configured to generate a prediction block of the current block by motion compensation. The inter-frame predictor 124 is configured to search for a block that is most similar to the current block in a reference image that has been encoded and decoded earlier than the current image, and use the searched block to generate a prediction block of the current block. Then, the inter-frame predictor is configured to generate a motion vector corresponding to the displacement between the current block in the current image and the prediction block in the reference image. Typically, motion estimation is performed on the luminance (luma) component, and the motion vector calculated based on the luminance component is used for both the luminance component and the chrominance component. The motion information including information about the reference image and information about the motion vector used to predict the current block is encoded by the entropy encoder 155 and transmitted to the video decoding device.

[0048] The subtractor 130 is configured to subtract the prediction block generated by the intra predictor 122 or the inter predictor 124 from the current block to generate a residual block.

[0049] The transformer 140 may be configured to segment the residual block into one or more transform blocks and apply a transform to one or more transform blocks, thereby transforming the residual values ​​of the transform block from the pixel domain to the frequency domain. In the frequency domain, a transform block is referred to as a coefficient block containing one or more transform coefficient values. A two-dimensional transform kernel may be used for the transform, while a one-dimensional transform kernel may be used for horizontal transform and vertical transform, respectively. The transform kernel may be based on a discrete cosine transform (DCT), a discrete sine transform (DST), etc.

[0050] The transformer 140 may be configured to transform the residual signal in the residual block using the entire size of the residual block as a transform unit. In addition, the transformer 140 may be configured to partition the residual block into two sub-blocks in the horizontal direction or the vertical direction, and to transform only one of the two sub-blocks. Accordingly, the size of the transform block may be different from the size of the residual block (and thus the size of the prediction block). Non-zero residual sample values ​​may not exist or may be very rare in the untransformed sub-block. The residual samples of the untransformed sub-block are not signaled, and the residual samples of the untransformed sub-block may be regarded as "0" by the video decoding device. There may be multiple partition types according to the partition direction and the partition ratio. The transformer 140 may provide information about the coding mode (or transform mode) of the residual block to the entropy encoder 155 (e.g., information indicating whether to transform the residual block or the residual sub-block, information indicating the partition type selected to partition the residual block into sub-blocks, and information identifying the sub-blocks on which the transform is performed). The entropy encoder 155 may be configured to encode information about an encoding mode (or a transform mode) of the residual block.

[0051] The quantizer 145 is configured to quantize the transform coefficients output from the transformer 140, and output the quantized transform coefficients to the entropy encoder 155. For some blocks or frames, the quantizer 145 may be configured to directly quantize the relevant residual block without transformation.

[0052] The rearrangement unit 150 can be configured to rearrange the coefficient values ​​of the quantized residual values. The rearrangement unit 150 can be configured to change the 2-dimensional coefficient array into a 1-dimensional coefficient sequence by coefficient scanning. For example, the rearrangement unit 150 can be configured to scan the coefficients from the DC coefficient to the coefficients in the high-frequency region using a zig-zag scan or a diagonal scan to output a 1-dimensional coefficient sequence. According to the size of the transform unit and the intra-frame prediction mode, a vertical scan or a horizontal scan can be used instead of a zig-zag scan, wherein the vertical scan scans the two-dimensional array of coefficients in the column direction, and the horizontal scan scans the coefficients in the two-dimensional block shape in the row direction. In other words, the scanning mode to be used can be determined among the zig-zag scan, the diagonal scan, the vertical scan, and the horizontal scan according to the size of the transform unit and the intra-frame prediction mode.

[0053] The entropy encoder 155 is configured to encode the one-dimensional quantized transform coefficient output from the rearrangement unit 150 using various encoding techniques such as Context-based Adaptive Binary Arithmetic Code (CABAC) and exponential Golomb to generate a bitstream.

[0054] In addition, the entropy encoder 155 is configured to encode information related to block partitioning (e.g., CTU size, CU partition flag, QT partition flag, MTT partition type, and MTT partition direction) so that the video decoding device can partition the block in the same manner as the video encoding device. In addition, the entropy encoder 155 is configured to encode information about a prediction type indicating whether the current block is encoded by intra-frame prediction or inter-frame prediction, and encode intra-frame prediction information (i.e., information about an intra-frame prediction mode) or inter-frame prediction information (i.e., information about a reference image index and a motion vector) according to the prediction type.

[0055] The inverse quantizer 160 is configured to inversely quantize the quantized transform coefficient output from the quantizer 145 to generate a transform coefficient. The inverse transformer 165 is configured to transform the transform coefficient output from the inverse quantizer 160 from the frequency domain to the spatial domain and reconstruct a residual block.

[0056] The adder 170 is configured to add the reconstructed residual block and the prediction block generated by the predictor 120 to reconstruct the current block. Pixels in the reconstructed current block are used as reference pixels when performing intra prediction of a subsequent block.

[0057] The filtering unit 180 is configured to filter the reconstructed pixels to reduce blocking artifacts, ringing artifacts, and blurring artifacts caused by block-based prediction and transform / quantization. The filtering unit 180 may include a deblocking filter 182 and a sample adaptive offset (SAO) filter 184.

[0058] The deblocking filter 180 is configured to filter the boundary between the reconstructed blocks to remove the block artifacts caused by block-by-block encoding / decoding, and the SAO filter 184 is configured to perform additional filtering on the deblocking filtered video. The SAO filter 184 is a filter for compensating for the difference between the reconstructed pixels and the original pixels caused by lossy coding.

[0059] The reconstructed blocks filtered by the deblocking filter 182 and the SAO filter 184 are stored in the memory 190. Once all blocks in one image are reconstructed, the reconstructed image can be used as a reference image for inter-frame prediction of blocks in subsequent images to be encoded.

[0060] Figure 4 is an exemplary functional block diagram of a video decoding device capable of implementing the technology of the present invention. Figure 4A video decoding apparatus and elements of the apparatus are described below.

[0061] The video decoding apparatus may include an entropy decoder 410 , a rearrangement unit 415 , an inverse quantizer 420 , an inverse transformer 430 , a predictor 440 , an adder 450 , a filtering unit 460 , and a memory 470 .

[0062] Similar to Figure 1 In the video encoding device of the present invention, each element of the video decoding device can be implemented by hardware, software, or a combination of hardware and software. In addition, the function of each element can be implemented by software, and the microprocessor can be implemented to implement the software function corresponding to each element.

[0063] The entropy decoder 410 is configured to determine a current block to be decoded by decoding a bitstream generated by a video encoding device and extracting information related to block partitioning, and extract prediction information required to reconstruct the current block and information about a residual signal, etc.

[0064] The entropy decoder 410 is configured to extract information about the size of the CTU from a sequence parameter set (SPS) or a picture parameter set (PPS), determine the size of the CTU, and divide the image into CTUs of the determined size. Then, the decoder is configured to determine the CTU as the highest level (i.e., the root node) of the tree structure and extract the partitioning information about the CTU to partition the CTU using the tree structure.

[0065] For example, when the CTU is segmented using the QTBTTT structure, the first flag (QT_split_flag) related to the segmentation of the QT is extracted to segment each node into four nodes of the sublayer. For the node corresponding to the leaf node of the QT, the second flag (MTT_split_flag) related to the segmentation of the MTT and the information about the segmentation direction (vertical / horizontal) and / or the segmentation type (binary / trifurcated) are extracted to segment the corresponding leaf node with the MTT structure. Thus, each node below the leaf node of the QT is recursively segmented with the BT or TT structure.

[0066] As another example, when a CTU is split using a QTBTTT structure, a CU split flag (split_cu_flag) indicating whether the CU is split may be extracted. When the corresponding block is split, a first flag (QT_split_flag) may be extracted. In a split operation, after zero or more recursive QT splits, zero or more recursive MTT splits may occur for each node. For example, a CTU may directly undergo MTT splitting without undergoing QT splitting, or may only undergo QT splitting multiple times.

[0067] As another example, when the CTU is segmented using the QTBT structure, the first flag (QT_split_flag) related to the QT segmentation is extracted, and each node is segmented into four nodes of the lower layer. Then, the segmentation flag (split_flag) indicating whether the node corresponding to the leaf node of the QT is further segmented with the BT and the segmentation direction information are extracted.

[0068] Once the current block to be decoded is determined by tree structure partitioning, the entropy decoder 410 is configured to extract information about the prediction type indicating whether the current block is intra-predicted or inter-predicted. When the prediction type information indicates intra-prediction, the entropy decoder 410 is configured to extract syntax elements for intra-prediction information (intra-prediction mode) of the current block. When the prediction type information indicates inter-prediction, the entropy decoder 410 is configured to extract syntax elements for inter-prediction information, that is, information indicating a motion vector and a reference image referenced by the motion vector.

[0069] The entropy decoder 410 is configured to extract information about the encoding mode of the residual block from the bitstream (e.g., information about whether to encode the residual block or only encode a sub-block of the residual block, information indicating a partition type selected to partition the residual block into sub-blocks, information identifying the encoded residual sub-block, a quantization parameter, etc.). The entropy decoder 410 is also configured to extract information about the quantized transform coefficient of the current block as the information about the residual signal.

[0070] The rearrangement unit 415 may be configured to change a sequence of one-dimensional quantized transform coefficients entropy-decoded by the entropy decoder 410 into a 2-dimensional coefficient array (ie, block) in the reverse order of coefficient scanning performed by the video encoding apparatus.

[0071] The inverse quantizer 420 is configured to inversely quantize the quantized transform coefficients. The inverse transformer 430 is configured to inversely transform the inversely quantized transform coefficients from the frequency domain to the spatial domain based on the information about the encoding mode of the residual block to reconstruct the residual signal, thereby generating a reconstructed residual block of the current block.

[0072] When the information about the encoding mode of the residual block indicates that the residual block of the current block has been encoded by the video encoding device, the inverse transformer 430 uses the size of the current block (and therefore the size of the residual block to be reconstructed) as the transform unit of the inverse quantized transform coefficient to perform an inverse transform, thereby generating a reconstructed residual block of the current block.

[0073] When the information about the encoding mode of the residual block indicates that only one subblock of the residual block has been encoded by the video encoding device, the inverse transformer 430 uses the size of the transformed subblock as the transform unit of the inverse quantized transform coefficient to perform inverse transform, thereby reconstructing the residual signal of the transformed subblock, and fills the residual signal of the untransformed subblock with the value "0" to generate a reconstructed residual block of the current block.

[0074] The predictor 440 may include an intra predictor 442 and an inter predictor 444. The intra predictor 442 is activated when the prediction type of the current block is intra prediction, and the inter predictor 444 is activated when the prediction type of the current block is inter prediction.

[0075] The intra-frame predictor 442 is configured to determine the intra-frame prediction mode of the current block in multiple intra-frame prediction modes based on the syntax elements of the intra-frame prediction mode extracted from the entropy decoder 410, and predict the current block using reference pixels around the current block according to the intra-frame prediction mode. In addition, the intra-frame predictor 442 can be configured to generate a prediction block of the current block using matrix-based intra-frame prediction (MIP), which will be described later. The intra-frame predictor 442 can be configured to generate a prediction block of the current block using a boundary vector derived from a sample reconstructed on the left side of the current block and a sample reconstructed above the current block, as well as a predefined matrix and an offset vector.

[0076] The inter predictor 444 is configured to determine a motion vector of a current block and a reference image referred to by the motion vector using a syntax element of the intra prediction mode extracted by the entropy decoder 410 , and predict the current block based on the motion vector and the reference image.

[0077] The adder 450 is configured to reconstruct the current block by adding the residual block output from the inverse transformer 430 to the prediction block output from the inter predictor 444 or the intra predictor 442. When intra predicting a subsequent block to be decoded, pixels in the reconstructed current block are used as reference pixels.

[0078] The filtering unit 460 may include a deblocking filter 462 and an SAO filter 464. The deblocking filter 462 performs deblocking filtering on the boundaries between the reconstructed blocks to remove block artifacts caused by block-by-block decoding. The SAO filter 464 is configured to perform additional filtering on the reconstructed blocks after deblocking filtering the corresponding offsets to compensate for the difference between the reconstructed pixels and the original pixels caused by lossy encoding. The reconstructed blocks filtered by the deblocking filter 462 and the SAO filter 464 are stored in the memory 470. When all blocks in one image are reconstructed, the reconstructed image is used as a reference image for inter-frame prediction of blocks in subsequent images to be encoded.

[0079] The techniques of the present invention generally relate to intra-frame prediction coding. The following description focuses primarily on decoding techniques, that is, the operation of a video decoder. Since the encoding techniques are in contrast to the decoding techniques, which are described in detail, the encoding techniques are briefly described.

[0080] When discussing the next generation video coding standard (i.e., Versatile Video Coding (VVC)), several new coding tools were introduced that can achieve better coding performance than High Efficiency Video Coding (HEVC). Multiple Transform Selection (MTS), a technology introduced in VVC, relies on three triangular transforms, including DCT-2, DST-7, and DCT-8, and the video encoder can be configured to select horizontal transforms and vertical transforms, which minimizes the rate-distortion penalty.

[0081] Table 2

[0082]

[0083] For a block to which MTS is applied, one or more syntax elements may be encoded to signal the transform kernels used in the horizontal and vertical directions, respectively.

[0084] As an example, three flags (MTS_CU_flag, MTS_Hor_flag, MTS_Ver_flag) may be used with a predefined mapping table such as Table 3.

[0085] Table 3

[0086]

[0087] First, a flag (MTS_CU_flag) indicating whether DCT-2 is used in both the horizontal and vertical directions can be signaled. When DCT-2 is not applied in these two directions, a flag (MTS_Hor_flag) indicating the transform kernel to be applied between DST-7 and DCT-8 in the horizontal direction can be signaled, and a flag (MTS_Ver_flag) indicating the transform kernel to be applied between DST-7 and DCT-8 in the vertical direction can be signaled. DST-7 and DCT-8 can be used to achieve a block size of 32×32, while DCT-2 can be used to achieve a block size of 64×64. Accordingly, when the size (width and height) of the block is less than or equal to 32 and the block has at least one non-zero residual signal, MTS_CU_flag can be signaled. When MTS_CU_flag is not present, MTS_CU_flag can be inferred to be 0.

[0088] Alternatively, a syntax element (tu_mts_idx) indicating an MTS core index may be used with a predefined mapping table such as Table 4. In addition, tu_mts_idx indicates an MTS core index, and a core for the horizontal direction and the vertical direction is selected according to each index value. In Table 4, trTypeHor and trTypeVer indicate the type of core for the horizontal direction and the vertical direction. In particular, 0 indicates DCT-2, 1 indicates DST-7, and 2 indicates DCT-8. For example, when tu_mts_idx=2, DCT-8 is used in the horizontal direction and DST-7 is used in the vertical direction.

[0089] Table 4

[0090] tu_mts_idx 0 1 2 3 4 trTypeHor 0 1 2 1 2 trTypeVer 0) 1 1 2 2

[0091] In addition, tu_mts_idx is signaled only when sps_explicit_mts_enabled_flag in SPS is 1. This is called ExplicitMTS because the MTS index is explicitly signaled. On the other hand, tu_mts_idx may not be explicitly indicated, but is implicitly derived by the encoder and decoder, which is called ImplicitMTS. When sps_explicit_mts_enabled_flag in SPS is 0, ImplicitMTS can be performed. When ImplicitMTS is used, the selection of the transform kernel can be performed based on information such as the intra-frame prediction mode, inter-frame prediction mode, and size of the block for the block. For example, when the width of the block is greater than or equal to 4 and less than or equal to 16, DST-7 is used as the kernel in the horizontal direction. Otherwise, DCT-2 is used. In addition, when the height of the block is greater than or equal to 4 and less than or equal to 16, DST-7 is used as the kernel in the vertical direction. Otherwise, DCT-2 is used. In other words, when the block size is small, DST-7 is used because the directionality of the prediction residual signal may be high. For relatively large blocks, DCT-2 may be used as the kernel because the relatively uniform nature of the residual signal is strong.

[0092] Table 5 shows a portion of the transform unit syntax specific to VVC draft 5. In the syntax below, elements highlighted in grey are intended to provide understanding.

[0093] Table 5

[0094]

[0095] A transform block may be encoded without a transform operation, which is indicated by a syntax element transform_skip_flag signaled for a transform block having at least one non-zero coefficient. A high-level control of the encoding mode of the transform block is achieved through transform_skip_enabled_flag and log2_transform_skip_max_size_minus2, which are elements in a High-Level Syntax (HLS) such as a Sequence Parameter Set (SPS). When transform_skip_enabled_flag is 1, transform_skip_flag is encoded for transform blocks whose height and width are less than or equal to "1<<(log2_transform_skip_max_size_minus2+2)". When transform_skip_flag is 1, the associated transform block is encoded in transform skip mode. Otherwise, an MTS-based transform may be applied to the associated transform block.

[0096] Matrix-based intra prediction (MIP) is a new intra prediction technology introduced in VTM5.0. The original idea was to use a neural network-based intra prediction technology, that is, to use a multi-layer neural network to predict the current PU pixel value based on adjacent reconstructed pixels. However, due to the high complexity of the prediction method using a neural network, an intra prediction technology based on affine linear transformation using a pre-trained matrix was introduced.

[0097] To predict a rectangular block PU with width W and height H, MIP takes as input H reconstructed samples on the left side of the block and W reconstructed samples above the block. The final predicted pixels are obtained by averaging, matrix-vector multiplication, linear interpolation, etc.

[0098] The sizes of blocks to which MIP is applied are classified into the following three categories.

[0099]

[0100] According to idx(W, H), the number of MIP modes (numModes), boundary size (boundarySize) and prediction block size (predW, predH, predC) are defined as follows. In the following table, MipSizeId=idx(W, H).

[0101] Table 6

[0102] MipSizeId numModes boundarySize pred pred predC 0 35 2 4 4 4 1 19 4 4 4 4 2 11 4 Min(nTbW,8) Min(nTbH,8) 8

[0103] Figure 5 is a conceptual diagram showing the main process of the MIP technology that can be used in the technology of the present invention.

[0104] (1) Averaging

[0105] The main purpose of this process is to normalize the reference samples. Depending on the block size and shape (i.e., MipSizeId), 4 or 8 samples are obtained. When the width and height of the current block are both 4 (i.e., W=H=4), a total of 4 samples are obtained, including 2 on the left and 2 above (boundarySize=2). In another case, a total of 8 samples are obtained, including 4 on the left and 4 above (boundarySize=4).

[0106] like Figure 5 As shown, the upper adjacent sample is bdry top The left adjacent sample is represented by bdry left Indicates. top and bdry left Perform averaging separately to obtain the downsampled sample set and Averaging is a downsampling process as follows.

[0107]

[0108] In the above equation, bDwn represents the downsampled scaling value (nTbs / boundarySize), and refS represents the original reference sample. For the left neighboring sample, the calculated redS is stored as For the upper adjacent sample, the calculated redS is stored as

[0109] Concatenate the downsampled reference samples into a vector of length 4 or 8. The reduced boundary vector bdry is input to the vector-matrix multiplication. red is defined as follows. For example, when W = H = 4 and the MIP mode is less than 18, and When W = H = 4 and the MIP mode is greater than or equal to 18, and The order of splicing and In the following equations, "mode" represents the MIP mode.

[0110]

[0111] (2) Matrix-vector multiplication

[0112] In this process, from the reduced boundary vector bdry red Generate the downsampled prediction signal pred of the current block red .pred red is the sum of the matrix-vector product and the offset and is calculated as follows.

[0113] pred red =A.bdry red +b

[0114] pred red The size of W red ×H red . Define W according to the size and shape of the current block red and H red , as shown below. The number of rows in matrix A is related to W red ×H red The same number of columns, and 4 columns when W = H = 4, or 8 columns in other cases. The offset vector b is of size W red ×H red Vector.

[0115]

[0116] For each class of block sizes, a set S of matrices A and offset vectors b that can be used for blocks is predefined 0 , S 1 and S 2 The index (0, 1, 2) of the set S is selected according to the above MipSizeId (ie, idx(W, H)), and the MIP mode applied to the current block is selected from the set S. 0 , S 1 and S 2 An extraction matrix A and an offset vector b.

[0117] Set S 0 From 18 matrices A 0 and 18 16-dimensional offset vectors b 0 and for a 4×4 block, each matrix A 0 There are 16 rows and 4 columns. Set S 1 From 10 matrices A 1 and 10 16-dimensional offset vectors b 1 and for blocks of size 4×8, 8×4, and 8×8, each matrix A 1 has 16 rows and 8 columns. Finally, the set S 2 The 6 matrices A 2 and 6 64-dimensional offset vectors b 2, and for all other block shapes, each matrix A 2 It has 64 rows and 8 columns.

[0118] (3) Pixel interpolation

[0119] Interpolation is an upsampling process. As mentioned above, pred red is the downsampled prediction signal of the original block. Specifically, the downsampled prediction block with sizes predW and predH is defined as follows.

[0120] pred red [x][y], where x = 0...predW1, y = 0...predH-1

[0121] A prediction block with an original block size (nTbW, nTbH) generated by linearly interpolating the prediction signal at the remaining positions in each direction is defined as follows.

[0122] predSamples[x][y], where x=0...nTbW-1, y=0...nTbH-1

[0123] According to the horizontal upsampling scaling factor upHor (=nTbW / predW) and the vertical upsampling scaling factor upVer (=nTbH / predH), the following is described from pred red Fill part or all of predSamples.

[0124] predSamples[(x+1)×upHor-1][(y+1)×upVer-1]=pred red [x][y]

[0125] When upHor=1, pred red All horizontal positions of predSamples in are filled. When upVer=1, pred red All vertical positions of predSamples in are filled.

[0126] Then, bilinear interpolation is used to fill the remaining blank samples of predSamples. The horizontal interpolation and vertical interpolation are upsampling processes. For the interpolation of the left and top samples in predSamples, the downsampled samples are The value assigned to predSamples[x][-1] and the original reference sample to the left is assigned to the value of predSamples[-1][y].

[0127] (4) Signaling the MIP intra prediction mode

[0128] For each coding unit (CU) that is intra-prediction encoded, a flag indicating whether a matrix-based intra prediction mode (i.e., MIP mode) is applied is transmitted. In VVC Draft 5, in order to signal the MIP mode, the MPM list is used in the same way as the conventional intra prediction mode (hereinafter referred to as "conventional intra prediction mode") that is different from the matrix-based intra prediction. For example, intra_mip_mpm_flag, intra_mip_mpm_idx, and intra_mip_mpm_remainder are used to signal the MIP mode. Intra_mip_mpm_idx is encoded using a truncated binary code, and intra_mip_mpm_remainder is encoded using a fixed length code.

[0129] Depending on the size of the coding block (CU), up to 35 MIP modes can be supported. For example, for a CU with max(W, H) ≤ 8 and W×H < 32, 35 modes are available. In addition, 19 prediction modes and 11 prediction modes are used for CUs with max(W, H) = 8 and max(W, H) > 8, respectively. In addition, a pair of modes (ie, two modes) can share matrices and offset vectors to reduce memory requirements. The specific shared mode calculation is as follows. For example, for a 4×4 coding block, mode 19 uses the transposed matrix of the matrix assigned to mode 2.

[0130]

[0131] When there is a block to which MIP is applied near a block to which a conventional intra prediction mode is applied instead of MIP (hereinafter referred to as a "conventional block"), a mapping table defined between the MIP mode and the conventional mode can be used for MPM derivation of the conventional block. The mapping table is used to derive a conventional mode of similar characteristics from the MIP mode of the adjacent block to which MIP is applied. The conventional mode derived in this way is used for MPM derivation of the conventional block. Similarly, even when MIP is applied to a collocated luminance block used in chrominance DM derivation, the mapping table can be used to derive the conventional mode of the collocated luminance block, and the derived conventional mode is used for chrominance DM derivation. The following equation expresses the mapping between the conventional mode and the MIP mode using a mapping table.

[0132] predmode MIP =map_regular_to_mip idx [predmode regular ]

[0133] predmode regular =map_mip_to_regular idx [predmodeMIP ]

[0134] As mentioned above, in VVC draft 5, when predicting a block based on MIP, the reconstructed neighboring samples bdry on the left side of the block are always used. left and the reconstructed adjacent samples bdry above the block top Used as reference samples. In the case of blocks whose texture has directional characteristics, this method may deteriorate the prediction performance. In addition, in VVC draft 5, in order to interpolate the top samples in predSamples, the downsampled sample set The value assigned to predSamples[x][-1] is as follows Figure 5 As shown, the interpolation process is more complicated than necessary. In addition, in VVC draft 5, when the MPM list is used to signal both MIP mode and normal mode, the implementation may be very complicated due to many checks and conditions, such as mapping requirements between MIP mode and normal mode.

[0135] In view of the above, the present invention proposes some improved techniques for MPM coding. According to one aspect of the present invention, a method is provided, which can reduce the implementation complexity of the MIP mode and take into account the possible directional characteristics of the texture of the block.

[0136] Figure 6 is a flowchart illustrating an exemplary process of predicting a block based on MIP according to an embodiment of the present invention.

[0137] 1. Decoding MIP mode (S610)

[0138] For a CU encoded in intra prediction mode, the video decoder decodes a flag indicating whether the intra prediction type is MIP. When MIP is applied to the current CU, the video decoder decodes a syntax element indicating the MIP mode used in the current CU from multiple available MIP modes.

[0139] Unlike conventional intra prediction modes (i.e., conventional intra prediction modes), the MPM list may not be used to signal the MIP mode. Instead, for example, a syntax element (e.g., intra_mip_mode) may be used that can be encoded with a truncated binary code and indicates the MIP mode used in the current CU among multiple MIP modes.

[0140] The following provides a portion of an exemplary coding unit syntax proposed based on VVC draft 5. In the following syntax, the grey elements are used to provide understanding.

[0141] Table 7

[0142]

[0143] When intra_mip_flag[x0][y0] is 1, it indicates that the intra prediction type of the current block is MIP. When intra_mip_flag[x0][y0] is 0, it indicates that the intra prediction type of the current block is regular intra prediction, not MIP. When intra_mip_flag[x0][y0] is not present, it can be inferred to be equal to 0. intra_mip_mode[x0][y0] specifies the MIP mode for the current block in the MIP.

[0144] The neighboring samples of the current coding block to be used for generating the prediction signal can be determined differently according to the MIP mode. As an example, for a coding block with a 4×4 size, 35 MIP modes are available, and the neighboring samples to be used for generating the prediction signal can be determined according to the range of the mode, as shown below.

[0145]

[0146] When the pixels of the current coding block have horizontal characteristics, it may be advantageous to generate the prediction signal using the left neighboring samples. Similarly, when the pixels of the current coding block have vertical characteristics, it may be advantageous to generate the prediction signal using the upper neighboring samples.

[0147] In order to select a MIP mode suitable for the directional characteristics of the block, the video encoder may be configured to apply a conventional mode of intra prediction to the block, and calculate a distortion value of the block for each conventional mode. When the distortion value of the horizontal prediction mode (e.g., INTRA_ANGULAR_14 to INTRA_ANGULAR_22) is less than the distortion value of other modes, the horizontal direction may be determined as the dominant direction of the current coding block. Conversely, when the distortion value of the vertical prediction mode (e.g., INTRA_ANGULAR_46 to INTRA_ANGULAR_54) is less than the distortion value of other modes, the vertical direction may be determined as the dominant direction of the current coding block.

[0148] When the horizontal direction is the dominant direction of the current coding block, the encoder assigns one of the MIP modes that only use the left neighboring samples to the current coding block. On the other hand, when the vertical direction is the dominant direction, the encoder can assign one of the MIP modes that only use the upper neighboring samples to the current coding block.

[0149] In the following, for simplicity, the boundary vector and the method of determining the predicted sample according to the boundary vector will be described for the case of using the left neighboring samples and the case of using the upper neighboring samples. When both the left neighboring samples and the upper neighboring samples are used at the same time, the process can be the same as the reference Figure 5 Basically the same as described.

[0150] 2-1. Boundary vector generation - using left adjacent samples (S620)

[0151] Figures 7a to 7c is a conceptual diagram illustrating an exemplary method of constructing a boundary vector input to a matrix-vector multiplication operation using left neighboring samples.

[0152] As an example, Figure 7a As shown, the size (height) of the current coding block and the boundary vector bdry red When the sizes are the same, the left adjacent sample set bdry can be used left To fill the border vector bdry of the same size red For example, each of the left neighboring samples may be included in the entries of the boundary vector.

[0153] As another example, Figure 7b As shown, the boundary vector bdry can be filled by calculating the average value of the two pixels in each row using the two left columns adjacent to the coding block. red .

[0154] As another example, Figure 7c As shown, we can use the adjacent sample set bdry from the left left The obtained downsampled sample set To fill the border vector bdry red . It can be obtained by averaging the bdry of each two samples. left To obtain

[0155] Depending on the size of the coding block, bdryleft and For example, as shown in the following equation, when H ≤ 8, use bdryleft. Otherwise, you can use

[0156]

[0157] 2-2. Boundary vector generation - using upper adjacent samples (S620)

[0158] Figures 8a to 8c is a conceptual diagram illustrating an exemplary method of constructing a boundary vector input to a matrix-vector multiplication operation using upper neighboring samples.

[0159] As an example, Figure 8aAs shown, when the size (width) of the current coding block is the same as the size of the boundary vector bdryred, the upper adjacent sample set can be used to fill the boundary vector bdryred of the same size. For example, each of the upper adjacent samples can be included in the entry of the boundary vector.

[0160] As another example, Figure 8b As shown, the boundary vector bdryred can be filled by calculating the average value of the two pixels in each column using the two upper rows adjacent to the coding block.

[0161] As another example, Figure 8c As shown, we can use the adjacent sample set bdry from above t The downsampled sample set obtained by op To fill the boundary vector bdryred. Alternatively, it can be obtained by averaging every two samples of bdrytop

[0162] Depending on the size of the encoding block, bdrytop and For example, as shown in the following equation, when H ≤ 8, bdrytop can be used. Otherwise,

[0163]

[0164] 3. Matrix-vector multiplication and offset addition (S630)

[0165] The video decoder is configured to use the boundary vector bdry red A matrix-vector multiplication is performed and the offset vector is added thereto. The matrix A and the offset vector b can be determined according to the decoded MIP mode k. As a result, a prediction signal of the original block or a subsampled prediction signal pred is generated. red .pred red The calculation is as follows.

[0166] pred red =A.bdry red +b

[0167] Here, pred red The size of W red ×H red .

[0168] When constructing the boundary vector input to the matrix-vector multiplication using the left neighboring samples, W can be determined according to the size of the coding block. red and H red , as shown below.

[0169]

[0170] When using the upper adjacent samples to construct the boundary vector input to the matrix-vector multiplication, W can be determined according to the size of the coding block. red and H red , as shown below.

[0171]

[0172] When the generated pred red When the size is less than the number of samples in the coded block, interpolation is required.

[0173] 4. Linear interpolation (S640)

[0174] Fig. 9 The left neighboring samples, the boundary vector and the predicted samples obtained thereby are shown. Fig. 9 The pixels indicated by the triangle in the prediction block predSamples[x][y] for the current coding block have the pixels generated from pred red The value assigned.

[0175] refer to Fig. 9 , various interpolation methods can be used to determine the empty pixels in the prediction block predSamples[x][y].

[0176] For example, as in the case of pixel 1, the left neighboring samples, the upper neighboring samples, and the pred allocated below the empty pixel can be used. red As another example, as in the case of pixel 2, the value of the empty pixel can be determined by using the upper neighboring sample and the pred allocated below the empty pixel. red As another example, as in the case of pixel 3, the pred values ​​allocated above and below the empty pixel can be used. red As another example, in the case of pixel 4, the left neighboring samples, the pred values ​​assigned above and below the empty pixel can be used. red The two values ​​determine the empty pixel value.

[0177] Fig.10 The upper neighboring samples, the boundary vectors and the predicted samples obtained thereby are shown. Fig.10 The pixels indicated by the triangle in the prediction block predSamples[x][y] for the current coding block have the same pixel values ​​as those generated according to pred red The value assigned.

[0178] refer to Fig.10, various interpolation methods can be used to determine the empty pixels in the prediction block predSamples[x][y]. For example, as in the case of pixel 1, the left neighboring samples, the upper neighboring samples, and the pred allocated below the empty pixel can be used. red As another example, in the case of pixel 2, the left neighboring samples, the pred values ​​assigned above and below the empty pixel can be used. red As another example, as in the case of pixel 3, the pred values ​​assigned above and below the empty pixel can be used. red As another example, in the case of pixel 4, the upper neighboring sample and the pred assigned below the empty pixel can be used. red The value of determines the empty pixel value.

[0179] and Fig. 9 and Fig.10 Different from the example shown in , in order to determine the empty pixels in the prediction block predSamples[x][y], the original reference sample bdry above can be top Assigned to predSamples[x][-1], and the left original reference sample bdry left Assigned to predSamples[-1][y].

[0180] In the above embodiments, a scheme for selectively utilizing neighboring samples of a current block according to a MIP mode is introduced, so that the directionality of the block can be considered when generating a prediction block based on MIP.

[0181] According to another aspect of the present invention, instead of changing the neighboring sample set used to generate the boundary vector according to the MIP mode, the values ​​of the matrix A and the offset b applied to the boundary vector may be modified.

[0182] The matrix A and offset b applied to the matrix-vector multiplication and offset addition of the boundary vector can be divided into two parts as follows.

[0183]

[0184] When the values ​​of A2 and b2 are approximately 0, the prediction signal pred red Mainly affected by the adjacent pixels on the left This may be useful for blocks whose texture has horizontal directionality. When the values ​​of A1 and b1 are approximately 0, the prediction signal pred red Mainly affected by the upper adjacent pixels This may be useful for blocks whose textures have a vertical directionality.

[0185] The video encoder and the video decoder may use modified values ​​of the matrix A and the offset b to be applied to the boundary vector according to the MIP mode of the current block. For example, for a 4×4 CU, 35 MIP modes may be available, and the values ​​of the matrix A and the offset b to be applied to the boundary vector may be modified according to the range of the mode.

[0186]

[0187] As mentioned above, VVC draft 5 supports up to 35 MIP modes depending on the size and shape of the CU. For example, for a CU with max(W, H) ≤ 8 and W×H < 32, 35 modes are available, for a CU with max(W, H) = 8, 19 prediction modes are available, and for a CU with max(W, H) > 8, 11 prediction modes are available. In addition, a pair of modes (two modes) can share matrices and offset vectors to reduce memory requirements. For example, for a 4×4 coding block, mode 19 uses the transposed matrix of the matrix assigned to mode 2. In addition, by cascading in an order determined by the MIP mode and the size of the block and Generates the boundary vector bdry to be input to the vector-matrix multiplication red .

[0188] An improved method can be used that can achieve a substantially equivalent level of coding efficiency while reducing complexity. According to another aspect of the present invention, instead of using the transpose of a matrix used by another mode in one mode, a configuration can be used to change the concatenated boundary vector bdry. red of and The new scheme of the order, the boundary vector bdry red The video encoder may signal a syntax element indicating that the vector matrix multiplication for each mode is to be performed. and Concatenate to form the boundary vector bdry for each mode red According to the scheme, the number of available MIP modes can be reduced by half compared with the conventional method, and the coding complexity in terms of boundary vector generation and vector-matrix multiplication can be reduced.

[0189] As described with respect to the syntax structure of Table 5, for each of the transform units satisfying a specific condition, transform_skip_flag, which is a syntax element related to skipping transform, is signaled. When transform_skip_flag is not true (TRUE), tu_mts_idx, which is a syntax element related to multiple transform selection, must be transmitted according to the flag (sps_explicit_mts_inter_enabled_flag, sps_explicit_mts_intra_enabled_flag) in the SPS, thereby imposing a burden depending on the number of bits to be transmitted in the bitstream.

[0190] However, multiple transform selection and transform skipping are closely related to the characteristics of the residual signal, and therefore, it may not be efficient to always signal transform-related syntax elements regardless of the prediction mode used in the block. It should be noted that, considering the MIP process including the averaging operation and the interpolation operation, the residual signal of the block to which the MIP is applied has a low-frequency component that dominates in the transform domain, and it is possible that the MIP will not be applied to blocks with low correlation between pixels, such as blocks containing complex motion and texture. Therefore, it is possible that transform skipping will not be applied to blocks using MIP.

[0191] In view of the above, a method of always signaling tu_mts_idx as a syntax element related to multiple transform selection and transform_skip_flag as a syntax element related to transform skipping may not be very efficient regardless of whether MIP is applied to a block.

[0192] According to one aspect of the present invention, signaling transform-related syntax elements may be omitted depending on whether MIP is applied, and the omitted syntax elements may be implicitly inferred by a video decoder.

[0193] In an implementation scheme, the MTS technique can be implicitly applied based on the MIP flag. As shown in the following table, transform_skip_flag and tu_mts_idx can be encoded only when intra_mip_flag is not true. When intra_mip_flag is true, transform_skip_flag is inferred to be 0, tu_mts_idx is also inferred to be 0, and implicit MTS is inferred to be applied. Accordingly, the transform kernel in the horizontal and vertical directions can be determined based on the block size. In the following syntax, the gray elements are used to indicate potential changes in the syntax or provide their understanding.

[0194] Table 8

[0195]

[0196] In another embodiment, the MTS technique can be applied explicitly based on the MIP flags. As shown in the following table, transform_skip_flag and tu_mts_idx can be encoded only when intra_mip_flag is not true. When intra_mip_flag is true, transform_skip_flag is inferred to be 0, tu_mts_idx is explicitly signaled, and explicit MTS is applied. In the following syntax, gray elements are used to indicate potential changes in the syntax or provide their understanding.

[0197] Table 9

[0198]

[0199] In another possible implementation, MIP may not be explicitly applied to blocks with low correlation between pixels, such as blocks of smaller size containing complex motion and texture. Transform skip (TS) may not be applied to a 4×4 CU to which MIP is applied. In addition, MIP may not be applied to a 4×4 CU to which TS is applied. In addition, MIP may not be applied together with TS to a 4×4 CU.

[0200] In the traditional method, intra-frame prediction coding using the Most Probable Mode (MPM) can be used. For example, in HEVC, a list of three MPMs is configured from the intra-frame prediction modes of the left block and the upper block. The disadvantage of this method is that more modes (intra-frame modes other than MPM) belong to non-MPMs that need to be encoded with more bits. Several methods have been proposed to expand the number of MPMs to 3 or more items (for example, 6 MPM modes). However, configuring such an MPM list with more items may require more checks and conditions, which may make the implementation plan more complicated.

[0201] As described above, in VVC draft 5, where each MPM list is used to signal MIP mode and normal mode, the implementation may be complicated due to many checks and conditions, such as the requirement for mapping between MIP mode and normal mode in order to configure the MIP list.

[0202] In order to keep the configuration of the MPM list low in complexity, the MPM list including six MPM candidates can be configured using the intra-frame prediction modes of the left neighboring blocks and the upper neighboring blocks adjacent to the current block. The MPM candidates may include: a default intra-frame prediction mode (e.g., a planar mode), an intra-frame prediction mode of a neighboring block, and an intra-frame prediction mode derived from the intra-frame prediction mode of a neighboring block. When the intra-frame prediction mode of a neighboring block is not used (e.g., when an inter-frame prediction is performed on a neighboring block, or when the neighboring block is located in a different slice or another tile), the intra-frame prediction mode of the neighboring block can be set to a planar mode (Planar).

[0203] According to the type of intra prediction mode of the left block mode (left mode) and the upper block mode (above mode), there are 4 cases. When the left mode and the upper mode are different from each other and the two modes are directional modes, 4 more cases can be included according to the difference between the left mode and the upper mode to generate the MPM list. In the following table, Max refers to the larger mode between the left mode and the upper mode, and MIN refers to the smaller mode between the left mode and the upper mode.

[0204] Table 10

[0205]

[0206] Due to the characteristics of the MIP technique including averaging and interpolation operations, the residual signal of the block to which the MIP is applied may have a low-frequency component that is dominant in the transform domain. Note that this residual characteristic is similar to the residual signal of the block to which the planar mode or DC mode is applied. Therefore, it may be useful to consider the similarity of this residual signal when deriving the MPM list of the block encoded in the conventional intra prediction mode.

[0207] According to one aspect of the present invention, when deriving an MPM list for a block encoded in a conventional intra prediction mode (i.e., a conventional block), when encoding a neighboring block in a MIP mode, the intra prediction mode of the neighboring block may be considered as a plane mode (or a DC mode). As an example, when the MIP mode is applied to a neighboring block, the plane mode (or the DC mode) may be added to the MPM list to replace the MIP mode of the neighboring block.

[0208] Similarly, if MIP is applied to juxtaposed luma blocks, the intra-frame prediction mode of the luma block can be regarded as a plane mode (or DC mode) even when chroma DM (direct mode) is derived, instead of using a mapping table between the MIP mode and the conventional mode. Accordingly, the encoder parses the syntax element specifying the intra-frame prediction mode for the chroma block, and the syntax element can indicate that the intra-frame prediction mode of the chroma block adopts the intra-frame prediction mode of the juxtaposed luma block. In this case, when MIP is applied to juxtaposed luma blocks, the intra-frame prediction mode of the luma block can be regarded as a plane mode (or DC mode). In other words, when MIP is applied to juxtaposed luma blocks in a chroma direct mode (DM), it can be determined that the intra-frame prediction mode of the chroma block is a plane mode (or DC mode).

[0209] The video encoder may signal a 1-bit flag (e.g., mpm_flag) indicating whether the intra-prediction mode of the current block corresponds to an MPM. Typically, when the intra-prediction mode of the current block corresponds to an MPM, an MPM index indicating one of the six MPMs is additionally signaled. Note that in the above table, the plane mode is always included in the MPM list. Therefore, it would be effective for the encoder to explicitly signal whether the intra-prediction mode of the current block is a plane mode (e.g., using a 1-bit flag), and when the intra-prediction mode of the current block is the same as one of the other five MPMs, to additionally signal an MPM index indicating one of the other five MPMs. When the intra-prediction mode of the current block does not correspond to any MPM, a syntax element indicating one of the remaining 61 non-MPMs other than the six MPMs may be encoded using a truncated binary code.

[0210] Fig.11 is a flowchart illustrating a method of decoding video data according to an embodiment of the present invention.

[0211] The video decoder may be configured to decode a syntax element indicating an intra prediction type of a current block of video data from a bitstream (S1110). The intra prediction type includes matrix-based intra prediction (MIP) and conventional intra prediction. The syntax element may be a truncated binary code specified as one of a plurality of MIP modes allowed for the size and shape of the current block.

[0212] The video decoder may be configured to generate a prediction block of the current block by selectively performing MIP or normal intra prediction based on the intra prediction type of the current block.

[0213] As part of generating a prediction block for a current block by performing conventional intra prediction, the video decoder may be configured to perform the following operations (S1120 to S1140). The video decoder may configure a most probable mode (MPM) list for the current block (S1120) by deriving an MPM candidate based on a conventional intra prediction mode of a neighboring block adjacent to the current block, and derive a conventional intra prediction mode for the current block based on the MPM list (S1130). When deriving the MPM candidate based on the conventional intra prediction mode of the neighboring block, the video decoder may be configured to set (regard) the conventional intra prediction mode of the neighboring block to a planar mode when the intra prediction type of the neighboring block is a matrix-based intra prediction. The decoder may be configured to generate a prediction block for the current block based on the conventional intra prediction mode of the current block (S1140).

[0214] As part of generating a prediction block for a current block by performing matrix-based intra prediction, the video decoder may be configured to perform the following operations (S1121 to S1151). The video decoder may be configured to decode a syntax element indicating a matrix-based intra prediction mode for the current block from a bitstream to determine a matrix-based intra prediction mode for the current block (S1121). The video decoder may be configured to derive a boundary vector based on the size and shape of the current block using neighboring samples adjacent to the current block (S1131), and generate a predicted sample of the current block based on a matrix-vector multiplication between a matrix predefined by the matrix-based intra prediction mode of the current block and the boundary vector (S1141). The video decoder may be configured to derive a prediction block for the current block by performing linear interpolation and cropping based on the predicted samples (S1151).

[0215] The video decoder may be configured to derive the boundary vector from one or both sets selected between a left neighboring sample set and an upper neighboring sample set adjacent to the current block according to a matrix-based intra prediction mode of the current block.

[0216] The video decoder is configured to decode a syntax element from a bitstream, the syntax element indicating a concatenation order of a first term of a boundary vector derived from left neighboring samples adjacent to a current block and a second term of the boundary vector derived from top neighboring samples adjacent to the current block. The video decoder may be configured to generate the boundary vector by concatenating the first term and the second term according to the indicated concatenation order.

[0217] The entries of the boundary vector may be filled with values ​​downsampled from left neighboring samples adjacent to the current block, or filled with left neighboring samples adjacent to the current block, depending on the size and shape of the current block.

[0218] The video decoder may assign the predicted samples to positions in the prediction block to derive the prediction block of the current block based on the predicted samples. To generate predicted sample values ​​for positions in the prediction block to which the predicted samples are not assigned, linear interpolation may be performed on the predicted samples, the left neighboring samples adjacent to the current block, and the top neighboring samples adjacent to the current block.

[0219] It should be understood that the exemplary embodiments described above can be implemented in many different ways. The functions or methods described in one or more examples can be implemented as hardware, software, or any combination thereof. It should be understood that the functional components described herein have been labeled as "units" to further emphasize their implementation independence. The various functions or methods described in the present invention can be implemented with instructions stored in a non-volatile recording medium, which can be read and executed by one or more processors. Non-volatile recording media include, for example, all types of recording devices, wherein data is stored in a form readable by a computer system. For example, non-volatile recording media include storage media, such as erasable programmable read-only memory (EPROM), flash drives, optical disk drives, magnetic hard drives, and solid-state drives (SSD).

[0220] Although exemplary embodiments of the present invention have been described for illustrative purposes, it will be appreciated by those skilled in the art that various modifications and changes are possible without departing from the spirit and scope of the present invention. For the sake of brevity and clarity, exemplary embodiments have been described. Accordingly, it will be appreciated by those of ordinary skill that the scope of the embodiments is not limited by the embodiments explicitly described above, but is included in the claims and their equivalents.

Claims

1. A device for decoding video data, the device comprising at least one processor, wherein the at least one processor is configured to: decoding, from a bitstream, a syntax element indicating an intra prediction type for a current block of video data, the intra prediction type indicated by matrix-based intra prediction and conventional intra prediction; generating a prediction block for the current block by selectively performing matrix-based intra prediction or conventional intra prediction based on the intra prediction type of the current block indicated by the syntax element; Decoding residual data of a residual block specifying a current block from a bitstream; Reconstruct the current block based on the prediction block and the residual block, in, When performing conventional intra prediction, the processor is configured to generate a prediction block of the current block by the following process, the process comprising: deriving a most probable mode candidate based on a conventional intra prediction mode of each of neighboring blocks adjacent to the current block, and configuring a most probable mode list of the current block; and Derives the conventional intra prediction mode for the current block based on the most probable mode list, wherein, in response to determining that the intra prediction type of the neighboring block is matrix-based intra prediction, setting the conventional intra prediction mode of the neighboring block to a planar mode, Wherein, when performing matrix-based intra prediction, the processor is configured to generate a prediction block of the current block by the following processing, the processing comprising: decoding, from a bitstream, a syntax element indicating a matrix-based intra prediction mode for a current block; Based on the width and height of the current block, the boundary vector is derived using neighboring samples adjacent to the current block; generating predicted samples of the current block based on a matrix-vector multiplication between the boundary vector and a matrix predefined for the matrix-based intra prediction mode; and Based on the predicted samples, the predicted block of the current block is derived. Among them, using adjacent samples adjacent to the current block to derive the boundary vector includes: decoding a syntax element from a bitstream, the syntax element indicating a concatenated order of a first term of a boundary vector derived from left neighboring samples adjacent to a current block and a second term of a boundary vector derived from top neighboring samples adjacent to the current block; and The first term and the second term are concatenated according to the concatenation order to generate a boundary vector.

2. The apparatus for decoding video data according to claim 1, wherein: The syntax element indicating the matrix-based intra prediction mode for the current block is a truncated binary code that specifies one of a plurality of matrix-based intra prediction modes allowed for the width and height of the current block.

3. The apparatus for decoding video data according to claim 1, wherein: Depending on the width and height of the current block, the entries of the boundary vector are filled with values ​​downsampled from the left neighboring samples adjacent to the current block, or the entries of the boundary vector are filled with the left neighboring samples adjacent to the current block.

4. The apparatus for decoding video data according to claim 1, wherein: The prediction block of the current block is derived based on the predicted samples including: assigning predicted samples to positions in the prediction block; and Predicted sample values ​​are generated for positions in the prediction block to which predicted samples are not allocated using linear interpolation of the predicted samples, left neighboring samples adjacent to the current block, and top neighboring samples adjacent to the current block.

5. The apparatus for decoding video data according to claim 1, wherein: The current block is a luminance block including a luminance component.

6. The apparatus for decoding video data according to claim 5, wherein: When matrix-based intra prediction is performed on a luma block and the intra prediction mode of the luma block is used as the intra prediction mode of a chroma block corresponding to the luma block, a normal intra prediction mode for the luma block is set to a planar mode.

7. An apparatus for encoding video data, the apparatus comprising at least one processor, the at least one processor being configured to: encoding into a bitstream a syntax element indicating a type of intra prediction for a current block of video data, the intra prediction type indicated by matrix-based intra prediction and conventional intra prediction; generating a prediction block of the current block by selectively performing matrix-based intra prediction or conventional intra prediction based on an intra prediction type of the current block; Use the prediction block to generate a residual block of the current block; Encode the residual data of the specified residual block, in, When performing conventional intra prediction, the processor is configured to generate a prediction block of the current block by the following process, the process comprising: determining a conventional intra prediction mode for a current block; deriving a most probable mode candidate based on a conventional intra prediction mode of each of neighboring blocks adjacent to the current block to configure a most probable mode list for the current block, wherein when the intra prediction type of the neighboring block is matrix-based intra prediction, the conventional intra prediction mode of the neighboring block is set to a planar mode; and encoding at least one syntax element indicating a normal intra prediction mode for the current block into a bitstream based on the most probable mode list, and Wherein, when performing matrix-based intra prediction, the processor is configured to generate a prediction block of the current block by the following processing, the processing comprising: encoding into a bitstream a syntax element indicating a matrix-based intra prediction mode for a current block; Based on the width and height of the current block, the boundary vector is derived using neighboring samples adjacent to the current block; generating predicted samples of the current block based on a matrix-vector multiplication between the boundary vector and a matrix predefined for the matrix-based intra prediction mode; and Based on the predicted samples, the predicted block of the current block is derived. Among them, using adjacent samples adjacent to the current block to derive the boundary vector includes: A syntax element is encoded into a bitstream, wherein the syntax element indicates a concatenation order of a first item of a boundary vector derived from left neighboring samples adjacent to a current block and a second item of the boundary vector derived from upper neighboring samples adjacent to the current block; and the first item and the second item are concatenated according to the concatenation order to generate a boundary vector.

8. An apparatus for transmitting a bitstream comprising encoded data for a block of video data, the apparatus comprising at least one processor, the at least one processor being configured to: Generate a bitstream by encoding the current block using intra prediction based on the encoding method; Sending a bitstream to a video decoding device, in, The processor is configured to generate a bitstream by the following process, the process comprising: encoding into a bitstream a syntax element indicating a type of intra prediction for a current block of video data, the intra prediction type indicated by matrix-based intra prediction and conventional intra prediction; generating a prediction block of the current block by selectively performing matrix-based intra prediction or conventional intra prediction based on an intra prediction type of the current block; Use the prediction block to generate a residual block of the current block; Encode the residual data of the specified residual block, Among them, generating a prediction block of the current block by performing conventional intra prediction includes: determining a conventional intra prediction mode for a current block; deriving a most probable mode candidate based on a conventional intra prediction mode of each of neighboring blocks adjacent to the current block to configure a most probable mode list for the current block, wherein when the intra prediction type of the neighboring block is matrix-based intra prediction, the conventional intra prediction mode of the neighboring block is set to a planar mode; and encoding at least one syntax element indicating a normal intra prediction mode for the current block into a bitstream based on the most probable mode list, and The step of generating a prediction block of a current block by performing matrix-based intra prediction includes: encoding into a bitstream a syntax element indicating a matrix-based intra prediction mode for a current block; Based on the width and height of the current block, the boundary vector is derived using neighboring samples adjacent to the current block; generating predicted samples of the current block based on a matrix-vector multiplication between the boundary vector and a matrix predefined for the matrix-based intra prediction mode; and Based on the predicted samples, the predicted block of the current block is derived. Among them, using adjacent samples adjacent to the current block to derive the boundary vector includes: encoding a syntax element into a bitstream, the syntax element indicating a concatenated order of a first term of a boundary vector derived from left neighboring samples adjacent to a current block and a second term of a boundary vector derived from top neighboring samples adjacent to the current block; and The first term and the second term are concatenated according to the concatenation order to generate a boundary vector.

Images