Method and apparatus for intra-predictive coding of video data - Patents.com

Abstract

To provide a method and apparatus for intra-prediction coding of video data.SOLUTION: An apparatus for decoding video data comprises: a decoder configured to decode, from a bitstream, a syntax element indicating an intra-prediction type of a current block of the video data; and an intra-predictor configured to generate a prediction block for the current block by selectively performing matrix based intra-prediction or regular intra-prediction based on the intra-prediction type of the current block indicated by the syntax element.SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] The present invention relates to encoding and decoding of moving image data. [Background technology]

[0002] Video data has a larger amount of data than audio data or still image data, and therefore requires a large amount of hardware resources, including memory, to store or transmit the video data without compression processing.

[0003] Therefore, in general, when video data is stored or transmitted, the video data is compressed using an encoder and then stored or transmitted, and the compressed video data is received by a decoder, decompressed, and played back. Examples of such video compression technologies include H.264 / AVC and High Efficiency Video Coding (HEVC), which has improved coding efficiency by about 40% compared to H.264 / AVC.

[0004] However, as picture sizes, resolutions, and frame rates gradually increase, the amount of data to be encoded also increases. This has created a demand for new compression techniques that offer better encoding efficiency and greater image quality improvement than conventional compression techniques. Summary of the Invention [Problem to be solved by the invention]

[0005] It is an object of the present invention to provide improved techniques for intra-predictive coding of blocks of video data. [Means for solving the problem]

[0006] A method according to one aspect of the present invention is a method for decoding video data, comprising the steps of: decoding a syntax element indicating an intra-prediction type of a current block of video data from a bitstream, the intra-prediction type including matrix based intra prediction (MIP) and regular intra prediction; and selectively performing matrix based intra prediction or regular intra prediction based on the intra-prediction type of the current block indicated by the syntax element to generate a predicted block for the current block.

[0007] The method is a part of the step of generating a predicted block for the current block by performing the regular intra prediction, and the method includes the steps of deriving Most Probable Mode (MPM) candidates based on regular intra prediction modes of neighboring blocks adjacent to the current block to construct an MPM list for the current block, and deriving a regular intra prediction mode for the current block based on the MPM list. When deriving the MPM candidates, if the intra prediction type of the neighboring blocks is matrix-based intra prediction, the regular intra prediction mode of the neighboring blocks is set to planar mode.

[0008] As part of the step of performing matrix-based intra prediction to generate a predictive block for the current block, the method includes the steps of: decoding a syntax element indicating a matrix-based intra prediction mode for the current block from the bitstream; deriving a boundary vector using surrounding samples adjacent to the current block based on a size and shape of the current block; generating predictive samples for the current block based on a matrix-vector multiplication between a matrix predefined for the matrix-based intra prediction mode and the boundary vector; and deriving a predictive block for the current block based on the predictive samples.

[0009] An apparatus according to one aspect of the present invention is an apparatus for decoding video data, characterized in that it includes: a decoding unit that decodes a syntax element indicating an intra prediction type of a current block of video data from a bitstream, the intra prediction type including matrix-based intra prediction and regular intra prediction; and an intra prediction unit that selectively performs matrix-based intra prediction or regular intra prediction based on the intra prediction type of the current block indicated by the syntax element to generate a predicted block for the current block.

[0010] As part of performing the regular intra prediction to generate a predicted block for the current block, the intra prediction unit derives MPM candidates based on regular intra prediction modes of neighboring blocks adjacent to the current block to construct an MPM list for the current block, and derives a regular intra prediction mode for the current block based on the MPM list. When deriving the MPM candidates, the intra prediction unit sets (considers) the regular intra prediction mode of the neighboring blocks to a planar mode if the intra prediction type of the neighboring blocks is matrix-based intra prediction.

[0011] As part of performing the matrix-based intra prediction to generate a predictive block for the current block, the intra prediction unit performs the steps of decoding a syntax element indicating a matrix-based intra prediction mode for the current block from the bitstream, deriving a boundary vector using surrounding samples adjacent to the current block based on the size and shape of the current block, generating predictive samples for the current block based on a matrix-vector multiplication between a matrix predefined for the matrix-based intra prediction mode and the boundary vector, and deriving a predictive block for the current block based on the predictive samples. Effect of the Invention

[0012] According to the present invention, it is possible to provide a coding technique that reduces the complexity of intra prediction and improves coding efficiency. [Brief description of the drawings]

[0013] [Figure 1] 1 is an exemplary block diagram of a video encoding device capable of implementing techniques of the present invention; [Diagram 2] FIG. 13 is a diagram for explaining a method of dividing blocks using the QTBTTT structure. [Figure 3a] FIG. 2 is a diagram illustrating multiple intra-prediction modes. [Figure 3b] FIG. 2 is a diagram illustrating multiple intra-prediction modes, including a wide-angle intra-prediction mode. [Figure 4] 1 is an exemplary block diagram of a video decoding device capable of implementing the techniques of the present invention; [Diagram 5] FIG. 1 is a schematic diagram illustrating the main processes of the MIP technology used in the present technique. [Figure 6] 1 is a flowchart illustrating a general process of MIP-based block prediction according to one embodiment of the present invention. [Figure 7a]FIG. 13 is a conceptual diagram illustrating an example method for constructing boundary vectors input to a matrix-vector multiplication operation using samples to the left. [Figure 7b] FIG. 13 is a conceptual diagram illustrating an example method for constructing boundary vectors input to a matrix-vector multiplication operation using samples to the left. [Figure 7c] FIG. 13 is a conceptual diagram illustrating an example method for constructing boundary vectors input to a matrix-vector multiplication operation using samples to the left. [Figure 8a] FIG. 13 is a conceptual diagram illustrating an example method for constructing boundary vectors input to a matrix-vector multiplication operation using upper neighbor samples. [Figure 8b] FIG. 13 is a conceptual diagram illustrating an example method for constructing boundary vectors input to a matrix-vector multiplication operation using upper neighbor samples. [Figure 8c] FIG. 13 is a conceptual diagram illustrating an example method for constructing boundary vectors input to a matrix-vector multiplication operation using upper neighbor samples. [Figure 9] The sample to the left, the boundary vector, and the predicted sample predicted from it are shown. [Figure 10] The upper neighbor sample, the boundary vector, and the predicted sample predicted from it are shown. [Figure 11] 4 is a flow chart illustrating a method for decoding video data according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Some embodiments of the present invention will be described in detail below with reference to the drawings. When identifying elements in each drawing, the same elements are assigned the same reference numerals as much as possible even if they are displayed in different drawings. In describing the present invention, if it is determined that a detailed description of related known configurations or functions would obscure the gist of the present invention, the detailed description will be omitted.

[0015] 1 is an exemplary block diagram of a video encoding device embodying the technology of the present invention. The video encoding device and its sub-components will be described below with reference to FIG.

[0016] The video encoding device includes a picture division unit 110, a prediction unit 120, a subtractor 130, a transformation unit 140, a quantization unit 145, a rearrangement unit 150, an entropy encoding unit 155, an inverse quantization unit 160, an inverse transform unit 165, an adder 170, a filter unit 180, and a memory 190.

[0017] Each component of the video encoding device may be implemented as hardware or software, or a combination of hardware and software. Furthermore, the function of each component may be implemented as software, and a microprocessor may execute the software function corresponding to each component.

[0018] A video is composed of multiple pictures. Each picture is divided into multiple regions, and coding is performed for each region. For example, a picture is divided into one or more tiles and / or slices. Here, one or more tiles are defined as a tile group. Each tile and / or slice is divided into one or more coding tree units (CTUs). Each CTU is divided into one or more coding units (CUs) by a tree structure. Information applied to each CU is coded as a CU syntax, and information commonly applied to CUs included in one CTU is coded as a CTU syntax. Information commonly applied to all blocks in one slice is coded as a slice header syntax, and information commonly applied to all blocks constituting one picture is coded in a picture parameter set (PPS) or picture header. Furthermore, information commonly referred to by multiple pictures is coded in a sequence parameter set (SPS). Information commonly referenced by one or more SPSs is coded into a video parameter set (VPS). Furthermore, information commonly applied to one tile or tile group may be coded as syntax of a tile or tile group header.

[0019] The picture division unit 110 determines the size of a coding tree unit (CTU), and encodes information about the size of the CTU (CTU size) as an SPS or PPS syntax and transmits the information to a video decoding device.

[0020] The picture division unit 110 divides each picture constituting a video into a plurality of CTUs (coding tree units) having a predetermined size, and then recursively divides the CTUs using a tree structure. Leaf nodes in the tree structure become coding units (CUs), which are basic units of encoding.

[0021] The tree structure may be a quadtree (QT) in which an upper node (or parent node) is divided into four lower nodes (or child nodes) of the same size, a binary tree (BT) in which an upper node is divided into two lower nodes, a ternary tree (TT) in which an upper node is divided into three lower nodes in a ratio of 1:2:1, or a structure that combines two or more of the QT, BT, and TT structures. For example, a QTBT (QuadTree plus BinaryTree) structure or a QTBTTT (QuadTree plus BinaryTree TernaryTree) structure may be used. Here, BTTT is collectively called MTT (Multiple-Type Tree).

[0022] FIG. 2 shows a QTBTTTT splitting tree structure. As shown in FIG. 2, a CTU is first split into a QT structure. The quadtree split is repeated until the size of a splitting block reaches the minimum block size (MinQTSize) of a leaf node allowed in QT. A first flag (QT_split_flag) indicating whether each node of the QT structure is split into four nodes of a lower layer is coded by the entropy coding unit 155 and signaled in the video decoding device. If a leaf node of the QT is not larger than the maximum block size (MaxBTSize) of a root node allowed in BT, it is further split into one or more of a BT structure or a TT structure. In the BT structure and / or the TT structure, there may be multiple splitting directions. For example, there may be two directions, that is, a direction in which the block of the corresponding node is split horizontally and a direction in which the block is split vertically. As shown in FIG. 2, when MTT splitting is started, a second flag (MTT_split_flag) indicating whether or not a node has been split, and a flag indicating a split direction (vertical or horizontal) and / or a flag indicating a split type (Binary or Ternary) if a node has been split are coded by the entropy coding unit 155 and signaled to the video decoding device. As a countermeasure, a CU split flag (split_cu_flag) indicating whether or not a node is split may be coded before coding a first flag (QT_split_flag) indicating whether or not each node is split into four nodes in a lower layer. When the value of the CU split flag (split_cu_flag) indicates that no splitting is to be performed, the block of the node becomes a leaf node in the split tree structure, and becomes a CU (coding unit), which is a basic unit of coding. When the value of the CU split flag (split_cu_flag) indicates that splitting is to be performed, the video encoding device starts coding from the first flag as described above.

[0023] When QTBT is used as another example of a tree structure, there are two methods: a type in which the block of the corresponding node is split horizontally into two blocks of the same size (i.e., symmetric horizontal splitting) and a type in which the block of the corresponding node is split vertically (i.e., symmetric vertical splitting). A split flag (split_flag) indicating whether each node of the BT structure is split into a block of a lower layer and split type information indicating the type of split are coded by the entropy coding unit 155 and transmitted to the video decoding device. Meanwhile, there may also be a type in which the block of the corresponding node is split into two blocks that are asymmetric with respect to each other. The asymmetric type may include a form in which the block of the corresponding node is split into two rectangular blocks having a size ratio of 1:3, or a form in which the block of the corresponding node is split in a diagonal direction.

[0024] The CU has various sizes according to the QTBT or QTBTTT division from the CTU. Hereinafter, the block corresponding to the CU to be encoded or decoded (i.e., the leaf node of QTBTTTT) is referred to as the "current block". By adopting the QTBTTT division, the shape of the current block may be not only square but also rectangular.

[0025] The prediction unit 120 predicts the current block to generate a predicted block. The prediction unit 120 includes an intra prediction unit 122 and an inter prediction unit .

[0026] In general, each current block in a picture is predictively coded. In general, prediction of the current block is performed using intra prediction techniques (using data from the picture containing the current block) or inter prediction techniques (using data from a picture coded before the picture containing the current block). Inter prediction includes both unidirectional prediction and bidirectional prediction.

[0027] The intra prediction unit 122 predicts pixels in a current block using pixels (reference pixels) located around the current block in a current picture including the current block. There are a number of intra prediction modes depending on the prediction direction. For example, as shown in FIG. 3a, the intra prediction modes include two non-directional modes including a planar mode and a DC mode, and 65 directional modes. Different neighboring pixels and calculation formulas are defined to be used depending on each prediction mode. The following table lists the numbers and names of the intra prediction modes.

[0028] [Table 1]

[0029] For efficient directional prediction of the rectangular current block, directional modes (intra prediction modes 67 to 80 and -1 to -14) indicated by dotted arrows in FIG. 3b are further used. These are called "wide angle intra-prediction modes". In FIG. 3b, the arrows indicate the corresponding reference samples used for prediction, but do not indicate the prediction direction. The prediction direction is opposite to the direction indicated by the arrows. The wide angle intra-prediction modes are modes that predict a specific directional mode in the opposite direction without additional bit transmission when the current block is rectangular. In this case, among the wide angle intra-prediction modes, a part of the wide angle intra-prediction modes available for the current block is determined according to the ratio of the width and height of the rectangular current block. For example, wide-angle intra prediction modes with angles less than 45 degrees (intra prediction modes 67 to 80) are available when the current block is a rectangle whose height is smaller than its width, and wide-angle intra prediction modes with angles greater than -135 degrees (intra prediction modes -1 to -14) are available when the current block is a rectangle whose height is greater than its width.

[0030] The intra predictor 122 determines the intra prediction mode to use to encode the current block. In some examples, the intra predictor 122 may encode the current block using several intra prediction modes and select an appropriate intra prediction mode to use from the tested modes. For example, the intra predictor 122 may calculate a rate-distortion value using a rate-distortion analysis for several tested intra prediction modes and select the intra prediction mode with the best rate-distortion characteristics among the tested modes.

[0031] The intra prediction unit 122 selects one intra prediction mode from among a plurality of intra prediction modes, and predicts a current block using surrounding pixels (reference pixels) and an arithmetic equation determined by the selected intra prediction mode. Information about the selected intra prediction mode is encoded by the entropy encoding unit 155 and transmitted to the video decoding device.

[0032] In addition, the intra prediction unit 122 may generate a prediction block for the current block using matrix-based intra prediction (MIP), which will be described later. The intra prediction unit 122 may generate a prediction block for the current block using a boundary vector derived from samples reconstructed to the left of the current block and samples reconstructed to the upper side of the current block, a predefined matrix, and an offset vector.

[0033] The inter prediction unit 124 generates a prediction block for the current block through a motion compensation process. The inter prediction unit 124 searches for a block most similar to the current block in a reference picture that has been encoded and decoded prior to the current picture, and generates a prediction block for the current block using the searched block. Then, the inter prediction unit 124 generates a motion vector corresponding to a displacement between the current block in the current picture and the prediction block in the reference picture. In general, motion estimation is performed on a luma component, and a motion vector calculated based on the luma component is used for both the luma component and the chroma component. Motion information including information on the reference picture used to predict the current block and information on the motion vector is coded by the entropy coding unit 155 and transmitted to the video decoding device.

[0034] The subtractor 130 subtracts the predicted block generated by the intra predictor 122 or the inter predictor 124 from the current block to generate a residual block.

[0035] The transform unit 140 divides the residual block into one or more transform blocks, applies a transform to the one or more transform blocks, and transforms the residual values ​​of the transform blocks from the pixel domain to the frequency domain. In the frequency domain, the transformed blocks are called coefficient blocks that contain one or more transform coefficient values. A two-dimensional transform kernel is used for the transform, and one-dimensional transform kernels are used for the horizontal and vertical transforms, respectively. The transform kernels are based on the discrete cosine transform (DCT), the discrete sine transform (DST), etc.

[0036] The transform unit 140 transforms the residual signal in the residual block using the entire size of the residual block as a transform unit. The transform unit 140 also divides the residual block into two sub-blocks in the horizontal or vertical direction, and performs transform on only one of the two sub-blocks. Therefore, the size of the transform block may be different from the size of the residual block (and therefore the size of the prediction block). Non-zero residual sample values ​​are absent or very rare in the sub-blocks where transform is not performed. Residual samples of the sub-blocks where transform is not performed are not signaled and are all considered to be "0" by the image decoding apparatus. There are multiple partition types depending on the division direction and division ratio. The transform unit 140 provides information regarding the coding mode (or transform mode) of the residual block (e.g., information including information indicating whether the residual block has been transformed or the residual sub-block has been transformed, information indicating the partition type selected for dividing the residual block into sub-blocks, information identifying the sub-blocks on which transform is performed, etc.) to the entropy coding unit 155. The entropy coding unit 155 codes information about the coding mode (or transform mode) of the residual block.

[0037] The quantization unit 145 quantizes the transform coefficients output by the transform unit 140 and outputs the quantized transform coefficients to the entropy coding unit 155. For any block or frame, the quantization unit 145 may immediately quantize the associated residual block without any transform.

[0038] The sorting unit 150 sorts coefficient values ​​for the quantized residual values. The sorting unit 150 changes a two-dimensional coefficient array into a one-dimensional coefficient sequence through coefficient scanning. For example, the sorting unit 150 uses a zig-zag scan or a diagonal scan to scan from a DC coefficient to a high-frequency region coefficient and outputs a one-dimensional coefficient sequence. Depending on the size of the transform unit and the intra prediction mode, a vertical scan in which a two-dimensional coefficient array is scanned in the column direction and a horizontal scan in which a two-dimensional block shape coefficient is scanned in the row direction may be used instead of the zig-zag scan. That is, depending on the size of the transform unit and the intra prediction mode, a scan method to be used may be determined among the zig-zag scan, the diagonal scan, the vertical scan, and the horizontal scan.

[0039] The entropy coding unit 155 generates a bitstream by coding the sequence of one-dimensional quantized transform coefficients output from the reordering unit 150 using various coding schemes such as Context-based Adaptive Binary Arithmetic Code (CABAC) or Exponential Golomb.

[0040] In addition, the entropy coding unit 155 codes information such as a CTU size, a CU division flag, a QT division flag, an MTT division type, and an MTT division direction related to block division, so that the video decoding device can divide the block in the same manner as the video coding device. In addition, the entropy coding unit 155 codes information related to a prediction type indicating whether the current block is coded by intra prediction or inter prediction, and codes intra prediction information (i.e., information related to an intra prediction mode) or inter prediction information (information related to a reference picture and a motion vector) according to the prediction type.

[0041] The inverse quantization unit 160 generates transform coefficients by inverse quantizing the quantized transform coefficients output from the quantization unit 145. The inverse transform unit 165 transforms the transform coefficients output from the inverse quantization unit 160 from the frequency domain to the spatial domain to reconstruct the residual block.

[0042] The adder 170 reconstructs the current block by adding the reconstructed residual block and the prediction block generated by the prediction unit 120. The pixels in the reconstructed current block are used as reference pixels when intra-predicting the next block.

[0043] The filter unit 180 performs filtering on the restored pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. that occur due to block-based prediction and transform / quantization. The filter unit 180 includes a deblocking filter 182 and a Sample Adaptive Offset (SAO) filter 184.

[0044] The deblocking filter 180 filters the boundaries between the reconstructed blocks to remove blocking artifacts caused by block-based encoding / decoding, and the SAO filter 184 performs additional filtering on the deblocking filtered image. The SAO filter 184 is a filter used to compensate for differences between reconstructed pixels and original pixels caused by lossy coding.

[0045] The reconstructed blocks filtered through the deblocking filter 182 and the SAO filter 184 are stored in a memory 190. Once all blocks in a picture are reconstructed, the reconstructed picture is used as a reference picture for inter-predicting blocks in a picture to be coded later.

[0046] 4 is an exemplary functional block diagram of a video decoding device embodying the technology of the present invention. In the following, the video decoding device and its lower components will be described with reference to FIG.

[0047] The video decoding apparatus includes an entropy decoding unit 410 , a reordering unit 415 , an inverse quantization unit 420 , an inverse transform unit 430 , a prediction unit 440 , an adder 450 , a filter unit 460 , and a memory 470 .

[0048] Like the video encoding device of Fig. 1, each component of the video decoding device may be implemented in hardware or software, or in a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, and a microprocessor may execute the software functions corresponding to each component.

[0049] The entropy decoding unit 410 determines a current block to be decoded by decoding a bitstream generated by a video encoding device and extracting information related to block division, and extracts prediction information and information related to a residual signal required to restore the current block.

[0050] The entropy decoding unit 410 extracts information about the CTU size from a sequence parameter set (SPS) or a picture parameter set (PPS) to determine the size of the CTU, and divides the picture into CTUs of the determined size. The entropy decoding unit 410 then determines the CTU as the top layer of a tree structure, i.e., the root node, and divides the CTU using the tree structure by extracting division information about the CTU.

[0051] For example, when splitting a CTU using the QTBTTTT structure, first, the first flag (QT_split_flag) related to the split of QT is extracted, and each node is split into four nodes in the lower layer. Then, for a node that corresponds to a leaf node of QT, the second flag (MTT_split_flag) related to the split of MTT and the split direction (vertical / horizontal) and / or split type (binary / ternary) information are extracted, and the leaf node is split into an MTT structure. In this way, each node below the leaf node of QT is recursively split into a BT or TT structure.

[0052] As another example, when splitting a CTU using the QTBTTTT structure, a CU split flag (split_cu_flag) indicating whether or not to split a CU may be extracted first, and if the corresponding block is split, a first flag (QT_split_flag) may be extracted. During the splitting process, each node may undergo 0 or more repeated QT splits followed by 0 or more repeated MTT splits. For example, a CTU may undergo an immediate MTT split, or conversely, may undergo only multiple QT splits.

[0053] As another example, when splitting a CTU using the QTBT structure, the first flag (QT_split_flag) related to the split of the QT is extracted and each node is split into four nodes in the lower layer. Then, for nodes that correspond to leaf nodes of the QT, the split flag (split_flag) indicating whether to further split into BTs and the split direction information are extracted.

[0054] Meanwhile, when the entropy decoding unit 410 determines the current block to be decoded through the division of the tree structure, it extracts information on a prediction type indicating whether the current block is intra-predicted or inter-predicted. If the prediction type information indicates intra-prediction, the entropy decoding unit 410 extracts syntax elements on intra-prediction information (intra-prediction mode) of the current block. If the prediction type information indicates inter-prediction, the entropy decoding unit 410 extracts syntax elements on the inter-prediction information, i.e., information indicating a motion vector and a reference picture to which the motion vector refers.

[0055] Meanwhile, the entropy decoding unit 410 extracts information about the coding mode of the residual block (e.g., information about whether the residual block is coded or only sub-blocks of the residual block are coded, information indicating a partition type selected for dividing the residual block into sub-blocks, information identifying the coded residual sub-blocks, quantization parameters, etc.) from the bitstream. In addition, the entropy decoding unit 410 extracts information about the quantized transform coefficients of the current block as information about the residual signal.

[0056] The reordering unit 415 rearranges the sequence of one-dimensional quantized transform coefficients entropy decoded by the entropy decoding unit 410 back into a two-dimensional coefficient array (i.e., a block) in the reverse order of the coefficient scanning performed by the video coding apparatus.

[0057] The inverse quantization unit 420 inverse quantizes the quantized transform coefficients, and the inverse transform unit 430 generates a reconstructed residual block for the current block by inverse transforming the inverse quantized transform coefficients from the frequency domain to the spatial domain based on information about the coding mode of the residual block to reconstruct the residual signal.

[0058] When the information regarding the coding mode of the residual block indicates that the residual block of the current block has been coded in the video coding device, the inverse transform unit 430 performs an inverse transform on the dequantized transform coefficients using the size of the current block (and therefore the size of the residual block to be reconstructed) as a transform unit, and generates a reconstructed residual block for the current block.

[0059] In addition, when the information regarding the coding mode of the residual block indicates that only one sub-block of the residual block has been coded in the video coding device, the inverse transform unit 430 performs an inverse transform on the dequantized transform coefficients using the size of the transformed sub-block as a transform unit to restore the residual signal of the transformed sub-block, and creates a residual block restored to the current block by filling the residual signal of the non-transformed sub-block with a “0” value.

[0060] The prediction unit 440 includes an intra prediction unit 442 and an inter prediction unit 444. The intra prediction unit 442 is activated when the prediction type of the current block is intra prediction, and the inter prediction unit 444 is activated when the prediction type of the current block is inter prediction.

[0061] The intra prediction unit 442 determines an intra prediction mode of the current block from among a plurality of intra prediction modes based on a syntax element for the intra prediction mode extracted from the entropy decoding unit 410, and predicts the current block using reference pixels surrounding the current block according to the intra prediction mode. The intra prediction unit 442 may generate a prediction block for the current block using a matrix-based intra prediction (MIP) method described below. The intra prediction unit 422 may generate a prediction block for the current block using a boundary vector derived from samples reconstructed to the left of the current block and samples reconstructed to the upper side of the current block, and a predefined matrix and offset vector.

[0062] The inter prediction unit 444 determines a motion vector for the current block and a reference picture to which the motion vector refers using a syntax element for the intra prediction mode extracted from the entropy decoding unit 410, and predicts the current block using the motion vector and the reference picture.

[0063] The adder 450 adds the residual block output from the inverse transform unit and the predicted block output from the inter prediction unit or the intra prediction unit to reconstruct a current block. The reconstructed pixels in the current block are used as reference pixels when intra predicting a block to be decoded later.

[0064] The filter unit 460 includes a deblocking filter 462 and an SAO filter 464. The deblocking filter 462 deblocks and filters the boundaries between the reconstructed blocks to remove blocking artifacts caused by block-by-block decoding. The SAO filter 464 performs additional filtering on the reconstructed blocks after deblocking filtering to compensate for differences between reconstructed pixels and original pixels caused by lossy coding. The reconstructed blocks filtered through the deblocking filter 462 and the SAO filter 464 are stored in the memory 470. When all the blocks in one picture are reconstructed, the reconstructed picture is used as a reference picture for inter-predicting blocks in a picture to be coded later.

[0065] The techniques of the present invention generally relate to intra-prediction coding. The following description focuses primarily on the decoding technique, i.e., the operation of a video decoder, and the description of the encoding technique will be simplified since it is the opposite of the decoding technique described generically.

[0066] In the discussion of the next generation video coding standard (VVC), several new coding tools have been introduced that enable better coding performance compared to the High Efficiency Video Coding (HEV) standard. The Multiple Transform Selection (MTS) technique introduced in VVC relies on three trigonometric transforms including DCT-2, DST-7, and DCT-8, and the video coder selects the horizontal and vertical transforms that maximize the rate-distortion cost.

[0067] [Table 2]

[0068] For the blocks to which MTS is applied, one or more syntax elements are coded to signal which transform kernels to use in the horizontal and vertical directions.

[0069] As an example, three flags (MTS_CU_flag, MTS_Hor_flag, MTS_Ver_flag) are used with a predefined mapping table as shown in Table 3.

[0070] [Table 3]

[0071] First, one flag (MTS_CU_flag) is signaled to indicate whether DCT-2 is used in both horizontal and vertical directions. If DCT-2 is not applied in both directions, one flag (MTS_Hor_flag) is signaled to indicate which transform kernel, DST-7 or DCT-8, is applied in the horizontal direction, and one flag (MTS_Ver_flag) is signaled to indicate which transform kernel, DST-7 or DCT-8, is applied in the vertical direction. DST-7 and DCT-8 are used for block sizes up to 32x32, and DCT-2 is used for block sizes up to 64x64. Thus, MTS_CU_flag is signaled if the size (width and height) of the block is less than or equal to 32 and the block has at least one non-zero residual signal. If MTS_CU_flag is not present, MTS_CU_flag is inferred to be 0.

[0072] As a countermeasure, we use a syntax element (tu_mts_idx) that represents an MTS kernel index, along with a predefined mapping table as shown in Table 4. tu_mts_idx is represented by an MTS kernel index, and each index value selects a kernel for the horizontal and vertical directions. In Table 4, trTypeHor and trTypeVer indicate the type of kernel for the horizontal and vertical directions. 0 means DCT-2, 1 means DST-7, and 2 means DCT-8. For example, when tu_mts_idx=2, DCT-8 is used for the horizontal direction and DST-7 is used for the vertical direction.

[0073] [Table 4]

[0074] tu_mts_idx is signaled only when sps_explicit_mts_enabled_flag is 1 in the SPS. Since it explicitly signals the MTS index, this is called ExplicitMTS. On the other hand, tu_mts_idx is not explicitly indicated and may be derived implicitly from the encoder and decoder, which is called ImplicitMTS. ImplicitMTS is performed when sps_explicit_mts_enabled_flag is 0 in the SPS. When ImplicitMTS is used, the selection of the transform kernel is performed based on information such as the intra prediction mode, inter prediction mode, block size, etc. used for the block. For example, if the width of the block is 4 or more and 16 or less, DST-7 is used as the horizontal kernel, and DCT-2 is used otherwise. Also, if the height of the block is 4 or more and 16 or less, DST-7 is used as the vertical kernel, and DCT-2 is used otherwise. That is, when the block size is small, the directionality of the prediction residual signal is large, so the DST-7 kernel is used, whereas when the block size is relatively large, the characteristics of the residual signal having a more uniform characteristic are strong, so the DCT-2 kernel is used.

[0075] Table 5 shows some of the transform unit syntax specified in VVC Draft 5. In the syntax below, grey highlighting of elements is used to aid understanding.

[0076] [Table 5]

[0077] Transform blocks are coded without transform operation, which is indicated by the syntax element transform_skip_flag signaled for transform blocks with at least one non-zero coefficient. High-level control of coding mode of transform blocks is achieved by High-Level Syntax (HLS) elements transform_skip_enabled_flag and log2_transform_skip_max_size_minus2, such as SPS (Sequential Parameter Set). If transform_skip_enabled_flag is 1, then transform_skip_flag is coded for transform blocks whose height and width are less than or equal to "1 << (log2_transform_skip_max_size_minus2+2)". If transform_skip_flag is 1, then the associated transform block is coded in transform skip mode, otherwise apply MTS-based transform to the associated transform block.

[0078] Meanwhile, Matrix-based Intra Prediction (MIP) is a new intra prediction technique introduced in VTM5.0. The original idea is to use a neural network-based intra prediction technique, i.e., a multi-layer neural network to predict the current PU pixel value based on the neighboring reconstructed pixels. However, due to the high complexity of the prediction method using neural networks, we introduced an intra prediction technique based on affine linear transformation with a pre-trained matrix.

[0079] To predict a rectangular block PU with width W and height H, MIP takes H samples reconstructed on the left side of the block and W samples reconstructed on the top side of the block as input, and finally the predicted pixel is obtained by averaging, matrix-vector multiplication, linear interpolation, etc.

[0080] The block sizes to which MIP is applied are classified into three categories as follows:

[0081] TIFF0007627013000006.tif21146

[0082] The number of MIP modes (numModes), boundary size (boundarySize), and prediction block size (predW, predH, predC) are defined as follows according to idx(W,H): In the table below, MipSizeId=idx(W,H).

[0083] [Table 6]

[0084] FIG. 5 is a schematic diagram illustrating the main processes of the MIP technology used in the present technique.

[0085] (1) Averaging The main purpose of this step is to normalize the reference samples. Depending on the block size and shape (i.e. depending on MipSizeId), 4 or 8 samples are obtained. If both width and height of the current block are 4 (W=H=4), then a total of 4 samples are obtained, including 2 from the left and 2 from the top (boundarySize=2). In the remaining cases, a total of 8 samples are obtained, including 4 from the left and 4 from the top (boundarySize=4).

[0086] TIFF0007627013000008.tif26170

[0087] TIFF0007627013000009.tif9149

[0088] In the above formula, bDwn is the downsampling scale value (nTbs / boundarySize), and refS refers to the original reference sample. TIFF0007627013000010.tif9165

[0089] The downsampled reference samples are stitched into vectors of length 4 or 8. Reduced boundary vectors input to the vector matrix multiplication operation TIFF0007627013000011.tif7128 is defined as follows. For example, if W=H=4 and MIP mode is less than 18, TIFF0007627013000012.tif8150W=H=4 and MIP mode is 18 or higher, TIFF0007627013000013.tif10128In the formula below, "mode" means MIP mode.

[0090] TIFF0007627013000014.tif45146

[0091] (2) Matrix-Vector Multiplication TIFF0007627013000015.tif17170

[0092] TIFF0007627013000016.tif8146

[0093] TIFF0007627013000017.tif11170 Matrix A is W red *H red The offset vector b has only one row, and four columns when W=H=4, otherwise eight columns. red *H red A vector of sizes.

[0094] TIFF0007627013000018.tif27146

[0095] A set of matrices A and offset vectors b to use for the blocks S 0 , S 1 , S2 is predefined for each block size category. The index (0, 1, 2) of the set S is selected by the above MipSizeId (i.e., idx(W,H)), and the matrix A and offset vector b are stored in the set S. 0 , S 1 , S 2 The current block is extracted by a MIP mode applied to one of the sets.

[0096] Set S 0 is a set of 18 matrices A, each with 16 rows and 4 columns. 0 and 18 16-dimensional offset vectors b 0 The set S is used for 4×4 blocks. 1 is a set of 10 matrices A, each with 16 rows and 8 columns. 1 and ten 16-dimensional offset vectors b 1 and used for blocks of size 4×8, 8×4, and 8×8. Finally, the set S 2 is a set of six matrices A, each with 64 rows and 8 columns. 2 and six 64-dimensional offset vectors b 2 and is used for all remaining block shapes.

[0097] (3) Pixel Interpolation Interpolation is an upsampling process. As mentioned above, TIFF0007627013000019.tif8128 is the downsampled prediction signal of the original block, where the downsampled prediction block with size predW and predH is defined as follows:

[0098] predred[x][y],with x=0..predW 1,y=0..predH‐1

[0099] A prediction block of the original block size (nTbW, nTbH) generated by linearly interpolating the prediction signal at the remaining positions along each direction is defined as follows.

[0100] predSamples[x][y],with x=0..nTbW‐1,y=0..nTbH‐1

[0101] Depending on the horizontal and vertical upsampling scale factors upHor (=nTbW / predW) and upVer (=nTbH / predH), Some or all of predSamples are filled from TIFF0007627013000020.tif6128.

[0102] predSamples[(x+1)* upHor‐1][(y+1)*upVer‐1]=predred [x][y]

[0103] If upHor=1, If all horizontal positions of predSamples are filled from TIFF0007627013000021.tif6128 and upVer=1, All vertical positions of predSamples are filled from TIFF0007627013000022.tif6128.

[0104] After this, bi-linear interpolation is used to fill the remaining free samples in predSamples. Horizontal and vertical interpolation are the upsampling processes. To interpolate the left and top samples in predSamples, the downsampled samples are used. TIFF0007627013000023.tif7128 is assigned the predSamples[x][-1] value, and the original reference sample on the left TIFF0007627013000024.tif7128 is assigned to the predSamples[-1][y] value.

[0105] (4) MIP intra prediction mode signaling A flag indicating whether to apply a matrix-based intra prediction mode (i.e., MIP mode) to each coding unit CU that performs intra prediction coding is transmitted. In the VVC5 draft, for signaling the MIP mode, an MPM list is used similarly to the existing intra prediction mode (hereinafter, "regular intra prediction mode"), rather than the matrix-based intra prediction. For example, intra_mip_mpm_flag, intra_mip_mpm_idx, and intra_mip_mpm_remainder are used for signaling the MIP mode. intra_mip_mpm_idx is coded into a truncated binary code, and intra_mip_mpm_remainder is coded into a fixed length code.

[0106] Depending on the size of the coding block CU, up to 35 MIP modes are supported. For example, for CUs with max(W,H)<=8 and W*H<32, 35 modes are available. And for CUs with max(W,H)=8 and max(W,H)>8, 19 and 11 prediction modes are used, respectively. Furthermore, a pair of modes (two modes) share matrices and offset vectors to reduce memory requirements. The specific sharing mode is calculated as follows: For example, for a 4×4 coding block, mode 19 uses the transposed matrix of the matrix assigned to mode 2.

[0107] TIFF0007627013000025.tif33136

[0108] When a block to which a normal intra prediction mode is applied instead of matrix-based intra prediction (MIP) (hereinafter referred to as a "normal block") has a block to which MIP is applied, a mapping table defined between the MIP mode and the normal mode is used for MPM induction of the normal block. The mapping table is used to derive a normal mode with similar characteristics from the MIP mode of the adjacent block to which MIP is applied. The normal mode derived in this manner is used for MPM induction of the normal block. Similarly, when a luminance block at the same position used in chroma DM induction applies MIP, a normal mode is derived using the mapping table and used for chroma DM induction. The following equation expresses mapping between normal modes and MIP modes using the mapping table.

[0109] TIFF0007627013000026.tif18128

[0110] As explained above, in VVC Draft 5, when predicting a block using MIP-based prediction, the reconstructed neighboring samples on the left side of the block are always used. TIFF0007627013000027.tif7128 and the reconstructed surrounding samples in the upper part of the block TIFF0007627013000028.tif7128 is used. Such an approach may hinder prediction performance in the case of blocks whose texture has directional characteristics. Also, in VVC Draft 5, for the interpolation of the upper samples in predSamples, a downsampled sample set is used, as shown in Figure 5. Using TIFF0007627013000029.tif7128 as the predSamples[x][-1] value makes the interpolation process unnecessarily complicated. Furthermore, VVC Draft 5 uses MPM lists for signaling MIP mode and normal mode, respectively, which requires mapping between MIP mode and normal mode, and many other checks and conditions can make the implementation very complicated.

[0111] In view of this, the present invention proposes some improved techniques for MPM coding. According to one aspect of the present invention, an approach is proposed that reduces the complexity of the MIP mode implementation and takes into account the directional characteristics of the block texture.

[0112] FIG. 6 is a flow chart illustrating a general process for MIP-based block prediction according to one embodiment of the present invention.

[0113] 1. MIP mode decoding step (S610) The video decoder decodes, for a coding unit (CU) coded in an intra-prediction mode, a flag indicating whether the intra-prediction type is matrix-based intra-prediction (MIP), and if MIP is applied, a syntax element indicating which of multiple available MIP modes to use for the current coding unit.

[0114] Unlike existing intra prediction modes (regular intra prediction modes), MIP modes may not use an MPM list for signaling, for example, using one syntax element (e.g., intra_mip_mode) that may be coded with a truncated binary code to indicate the MIP mode to be used for the current coding unit among multiple MIP modes.

[0115] A portion of an example transform unit syntax proposed based on the VVC5 draft is provided below: In the syntax below, grey highlighting of elements is used to aid in understanding.

[0116] [Table 7]

[0117] If intra_mip_flag[x0][y0] is 1, it indicates that the intra prediction type of the current block is matrix-based intra prediction (MIP). If intra_mip_flag[x0][y0] is 0, it indicates that the intra prediction type of the current block is regular intra prediction instead of matrix-based intra prediction. If intra_mip_flag[x0][y0] is not present, it is inferred to be equal to 0. intra_mip_mode[x0][y0] specifies the MIP mode to use for the current block in matrix-based intra prediction (MIP).

[0118] The neighboring samples of the current coding block used to generate the prediction signal are determined differently depending on the MIP mode. In one example, for a coding block of 4×4 size, for example, 35 MIP modes are available, and the neighboring samples used to generate the prediction signal are determined according to the range of modes as follows:

[0119] TIFF0007627013000031.tif21133

[0120] If the pixels of the current coding block have horizontal characteristics, it is advantageous to use the samples of the left neighbor to generate the prediction signal, and similarly, if the pixels of the current coding block have vertical characteristics, it is advantageous to use the samples of the upper neighbor to generate the prediction signal.

[0121] In order to select a MIP mode suitable for the directional characteristics of a block, the video encoder applies a normal mode of intra prediction mode and calculates a distortion value of the block according to each mode. If the distortion value of a horizontal prediction mode (e.g., INTRA_ANGULAR_14 to INTRA_ANGULAR_22) is smaller than that of other modes, the horizontal direction is determined as the main direction of the current coding block. Conversely, if the distortion value of a vertical prediction mode (e.g., INTRA_ANGULAR_46 to INTRA_ANGULAR_54) is smaller than that of other modes, the vertical direction is determined as the main direction of the current coding block.

[0122] If the horizontal direction is the main direction, the encoder assigns to the current coding block one of the MIP modes that only use samples from the left neighbor, and conversely, if the vertical direction is the main direction, the encoder assigns to the current coding block one of the MIP modes that only use samples from the upper neighbor.

[0123] In the following, for convenience of explanation, the case of using the left neighbor sample and the case of using the upper neighbor sample are distinguished, and the method of determining the boundary vector and the predicted sample therefrom is described. The process of using both the left neighbor sample and the upper neighbor sample is substantially the same as that described with reference to FIG.

[0124] 2-1. Creating a boundary vector - Using the sample to the left (S620) 7a-7c are conceptual diagrams illustrating an exemplary method for constructing boundary vectors for input into a matrix-vector multiplication operation using samples to the left.

[0125] As an example, as illustrated in FIG. 7a, the size (height) of the current coding block and the boundary vector If TIFF0007627013000032.tif7128 is the same size, the sample set to the left Same size border vector using TIFF0007627013000033.tif8128 Fill in TIFF0007627013000034.tif8128, e.g. by including each of the left-neighboring samples in an entry in the boundary vector.

[0126] As another example, as shown in Fig. 7b, we calculate the boundary vector by using the two adjacent columns on the left to calculate the average between the two pixels in each row. Fill TIFF0007627013000035.tif8128.

[0127] As another example, as shown in Fig. 7c, Downsampled sample set taken from TIFF0007627013000036.tif8128 TIFF0007627013000037.tif19128

[0128] TIFF0007627013000038.tif13170

[0129] TIFF0007627013000039.tif13145

[0130] 2-2. Generation of boundary vectors - Using the upper neighboring sample (S620) 8a-8c are conceptual diagrams illustrating an exemplary method for constructing boundary vectors input to a matrix-vector multiplication operation using upper neighbor samples.

[0131] As an example, as illustrated in FIG. 8a, the size (width) of the current coding block and the boundary vector If TIFF0007627013000040.tif7128 is the same size, the upper neighboring sample set Same size border vector using TIFF0007627013000041.tif7128 Fill in TIFF0007627013000042.tif7128, e.g. by including each of the upper neighboring samples in an entry in the boundary vector.

[0132] As another example, as shown in Fig. 8b, we use the two adjacent rows on the top to calculate the average between two pixels in each column to find the boundary vector Fill TIFF0007627013000043.tif7128.

[0133] TIFF0007627013000044.tif21170

[0134] TIFF0007627013000045.tif18170

[0135] TIFF0007627013000046.tif15152

[0136] 3. Matrix-vector multiplication and offset addition (S630) The video decoder uses the boundary vector A matrix-vector multiplication operation is performed on TIFF0007627013000047.tif7130, and an offset vector is added. The matrix A and offset vector b are determined according to the decoded MIP mode k. TIFF0007627013000048.tif12170

[0137] TIFF0007627013000049.tif7128

[0138] TIFF0007627013000050.tif7128

[0139] If the boundary vector to be input to the matrix-vector multiplication operation is constructed using the sample to the left, W red and H red is determined by the size of the coding block as follows:

[0140] TIFF0007627013000051.tif25128

[0141] When the boundary vector to be input to the matrix-vector multiplication operation is constructed using the upper neighboring sample, W redand H red is determined according to the size of the coding block as follows:

[0142] TIFF0007627013000052.tif27128

[0143] Generated If the size of TIFF0007627013000053.tif7128 is smaller than the number of samples in the coding block, interpolation is required.

[0144] 4. Linear Interpolation (S640) FIG. 9 shows the left neighbor sample, the boundary vector, and the predicted sample predicted from it. In FIG. 9, the pixels indicated by triangles in the prediction block predSamples[x][y] for the current coding block are generated. This value is assigned from TIFF0007627013000054.tif7128.

[0145] Referring to FIG. 9, various interpolation methods are used to determine the empty pixels in the prediction block predSamples[x][y].

[0146] TIFF0007627013000055.tif35170

[0147] FIG. 10 shows the upper neighbor samples, boundary vectors, and the predicted samples derived therefrom. In FIG. 10, the pixels indicated by triangles in the predicted block predSamples[x][y] for the current coding block are the generated samples. This value is assigned from TIFF0007627013000056.tif8128.

[0148] Referring to FIG. 10, various interpolation methods are used to determine the empty pixels in the prediction block predSamples[x][y]. TIFF0007627013000057.tif31170

[0149] Unlike the examples shown in FIG. 9 and FIG. 10, the upper original reference samples are used to determine the empty pixels in the prediction block predSamples[x][y]. TIFF0007627013000058.tif7128 is assigned the predSamples[x][-1] value, and the original reference sample on the left TIFF0007627013000059.tif7128 may be assigned to the predSamples[-1][y] value.

[0150] The above embodiment introduces a method of selectively using neighboring samples of a current block according to a MIP mode so that the direction of a block is taken into consideration when generating a predicted block based on MIP.

[0151] According to another aspect of the present invention, instead of changing the set of adjacent samples used to generate the boundary vector in the MIP mode, a method of modifying the values ​​of the matrix A and the offset b applied to the boundary vector may be used.

[0152] The matrix A and offset b applied to the matrix-vector multiplication and offset addition operations on the boundary vectors are divided into two parts as follows.

[0153] TIFF0007627013000060.tif30145

[0154] If A2 and b2 have values ​​close to zero, the predicted signal TIFF0007627013000061.tif7128 is mainly the pixel to the left TIFF0007627013000062.tif8128, which is useful for blocks whose texture has a horizontal orientation. If A1 and b1 have values ​​close to 0, the predicted signal TIFF0007627013000063.tif8128 is mainly the upper adjacent pixel Inspired by TIFF0007627013000064.tif8128, this is useful for blocks where the block texture has a vertical direction.

[0155] The video encoder and decoder modify and use the values ​​of matrix A and offset b applied to the boundary vector according to the MIP mode of the current block. For example, for a coding unit of 4×4 size, for example, 35 MIP modes are available, and the values ​​of matrix A and offset b applied to the boundary vector are modified and used according to the range of the modes.

[0156] TIFF0007627013000065.tif17128

[0157] As mentioned above, VVC draft 5 supports up to 35 MIP modes depending on the size and shape of the coding block CU. For example, 35 modes are available for CUs with max(W,H)<=8&&W*H<32, and 19 and 11 prediction modes are used for CUs with max(W,H)=8 and max(W,H)>8, respectively. Furthermore, a pair of modes (two modes) share matrices and offset vectors to reduce memory requirements. For example, for a 4x4 coding block, mode 19 uses the transposed matrix of the matrix assigned to mode 2. TIFF0007627013000066.tif11170

[0158] An improved approach is used that achieves substantially the same level of coding efficiency while reducing complexity. According to another aspect of the present invention, instead of the conventional method in which one mode uses a transposed matrix used by the other mode, TIFF0007627013000067.tif21170 According to this method, the number of available MIP modes can be reduced to half the level compared to existing methods, and coding complexity for boundary vector generation and vector matrix multiplication operations can be reduced.

[0159] On the other hand, as explained in relation to the syntax structure of Table 5, for transform units that meet certain conditions, the transform skip-related syntax element transform_skip_flag is signaled every time, and if transform_skip_flag is not true, the multiple transform selection-related syntax element tu_mts_idx needs to be transmitted depending on the flags in the SPS (sps_explicit_mts_inter_enabled_flag, sps_explicit_mts_intra_enabled_flag), which puts a strain on the number of bits transmitted in the bitstream.

[0160] However, such multiple transform selection and transform skipping are closely related to the characteristics of the residual signal, and therefore, it may not be efficient to always signal the transform-related syntax elements regardless of the prediction mode used for the block. The inventors note that, considering the process of the matrix-based intra prediction (MIP) technique, which includes an average operation and an interpolation operation, the residual signal of a block to which MIP is applied has a characteristic in which low-frequency components predominate in the transform domain, and MIP is likely not applied to blocks with little correlation between pixels, such as blocks containing complex motion and texture. Therefore, it is likely that transform skipping is not applied to blocks using MIP.

[0161] From this perspective, the approach of always signaling the multiple transform selection related syntax element tu_mts_idx and the transform skip related syntax element transform_skip_flag regardless of whether MIP is applied to the block or not may not be very efficient.

[0162] According to one aspect of the present invention, depending on whether MIP is applied, signaling of transform-related syntax elements can be omitted, and the omitted syntax elements are implicitly inferred by the video decoder.

[0163] In one embodiment, the Multiple Transform Selection (MTS) technique is implicitly applied depending on the MIP flag. As shown in the table below, transform_skip_flag and tu_mts_idx are coded only if intra_mip_flag is not true. If intra_mip_flag is true, transform_skip_flag is inferred to be 0, tu_mts_idx is also inferred to be 0, and implicit MTS is inferred to be applied. Thus, the horizontal and vertical transform kernels are determined by the block size. In the syntax below, grey highlighting of elements is used to indicate potential changes in the syntax or to aid understanding.

[0164] [Table 8]

[0165] In another embodiment, the Multiple Transform Selection (MTS) technique is explicitly applied via the MIP flags. As per the table below, transform_skip_flag and tu_mts_idx are coded only if intra_mip_flag is not true. If intra_mip_flag is true, transform_skip_flag is inferred as 0, tu_mts_idx is explicitly signaled, and explicit MTS is applied. In the syntax below, grey highlighting of elements is used to indicate potential changes in the syntax or to aid understanding.

[0166] [Table 9]

[0167] In other possible implementations, MIP may not be explicitly applied to blocks with low correlation between pixels, such as small-sized blocks with complex motion or texture. Transform Skip (TS) may not be applied to 4x4 CUs with MIP applied. Also, MIP may not be applied to 4x4 CUs with Transform Skip (TS) applied. Furthermore, MIP may not be applied to 4x4 CUs with TS applied.

[0168] In conventional approaches, intra-predictive coding using Most Probable Modes (MPMs) is used. For example, in HEVC, a list of three MPMs is composed of intra-prediction modes of the left and top blocks. The drawback of such an approach is that more modes (intra modes, not MPMs) belong to non-MPMs, which should be coded with more bits. Several methods have been proposed to extend the number of MPMs to more than three entries (e.g., six MPM modes). However, this makes the implementation more complex, since more checks and conditions are required to construct such an MPM list with more entries.

[0169] As mentioned above, in VVC Draft 5, MPM lists are used for signaling MIP mode and normal mode, respectively, which makes implementation very complicated due to many checks and conditions, such as requiring mapping between MIP mode and normal mode to configure the MIP list.

[0170] In order to keep the complexity of the MPM list configuration low, an MPM list including six MPM candidates is configured using the intra prediction modes of the left and upper blocks adjacent to the current block. The MPM candidates are configured from a default intra prediction mode (e.g., PLANAR mode), the intra prediction mode of the surrounding blocks, and an intra prediction mode derived from the intra prediction mode of the surrounding blocks. If the intra prediction mode of the surrounding blocks is not used (e.g., if the surrounding blocks are inter-predicted or located in other slices or other tiles), the intra prediction mode of the intra prediction mode of the surrounding blocks is set to planar.

[0171] There are four main cases depending on the type of intra prediction mode for the left block mode (Left) and the upper block mode (Above), and when 'Left' and 'Above' are different and both modes are directional modes, the MPM list is created by further distinguishing between 4 cases depending on the difference between 'Left' and 'Above'. In the table below, 'Max' refers to the larger mode between 'Left' and 'Above', and 'MIN' refers to the smaller mode between 'Left' and 'Above'.

[0172] [Table 10]

[0173] Due to the characteristics of the matrix-based intra prediction (MIP) technique, which includes averaging and interpolation operations, the residual signal of a block to which MIP is applied is dominated by low-frequency components in the transform domain. It should be noted that such residual characteristics are similar to the residual signal of a block to which planar mode or DC mode is applied. Therefore, it is useful to take such similarity of residual signals into account when deriving an MPM list for a block to be coded in a regular intra prediction mode.

[0174] According to one aspect of the present invention, when deriving an MPM list for a block coded in a normal intra prediction mode (i.e., a normal block), if a surrounding block is coded in an MIP mode, the intra prediction mode of the surrounding block is considered to be a planar mode (or a DC mode). For example, if an MIP mode is applied to a surrounding block, a planar mode (or a DC mode) is added to the MPM list instead of the MIP mode of the surrounding block.

[0175] Similarly, when chroma DM (direct mode) is induced, if MIP is applied to the co-located luma block, the intra prediction mode of the luma block is considered to be the planar mode (or DC mode) instead of using a mapping table between the MIP mode and the normal mode. Thus, the encoder parses a syntax element specifying the intra prediction mode for a chroma block, and if the syntax element instructs that the intra prediction mode of the chroma block should be the intra prediction mode of the co-located luma block as is, and if MIP is applied to the co-located luma block, the intra prediction mode of the luma block is considered to be the planar mode (or DC mode). In other words, if MIP is applied to the co-located luma block in chroma DM (direct mode), the intra prediction mode of the chroma block is determined to be the planar mode (or DC mode).

[0176] The video encoder signals a 1-bit flag (e.g., mpm_flag) indicating whether the intra prediction mode of the current block corresponds to the MPM. Typically, if the intra prediction mode of the current block corresponds to the MPM, an MPM index indicating one of the six MPMs (i.e., planar mode) is additionally signaled. Note that in the above table, the planar mode is always included in the MPM list. Therefore, it is efficient for the encoder to explicitly signal (e.g., using a 1-bit flag) whether the intra prediction mode of the current block is planar mode if the intra prediction mode of the current block is planar mode, and to additionally signal an MPM index indicating one of the remaining five MPMs if the intra prediction mode of the current block is the same as one of the remaining five MPMs. If the intra prediction mode of the current block does not correspond to an MPM, a syntax element indicating one of the remaining 61 non-MPMs other than the six MPMs is encoded using a truncated binary code.

[0177] FIG. 11 is a flow chart illustrating a method for decoding video data according to one embodiment of the present invention.

[0178] The video decoder decodes (S1110) a syntax element indicating an intra-prediction type for a current block of video data from the bitstream. The intra-prediction types include matrix based intra prediction (MIP) and regular intra prediction. The syntax element is a truncated binary code that specifies one of a number of matrix based intra-prediction modes allowed for the size and shape of the current block.

[0179] The video decoder selectively performs matrix-based intra prediction or regular intra prediction based on an intra prediction type of the current block to generate a prediction block for the current block.

[0180] The video decoder performs the following steps (S1120 to S1140) as part of performing regular intra prediction to generate a predicted block for a current block. The video decoder derives Most Probable Mode (MPM) candidates based on regular intra prediction modes of neighboring blocks of the current block to construct an MPM list for the current block (S1120), and derives a regular intra prediction mode for the current block based on the MPM list (S1130). In deriving MPM candidates based on regular intra prediction modes of the neighboring blocks, the video decoder sets (deems) the regular intra prediction mode of the neighboring blocks to planar mode if the intra prediction type of the neighboring blocks is matrix-based intra prediction. The decoder generates a predicted block for the current block using the regular intra prediction mode of the current block (S1140).

[0181] The video decoder performs the following steps (S1121 to S1151) as part of performing matrix-based intra prediction to generate a predicted block for a current block. The video decoder decodes a syntax element indicating a matrix-based intra prediction mode for a current block from a bitstream to determine a matrix-based intra prediction mode for the current block (S1121). The video decoder derives a boundary vector using neighboring samples of the current block based on the size and shape of the current block (S1131), and generates a predicted sample for the current block based on a matrix-vector multiplication between a matrix predefined for the matrix-based intra prediction mode for the current block and the boundary vector (S1141). The video decoder performs linear interpolation, clipping, etc. based on the predicted sample to derive a predicted block for the current block (S1151).

[0182] The video decoder derives the boundary vector from one or two selected sets of left surrounding samples and a set of upper surrounding samples adjacent to the current block, depending on the matrix-based intra prediction mode for the current block.

[0183] The video decoder decodes from the bitstream a syntax element indicating a splicing order between a first entry of a boundary vector derived from a left surrounding sample adjacent to the current block and a second entry of a boundary vector derived from a top surrounding sample adjacent to the current block, and the video decoder splices the first entry and the second entry according to the indicated splicing order to generate a boundary vector.

[0184] The entries of the boundary vector are filled with downsampled values ​​from the left neighboring samples adjacent to the current block or with the left neighboring samples adjacent to the current block, depending on the size and shape of the current block.

[0185] The video decoder assigns the prediction samples to positions in the prediction block to derive a prediction block for the current block based on the prediction samples, and performs linear interpolation on the prediction samples, left surrounding samples adjacent to the current block, and upper surrounding samples adjacent to the current block to generate prediction sample values ​​for positions in the prediction block to which no prediction samples are assigned.

[0186] The exemplary embodiments described above may be embodied in many other ways. The functions or methods described in one or more examples may be embodied in hardware, software, firmware, or any combination thereof. The functional components described herein are labeled as "Units" to specifically emphasize their implementation independence.

[0187] Meanwhile, various functions or methods described herein may be embodied as instructions stored in a non-transitory recording medium that can be read and executed by one or more processors. Non-transitory recording media include, for example, any type of recording device that stores data in a form readable by a computer system. For example, non-transitory recording media include storage media such as erasable programmable read only memory (EPROM), flash drives, optical drives, magnetic hard drives, and solid-state drives (SSDs).

[0188] The above description is merely illustrative of some embodiments according to the technical idea of ​​the present invention, and various modifications and variations are possible within the scope of the essential characteristics of the present invention, if one has ordinary knowledge in the technical field to which the present invention belongs. Therefore, the illustrated embodiments are for the purpose of explanation, not for the purpose of limiting the technical idea of ​​the present invention, and the scope of the technical idea of ​​the present invention is not limited by such embodiments.

[0189] CROSS-REFERENCE TO RELATED APPLICATION This patent application claims priority to patent application number 10-2019-0075163 filed in Korea on June 24, 2019, patent application number 10-2019-0080749 filed in Korea on July 4, 2019, and patent application number 10-2020-0077014 filed in Korea on June 24, 2020, the entireties of which are incorporated herein by reference. [Explanation of symbols]

[0190] 110 Picture division section 120, 440 Prediction Department 122, 442 Intra prediction section 124, 444 Inter Prediction Section 130 Subtractor 140 Conversion unit 145 Quantization section 150, 415 Sorting section 155 Entropy coding unit 160, 420 Inverse quantization section 165, 430 Reverse conversion unit 170, 450 adder 180, 460 Filter section 182, 462 Deblocking Filter 184, 464 SAO filter 190, 470 Memory 410 Entropy Decoding Unit

Claims

1. 1. A method for decoding video data, comprising the steps of: decoding a syntax element indicating an intra prediction type for a current block of video data from a bitstream, the intra prediction type including matrix based intra prediction (MIP) and regular intra prediction; selectively performing matrix-based intra prediction or regular intra prediction based on an intra prediction type of the current block indicated by the syntax element to generate a predicted block for the current block; decoding residual data from the bitstream identifying a residual block of the current block; reconstructing the current block based on the predicted block and the residual block; Including, The step of performing regular intra prediction to generate a predicted block for the current block comprises: deriving Most Probable Mode (MPM) candidates based on normal intra-prediction modes of neighboring blocks of the current block to construct an MPM list for the current block; deriving a regular intra prediction mode for the current block based on the MPM list; The step of performing matrix-based intra prediction to generate a predicted block for the current block includes: decoding, from the bitstream, a syntax element indicating a matrix-based intra-prediction mode for the current block; deriving a boundary vector by downsampling surrounding samples adjacent to the current block based on a width and height of the current block; generating a prediction sample for the current block based on a matrix-vector multiplication between a predefined matrix for the matrix-based intra prediction mode and the boundary vector; deriving the prediction block for the current block based on the prediction samples; deriving a boundary vector by downsampling surrounding samples adjacent to the current block, decoding from the bitstream a syntax element indicating a splicing order between a first entry of the boundary vector downsampled from a left-side neighboring sample adjacent to the current block and a second entry of the boundary vector downsampled from a top-side neighboring sample adjacent to the current block; splicing the first entry and the second entry according to the splicing order to generate the boundary vector; The method is characterized in that the current block is a luma block consisting of luma components, the matrix-based intra prediction is performed on the luma block, and if the intra prediction mode of the chroma block corresponding to the luma block uses the intra prediction mode of the luma block as is, the normal intra prediction mode for the luma block is set to planar mode.

2. The syntax element indicating a matrix-based intra prediction mode for the current block may include:

2. The method of claim 1, wherein the matrix-based intra-prediction mode is a truncated binary code that specifies one of a plurality of matrix-based intra-prediction modes that are allowed for a width and height of the current block.

3. deriving a boundary vector by downsampling neighboring samples of the current block, 3. The method of claim 2, further comprising the step of deriving the boundary vector from one or two selected sets of left and upper surrounding samples adjacent to the current block depending on a matrix-based intra prediction mode for the current block.

4. deriving a prediction block for the current block based on the prediction samples, assigning the prediction samples to positions within the prediction block; and generating predicted sample values ​​for locations in the predictive block not assigned to the predicted samples using linear interpolation on the predicted samples, left neighboring samples adjacent to the current block, and upper neighboring samples adjacent to the current block.

5. 1. A method for encoding video data, comprising the steps of: encoding a syntax element into the bitstream indicating an intra-prediction type for a current block of video data, the intra-prediction type including matrix-based intra-prediction and regular intra-prediction; selectively performing matrix-based intra prediction or regular intra prediction based on an intra prediction type of the current block to generate a predicted block for the current block; generating a residual block of the current block using the predicted block; encoding residual data specifying the residual block into the bitstream; Including, The step of performing regular intra prediction to generate a predicted block for the current block comprises: determining a regular intra prediction mode of the current block; deriving Most Probable Mode (MPM) candidates based on normal intra prediction modes of neighboring blocks of the current block to construct an MPM list for the current block; encoding at least one syntax element into the bitstream, the syntax element indicating a normal intra-prediction mode of the current block based on the MPM list; The step of performing matrix-based intra prediction to generate a predicted block for the current block includes: encoding a syntax element indicating a matrix-based intra-prediction mode for the current block into the bitstream; deriving a boundary vector by downsampling surrounding samples adjacent to the current block based on a width and height of the current block; generating a prediction sample for the current block based on a matrix-vector multiplication between a predefined matrix for the matrix-based intra prediction mode and the boundary vector; deriving a prediction block for the current block based on the prediction samples; deriving a boundary vector by downsampling neighboring samples of the current block, encoding, into the bitstream, a syntax element indicating a splicing order between a first entry of the boundary vector downsampled from a left neighboring sample adjacent to the current block and a second entry of the boundary vector downsampled from a top neighboring sample adjacent to the current block; splicing the first entry and the second entry according to the splicing order to generate the boundary vector; The method is characterized in that the current block is a luma block consisting of luma components, the matrix-based intra prediction is performed on the luma block, and if the intra prediction mode of a chroma block corresponding to the luma block uses the intra prediction mode of the luma block as is, the normal intra prediction mode for the luma block is set to planar mode.

6. The syntax element indicating a matrix-based intra prediction mode for the current block may include:

6. The method of claim 5, wherein the matrix-based intra-prediction mode is a truncated binary code that specifies one of a plurality of matrix-based intra-prediction modes that are allowed for a width and height of the current block.

7. deriving a boundary vector by downsampling neighboring samples of the current block, 7. The method of claim 6, further comprising deriving the boundary vector from one or two of a set of left surrounding samples and a set of upper surrounding samples adjacent to the current block, selected depending on a matrix-based intra prediction mode for the current block.

8. deriving a prediction block for the current block based on the prediction samples, assigning the prediction samples to positions within the prediction block; 7. The method of claim 6, further comprising: generating predicted sample values ​​for locations in the predicted block not assigned to the predicted samples using linear interpolation on the predicted samples, left neighboring samples adjacent to the current block, and upper neighboring samples adjacent to the current block.

9. 1. A computer-readable medium having stored thereon a method for causing a processor to transmit a bitstream including encoded data for a block of video data, the method comprising: The method comprises: encoding the video data blocks into a bitstream; transmitting the bitstream to a video decoding device; generating a residual block of a current block using the predicted block; encoding residual data specifying the residual block into the bitstream; Including, The step of encoding the video data blocks into a bitstream comprises: encoding a syntax element into the bitstream indicating an intra-prediction type for a current block of video data, the intra-prediction type including matrix-based intra-prediction and regular intra-prediction; selectively performing matrix-based intra prediction or regular intra prediction based on an intra prediction type of the current block to generate a predicted block for the current block; Including, The step of performing regular intra prediction to generate a predicted block for the current block comprises: determining a regular intra prediction mode of the current block; deriving Most Probable Mode (MPM) candidates based on normal intra prediction modes of neighboring blocks of the current block to construct an MPM list for the current block; encoding at least one syntax element into the bitstream, the syntax element indicating a normal intra-prediction mode of the current block based on the MPM list; The step of performing matrix-based intra prediction to generate a predicted block for the current block includes: encoding a syntax element indicating a matrix-based intra-prediction mode for the current block into the bitstream; deriving a boundary vector by downsampling surrounding samples adjacent to the current block based on a width and height of the current block; generating a prediction sample for the current block based on a matrix-vector multiplication between a predefined matrix for the matrix-based intra prediction mode and the boundary vector; deriving a prediction block for the current block based on the prediction samples; deriving a boundary vector by downsampling neighboring samples of the current block, encoding, into the bitstream, a syntax element indicating a splicing order between a first entry of the boundary vector downsampled from a left neighboring sample adjacent to the current block and a second entry of the boundary vector downsampled from a top neighboring sample adjacent to the current block; splicing the first entry and the second entry according to the splicing order to generate the boundary vector; The current block is a luma block consisting of luma components, and the matrix-based intra prediction is performed on the luma block. If the intra prediction mode of a chroma block corresponding to the luma block uses the intra prediction mode of the luma block as is, a normal intra prediction mode for the luma block is set to a planar mode.

Images