Method and apparatus for video coding using matrix-based intra prediction

Abstract

The present embodiments provide video coding methods and apparatuses utilizing matrix-based intra prediction. In the present embodiments, an image decoding apparatus obtains a weight matrix for applying matrix-based intra prediction (MIP) to a current block, and generates a prediction block of the current block by applying the weight matrix to reference pixels of the current block. The image decoding apparatus derives a virtual intra prediction mode based on neighboring / non-neighboring samples of the current block, and selects a separable / non-separable inverse transform kernel of the current block based on the virtual intra prediction mode. The image decoding apparatus reconstructs transform coefficients of the current block from a bitstream, and generates a residual block of the current block by applying the separable / non-separable inverse transform kernel of the current block to the transform coefficients.

Description

Technical Field

[0001] This invention relates to a video encoding and decoding method and apparatus that utilizes matrix-based intra-frame prediction. Background Technology

[0002] The statements in this section are provided only as background information in relation to the present invention and do not necessarily constitute prior art.

[0003] Compared to audio or still image data, video data is much larger in volume, and unprocessed or uncompressed video data requires significant hardware resources (including memory) to store or transmit it in its raw form.

[0004] Accordingly, when storing or transmitting video data, an encoder is typically used to compress the video data for storage or transmission, and a decoder receives the compressed video data and performs decompression and reconstruction. Such video compression technologies include H.264 / AVC, High Efficiency Video Coding (HEVC), and Versatile Video Coding (VVC), which offers approximately 30% or more improvement in coding efficiency compared to HEVC.

[0005] However, as video size, resolution, and frame rate continue to increase, and the amount of data that needs to be encoded and decoded also increases accordingly, a new compression technology is needed that is more efficient in terms of encoding and decoding and more effective in improving video quality than existing compression technologies.

[0006] In particular, the demand for high-resolution, high-quality video (e.g., High Definition (HD) and Ultra High Definition (UHD) video) has been increasing in various fields recently. As the resolution and quality of image data become higher, the amount of information or bits transmitted also increases compared to existing image data. This increase in the amount of information or bits transmitted leads to increased transmission and storage costs. Furthermore, there is a growing interest in and demand for immersive media such as Virtual Reality (VR), Artificial Reality (AR) content, and holograms, and the broadcasting of videos / images with characteristics different from real-world images (e.g., game footage) is increasing. Accordingly, there is a need for an efficient video compression technology to efficiently compress, transmit, store, and play information from high-resolution, high-quality video with all the aforementioned characteristics. Summary of the Invention

[0007] Technical issues The present invention aims to provide a video coding and decoding method and apparatus for performing intra-frame prediction using matrix-based intra-frame prediction (MIP) with multiple candidates.

[0008] The embodiments of the present invention aim to provide a video coding and decoding method and apparatus for performing intra-frame prediction using a MIP having multiple weight matrices.

[0009] The embodiments of the present invention aim to provide a video encoding and decoding method and apparatus for selecting a separable / non-separable transform core of a MIP.

[0010] According to embodiments of the present invention, a video encoding / decoding method performed by a video encoding / decoding apparatus is provided. According to embodiments of the present invention, a video encoding / decoding apparatus for performing video encoding / decoding is provided.

[0011] According to embodiments of the present invention, a computer-readable digital recording medium is provided that stores encoded video information generated according to a video encoding method disclosed in at least one embodiment of the present invention. According to embodiments of the present invention, a computer-readable digital recording medium storing encoded information or encoded image information is provided that enables the video decoding method disclosed in at least one embodiment of the present invention to be executed by a video decoding apparatus.

[0012] Additionally, embodiments of the present invention provide a method or apparatus for transmitting video data, the video data comprising a bitstream generated based on a video encoding method disclosed in at least one embodiment of the present invention.

[0013] Technical solution At least one aspect of the present invention provides a method for reconstructing a current block, performed by a video decoding apparatus. The method includes: obtaining a weight matrix for applying matrix-based intra-frame prediction (MIP) to the current block. The method further includes: generating a prediction block of the current block by applying the weight matrix to reference pixels of the current block. The method further includes: deriving a virtual intra-frame prediction mode based on neighboring / non-neighboring samples of the current block. The method further includes: selecting a separable / non-separable inverse transform kernel for the current block based on the virtual intra-frame prediction mode. The method further includes: reconstructing transform coefficients of the current block from a bitstream. The method further includes: generating a residual block of the current block by applying the separable / non-separable inverse transform kernel of the current block to the transform coefficients.

[0014] Another aspect of the invention provides a method for encoding a current block, performed by a video coding apparatus. The method includes: obtaining a weight matrix for applying matrix-based intra-frame prediction (MIP) to the current block. The method further includes: generating a prediction block for the current block by applying the weight matrix to reference pixels of the current block. The method further includes: deriving a virtual intra-frame prediction mode based on neighboring / non-neighboring samples of the current block. The method further includes: selecting a separable / non-separable transform kernel for the current block based on the virtual intra-frame prediction mode. The method further includes: generating a residual block for the current block by subtracting the prediction block from the current block. The method further includes: generating transform coefficients by applying the separable / non-separable kernel of the current block to the residual block.

[0015] Another aspect of the present invention provides a method for providing video data to a video decoding apparatus. The method includes encoding the video data into a bitstream and transmitting the bitstream to the video decoding apparatus. Encoding the video data includes: obtaining a weight matrix for applying matrix-based intra-frame prediction (MIP) to a current block. Encoding the video data further includes: generating a prediction block for the current block by applying the weight matrix to reference pixels of the current block. Encoding the video data further includes: deriving a virtual intra-frame prediction mode based on neighboring / non-neighboring samples of the current block. Encoding the video data further includes: selecting a separable / non-separable transform kernel for the current block based on the virtual intra-frame prediction mode. Encoding the video data further includes: generating a residual block for the current block by subtracting the prediction block from the current block. Encoding the video data further includes: generating transform coefficients by applying the separable / non-separable kernel of the current block to the residual block.

[0016] Beneficial effects As described above, the present invention provides a video encoding / decoding method and apparatus that utilizes a MIP with multiple candidate transforms (MIPs) to perform intra-frame prediction, utilizes a MIP with multiple weight matrices to perform intra-frame prediction, and selects a separable / non-separable transform kernel for the MIP. Therefore, the video encoding / decoding method and apparatus improve objective video encoding / decoding efficiency and enhance subjective video quality. Attached Figure Description

[0017] Figure 1 This is a block diagram of a video encoding device that can implement the technology of the present invention.

[0018] Figure 2 This paper demonstrates a method for segmenting blocks using a quadtree plus binary tree ternary tree (QTBTTT) structure.

[0019] Figure 3a and Figure 3b Several intra-prediction modes, including a wide-angle intra-prediction mode, are shown.

[0020] Figure 4 The adjacent blocks of the current block are shown.

[0021] Figure 5 This is a block diagram of a video decoding device that can implement the technology of the present invention.

[0022] Figure 6 This is an example schematic diagram illustrating the search region used in the IntraTMP (Intra-Template Matching Prediction) technique.

[0023] Figure 7 This is an example diagram illustrating a method for deriving one or more intra-predictive modes in Template-based intra-modederivation (TIMD) technology.

[0024] Figure 8 This is an example schematic diagram illustrating a method of combining an intra-prediction mode and a segmentation mode using a signal to emit a Spatial Geometric Partitioning Mode (SGPM).

[0025] Figure 9 This is an example diagram showing the template of the current block with SGPM mode applied.

[0026] Figure 10 This is an example schematic diagram illustrating a reference template for matrix-based intra-prediction (MIP).

[0027] Figure 11 This is a flowchart illustrating a method for encoding the current block performed by a video encoding apparatus according to an embodiment of the present invention.

[0028] Figure 12 This is a flowchart illustrating a method for reconstructing the current block performed by a video decoding device according to an embodiment of the present invention. Detailed Implementation

[0029] In the following description, some embodiments of the invention are described in detail with reference to the accompanying illustrative drawings. In the following description, the same reference numerals denote the same elements, although these elements are shown in different drawings. Furthermore, in the following description of some embodiments, detailed descriptions of relevant known components and functions may be omitted for clarity and brevity when it is considered that such detailed descriptions obscure the subject matter of the invention.

[0030] Figure 1 This is a block diagram of a video encoding apparatus that can implement the technology of the present invention. In the following, reference is made to... Figure 1The diagram illustrates a video encoding device and its components.

[0031] The encoding device may include: an image segmenter 110, a predictor 120, a subtractor 130, a transformer 140, a quantizer 145, a reordering unit 150, an entropy encoder 155, an inverse quantizer 160, an inverse transformer 165, an adder 170, a loop filter unit 180, and a memory 190.

[0032] Each component of the encoding device can be implemented as hardware or software, or a combination of hardware and software. Furthermore, the function of each component can be implemented as software, and the microprocessor can be implemented to execute the software function corresponding to each component.

[0033] A video consists of one or more sequences of images. Each image is segmented into multiple regions, and encoding is performed on each region. For example, an image is segmented into one or more tiles and / or slices. Here, one or more tiles can be defined as a tile group. Each tile and / or slice is segmented into one or more coding tree units (CTUs). Additionally, each CTU is segmented into one or more coding units (CUs) in a tree structure. Information applied to each coding unit (CU) is encoded as the syntax of the CU, and information commonly applied to CUs included in a CTU is encoded as the syntax of the CTU. Furthermore, information commonly applied to all blocks in a slice is encoded as the syntax of the slice header, and information applied to all blocks constituting one or more images is encoded as a picture parameter set (PPS) or picture header. Additionally, information commonly referenced by multiple images is encoded as a sequence parameter set (SPS). Furthermore, information commonly referenced by one or more SPSs is encoded as a video parameter set (VPS). Furthermore, information commonly applied to a tile or tile group can also be encoded as the syntax of the tile or tile group header. The syntax included in SPS, PPS, slice headers, and tile or tile group headers can be called high-level syntax.

[0034] Image splitter 110 determines the size of the coding tree unit (CTU). Information about the size of the CTU (CTU size) is encoded into SPS or PPS syntax and transmitted to the video decoding device.

[0035] Image segmenter 110 divides each image constituting the video into multiple coding tree units (CTUs) of a predetermined size, and then recursively segments the CTUs using a tree structure. The leaf nodes in the tree structure are called coding units (CUs), which are the basic units of encoding.

[0036] A tree structure can be a quadtree (QT) that divides a higher node (or parent node) into four lower nodes (or child nodes) of the same size. A tree structure can also be a binary tree (BT) that divides a higher node into two lower nodes. A tree structure can also be a ternary tree (TT) that divides a higher node into three lower nodes in a 1:2:1 ratio. A tree structure can also be a combination of two or more of the QT, BT, and TT structures. For example, a quadtree plus binary tree (QTBT) structure can be used, or a quadtree plus binary tree ternary tree (QTBTTT) structure can be used. Here, a binary tree ternary tree (BTTT) is added to the tree structure to form a multiple-type tree (MTT).

[0037] Figure 2 This is a schematic diagram illustrating a method for segmenting blocks using a QTBTTT structure.

[0038] like Figure 2 As shown, the CTU can first be segmented into a QT structure. Quadtree segmentation can be recursive until the size of the segmented blocks reaches the minimum block size (MinQTSize) allowed for leaf nodes in the QT. The entropy encoder 155 encodes a first flag (QT_split_flag) indicating whether each node of the QT structure is segmented into the four nodes below it and signals this flag to the video decoding device. When the leaf node of the QT is not larger than the maximum block size (MaxBTSize) allowed for the root node in the BT, the leaf node can be further segmented using at least one of the BT or TT structures. Multiple segmentation directions can exist in the BT and / or TT structures. For example, two directions can exist: a horizontal direction for segmenting the blocks of the corresponding node and a vertical direction for segmenting the blocks of the corresponding node. Figure 2 As shown, when MTT splitting begins, the entropy encoder 155 encodes a second flag (mtt_split_flag) indicating whether a node has been split, as well as a flag indicating the splitting direction (vertical or horizontal) and / or the splitting type (binary or ternary) if the node has been split, and signals this to the video decoding device.

[0039] Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node has been split into the four lower-level nodes, the CU splitting flag (split_cu_flag) indicating whether a node has been split can also be encoded. When the value of the CU splitting flag (split_cu_flag) indicates that each node has not been split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a CU, which is the basic unit of encoding. When the value of the CU splitting flag (split_cu_flag) indicates that each node has been split, the video encoding device begins encoding the first flag using the above scheme.

[0040] When QTBT is used as another example of a tree structure, two types can exist: a type where the corresponding node's block is horizontally divided into two blocks of the same size (i.e., symmetrical horizontal division) and a type where the corresponding node's block is vertically divided into two blocks of the same size (i.e., symmetrical vertical division). The entropy encoder 155 encodes a split flag (split_flag) indicating whether each node of the BT structure is divided into blocks of the lower layer and split type information indicating the split type, and transmits this information to the video decoding device. Alternatively, a type can exist where the corresponding node's block is divided into two blocks that are asymmetrically divided. Asymmetrical forms can include dividing the corresponding node's block into two rectangular blocks with a size ratio of 1:3, or it can also include dividing the corresponding node's block along a diagonal direction.

[0041] The CU can have various sizes depending on the QTBT or QTBTTT segmentation from the CTU. In the following text, the block corresponding to the CU to be encoded or decoded (i.e., the leaf node of the QTBTTT) is called the "current block". When using QTBTTT segmentation, the current block can also be rectangular in shape in addition to a square shape.

[0042] Predictor 120 predicts the current block to generate a prediction block. Predictor 120 includes an intra-frame predictor 122 and an inter-frame predictor 124.

[0043] Typically, each block of the current image can be predictively encoded. This prediction can usually be performed using intra-frame prediction (which utilizes data from the image containing the current block) or inter-frame prediction (which utilizes data from an image encoded before the image containing the current block). Inter-frame prediction includes both one-way and two-way prediction.

[0044] Intra-predictor 122 predicts pixels in the current block by utilizing pixels (reference pixels) that are adjacent to the current block in the current image, including the current block. Depending on the prediction direction, there are various intra-prediction modes. For example, such as... Figure 3a As shown, various intra-frame prediction modes can include two non-directional modes: Planar mode and DC mode, and can include 65 directional modes. The neighboring pixels and algorithm equations to be used are defined differently for each prediction mode.

[0045] To perform efficient orientation prediction for the current block with a rectangular shape, additional methods such as... can be used. Figure 3b The dashed arrows indicate the directional modes (intra-prediction modes #67 to #80, #-1 to #-14). These directional modes can be referred to as "wide-angle intra-prediction modes." Figure 3b In the diagram, the arrows indicate the corresponding reference samples used for prediction, not the prediction direction. The prediction direction is opposite to the direction indicated by the arrow. When the current block has a rectangular shape, the wide-angle intra-frame prediction mode is a mode that performs prediction in the opposite direction to the specific direction mode without additional bit transmission. In this case, in the wide-angle intra-frame prediction mode, some wide-angle intra-frame prediction modes available for the current block can be determined by the ratio of the width to the height of the current block with a rectangular shape. For example, when the current block has a rectangular shape with a height less than its width, wide-angle intra-frame prediction modes with angles less than 45 degrees (intra-frame prediction modes #67 to #80) are available. When the current block has a rectangular shape with a width greater than its height, wide-angle intra-frame prediction modes with angles greater than -135 degrees (intra-frame prediction modes #-1 to #-14) are available.

[0046] Intra-predictor 122 can determine the intra-prediction to be used for encoding the current block. In some examples, intra-predictor 122 can encode the current block by utilizing multiple intra-prediction modes, and can also select the appropriate intra-prediction mode to use from test modes. For example, intra-predictor 122 can calculate rate-distortion values ​​by utilizing rate-distortion analysis of intra-prediction modes across multiple tests, and can also select the intra-prediction mode with the best rate-distortion characteristics from the test modes.

[0047] Intra-predictor 122 selects one intra-prediction mode from a variety of intra-prediction modes and predicts the current block by using neighboring pixels (reference pixels) determined according to the selected intra-prediction mode and an algorithm equation. Entropy encoder 155 encodes the information about the selected intra-prediction mode and transmits it to the video decoding device.

[0048] Inter-frame predictor 124 generates a predicted block for the current block by utilizing motion compensation processing. Inter-frame predictor 124 searches for the most similar block to the current block in a reference image that was encoded and decoded earlier than the current image, and generates the predicted block for the current block using the searched block. Additionally, a motion vector (MV) is generated, corresponding to the displacement between the current block in the current image and the predicted block in the reference image. Typically, motion estimation is performed on the luma component, and the motion vector calculated based on the luma component is used for both the luma and chroma components. The motion information, including information about the reference image and information about the motion vector used to predict the current block, is encoded by entropy encoder 155 and transmitted to the video decoding device.

[0049] The inter-frame predictor 124 can also perform interpolation on a reference image or reference block to increase prediction accuracy. In other words, subsamples are interpolated between two consecutive integer samples by applying filter coefficients to multiple consecutive integer samples, including two integer samples. When performing a process to search for the most similar block to the current block on the interpolated reference image, the motion vector can be represented with fractional-unit precision instead of integer-sample-unit precision. For each target region to be encoded, such as a cell like a slice, tile, CTU, or CU, the precision or resolution of the motion vector can be set differently. When this Adaptive Motion Vector Resolution (AMVR) is applied, information about the motion vector resolution to be applied to each target region should be signaled for each target region. For example, when the target region is a CU, information about the motion vector resolution applied to each CU is signaled. The information about the motion vector resolution can be information representing the precision of the motion vector difference to be described below.

[0050] On the other hand, the inter-frame predictor 124 can perform inter-frame prediction using bidirectional prediction. In the case of bidirectional prediction, two reference images and two motion vectors representing the location of the block most similar to the current block in each reference image are used. The inter-frame predictor 124 selects a first reference image and a second reference image from reference image list 0 (RefPicList0) and reference image list 1 (RefPicList1), respectively. The inter-frame predictor 124 also searches for the block most similar to the current block in each reference image to generate a first reference block and a second reference block. In addition, the predicted block of the current block is generated by averaging or weighted averaging the first reference block and the second reference block. In addition, motion information including information about the two reference images used to predict the current block and information about the two motion vectors is transmitted to the entropy encoder 155. Here, reference image list 0 may consist of images in pre-reconstructed images that are displayed before the current image in display order, and reference image list 1 may consist of images in pre-reconstructed images that are displayed after the current image in display order. However, although not particularly limited to this, pre-reconstructed images that are displayed after the current image in display order may be additionally included in reference image list 0. Conversely, pre-reconstructed images preceding the current image can also be additionally included in the reference image list 1.

[0051] Various methods can be used to minimize the number of bits consumed in encoding motion information.

[0052] For example, when the reference image and motion vector of the current block are the same as those of the adjacent blocks, the information that can identify the adjacent blocks is encoded to transmit the motion information of the current block to the video decoding device. This method is called merge mode.

[0053] In merge mode, the inter-frame predictor 124 selects a predetermined number of merge candidate blocks (hereinafter referred to as "merge candidates") from the neighboring blocks of the current block.

[0054] As adjacent blocks used to derive merge candidates, all or some of the following can be used: left block A0, lower left block A1, upper block B0, upper right block B1, and upper left block B2, which are adjacent to the current block in the current image. Figure 4 As shown. In addition to the current image containing the current block, blocks located within a reference image (which may be the same as or different from the reference image used to predict the current block) can also be used as merge candidates. For example, a co-located block of the current block within the reference image, or a block adjacent to that co-located block, can be additionally used as a merge candidate. If the number of merge candidates selected using the above method is less than a preset number, a zero vector is added to the merge candidates.

[0055] The inter-frame predictor 124 configures a merge list including a predetermined number of merge candidates by utilizing neighboring blocks. It selects a merge candidate from the merge candidates included in the merge list to be used as motion information for the current block, and generates merge index information for identifying the selected candidate. The generated merge index information is encoded by the entropy encoder 155 and transmitted to the video decoding device.

[0056] The merge skip mode is a special case of the merge mode. After quantization, when all transform coefficients used for entropy coding are close to zero, only the adjacent block selection information is transmitted, and the residual signal is not transmitted. By utilizing the merge skip mode, relatively high coding efficiency can be achieved for images with slight motion, still images, and screen content images.

[0057] From then on, merge mode and merge skip mode were collectively referred to as merge / skip mode.

[0058] Another approach for encoding motion information is the Advanced Motion Vector Prediction (AMVP) model.

[0059] In AMVP mode, the inter-frame predictor 124 derives motion vector prediction candidates for the current block by utilizing neighboring blocks of the current block. As neighboring blocks used to derive motion vector prediction candidates, they can use... Figure 4 The current block in the current image shown includes all or some of the adjacent blocks: left block A0, lower left block A1, upper block B0, upper right block B1, and upper left block B2. Furthermore, blocks located in a reference image (which may be the same as or different from the reference image used to predict the current block) can also be used as neighboring blocks for deriving motion vector prediction candidates, in addition to the current image containing the current block. For example, a co-located block of the current block in the reference image or a block adjacent to that co-located block can be used. If the number of motion vector candidates selected by the above method is less than a preset number, a zero vector is added to the motion vector candidates.

[0060] The inter-frame predictor 124 derives motion vector prediction candidates by utilizing motion vectors from neighboring blocks, and determines the motion vector prediction for the current block by utilizing these candidates. Furthermore, the motion vector difference is calculated by subtracting the motion vector prediction from the motion vector prediction for the current block.

[0061] Motion vector predictions can be obtained by applying predefined functions (e.g., median and average calculations) to motion vector prediction candidates. In this case, the video decoding device also knows the predefined functions. Furthermore, since the neighboring blocks used to derive the motion vector prediction candidates are blocks that have already been encoded and decoded, the video decoding device may also already know the motion vectors of the neighboring blocks. Therefore, the video encoding device does not need to encode the information used to identify the motion vector prediction candidates. Accordingly, in this case, information about the motion vector difference and information about the reference image used to predict the current block are encoded.

[0062] Alternatively, motion vector prediction can be determined by selecting any of the motion vector prediction candidates. In this case, additional encoding is performed on the information used to identify the selected motion vector prediction candidate, as well as the information about the motion vector difference used to predict the current block and the information about the reference image used to predict the current block.

[0063] Subtractor 130 generates a residual block by subtracting the predicted block generated by intra-predictor 122 or inter-predictor 124 from the current block.

[0064] Transformer 140 transforms the residual signal in the residual block, which has pixel values ​​in the spatial domain, into transform coefficients in the frequency domain. Transformer 140 can transform the residual signal in the residual block by using the entire size of the residual block as a transform unit, or it can divide the residual block into multiple sub-blocks and perform the transformation by using the sub-blocks as transform units. Alternatively, the residual block is divided into two sub-blocks (i.e., a transformed region and a non-transformed region) to transform the residual signal by using only the transformed region sub-block as a transform unit. Here, the transformed region sub-block can be one of two rectangular blocks with a size ratio of 1:1 based on a horizontal axis (or a vertical axis). In this case, the entropy encoder 155 encodes a flag (cu_sbt_flag) indicating that only the transformed sub-block is used, as well as orientation (vertical / horizontal) information (cu_sbt_horizontal_flag) and / or position information (cu_sbt_pos_flag), and signals this information to the video decoding device. Furthermore, the size of the transformed region sub-blocks can be based on a 1:3 size ratio along the horizontal (or vertical) axis. In this case, the entropy encoder 155 additionally encodes the flag (cu_sbt_quad_flag) for dividing the corresponding segment and signals it to the video decoding device.

[0065] On the other hand, the transformer 140 can perform transformations on the residual blocks individually in the horizontal and vertical directions. Various types of transformation functions or transformation matrices can be used for this transformation. For example, a pair of transformation functions for horizontal and vertical transformations can be defined as a Multiple Transform Set (MTS). The transformer 140 can select the transformation function pair with the highest transformation efficiency in the MTS and can transform the residual block in each of the horizontal and vertical directions. The entropy encoder 155 encodes the information (mts_idx) about the transformation function pair in the MTS and signals it to the video decoding device.

[0066] Quantizer 145 quantizes the transform coefficients output from transformer 140 using quantization parameters and outputs the quantized transform coefficients to entropy encoder 155. Quantizer 145 can also quantize the relevant residual blocks immediately without transforming any block or frame. Quantizer 145 can also apply different quantization coefficients (scaling values) based on the position of the transform coefficients in the transform block. The quantization matrix applied to the quantized transform coefficients arranged in two dimensions can be encoded and signaled to the video decoding device.

[0067] The reordering unit 150 can perform a rearrangement of the coefficient values ​​on the quantized residual values.

[0068] The reordering unit 150 can transform a 2D coefficient array into a 1D coefficient sequence using coefficient scanning. For example, the reordering unit 150 can output a 1D coefficient sequence by scanning the DC coefficients to high-frequency region coefficients using a zig-zag scan or a diagonal scan. Depending on the size of the transform unit and the intra-frame prediction mode, a vertical scan scanning the 2D coefficient array in the column direction and a horizontal scan scanning the 2D block-type coefficients in the row direction can also be used instead of a zig-zag scan. In other words, the scanning method to be used can be determined from zig-zag scan, diagonal scan, vertical scan, and horizontal scan, depending on the size of the transform unit and the intra-frame prediction mode.

[0069] The entropy encoder 155 encodes the 1D quantized transform coefficient sequence output from the reordering unit 150 by utilizing various encoding schemes, including Context-based Adaptive Binary Arithmetic Code (CABAC) and Exponential Golomb, to generate a bitstream.

[0070] Furthermore, the entropy encoder 155 encodes block-segmentation related information (e.g., CTU size, CTU segmentation flag, QT segmentation flag, MTT segmentation type, MTT segmentation direction, etc.) so that the video decoding device can segment blocks equivalently to the video encoding device. Additionally, the entropy encoder 155 encodes information indicating whether the current block is encoded using intra-frame prediction or inter-frame prediction. The entropy encoder 155 encodes intra-frame prediction information (i.e., information about the intra-frame prediction mode) or inter-frame prediction information (the merge index in the case of merge mode, and information about the reference image index and motion vector difference in the case of AMVP mode) according to the prediction type. Furthermore, the entropy encoder 155 encodes quantization-related information (i.e., information about quantization parameters and information about the quantization matrix).

[0071] Inverse quantizer 160 inverse quantizes the quantized transform coefficients output from quantizer 145 to generate transform coefficients. Inverse transformer 165 transforms the transform coefficients output from inverse quantizer 160 from the frequency domain to the spatial domain to reconstruct the residual block.

[0072] Adder 170 adds the reconstructed residual block to the predicted block generated by predictor 120 to reconstruct the current block. When performing intra-frame prediction for the next sequential block, the pixels in the reconstructed current block can be used as reference pixels.

[0073] The loop filtering unit 180 performs filtering on the reconstructed pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc., that occur due to block-based prediction and transform / quantization. The loop filtering unit 180, as an in-loop filter, may include all or some of the following: a deblocking filter 182, a sample adaptive offset (SAO) filter 184, and an adaptive loop filter (ALF) 186.

[0074] Deblocking filter 182 filters the boundaries between reconstructed blocks to remove blocking artifacts caused by block unit encoding / decoding, and SAO filter 184 and ALF 186 perform additional filtering on the deblocked video. SAO filter 184 and ALF 186 are filters used to compensate for the difference between reconstructed pixels and original pixels caused by lossy coding. SAO filter 184 applies an offset in CTU units to enhance subjective image quality and coding efficiency. On the other hand, ALF 186 performs block unit filtering and applies different filters to compensate for distortion by dividing the boundaries of the corresponding blocks and the degree of variation. Information about the filter coefficients to be used for ALF can be encoded and signaled to the video decoding device.

[0075] The reconstructed blocks filtered by deblocking filter 182, SAO filter 184, and ALF 186 are stored in memory 190. When all blocks in an image are reconstructed, the reconstructed image can be used as a reference image for inter-frame prediction of blocks in subsequent images to be encoded.

[0076] Video encoding devices can store the bitstream of encoded video data in a non-volatile storage medium or send the bitstream to a video decoding device via a communication network.

[0077] Figure 5 This is a functional block diagram of a video decoding device that can implement the technology of the present invention. In the following text, reference is made to... Figure 5 This describes a video decoding device and its components.

[0078] The video decoding device may include: an entropy decoder 510, a reordering unit 515, an inverse quantizer 520, an inverse transformer 530, a predictor 540, an adder 550, a loop filter unit 560, and a memory 570.

[0079] Similar to Figure 1 The video encoding device and the video decoding device each component can be implemented as hardware or software, or a combination of hardware and software. Furthermore, the function of each component can be implemented as software, and the microprocessor can also be implemented to execute the software functions corresponding to each component.

[0080] The entropy decoder 510 extracts information related to block segmentation by decoding the bitstream generated by the video encoding device to determine the current block to be decoded, and extracts the prediction information and information about the residual signal required to reconstruct the current block.

[0081] The entropy decoder 510 determines the size of the CTU by extracting information about the CTU size from the sequence parameter set (SPS) or the picture parameter set (PPS), and segments the image into CTUs of a defined size. Furthermore, the CTU is identified as the highest level (i.e., the root node) of the tree structure, and segmentation information for the CTU can be extracted to segment the CTU by utilizing the tree structure.

[0082] For example, when segmenting a CTU using a QTBTT structure, the first flag (QT_split_flag) related to the QT segmentation is first extracted to split each node into four lower-level nodes. Additionally, a second flag (mtt_split_flag), segmentation direction (vertical / horizontal), and / or segmentation type (binary / ternary) related to the MTT segmentation are extracted relative to the nodes corresponding to the leaf nodes of the QT to segment the corresponding leaf nodes using the MTT structure. Thus, each node below a leaf node of the QT is recursively segmented using either a BT or TT structure.

[0083] As another example, when splitting the CTU using the QTBTTT structure, a CU split flag (split_cu_flag) indicating whether the CU has been split is extracted. A first flag (QT_split_flag) can also be extracted when the corresponding block is split. During the splitting process, for each node, zero or more recursive MTT splits may occur after zero or more recursive QT splits. For example, with respect to the CTU, MTT splits may occur immediately, or conversely, only multiple QT splits may occur.

[0084] As another example, when segmenting a CTU using a QTBT structure, the first flag (QT_split_flag) associated with the segmentation of the QT is extracted to split each node into four nodes at the lower level. Additionally, a split flag (split_flag) indicating whether the node corresponding to a leaf node of the QT will be further segmented using BT, as well as segmentation direction information, is extracted.

[0085] On the other hand, when the entropy decoder 510 determines the current block to be decoded by utilizing the segmentation of the tree structure, the entropy decoder 510 extracts information about the prediction type indicating whether the current block is predicted intra-frame or inter-frame. When the prediction type information indicates intra-frame prediction, the entropy decoder 510 extracts the syntax elements for the intra-frame prediction information (intra-frame prediction mode) for the current block. When the prediction type information indicates inter-frame prediction, the entropy decoder 510 extracts information representing the syntax elements for the inter-frame prediction information, namely, the motion vector and the reference picture for the motion vector reference.

[0086] In addition, the entropy decoder 510 extracts quantization-related information and extracts information about the quantization transform coefficients of the current block as information about the residual signal.

[0087] The reordering unit 515 can again change the 1D quantization transform coefficient sequence entropy decoded by the entropy decoder 510 into a 2D coefficient array (i.e., a block) in the reverse order of the coefficient scan order performed by the video encoding device.

[0088] The inverse quantizer 520 performs inverse quantization on the quantized transform coefficients, and performs inverse quantization on the quantized transform coefficients using quantization parameters. The inverse quantizer 520 can also apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in 2D. The inverse quantizer 520 can perform inverse quantization by applying a matrix of quantization coefficients (scaling values) from the video encoding device to a 2D array of quantized transform coefficients.

[0089] The inverse transformer 530 reconstructs the residual signal by inversely transforming the inverse-quantized transform coefficients from the frequency domain to the spatial domain, thereby generating the residual block of the current block.

[0090] Furthermore, when the inverse transformer 530 performs an inverse transform on a portion (sub-block) of the transform block, it extracts a flag (cu_sbt_flag) indicating that only the sub-blocks of the transform block are transformed, the orientation (vertical / horizontal) information (cu_sbt_horizontal_flag) of the sub-blocks, and / or the position information (cu_sbt_pos_flag) of the sub-blocks. The inverse transformer 530 also inverse transforms the transform coefficients of the corresponding sub-blocks from the frequency domain to the spatial domain to reconstruct the residual signal, and fills the regions not inversely transformed with values ​​of "0" as the residual signal to generate the final residual block of the current block.

[0091] Furthermore, when applying MTS, the inverse transformer 530 determines the transform function or transform matrix to be applied in both the horizontal and vertical directions by utilizing the MTS information (mts_idx) signaled from the video encoding device. The inverse transformer 530 also performs inverse transform on the transform coefficients in the transform block in both the horizontal and vertical directions using the determined transform function.

[0092] Predictor 540 may include intra-predictor 542 and inter-predictor 544. Intra-predictor 542 is activated when the prediction type of the current block is intra-prediction, and inter-predictor 544 is activated when the prediction type of the current block is inter-prediction.

[0093] Intra-predictor 542 determines the intra-prediction mode for the current block among multiple intra-prediction modes based on the syntax elements for the intra-prediction mode extracted from entropy decoder 510. Intra-predictor 542 also predicts the current block by utilizing neighboring reference pixels of the current block according to the intra-prediction mode.

[0094] The inter-frame predictor 544 determines the motion vector and the reference picture of the motion vector reference for the current block by using the syntax elements for the inter-frame prediction mode extracted from the entropy decoder 510, and predicts the current block by using the motion vector and the reference picture.

[0095] Adder 550 reconstructs the current block by adding the residual block output from inverse transform 530 to the predicted block output from inter-frame predictor 544 or intra-frame predictor 542. Pixels within the reconstructed current block are used as reference pixels when performing intra-frame prediction on a subsequent block to be decoded.

[0096] The loop filtering unit 560, acting as an in-loop filter, may include a deblocking filter 562, a SAO filter 564, and an ALF 566. The deblocking filter 562 performs deblocking filtering on the boundaries between reconstructed blocks to remove block artifacts caused by block unit decoding. The SAO filter 564 and ALF 566 perform additional filtering on the reconstructed blocks after deblocking to compensate for differences between reconstructed pixels and original pixels caused by lossy encoding. The filter coefficients of the ALF are determined by utilizing information about the filter coefficients decoded from the bitstream.

[0097] The reconstructed blocks filtered by deblocking filter 562, SAO filter 564, and ALF 566 are stored in memory 570. When all blocks in an image are reconstructed, the reconstructed image can be used as a reference image for inter-frame prediction of blocks in subsequent images to be encoded.

[0098] In some embodiments, the present invention relates to encoding and decoding video images as described above. More specifically, the present invention provides a video encoding / decoding method and apparatus for performing intra-frame prediction using a matrix-based intra-prediction (MIP) with multiple candidates. The present invention provides a video encoding / decoding method and apparatus for performing intra-frame prediction using an MIP with multiple weight matrices. Additionally, a video encoding / decoding method and apparatus for selecting a separable / non-separable transform kernel of the MIP is provided.

[0099] The following implementation scheme can be performed by the intra-frame predictor 122 in the video encoding apparatus. The following implementation scheme can also be performed by the intra-frame predictor 542 in the video decoding apparatus.

[0100] When encoding the current block, the video encoding apparatus can generate signaling information associated with this embodiment in terms of optimizing rate distortion. The video encoding apparatus can encode the signaling information using the entropy encoder 155 and can send the encoded signaling information to the video decoding apparatus. The video decoding apparatus can decode the signaling information associated with the decoding of the current block from the bitstream using the entropy decoder 510.

[0101] In the following description, the term "target block" may be used interchangeably with the current block or coding unit (CU), or may refer to some region of a coding unit.

[0102] In addition, a flag value of true indicates that the flag is set to 1. Conversely, a flag value of false indicates that the flag is set to 0.

[0103] Decoder side includes Figure 1 The video encoding apparatus shown includes all or part of the inverse quantizer 160, inverse transformer 165, predictor 120, adder 170, loop filter unit 180, and memory 190. Alternatively, the decoder side includes... Figure 5 The video decoding apparatus shown includes all or part of the inverse quantizer 520, inverse converter 530, predictor 540, adder 550, loop filter unit 560, and memory 570. Regarding a series of decoding processes, the decoder side of the video encoding apparatus and the decoder side of the video decoding apparatus perform the same operations. The video encoding apparatus determines information related to the operation on the decoder side and notifies the video decoding apparatus of the determined information via a signal. The video decoding apparatus can decode the information notified via the signal and can operate the decoder side based on the decoded information.

[0104] I-1. Intra-frame prediction techniques in VVC In intra-frame prediction in VVC, the direction (angle) prediction is subdivided into up to 65 sub-directions, such as... Figure 3a As illustrated, prediction modes 2 through 66 (predModeIntra) can be used depending on the prediction angle of the intra-prediction mode. By introducing Wide-Angle Intra Prediction (WAIP), prediction modes -14 through -1 and 67 through 80, which are directional modes with larger angles, can be used based on the block's aspect ratio. In intra-prediction, prediction blocks can be generated based on 67 Intra Prediction Modes (IPMs). The 67 IPMs refer to 67 intra-prediction modes emitted by the signal based on the block's aspect ratio, including non-directional prediction modes, planar modes, and DC modes from prediction modes -14 through 80.

[0105] VVC's intra-prediction techniques enable the generation of intra-predicted blocks using techniques such as Multiple Reference Line Intra Prediction (MRLP), Cross-Component Linear Model (CCLM), Position Dependent Intra Prediction Combination (PDPC), Intra Sub-Partition (ISP), and Matrix-based Intra Prediction (MIP).

[0106] I-2. Template Matching Prediction Template Matching Prediction (TMP) searches for a prediction block with the minimum difference between templates within a predefined reconstruction region (i.e., the search region) of the current frame. The template of the current block (hereinafter referred to as the "current template") consists of its upper and left neighboring samples. In this case, the difference between the template of the current block and the template of the prediction block found in the search region is defined as the cost function. Template Matching Prediction identifies the prediction block with the minimum cost as the prediction block of the current block. The Sum of Absolute Differences (SAD) can be used as the cost function.

[0107] In the Enhanced Compression Model (ECM) (the next-generation technology of VVC), Intra Template Matching Prediction (IntraTMP) sets up L-shaped left and top templates around the current block. After searching for the most similar template within the pre-recovery region of the current frame, IntraTMP uses a block that is adjacent to the found template and has the same size as the current block as the predicted block. IntraTMP uses the Sum of Absolute Differences (SAD) as a cost function to search for similar templates. The video encoding device transmits whether IntraTMP mode is used, and the video decoding device can perform the same template matching operation when IntraTMP mode is applied.

[0108] Figure 6 This is an example diagram illustrating the search area used in IntraTMP technology.

[0109] To conserve memory, the possible search area can be limited to the current coding tree unit (CTU) where the current block resides, the top-left CTU, the top CTU, and the left-side CTU. Figure 6 In this context, the search regions are denoted as R1 to R4. Within the possible search regions, the area generated by multiplying the width and height (w, h) of the current block by a preset constant a can be adaptively set as the search range. For example, by setting a=5, the search ranges SearchRange_w and SearchRange_h can be determined as shown in Equation 1.

[0110] [Equation 1] SearchRange_w = a × BlkW SearchRange_h = a × BlkH In Equation 1, BlkW and BlkH represent the width and height of the current block.

[0111] The template search area in IntraTMP mode can be predefined according to the protocol between the video encoding device and the video decoding device.

[0112] IntraTMP technology can include sub-modes, such as a mode that utilizes a single template (TMP Single), a technique that fuses multiple templates (TMP Fusion), a sub-pixel precision mode, and a linear filter model mode.

[0113] In single-template mode, the template search process comprises two steps. In the first step, a search is performed at 3-pixel intervals, and the block vectors (BVs) of 30 candidate templates (i.e., templates for candidate reference blocks) are included in the candidate list. In the second step, 19 BVs are ultimately determined by performing an additional search on the 30 BVs in the candidate list within adjacent 3×3 regions at 1-pixel intervals. Subsequently, the optimal reference block is selected for rate-distortion optimization, and the candidate list index indicating the final selected BV is signaled from the video encoding device to the video decoding device.

[0114] TMP fusion uses weights to combine multiple reference templates and reference blocks.

[0115] The subpixel precision mode searches for similar templates based on 1 / 2 pixel (Pel), 1 / 4 pixel, and 3 / 4 pixel precision.

[0116] The linear filter model mode uses a 6-tap filter to filter the prediction block. The 6-tap filter is a cross-shaped 5-tap filter with added bias. The filter coefficients can be calculated based on the relationship between the current template and the reference template. The current block is predicted using a reference block with the generated filter applied.

[0117] I-3. Decoder-Side Intra Mode Derivation (DIMD) Technology The DIMD method uses reconstructed pixels adjacent to the current block as templates and derives the intra-frame prediction mode by performing gradient analysis on the templates. DIMD extracts angle (direction) and magnitude information by analyzing the vertical and horizontal gradients within the template. The DIMD method calculates the horizontal gradient Gx and vertical gradient Gy by applying a horizontal Sobel filter and a vertical Sobel filter to the pixel position at the center of the template, respectively. The direction at the corresponding pixel position is calculated using atan(Gy / Gx), and the magnitude of the angle (direction) can be calculated by summing the absolute values ​​of Gx and Gy. The size of the template can be equal to or greater than the size of the Sobel filter. The template can include neighboring samples from the lower left and upper right sides of the current block.

[0118] The angle and magnitude calculated for each pixel position at the center of the template region are used to generate a Histogram of Gradients (HoG), which is constructed by accumulating the angle along the x-axis and the magnitude along the y-axis. The DIMD method can derive or determine the corresponding intra-prediction mode and its corresponding weights by mapping multiple angles (e.g., up to five) with relatively high accumulated magnitude values ​​on the HoG to directional patterns. The intra-prediction mode determined by the DIMD method can be called the DIMD mode and can then be used to generate the Most Probable Mode (MPM).

[0119] To form the final prediction block, non-directional predictors (based on plane or block vectors) can be used to perform a weighted sum of predictors for the intra-prediction mode determined by DIMD.

[0120] On the other hand, in intra-frame prediction, MPM technology utilizes the intra-frame prediction modes of neighboring blocks when performing intra-frame prediction on the current block. The video coding apparatus generates an MPM list to include intra-frame prediction modes derived from predefined locations spatially adjacent to the current block. The video coding apparatus can improve the coding efficiency of intra-frame prediction modes by transmitting the index of the MPM list instead of the index of the prediction mode.

[0121] I-4. Template-Based Intra Mode Derivation (TIMD) Technique The TIMD technique performs template prediction by applying intra-prediction modes from the MPM list to template regions adjacent to the current block. TIMD calculates the sum of absolute transformed differences (SATD) cost between the predicted values ​​and the reconstructed template values ​​in the template region. Based on the calculated cost, TIMD selects the mode costMode1 (hereinafter referred to as the first mode), representing the minimum cost, and the mode costMode2 (hereinafter referred to as the second mode), representing the second minimum cost. A non-directional mode with a lower SATD cost (costMode3) in the DC or plane is selected as the third mode. Non-directional modes can also be used if the following conditions are met: - The third non-directional mode is different from the first and second modes.

[0122] - CostMode3 < 1.5 × costMode1.

[0123] When both of the above conditions are true, the prediction blocks based on the three intra-frame prediction modes are combined using weights, as shown in Equation 2.

[0124] [Equation 2] If either of the above two conditions is false, then the first mode and the second mode are used. The TIMD technique determines whether to combine the weights of the two TIMD modes according to Equation 3.

[0125] [Equation 3] costMode2 < 2×costMode1 If Equation 3 is satisfied, the prediction blocks based on the two modes are weighted and combined. Conversely, if Equation 3 is not satisfied, the prediction mode with the lower cost is used. When combining the two modes, according to Equation 4, the larger weight is assigned to the mode with the lower SATD cost.

[0126] [Equation 4] Weight 1 = costMode2 / (costMode1 + costMode2) Weight 2 = 1 - Weight 1 Figure 7 This is an example schematic diagram illustrating a method for deriving one or more intra-frame prediction modes in TIMD technology.

[0127] The size of the template region can be determined based on the size of the current block, such as Figure 7As shown. If the width (W) or height (H) of the current block is greater than 8, then set the template size L1 or L2 to 4. If the width or height of the current block is 8 or less, then set L1 or L2 to 2.

[0128] I-5. Spatial Geometric Partitioning Mode (SGPM) Technology The SGPM technique divides the current block into two regions and performs different intra-frame predictions for each region. For example... Figure 8 As shown, the current block can be segmented into two parts with different intra-prediction modes ("intra_pred_mode0" and "intra_pred_mode1") using the segmentation mode. This information can be represented by a single candidate index ("sgpm_cand_idx") indicating a combination of one segmentation mode ("segmentation mode idx") and two intra-prediction modes ("intra_pred_mode0_idx" and "intra_pred_mode0_idx"), as shown. Figure 8 exemplified.

[0129] The candidates in the list are reconstructed in the order of the lowest SAD cost in the template region, such as... Figure 9 As shown. In other words, templates can be used to generate candidate lists. Figure 9 Template 1120 is shown, represented by the current block 1110 and two template parts 1120a and 1120b.

[0130] For each possible combination of a segmentation mode and two intra-prediction modes, a prediction is generated for template 1120, and the blending weights for the current block are extended toward template 1120. Alternatively, each template region can utilize a single value as a blending weight based on the segmentation boundary. For example, values ​​0 or 1 can be used as weights for each template region. The aforementioned combinations are ordered in ascending order according to the sum of absolute differences (SAD) cost between the template's prediction and reconstruction. The size of the candidate list can be set to, for example, 16, and these candidates are considered to be the most probable SGPM combinations for the current block. The video encoding and decoding devices construct the same candidate list using template 1120. The video encoding device selects the best candidate and signals the index of the selected candidate to the video decoding device.

[0131] To reduce the complexity of generating the candidate list, both the number of available segmentation modes and the number of available intra-prediction modes can be limited. For example, in the SGPM used in ECM, 26 segmentation modes and 9 intra-prediction modes are used to form a combination. The 9 intra-prediction modes can include... Figure 3aSome of the exemplified conventional intra-frame prediction modes (e.g., directional modes parallel to the segmentation boundary, directional modes perpendicular to the segmentation boundary, and planar modes), and further include block vector-based prediction modes obtained from neighboring blocks (e.g., IntraTMP mode or IBC mode).

[0132] I-6. Merged Intra Mode Derivation (MIMD) technique The merged intra-frame mode derivation mode uses information extracted from neighboring blocks to derive the merged histogram of gradient (MHoG).

[0133] The MIMD mode utilizes N neighboring blocks (where N is a positive integer greater than or equal to 2) predicted according to at least one directional intra-prediction mode. If a neighboring block i is predicted according to DIMD or MIMD, then the HoG or MHoG of that block is considered to be Hi. Hi(m) represents the magnitude corresponding to the directional prediction mode m. If a neighboring block i is predicted based on the directional intra-prediction mode m, then Hi(m) is set to be equal to A, and is set to satisfy Hi(k) = 0, k ≠ i. A can be a value that depends on the size of the current block. If a neighboring block i is predicted by SGPM or TIMD and one or more intra-prediction modes are available, then the corresponding intra-prediction mode can be considered in the derivation of Hi.

[0134] The MIMD pattern can be used to compute MHoG as shown in Equation 5 by utilizing Hi extracted from neighboring blocks as described above.

[0135] [Equation 5] The MIMD mode can be derived based on the intra prediction modes with the highest amplitude in the MHoG (e.g., up to 5).

[0136] I-7. Based on emerging Occurrence-Based Intra Coding (OBIC) technology OBIC technology derives the intra-prediction mode (HoC) on the decoder side based on neighboring reference sample information of the current block, and combines the HoC with existing prediction modes (planar or non-directional prediction based on block vectors) to generate the final prediction block. The video decoding device can generate the HoC by counting the frequencies of intra-prediction modes used to predict neighboring reference samples on a sample-by-sample basis. The video decoding device predefines neighboring block regions that are directly or not directly adjacent to the current block, collects the intra-prediction modes for the corresponding blocks, and constructs the HoC based on the occurrence frequency of the intra-prediction modes. The occurrence frequency is calculated based on the number of samples encoded with a specific intra-prediction mode. For example, if a block ui with width uiWidth and height uiHeight is encoded using a specific intra-prediction mode (IPM), the HoC[IPM] is calculated as shown in Equation 6.

[0137] [Equation 6] HoC[IPM] += uiWidth × uiHeight In Equation 6, IPM can be... Figure 3a One of the 67 intra-frame prediction modes shown.

[0138] The video decoding apparatus selects five IPMs based on the order of HoC modes. The apparatus generates prediction blocks based on the five selected prediction modes and produces a final prediction block by weighted summation of the generated prediction blocks. Similar to DIMD, to form the final prediction block, a weighted summation of the prediction values ​​of the intra-prediction modes determined by HoC can be performed using non-directional prediction values ​​(based on planes or block vectors). Alternatively, the weights associated with each prediction mode can be set based on the HoC values ​​in ascending order of HoC.

[0139] The video decoding device applies HoC-based intra-prediction mode only to luma blocks. HoC-based intra-prediction mode is not applied to blocks containing fewer than 64 samples. The HoC-based intra-prediction mode is signaled using PU-level flags. The HoC-based intra-prediction mode can be used as a sub-mode of DIMD, and the corresponding flag can be transmitted after the DIMD flag.

[0140] I-8. Matrix-based Intra-Injection Prediction (MIP) Matrix-based intra prediction (MIP) derives a replacement prediction sample for the existing intra prediction by multiplying a reference pixel by a weight matrix. The process of deriving the prediction sample can be represented as shown in Equation 7.

[0141] [Equation 7] In Equation 7, the predicted sample P(x,y) is calculated by multiplying the reference pixel r(k) by the weight matrix F(x,y,k). k represents the index of the reference pixel.

[0142] This set of reference pixels is called the reference template. A typical MIP reference template can be like this: Figure 10 As illustrated, the reference template corresponds to the input of the MIP. The reference template includes a reconstructed 4-line sample (T2=4) adjacent to the left side of the current block and a reconstructed 4-line sample (T1=4) adjacent to the top side of the current block, and has an L-shape. The width and height of the reference template have values ​​of 2W and 2H, respectively. Here, W represents the width of the current block, H represents the height of the current block, and r represents the reference pixel or reference template.

[0143] The MIP is activated for a block of a specified size or smaller associated with a preset intra-prediction mode (e.g., intra-prediction modes 0, 1, 2, 6, 10, ...). When the MIP is activated, the intra-prediction according to the corresponding intra-prediction mode is replaced by the MIP. Therefore, when the MIP is applied, no additional information other than the intra-prediction mode information is emitted by the signal.

[0144] The following embodiments are described with respect to a video decoding apparatus, but can be implemented in an equivalent or similar manner in a video encoding apparatus. Alternatively, the following embodiments are described with respect to the decoder side of a video decoding apparatus, but can be implemented in an equivalent or similar manner in a video encoding apparatus.

[0145] II. Embodiments according to the present invention As an embodiment of the present invention, matrix-based intra-frame prediction utilizing multiple candidates is described.

[0146] In this embodiment, the video decoding device performs matrix-based intra-frame prediction (MIP) with multiple candidates. MIP candidates can be derived in various ways based on the location of reference pixels, the characteristics of reference pixels, the shape of reference pixels (e.g., a set of reference pixels or a reference template), the type of weight matrix, etc.

[0147] As an example, pixel-level values ​​can be used as characteristics of a reference pixel. A video decoding device can divide the reference pixel into N groups based on predetermined reference level values, and can derive MIP candidates for each of the N groups. As another example, primary / secondary pixels can be used as characteristics of a reference pixel. When interpolation is applied to the reference pixel, the video decoding device can derive one MIP candidate using only the pixels at the primary pixel position in the reference pixel, derive another MIP candidate using only the pixels at the secondary pixel position, and derive yet another MIP candidate based on the reference pixel corresponding to the primary + secondary pixels.

[0148] A video decoding device can construct a candidate list of MIPs by executing candidate derivation methods in a predetermined order. For example, when constructing each candidate list in the order of three methods A, B, and C, the video decoding device can generate X MIP candidates (A1, A2, ..., AX) based on method A, Y MIP candidates (B1, B2, ..., BY) based on method B, and Z MIP candidates (C1, C2, ..., CZ) based on method C. Table 1 illustrates candidate lists constructed according to the above methods. The method for constructing the MIP candidate list can be predefined according to the protocol between the video encoding device and the video decoding device.

[0149] [Table 1] As an example, MIP candidate information for the current block can be explicitly signaled. When there are four MIP candidates, the video encoder can signal a 2-bit encoded MIP index using fixed-length code (FLC). When constructing the front of the MIP candidate list based on a more efficient method or a method for deriving candidates that are likely to be frequently selected, the video encoder can signal MIP indices encoded in variable-length code (VLC). Accordingly, fewer bits can be allocated to candidates located earlier in the candidate list.

[0150] As another embodiment of the invention, matrix-based intra-frame prediction utilizing multiple weight matrices is described.

[0151] In this embodiment, the video decoding device utilizes multiple weight matrices to perform MIP. As an example, the video decoding device can use M weight matrices based on the positions of reference pixels in the MIP. The M weight matrices can form a candidate list as described above. For example, a total of three weight matrices can be used, such as a weight matrix for L-shaped reference pixels, a weight matrix for the left reference pixels, and a weight matrix for the top reference pixels. Subsequently, the video decoding device can implicitly use the corresponding weight matrices according to the reference pixel selection method, or explicitly use the weight matrix information emitted by the signal (hereinafter referred to as weightMatrixIdx).

[0152] Alternatively, the video decoding device can use N weight matrices based on reference values ​​based on neighboring block information. The reference values ​​for neighboring block information can be the average of reference pixel-level values, the median of reference pixel-level values, or directional information of neighboring blocks of the current block (e.g., intra-prediction mode, gradient, etc.).

[0153] For example, if the reference value is the average value of the reference pixel level values, the range of the level values is divided into three intervals, and two thresholds (th1, th2) for distinguishing the three intervals may be predefined according to the agreement between the video encoding apparatus and the video decoding apparatus. The reference value and the two thresholds may be utilized as shown in Equation 8.

[0154] [Equation 8] In Equation 8, if the average value avglev satisfies 0≤avglev<th1, the video decoding apparatus uses the first weight matrix (weightMatrixIdx=0). If the average value satisfies th1≤avglev<th2, the video decoding apparatus uses the second weight matrix (weightMatrixIdx=1). If the average value satisfies th2≤avglev, the video decoding apparatus may use the third weight matrix (weightMatrixIdx=2). In the above method, since the weight matrix is determined according to the reference value based on adjacent block information, information indicating the weight matrix (hereinafter, referred to as weight matrix information) is not signaled.

[0155] As another example, when a plurality of weight matrices are used, the weight matrix information may be explicitly signaled. For example, a plurality of weight matrices may be constructed regardless of the intra prediction mode. In the case of an intra prediction mode to which MIP is applied, the video encoding apparatus may perform prediction of the current block based on K weight matrices, and then signal the weight matrix information (e.g., an index) that provides the minimum cost value according to a predetermined cost function. By utilizing the signaled information, the video decoding apparatus may verify the weight matrix among the K weight matrices actually used for intra prediction.

[0156] As another example, a plurality of different weight matrices may be defined for each intra prediction mode. That is, K may be defined for the intra prediction mode number A to which MIP is applied A weight matrices, and K may be defined for the mode number B B weight matrices. In this case, K A weight matrices and K B weight matrices are different matrices. K A and K B may be the same value or different values.

[0157] As another example, a plurality of different weight matrices may be defined according to the size of the current block. For example, defining different weight matrices according to the block size may be performed, such as defining K for 4×4 blocks to which MIP is applied 4×4 weight matrices, and defining K for 4×16 blocks 4×16Weight matrix. K 4×4 and K 4×16 It can be the same value or different values.

[0158] As another embodiment of the invention, the selection of separable / non-separable transform kernels in matrix-based intra-frame prediction is described.

[0159] In conventional MIP technology, when determining the separable / non-separable transform set / transform kernel from the separable / non-separable transform set, the video decoding device uses information from the intra prediction mode replaced by the MIP. However, a problem may arise where the characteristics of the residual remaining after performing MIP prediction may differ from those of the residual based on conventional intra prediction. Accordingly, in this embodiment, instead of the intra prediction mode of the current block, the video decoding device derives / specifies a new virtual intra prediction mode (VIPM) for selecting the separable / non-separable transform kernel of the MIP.

[0160] Separable transforms apply transform sets / kernels separately in the horizontal and vertical directions of the transform block. Non-separable transforms transform the transform block into a one-dimensional vector and then apply a single transform set / kernel. An example of a non-separable transform is the low-frequency non-separable transform (LFNST), which is used for quadratic transforms and can be applied after a first-order transform that is a separable transform.

[0161] As an embodiment of the present invention, it is described that a virtual intra-frame prediction mode is derived by utilizing the gradients of neighboring / non-neighboring samples.

[0162] In this embodiment, the video decoding device derives a virtual intra-frame prediction mode for determining separable / non-separable transform sets / kernels using gradients from neighboring / non-neighboring samples. Here, non-neighboring samples refer to samples in regions defined by a protocol between the video encoding and decoding devices, or samples in any region indicated by additional information from the current block.

[0163] As an example, a video decoding device computes all gradients (or slopes) of neighboring / non-neighboring samples by applying boundary detection filters (such as Sobel filters, Roberts cross filters, Prewitt filters, Scharr filters, and Laplacian filters) to samples neighboring / non-neighboring the current block, and generates a Histogram of Gradients (HoG) based on these gradients. The HoG can be generated by accumulating the magnitude (i.e., amplitude) of the gradients associated with their directions. The video decoding device can select the gradient with the maximum amplitude from the gradient histogram, map the direction of the selected gradient to an intra-prediction mode, and determine the mapped intra-prediction mode as the virtual intra-prediction mode for the current block. Here, mapping is matching or approximating the direction of the gradient to, for example... Figure 3a The process of one type of intra-frame prediction mode is shown.

[0164] As another example, a fixed number of gradients are selected from the gradient histogram in any order or in order of maximum magnitude, and the direction of each selected gradient is mapped to an intra-prediction mode, such that the mapped prediction mode can be used as a candidate for the virtual intra-prediction mode of the current block. The video coding device can perform encoding (e.g., prediction / transformation) on the candidate modes (i.e., intra-prediction mode information and virtual intra-prediction mode of the current block), and then determine the candidate mode that is best in terms of rate distortion or has the minimum cost value for a defined cost function as the virtual intra-prediction mode of the current block. The video coding device can signal relevant information to the video decoding device (e.g., whether to utilize the virtual intra-prediction mode, transform set index / transform kernel index / virtual intra-prediction mode, etc.).

[0165] As another example, the video decoding device selects a predetermined number of gradients from the gradient histogram in any order or in order of maximum magnitude, and maps the direction of each selected gradient to an intra-prediction mode. Thereafter, the video decoding device can determine the intra-prediction mode with the smallest mode number or the intra-prediction mode with the largest mode number among the assigned intra-prediction modes as the virtual intra-prediction mode for the current block.

[0166] When a method is applied to derive a virtual intra-prediction mode using the gradients of neighboring / non-neighboring samples, if all magnitudes of the gradient histogram are less than or equal to a specific value, the video decoding device can determine the virtual intra-prediction mode of the current block as a planar mode, a DC mode, or an arbitrary predetermined mode.

[0167] As another embodiment of the invention, a virtual intra-frame prediction mode is described based on the gradient histogram of neighboring / non-neighboring blocks.

[0168] In this embodiment, the video decoding apparatus derives a virtual intra-frame prediction mode for determining separable / non-separable transform sets / kernels using gradient histograms of neighboring / non-neighboring blocks. Here, non-neighboring samples refer to samples in regions defined according to the protocol between the video encoding and decoding apparatus, or samples in any region indicated by additional information of the current block.

[0169] The gradient histogram used to determine the virtual intra-prediction mode of the current block can be generated as follows.

[0170] As an example, if the intra-prediction mode of the neighboring / non-neighboring block is a decoder-side intra-mode derivation (DIMD) mode or a merged intra-mode derivation (MIMD) mode, the video decoding device can use the current mode amplitude value as is or after normalization, derived from the Histogram of Gradients (HoG) or Merged Histogram of Gradients (MHoG) of the corresponding block.

[0171] As another example, if the intra-prediction mode of the neighboring / non-neighboring block is not a decoder-side intra-mode derivation mode (non-DIMD) or not a merged intra-mode derivation mode (non-MIMD), the video decoding device may use a value accumulated as a real multiple of any predetermined value (e.g., a real multiple of the number of pixels in the current block or a real multiple of the number of pixels in the neighboring / non-neighboring block) as the magnitude value of the intra-prediction mode of the corresponding block.

[0172] If the neighboring / non-neighboring blocks are blocks predicted according to a pattern having two or more intra-prediction modes (e.g., Spatial Geometry Partitioning (SGPM) or Template-Based Intra-Mode Derivation (TIMD) modes), the video decoding apparatus can use a value accumulated as an arbitrary fixed real multiple (e.g., a real multiple of the number of pixels in the current block or a real multiple of the number of pixels in the neighboring / non-neighboring blocks) as the magnitude value for each intra-prediction mode.

[0173] After deriving multiple gradient histograms using neighboring / non-neighboring blocks, the video decoding device can use the accumulated histogram of the multiple gradient histograms as is, and use it as the gradient histogram / merged gradient histogram of the current block. Alternatively, the video decoding device can generate the gradient histogram / merged gradient histogram of the current block by combining / normalizing the accumulated histograms using calculations such as averaging.

[0174] The video decoding device can select the gradient with the largest magnitude from the gradient histogram / merged gradient histogram generated based on information from neighboring / non-neighboring blocks, map the direction of the selected gradient to an intra-prediction mode, and determine the mapped intra-prediction mode as the virtual intra-prediction mode for the current block.

[0175] As another example, the video coding device can select a fixed number of gradients from the gradient histogram / merged gradient histogram in any order or in order of maximum magnitude. The video coding device can then map the direction of each selected gradient to an intra-prediction mode and use the mapped prediction modes as candidates for the virtual intra-prediction mode of the current block. The video coding device can perform encoding (e.g., prediction / transformation) on the candidate modes (i.e., intra-prediction mode information for the current block and the virtual intra-prediction mode), and then determine the candidate mode that is best in terms of rate-distortion or has the minimum cost value for a defined cost function as the virtual intra-prediction mode for the current block. The video coding device can signal relevant information (e.g., whether to utilize the virtual intra-prediction mode, transform set index / transform kernel index / virtual intra-prediction mode, etc.) to the video decoding device.

[0176] As another example, the video decoding device selects a predetermined number of gradients from the gradient histogram / merged gradient histogram in any order or in order of maximum magnitude, and maps the direction of each selected gradient to an intra-prediction mode. Thereafter, the video decoding device can determine the intra-prediction mode with the smallest mode number or the intra-prediction mode with the largest mode number among the assigned intra-prediction modes as the virtual intra-prediction mode for the current block.

[0177] When a method is applied to derive a virtual intra-prediction mode using gradient histograms / merged gradient histograms of neighboring / non-neighboring samples, if all magnitudes of the gradient histogram / merged gradient histogram are less than or equal to a specific value, the video decoding device can determine the virtual intra-prediction mode of the current block as planar mode, DC mode, or any predetermined mode.

[0178] As another embodiment of the invention, a virtual intra-frame prediction mode is described based on the occurrence histogram of neighboring / non-neighboring blocks.

[0179] In this embodiment, the video decoding apparatus derives a virtual intra-frame prediction mode for determining separable and non-separable transform sets and kernels using the occurrence histogram (HoC) of neighboring / non-neighboring blocks. Here, non-neighboring samples refer to samples in regions defined according to the protocol between the video encoding apparatus and the video decoding apparatus, or samples in any region indicated by additional information of the current block.

[0180] The occurrence histogram for determining the virtual intra-prediction mode of the current block can be generated as follows.

[0181] As an example, a video decoding device can construct a histogram of intra-prediction patterns for neighboring / non-neighboring blocks, and can use the constructed histogram as an occurrence histogram.

[0182] As another example, the video decoding device can accumulate the amplitude histogram of the frequency corresponding to the intra-prediction mode of the neighboring / non-neighboring blocks by any fixed real multiple (e.g., a real multiple of the number of pixels in the current block or a real multiple of the number of pixels in the neighboring / non-neighboring blocks), and can use the accumulated histogram as the occurrence histogram.

[0183] As another example, if the neighboring / non-neighboring blocks are blocks with two or more intra-prediction modes (e.g., SGPM, TIMD, and DIMD), the video decoding device can use a value accumulated as a real multiple of any fixed value (e.g., a real multiple of the number of pixels in the current block or a real multiple of the number of pixels in the neighboring / non-neighboring blocks) as the amplitude value for each intra-prediction mode.

[0184] The video decoding device can determine the intra-prediction mode with the largest amplitude value as the virtual intra-prediction mode for the current block from the occurrence histogram generated based on the intra-prediction modes of neighboring / non-neighboring blocks.

[0185] As another example, a number of intra-prediction modes determined from the histogram of occurrence, either in any order or in the order of maximum magnitude, can be used as candidates for the virtual intra-prediction mode of the current block. The video coding device performs encoding (e.g., prediction / transformation) on the candidate modes (i.e., the intra-prediction mode information for the current block and the virtual intra-prediction mode). Thereafter, the video coding device can determine the candidate mode that is best in terms of rate-distortion or has the minimum cost value for a given cost function as the virtual intra-prediction mode for the current block. The video coding device can signal relevant information (e.g., whether to utilize the virtual intra-prediction mode, transform set index / transform kernel index / virtual intra-prediction mode, etc.) to the video decoding device.

[0186] As another example, the video decoding device selects a predetermined number of intra-prediction modes from the occurrence histogram in any order or in the order of maximum magnitude. The video decoding device can then determine the intra-prediction mode with the smallest mode number or the intra-prediction mode with the largest mode number among the selected intra-prediction modes as the virtual intra-prediction mode for the current block.

[0187] When a method is applied to derive a virtual intra-frame prediction mode using the occurrence histogram of neighboring / non-neighboring samples, if all amplitudes of the occurrence histogram are less than or equal to a specific value, the video decoding device can determine the virtual intra-frame prediction mode of the current block as a planar mode, a DC mode, or an arbitrary predetermined mode.

[0188] As another embodiment of the invention, the specification of an arbitrary intra-frame prediction mode is described.

[0189] In this embodiment, the video decoding apparatus designates an arbitrary intra-prediction mode as a virtual intra-prediction mode for determining the separable / non-separable transform set / kernel. The higher the prediction accuracy of the MIP, the more random the residuals after prediction may behave. That is, the directionality of the current block is largely removed, and therefore a specific directionality may not appear. Therefore, an arbitrary intra-prediction mode is used to determine the separable / non-separable transform set / kernel, and information related to the arbitrary intra-prediction mode can be pre-agreed between the video encoding and decoding apparatus. For example, if the intra-prediction mode is replaced with a MIP, the video decoding apparatus can use a planar mode to determine the separable / non-separable transform set / kernel.

[0190] In the following text, see references Figure 11 and Figure 12 This describes a method for predicting / transforming the current block using matrix-based intra-frame prediction.

[0191] Figure 11 This is a flowchart illustrating a method for encoding the current block performed by a video encoding apparatus according to an embodiment of the present invention.

[0192] If the current block is predicted in an intra-prediction mode associated with matrix-based intra-prediction (MIP) applications, the video coding apparatus may perform the following steps.

[0193] The video encoding device obtains a weight matrix (S1100) for applying the MIP to the current block.

[0194] As an example, a video coding apparatus can generate a list of candidate weight matrices based on the location of reference pixels, the shape of a group of reference pixels, or the type of the weight matrix. The video coding apparatus can then derive the weight matrix from the list. Subsequently, the video coding apparatus encodes the indices indicating the weight matrix.

[0195] As another example, the video coding apparatus can implicitly determine the weight matrix. The video coding apparatus can calculate reference values ​​based on neighboring block information. Here, the reference values ​​can be the average value of the reference pixel-level values, the median value of the reference pixel-level values, the orientation information of the current block's neighboring blocks, etc. The video coding apparatus can compare the calculated reference values ​​with a preset threshold and determine the weight matrix based on the comparison result.

[0196] The video coding device generates a prediction block for the current block by applying a weight matrix to the reference pixels of the current block (S1102).

[0197] The video coding device derives the virtual intra-frame prediction mode based on the neighboring / non-neighboring samples of the current block (S1104).

[0198] As an example, a video coding apparatus can compute gradients by applying boundary detection filters to neighboring / non-neighboring samples and use these gradients to generate a gradient histogram. The video coding apparatus can select the gradient with the maximum magnitude from the gradient histogram, map the direction of the selected gradient to an intra-prediction mode, and determine the mapped intra-prediction mode as a virtual intra-prediction mode. Here, the mapping is matching or approximating the direction of the gradient to, for example... Figure 3a The process of one type of intra-frame prediction mode is shown.

[0199] As another example, the video coding apparatus can obtain a flag from a higher layer indicating whether to utilize a virtual intra-prediction mode. If the flag is verified and found to be true, thus determining that a virtual intra-prediction mode should be utilized, the video coding apparatus can determine the virtual intra-prediction mode as follows.

[0200] The video coding apparatus can compute gradients by applying boundary detection filters to neighboring / non-neighboring samples and can use these gradients to generate a gradient histogram. The video coding apparatus can generate candidate modes by selecting a predetermined number of gradients from the gradient histogram and mapping the directions of the selected gradients to intra-prediction modes. The video coding apparatus can determine a virtual intra-prediction mode based on the candidate modes, either for rate-distortion optimization or optimization of a predetermined cost function. Subsequently, the video coding apparatus can encode the determined virtual intra-prediction mode or encode information indicating the determined virtual intra-prediction mode.

[0201] As another example, if the intra-prediction mode of neighboring / non-neighboring blocks is DIMD or MIMD mode, the virtual intra-prediction mode can be derived as follows. The video coding apparatus can generate an accumulated histogram by combining the HoG or MHoG of neighboring / non-neighboring blocks. The video coding apparatus can select the gradient with the largest magnitude from the accumulated histogram, map the direction of the selected gradient to an intra-prediction mode, and determine the mapped intra-prediction mode as the virtual intra-prediction mode.

[0202] As another example, regarding neighboring / non-neighboring blocks, the video coding apparatus can generate a histogram by accumulating the frequency of each intra-prediction mode. The video coding apparatus can select the gradient with the largest amplitude from the histogram, map the direction of the selected gradient to an intra-prediction mode, and determine the mapped intra-prediction mode as a virtual intra-prediction mode.

[0203] As another example, regarding neighboring / non-neighboring blocks, the video coding apparatus can generate a histogram using values ​​accumulated as real multiples of the number of pixels in the neighboring / non-neighboring blocks as the magnitude value for each intra-prediction mode. The video coding apparatus can select the gradient with the maximum magnitude from the histogram, map the direction of the selected gradient to an intra-prediction mode, and determine the mapped intra-prediction mode as a virtual intra-prediction mode.

[0204] As another example, if neighboring / non-neighboring blocks have two or more intra-prediction modes, the virtual intra-prediction mode can be derived as follows. Regarding neighboring / non-neighboring blocks, the video coding apparatus can generate a histogram using values ​​accumulated as real multiples of the number of pixels in the neighboring / non-neighboring blocks as the magnitude value for each intra-prediction mode. The video coding apparatus can select the gradient with the largest magnitude from the histogram, map the direction of the selected gradient to an intra-prediction mode, and determine the mapped intra-prediction mode as the virtual intra-prediction mode.

[0205] The video coding device selects the separable / non-separable transform kernel for the current block based on the virtual intra-frame prediction mode (S1106).

[0206] The video encoding device generates a residual block for the current block by subtracting the prediction block from the current block (S1108).

[0207] The video coding device generates transform coefficients by applying the separable / non-separable transform kernel of the current block to the residual block (S1110).

[0208] Subsequently, the video encoding device can generate quantized transform coefficients by quantizing the transform coefficients, and can encode the quantized transform coefficients.

[0209] Figure 12This is a flowchart illustrating a method for reconstructing the current block performed by a video decoding device according to an embodiment of the present invention.

[0210] If the current block is predicted in the intra-prediction mode corresponding to the application of MIP, the video decoding device may perform the following steps.

[0211] The video decoding device obtains a weight matrix (S1200) for applying the MIP to the current block.

[0212] As an example, a video decoding device can decode indices indicating weight matrices from a bitstream. The video decoding device can generate a list of candidate weight matrices based on the location of reference pixels, the shape of a group of reference pixels, or the type of the weight matrix. The video decoding device can then derive the weight matrix from the list using the indices.

[0213] As another example, the video decoding device can implicitly determine the weight matrix. The video decoding device can calculate reference values ​​based on neighboring block information. Here, the reference values ​​can be the average value of the reference pixel-level values, the median value of the reference pixel-level values, the orientation information of the neighboring blocks of the current block, etc. The video decoding device can compare the calculated reference values ​​with a preset threshold and determine the weight matrix based on the comparison result.

[0214] The video decoding device generates a prediction block for the current block by applying a weight matrix to the reference pixels of the current block (S1202).

[0215] The video decoding device derives the virtual intra-frame prediction mode based on the neighboring / non-neighboring samples of the current block (S1204).

[0216] As an example, a video decoding device can compute gradients by applying boundary detection filters to neighboring / non-neighboring samples, and can use these gradients to generate a gradient histogram. The video decoding device can select the gradient with the maximum magnitude from the gradient histogram, map the direction of the selected gradient to an intra-prediction mode, and determine the mapped intra-prediction mode as a virtual intra-prediction mode. Here, the mapping is matching or approximating the direction of the gradient to, for example... Figure 3a The process of one type of intra-frame prediction mode is shown.

[0217] As another example, a video decoding device can decode a flag from the bitstream indicating whether to utilize a virtual intra-prediction mode. If the flag is verified and found to be true, thus determining that a virtual intra-prediction mode should be utilized, the video decoding device can decode the virtual intra-prediction mode from the bitstream or decode information indicating the virtual intra-prediction mode.

[0218] As another example, if the intra-prediction mode of the neighboring / non-neighboring blocks is DIMD or MIMD mode, the virtual intra-prediction mode can be derived as follows. The video decoding device can generate an accumulated histogram by combining the HoG or MHoG of the neighboring / non-neighboring blocks. The video decoding device can select the gradient with the largest magnitude from the accumulated histogram, map the direction of the selected gradient to the intra-prediction mode, and determine the mapped intra-prediction mode as the virtual intra-prediction mode.

[0219] As another example, regarding neighboring / non-neighboring blocks, the video decoding device can generate a histogram by accumulating the frequency of each intra-prediction mode. The video decoding device can select the gradient with the largest amplitude from the histogram, map the direction of the selected gradient to an intra-prediction mode, and determine the mapped intra-prediction mode as a virtual intra-prediction mode.

[0220] As another example, regarding neighboring / non-neighboring blocks, the video decoding device can generate a histogram using values ​​accumulated as real multiples of the number of pixels in the neighboring / non-neighboring blocks as the magnitude value for each intra-prediction mode. The video decoding device can select the gradient with the maximum magnitude from the histogram, map the direction of the selected gradient to an intra-prediction mode, and determine the mapped intra-prediction mode as a virtual intra-prediction mode.

[0221] As another example, if neighboring / non-neighboring blocks have two or more intra-prediction modes, the virtual intra-prediction mode can be derived as follows. Regarding neighboring / non-neighboring blocks, the video decoding device can generate a histogram using values ​​accumulated as real multiples of the number of pixels in the neighboring / non-neighboring blocks as the magnitude value for each intra-prediction mode. The video decoding device can select the gradient with the largest magnitude from the histogram, map the direction of the selected gradient to an intra-prediction mode, and determine the mapped intra-prediction mode as the virtual intra-prediction mode.

[0222] The video decoding device selects the separable / non-separable inverse transform kernel for the current block based on the virtual intra-frame prediction mode (S1206).

[0223] The video decoding device reconstructs the transform coefficients of the current block from the bitstream (S1208). For example, the video decoding device can decode the quantized transform coefficients from the bitstream and can generate transform coefficients by inverse quantizing the quantized transform coefficients.

[0224] The video decoding device generates a residual block of the current block by applying the separable / non-separable inverse transform kernel of the current block to the transform coefficients (S1210).

[0225] Subsequently, the video decoding device can generate a reconstructed block for the current block by adding the predicted block and the residual block.

[0226] Although the steps in the various flowcharts are described in sequence, these steps only illustrate the technical ideas of some embodiments of the invention. Therefore, those skilled in the art to which this invention pertains can perform these steps by changing the order described in the various figures or by performing two or more steps in parallel. Thus, the steps in the various flowcharts are not limited to the chronological order shown.

[0227] It should be understood that the above description presents illustrative embodiments that can be implemented in various other ways. The functionality described in some embodiments can be implemented by hardware, software, firmware, and / or combinations thereof. It should also be understood that functional components described in this invention are designated as "...units" to emphasize their potential for independent implementation.

[0228] On the other hand, the various methods or functions described in some embodiments can be implemented as instructions stored in a non-volatile recording medium, which can be read and executed by one or more processors. The non-volatile recording medium can include various types of recording devices, such as those storing data in a computer system-readable form. For example, the non-volatile recording medium can include storage media such as erasable programmable read-only memory (EPROM), flash memory drives, optical disk drives, magnetic hard disk drives, and solid-state drives (SSDs).

[0229] Although embodiments of the invention have been described for illustrative purposes, those skilled in the art will understand that various modifications, additions, and substitutions can be made without departing from the spirit and scope of the invention. Therefore, embodiments of the invention have been described for the sake of brevity and clarity. The scope of the technical concept of the embodiments of the invention is not limited by the examples. Accordingly, those skilled in the art will understand that the scope of the invention should not be limited by the embodiments explicitly described above, but rather by the claims and their equivalents.

[0230] Cross-reference to related applications This application claims priority and benefit to Korean Patent Application No. 10-2024-0041903, filed on March 27, 2024, and Korean Patent Application No. 10-2025-0030905, filed on March 10, 2025, the entire contents of each of which are incorporated herein by reference.

Claims

1. A method for reconstructing a current block, performed by a video decoding device, the method comprising: Obtain the weight matrix used to apply matrix-based intra-frame prediction (MIP) to the current block; The predicted block for the current block is generated by applying a weight matrix to the reference pixels of the current block; The virtual intra-frame prediction mode is derived based on the neighboring / non-neighboring samples of the current block. Select the separable / non-separable inverse transform kernel for the current block based on the virtual intra-frame prediction mode; Reconstruct the transform coefficients of the current block from the bitstream; as well as The residual block of the current block is generated by applying the separable / non-separable inverse transform kernel of the current block to the transform coefficients.

2. The method according to claim 1, wherein, The weight matrix is ​​obtained by: Decode the index from the bitstream; Generate a list of candidate weight matrices based on the location of reference pixels, the shape of the reference pixel group, or the type of the weight matrix; and Derive the weight matrix from the list using indexes.

3. The method according to claim 1, wherein, The weight matrix is ​​obtained by: The reference value is calculated based on the reference pixel level value of the current block; and The weight matrix is ​​determined based on the comparison between reference values ​​and preset thresholds.

4. The method according to claim 1, wherein, The derivation of the virtual intra-frame prediction mode includes: The gradient is calculated by applying a boundary detection filter to neighboring / non-neighboring samples; Using gradients to generate gradient histograms; and The gradient with the largest magnitude is selected from the gradient histogram, the direction of the selected gradient is mapped to the intra-prediction mode, and the mapped intra-prediction mode is determined as the virtual intra-prediction mode.

5. The method according to claim 1, wherein, The derivation of the virtual intra-frame prediction mode includes: A flag indicating whether virtual intra-frame prediction mode is used from the bitstream decoding; and Verify the flag, and When deciding to utilize the virtual intra-frame prediction mode, deriving the virtual intra-frame prediction mode further includes decoding the virtual intra-frame prediction mode from the bitstream.

6. The method according to claim 1, wherein, When the intra-prediction mode of neighboring / non-neighboring blocks is a decoder-side intra-prediction mode or a merged intra-prediction mode, the derivation of the virtual intra-prediction mode includes: A cumulative histogram is generated by combining gradient histograms of neighboring / non-neighboring blocks or merging gradient histograms; and The gradient with the largest magnitude is selected from the cumulative histogram, the direction of the selected gradient is mapped to the intra-prediction mode, and the mapped intra-prediction mode is determined as the virtual intra-prediction mode.

7. The method according to claim 1, wherein, The derivation of the virtual intra-frame prediction mode includes: For neighboring / non-neighboring blocks, a histogram is generated by accumulating the frequencies of each intra-frame prediction mode; and Select the gradient with the largest magnitude from the histogram, map the direction of the selected gradient to the intra-prediction mode, and determine the mapped intra-prediction mode as the virtual intra-prediction mode.

8. The method according to claim 1, wherein, The derivation of the virtual intra-frame prediction mode includes: For neighboring / non-neighboring blocks, a histogram is generated using the accumulated values, which are real multiples of the number of pixels in the neighboring / non-neighboring blocks, as the magnitude values ​​for each intra-frame prediction mode; and Select the gradient with the largest magnitude from the histogram, map the direction of the selected gradient to the intra-prediction mode, and determine the mapped intra-prediction mode as the virtual intra-prediction mode.

9. The method according to claim 1, wherein, When neighboring / non-neighboring blocks have two or more intra-prediction modes, deriving the virtual intra-prediction mode includes: For neighboring / non-neighboring blocks, a histogram is generated using the accumulated values, which are real multiples of the number of pixels in the neighboring / non-neighboring blocks, as the magnitude values ​​for each intra-frame prediction mode; and Select the gradient with the largest magnitude from the histogram, map the direction of the selected gradient to the intra-prediction mode, and determine the mapped intra-prediction mode as the virtual intra-prediction mode.

10. A method for encoding a current block, performed by a video encoding apparatus, the method comprising: Obtain the weight matrix used to apply matrix-based intra-frame prediction (MIP) to the current block; The predicted block for the current block is generated by applying a weight matrix to the reference pixels of the current block; The virtual intra-frame prediction mode is derived based on the neighboring / non-neighboring samples of the current block. Select the separable / non-separable transform kernel for the current block based on the virtual intra-frame prediction mode; The residual block of the current block is generated by subtracting the predicted block from the current block; as well as Transformation coefficients are generated by applying the separable / non-separable kernel of the current block to the residual block.

11. The method according to claim 10, wherein, The weight matrix is ​​obtained by: Generate a list of candidate weight matrices based on the location of the reference pixels, the shape of the reference pixel group, or the type of the weight matrix; Deriving the weight matrix from the list; and The indices of the indicator weight matrix are encoded.

12. The method according to claim 10, wherein, The derivation of the virtual intra-frame prediction mode includes: The gradient is calculated by applying a boundary detection filter to neighboring / non-neighboring samples; Using gradients to generate gradient histograms; and The gradient with the largest magnitude is selected from the gradient histogram, the direction of the selected gradient is mapped to the intra-prediction mode, and the mapped intra-prediction mode is determined as the virtual intra-prediction mode.

13. The method according to claim 10, wherein, The derivation of the virtual intra-frame prediction mode includes: Obtain a flag from higher levels indicating whether to utilize virtual intra-frame prediction mode; and Verify the flag, and When deciding to use the virtual intra-frame prediction mode, deriving the virtual intra-frame prediction mode further includes: The gradient is calculated by applying a boundary detection filter to neighboring / non-neighboring samples; Use gradients to generate gradient histograms; A preset number of gradients are selected from the gradient histogram, and the directions of the selected gradients are mapped to intra-frame prediction modes to generate candidate modes; In rate-distortion optimization or optimization using a pre-defined cost function, the virtual intra-frame prediction mode is determined based on candidate modes; and Encode the virtual intra-frame prediction mode.

14. The method of claim 10, wherein, The derivation of the virtual intra-frame prediction mode includes: For neighboring / non-neighboring blocks, a histogram is generated by accumulating the frequencies of each intra-frame prediction mode; and Select the gradient with the largest magnitude from the histogram, map the direction of the selected gradient to the intra-prediction mode, and determine the mapped intra-prediction mode as the virtual intra-prediction mode.

15. A method for providing video data to a video decoding apparatus, the method comprising: Encode video data into a bitstream; as well as Send the bitstream to the video decoding device. The encoded video data includes: Obtain the weight matrix used to apply matrix-based intra-frame prediction (MIP) to the current block; The predicted block for the current block is generated by applying a weight matrix to the reference pixels of the current block; The virtual intra-frame prediction mode is derived based on the neighboring / non-neighboring samples of the current block. Select the separable / non-separable transform kernel for the current block based on the virtual intra-frame prediction mode; The residual block of the current block is generated by subtracting the predicted block from the current block; and Transformation coefficients are generated by applying the separable / non-separable kernel of the current block to the residual block.

Images