Video encoding method and device for setting reference region during matrix-based intra prediction, and video decoding method and device

Abstract

A video decoding method according to an embodiment comprises: obtaining flag information indicating whether MIP is applied and prediction mode information of a current block; when the flag information indicates that MIP is applied and a prediction mode of the current block is an MIP prediction mode, setting a reference region used for MIP on the basis of condition information of the current block; determining a matrix used for MIP; and obtaining a prediction sample of the current block on the basis of the reference region and the matrix.

Description

Video encoding method and device for setting a reference region during matrix-based intra-prediction, and video decoding method and device

[0001] The present disclosure relates to the field of video encoding and decoding. More specifically, the present disclosure relates to a method for establishing a reference region for matrix-based intra prediction, and to a video encoding, decoding method and apparatus utilizing such matrix-based intra prediction.

[0002] With the development and widespread adoption of hardware capable of playing and storing high-resolution or high-quality video content, the need for codecs that can effectively encode or decode such content is increasing. Recently, methods to effectively compress this high-resolution or high-quality video content are being implemented. Representative examples of codecs include HEVC (High Efficiency Video Coding) and VVC (Versatile Video Coding).

[0003] In HEVC, a single picture is divided into one or more tiles / slices and then further divided into multiple Coding Tree Units (CTUs). VVC, on the other hand, can first divide a single picture into multiple sub-pictures. A sub-picture is defined as a group of rectangular slices and was added to VVC to support the ability to partially and independently encode, decode, and transmit the picture. A single sub-picture can be divided into tiles / slices, similar to HEVC. Additionally, VVC introduces the "brick" as a new picture division structure. Bricks are created by dividing tiles horizontally and serve as the basic unit for parallel processing. To process higher resolution video than HEVC, VVC uses Coding Tree Units (CTUs) of up to 256x256, which is 16 times larger than those in HEVC, as the basic unit for encoding and decoding.

[0004] In the case of intra-block prediction, to eliminate redundancy within the screen, a prediction block is generated using restoration pixels adjacent to the current coding block, and a difference value is created between this and the current coding block. In the case of inter-block prediction, unlike intra-block prediction, the prediction block is generated by searching for the block most similar to the current coding block in the previous or subsequent frame.

[0005] Numerous tools have been proposed to improve the efficiency and performance of intra coding. For example, in addition to directional intra prediction modes using DC, Planar, and reference samples of a predetermined direction, various intra coding methods have been proposed, such as DIMD (Decoder Side Intra Mode Derivation), TIMD (Template based Intra Mode Derivation), EIP (Edge-based Intra Prediction), OBIC (Orientation-Based Intra Coding), MIP (Matrix based Intra Prediction), SGPM (Spatial Geometric Partitioning Mode), IntraTMP (Intra Template Matching Prediction), IBC (Intra Block Copy), MRL (Multiple Reference Lines), and ISP (Intra Sub-partitions).

[0006] In response to the demand for continuous coding improvement, it is necessary to enhance coding efficiency by improving various existing coding tools.

[0007] The present disclosure is intended to provide a video encoding / decoding method and apparatus that adaptively sets a surrounding reference pixel region used in matrix-based intra-prediction to improve coding efficiency.

[0008] The embodiments of the present disclosure disclose a video decoding method and apparatus, and a video encoding method and apparatus that can improve the prediction efficiency of MIP and thereby improve coding efficiency by variably setting the number of reference lines of a reference area used in MIP or the size of an extended reference area based on various additional information related to a current block, or by variably setting the number of reference lines of a reference area or the size of an extended reference area based on information regarding a reference area that is explicitly signaled, thereby allowing the reference area used in MIP to be set in various ways.

[0009] According to the embodiments of the present disclosure, video coding efficiency can be improved by efficiently determining the area of ​​reference pixels used to generate the predicted value of the current block according to the MIP mode, taking into account image characteristics.

[0010] Figure 1 is a block diagram illustrating the configuration and operation of a video encoder for encoding images.

[0011] FIG. 2 is a diagram illustrating an example of a method for dividing blocks of an image.

[0012] FIGS. 3 and FIGS. 4 are drawings for illustrating embodiments of an intra-prediction method.

[0013] Figure 5 is a block diagram illustrating the configuration and operation of a video decoder for decoding an image.

[0014] FIG. 6 is a diagram illustrating the process of generating a prediction sample of the current block according to the MIP mode according to one embodiment of the present invention.

[0015] FIGS. 7a and 7b illustrate an example of a process for generating a predicted sample of a current block according to an MIP mode according to an embodiment of the present invention.

[0016] FIG. 8 is a diagram illustrating an interpolation process performed in MIP mode in one embodiment of the present invention.

[0017] FIG. 9 is a diagram illustrating the process of configuring an MPM list according to an embodiment of the present invention.

[0018] FIGS. 10a and 10b are drawings for explaining a method of determining the cost when configuring an MPM candidate according to one embodiment of the present disclosure.

[0019] FIG. 11 is a drawing showing an example of a reference area used in MIP mode according to one embodiment of the present disclosure.

[0020] FIG. 12 is a drawing showing another example of a reference area used in MIP mode according to one embodiment of the present disclosure.

[0021] FIG. 13 is a drawing illustrating a reference mode that selectively applies surrounding pixels included in a reference area according to one embodiment of the present disclosure.

[0022] FIG. 14 is a flowchart of a video encoding method according to one embodiment of the present disclosure.

[0023] FIG. 15 is a flowchart of a video decoding method according to one embodiment of the present disclosure.

[0024] A video decoding method according to one embodiment of the present disclosure comprises: a step of obtaining flag information indicating whether Matrix-based Intra prediction (MIP) is applied and prediction mode information of a current block; a step of setting a reference region used for the MIP based on condition information of the current block including the size of the current block when the flag information indicates that the MIP is applied and the prediction mode of the current block determined based on the prediction mode information is the MIP prediction mode; a step of determining a matrix used for the MIP; and a step of obtaining a prediction sample of the current block based on the reference region and the matrix.

[0025] A video encoding method according to one embodiment of the present disclosure comprises: a step of performing intra prediction for a current block by applying a plurality of intra prediction modes, including an MIP, to the current block; a step of generating flag information indicating whether the MIP is applied to the current block; and a step of encoding the flag information and the intra prediction mode information of the current block, wherein the step of performing intra prediction for the current block includes: a step of setting a reference region used for the MIP based on condition information of the current block including the size and shape of the current block when performing MIP intra prediction for the current block; a step of determining a matrix used for the MIP; and a step of obtaining a prediction sample of the current block based on the reference region and the matrix.

[0026] In addition, embodiments of the present invention disclose a video decoding device and a decoding device for performing the aforementioned video decoding method and encoding method, and a computer-readable recording medium having a program for executing the video decoding method and encoding method on a computer.

[0027] Hereinafter, a video encoding, decoding method, and apparatus according to an embodiment of the present invention will be described in detail with reference to the attached drawings.

[0028] In the following description of the present invention, specific descriptions of related known functions or configurations will be omitted if it is determined that such detailed descriptions could unnecessarily obscure the essence of the invention. Furthermore, the terms described below are defined in consideration of their functions within the present invention, and these definitions may vary depending on the intentions or practices of the user or operator. Therefore, their definitions should be based on the overall content of the present invention.

[0029] In addition, the preferred embodiments of the present invention described below will focus on explaining the functional configurations that must be additionally provided for the present invention, while omitting as much as possible the system functional configurations that are already provided in each system functional configuration or are ordinarily provided in the technical field to which the present invention belongs, in order to efficiently explain the technical components constituting the present invention.

[0030] If a person skilled in the art to which the present invention pertains can easily understand the function of a component that has been used in the past among the functional configurations that are omitted and not illustrated below, and can also clearly understand the relationship between the component that was omitted as above and the component added for the present invention.

[0031] In this specification, a device that encodes an image to generate a video signal bitstream is referred to as an encoding device, an encoding device, or an encoder, and a device that decodes a video signal bitstream to restore an image is referred to as a decoding device, a decoding device, or a decoder.

[0032] A pixel or pel refers to the smallest unit that constitutes an image, and the terms pixel and sample may be used interchangeably. Generally, a sample can represent a pixel or its value, and it may represent only the pixel or its value of the luminance component, or only the pixel or its value of the chroma component.

[0033] Furthermore, the term "unit" is used to refer to a basic unit of image processing or a specific location within a picture, representing an image region containing at least one of the luminance component and the chrominance component. Specifically, the term "unit" can be used as a concept encompassing the Coding Tree Unit (CTU), Coding Unit (CU), Prediction Unit (PU), and Transform Unit (TU). Additionally, the term "block" represents an image region containing a specific component among the luminance and chrominance components, and an MxN block can represent a set of samples or transform coefficients consisting of M columns and N rows. Here, terms such as unit, block, partition, signal, and region may be used interchangeably.

[0034] Meanwhile, the term "picture" refers to a field or a frame, and these terms may be used interchangeably. For example, if the image is interlaced, a single frame is separated into an odd (or top) field and an even (or bottom) field, and each field is configured as a single picture unit for encoding or decoding. And if the image is sequential, a single frame is configured as a picture for encoding or decoding.

[0035] FIG. 1 is a block diagram illustrating an encoding device according to an embodiment of the present invention, and is intended to explain the configuration and operation of a video encoder for encoding images.

[0036] Referring to FIG. 1, the video encoder (100) may be configured to include a conversion unit (110), a quantization unit (120), an inverse quantization unit (130), an inverse conversion unit (140), a filtering unit (150), a prediction unit (160), a DPB (Decoded Picture Buffer, 170), and an entropy coding unit (180).

[0037] The conversion unit (110) converts the residual signal, which is the difference between the input video signal and the prediction signal generated by the prediction unit (160), to obtain a conversion coefficient value.

[0038] For example, the Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), or Wavelet Transform may be used.

[0039] The transformation kernel used for the transformation of a residual block may be a transformation kernel having separable characteristics of vertical transformation and horizontal transformation. In this case, the transformation of the residual block can be performed by separating it into a vertical transformation and a horizontal transformation. For example, an encoder may perform a vertical transformation by applying a transformation kernel in the vertical direction of the residual block. Additionally, an encoder may perform a horizontal transformation by applying a transformation kernel in the horizontal direction of the residual block.

[0040] Meanwhile, a transformation kernel can be used as a term referring to a set of parameters used for transforming a residual signal, such as a transformation matrix, transformation array, transformation function, or transformation; it can be any one of a plurality of available kernels, and transformation kernels based on different transformation types may be used for vertical transformation and horizontal transformation, respectively.

[0041] Transformation coefficients may be distributed such that higher coefficients are found towards the top-left corner of the block, and coefficients close to '0' are found towards the bottom-right corner. Additionally, as the current block size increases, there is a higher likelihood of '0' coefficients existing in the bottom-right region; to reduce the transformation complexity of large blocks, only an arbitrary top-left region may be retained, while the remaining regions may be reset to '0'.

[0042] In addition, an error signal may exist only in some areas of a coding block, in which case the conversion process may be performed only on some arbitrary areas. For example, in a block of size 2Nx2N, an error signal may exist only in the first 2NxN block, in which case the conversion process is performed only on the first 2NxN block, but the second 2NxN block may not be encoded or decoded without the conversion process being performed.

[0043] The encoder may perform additional transformations before the transformation coefficients are quantized. The transformation method described above is referred to as a primary transform, and the additional transformation may be referred to as a secondary transform.

[0044] The second-order conversion can be optional for each residual block. For example, the encoder can improve coding efficiency by performing a second-order conversion for regions where it is difficult to concentrate energy in the low-frequency region with only the first-order conversion.

[0045] Specifically, a second transformation may be additionally performed on blocks where residual values ​​appear significantly in directions other than the horizontal or vertical direction of the residual block, and unlike the first transformation, the second transformation may not be performed separately into a vertical transformation and a horizontal transformation. The second transformation described above may be referred to as a Low Frequency Non-Separable Transform (LFNST).

[0046] The quantization unit (120) quantizes the conversion coefficient value output from the conversion unit (110).

[0047] In order to increase coding efficiency, instead of coding the picture signal as is, a method is used to predict the picture using an area that has already been coded through a prediction unit (160), and to obtain a restored picture by adding the residual value between the original picture and the predicted picture to the predicted picture.

[0048] To prevent mismatches from occurring between the encoder and decoder, when performing prediction in the encoder, information available in the decoder must be used; to this end, the encoder may perform a process of restoring the currently encoded block.

[0049] The inverse quantization unit (130) inversely quantizes the conversion coefficient value, and the inverse conversion unit (140) restores the residual value using the inversely quantized conversion coefficient value.

[0050] The filtering unit (150) performs filtering operations using a deblocking filter, a Sample Adaptive Offset (SAO), an Adaptive Loop Filter (ALF), etc., to improve the quality of the restored picture and enhance encoding efficiency.

[0051] A deblocking filter is a filter for removing distortion within blocks generated at the boundaries between blocks in a restored picture, and the encoder can determine whether to apply the deblocking filter to a boundary based on the distribution of pixels included in a few columns or rows relative to an arbitrary boundary within a block.

[0052] When a deblocking filter is applied, the filtering unit (150) can apply a long filter, a strong filter, or a weak filter depending on the deblocking filtering strength, and can process horizontal filtering and vertical filtering in parallel.

[0053] Sample Adaptive Offset (SAO) can be used to correct the offset from the original image on a pixel-by-pixel basis for a residual block to which a deblocking filter has been applied. The filtering unit (150) may use a Band Offset method to correct the offset for a specific picture by dividing the pixels included in the image into a certain number of regions, determining the region to perform offset correction, and applying the offset to the region. Additionally, the filtering unit (150) may use an Edge Offset method to apply the offset by considering the edge information of each pixel.

[0054] An Adaptive Loop Filter (ALF) is a method that divides pixels contained in an image into specific groups, determines a single filter to apply to each group, and performs differential filtering for each group. Information regarding whether to apply an Adaptive Loop Filter can be signaled at the coding unit level, and the shape and filter coefficients of the ALF filter to be applied may vary depending on each block. Additionally, the same type of Adaptive Loop Filter may be applied regardless of the characteristics of the target block.

[0055] The filtered picture can be stored in DPB (170) to be used as a reference picture.

[0056] The prediction unit (160) includes an intra prediction unit (161) and an inter prediction unit (165), and the intra prediction unit (161) performs intra prediction within the current picture. Various intra prediction methods such as DIMD (Decoder Side Intra Mode Derivation), TIMD (Template based Intra Mode Derivation), EIP (Edge-based Intra Prediction), OBIC (Orientation-Based Intra Coding), MIP (Matrix based Intra Prediction), SGPM (Spatial Geometric Partitioning Mode), IntraTMP (Intra Template Matching Prediction), IBC (Intra Block Copy), MRL (Multiple Reference Lines), and ISP (Intra Sub-partitions) may be applied as intra prediction methods.

[0057] The inter prediction unit (165) performs an inter prediction that predicts the current picture using a reference picture stored in the DPB (170).

[0058] The intra prediction unit (161) performs intra prediction from the restored regions within the current picture and transmits intra encoding information to the entropy coding unit (180). Here, the intra encoding information may include an intra prediction mode, an MPM (Most Probable Mode) flag, an MPM index, information regarding a reference sample, and at least one of various parameter information required for intra prediction according to each intra prediction method.

[0059] The inter prediction unit (165) refers to a specific area of ​​the restored reference picture to find the part most similar to the current area and obtains a motion vector value which is the distance between the areas, and transmits motion information for the obtained reference area (reference direction indicator information (L0 prediction, L1 prediction, bidirectional prediction), reference picture index, motion vector information, etc.) to the entropy coding unit (180).

[0060] Additionally, the inter prediction unit (165) performs motion compensation using motion information to generate a prediction block for the current block, and transmits inter encoding information containing motion information for a reference area to the entropy coding unit (180).

[0061] Meanwhile, quantized transformation coefficients in the form of a two-dimensional array can be rearranged into a one-dimensional array for entropy coding.

[0062] The method for scanning quantized transform coefficients can be determined by the size of the transform block and the intra-prediction mode, and diagonal, vertical, or horizontal scans can be applied, and the scan information can be signaled in block units or derived from a decoder according to a defined rule.

[0063] The entropy coding unit (180) generates a bitstream by entropy coding information representing quantized conversion coefficients, intra-coding information, and inter-coding information, etc. For this purpose, a variable length coding (VLC) method and an arithmetic coding method may be used.

[0064] Variable Length Coding (VLC) converts input symbols into a sequence of codewords, the length of which can be variable. For example, frequently occurring symbols can be represented by short codewords, while infrequently occurring symbols can be represented by long codewords.

[0065] As a variable-length coding method, Context-based Adaptive Variable Length Coding (CAVLC) can be used.

[0066] Arithmetic coding utilizes the probability distribution of each data symbol to convert consecutive data symbols into a single prime number, thereby obtaining the optimal prime bit required to represent each symbol.

[0067] As an arithmetic coding method, the Context-based Adaptive Binary Arithmetic Code (CABAC) method can be used.

[0068] The generated bitstream is encapsulated with NAL (Network Abstraction Layer) units as the basic unit.

[0069] NAL units are classified into VCL (Video Coding Layer) NAL units containing video data and non-VCL NAL units containing parameter information for decoding video data, and various types of VCL or non-VCL NAL units may exist.

[0070] A NAL unit consists of NAL header information and data, which is a Raw Byte Sequence Payload (RBSP), and the NAL header information includes summary information about the RBSP. The RBSP of a VCL NAL unit contains encoded integer coding tree units.

[0071] In order to decode a bitstream in a decoder, the bitstream must first be separated into NAL units, and then each separated NAL unit must be decoded. Meanwhile, the information required for decoding the bitstream can be transmitted in the Picture Parameter Set (PPS), Sequence Parameter Set (SPS), Video Parameter Set (VPS), etc.

[0072] Meanwhile, the configuration and operation of the encoder described with reference to FIG. 1 are according to an embodiment of the present invention, and some configurations may be omitted or added as needed.

[0073] Additionally, a single picture can be divided and encoded into sub-pictures, slices, tiles, etc. A sub-picture may include one or more slices or tiles. If a single picture is divided and encoded into multiple slices or tiles, it can be displayed on the screen only after all slices or tiles within the picture have been fully decoded.

[0074] When a single picture is encoded into multiple sub-pictures, any single sub-picture may be decoded and displayed on the screen. A slice may include multiple tiles or sub-pictures, and a tile may include multiple sub-pictures or slices.

[0075] Sub-pictures, slices, and tiles can be encoded or decoded independently of each other, which is effective for parallel processing and improving processing speed; however, since encoded information from adjacent sub-pictures, slices, and tiles cannot be utilized, the bit size may increase.

[0076] And sub-pictures, slices, and tiles can be divided into multiple Coding Tree Units (CTUs) and encoded.

[0077] The coding tree unit may consist of a 128x128 luminance coding tree block (Coding Tree Block, CTB) and two 64x64 color difference coding tree blocks.

[0078] A single coding tree unit may not be divided and may constitute a single coding unit (CU) itself, or it may be divided into multiple coding units as shown in FIG. 2. A coding unit may consist of a luminance coding block (CB) and two color difference coding blocks.

[0079] A single coding unit may consist of a single transform unit (TU) or be divided into multiple transform units. A transform unit may consist of a luminance transform block (TB) and two chrominance transform blocks.

[0080] Here, a coding unit represents a basic unit for processing a picture during the prediction, transformation, quantization, entropy coding, and decoding processes, and the size and shape of the coding unit within a single picture may not be constant.

[0081] The coding unit may have a square or non-square shape, and the non-square coding unit may include a vertical coding unit in which the height is greater than the width and a horizontal coding unit in which the width is greater than the height.

[0082] The coding tree unit is first partitioned into a Quad Tree (QT) structure, so that a single node of size 2NX2N can be partitioned into four nodes of size NXN. Meanwhile, the Quad Tree partitioning can be performed recursively, and not all nodes need to be partitioned to the same depth.

[0083] The leaf nodes of a quad tree can be further divided into a Multi-Type Tree (MTT) structure. For example, in a Multi-Type Tree structure, a single node can be divided into a binary or ternary tree structure of horizontal or vertical splitting. Accordingly, there can be four types of splitting structures in a Multi-Type Tree structure: vertical binary splitting, horizontal binary splitting, vertical ternary splitting, and horizontal ternary splitting.

[0084] In each tree structure, the width and height of the nodes can both have powers of 2. For example, in a binary tree (BT) structure, a node of size 2NX2N can be divided into 2 NX2N nodes by a vertical binary split and into 2 2NXN nodes by a horizontal binary split.

[0085] In addition, in a ternary tree (TT) structure, a node of size 2NX2N can be divided into (N / 2)X2N, NX2N, and (N / 2)X2N nodes by vertical ternary partitioning, and into 2NX(N / 2), 2NXN, and 2NX(N / 2) nodes by horizontal ternary partitioning. This multi-type tree partitioning can be performed recursively.

[0086] Leaf nodes of a multi-type tree can serve as coding units, and if a coding unit is not larger than the maximum transformation length, it can be used as a unit of prediction and transformation without further splitting. On the other hand, if the width or height of a coding unit is greater than the maximum transformation length, it can be split into multiple transformation units without explicit signaling regarding splitting.

[0087] The tree partitioning structure described above may have the same form (Single Tree) for the luminance block and the chrominance block, or different forms (Dual Tree) for the luminance block and the chrominance block.

[0088] Meanwhile, regarding the block division from the coding tree unit (CTU) to the coding unit (CU) as described above, the final coding unit can be determined by selecting the division structure with the smallest rate-distortion cost (RD cost) value within the allowable size and depth conditions through the rate-distortion optimization (RDO) process.

[0089] Hereinafter, embodiments of an intra-prediction method will be described in more detail with reference to FIGS. 3 and FIGS. 4.

[0090] For intra prediction, intra prediction mode information may be signaled, and the intra prediction mode information may indicate any one of a plurality of intra prediction modes. The plurality of intra prediction modes may include various methods such as directional intra prediction mode as illustrated in FIG. 3, IntraTMP, IBC, DIMD, TIMD, OBIC, MIP, SGPM, MRL, ISP, etc. The video encoder (100) includes the intra prediction mode information applied to the current block in the bitstream and transmits it to the video decoder (200), and the video decoder (200) can determine the intra prediction mode of the current block by parsing the intra prediction information included in the bitstream.

[0091] For example, as illustrated in FIG. 3, the directional intra prediction mode may include a planar mode, a DC mode, and 65 directional modes, and each intra prediction mode may be indicated through an intra prediction mode index.

[0092] Intra-prediction mode index "0" indicates a planar mode, intra-prediction mode index "1" indicates a DC mode, and intra-prediction mode indices "2" through "66" can each indicate different directional modes.

[0093] Directional modes each indicate different angles within a preset angle range, for example, a directional mode can indicate an angle within an angle range between 45 degrees and -135 degrees clockwise.

[0094] In this case, the intra prediction mode index "2" indicates a horizontal diagonal (HDIA) mode, the intra prediction mode index "18" indicates a horizontal (HOR) mode, the intra prediction mode index "34" indicates a diagonal (DIA) mode, the intra prediction mode index "50" indicates a vertical (VER) mode, and the intra prediction mode index "66" indicates a vertical diagonal (VDIA) mode.

[0095] If the current block is a non-square block, 20 additional wide angular modes indicating angles greater than 45 degrees clockwise or less than -135 degrees may be used.

[0096] Based on the intra prediction mode information as described above, reference samples to be used for intra prediction for the current block are determined.

[0097] For example, if the intra prediction mode index indicates a specific directional mode, a reference sample corresponding to the angle from the current sample in the current block is used for prediction regarding the current sample. For intra prediction, surrounding already restored samples are used as reference samples, and the reference samples may be restored samples located to the left or above the current block.

[0098] Referring to FIG. 4, the reference samples may be samples adjacent to the left boundary and the upper boundary of the current block.

[0099] For example, if the current block size is NxN and samples from a single reference line adjacent to the current block are used for intra prediction, reference samples can be established using (2N*2+1) surrounding samples located to the left (L, Left), top (T, Top), and top-left (TL, Top-left) of the current block.

[0100] Meanwhile, samples of multiple reference lines (MRLs) may be used for intra prediction of the current block, and the multiple reference lines may consist of n reference lines located within a pre-set range from the current block. In this case, separate reference line index information indicating the reference lines to be set as reference pixels may be signaled.

[0101] In addition, if at least some of the samples to be used as reference samples have not yet been restored, reference samples can be obtained through a reference sample padding process, and a reference sample filtering process can be performed to reduce the error of intra-prediction.

[0102] FIG. 5 is a block diagram illustrating a decoding device according to an embodiment of the present invention, and is intended to explain the configuration and operation of a video decoder for decoding an image.

[0103] Referring to FIG. 5, the video decoder (200) may be configured to include an entropy decoding unit (210), an inverse quantization unit (220), an inverse transform unit (230), a filtering unit (240), a prediction unit (260), and a DPB (Decoded Picture Buffer, 270).

[0104] The entropy decoding unit (210) entropies decodes the bitstream to extract conversion coefficient information, intra-coding information, inter-coding information, etc. for each region.

[0105] For example, the entropy decoding unit (210) can obtain a binary code for conversion coefficient information of a specific region from the bitstream, and can obtain quantized conversion coefficients by debinding the binary code.

[0106] The inverse quantization unit (220) inversely quantizes the quantized transformation coefficients, and the inverse transformation unit (230) restores the residual value using the inversely quantized transformation coefficients. After performing a first-order inverse transformation on a transformation block containing the inversely quantized transformation coefficients, the inverse transformation unit (230) can additionally perform a second-order inverse transformation to obtain the residual value.

[0107] Meanwhile, the residual value obtained from the inverse conversion unit (230) is added to the predicted value obtained from the prediction unit (260) to restore the original pixel value.

[0108] The filtering unit (240) performs filtering operations using a deblocking filter, a sample adaptive offset, an adaptive loop filter, etc., to improve the quality of the restored picture, and the filtered picture can be stored in the DPB (270) to be output or used as a reference picture for the next picture.

[0109] The prediction unit (260) includes an intra prediction unit (261) and an inter prediction unit (265), and generates a prediction picture by utilizing the encoding type decoded through the entropy decoding unit (210), the conversion coefficient for each region, intra / inter encoding information, etc.

[0110] To restore the current block being decrypted, the decrypted regions of the current picture containing the current block or other pictures may be utilized. A picture (or tile / slice) that performs intra prediction or intra-BC prediction using only the current picture for restoration is called an intra picture or I picture (or tile / slice), and a picture (or tile / slice) capable of performing intra prediction, inter prediction, and intra-BC prediction is called an inter picture (or tile / slice).

[0111] Meanwhile, a picture (or tile / slice) that uses at most one motion vector and reference picture index to predict sample values ​​of each block among the inter-pictures (or tiles / slices) is called a predictive picture or P picture (or tile / slice), and a picture (or tile / slice) that uses at most two motion vectors and reference picture indices is called a Bi-predictive picture or B picture (or tile / slice).

[0112] That is, the P picture (or tile / slice) uses at most one set of motion information to predict each block, and the B picture (or tile / slice) uses at most two sets of motion information to predict each block. Here, a set of motion information may include one or more motion vectors and one reference picture index.

[0113] The intra prediction unit (261) generates a prediction block using intra encoding information and restored samples within the current picture, and the intra encoding information may include at least one of an intra prediction mode, an MPM (Most Probable Mode) flag, an MPM index, information regarding a reference sample, and various parameter information required for intra prediction according to each intra prediction method.

[0114] The intra prediction unit (261) can predict the sample values ​​of the current block by using the restored samples located to the left and / or above the current block as reference samples.

[0115] For example, the reference samples may be samples adjacent to the left boundary of the current block and / or samples adjacent to the upper boundary, and among the samples of the surrounding blocks of the current block, samples located on a line within a preset distance from the left boundary of the current block and / or samples located on a line within a preset distance from the upper boundary of the current block. In this case, the surrounding blocks of the current block may include at least one of the left (L) block, upper (A) block, lower-left (BL) block, upper-right (AR) block, or upper-left (AL) block adjacent to the current block.

[0116] The inter prediction unit (265) generates a prediction block using reference picture and inter encoding information stored in the DPB (270), and the inter encoding information may include a set of motion information of the current block for the reference block (reference picture index, motion vector information, etc.).

[0117] Meanwhile, inter-prediction can include L0 prediction, L1 prediction, and bi-prediction.

[0118] L0 prediction means a prediction using one reference picture included in the L0 picture list, and L1 prediction means a prediction using one reference picture included in the L1 picture list. To do this, a set of motion information (e.g., motion vector and reference picture index) may be required.

[0119] In the bidirectional prediction method, up to two reference regions can be used, and the two reference regions may exist in the same reference picture or in different pictures. Accordingly, in the up to two sets of motion information used in the bidirectional prediction method, two motion vectors may correspond to the same reference picture index or to different reference picture indices.

[0120] At this time, the reference pictures are pictures located temporally before or after the current picture, and can be completed pictures that have already been restored, and the two reference regions used in the bidirectional prediction method can be regions selected from the L0 picture list and the L1 picture list, respectively.

[0121] The inter prediction unit (265) can obtain a reference block of the current block using a motion vector and a reference picture index, and the reference block exists within a reference picture corresponding to the reference picture index.

[0122] In addition, a sample value of a block specified by a motion vector or an interpolated value thereof can be used as a predictor for the current block. For motion prediction with subpel-level pixel accuracy, an 8-tap interpolation filter can be used for the luminance signal and a 4-tap interpolation filter can be used for the chrominance signal.

[0123] Meanwhile, the configuration and operation of the decoder described with reference to FIG. 5 is according to an embodiment of the present invention, and some configurations may be omitted or added as needed, and the decoder can decode an image by performing the reverse process of the encoding method of the encoder described above.

[0124] Meanwhile, as previously described, a matrix-based intra prediction (hereinafter referred to as MIP) may be used as one of the multiple intra prediction modes. MIP may also be referred to as MPDIP (Matrix-based position-Dependent Intra-Prediction). Below, the process of generating prediction samples according to the MIP mode in the intra prediction unit (161, 261) is described.

[0125] FIG. 6 is a diagram illustrating the process of generating a prediction sample of the current block according to the MIP mode according to one embodiment of the present invention.

[0126] When the MIP mode is applied to the current block, the intra prediction unit (161, 261) performs a matrix multiplication procedure using surrounding reference samples and, if necessary, further performs a horizontal / vertical interpolation procedure to obtain prediction samples for the current block.

[0127] In order to predict samples of a current block of size W*H with width (W) and height (H) in MIP mode, reference samples (Reference(r(k)) of a reference region consisting of restored left peripheral reference samples and upper peripheral reference samples of the current block are used as input values. As shown in Equation 1 below, a predicted sample (P(x,y)) of the current block at position (x,y) can be obtained through matrix multiplication operations between the reference samples (r(k)) and the matrix (M(x,y,k)).

[0128]

[0129] In mathematical formula 1, k is a reference sample index pointing to one of a plurality of MIP matrices, and one of the plurality of MIP matrices can be set for each predicted pixel location (x,y) of the current block. Each of the plurality of MIP matrices includes weight vectors composed of weights that are multiplied to surrounding reference pixels for each predicted pixel location.

[0130] FIGS. 7a and 7b illustrate an example of a process for generating a predicted sample of a current block according to an MIP mode according to an embodiment of the present invention.

[0131] To predict samples of a current block (700) of size W*H having width (W) and height (H), a reference sample (r(k)) used as an input for MIP mode can be obtained from the restored upper peripheral reference samples (710, 730) and left peripheral reference samples (720, 740) of the current block. For example, when the size of the current block (700) is 4*4, as shown in FIG. 7a, four upper peripheral reference samples (710) and four left peripheral reference samples (720) can be used as input values ​​for matrix multiplication operations. Additionally, as shown in FIG. 7b, when the size of the current block (700) is 4*4, peripheral reference pixels reconstructed through an averaging process for eight upper peripheral reference samples (730) and eight left peripheral reference samples (740) can be used as input values ​​for matrix multiplication operations. FIGS. 7a and 7b are merely examples, and the reference region including the reference pixels used in MIP mode is not limited thereto and may be changed. The method for setting the reference region will be described later.

[0132] It is assumed that the values ​​of surrounding reference pixels through the above process, or the values ​​of surrounding reference pixels reconstructed through the averaging process, are as shown in Table 1 below.

[0133] Reference Sample Reference Sample Value T1[0]100 T1[1]102 T1[2]101 T1[3]99 T2[0]103 T2[1]105 T2[2]107 T2[3]108

[0134] In addition, it is assumed that the MIP matrix applied to each pixel position of the current 4x4 block is set as shown in Table 2 below.

[0135] Pixel position (x,y) Weight vector F(x,y,k)(0,0)[0.20, 0.15, 0.10, 0.05, 0.25, 0.15, 0.07, 0.03](1,0)[0.25, 0.20, 0.10, 0.05, 0.20, 0.10, 0.07, 0.03](0,1)[0.10, 0.10, 0.05, 0.05, 0.30, 0.25, 0.10, 0.05](1,1)[0.15, 0.10, 0.10, 0.05, 0.25, 0.20, 0.10, 0.05]

[0136] For example, the predicted sample P(0,0) at the (0,0) position of the current block (700) and the predicted sample P(1,1) at the (1,1) position can be obtained as shown in the following mathematical formulas 2 and 3, respectively, through a weighted sum using the reference sample value of the surrounding pixels in Table 1 and the weight vector applied to the (0,0) and (1,1) pixel positions in Table 2.

[0137]

[0138]

[0139] FIG. 8 is a diagram illustrating an interpolation process performed in MIP mode in one embodiment of the present invention.

[0140] Referring to FIG. 8, when the prediction mode of the current block is MIP mode, only the pixel values ​​of the current block's specific locations (A1, A2, B1, B2, etc.) are predicted by applying the surrounding reference sample values ​​and the MIP matrix as described in Equation 1 above, and the pixel values ​​of the remaining locations can be predicted through a horizontal or vertical interpolation process using adjacent surrounding pixels. Pixels (a1 to a6, b1 to b6) located in the same row as the pixels of the specific locations (A1, A2, B1, B2, etc.) are predicted through horizontal linear interpolation using adjacent available surrounding pixels, and the remaining pixels (810) not located in the same row as the pixels of the specific locations (A1, A2, B1, B2, etc.) can be predicted through vertical linear interpolation using adjacent available surrounding pixels.

[0141] Meanwhile, whether the prediction mode of the current block is MIP mode can be signaled through a specific intra prediction mode. Additionally, depending on the size of the current block, a specific intra prediction mode can be configured to be replaced by the MIP mode in different ways. For example, depending on the size of the current block (W*H), a specific intra prediction mode can be replaced by the MIP mode as shown in Table 3 below.

[0142] Current block size Intra prediction mode replaced by MIP mode W≤16 && H≤160, 1, (2+2*k) Others 0, 1, (2+4*k)

[0143] According to Table 3, if the intra prediction mode of the current block corresponds to a specific intra prediction mode that is replaced by the MIP mode, the prediction mode of the current block may be determined to be the MIP mode. For example, if the width (W) and height (H) of the current block are both 16 or less, and the intra prediction mode of the current block corresponds to 0 (planar mode), 1 (DC mode), or (2+2*k) (k is an integer), the prediction mode of the current block is determined to be the MIP mode. As another example, if the size of the current block is 8*16 and the intra prediction mode of the current block corresponds to one of {0, 1, 2, 4, 6, 8, 10, 12}, the prediction mode of the current block is determined to be the MIP mode. In addition, if the width (W) of the current block is not less than or equal to 16 or the height (H) is not less than or equal to 16, and the intra prediction mode of the current block corresponds to one of 0 (planar mode), 1 (DC mode), or (2+4*k), the prediction mode of the current block is determined to be MIP mode. As another example, if the size of the current block is 16*32 and the intra prediction mode of the current block corresponds to one of {0, 1, 2, 6, 10, 14, 18}, the prediction mode of the current block is determined to be MIP mode.

[0144] The MIP mode may be restricted depending on the size and shape of the current block, not limited to Table 3 described above. The MIP mode may not be applied if the size of the current block exceeds a predetermined size, if the shape of the current block is excessively large in only one direction (horizontal or vertical), if the ratio of width to height exceeds a certain value, or if the current block is not rectangular. For example, the MIP mode may be restricted for blocks that are excessively long in only one direction (horizontal or vertical), such as 4*32, 32*4, 8*32, or 32*8, or for blocks where both width and height exceed a predetermined threshold size (e.g., 32*32). In cases where the MIP mode is restricted in this way, the intra prediction mode of the current block is applied as is, and the intra prediction mode is not replaced by the MIP mode.

[0145] FIG. 9 is a diagram illustrating the process of configuring an MPM list according to an embodiment of the present invention.

[0146] Referring to FIG. 9, six MPM lists can be configured based on the intra prediction modes of the surrounding blocks (A) adjacent to the upper side of the current block and the surrounding blocks (L) adjacent to the left side. For example, if the size of the current block is W*H and the top-left corner position of the current block is (0,0), the surrounding block (A) adjacent to the upper side may be a block containing the surrounding pixels at the position (W-1, -1), and the surrounding block (L) adjacent to the left side may be a block containing the surrounding pixels at the position (-1, H-1).

[0147] If the intra prediction mode of the surrounding blocks (A, L) is not available, the prediction mode of the surrounding blocks can be set to planar mode to construct the MPM list. If the intra prediction mode of the surrounding blocks (A, L) is a non-angular mode such as planar or DC, a 6-MPM list including {Planar, DC, V, H, V-4, V+4} (V is vertical mode, H is horizontal mode) can be constructed. If the intra prediction mode of the surrounding blocks (A, L) is an angular mode, the MPM list can be constructed using the surrounding intra prediction modes of that angular mode. For example, the MPM list can be constructed using an intra prediction mode to which a predetermined offset (±1, ±2, ��) is applied to the intra prediction mode of the surrounding blocks (A, L). In this case, considering redundancy, prediction modes already included in the MPM list are not added to the MPM list.

[0148] Additionally, an MPM list may be constructed based on the intra prediction modes of the surrounding blocks adjacent to the upper side of the current block (A), adjacent to the left side (L), adjacent to the upper right side (AR), adjacent to the lower left side (BL), and adjacent to the upper left side (AL). Specifically, an MPM list containing 22 MPM candidates may be constructed based on the intra prediction modes of the surrounding blocks L, A, BL, AR, and AL, and the surrounding intra prediction modes and default modes to which a predetermined offset (±1, ±2, ��) is applied to the first two directional intra prediction modes available among them. Here, the default modes may include a horizontal mode, a vertical mode, a diagonal mode in the upper-left direction, and a DIMD mode.

[0149] A PMPM (Primary MPM) containing the top 6 intra-prediction modes from the list of 22 MPMs may be configured, or a PMPM containing the top 5 intra-prediction modes and a Planar mode may be configured. The remaining 16 intra-prediction modes not included in the PMPM constitute a SMPM (Secondary MPM).

[0150] Meanwhile, when configuring MPM candidates included in the MPM list, the MPM candidates can be sorted in ascending order based on costs such as SATD (Sum of Absolute Transformed Differences) to determine the order of the MPM candidates included in the MPM list. That is, the smaller the SATD, the higher the rank (smaller mpm_idx) the MPM candidate can be determined.

[0151] FIGS. 10a and 10b are drawings for explaining a method of determining the cost when configuring an MPM candidate according to one embodiment of the present disclosure.

[0152] Referring to FIG. 10a, a prediction mode according to an MPM candidate is applied to an upper template (1012) of a predetermined size composed of upper surrounding pixels of the current block (1011) to obtain a predicted value of the upper template (1012) from L-shaped surrounding reference pixels (1013), and a first cost such as SATD can be calculated based on the value of the upper template (1012) and the predicted value. Additionally, referring to FIG. 10b, a prediction mode according to an MPM candidate is applied to a left template (1022) of a predetermined size composed of left surrounding pixels of the current block (1021) to obtain a predicted value of the left template (1022) from L-shaped surrounding reference pixels (1023), and a second cost such as SATD can be calculated based on the value and predicted value of the left template (1022). The first and second costs obtained from the upper template (1012) and the left template (1022) can be summed to obtain the final template cost. Based on this template cost, MPM candidates included in the MPM list can be sorted. As described below, when a specific MPM candidate in the MPM list is replaced with an MIP mode, the template cost can be calculated according to the MIP mode when calculating the template cost according to the said MPM candidate.

[0153] Whether the MPM list is in use can be signaled explicitly or implicitly. For example, if the MPM list is explicitly signaled, the video encoder (100) may transmit flag information (MPM_flag) indicating whether the MPM list is in use to the video decoder (200). If MPM_flag indicates that the MPM list is in use, the MPM index (mpm_idx) may be transmitted as intra prediction mode information instead of the intra prediction mode index information. In this specification, the intra prediction mode information of the current block may be the mpm index when the MPM list is in use, and general intra prediction mode index information when the MPM list is not in use. Whether the MPM list is in use may also be implicitly derived based on additional information related to the current block.

[0154] Meanwhile, in one embodiment, a specific intra prediction mode within an MPM list or a non-MPM list may be replaced with an MIP mode, and in this case, if the prediction mode of the current block corresponds to a specific intra prediction mode that has been replaced with an MIP mode, the prediction mode of the current block may be determined as an MIP mode. Additionally, a specific intra prediction mode may be configured to be replaced with an MIP mode in different ways depending on the size of the current block. For example, depending on the size of the current block (W*H), a specific intra prediction mode that is replaced with an MIP mode from an initial intra prediction mode (Intra_Prediction_Mode) may be determined as shown in Table 4 below.

[0155] Current block size Intra prediction mode replaced by MIP mode W≤16 && H≤16((Intra_Prediction_Mode -2)+1>>1)*2+2 Others((Intra_Prediction_Mode -2)+2>>1)*4+2

[0156] For example, if the current block's width (W) and height (H) are both 16 or less and Intra_Prediction_Mode is 3, the intra prediction mode with a value of 4 ((3-2)+1>>1)*2+2) is replaced with the MIP mode. That is, the intra prediction mode index 4 is set to point to the MIP mode instead of pointing to the existing directional intra prediction mode. If Intra_Prediction_Mode is 6, the value of ((6-2)+1>>1)*2+2 is 6, so the intra prediction mode with a value of 6 is replaced with the MIP mode. If Intra_Prediction_Mode is 2, the value of ((2-2)+1>>1)*2+2 is 2, so the intra prediction mode with a value of 2 is replaced with the MIP mode. If Intra_Prediction_Mode is 8, the value of ((8-2)+1>>1)*2+2 is 8, so the intra prediction mode with a value of 8 is replaced with the MIP mode. In this way, among all Intra_Prediction_Modes, the index of the intra prediction mode pointing to the MIP mode can be determined.

[0157] As another example, if the current block width (W) is greater than 16 or the height (H) is greater than 16 and Intra_Prediction_Mode is 3, the intra prediction mode with a value of 6 ((3-2)+2)>>1)*4+2 is replaced with MIP mode. If Intra_Prediction_Mode is 6, the intra prediction mode with a value of 14 ((6-2)+2>>1)*4+2) is replaced with MIP mode.

[0158] In this way, specific intra prediction modes can be determined to point to MIP modes. When the intra prediction mode of the current block is signaled through the MPM (Most Probable Mode) list, whether the prediction mode of the current block is a MIP mode can be determined by whether it corresponds to a specific intra prediction mode that has been replaced to point to a MIP mode within the MPM list. For example, as in the example above, if the width (W) and height (H) of the current block are both 16 or less, if the intra prediction mode of the current block is one of the intra prediction modes that has been replaced to a MIP mode such as {2, 4, 6}, the prediction mode of the current block can be determined to be a MIP mode.

[0159] The MIP mode may be restricted based on the size and shape of the current block, not limited to the aforementioned Table 4. The MIP mode may not be applied if the size of the current block exceeds a predetermined size, if the shape of the current block is excessively large in only one direction (horizontal or vertical), if the ratio of width to height exceeds a certain value, or if the current block is not rectangular. For example, the MIP mode may be restricted for blocks that are excessively long in only one direction (horizontal or vertical), such as 4*32, 32*4, 8*32, or 32*8, or for blocks where both width and height exceed a predetermined threshold size (e.g., 32*32). In cases where the MIP mode is restricted in this manner, the intra prediction mode of the current block determined from the MPM list is applied as is, and the intra prediction mode is not replaced by the MIP mode.

[0160] An intra prediction mode replaced by an MIP mode may be added to the MPM list, and redundancy is considered when configuring the MPM list. If an intra prediction mode replaced by an MIP mode is already included in the MPM list, an intra prediction mode obtained by adding a predetermined offset to the intra prediction mode replaced by an MIP mode may be added to the MPM list. For example, if an intra prediction mode such as {2, 4, 6...} is replaced by an MIP mode and {2,4,6...} is already configured in the MPM list, and an MPM candidate cannot be newly added to the MPM list due to redundancy considerations because it is already included in the MPM list as one of {2,4,6...}, then an intra prediction mode obtained by adding a predetermined offset to the intra prediction mode of that MPM candidate may be added to the MPM list. In this case, the offset may be set according to the current block size. For example, if the width and height are 16 or less, the offset is set to 2, otherwise it is set to 4, and the value obtained by adding the offset to the corresponding intra prediction mode can be included in the MPM list. Even when adding an MPM candidate with the offset added to the MPM list, redundancy is considered, and if the MPM candidate with the offset added is also included in the MPM list, it is not added to the MPM list.

[0161] As mentioned above, when MPM candidates are sorted in ascending order based on the cost of SATD when configuring MPM candidates included in the MPM list, the cost of the MPM candidate corresponding to the MIP mode is obtained by applying the MIP mode.

[0162] Below, a method for setting a reference region containing reference pixels used to generate predicted values ​​according to MIP mode is described.

[0163] FIG. 11 is a drawing showing an example of a reference area used in MIP mode according to one embodiment of the present disclosure.

[0164] As described above, the MIP mode generates a predicted value through an operation using a reference region (r(k)) containing surrounding pixels restored prior to the current block and a predetermined MIP matrix based on the pixel location within the current block.

[0165] In one embodiment, the reference area may be adaptively set to improve the prediction efficiency of the MIP mode. For example, the number of upper reference lines (A_h) and the number of left reference lines (A_w) included in the reference area may be adaptively set. The video encoder (100) and the video decoder (200) may set the number of reference lines (A_w and A_h) according to a preset rule.

[0166] In one embodiment, the number of reference lines included in the reference area can be adaptively set according to information related to the current block. As information related to the current block, i) the size of the current block, ii) the shape (form) of the current block, iii) the slice type, iv) the Quantization Parameter (QP), and v) surrounding block information may be used.

[0167] i) When the number of reference lines (A_w and A_h) is set according to the size of the current block, for example, the number of reference lines (A_w and A_h) may be set to 2 for a size of 16*16 or less, and the number of reference lines (A_w and A_h) may be set to 1 for a size of 16*32 to 32*32. As described above, if the size of the current block exceeds a predetermined threshold size (e.g., 32*32), the MIP mode itself may be set not to be applied.

[0168] ii) When the number of reference lines (A_w and A_h) is set according to the shape of the current block, for example, if the width (W) of the current block is greater than the height (H), the number of left reference lines (A_w) may be set to be greater than the number of upper reference lines (A_h), or conversely, the number of left reference lines (A_w) may be set to be smaller than the number of upper reference lines (A_h). Also, if the height (H) of the current block is greater than the width (W), the number of left reference lines (A_w) may be set to be greater than the number of upper reference lines (A_h), or conversely, the number of left reference lines (A_w) may be set to be smaller than the number of upper reference lines (A_h). If the shape of the current block is excessively large in either the horizontal or vertical direction, for example, if the width / height ratio of the current block is excessively large or small, such as 4*32 or 32*4, the MIP mode itself may not be applied.

[0169] iii) Since the characteristics of the referenced restoration pixel differ depending on whether the slice type is intra-slice or inter-slice, the number of reference lines (A_w and A_h) can be set according to the type of slice to which the current block belongs.

[0170] iv) As the QP value decreases, the image compression rate decreases, but the quality of the reference pixels becomes similar to the original, and as the QP value increases, the quality decreases, so the number of reference lines (A_w and A_h) can be adaptively set according to the QP value to improve prediction accuracy.

[0171] v) The number of reference lines (A_w and A_h) can be adaptively determined based on various information of surrounding blocks adjacent to the current block on the left and upper sides, such as QP, prediction mode, BS value, and partitioning information of the surrounding blocks on the left and upper sides.

[0172] Meanwhile, the video encoder (100) may determine the optimal number of reference lines (A_w and A_h) based on cost and include information about the number of reference lines (A_w and A_h) in one of the data units such as sequence, GOP, frame, picture, slice, CTU, CT, TU, etc., and transmit it to the video decoder (200).

[0173] FIG. 12 is a drawing showing another example of a reference area used in MIP mode according to one embodiment of the present disclosure.

[0174] If the size of the current block is denoted as (B_w)*(B_h), the reference area can be expanded to include the upper right peripheral area (1210) and lower left peripheral area (1220) of the current block in addition to the upper peripheral area (1230) and left peripheral area (1240). Whether to include the upper left peripheral area (1250) in the expanded area can be determined adaptively, and the upper left peripheral area (1250) can be excluded from the reference area. The number of upper reference lines (A_h) or the number of left reference lines (A_w) can be determined adaptively by considering the size of the current block, etc., as described above.

[0175] The upper right peripheral area (1210) and the lower left peripheral area (1220) can be adaptively set according to the size of the current block. The width (C_w) of the upper right peripheral area (1210) included in the reference area can be set to the width (B_w) of the current block * k1 (k1 is a positive integer), and the height (C_h) of the lower left peripheral area (1220) can be set to the height (B_h) of the current block * k2 (k2 is a positive integer). For example, k1 and k2 can be 2. That is, when the size of the current block is W * H (W is the width of the current block, H is the height of the current block), a reference area including 2W upper peripheral pixels and 2H left peripheral pixels can be determined.

[0176] In addition, the number of surrounding reference pixels included in the reference area may be limited to a predetermined threshold. For example, the number of reference pixels included in the reference area may be limited to 50 or fewer. If the number of reference pixels exceeds the predetermined threshold due to the expansion of the reference area, the size of the expanded area (C_w, C_h) included in the reference area may be reduced by a predetermined ratio so that the number of pixels within the reference area becomes less than or equal to the predetermined threshold.

[0177] Additionally, the width (C_w) of the upper right peripheral area (1210) included in the reference area and the height (C_h) of the lower left peripheral area (1220) can be adaptively set according to the shape of the current block. For example, if the current block is non-square and the width of the current block is greater than the height, C_h can be set to a value smaller than C_w, or conversely, C_w can be set to a value smaller than C_h.

[0178] In addition, the width (C_w) of the upper right peripheral area (1210) included in the reference area and the height (C_h) of the lower left peripheral area (1220) can be adaptively set according to the slice type of the current block, the QP of the current block, the QP of the peripheral block, the prediction mode, prediction information, BS value, division information, etc.

[0179] Additionally, the video encoder (100) may determine the width (C_w) of the optimal upper right peripheral area (1210) and the height (C_h) of the lower left peripheral area (1220) based on cost, and may transmit the information regarding the width (C_w) of the upper right peripheral area (1210) and the height (C_h) of the lower left peripheral area (1220) to the video decoder (200) by including it in one of the data units such as sequence, GOP, frame, picture, slice, CTU, CT, TU, etc.

[0180] FIG. 13 is a drawing illustrating a reference mode that selectively applies surrounding pixels included in a reference area according to one embodiment of the present disclosure.

[0181] In one embodiment, as a surrounding reference pixel used for prediction according to the MIP mode, one of the following may be applied to perform the MIP mode: i) a first reference mode used for the MIP mode by including both upper surrounding pixels (1310) and left surrounding pixels (1320) in the reference area; ii) a second reference mode used for the MIP mode by including only upper surrounding pixels (1310) in the reference area; and iii) a third reference mode used for the MIP mode by including only left surrounding pixels (1320) in the reference area. Whether to use any of the first to third reference modes may be implicitly determined based on the prediction information of the current block, size, QP, slice type, MPM index, the ratio of the width and height of the current block, information on the prediction tool used for intra / inter prediction in the surrounding block, MRL information, reference value of the block vector, and the values ​​of L-shaped surrounding pixels (1310, 1320). That is, whether to use any of the first to third reference modes can be determined based on the various information mentioned above.

[0182] Additionally, the video encoder (100) may determine one of the first to third reference modes based on cost and transmit the determined reference mode information to the video decoder (200) by including it in one of the data units such as sequence, GOP, frame, picture, slice, CTU, CT, TU, etc. At this time, the video encoder (100) may adaptively configure and transmit the order of the first to third reference modes by utilizing information from surrounding restoration blocks. For example, the default order is signaled with a predetermined index assigned in the order of the first reference mode, the second reference mode, and the third reference mode, but if the third reference mode is used in both the left and upper surrounding blocks, the order may be changed to the third reference mode, the first reference mode, and the second reference mode, and the assigned index may be signaled. In this way, by changing the reference mode order using surrounding information, bits can be saved through variable length coding during reference mode coding.

[0183] Additionally, the video encoder (100) may arbitrarily select only two reference modes as candidates based on reference mode information used in surrounding blocks, select one of these reference modes, and signal the finally selected reference mode information. For example, if the reference modes used on the left and upper sides are the second reference mode and the third reference mode, the video encoder (100) may form a candidate group consisting only of the second reference mode and the third reference mode, determine the optimal reference mode in terms of cost among them, include the corresponding reference mode information in the bitstream, and transmit it to the video decoder (200). If only the second reference mode is used on both the left and upper sides, the second reference mode and the first reference mode having the highest priority in the basic order may be combined to form a candidate group consisting only of the second reference mode and the first reference mode, determine the optimal reference mode among them, include the corresponding reference mode information in the bitstream, and transmit it to the video decoder (200).

[0184] Meanwhile, a boundary condition can be set when surrounding reference pixels included in the reference area are unavailable. Cases corresponding to the boundary condition include when the current block is located at one of the left boundary edge, upper-left boundary edge, lower-left boundary edge, top boundary edge, right boundary edge, upper-right boundary edge, or lower-right boundary edge of a frame, slice, picture, tile, etc., and surrounding pixels of the current block are unavailable; when the current block is adjacent to a data unit such as a CTU or CU that cannot be utilized because it is not managed in a buffer such as a DPB; or when the current block references a pixel that is not set to an available state according to the availability check process.

[0185] In this way, if a boundary condition is met, that is, if at least some of the surrounding reference pixels cannot be used as reference pixels for the current block, the number of surrounding pixels required for MIP may be insufficient. In this case, if the number of surrounding pixels required for MIP is insufficient, the MIP mode may be configured not to be applied to the current block. Additionally, if even one of the surrounding pixels required for MIP prediction is unavailable, the video encoder (100) and the video decoder (200) may determine that the MIP mode is not applied.

[0186] Instead of applying the MIP mode when at least some of the surrounding pixels are unavailable, the values ​​of the surrounding pixels can be determined by extending the values ​​of other available surrounding pixels and used for MIP prediction. For example, if the number of available surrounding pixels is less than the number of surrounding pixels required for MIP, MIP prediction can be performed by determining the pixel values ​​of the unavailable surrounding pixels using at least one of the following: interpolation or extrapolation of available surrounding pixels, the average or weighted average of available surrounding pixels, the transpose of available surrounding pixels, available previous pixels obtained through template matching, or extension of available surrounding pixels. For example, if 40 neighboring pixels are required for the MIP operation but the number of available neighboring pixels is less than 40, the value of an unavailable neighboring pixel can be determined by using at least one of the following methods: interpolating the available neighboring pixel value, copying and using the nearest available neighboring pixel value, replacing it with the average or weighted average value of the available neighboring pixels, transposing and copying the available neighboring pixels, or copying the value of an available previous pixel found through template matching for the unavailable neighboring pixel.

[0187] Additionally, if the left reference area is unavailable, the available reference area on the upper side can be copied, or the available reference area on the upper side can be expanded by a factor of 2 and used as the pixel value of the unavailable surrounding pixels of the left reference area. Additionally, if the upper reference area is unavailable, the available reference area on the left side can be copied, or the available reference area on the left side can be expanded by a factor of 2 and used as the pixel value of the unavailable surrounding pixels of the upper reference area.

[0188] If the current block corresponds to a boundary condition but is part of a 360-degree image, there may exist available surrounding pixels, i.e., available wrap-around pixels, which are not immediately adjacent to the current block but are actually adjacent to the current block in the 360-degree image. When such wrap-around pixels are available, the value of the unavailable surrounding pixels can be replaced with the value of these wrap-around pixels. For example, if the current block is part of a 360-degree image, and the current block is located near the left boundary and there are unavailable surrounding pixels, the value of a pixel located near the right boundary at the same height as the unavailable surrounding pixels can be used as the value of the unavailable surrounding pixels.

[0189] Additionally, the video encoder (100) may include information on how to determine unavailable surrounding reference pixels in one of the data units such as sequence, GOP, frame, picture, slice, CTU, CT, TU, etc., and transmit it to the video decoder (200).

[0190] In one embodiment, adaptive linear filtering may be performed on surrounding pixels included in the reference region. For example, 1:2:1-based linear filtering may be applied to surrounding pixels, but is not limited thereto, and various filters used in conventional intra prediction may be applied to surrounding pixels.

[0191] Whether to apply filtering to surrounding pixels can be determined adaptively. Whether to apply filtering to surrounding pixels can be implicitly derived based on at least one of the current block's prediction information, block size, QP, slice type, MPM index, ratio of the current block's width to height, the prediction mode information of surrounding blocks or the intra / inter tool information used, MRL information, the reference value of the block vector, and L-shaped surrounding pixel values.

[0192] Linear filtering may be applied to all reference pixels in the reference area used for MIP prediction, or, if the reference area contains multiple reference lines, it may be configured to apply linear filtering only to specific reference lines. For example, linear filtering may be applied only to the reference line closest to the current block, or linear filtering may be applied only to the line closest to the current block and a reference line selected in the MRL.

[0193] The video encoder (100) may include linear filtering information for a reference pixel in one of the data units such as a sequence, GOP, frame, picture, slice, CTU, CT, TU, etc. and transmit it to the video decoder (200). As linear filtering information, it may include whether linear filtering is applied to all reference areas, whether linear filtering is applied only to reference areas used in existing intra prediction, whether linear filtering is applied only to specific reference lines among the reference areas, whether only some of the upper reference area and left reference area are adaptively filtered, and information pointing to the reference area among the upper reference area and left reference area that is linearly filtered.

[0194] Meanwhile, the matrix used in MIP mode may be a matrix trained either online or offline. When using a trained matrix, the matrix can be generated based on input parameters such as the current block width, height, intra-prediction mode information, and reference lines. When training the matrix online or offline, the cost can be calculated based on the aforementioned input parameters to determine the optimal matrix according to those parameters. Subsequently, a suitable matrix can be selected and utilized in MIP based on information such as the current block width, height, intra-prediction mode information, and reference lines being coded.

[0195] Meanwhile, since the matrix used for MIP is determined based on the current block size and intra-prediction mode, a separate matrix must be stored for each. This can place a burden on the memory of the encoder and decoder.

[0196] Accordingly, in one embodiment, elements included in a reference matrix (Fmax) set for the largest available block size are subsampled based on the size of the current block, and the subsampled matrix can be used in MIP mode.

[0197] Specifically, a reference matrix (F) applied to a reference block size (e.g., 32*32), which is the largest block size to which MIP is applicable. max (x, y, k) (k is the reference sample index) is generated in advance through online and offline learning.

[0198] The matrix applied to blocks smaller than the maximum block size to which MIP is applicable (e.g., 32*32) is a reference matrix (F) according to a subsampling rate determined based on the size of the largest reference block and the current block. max It can be determined by sampling some elements within (x,y,k)). The maximum block size to which MIP is applicable is W. max * H max ,If the current block size is W*H, the matrix (F) used for the MIP of the current block WⅹH (x,y,k)) can be obtained based on the following mathematical formula 4.

[0199]

[0200] According to mathematical formula 4, the reference matrix (F max It can be determined which pixel location's weight vector to sample from among the pixel location-specific weight vectors included within (x,y,k)). For example, the reference matrix (F max When (x,y,k)) is set for an 8*8 block and the current block size is 4*4, the reference matrix (F) applied to the 8*8 block is a weight vector applied to the predicted value of the pixel at position (a,b) within the current block. max F applied to the pixel at position (2a, 2b) among the weight vectors of (x,y,k). max (2x,2y,k) is used. That is, the weight vector applied to the pixel at position (0,0) of the 8*8 reference block is used as the weight vector applied to the predicted value of the pixel at position (0,0) of the current block, the weight vector applied to the pixel at position (0,2) of the 8*8 reference block is used as the weight vector applied to the predicted value of the pixel at position (0,1) of the current block, the weight vector applied to the pixel at position (2,0) of the 8*8 reference block is used as the weight vector applied to the predicted value of the pixel at position (1,0) of the current block, and the weight vector applied to the pixel at position (2,2) of the 8*8 reference block is used as the weight vector applied to the predicted value of the pixel at position (1,1) of the current block.

[0201] In addition, each element included in the subsampled matrix is ​​normalized according to the following Equation 5 so that the sum of the elements (weights) applied to a specific position in the current block becomes a predetermined value (e.g., 1), and finally, the normalized subsampled matrix (F' WⅹH (x,y,k)) can be used as a matrix for predicting the MIP of the current block of size W*H.

[0202]

[0203] That is, when predicting the MIP of the current block of size W*H, a prediction sample (P(x,y)) at the (x,y) position within the current block is obtained using a normalized subsampled matrix according to the following mathematical formula 6.

[0204]

[0205] For example, the process of setting the matrix used for MIP prediction of the current 4x4 block from the reference matrix set for an 8x8 block is explained as follows.

[0206] Reference matrix (F) applied to an 8*8 block max Assume that the weight vectors applied to some pixel locations among the weight vectors included in (x,y,k) are as shown in Table 5 below.

[0207] Weight vector (F) applied to position (x, y) within the reference block max(x,y,k))(0,0)[0.14, 0.10, 0.06, 0.04, 0.22, 0.20, 0.14, 1.10](2,0)[0.12, 0.10, 0.08, 0.04, 0.23, 0.19, 0.14, 0.10](4,0)[0.11, 0.09, 0.08, 0.05, 0.24, 0.20, 0.13, 0.10](6,0)[0.10, 0.09, 0.07, 0.04, 0.25, 0.21, 0.13, 0.11](0,2)[0.13, 0.09, 0.06, 0.05, 0.23, 0.20, 0.14, 0.10](2,2)[0.12, 0.10, 0.07, 0.05, 0.23, 0.20, 0.14, 0.09](4,0)[0.11, 0.09, 0.07, 0.05, 0.24, 0.20, 0.14, 0.10](6,2)[0.10, 0.09, 0.08, 0.04, 0.25, 0.20, 0.14, 0.10]

[0208] As described above, the weight vector applied to the pixel at position (0,0) of the 8*8 reference block is used as the weight vector applied to the predicted value of the pixel at position (0,0) of the current block, the weight vector applied to the pixel at position (0,2) of the 8*8 reference block is used as the weight vector applied to the predicted value of the pixel at position (0,1) of the current block, the weight vector applied to the pixel at position (2,0) of the 8*8 reference block is used as the weight vector applied to the predicted value of the pixel at position (1,0) of the current block, and the weight vector applied to the pixel at position (2,2) of the 8*8 reference block is used as the weight vector applied to the predicted value of the pixel at position (1,1) of the current block. That is, according to the following Table 6, the weight vector applied to the pixel at the subsampling position of the reference block is used as the weight vector applied to the position of the corresponding 4*4 block.

[0209] Weight vector (F) applied to the (x, y) position based on the (x, y) position within the block and the corresponding (x, y) position within the block.4X4 (x,y,k))(0,0)(0,0)[0.14, 0.10, 0.06, 0.04, 0.22, 0.20, 0.14, 1.10](1,0)(2,0)[0.12, 0.10, 0.08, 0.04, 0.23, 0.19, 0.14, 0.10](0,1)(0,2)[0.13, 0.09, 0.06, 0.05, 0.23, 0.20, 0.14, 0.10](1,1)(2,2)[0.12, 0.10, 0.07, 0.05, 0.23, 0.20, 0.14, 0.09]

[0210] Since the sum of the weights corresponding to the (1,0), (0,1), and (1,1) positions among the weight vectors in Table 6 is not 1, each weight is divided by the sum of the weights according to the aforementioned mathematical formula 5 to normalize the sum of all weights to 1, and through the normalization process for the subsampled matrix of Table 6, normalized subsampled weight vectors applied to each pixel position of a 4*4 block can be obtained as shown in the following Table 7.

[0211] Summarized weight vector (F) of position weights at (x, y) within a 4*4 block 4X4 (x,y,k))(0,0)1.00[0.14, 0.10, 0.06, 0.04, 0.22, 0.20, 0.14, 1.10](1,0)0.90[0.133, 0.111, 0.089, 0.044, 0.256, 0.211, 0.156, 0.111](0,1)0.96[0.135, 0.094, 0.063, 0.052, 0.240, 0.208, 0.146, 0.104](1,1)0.90[0.133, 0.111, 0.078, 0.056, 0.256, 0.222, 0.156, 0.100]

[0212] Thus, in one embodiment, if the largest block size to which MIP is applicable is 32*32, then the reference matrix (F) applied to the 32*32 block size maxOnly ) is generated in advance through online and offline learning, and the matrix applied to blocks smaller than 32*32 is a reference matrix (F) according to the subsampling ratio determined by the size of the largest block and the current block. max A matrix applied to a small block can be determined by sampling some elements within it. Additionally, each element included in the subsampled matrix is ​​normalized such that the sum of the elements (weights) applied to a specific position in the current block becomes a specific value (e.g., 1), and finally, the normalized subsampled matrix can be used as a matrix for predicting the MIP of the small block.

[0213] Meanwhile, the video encoder (100) may include information about the matrix in one of the data units such as sequence, GOP, frame, picture, slice, CTU, CT, TU, etc. and transmit it to the video decoder (200). For example, the video encoder (100) may form a matrix candidate group and signal information about the applied matrix candidate within the matrix candidate group, or signal a subsampling rate or normalization information regarding subsampling.

[0214] FIG. 14 is a flowchart of a video encoding method according to one embodiment of the present disclosure.

[0215] Referring to FIG. 14, the intra prediction unit (161) performs intra prediction for the current block by applying a plurality of intra prediction modes, including an MIP mode, to the current block (S1410). The video encoder (100) can apply various prediction modes, including a plurality of intra prediction modes and an inter prediction mode, and determine the optimal prediction mode based on RDO (Rate-Distortion Optimization).

[0216] As described above, when performing MIP intra prediction for the current block, the intra prediction unit (161) can set a reference area used for MIP based on condition information of the current block, including the size and shape of the current block. Then, the intra prediction unit (161) determines a matrix used for MIP and obtains a prediction sample of the current block based on the reference pixels of the set reference area and the matrix.

[0217] Additionally, the video encoder (100) generates flag information indicating whether MIP is applied to the current block (S1420).

[0218] And, the video encoder (100) encodes the intra prediction mode information of the current block when an intra prediction mode including MIP mode is applied to the flag information and the current block (S1430).

[0219] As described above, whether the prediction mode of the current block is MIP mode can be signaled through a specific intra prediction mode. Additionally, depending on the size of the current block, the specific intra prediction mode can be configured to be replaced by the MIP mode in different ways. If the prediction mode of the current block is MIP mode, specific intra prediction mode information replaced to point to the MIP mode can be encoded as the intra prediction mode information of the current block.

[0220] Additionally, a specific intra prediction mode within an MPM list or a non-MPM list may be replaced with an MIP mode, in which case the intra prediction mode information replaced to point to the MIP mode may be encoded as the prediction mode of the current block. If the intra prediction mode of the current block is signaled through an MPM list, and the prediction mode of the current block is an MIP mode, then the specific intra prediction mode information replaced to point to the MIP mode within the MPM list or a non-MPM list may be encoded as the intra prediction mode information of the current block.

[0221] The video encoder (100) includes information about the MIP in one of the data units such as sequence, GOP, frame, picture, slice, CTU, CT, TU, and transmits it to the video decoder (200), and if the current block is a CU, it may include intra prediction mode information in the syntax element of the CU.

[0222] Additionally, as described above, the video encoder (100) may transmit to the video decoder (200) information regarding the number of reference lines (A_w and A_h), information regarding the width (C_w) of the upper right peripheral area (1210) and the height (C_h) of the lower left peripheral area (1220), information regarding the method for determining unavailable peripheral reference pixels, information regarding the reference mode determined among the first to third reference modes, information regarding the method for determining unavailable peripheral reference pixels, and information regarding the matrix, by including them in one of the data units such as sequence, GOP, frame, picture, slice, CTU, CT, TU, etc.

[0223] FIG. 15 is a flowchart of a video decoding method according to one embodiment of the present disclosure.

[0224] The video decoder (200) obtains flag information indicating whether MIP is applied and prediction mode information of the current block from the bitstream (S1510).

[0225] The video decoder (200) sets a reference area used for MIP based on condition information of the current block including the size of the current block, when flag information indicates that MIP is applied and the prediction mode of the current block determined based on prediction mode information is the MIP prediction mode (S1520).

[0226] As described above, whether the prediction mode of the current block is MIP mode can be signaled through a specific intra prediction mode. Additionally, depending on the size of the current block, a specific intra prediction mode can be configured to be replaced by the MIP mode in different ways. If the prediction mode of the current block corresponds to a specific intra prediction mode that has been replaced by the MIP mode, the prediction mode of the current block can be determined as MIP mode.

[0227] Additionally, a specific intra prediction mode within the MPM list or non-MPM list may be replaced with an MIP mode, and in this case, if the intra prediction mode of the current block corresponds to the specific intra prediction mode that was replaced to point to the MIP mode within the MPM list or non-MPM list, the prediction mode of the current block may be determined as an MIP mode.

[0228] Additionally, as described above, when the prediction mode of the current block is MIP mode, the intra prediction unit (261) may set a reference area based on the condition information of the current block, or, when information regarding the reference area is explicitly signaled, parse information related to the reference area from the bitstream to set the reference area. When the reference area is implicitly set based on the condition information of the current block, at least one of the ratio of the width and height of the current block, the slice type, QP, and information applied in the surrounding blocks of the current block may be used as the condition information. When setting the reference area, the number of reference lines may be determined based on the condition information of the current block. In addition, the size of the upper right surrounding pixels and the number of lower left surrounding pixels included in the reference area of ​​the current block may be determined based on the size of the current block. For example, when the size of the current block is W*H, the surrounding pixels included in the reference area may be determined using 2W upper surrounding pixels and 2H left surrounding pixels. In this case, the number of surrounding pixels included in the reference area may be limited to a predetermined threshold (e.g., 50 or fewer).

[0229] The intra prediction unit (261) determines whether the size of the current block corresponds to the size of the block for which the MIP is available in advance, and can set the reference area of ​​the current block only if the size of the current block corresponds to the size of the block for which the MIP is available. If the size of the current block does not satisfy the condition of the size of the block for which the MIP is available, the intra prediction unit (261) can perform a prediction according to the original intra prediction mode without performing the MIP mode, even if the intra prediction mode of the current block corresponds to a mode that has been replaced with the MIP mode. For example, if the 2nd intra prediction mode is replaced with the MIP mode and is set to point to the MIP mode, and the size of the current block corresponds to a size for which the MIP cannot be applied, the intra prediction unit (261) can perform an intra prediction according to the existing 2nd intra prediction mode instead of a prediction according to the MIP mode for the current block.

[0230] If the prediction mode of the current block is the MIP mode, the intra prediction unit (261) determines the matrix used for MIP (S1530). As described above, the matrix used for MIP can be determined based on at least one of the size of the current block and the intra prediction mode. Additionally, the matrix used for MIP can be obtained by subsampling a reference matrix set for a block of a reference size based on the size of the current block.

[0231] The intra prediction unit (261) obtains a prediction sample for each pixel location within the current block based on a set reference area and matrix. As described above, a prediction sample for each pixel location can be obtained by calculating the weighted sum of the weight vectors within the reference area and the weight vectors set for each pixel location.

[0232] The methods described above in this specification may be performed through a processor of a video encoder or a video decoder. Additionally, the encoder may generate a bitstream that is decoded by a video signal processing method, and the bitstream generated by the encoder may be stored in a computer-readable non-transient storage medium (recording medium).

[0233] The embodiments of the present invention described above may be implemented through various means. For example, the embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof.

[0234] Some embodiments may also be implemented in the form of a recording medium containing computer-executable instructions, such as program modules executed by a computer. A computer-readable medium may be any available medium accessible by a computer and includes both volatile and non-volatile media, and both removable and non-removable media.

[0235] Additionally, computer-readable media may include both computer storage media and communication media. Computer storage media include both volatile and non-volatile, removable and non-removable media implemented by any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data. Communication media typically include other data of modulated data signals, such as computer-readable instructions, data structures, or program modules, or other transmission mechanisms, and include any information transmission media.

Claims

1. A step of obtaining flag information indicating whether Matrix-based Intra prediction (MIP) is applied and prediction mode information of the current block; If the above flag information indicates that MIP is applied and the prediction mode of the current block determined based on the above prediction mode information is the MIP prediction mode, a step of setting a reference region used for the MIP based on condition information of the current block including the size of the current block; A step of determining a matrix used in the above MIP; and A video decoding method comprising the step of obtaining a predicted sample of the current block based on the reference region and matrix.

2. In Paragraph 1, The step of setting a reference area used in the above MIP is A video decoding method further comprising a step of determining whether the size of the current block corresponds to the size of a block available to a preset MIP, and, if the size of the current block corresponds to the size of a block available to the MIP, setting a reference area of ​​the current block.

3. In Paragraph 1, The condition information of the current block above It includes the ratio of the width and height of the current block, the slice type, the QP (Quantization parameter), and information applied in the surrounding blocks of the current block, The step of setting the above reference area A video decoding method characterized by determining the number of reference lines of a reference area used in the MIP based on condition information of the current block.

4. In Paragraph 1, The step of setting the above reference area Based on the size of the current block, the size of the upper-right surrounding pixels and the number of lower-left surrounding pixels of the current block included in the reference area are determined, A video decoding method characterized in that the number of surrounding pixels included in the above reference area is less than or equal to a predetermined threshold.

5. In Paragraph 4, A video decoding method characterized by determining surrounding pixels included in the reference area using 2W upper surrounding pixels and 2H left surrounding pixels, where the size of the current block is W*H (W is the width of the current block and H is the height of the current block).

6. In Paragraph 1, The step of setting the above reference area A video decoding method characterized by determining one of a reference mode that includes both the left peripheral pixels and the upper peripheral pixels of the current block in the reference area of ​​the MIP, a second reference mode that includes only the upper peripheral pixels of the current block in the reference area of ​​the MIP, and a third reference mode that includes only the left peripheral pixels of the current block in the reference area of ​​the MIP, and setting the reference area using the peripheral pixels according to the reference mode.

7. In Paragraph 1, The step of setting the above reference area A step of determining whether surrounding pixels of the current block are available; and A video decoding method further comprising the step of determining the pixel value of an unavailable surrounding pixel using at least one of interpolation, extrapolation, average or weighted average of available surrounding pixels, transpose of available surrounding pixels, extension of available surrounding pixels, and wrap-around pixels when the number of available surrounding pixels is smaller than the number of surrounding pixels required for the above MIP.

8. In Paragraph 1, The step of setting the above reference area A video decoding method further comprising the step of adaptively performing linear filtering on surrounding pixels included in the above reference region.

9. In Paragraph 7, A video decoding method in which, when the reference area contains surrounding pixels of multiple reference lines, linear filtering is performed only on surrounding pixels included in some of the multiple reference lines.

10. In Paragraph 1, The step of determining the matrix used in the above MIP is A video decoding method characterized by determining the matrix based on at least one of the size of the current block and the intra-prediction mode information of the current block.

11. In Paragraph 1, The step of determining the matrix used in the above MIP is A video decoding method characterized by obtaining a subsampled matrix used for the MIP of the current block by subsampling elements included in a reference matrix (Fmax) set for the largest block size available for the above MIP based on the size of the current block.

12. In Paragraph 11, The step of determining the matrix used in the above MIP is A video decoding method characterized by further including a step of normalizing each element included in the above-mentioned subsampled matrix.

13. A video decoding device that performs the method of any one of claims 1 to 12.

14. A computer-readable recording medium storing a program for executing the method of any one of paragraphs 1 through 12 on a computer.

15. A step of performing intra prediction for the current block by applying a plurality of intra prediction modes, including Matrix-based Intra prediction (MIP), to the current block; A step of generating flag information indicating whether the MIP is applied to the current block; and The method includes the step of encoding the above flag information and the intra prediction mode information of the above current block, The step of performing intra prediction of the above current block is When performing MIP intra prediction for the current block, a step of setting a reference region used for the MIP based on condition information of the current block, including the size and shape of the current block; A step of determining a matrix used in the above MIP; and A video encoding method comprising the step of obtaining a predicted sample of the current block based on the reference region and matrix.

16. A video encoding device that performs the method of paragraph 15.

17. A computer-readable recording medium storing a program for executing the method of paragraph 15 on a computer.

Images