Video signal processing method and device therefor

Abstract

A video signal decoding device may comprise a processor, wherein the processor derives an intra prediction mode for prediction of a current block, the intra prediction mode is derived from one or more neighboring blocks of the current block, the intra prediction mode is derived from a prediction mode stored in an intra prediction mode (IPM) buffer when an encoding mode of the one or more neighboring blocks is not an intra mode, and the current block is predicted on the basis of the derived intra prediction mode.

Description

Video signal processing method and device therefor

[0001] The present invention relates to a method and device for processing a video signal, and more particularly, to a method and device for processing a video signal for encoding or decoding a video signal.

[0002] Compression coding refers to a series of signal processing techniques for transmitting digitized information over communication lines or storing it in a format suitable for storage media. Objects subject to compression coding include audio, video, and text, and the technology that performs compression coding on video in particular is called video compression. Compression coding of video signals is achieved by removing redundant information by considering spatial, temporal, and probabilistic correlations. However, with the recent advancements in various media and data transmission media, there is a growing demand for more efficient video signal processing methods and devices.

[0003] The purpose of this specification is to provide a video signal processing method and a device therefor to improve the coding efficiency of a video signal.

[0004] The present specification provides a video signal processing method and a device therefor.

[0005] In the present specification, a video signal decoding device includes a processor, wherein the processor derives an intra prediction mode for prediction of a current block, the intra prediction mode is derived from one or more surrounding blocks of the current block, and when an encoding mode of the one or more surrounding blocks is not an intra mode, the intra prediction mode is derived from a prediction mode stored in an intra prediction mode (IPM) buffer, and the current block can be predicted based on the derived intra prediction mode.

[0006] In the present specification, a video signal encoding device includes a processor, wherein the processor can obtain a bitstream decoded by a decoding method. In addition, in the present specification, a computer-readable non-transitory storage medium can obtain a bitstream decoded by a decoding method.

[0007] In the present specification, a decoding method may include a step of deriving an intra prediction mode for predicting a current block, the intra prediction mode being derived from one or more surrounding blocks of the current block, and when an encoding mode of the one or more surrounding blocks is not an intra mode, the intra prediction mode being derived from a prediction mode stored in an intra prediction mode (IPM) buffer; and a step of predicting the current block based on the derived intra prediction mode.

[0008] In the present specification, the encoding mode of the one or more surrounding blocks may be any one of intra TMP (template matching prediction), EIP (extrapolation filter-based intra prediction) mode, inter mode, and IBC (intra block copy) mode.

[0009] In the present specification, the one or more surrounding blocks may be blocks located at least one of the left, lower left, upper, upper right, and upper left of the current block.

[0010] In the present specification, the mode stored in the IPM buffer may be an intra prediction mode related to one or more surrounding blocks.

[0011] In the present specification, the intra prediction mode related to the one or more surrounding blocks may be a prediction mode derived by applying DIMD (decoder side intra mode derivation) to the surrounding blocks of the one or more surrounding blocks.

[0012] In the present specification, the intra prediction mode related to the one or more surrounding blocks may be a prediction mode having the highest frequency among prediction modes derived by applying DIMD (decoder side intra mode derivation) to the surrounding blocks of the one or more surrounding blocks.

[0013] This specification provides a method for efficiently processing a video signal.

[0014] The effects that can be obtained from this specification are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention pertains from the description below.

[0015] FIG. 1 is a schematic block diagram of a video signal encoding device according to one embodiment of the present invention.

[0016] FIG. 2 is a schematic block diagram of a video signal decoding device according to one embodiment of the present invention.

[0017] Figure 3 illustrates an embodiment in which a coding tree unit within a picture is divided into coding units.

[0018] Figure 4 illustrates one embodiment of a method for signaling the splitting of a quad tree and a multi-type tree.

[0019] Figures 5 and 6 illustrate the intra prediction method according to an embodiment of the present invention in more detail.

[0020] Figure 7 is a diagram showing the locations of surrounding blocks used to construct a motion candidate list in inter prediction.

[0021] FIG. 8 is a diagram showing the relationship between a luma sample and a chroma sample according to one embodiment of the present invention.

[0022] FIG. 9 is a diagram showing a reference sample required for Cross-component Linear Model (CCLM) prediction according to one embodiment of the present invention.

[0023] FIG. 10 is a diagram showing a mode in which CCLM is applied according to one embodiment of the present invention.

[0024] FIG. 11 is a diagram illustrating a CCLM mode using two linear models according to one embodiment of the present specification.

[0025] Fig. 12 is a drawing showing a division structure of a block according to one embodiment of the present invention.

[0026] FIG. 13 is a diagram illustrating a method for optimizing a linear model for CCLM according to one embodiment of the present invention.

[0027] FIG. 14 is a diagram illustrating a gradient linear model (GLM) according to one embodiment of the present invention.

[0028] Figure 15 is a diagram showing a filter for obtaining a slope value used in GLM.

[0029] FIG. 16 illustrates an example of a reference region and filter shape used to derive CCCM parameters according to one embodiment of the present invention.

[0030] Figures 17 and 18 illustrate luminance samples before downsampling used to derive chrominance samples in CCCM-ND mode according to one embodiment of the present invention.

[0031] FIG. 19 shows a downsampling filter and sample positions applied to a luminance block according to one embodiment of the present invention.

[0032] FIG. 20 is a diagram illustrating a direct block vector according to one embodiment of the present invention.

[0033] Figures 21 and 22 show additional locations for application of DBV according to one embodiment of the present invention.

[0034] FIG. 23 is a diagram showing a type of conversion kernel according to one embodiment of the present invention.

[0035] FIG. 24 is a diagram showing the 0th (lowest frequency component of the corresponding transform kernel) basis function of DCT-II, DCT-V, DCT-VIII, DST-I, and DST-VII transforms according to one embodiment of the present invention.

[0036] FIG. 25 and FIG. 26 are diagrams showing a set of transformation kernels according to one embodiment of the present invention.

[0037] FIG. 27 is a diagram illustrating a process for restoring a residual signal according to one embodiment of the present invention.

[0038] FIG. 28 is a diagram showing a region-of-interest (ROI) of a block to which a secondary transformation is applied according to one embodiment of the present invention.

[0039] FIG. 29 is a diagram showing a method of applying a second-order transform (LFNST) according to one embodiment of the present invention.

[0040] FIG. 30 is a diagram showing a mapping relationship between an intra prediction mode and a set of transformation kernels for secondary transformation according to one embodiment of the present invention.

[0041] FIG. 31 is a diagram showing the relationship between the input vector of the secondary transformation and the intra prediction mode according to one embodiment of the present invention.

[0042] Figure 32 shows the application block and kernel size of NSPT (Non-Separable Primary Transform) according to one embodiment of the present invention.

[0043] Fig. 33 illustrates an intra prediction mode derivation method for applying MTS / LFNST / NSPT according to one embodiment of the present invention.

[0044] Figure 34 illustrates syntax elements related to DBV according to one embodiment of the present invention.

[0045] Figure 35 illustrates the operation of BVG-CCCM (Block Vector Guided CCCM) according to one embodiment of the present invention.

[0046] Figure 36 illustrates syntax elements related to BVC-CCCM according to one embodiment of the present invention.

[0047] Figure 37 illustrates TIMD according to one embodiment of the present invention.

[0048] FIG. 38 illustrates a method for generating a final prediction block using an inter prediction block and an intra prediction block according to one embodiment of the present invention.

[0049] FIG. 39 illustrates a method for deriving an intra prediction mode from a luma block corresponding to a chroma block according to one embodiment of the present invention.

[0050] FIG. 40 illustrates a method for determining a reference pixel line based on a template according to one embodiment of the present invention.

[0051] Fig. 41 shows a block vector related to an IBC encoding method according to one embodiment of the present invention.

[0052] FIG. 42 illustrates a method for predicting a current block using RRIBC in the horizontal direction according to one embodiment of the present invention.

[0053] FIG. 43 illustrates a method for predicting a current block using RRIBC in the vertical direction according to one embodiment of the present invention.

[0054] Figure 44 shows a block vector of a block encoded in Intra TMP mode according to one embodiment of the present invention.

[0055] Figure 45 illustrates a case where a current block is divided by GPM mode according to one embodiment of the present invention and the divided area is encoded in IBC mode.

[0056] Figure 46 illustrates how a current block is encoded in IBC-CIIP mode according to one embodiment of the present invention.

[0057] Figure 47 illustrates a method for deriving a multi-transform set and a LFNST / NSPT set according to one embodiment of the present invention.

[0058] Figure 48 shows a mapping table according to one embodiment of the present invention.

[0059] Figure 49 illustrates a conversion type set table according to one embodiment of the present invention.

[0060] Figure 50 shows a conversion type combination table according to one embodiment of the present invention.

[0061] Figure 51 shows a threshold value table for an IDT conversion type according to one embodiment of the present invention.

[0062] Figure 52 shows block boundaries and samples around the boundaries in a deblocking filtering process according to one embodiment of the present invention.

[0063] Figures 53 and 54 illustrate the concept of matrix-based intra-screen prediction according to one embodiment of the present invention.

[0064] FIG. 55 is a diagram showing the locations of surrounding pixels used to derive directional information according to one embodiment of the present invention.

[0065] FIG. 56 is a diagram illustrating a method for mapping directional modes according to one embodiment of the present invention.

[0066] FIG. 57 is a diagram showing a histogram for deriving an intra prediction directional mode according to one embodiment of the present invention.

[0067] Figure 58 shows samples used when an intra prediction mode is derived in a DIMD manner for a block to which MIP is applied according to one embodiment of the present invention.

[0068] FIG. 59 shows a positional relationship for sample-based blending when performing a DIMD prediction method according to one embodiment of the present invention.

[0069] Figures 60 to 62 illustrate an EIP (Extrapolation filter-based intra prediction mode) according to one embodiment of the present invention.

[0070] FIG. 63 illustrates a method for deriving an intra prediction mode for inter prediction according to one embodiment of the present invention.

[0071] Figure 64 is a flowchart illustrating a method for predicting a current block according to one embodiment of the present invention.

[0072] The terms used in this specification have been selected from widely used and current terms, taking into account the functions of the present invention. However, these terms may vary depending on the intentions of those skilled in the art, customs, or the emergence of new technologies. Furthermore, in certain cases, the applicant may arbitrarily select terms, in which case their meanings will be described in the description of the relevant invention. Therefore, it should be noted that the terms used in this specification should be interpreted based on their substantive meaning and the overall content of this specification, rather than simply their names.

[0073] In this specification, 'A and / or B' may be interpreted to mean 'comprising at least one of A or B'.

[0074] In this specification, some terms may be interpreted as follows. Coding may be interpreted as encoding or decoding, depending on the case. In this specification, a device that encodes a video signal to generate a video signal bitstream is referred to as an encoding device or encoder, and a device that decodes a video signal bitstream to restore a video signal is referred to as a decoding device or decoder. In addition, in this specification, a video signal processing device is used as a term that includes both an encoder and a decoder. Information is a term that includes values, parameters, coefficients, elements, etc., and since the meaning may be interpreted differently depending on the case, the present invention is not limited thereto. 'Unit' is used to mean a basic unit of image processing or a specific location of a picture, and refers to an image area that includes at least one of a luminance component and a chroma component. In addition, 'block' refers to an image area including specific components among luminance components and chrominance components (i.e., Cb and Cr). However, depending on the embodiment, terms such as 'unit', 'block', 'partition', 'signal', and 'region' may be used interchangeably. In addition, in this specification, 'current block' means a block that is currently scheduled to be encoded, and 'reference block' means a block that has already been encoded or decoded and is used as a reference in the current block. In addition, in this specification, terms such as 'luma', 'luminance', and 'Y' may be used interchangeably. In addition, in this specification, terms such as 'chroma', 'chroma', 'color difference', and 'Cb or Cr' may be used interchangeably, and since chrominance components are divided into two, Cb and Cr, each chrominance component may be used separately. In addition, in this specification, a unit may be used as a concept including all of a coding unit, a prediction unit, and a transformation unit.A picture refers to a field or a frame, and depending on the embodiment, the terms may be used interchangeably. Specifically, if the captured image is an interlace image, one frame is divided into an odd (or odd, top) field and an even (or even, bottom) field, and each field is configured as one picture unit and can be encoded or decoded. If the captured image is a progressive image, one frame is configured as a picture and can be encoded or decoded. In addition, in this specification, the terms 'error signal', 'residual signal', 'residual signal', 'residual signal', and 'differential signal' may be used interchangeably. In addition, in this specification, the terms 'intra prediction mode', 'intra prediction directional mode', 'intra-screen prediction mode', and 'intra-screen prediction directional mode' may be used interchangeably. In addition, in this specification, the terms 'motion', 'movement', and the like may be used interchangeably. In addition, in this specification, 'left', 'upper left', 'upper left', 'upper right', 'right', 'lower right', 'lower left', and 'lower left' can be used interchangeably with 'left', 'upper left', 'top', 'upper right', 'right', 'lower right', 'bottom', and 'lower left'. In addition, element and member can be used interchangeably with each other. POC (Picture Order Count) indicates the temporal position information of a picture (or frame), can be the playback order displayed on the screen, and can have a unique POC for each picture.

[0075] FIG. 1 is a schematic block diagram of a video signal encoding device (100) according to one embodiment of the present invention. Referring to FIG. 1, the encoding device (100) of the present invention includes a transformation unit (110), a quantization unit (115), an inverse quantization unit (120), an inverse transformation unit (125), a filtering unit (130), a prediction unit (150), and an entropy coding unit (160).

[0076] The transform unit (110) obtains a transform coefficient value by transforming the residual signal, which is the difference between the input video signal and the prediction signal generated by the prediction unit (150). For example, a discrete cosine transform (DCT), a discrete sine transform (DST), or a wavelet transform may be used. The discrete cosine transform and the discrete sine transform divide the input picture signal into blocks and perform the transformation. The coding efficiency may vary depending on the distribution and characteristics of the values ​​within the transform domain during the transformation. The transform kernel used for the transformation on the residual block may be a transform kernel having separable characteristics of vertical transformation and horizontal transformation. In this case, the transformation on the residual block may be performed separately as vertical transformation and horizontal transformation. For example, the encoder may perform vertical transformation by applying the transform kernel in the vertical direction of the residual block. Additionally, the encoder can perform horizontal transformation by applying a transformation kernel in the horizontal direction of the residual block. In the present disclosure, the transformation kernel may be used as a term referring to a set of parameters used for transformation of the residual signal, such as a transformation matrix, a transformation array, a transformation function, or a transformation. For example, the transformation kernel may be any one of a plurality of available kernels. Additionally, transformation kernels based on different transformation types may be used for each of the vertical transformation and the horizontal transformation.

[0077] The transformation coefficients are distributed in a way that increases toward the upper left corner of the block, and decreases toward 0 toward the lower right corner. As the current block size increases, there is a high probability of a significant number of 0 coefficients in the lower right corner. To reduce the transformation complexity of large blocks, the remaining regions can be reset to 0, leaving only the upper left corner as an arbitrary region.

[0078] Additionally, error signals may exist only in some regions of a coding block. In this case, the conversion process may be performed only on some arbitrary regions. For example, in a block of size 2Nx2N, an error signal may exist only in the first 2NxN block, and the conversion process may be performed only on the first 2NxN block, but the conversion process may not be performed on the second 2NxN block and may not be encoded or decoded. Here, N can be any positive integer.

[0079] The encoder may perform an additional transform before the transform coefficients are quantized. The aforementioned transform method may be referred to as a primary transform, and the additional transform may be referred to as a secondary transform. The secondary transform may be optional for each residual block. In one embodiment, the encoder may improve coding efficiency by performing the secondary transform on areas where it is difficult to concentrate energy in the low-frequency region using only the primary transform. For example, the secondary transform may be additionally performed on blocks where residual values ​​appear large in directions other than the horizontal or vertical direction of the residual block. Unlike the primary transform, the secondary transform may not be performed separately into a vertical transform and a horizontal transform. Such a secondary transform may be referred to as a Low Frequency Non-Separable Transform (LFNST).

[0080] The quantization unit (115) quantizes the transformation coefficient value output from the transformation unit (110).

[0081] In order to increase coding efficiency, rather than coding the picture signal as it is, a method is used to predict a picture using an already coded area through a prediction unit (150), and to obtain a restored picture by adding the residual value between the original picture and the predicted picture to the predicted picture. In order to prevent mismatches from occurring in the encoder and decoder, when the encoder performs prediction, information that is also available to the decoder must be used. To this end, the encoder performs a process of restoring the encoded current block. The inverse quantization unit (120) inversely quantizes the transform coefficient value, and the inverse transform unit (125) restores the residual value using the inverse quantized transform coefficient value. Meanwhile, the filtering unit (130) performs a filtering operation to improve the quality of the restored picture and enhance the coding efficiency. For example, a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter may be included. The filtered picture is stored in the Decoded Picture Buffer (DPB, 156) to be output or used as a reference picture.

[0082] A deblocking filter is a filter that removes distortion within blocks generated at the boundaries between blocks in a reconstructed picture. The encoder can determine whether to apply a deblocking filter to a certain boundary based on the distribution of pixels in several columns or rows based on an arbitrary boundary (edge) within the block. When applying a deblocking filter to a block, the encoder can apply a long filter, a strong filter, or a weak filter depending on the deblocking filtering strength. Additionally, horizontal and vertical filtering can be processed in parallel. Sample adaptive offset (SAO) can be used to correct the offset from the original image on a pixel-by-pixel basis for the residual block to which the deblocking filter has been applied. To correct the offset for a specific picture, the encoder can divide the pixels contained in the image into a certain number of regions, determine the regions to perform offset correction, and apply the offset to those regions (Band Offset). Alternatively, the encoder can use a method (Edge Offset) that applies an offset by considering the edge information of each pixel. An adaptive loop filter (ALF) is a method that divides pixels included in an image into a predetermined group, determines a filter to be applied to each group, and performs filtering differently for each group. Information regarding whether to apply ALF can be signaled on a coding unit basis, and the shape and filter coefficients of the ALF filter to be applied can vary depending on each block. In addition, an ALF filter of the same shape (fixed shape) can be applied regardless of the characteristics of the target block to which it is applied.

[0083] The prediction unit (150) includes an intra prediction unit (152) and an inter prediction unit (154). The intra prediction unit (152) performs intra prediction within a current picture, and the inter prediction unit (154) performs inter prediction to predict the current picture using a reference picture stored in a decoded picture buffer (156). The intra prediction unit (152) performs intra prediction from reconstructed regions within the current picture and transfers intra encoding information to the entropy coding unit (160). The intra encoding information may include at least one of an intra prediction mode, an MPM (Most Probable Mode) flag, an MPM index, and information about a reference sample. The inter prediction unit (154) may be configured to include a motion estimation unit (154a) and a motion compensation unit (154b). The motion estimation unit (154a) refers to a specific area of ​​the restored reference picture to find the part most similar to the current area and obtains a motion vector value, which is the distance between the areas. The motion information (reference direction indication information (L0 prediction, L1 prediction, bidirectional prediction), reference picture index, motion vector information, etc.) for the reference area obtained by the motion estimation unit (154a) is transferred to the entropy coding unit (160) so that it can be included in the bitstream. Using the motion information transferred from the motion estimation unit (154a), the motion compensation unit (154b) performs inter-motion compensation to generate a prediction block for the current block. The inter-prediction unit (154) transfers inter-encoding information including motion information for the reference area to the entropy coding unit (160).

[0084] According to an additional embodiment, the prediction unit (150) may include an intra block copy (IBC) prediction unit (not shown). The IBC prediction unit performs IBC prediction on reconstructed samples in the current picture and transfers IBC encoding information to the entropy coding unit (160). The IBC prediction unit obtains a block vector value indicating a reference region used for prediction of the current region by referring to a specific region in the current picture. The IBC prediction unit may perform IBC prediction using the obtained block vector value. The IBC prediction unit transfers the IBC encoding information to the entropy coding unit (160). The IBC encoding information may include at least one of size information of the reference region, block vector information (index information for block vector prediction of the current block within a motion candidate list, and block vector difference information).

[0085] When the picture prediction as above is performed, the transformation unit (110) obtains a transformation coefficient value by transforming the residual value between the original picture and the predicted picture. At this time, the transformation can be performed in units of specific blocks within the picture, and the size of the specific block can be varied within a preset range. The quantization unit (115) quantizes the transformation coefficient value generated by the transformation unit (110) and transfers the quantized transformation coefficient to the entropy coding unit (160).

[0086] The quantized transform coefficients in the form of a two-dimensional array can be rearranged into a one-dimensional array for entropy coding. The method of scanning the quantized transform coefficients can be determined by which scanning method is used depending on the size of the transform block and the prediction mode within the screen. For example, diagonal, vertical, and horizontal scanning can be applied. This scanning information can be signaled on a block-by-block basis and can be derived according to predetermined rules.

[0087] The entropy coding unit (160) entropy-codes information representing quantized transform coefficients, intra-coding information, inter-coding information, etc. to generate a video signal bitstream. The entropy coding unit (160) may use a variable length coding (VLC) method and an arithmetic coding method. The variable length coding (VLC) method converts input symbols into continuous codewords, and the length of the codewords may be variable. For example, frequently occurring symbols are expressed as short codewords, and infrequently occurring symbols are expressed as long codewords. A context-based adaptive variable length coding (CAVLC) method may be used as a variable length coding method. Arithmetic coding converts continuous data symbols into a single prime number by using the probability distribution of each data symbol, and arithmetic coding can obtain the optimal prime number bits required to express each symbol. Context-based Adaptive Binary Arithmetic Code (CABAC) can be used as an arithmetic coding.

[0088] CABAC is a binary arithmetic coding method that uses multiple context models generated based on experimentally obtained probabilities. The context models can also be referred to as context models. First, if the symbols are not in binary form, the encoder binarizes each symbol using exp-Golomb, etc. The binarized 0 or 1 can be described as a bin. The CABAC initialization process is divided into context initialization and arithmetic coding initialization. Context initialization is the process of initializing the occurrence probability of each symbol, and is determined by the symbol type, quantization parameter (QP), and slice type (I, P, B). A context model with this initialization information can use probability-based values ​​obtained through experiments. The context model provides the occurrence probability of the Least Probable Symbol (LPS) or Most Probable Symbol (MPS) for the symbol to be currently encoded, as well as information (valMPS) on which bin value corresponds to the MPS between 0 and 1. One of several context models is selected through the context index (ctxIdx), and the context index can be derived from information about the current block to be encoded or information about the surrounding blocks. Initialization for binary arithmetic coding is performed based on the probability model selected from the context model. Binary arithmetic coding is performed by dividing the data into probability intervals based on the occurrence probabilities of 0 and 1, and then encoding is performed through a process in which the probability interval corresponding to the bin to be processed becomes the entire probability interval for the bin to be processed next. The location information within the probability interval in which the last bin has been processed is output. However, since the probability interval cannot be divided infinitely, if it is reduced to a certain size, a renormalization process is performed to expand the probability interval and output the corresponding location information. In addition, after each bin is processed, a probability update process can be performed in which the probability for the next bin to be processed is newly set based on the information of the processed bin.

[0089] The above generated bitstream is encapsulated into NAL (Network Abstraction Layer) units as basic units. NAL units are divided into VCL (Video Coding Layer) NAL units containing video data and non-VCL NAL units containing parameter information for decoding video data, and there are various types of VCL or non-VCL NAL units. A NAL unit consists of NAL header information and RBSP (Raw Byte Sequence Payload) data, and the NAL header information includes summary information about the RBSP. The RBSP of a VCL NAL unit includes an integer number of encoded coding tree units. In order for a video decoder to decode a bitstream, the bitstream must first be divided into NAL unit units, and then each divided NAL unit must be decoded. Meanwhile, information necessary for decoding a video signal bitstream can be transmitted by being included in a Picture Parameter Set (PPS), a Sequence Parameter Set (SPS), a Video Parameter Set (VPS), etc.

[0090] Meanwhile, the block diagram of FIG. 1 illustrates an encoding device (100) according to one embodiment of the present invention, and the blocks shown separately logically distinguish elements of the encoding device (100). Accordingly, the elements of the encoding device (100) described above may be mounted as one chip or as multiple chips depending on the design of the device. According to one embodiment, the operations of each element of the encoding device (100) described above may be performed by a processor (not shown).

[0091] Fig. 2 is a schematic block diagram of a video signal decoding device (200) according to one embodiment of the present invention. Referring to Fig. 2, the decoding device (200) of the present invention includes an entropy decoding unit (210), an inverse quantization unit (220), an inverse transformation unit (225), a filtering unit (230), and a prediction unit (250).

[0092] The entropy decoding unit (210) entropy decodes the video signal bitstream to extract transform coefficient information, intra-coding information, inter-coding information, etc. for each region. For example, the entropy decoding unit (210) can obtain a binarization code for transform coefficient information of a specific region from the video signal bitstream. In addition, the entropy decoding unit (210) inversely binarizes the binarization code to obtain a quantized transform coefficient. The inverse quantization unit (220) inversely quantizes the quantized transform coefficient, and the inverse transform unit (225) restores the residual value using the inverse quantized transform coefficient. The video signal processing device (200) restores the original pixel value by adding the residual value obtained by the inverse transform unit (225) and the prediction value obtained by the prediction unit (250).

[0093] Meanwhile, the filtering unit (230) performs filtering on the picture to improve the image quality. This may include a deblocking filter to reduce block distortion and / or an adaptive loop filter to remove distortion from the entire picture. The filtered picture is output or stored in the decoded picture buffer (DPB, 256) to be used as a reference picture for the next picture.

[0094] The prediction unit (250) includes an intra prediction unit (252) and an inter prediction unit (254). The prediction unit (250) generates a prediction picture by utilizing the encoding type decoded through the entropy decoding unit (210) described above, the transform coefficients for each region, intra / inter encoding information, etc. In order to restore the current block on which decoding is performed, the decoded region of the current picture or other pictures including the current block may be used. A picture (or tile / slice) that uses only the current picture for restoration, i.e., performs intra prediction or intra BC prediction, is called an intra picture or I picture (or tile / slice), and a picture (or tile / slice) that can perform all of intra prediction, inter prediction, and intra BC prediction is called an inter picture (or tile / slice). A picture (or tile / slice) that uses at most one motion vector and reference picture index to predict sample values ​​of each block among inter-pictures (or tiles / slices) is called a predictive picture or P-picture (or tile / slice), and a picture (or tile / slice) that uses at most two motion vectors and reference picture indices is called a bi-predictive picture or B-picture (or tile / slice). In other words, a P-picture (or tile / slice) uses at most one motion information set to predict each block, and a B-picture (or tile / slice) uses at most two motion information sets to predict each block. Here, a motion information set includes one or more motion vectors and one reference picture index.

[0095] The intra prediction unit (252) generates a prediction block using intra encoding information and reconstructed samples within the current picture. As described above, the intra encoding information may include at least one of an intra prediction mode, a Most Probable Mode (MPM) flag, and an MPM index. The intra prediction unit (252) predicts sample values ​​of the current block using reconstructed samples located on the left and / or above the current block as reference samples. In the present disclosure, the reconstructed samples, the reference samples, and the samples of the current block may represent pixels. Additionally, the sample values ​​may represent pixel values.

[0096] According to one embodiment, the reference samples may be samples included in a neighboring block of the current block. For example, the reference samples may be samples adjacent to a left boundary of the current block and / or samples adjacent to an upper boundary. In addition, the reference samples may be samples located on a line within a preset distance from a left boundary of the current block and / or samples located on a line within a preset distance from an upper boundary of the current block among samples of neighboring blocks of the current block. In this case, the neighboring blocks of the current block may include at least one of a left (L) block, an upper (A) block, a lower left (BL) block, an above right (AR) block, or an above left (AL) block adjacent to the current block.

[0097] The inter prediction unit (254) generates a prediction block using the reference picture and inter encoding information stored in the decoded picture buffer (256). The inter encoding information may include a set of motion information (reference picture index, motion vector information, etc.) of the current block with respect to the reference block. Inter prediction may include L0 prediction, L1 prediction, and bi-prediction. L0 prediction is prediction using one reference picture included in the L0 picture list, and L1 prediction means prediction using one reference picture included in the L1 picture list. For this, one set of motion information (e.g., motion vector and reference picture index) may be required. In the bi-prediction method, up to two reference areas can be used, and these two reference areas may exist in the same reference picture or may exist in different pictures, respectively. That is, in the bi-prediction method, up to two sets of motion information (e.g., motion vectors and reference picture indices) can be used, and the two motion vectors may correspond to the same reference picture index or may correspond to different reference picture indices. In this case, the reference pictures are pictures that are located temporally before or after the current picture, and may be completed pictures that have already been restored. According to one embodiment, the two reference areas used in the bi-prediction method may be areas selected from each of the L0 picture list and the L1 picture list.

[0098] The inter prediction unit (254) can obtain a reference block of the current block using a motion vector and a reference picture index. The reference block exists in a reference picture corresponding to the reference picture index. In addition, a sample value of a block specified by the motion vector or an interpolated value thereof can be used as a predictor of the current block. For motion prediction with sub-pel unit pixel accuracy, for example, an 8-tap interpolation filter can be used for a luminance signal and a 4-tap interpolation filter can be used for a chrominance signal. However, the interpolation filter for sub-pel unit motion prediction is not limited thereto. In this way, the inter prediction unit (254) performs motion compensation to predict the texture of the current unit from a previously restored picture. At this time, the inter prediction unit can use a motion information set.

[0099] According to an additional embodiment, the prediction unit (250) may include an IBC prediction unit (not shown). The IBC prediction unit may reconstruct the current region by referring to a specific region including reconstructed samples within the current picture. The IBC prediction unit may perform IBC prediction using IBC encoding information obtained from the entropy decoding unit (210). The IBC encoding information may include block vector information.

[0100] A restored video picture is generated by adding the predicted value output from the intra prediction unit (252) or inter prediction unit (254) and the residual value output from the inverse transformation unit (225). That is, the video signal decoding device (200) restores the current block using the predicted block generated from the prediction unit (250) and the residual obtained from the inverse transformation unit (225).

[0101] Meanwhile, the block diagram of FIG. 2 illustrates a decoding device (200) according to one embodiment of the present invention, and the blocks shown separately logically distinguish the elements of the decoding device (200). Accordingly, the elements of the aforementioned decoding device (200) may be mounted as one chip or as multiple chips depending on the design of the device. According to one embodiment, the operations of each element of the aforementioned decoding device (200) may be performed by a processor (not shown).

[0102] Meanwhile, the technology proposed in this specification is applicable to both the methods and devices of the encoder and decoder, and the parts described as signaling and parsing may be described for convenience of explanation. In general, signaling can be described as encoding each syntax from the encoder's perspective, and parsing can be described as interpreting each syntax from the decoder's perspective. That is, each syntax can be included in the bitstream from the encoder and signaled, and the decoder can parse the syntax and use it in the restoration process. At this time, the sequence of bits for each syntax listed in the prescribed hierarchical structure can be referred to as a bitstream.

[0103] A picture can be encoded by dividing it into sub-pictures, slices, tiles, etc. A sub-picture can include one or more slices or tiles. When a picture is encoded by dividing it into multiple slices or tiles, all slices or tiles within the picture must be decoded before it can be displayed on the screen. On the other hand, when a picture is encoded into multiple sub-pictures, only any sub-picture can be decoded and displayed on the screen. A slice can include multiple tiles or sub-pictures, or a tile can include multiple sub-pictures or slices. Sub-pictures, slices, and tiles can be encoded or decoded independently, which is effective for parallel processing and improving processing speed. However, there is a disadvantage in that the amount of bits increases because the encoded information of adjacent sub-pictures, slices, and tiles cannot be used. Sub-pictures, slices, and tiles can be encoded by dividing them into multiple coding tree units (CTUs).

[0104] FIG. 3 illustrates an embodiment in which a Coding Tree Unit (CTU) within a picture is divided into Coding Units (CUs). In the process of coding a video signal, a picture may be divided into a sequence of Coding Tree Units (CTUs). A Coding Tree Unit may be composed of a luminance (luma) Coding Tree Block (CTB), two chroma (chroma) Coding Tree Blocks, and their encoded syntax information. One Coding Tree Unit may be composed of one Coding Unit, or one Coding Tree Unit may be split into multiple Coding Units. One Coding Unit may be composed of a luminance Coding Block (CB), two chroma Coding Blocks, and their encoded syntax information. One Coding Block may be split into multiple Sub-Coding Blocks. One Coding Unit may be composed of one Transform Unit (TU), or one Coding Unit may be split into multiple Transform Units. A single transform unit may consist of a luminance transform block (TB), two chrominance transform blocks, and their encoded syntax information. A coding tree unit may be divided into multiple coding units. A coding tree unit may not be divided and may also be a leaf node. In this case, the coding tree unit itself may be a coding unit.

[0105] A coding unit refers to a basic unit for processing a picture in the video signal processing described above, such as intra / inter prediction, transformation, quantization, and / or entropy coding. The size and shape of a coding unit within a picture may not be constant. A coding unit may have a square or rectangular shape. A rectangular coding unit (or rectangular block) includes a vertical coding unit (or vertical block) and a horizontal coding unit (or horizontal block). In the present specification, a vertical block is a block whose height is greater than its width, and a horizontal block is a block whose width is greater than its height. In addition, a non-square block in the present specification may refer to a rectangular block, but the present invention is not limited thereto.

[0106] Referring to Fig. 3, the coding tree unit is first partitioned into a Quad Tree (QT) structure. That is, in the Quad Tree structure, one node with a size of 2NX2N can be partitioned into four nodes with a size of NXN. In this specification, the Quad Tree may also be referred to as a quaternary tree. The Quad Tree partitioning can be performed recursively, and not all nodes need to be partitioned to the same depth.

[0107] Meanwhile, the leaf node of the aforementioned quad tree can be further split into a multi-type tree (MTT) structure. According to an embodiment of the present invention, in the multi-type tree structure, one node can be split into a binary or ternary tree structure of horizontal or vertical split. That is, the multi-type tree structure has four split structures: vertical binary split, horizontal binary split, vertical ternary split, and horizontal ternary split. According to an embodiment of the present invention, in each of the above tree structures, the width and height of the node can both have a power of 2 value. For example, in the binary tree (BT) structure, a node of size 2NX2N can be split into two NX2N nodes by vertical binary split, and can be split into two 2NXN nodes by horizontal binary split. Also, in a Ternary Tree (TT) structure, a node of size 2NX2N can be split into nodes of size (N / 2)X2N, NX2N, and (N / 2)X2N by vertical ternary splitting, and into nodes of size 2NX(N / 2), 2NXN, and 2NX(N / 2) by horizontal ternary splitting. This multi-type tree splitting can be performed recursively.

[0108] A leaf node of a multi-type tree can be a coding unit. If the coding unit is not larger than the maximum transformation length, the coding unit can be used as a unit of prediction and / or transformation without further splitting. In one embodiment, if the width or height of the current coding unit is larger than the maximum transformation length, the current coding unit can be split into multiple transformation units without explicit signaling regarding the splitting. Meanwhile, in the quad tree and multi-type tree described above, at least one of the following parameters can be predefined or transmitted through an RBSP of a higher-level set, such as a PPS, an SPS, or a VPS. 1) CTU size: The size of the root node of the quad tree, 2) MinQtSize: The minimum allowed QT leaf node size, 3) MaxBtSize: The maximum allowed BT root node size, 4) MaxTT size (MaxTtSize): The maximum allowed TT root node size, 5) MaxMttDepth: The maximum allowed depth of an MTT split from a leaf node of a QT, 6) MinBT size (MinBtSize): The minimum allowed BT leaf node size, 7) MinTT size (MinTtSize): The minimum allowed TT leaf node size.

[0109] Fig. 4 illustrates one embodiment of a method for signaling splitting of a quad tree and a multi-type tree. Pre-configured flags may be used to signal splitting of the quad tree and the multi-type tree described above. Referring to Fig. 4, at least one of a flag 'split_cu_flag' indicating whether a node is split, a flag 'split_qt_flag' indicating whether a quad tree node is split, a flag 'mtt_split_cu_vertical_flag' indicating a splitting direction of a multi-type tree node, or a flag 'mtt_split_cu_binary_flag' indicating a splitting shape of a multi-type tree node may be used.

[0110] According to an embodiment of the present invention, a flag 'split_cu_flag' indicating whether a current node is split may be signaled first. If the value of 'split_cu_flag' is 0, it indicates that the current node is not split, and the current node becomes a coding unit. If the current node is a coding tree unit, the coding tree unit includes one coding unit that is not split. If the current node is a quad tree node 'QT node', the current node is a leaf node 'QT leaf node' of the quad tree and becomes a coding unit. If the current node is a multi-type tree node 'MTT node', the current node is a leaf node 'MTT leaf node' of the multi-type tree and becomes a coding unit.

[0111] When the value of 'split_cu_flag' is 1, the current node can be split into nodes of a quad tree or a multi-type tree depending on the value of 'split_qt_flag'. The coding tree unit is the root node of the quad tree and can be first split into a quad tree structure. In the quad tree structure, 'split_qt_flag' is signaled for each node 'QT node'. When the value of 'split_qt_flag' is 1, the node is split into four square nodes, and when the value of 'split_qt_flag' is 0, the node becomes a leaf node 'QT leaf node' of the quad tree, and the node is split into multi-type nodes. According to an embodiment of the present invention, quad tree splitting can be limited depending on the type of the current node. Quad tree splitting may be allowed if the current node is a coding tree unit (root node of a quad tree) or a quad tree node, and quad tree splitting may not be allowed if the current node is a multi-type tree node. Each quad tree leaf node 'QT leaf node' may be further split into a multi-type tree structure. As described above, if 'split_qt_flag' is 0, the current node may be split into multi-type nodes. To indicate the splitting direction and splitting shape, 'mtt_split_cu_vertical_flag' and 'mtt_split_cu_binary_flag' may be signaled. If the value of 'mtt_split_cu_vertical_flag' is 1, a vertical split of the node 'MTT node' is indicated, and if the value of 'mtt_split_cu_vertical_flag' is 0, a horizontal split of the node 'MTT node' is indicated.Additionally, if the value of 'mtt_split_cu_binary_flag' is 1, the node 'MTT node' is split into two rectangular nodes, and if the value of 'mtt_split_cu_binary_flag' is 0, the node 'MTT node' is split into three rectangular nodes.

[0112] The tree partitioning structure allows luminance blocks and chrominance blocks to be partitioned in the same manner. That is, the chrominance blocks can be partitioned by referring to the partitioning form of the luminance blocks. If the current chrominance block is smaller than a predetermined size, the chrominance block may not be partitioned even if the luminance block is partitioned.

[0113] The tree partitioning structure may have different forms for luminance blocks and chrominance blocks. In this case, partitioning information for the luminance block and partitioning information for the chrominance block may be signaled separately. Furthermore, not only the partitioning information but also the encoding information for the luminance block and chrominance block may differ. For example, at least one of the intra-coding modes of the luminance block and the chrominance block, and the encoding information for motion information, may differ.

[0114] A node to be divided into the smallest unit can be processed as a single coding block. If the current block is a coding block, the coding block can be divided into multiple sub-blocks (sub-coding blocks), and the prediction information of each sub-block can be the same or different. For example, if the coding unit is an intra mode, the intra prediction modes of each sub-block can be the same or different. Furthermore, if the coding unit is an inter mode, the motion information of each sub-block can be the same or different. Furthermore, each sub-block can be encoded or decoded independently. Each sub-block can be distinguished by a sub-block index (sbIdx). Furthermore, when a coding unit is divided into sub-blocks, it can be divided horizontally, vertically, or diagonally. In intra mode, the mode that divides the current coding unit into two or four sub-blocks horizontally or vertically is called ISP (Intra Sub-Partitions). In inter mode, the mode that divides the current coding block diagonally is called GPM (Geometric partitioning mode). In GPM mode, the position and direction of the diagonal line are derived using a predefined angle table, and the index information of the angle table is signaled.

[0115] Picture prediction (motion compensation) for coding is performed on coding units that are no longer divisible (i.e., leaf nodes of a coding tree unit). The basic unit performing this prediction is referred to below as a prediction unit or prediction block.

[0116] Hereinafter, the term "unit" used in this specification may be used as a replacement for the prediction unit, which is the basic unit for performing prediction. However, the present invention is not limited thereto, and can be understood more broadly as a concept that includes the coding unit.

[0117] Figures 5 and 6 illustrate an intra prediction method according to an embodiment of the present invention in more detail. As described above, the intra prediction unit predicts sample values ​​of the current block using restored samples located to the left and / or above the current block as reference samples.

[0118] First, FIG. 5 illustrates an embodiment of reference samples used for prediction of a current block in an intra prediction mode. According to an embodiment, the reference samples may be samples adjacent to a left boundary of the current block and / or samples adjacent to an upper boundary. As illustrated in FIG. 5, when the size of the current block is WXH and samples of a single reference line adjacent to the current block are used for intra prediction, reference samples may be set using at most 2W+2H+1 surrounding samples located on the left and / or upper sides of the current block.

[0119] Meanwhile, pixels of multiple reference lines may be used for intra prediction of the current block. The multiple reference lines may be composed of n lines located within a preset range from the current block. According to one embodiment, when pixels of multiple reference lines are used for intra prediction, separate index information indicating the lines to be set as reference pixels may be signaled, and this may be referred to as a reference line index.

[0120] In addition, if at least some of the samples to be used as reference samples have not yet been restored, the intra prediction unit may perform a reference sample padding process to obtain a reference sample. In addition, the intra prediction unit may perform a reference sample filtering process to reduce an error in intra prediction. That is, filtering may be performed on the surrounding samples and / or the reference samples obtained by the reference sample padding process to obtain filtered reference samples. The intra prediction unit predicts samples of the current block using the reference samples obtained in this manner. The intra prediction unit predicts samples of the current block using unfiltered reference samples or filtered reference samples. In the present disclosure, the surrounding samples may include samples on at least one reference line. For example, the surrounding samples may include adjacent samples on a line adjacent to a boundary of the current block.

[0121] Next, FIG. 6 illustrates an embodiment of prediction modes used for intra prediction. For intra prediction, intra prediction mode information indicating an intra prediction direction may be signaled. The intra prediction mode information indicates any one of a plurality of intra prediction modes constituting an intra prediction mode set. If the current block is an intra prediction block, the decoder receives intra prediction mode information of the current block from the bitstream. The intra prediction unit of the decoder performs intra prediction on the current block based on the extracted intra prediction mode information.

[0122] According to an embodiment of the present invention, an intra prediction mode set may include all intra prediction modes used for intra prediction (e.g., a total of 67 intra prediction modes). More specifically, the intra prediction mode set may include a planar mode, a DC mode, and a plurality of (e.g., 65) angular modes (i.e., directional modes). Each intra prediction mode may be indicated by a preset index (i.e., an intra prediction mode index). For example, as illustrated in FIG. 6, an intra prediction mode index 0 indicates a planar mode, and an intra prediction mode index 1 indicates a DC mode. In addition, intra prediction mode indices 2 to 66 may each indicate different angular modes. The angular modes each indicate different angles within a preset angular range. For example, an angular mode may indicate an angle within an angular range from 45 degrees to -135 degrees clockwise (i.e., a first angular range). The angular mode may be defined based on the 12 o'clock direction. At this time, intra prediction mode index 2 indicates horizontal diagonal (HDIA) mode, intra prediction mode index 18 indicates horizontal (HOR) mode, intra prediction mode index 34 indicates diagonal (DIA) mode, intra prediction mode index 50 indicates vertical (VER) mode, and intra prediction mode index 66 indicates vertical diagonal (VDIA) mode.

[0123] Meanwhile, the preset angle range may be set differently depending on the shape of the current block. For example, if the current block is a rectangular block, a wide-angle mode indicating an angle exceeding 45 degrees or less than -135 degrees in a clockwise direction may be additionally used. If the current block is a horizontal block, the angle mode may indicate an angle within an angle range (i.e., a second angle range) between (45+offset1) degrees and (-135+offset1) degrees in a clockwise direction. At this time, angle modes 67 to 76 that are outside the first angle range may be additionally used. In addition, if the current block is a vertical block, the angle mode may indicate an angle within an angle range (i.e., a third angle range) between (45-offset2) degrees and (-135-offset2) degrees in a clockwise direction. At this time, angle modes -10 to -1 that are outside the first angle range may be additionally used. According to an embodiment of the present invention, the values ​​of offset1 and offset2 may be determined differently depending on the ratio between the width and height of the rectangular block. Also, offset1 and offset2 can be positive.

[0124] According to an additional embodiment of the present invention, the plurality of angular modes constituting the intra prediction mode set may include a basic angular mode and an extended angular mode. In this case, the extended angular mode may be determined based on the basic angular mode.

[0125] According to one embodiment, the basic angle mode may be a mode corresponding to an angle used in intra prediction of the existing HEVC (High Efficiency Video Coding) standard, and the extended angle mode may be a mode corresponding to an angle newly added in intra prediction of the next-generation video codec standard. More specifically, the basic angle mode may be an angle mode corresponding to any one of the intra prediction modes {2, 4, 6, … , 66}, and the extended angle mode may be an angle mode corresponding to any one of the intra prediction modes {3, 5, 7, … , 65}. That is, the extended angle mode may be an angle mode between the basic angle modes within the first angle range. Therefore, the angle indicated by the extended angle mode may be determined based on the angle indicated by the basic angle mode.

[0126] According to another embodiment, the basic angle mode may be a mode corresponding to an angle within a preset first angle range, and the extended angle mode may be a wide-angle mode outside the first angle range. That is, the basic angle mode may be an angle mode corresponding to any one of the intra prediction modes {2, 3, 4, … , 66}, and the extended angle mode may be an angle mode corresponding to any one of the intra prediction modes {-14, –13, –12, … , –1} and {67, 68, … , 80}. The angle indicated by the extended angle mode may be determined as an opposite angle to the angle indicated by the corresponding basic angle mode. Accordingly, the angle indicated by the extended angle mode may be determined based on the angle indicated by the basic angle mode. Meanwhile, the number of extended angle modes is not limited thereto, and additional extended angles may be defined according to the size and / or shape of the current block. Meanwhile, the total number of intra prediction modes included in the intra prediction mode set may vary depending on the configuration of the basic angle mode and the extended angle mode described above.

[0127] In the above embodiment, the spacing between the extended angular modes may be set based on the spacing between the corresponding basic angular modes. For example, the spacing between the extended angular modes {3, 5, 7, … , 65} may be determined based on the spacing between the corresponding basic angular modes {2, 4, 6, … , 66}. In addition, the spacing between the extended angular modes {-14, -13, … , -1} may be determined based on the spacing between the corresponding opposite basic angular modes {53, 53, … , 66}, and the spacing between the extended angular modes {67, 68, … , 80} may be determined based on the spacing between the corresponding opposite basic angular modes {2, 3, 4, … , 15}. The angular spacing between the extended angular modes may be set to be equal to the angular spacing between the corresponding basic angular modes. In addition, the number of the extended angular modes in the intra prediction mode set may be set to be less than or equal to the number of the basic angular modes.

[0128] According to an embodiment of the present invention, an extended angular mode may be signaled based on a base angular mode. For example, a wide-angle mode (i.e., an extended angular mode) may replace at least one angular mode (i.e., a base angular mode) within a first angular range. The replaced base angular mode may be an angular mode corresponding to an opposite side of the wide-angle mode. That is, the replaced base angular mode is an angular mode corresponding to an angle in an opposite direction to an angle indicated by the wide-angle mode or an angle that is different from the angle in the opposite direction by a preset offset index. According to an embodiment of the present invention, the preset offset index is 1. An intra prediction mode index corresponding to the replaced base angular mode may be remapped to the wide-angle mode to signal the corresponding wide-angle mode. For example, the wide-angle modes {-14, -13, … , -1} may be signaled by intra prediction mode indices {52, 53, … , 66}, and the wide-angle modes {67, 68, … , 69} may be signaled by intra prediction mode indices {52, 53, … , 66}, respectively. , 80} can be signaled by intra prediction mode indices {2, 3, … , 15}, respectively. By having the intra prediction mode index for the basic angular mode signal the extended angular mode in this way, even if the configurations of the angular modes used for intra prediction of each block are different, the same set of intra prediction mode indices can be used to signal the intra prediction mode. Therefore, the signaling overhead due to changes in the intra prediction mode configuration can be minimized.

[0129] Meanwhile, whether to use the extended angle mode may be determined based on at least one of the shape and size of the current block. In one embodiment, if the size of the current block is larger than a preset size, the extended angle mode may be used for intra prediction of the current block, and otherwise, only the default angle mode may be used for intra prediction of the current block. In another embodiment, if the current block is a non-square block, the extended angle mode may be used for intra prediction of the current block, and if the current block is a square block, only the default angle mode may be used for intra prediction of the current block.

[0130] The intra prediction unit determines reference samples and / or interpolated reference samples to be used for intra prediction of the current block based on intra prediction mode information of the current block. If the intra prediction mode index indicates a specific angular mode, the reference sample or interpolated reference sample corresponding to the specific angle from the current sample of the current block is used for prediction of the current pixel. Therefore, different sets of reference samples and / or interpolated reference samples can be used for intra prediction depending on the intra prediction mode. After intra prediction of the current block is performed using the reference samples and intra prediction mode information, the decoder restores the sample values ​​of the current block by adding the residual signal of the current block obtained from the inverse transform unit to the intra prediction value of the current block.

[0131] Motion information used for inter prediction may include reference direction indication information (inter_pred_idc), reference picture indexes (ref_idx_l0, ref_idx_l1), and motion vectors (mvL0, mvL1). Reference picture list utilization information (predFlagL0, predFlagL1) may be set according to the reference direction indication information. As an example, in case of unidirectional prediction using an L0 reference picture, predFlagL0=1, predFlagL1=0 may be set. In case of unidirectional prediction using an L1 reference picture, predFlagL0=0, predFlagL1=1 may be set. In case of bidirectional prediction using both L0 and L1 reference pictures, predFlagL0=1, predFlagL1=1 may be set.

[0132] If the current block is a coding unit, the coding unit can be divided into multiple sub-blocks, and the prediction information of each sub-block can be the same or different. For example, if the coding unit is an intra mode, the intra prediction modes of each sub-block can be the same or different. In addition, if the coding unit is an inter mode, the motion information of each sub-block can be the same or different. In addition, each sub-block can be encoded or decoded independently. Each sub-block can be distinguished through a sub-block index (sbIdx).

[0133] The motion vector of the current block is likely to be similar to that of the surrounding blocks. Therefore, the motion vectors of the surrounding blocks can be used as motion vector predictors (mvp), and the motion vector of the current block can be derived using the motion vectors of the surrounding blocks. Furthermore, to improve the accuracy of the motion vector, the difference in motion vectors (mvd) between the optimal motion vector of the current block, found from the original image by the encoder, and the motion predictor can be signaled.

[0134] Motion vectors can have various resolutions, and the resolution of motion vectors can vary on a block-by-block basis. Motion vector resolution can be expressed in integer units, half-pixel units, quarter-pixel units, sixteenth-pixel units, and integer-4 pixel units. Since images such as screen content are in simple graphical forms such as characters, interpolation filters do not need to be applied, integer units and integer-4 pixel units can be selectively applied on a block-by-block basis. Blocks encoded in affine mode, which can express rotation and scale, have significant shape changes, so integer units, quarter-pixel units, and sixteenth-pixel units can be selectively applied on a block-by-block basis. Information on whether to selectively apply motion vector resolution on a block-by-block basis is signaled with amvr_flag. If applicable, which motion vector resolution to apply to the current block is signaled with amvr_precision_idx.

[0135] For blocks to which bidirectional prediction is applied, the weights between the two prediction blocks can be applied equally or differently when applying weighted average, and information about the weights is signaled through bcw_idx.

[0136] To improve the accuracy of the motion prediction value, Merge or advanced motion vector prediction (AMVP) methods can be selectively used on a block-by-block basis. The Merge method configures the motion information of the current block to be identical to the motion information of the adjacent blocks to the current block, and has the advantage of increasing the encoding efficiency of the motion information by spatially propagating the motion information without change in the homogeneous motion region. On the other hand, the AMVP method predicts motion information in the L0 and L1 prediction directions respectively to express accurate motion information and signals the most optimal motion information. After the decoder derives the motion information for the current block through the AMVP or Merge method, it uses the reference block located in the motion information derived from the reference picture as the prediction block for the current block.

[0137] A method for deriving motion information in Merge or AMVP may be to construct a motion candidate list using motion prediction values ​​derived from neighboring blocks of the current block, and then signal the index information for the optimal motion candidate. In the case of AMVP, since motion candidate lists are derived for each of L0 and L1, the optimal motion candidate indices (mvp_l0_flag, mvp_l1_flag) for each of L0 and L1 are signaled. In the case of Merge, since one motion candidate list is derived, one merge index (merge_idx) is signaled. The motion candidate lists derived from one coding unit may vary, and a motion candidate index or merge index may be signaled for each motion candidate list. In this case, a mode in which there is no information about the remaining blocks in a block encoded in Merge mode can be referred to as MergeSkip mode.

[0138] The motion candidates and motion information candidates in this specification may have the same meaning. Furthermore, the motion candidate list and motion information candidate list in this specification may have the same meaning.

[0139] SMVD (Symmetric MVD) is a method for reducing the bit volume of transmitted motion information by ensuring that the MVD (Motion Vector Difference) values ​​in the L0 and L1 directions are symmetrical in the case of bidirectional prediction. The MVD information in the L1 direction, which is symmetrical to the L0 direction, is not transmitted, and reference picture information in the L0 and L1 directions is also not transmitted and can be derived during the decoding process.

[0140] OBMC (Overlapped Block Motion Compensation) is a method that generates prediction blocks for the current block using motion information from surrounding blocks when the motion information between blocks differs. It then weights and averages these prediction blocks to generate a final prediction block for the current block. This method effectively reduces blocking artifacts that occur at block boundaries in motion-compensated images.

[0141] Typically, merge motion candidates have low motion accuracy. To improve the accuracy of these merge motion candidates, the MMVD (Merge mode with MVD) method can be used. The MMVD method is a method of compensating motion information using one candidate selected from several motion difference value candidates. Information about the compensation value of the motion information obtained through the MMVD method (e.g., an index indicating one candidate selected from among the motion difference value candidates) can be included in the bitstream and transmitted to the decoder. Compared to the conventional method of including the motion information difference value in the bitstream, the inclusion of information about the compensation value of the motion information in the bitstream can save bits.

[0142] The TM (Template Matching) method constructs a template using the surrounding pixels of the current block, finds the matching area with the highest similarity to the template, and then compensates for motion information. TM (Template Matching) is a method that performs motion prediction in the decoder without including motion information in the bitstream to reduce the size of the encoded bitstream. Since the decoder does not have the original image, it can roughly derive motion information for the current block using already reconstructed surrounding blocks.

[0143] The DMVR (Decoder-side Motion Vector Refinement) method is a method of compensating motion information through the correlation of already restored reference images to find slightly more accurate motion information. It is a method of using the bidirectional motion information of the current block to use the best matching point between reference blocks within an arbitrary set area of ​​two reference pictures as a new bidirectional motion. When this DMVR is performed, the encoder performs DMVR on a block basis to compensate for the motion information, and then divides the block into sub-blocks and performs DMVR on each sub-block basis to compensate for the motion information of the sub-block again. This can be called MP-DMVR (Multi-pass DMVR).

[0144] The LIC (Local Illumination Compensation) method is a method of compensating for luminance changes between blocks. It is a method of deriving a linear model using neighboring pixels adjacent to the current block, and then compensating for the luminance information of the current block through the linear model.

[0145] Existing video encoding methods perform motion compensation that only considers parallel translation in all directions, which reduces encoding efficiency when encoding videos containing motions commonly encountered in the real world, such as zooming in, zooming out, and rotation. To express such motions, an affine model-based motion prediction technique utilizing a four-parameter (rotation) or six-parameter (zoom in, zoom out, rotation) model can be applied.

[0146] Bi-Directional Optical Flow (BDOF) is used to compensate for predicted blocks by estimating pixel changes based on optical flow from a reference block of a block composed of bidirectional motion. The motion information derived from BDOF in VVC can be used to compensate for the motion of the current block.

[0147] PROF (Prediction Refinement with Optical Flow) is a technique for improving the accuracy of sub-block-level affine motion prediction to be similar to that of pixel-level motion prediction. Similar to BDOF, PROF calculates a correction value on a pixel-by-pixel basis for affine motion-compensated pixel values ​​on a sub-block-by-subblock basis based on optical flow to obtain a final prediction signal.

[0148] The CIIP (Combined Inter- / Intra-picture Prediction) method is a method of generating a final prediction block by weighting and averaging prediction blocks generated by the intra-picture prediction method and prediction blocks generated by the inter-picture prediction method when generating a prediction block for the current block.

[0149] The Intra Block Copy (IBC) method finds the most similar part to the current block in a previously restored area within the current picture and uses that reference block as a prediction block for the current block. At this time, information related to the block vector, which represents the distance between the current block and the reference block, can be included in the bitstream. The decoder can parse the block vector information contained in the BeastStream to calculate or set the block vector for the current block.

[0150] The BCW (Bi-prediction with CU-level Weights) method is a method that performs a weighted average on two motion-compensated prediction blocks by adaptively applying weights on a block-by-block basis, rather than generating a prediction block by averaging two motion-compensated prediction blocks from different reference pictures.

[0151] The Intra TMP (Template Matching Prediction) method is a method in which a video signal processing device constructs a reference template using pixel values ​​of neighboring blocks adjacent to the current block, finds the part most similar to the constructed reference template in an already restored area within the current picture, and then uses the reference block (the part found in the already restored area) as a prediction block for the current block.

[0152] The MHP (Multi-hypothesis prediction) method is a method of performing weight prediction using various prediction signals by transmitting additional motion information to unidirectional and bidirectional motion information during inter-screen prediction.

[0153] CCLM (Cross-component linear model) is a method of predicting chrominance signals by constructing a linear model using the high correlation between a luminance signal and the chrominance signals located at the same location as the luminance signal. A template is constructed using blocks adjacent to the current block that have been reconstructed, and the parameters for the linear model are derived from the template. Next, the reconstructed current luminance block is selectively downsampled to match the size of the chrominance block depending on the image format. Finally, the chrominance block of the current block is predicted using the downsampled luminance block and the linear model. At this time, a method using two or more linear models is called MMLM (Multi-model Linear mode). In addition, a prediction method that utilizes the correlation between different signals, such as CCLM and MMLM, can be called CCP (Cross-Component Prediction).

[0154] The CCP merge method is a method of predicting the chrominance block of the current block using the CCP model (models such as CCLM, MMLM, and CCCM) used in the surrounding blocks.

[0155] In independent scalar quantization, the input coefficient t k Restored coefficients t' for k is the associated quantization index q k depends only on t'. That is, the quantization index for any restored coefficient has a different value from the quantization indices for other restored coefficients. In this case, t' k is t k may be a value that includes quantization error, and may be different or the same depending on the quantization parameter. Here, t' kcan be named restored transform coefficients or inverse quantized transform coefficients, and the quantization indices can also be named quantized transform coefficients.

[0156] In Uniform Reconstruction Quantizers (URQ), the reconstructed coefficients are spaced at equal intervals. The distance between two adjacent reconstructed values ​​is referred to as the quantization step size. Reconstructed values ​​can include zeros, and the entire set of available reconstructed values ​​can be uniquely defined by the quantization step size. The quantization step size can vary depending on the quantization parameter.

[0157] In conventional methods, quantization reduces the set of acceptable reconstructed transform coefficients, and the number of elements in this set can be finite. This limits the ability to minimize the average error between the original and reconstructed images. Vector quantization can be used as a method to minimize this average error.

[0158] A simple form of vector quantization used in video encoding is sign data hiding. This is a method in which the encoder does not encode the sign of a non-zero coefficient, and the decoder determines the sign of the coefficient based on whether the sum of the absolute values ​​of all coefficients is even or odd. To achieve this, the encoder may increase or decrease the value of at least one coefficient, and at least one coefficient may be selected and adjusted to be optimal in terms of rate-distortion cost. In one implementation, a coefficient having a value close to the boundary of the quantization interval may be selected.

[0159] Another vector quantization method is trellis-coded quantization, which is used in video coding as an optimal path search technique to obtain optimized quantization values ​​in dependent quantization. Block-wise, quantization candidates for all coefficients in the block are arranged in a trellis graph, and the optimal trellis path between the optimized quantization candidates is searched for by considering the cost for rate-distortion. Specifically, dependent quantization applied to video coding can be designed so that the set of allowable reconstructed transform coefficients for a transform coefficient depends on the value of the transform coefficient preceding the current transform coefficient in the restoration order. At this time, by selectively using multiple quantizers according to the transform coefficients, the average error between the original image and the reconstructed image can be minimized, thereby increasing the encoding efficiency.

[0160] Among intra prediction encoding techniques, the Matrix Intra Prediction (MIP) method is a matrix-based intra prediction method. Unlike prediction methods that have directionality from pixels in neighboring blocks adjacent to the current block, it is a method of obtaining a prediction signal using a predefined matrix and offset values ​​for pixels on the left and top of neighboring blocks. In the MIP method, the matrix can be a matrix vector.

[0161] To derive the intra-prediction mode of the current block, a template, which is a random region adjacent to the current block and reconstructed, is used. The intra-prediction mode derived from the template's surrounding pixels can be used to reconstruct the current block. First, the decoder generates a prediction template for the template using the surrounding pixels (reference) adjacent to the template. The intra-prediction mode that generates the prediction template most similar to the already reconstructed template can then be used to reconstruct the current block. This method is called TIMD (Template intra-mode derivation).

[0162] Typically, an encoder can determine a prediction mode for generating a prediction block and generate a bitstream containing information about the determined prediction mode. A decoder can parse the received bitstream to set an intra-prediction mode. At this time, the bit amount of information about the prediction mode may be about 10% of the total bitstream size. To reduce the bit amount of information about the prediction mode, the encoder may not include information about the intra-prediction mode in the bitstream. Accordingly, the decoder can derive (determine) an intra-prediction mode for restoring the current block using the characteristics of the surrounding blocks, and can restore the current block using the derived intra-prediction mode. At this time, the decoder can use a method of applying a Sobel filter in the horizontal and vertical directions to each neighboring pixel (pixel) of the current block to infer directional information, and then mapping the directional information to the intra-prediction mode to derive the intra-prediction mode. The method by which the decoder derives the intra prediction mode using surrounding blocks can be described as Decoder side intra mode derivation (DIMD).

[0163] If the current block is a luminance block, the video signal processing device can derive an intra prediction mode through the DIMD and TIMD methods. If the current block is a chroma block, since a corresponding reconstructed luminance block exists, the video signal processing device can derive an intra prediction mode by applying the DIMD and TIMD methods using the reconstructed luminance block, and then use the intra prediction mode as an intra prediction mode of the chroma block. That is, the video signal processing device can derive directional information if the current block is a luminance block, and can not derive directional information if the current block is a chroma block, and can apply the directional information found in the luminance block to the chroma block. This mode can be referred to as a "DIMD Chroma" mode or a "TIMD Chroma" mode.

[0164] CCLM (Cross-component linear model) is a method of constructing a linear model by utilizing the high correlation between a luminance signal and a chrominance signal located at the same location as the luminance signal, and then predicting the chrominance signal through the linear model.

[0165] CCCM (Convolutional cross-component model) is a method of constructing a nonlinear model by utilizing the high correlation between a luminance signal and a chrominance signal located at the same location as the luminance signal, and then predicting the chrominance signal through the nonlinear model.

[0166] GLM (Gradient Linear Model) is a method of constructing a model by additionally reflecting the slope of luminance samples in a linear model such as CCLM, and then predicting the color difference signal through the model.

[0167] Figure 7 is a diagram showing the locations of surrounding blocks used to construct a motion candidate list in inter prediction.

[0168] Surrounding blocks can be blocks of spatial position or blocks of temporal position. A neighboring block spatially adjacent to a current block can be at least one of a left (A1) block, a left below (A0) block, an above (B1) block, an above right (B0) block, and an above left (B2) block. A neighboring block temporally adjacent to a current block can be a block that includes an upper left pixel position of a bottom right (BR) block of the current block in a corresponding picture (Collocated picture). If a neighboring block temporally adjacent to the current block is encoded in intra mode or a neighboring block temporally adjacent to the current block exists in an unusable position, a block that includes a horizontal and vertical center (Center, Ctr) pixel position of the current block in the picture corresponding to the current picture (Collocated picture) can be used as a temporal neighboring block. Motion candidate information derived from corresponding pictures can be referred to as a Temporal Motion Vector Predictor (TMVP). Only one TMVP can be derived from a single block, or a block can be divided into multiple sub-blocks, and then a separate TMVP candidate can be derived for each sub-block. The method for deriving TMVPs at the sub-block level can be referred to as a sub-block Temporal Motion Vector Predictor (sbTMVP).

[0169] Whether the methods described in this specification are applied can be determined based on at least one of information about the slice type (e.g., whether it is an I slice, a P slice, or a B slice), whether it is a tile, whether it is a subpicture, the size of the current block, the depth of the coding unit, whether the current block is a luminance block or a chrominance block, whether it is a reference frame or a non-reference frame, the temporal hierarchy according to the reference order and layer, etc. The information used to determine whether the methods described in this specification are applied can be information agreed upon in advance between the decoder and the encoder. In addition, such information can be determined according to a profile and a level. Such information can be expressed as a variable value, and information about the variable value can be included in the bitstream. That is, the decoder can determine whether the above-described methods are applied by parsing information about the variable value included in the bitstream. For example, whether the above-described methods are applied can be determined based on the horizontal length or the vertical length of the coding unit. The above-described methods can be applied if the horizontal or vertical length is 32 or more (e.g., 32, 64, 128, etc.). In addition, the above-described methods can be applied if the horizontal or vertical length is less than 32 (e.g., 2, 4, 8, 16). In addition, the above-described methods can be applied if the horizontal or vertical length is 4 or 8.

[0170] In this specification, the term "block" may be used interchangeably with "sample." Furthermore, the methods (examples) described herein may be applied at least at one of the CU, PU, ​​or TU levels, and the level at which the methods are applied and whether they are used may be indicated by separate information (e.g., syntax elements within a bitstream).

[0171] FIG. 8 is a diagram showing the relationship between a luma sample and a chroma sample according to one embodiment of the present invention.

[0172] Fig. 8 shows the positional relationship between luma samples and chroma samples in the horizontal and vertical directions. In addition, Fig. 8 shows the ratio relationship between luma samples and chroma samples. X in Fig. 8 may mean a luma sample, and O may mean a chroma sample.

[0173] Fig. 8(a) shows the positions of luma samples and chroma samples when the chroma format (relationship between luma samples and chroma samples) is 4:2:0 (or 4:1:1). There can be one corresponding chroma sample (Cb, Cr) for every four luma samples. Fig. 8(b) shows the positions of luma samples and chroma samples when the chroma format is 4:2:2. There can be two corresponding chroma samples for every four luma samples. Fig. 8(c) shows the positions of luma samples and chroma samples when the chroma format is 4:4:4. There can be four corresponding chroma samples for every four luma samples. One chroma sample can be located at the same position for each luma sample.

[0174] FIG. 9 is a diagram showing a reference sample required for Cross-component Linear Model (CCLM) prediction according to one embodiment of the present invention.

[0175] When a first color component and a second color component corresponding to a current block exist, the second color component can be predicted and / or restored based on the first color component. At this time, a model for the relationship between the first color component and the second color component can be used. That is, the second color component can be predicted and / or restored through the first color component based on the model for the relationship between the first color component and the second color component. The model for the relationship between the first color component and the second color component can be modeled from samples of the first color component that have already been restored and samples of the second color component that have already been restored. At this time, the samples of the first color component that have already been restored and the samples of the second color component that have already been restored can be samples around the current block. The second color component can be predicted and / or restored using the first color component that has already been restored based on the model for the relationship between the first color component and the second color component. At this time, the first color component can be a luma component, and the second color component can be a chroma component. Additionally, depending on the decoding and encoding order, the first color component may be a chroma component and the second color component may be a luma component. Additionally, the first color component and the second color component may be any one of the Y, U, and V components. Additionally, the first color component may refer to multiple color components.

[0176] CCLM prediction can be used for chroma intra prediction. Video compression performance can be more efficient if cross-component correlations existing in YUV 4:2:0 sequences are utilized. To reduce cross-component redundancy, in CCLM prediction, the chroma sample(s) of the current block can be predicted and / or reconstructed based on the reconstructed luma sample(s) of the current block. In this case, a linear model described below can be used. The linear model of this specification can be expressed by a linear equation, a linear equation, etc.

[0177]

[0178] Mathematical expression 1 represents a linear model used when restoring chroma samples based on already restored luma samples.

[0179] In mathematical expression 1, predC(i, j) may refer to the predicted value of the chroma sample at position (i, j) of the current block, and recL(i, j) may refer to the value of the already restored luma sample at position (i, j) of the current block. The upper left position of the current block may be (0, 0).

[0180] If the chroma sample density is lower than the luma sample density (e.g., YUV 4:2:0 image), the already restored luma sample may be a downsampled luma sample. Downsampling may mean the process of matching the number of luma samples to the number of chroma samples when the number of luma samples and chroma samples is not 1:1, as shown in Fig. 8(c). The value of the downsampled luma sample may be obtained by performing a weighted average according to a 6-tap filter (see Fig. 15). Parameters of Equation 1 and can be a value to minimize the regression error between the reconstructed luma sample (or downsampled luma sample) and the chroma sample around the current block. Parameter is a parameter based on mathematical expression 2 can be obtained based on mathematical formula 3.

[0181]

[0182]

[0183] In Equations 2 and 3, L(n) and C(n) can be reference samples of CCLM. In addition, L(n) in the above equation can be modeled (e.g., parameter and ) refers to the restored luma sample used in the process of obtaining the current block, and C(n) may refer to the restored chroma sample used in modeling. If the number of luma samples and the number of chroma samples are different, the luma sample may be downsampled and used. L(n) and C(n) may be samples around the current block (neighborhood). N may be the number of L(n) and C(n) pairs. Specifically, L(n) refers to the downsampled restored luma samples on the top and left of the current block, C(n) refers to the chroma samples on the top and left of the current block, and N may refer to twice the minimum value among the width and height of the coding block of the current chroma coding block. A video signal processing device (e.g., an encoder or a decoder) may downsample to make the neighboring samples on the longer boundary have the same value as the sample value on the shorter boundary for a non-square compressed block. Equations 2 and 3 are parameters. and This is just an example to obtain the parameter and can be obtained by other methods than the above-described mathematical formulas 2 and 3.

[0184] FIG. 10 is a diagram showing a mode in which CCLM is applied according to one embodiment of the present invention.

[0185] CCLM can be classified into various modes (Fig. 10(a) to Fig. 10(c)) depending on how the three reference samples are configured. At this time, CCLM can have a mode that basically uses one linear model (Mode 1 to Mode 3). Additionally, CCLM can have a mode that uses two linear models (Mode 4 to Mode 6). Including the mode that uses two linear models, CCLM can be classified into six modes. Referring to Fig. 10(a), the reference sample for CCLM can be the reference block (sample) above and the reference block to the left of the current block. Referring to Fig. 10(b), the reference sample for CCLM can be the reference block above the current block. Referring to Fig. 10(c), the reference sample for CCLM can be the block to the left of the current block. In the CCLM mode using the reference sample configured as in Fig. 10(a) to Fig. 10(c), one or two linear models can be used. The CCLM mode (mode 1) using the reference sample according to Fig. 10(a) and using one linear model can be indicated whether to use it by a syntax element included in the bitstream, and at this time, the syntax element can be expressed as LM_CHROMA_IDX. The CCLM mode (mode 2) using the reference sample according to Fig. 10(b) and using one linear model can be indicated whether to use it by a syntax element included in the bitstream, and at this time, the syntax element can be expressed as MDLM_T_IDX. The CCLM mode (mode 3) using the reference sample according to Fig. 10(c) and using two linear models can be indicated whether to use it by a syntax element included in the bitstream, and at this time, the syntax element can be expressed as MDLM_L_IDX.In addition, the CCLM mode (mode 4) using a reference sample according to Fig. 10(a) and using two linear models can be indicated whether to be used by a syntax element included in the bitstream, and the syntax element can be expressed as MMLM_CHROMA_IDX. The CCLM mode (mode 5) using a reference sample according to Fig. 10(b) and using two linear models can be indicated whether to be used by a syntax element included in the bitstream, and the syntax element can be expressed as MMLM_T_IDX. The CCLM mode (mode 6) using a reference sample according to Fig. 10(c) and using two linear models can be indicated whether to be used by a syntax element included in the bitstream, and the syntax element can be expressed as MMLM_L_IDX. When two linear models are used, each linear model can be a linear model for each group, when the two groups are divided according to the average value of the (downsampled) luma component reference sample values. In the present specification, when the chroma format is not 4:4:4, the luma component reference sample for CCLM can be a downsampled sample to match the ratio of chroma samples to luma samples to 1:1.

[0186] In addition, the configuration of the reference block used for CCLM described through FIG. 10 (e.g., when using the left block of the current block, when using the upper block of the current block, when using both the left and upper blocks of the current block, etc.) can also be applied to CCCM. For example, it can be divided into three modes depending on the reference block used for CCCM. The reference block for CCCM can be described in i) CCCM-LT mode if it is the left block and upper block of the current block, ii) CCCLM-L mode if it is the left block of the current block, and iii) CCCM-T mode if it is the upper block of the current block. At this time, the reference block can also be expressed as a template. For example, the upper block of the reference block can be described as an upper template, and the left block of the reference block can be described as a left template.

[0187] FIG. 11 is a diagram illustrating a CCLM mode using two linear models according to one embodiment of the present specification.

[0188] The average value of reference samples (luma component or chroma component) for CCLM can be a threshold value for obtaining a linear model. At this time, there can be more than one threshold value. Referring to Fig. 11, reference samples can be divided into multiple groups based on the threshold value, and a video signal processing device can obtain a linear model for each group. The threshold value can also be applied to a luma component block that has already been restored in the current block. Using Equation 4, which is a linear model obtained based on the threshold value, predC(I, j), which is a predicted value of a chroma block, can be obtained.

[0189]

[0190] Referring to Fig. 11 and mathematical expression 4, a first linear model can be obtained based on luma component reference samples having values ​​less than the average value, and a second linear model can be obtained based on luma component reference samples having values ​​greater than the average value. At this time, the parameters of the first linear model is 2, can be 1. The parameters of the second linear model is 1 / 2, can be 1.

[0191] The method of using two linear models can also be applied to CCCM. The video signal processing device can obtain the average value of the luminance sample based on the reference region sample values ​​of the CCCM, derive two CCCM models based on the average value, and then predict the chrominance block. The method of using two linear models in CCCM can be described as MM-CCCM (Multi-Model CCCM). MM-CCCM (Multi-Model CCCM) is a method of deriving two CCCM parameters based on the average value of the reference region (or the restored current luminance block).

[0192] GL-CCCM (Gradient and location based convolutional cross-component model) is an additional CCCM mode that uses gradient and location information. In the case of the existing CCCM mode, the video signal processing device can derive the chrominance sample for the current block using the luminance sample at the corresponding position from the predicted chrominance sample position, four samples around the luminance sample, and coefficient information. At this time, in the case of the GL-CCCM mode, the video signal processing device can derive the chrominance sample for the current block by reflecting the vertical and horizontal differences for the luminance sample at the corresponding position from the predicted chrominance sample position and eight samples around the luminance sample, and also using the position value of the current luminance sample and its coefficient information.

[0193] When CCCM mode is applied, the video signal processing device can apply a downsampling filter to match the resolution difference between the luminance block and the chrominance block. This is to reduce the resolution of the luminance block to that of the chrominance block. The mode that applies the downsampling filter can be described as CCCM-MDF (CCCM with multiple downsampling filters).

[0194] When the inter-coding mode is applied to the current block, the video signal processing device can derive linear and nonlinear models between the luminance prediction block (Y') derived using the motion information of the current block and the first chrominance prediction block (Cb', Cr') derived using the motion information of the current block (Derive filter), and then generate a reconstructed luminance block of the current block using the luminance residual block of the current block, and apply the derived linear and nonlinear models to the reconstructed luminance block of the current block (Apply filter) to generate a second chrominance prediction block of the current block. Thereafter, the video signal processing device can add the chrominance residual block to the second chrominance prediction block of the current block predicted using the linear and nonlinear models to finally generate the chrominance block (Cb, Cr) of the current block. This method can be described as a cross-component residual model (CCRM).

[0195] Fig. 12 is a drawing showing a division structure of a block according to one embodiment of the present invention.

[0196] Fig. 12(a) shows the QTBT partition structure of a luma component block in an I slice which is YUV 4:2:0, and Fig. 12(b) shows the chroma QTBT partition structure in an I slice which is YUV 4:2:0. The luma component block and the chroma component block can be partitioned into different structures depending on the partitioning form. If the partitioning form is a single tree, the luma component block and the chroma component block can be partitioned into the same structure, and in this case, the chroma component block and the luma component block can correspond 1:1. If the partitioning form is a separate tree, the luma component block and the chroma component block can be partitioned into different structures, as shown in Fig. 12, and in this case, the chroma component block and the luma component block may not correspond 1:1. The chroma block (gray shaded part) of Fig. 12(b) and the corresponding luma block (gray shaded part) of Fig. 12(a) have different partitioning structures.

[0197] In predicting a chroma component block using a luma component block, if the splitting form is a single tree, the prediction mode of the luma component block stored in the upper left position (TL) of Fig. 12(a) can be used, and if it is a separate tree, the prediction mode of the luma component block stored in the center position (CR) of the luma component block corresponding to the left block (recovery shaded portion) of the chroma component block of Fig. 12(b) can be used.

[0198] Additionally, the video signal device can determine whether a block vector exists at a preset position of a luma block corresponding to a chroma block. A block vector may exist for a block coded in IBC mode or Intra TMP mode.

[0199] If there is a block coded with Intra Block Copy (IBC) or Intra Template Matching (Intra TMP), a block vector may exist. The video signal processing device can search for a template matching block based on a template cost within a preset region based on a template composed of neighboring samples of the current block, for a current block coded with the Intra TMP mode. The video signal processing device can use the luma block of the matching block as a prediction value. The video signal processing device can derive a block vector based on the positional relationship between the matching block position and the current block. The video signal processing device can predict a chroma block using a reference block at a position indicated by the block vector and the template of the reference block. At this time, the video signal processing device can use a method such as CCCM / MM-CCCM / CCLM / GLM / GL-CCCM / CCP merge / CCCM-ND / CCCM-MDF to predict the chroma block.

[0200] FIG. 13 is a diagram illustrating a method for optimizing a linear model for CCLM according to one embodiment of the present invention.

[0201] Figure 12(a) is the mathematical expression 1 described above. and , and Fig. 12(b) is a drawing obtained to optimize the linear model. and This is a drawing showing the obtained for optimization. and are each in mathematical formula 1 and can be replaced, and the linear model can be as shown in Equation 5.

[0202]

[0203] Is + can be obtained with u, Is -u *yr can be obtained. At this time, u is an integer value between -4 and 4, which may be a value signaled by a syntax element included in the bitstream. In addition, the u value may be a preset value. y r may be the mean (or median, mode) of the (downsampled) luma component reference sample.

[0204] FIG. 14 is a diagram illustrating a gradient linear model (GLM) according to one embodiment of the present invention.

[0205] Fig. 14(a) shows the weight values ​​of the downsampling filter used when the chroma format is not 4:4:4. As shown in the right drawing of Fig. 14(c), the size of the current block may be 8x8, and the chroma format may be 4:2:0. The video signal processing device may multiply the values ​​of six reference samples (151) located above the current coding block of Fig. 14(c) by the filter weight values ​​corresponding to each of the six values, and obtain an average value thereof. At this time, the downsampled luma samples may be the left block of Fig. 14(c). One chroma sample may be located per position of each downsampled luma sample, and may correspond 1:1. Through the above-described method, the video signal processing device may obtain downsampled luma samples.

[0206] Below, a method is described for obtaining a GLM linear model based on the gradient value of a luma sample and obtaining a predicted value (C(i, j)) of a chroma sample of the current block using the GLM linear model.

[0207] The GLM linear model can be set to Equation 6 or Equation 7 for each current block depending on the condition. The condition may be a condition based on a cost value.

[0208]

[0209]

[0210] Mathematical formulas 6 and 7 and is the same as the value of the CCLM linear model of the above-described mathematical expression 1, and G(i, j) may be a gradient value corresponding to the already restored luma samples. recL(i, j) may be a value of the downsampled luma sample.

[0211] Mathematical expression 6 can be replaced with mathematical expression 8, and mathematical expression 7 can be replaced with mathematical expression 9.

[0212]

[0213]

[0214] B in mathematical expression 8 is the median value of the bit depth of the content, and chromaMean may be the average value of the chroma component reference samples. midValue in mathematical expression 9 is the median value of the bit depth of the content, and may be 512 when the bit depth is 10-bit. In addition, midValue may be the average value of each chroma component reference sample. In mathematical expressions 8 and 9, may be a coefficient corresponding to the i-th already restored downsampled luma sample value located around the chroma component sample to be predicted.

[0215] A filter for obtaining a gradient value can be described as a Sobel-based gradient pattern. There may be multiple filters for obtaining a gradient value. Fig. 14(b) illustrates one of multiple filters for obtaining a gradient value. The GLM linear model can be independently applied to chroma component Cb and Cr samples, respectively. In this case, whether the GLM linear model is applied to each chroma component can be signaled by a syntax element (flag) included in the bitstream. In addition, when the GLM linear model is applied to each chroma component sample, a syntax element indicating one of the filters (multiple Sobel-based gradient patterns) for the GLM linear model can be included in the bitstream and signaled. The syntax element indicating one filter can be described by a glm index.

[0216] GLM can operate in the six modes described above of CCLM or can be restricted to operate in only specific modes. In this case, the specific modes can be LM_CHROMA_IDX, MDLM_L_IDX, MDLM_T_IDX. In addition, GLM may not be applied to some of specific intra luma prediction modes. For example, GLM may not be applied to all or some of the non-directional DC mode and the planar mode. In addition, GLM may not be applied when the intra luma prediction mode is any one of the non-directional DC mode, the planar mode, and the MIP mode. In this case, the value of the syntax element (flag) indicating whether GLM is applied can be set to 0. Two linear models can also be used in GLM. The following describes the GLM operation method and operating conditions.

[0217] The conditions for GLM to be applied are as follows.

[0218] (Condition) Chroma mode==MMLM_CHROMA_IDX || Chroma mode==MMLM_L_IDX || Chroma mode==MMLM_T_IDX) && (Luma block width * Luma block height >= Reference value 1)

[0219] Reference value 1 is a positive integer, such as 16, 32, 64, or 128. In the above condition 1, the horizontal size of the luma block can be replaced with the horizontal size of the chroma block, and the vertical size of the luma block can be replaced with the vertical size of the chroma block. In addition, reference value 1 is a value determined based on the chroma format, such as 8, 16, 32, 64, or 128.

[0220] There may be multiple filters for obtaining a slope value, and the glm index may be signaled to indicate one of the multiple filters. The video signal processing device may determine the filter for obtaining the slope value by parsing the glm index included in the bitstream. At this time, there may be 4, 8, or 16 filters for obtaining the slope value, and the glm index may indicate any one of the 4, 8, or 16 filters. The glm index may be signaled with a fixed bit size. In addition, in order to reduce signaling complexity, the glm index may be configured in a method described below rather than with a fixed bit size.

[0221] i) A video signal processing device can rearrange filter candidates that can be used as filters for obtaining a gradient value based on a reference element, and signal / parse a glm index based on the rearranged filter candidates. The reference element may be a luma prediction mode or a coding mode (e.g., Decoder-side intra mode derivation (DIMD), Template-based intra mode derivation (TIMD)) of a current block. For example, the video signal processing device can classify luma prediction modes into a plurality of groups, and rearrange filter candidates for obtaining a gradient value for each group using frequently used filter candidates. Among the rearranged candidates, a candidate with the highest frequency of use may be mapped to a lowest glm index, and the glm index may be signaled with a size of 1 bit. The luma prediction modes may be classified into a plurality of groups based on the indexes of the luma prediction modes. At this time, the luma prediction modes may be classified into groups evenly (i.e., the same number per group) or unequally (i.e., unequal number per group). The indices of luma prediction modes included in each group may be continuous. In addition, luma prediction modes may be classified based on specific indices. For example, luma prediction modes may be classified into luma prediction modes corresponding to indices that are equal to or less than (or less than) a specific index and luma prediction modes corresponding to indices that are greater than (or equal to or greater than) the specific index. At this time, the specific indices may be multiple. That is, if there is one specific index, luma prediction modes may be classified into two groups, if there are two specific indices, luma prediction modes may be classified into three groups, and if there are three specific indices, luma prediction modes may be classified into four groups. At this time, the specific indices may be 18 (horizontal direction), 34 (diagonal direction), and 50 (vertical direction).In addition, non-directional prediction modes (e.g., DC mode, planar mode) can be classified into a separate group. The glm index can be signaled using a variable length code scheme. The variable length code scheme can be a truncated unary binarization scheme. There is a problem that the amount of information increases as the value of the glm index increases. However, if the occurrence frequency of a relatively low index is high, the glm index can be signaled with a small amount of information. Table 1 shows the structure of truncated unary binarization. Referring to Table 1, as the value of the index (v) increases, the amount of information can also increase.

[0222]

[0223] ii) The video signal processing device can calculate a cost value for each of the filters for obtaining a plurality of slope values, and reorder the filters based on the cost value. The video signal processing device can sort the filters in ascending order of the cost values ​​corresponding to the filters. A filter with the smallest cost value can be mapped to the smallest glm index. The video signal processing device can signal / parse a glm index indicating any one of the reordered filters. The cost value can be calculated based on chroma component reference samples and chroma component prediction samples located at the boundary of the current block. The chroma component prediction samples located at the boundary of the current block can be samples at specific locations.

[0224] Figure 15 is a diagram showing a filter for obtaining a slope value used in GLM.

[0225] Specifically, Fig. 15 is a diagram illustrating four Sobel-based gradient patterns that can be used in GLM. The circle located at the center of each filter may be the result of filtering.

[0226] Table 2 shows the filter values ​​of 16 Sobel-based gradient patterns that can be used in GLM.

[0227]

[0228] FIG. 16 illustrates an example of a reference region and filter shape used to derive CCCM parameters according to one embodiment of the present invention.

[0229] CCCM (Convolutional cross-component intra prediction model) is a method for predicting chroma blocks using a nonlinear model that utilizes the high correlation between a luma block (luma component block, luma component sample, luma sample) and a chroma block located in the same location as the luma block. That is, a video signal processing device can predict a chroma component sample corresponding to a luma component sample of the current block using a luma component sample of the current block. Then, the video signal processing device can restore the current block using the value of the predicted chroma component sample.

[0230] FIG. 16(a) shows the positional relationship between reference samples (vertical hatching, 161) for applying CCCM to a current prediction block (diagonal hatching, 162) according to one embodiment of the present invention and side samples (horizontal hatching) required when applying a cross-shaped filter. When the size of the current prediction block (162) is M (width) x N (height) and the number of luma component samples and chroma component samples is 1:1, the reference samples (161) may be configured as a reference sample area (161-1) of 2M x 6 size in the upper 6 rows with a size twice that of the width of the current prediction block (162), a reference sample area (161-2) of 6 x 2N size in the left 6 rows with a size twice that of the height of the current prediction block (162), and a reference sample area (161-3) of 6 x 6 size in the upper left of the current prediction block (162). That is, when the coordinates of the upper left sample of the current prediction block (162) are (0, 0), 161-1 may be (X1, Y1), 161-2 may be (X2, Y2), and 161-3 may be (X3, Y3). At this time, X1 may be a value from 0 to 2M-1, Y1 may be a value from -1 to -6, X2 may be a value from -1 to -6, Y2 may be a value from 0 to 2N-1, and X3 and Y3 may be values ​​from -1 to -6.

[0231] Fig. 16(b) illustrates a sample of a cross-shaped pattern according to an embodiment of the present invention. In the present specification, a sample of any shape pattern can be expressed as a sample filter. When the sample filter of the cross-shaped pattern of Fig. 16(b) is used for CCCM, the area where the filter is applied may be outside the reference sample area. In this case, an additionally required sample may be a side sample. Depending on the position of the C sample of Fig. 16(b), an area other than the reference sample may be required when the sample filter of the cross-shaped pattern is applied. In this case, if the area other than the reference sample is not available, the value of the area other than the reference sample may be padded with the value of the C sample and used. The sample at the C (Center) position of Fig. 16(b) may be a luma sample corresponding to the Cb and Cr chroma samples, and N (North), E (East), S (South), and W (West) may be luma samples adjacent to the luma sample at the C position. CCCM can be applied to each chroma component (i.e., Cb component, Cr component). Side samples may require one additional sample for areas other than the reference sample, depending on the location of the C sample. If the sample value existing at the location of the side sample is not available, the sample value at the unavailable location may be padded with the C sample value.

[0232] When a cross-shaped pattern sample filter for CCCM is applied, the predicted value (predChromaVal) of the chroma component sample can be calculated as in Equation 10.

[0233]

[0234] The P value in Equation 10 is a nonlinear term and can be calculated as ( C * C + midVal ) >> bitDepth . midValue can be the median value of the content bitdepth or the average value of each chroma component. bidDepth can represent the bit depth. In content with a bitDepth of 10 bits, the P value can be calculated as ( C * C + 512 ) >> 10 . The B value represents a bias term, an integer offset value, and can be the median value of the bitDepth content. In the case of 10-bit content, the B value can be 512. In addition, the B value can be the average value of the chroma component reference sample. In addition, the B value can be the (absolute value) difference between the average value of the luma component reference samples and the average value of each chroma component reference sample. C, N, S, E, and W can represent sample values ​​according to the positions in Fig. 16(b). The coefficients (C0, C1, ..., C6) of Equation 10 may be values ​​that minimize the MSE (Mean Square Error) value of the autocorrelation matrix for the luma component sample values ​​of the reference sample area and the cross-correlation vector for the chroma component sample values. The autocorrelation matrix may be obtained using LDL decomposition or Cholesky decomposition. In addition, the coefficients (C0, C1, ..., C6) of Equation 10 may be obtained using back-substitution.

[0235] C, N, S, E, W, P in Equation 10 can be replaced with C', N', S', E', W', P', respectively. C', N', S', E', W', P' can be calculated as follows.

[0236] C' = C - meanY, N' = N - meanY, S' = S - meanY, E' = E - meanY, W' = W - meanY, P' = P - meanNonlinY

[0237] meanY can be the average value of the luma component reference samples. meanNonlinY can be calculated as follows.

[0238] meanNonlinY = (meanY * meanY) >> bitdepth or meanNonlinY = (meanY * meanY + bitdepth>>1) >> bitdepth

[0239] Mathematical expression 10 can be changed to mathematical expression 11.

[0240]

[0241] Mathematical expression 11 may be in the form where C, N, S, E, W, P, and B in Mathematical expression 10 are replaced with C', N', S', E', W', P', and B', and meanChroma is added. meanChroma may be the average value of chroma component reference samples. B' in Mathematical expression 11 may be the (absolute value) difference between the average value of the luma component reference sample and the average value of each chroma component reference sample.

[0242] Depending on the luma samples used for CCCM, Equation 11 can be modified. For example, if only C, N, S, and E are used among C, N, S, E, and W, Equation 11 can be modified to a form that excludes the term corresponding to W (c4W'). Furthermore, Equation 11 can be modified to a form that excludes B'.

[0243] In addition, C', N', S', E', and W' in Equation 11 may be result values ​​according to any one pattern (filter) among multiple Sobel-based gradient patterns. That is, C', N', S', E', and W' in Equation 11 may be replaced with gradient values.

[0244] Additionally, the P value in Equation 10 can be changed as follows.

[0245] i) The P value can be obtained according to ( Cg * Cg + midVal ) >> bitDepth. Cg can be the gradient value of the sample at position C in Fig. 16(b). midVal can be the middle value of bitDepth, and in content with bitDepth of 10 bits, midVal can be 512. ii) The P value can be the average value of the gradient values ​​of the reference samples. iii) The P value can be the average value of the gradient values ​​at positions (C, N, E, S, W) in Fig. 16(b). iv) The P value can be obtained according to (meanG * meanG + bitDepth >> 1) >> bitdepth. meanG can be the average value of the gradient values ​​of the reference samples.

[0246] In addition, Fig. 16(c) shows the positional relationship of samples for the GL-CCCM mode. The video signal processing device can apply the GL-CCCM mode using the sample positional relationship of Fig. 16(c). The sample corresponding to each position in Fig. 16(c) is a down-sampled value according to the luma and chroma composition ratio, and the luma sample at position C can be a sample corresponding to the chroma to be predicted. The GL-CCCM (Gradient and location based convolutional cross-component model) mode can be a CCCM mode using gradient and position information. Mathematical expression 12 is a mathematical expression used in GL-CCCM.

[0247]

[0248] The video signal processing device can derive coefficients using the CCCM method using the luminance sample value at the C position in Fig. 16(c) and Gx (horizontal slope, horizontal difference), Gy (vertical slope, vertical difference), Y (value based on the vertical coordinate of C), X (value based on the horizontal coordinate of C), P, and B as input values, and generate a chroma prediction sample value. If the upper left position of the reference template used to derive the CCCM model, the upper left position of the current block, and the upper left position of the current block are (0,0), the position value of the current luminance sample corresponding to the values ​​of Y and X can be a value recalculated through the relative position of the current pixel, an arbitrary offset value, and an arbitrary shift value. At this time, the position value of the current luminance sample can be calculated through the relative position value of the current pixel + the difference value between the upper left position of the reference template used to derive the CCCM model at the upper left position of the current block + an arbitrary offset value) << an arbitrary shift value). At this time, the arbitrary offset value can be an integer and can be '8'. Additionally, the arbitrary shift value can be an integer, and can be '3'. In addition, Gx and Gy can be calculated using Equation 13 using the samples of Fig. 16(c).

[0249]

[0250] Below, when the chroma format is 4:2:2, the downward sampling process to match the number of chroma samples to the number of luma samples to 1:1 is described.

[0251] As explained through Figure 14, luma samples can be converted by applying a downsampling filter (Figure 14(a)) to the same number as the number of chroma samples.

[0252] Mathematical expression 13 is a filter-based linear model (FLM) used for downsampling.

[0253]

[0254] C in mathematical expression 14 is the value of the chroma component sample to be predicted, L i is the value of the i-th restored downsampled luma component sample located around the chroma component sample to be predicted, is L i The coefficient values ​​corresponding to can be obtained by using the method of calculating the coefficients (C0, C1, ..., C6) of the above-described mathematical expression 10. is an offset, N can be the number of luma component samples required to calculate the value of the chroma component sample to be predicted. The value of N can be an integer between 2 and 6, and can be 2 or 6. In mathematical expression 14, it can be replaced with a gradient value which is a result value for one of the multiple Sobel-based gradient patterns.

[0255] Figures 17 and 18 illustrate luminance samples before downsampling used to derive chrominance samples in CCCM-ND mode according to one embodiment of the present invention.

[0256] CCCM-ND (CCCM using non-downsampled luma samples) mode may be a method of predicting a chrominance block using samples of a luminance block before downsampling. Fig. 17 shows a chrominance sample position C to be predicted by a video signal in CCCM-ND mode and six luminance sample positions corresponding to the chrominance sample positions. At this time, the luminance samples may be samples before downsampling. The chrominance sample may be calculated using Equation 15. The chrominance sample may be calculated using a linear model using six luminance samples (L0, L1, L2, L3, L4, L5 samples of Fig. 17) and a non-linear model using four luminance samples (L0, L3, L2, L1 samples of Fig. 17). In Equation 15, offsetLuma and offsetchroma have the same meaning as the difference value described above. B can be calculated as '1 << (bitDepth - 1)'. The LDL decomposition method used in CCCM can be used to derive the coefficients (a0 ~ a10).

[0257]

[0258] The encoder can signal whether CCCM-ND is applied to the current block by including information in the bitstream. The decoder can parse this information to determine whether CCCM-ND is applied to the current block.

[0259] If CCCM-ND is applied to the current block, the encoder may not include information about GL-CCCM in the bitstream. If CCCM-ND is applied to the current block, the decoder may not parse information related to GL-CCCM and may set GL-CCCM not to be applied to the current block.

[0260] When GL-CCCM is applied to the current block, the video signal processing device derives a model for GL-CCCM using the downsampled luminance samples, and predicts chrominance samples for the current block using the downsampled luminance samples of the current block. At this time, in GL-CCCM, as in CCCM-ND, luminance samples before downsampling can be used to derive a model or predict chrominance samples.

[0261]

[0262] A chrominance sample can be predicted (calculated) through mathematical expression 16. The video signal processing device can derive a chrominance sample for the current block by reflecting a linear model using six luminance samples (luminance samples L0, L1, L2, L3, L4, L5 of FIG. 17) and a non-linear model using four luminance samples (luminance samples L0, L3, L2, L1 of FIG. 17), a luminance sample at a corresponding position at the chrominance sample position to be predicted, and vertical and horizontal differences (Gx, Gy in mathematical expression 16) for samples around the luminance sample (luminance samples L1 to L8 of FIG. 17), and also using the position value (X, Y in mathematical expression 16) of the current luminance sample and coefficient information. At this time, the video signal processing device can calculate Gx and Gy through mathematical expression 17. Mathematical expression 17 can be used for the samples of FIG. 17. In Equation 17, offsetLuma and offsetchroma have the same meanings as the difference values ​​described above. B can be calculated as '1 << (bitDepth - 1)'. The LDL decomposition method used in CCCM can be used to derive the coefficients (a0 ~ a14).

[0263]

[0264] To predict a chrominance sample, the video signal processing device can use the luminance samples of FIG. 18 (e.g., L6, L7, L8 located below the current chrominance sample) instead of the luminance samples of FIG. 17.

[0265] The video signal processing device can use luminance samples located at a specific distance (sample) away from the luminance sample corresponding to the current chrominance sample in addition to the surrounding luminance samples for CCCM-ND. For example, when the specific distance is 1, the video signal processing device can use samples located at L0 to L8 of FIG. 18, and when the specific distance is 2, the video signal processing device can use at least one of the luminance samples located at L0 to L8 and I0 to I15 of FIG. 18.

[0266] FIG. 19 shows a downsampling filter and sample positions applied to a luminance block according to one embodiment of the present invention.

[0267] In the CCCM method of predicting a chrominance block based on a luminance block, a video signal processing device can apply various downsampling filters to reduce the resolution of the luminance block to that of the chrominance block to match the resolution difference between the luminance block and the chrominance block. The CCCM mode that applies the downsampling filters can be described as CCCM-MDF (CCCM with multiple downsampling filters).

[0268] Fig. 19(a) shows multiple downsampling filters that can be applied to a luminance block. Filtering using the H filter in Fig. 19(a) is filtering that considers the value of the current luminance sample position more important than the surrounding samples. Filtering using the G1, G2, and G3 filters in Fig. 19(a) may be filtering methods designed according to the gradient between adjacent samples. Filtering using the G1 filter may be filtering based on changes in the horizontal direction. Filtering using the G2 filter may be filtering based on changes in the vertical direction. Filtering using the G3 filter may be filtering based on changes in the diagonal direction. C in Fig. 19(b) represents the position of the current chrominance sample, and W, E, N, S, NW, NE, SW, and SE represent the positions of samples surrounding C.

[0269]

[0270] Mathematical expression 18 represents a calculation formula for deriving a chrominance sample using at least one of the down-sampled luminance samples obtained based on filtering using the H, G1, G2, and G3 filters of FIG. 19.

[0271] In Equation 18, P represents a nonlinear model and can be the square of the sample value at the corresponding location. In Equation 18, B can be half the maximum value of the current sample or an intermediate bit depth value. For 10-bit content, the value of B can be 512.

[0272] Model 1 of Equation 18 may be a mathematical formula for deriving a chrominance sample using a down-sampled luminance block obtained using four filters (using the H, G1, G2, and G3 filters of FIG. 19). Model 2 of Equation 18 may be a mathematical formula for deriving a chrominance sample using a down-sampled luminance block obtained using the H, G1 filters of FIG. 19. Model 3 of Equation 18 may be a mathematical formula for deriving a chrominance sample using a down-sampled luminance block obtained using the H, G3 filters of FIG. 19.

[0273] The encoder can select the optimal model among the three chrominance prediction blocks derived using Equation 18, and then signal information about which model is used to predict the current chrominance block by including it in the bitstream. At this time, the information about which model is used can be applied to the chrominance signal Cb and the chrominance signal Cr, respectively, on a block-by-block basis. The decoder can predict each chrominance block using the model determined by parsing the information about which model is used for each chrominance signal (Cb, Cr).

[0274] Alternatively, the encoder can select the optimal model among the three chrominance prediction blocks derived through Equation 18, and then comprehensively signal information on which model to use when predicting the current chrominance blocks for the chrominance signals (Cb, Cr) and include it in the bitstream on a block-by-block basis. The decoder can parse the information on which model to use to generate the current chrominance prediction block and apply it equally to the current Cb chrominance block and the current Cr chrominance block to generate the chrominance prediction block.

[0275] When two CCCM models are used in the current block, the video signal processing device can derive the two models based on the average value of the down-sampled luminance block generated through filtering using the H filter of Fig. 19(a).

[0276] FIG. 20 is a diagram illustrating a direct block vector according to one embodiment of the present invention.

[0277] Referring to FIG. 20, a video signal processing device can use a direct block vector (DBV) for prediction of a chroma block.

[0278] Fig. 20(a) shows the relationship between luma and chroma described through Fig. 12. Fig. 20(a) shows a chroma block and a luma block corresponding to the chroma block. At this time, the luma block may include an upper left (TL), upper right (TR), center (C), lower left (BL), and lower right (BR) region. If the luma block is coded in IBC mode or Intra TMP mode, a block vector (BV) related to the luma block may exist. The vector related to the luma block may be adjusted according to the luma and chroma sampling ratio. The video signal processing device may check whether a block vector exists at a preset position of the luma block. At this time, the preset position may be located inside the luma block or outside the luma block. For example, if the preset position is located inside the luma block, the preset position may be TL, TR, C, BL, and BR. If a block vector exists at a preset position, the video signal processing device can use the chroma reference block indicated by the block vector at the preset position as a prediction value of the current chroma coding unit (block). Referring to Fig. 20(b), the upper left position of the current block may be (xCb, yCb), and the block vector at the upper left position may be (bvC[0], bvC[1]). At this time, the position in the reference block indicated by the block vector may be (xCb+bvC[0], yCb+bvC[1]). If multiple block vectors exist, the video signal processing device can use the optimal block vector based on the TM cost. Alternatively, the video signal processing device can use only the block vector that is searched first. At this time, the order of the area that the video signal processing device searches to determine whether a block vector exists may be predetermined.

[0279] Describes the chroma fusion prediction mode.

[0280] A video signal processing device can predict a final chroma prediction sample value corresponding to a luma block by combining a weighted average of an LM mode (e.g., MM-CCLM or MM-CCCM) and a non-LM mode. At this time, the final chroma prediction sample value can be predicted using mathematical expression 19.

[0281]

[0282] Pred0(i,j) in Equation 19 may be a sample value of a block (sample, pixel) at position (i, j) in a chroma block predicted by a non-LM mode. The non-LM mode may be at least one of a DM mode, a DIMD chroma mode, and four chroma basic modes. In this case, the four chroma basic modes may be a planar mode, a DC mode, a vertical direction mode (index 18 of the angular mode), and a horizontal direction mode (index 50 of the angular mode). pred1(i,j) may be a sample value of a block at position (i, j) in a chroma block predicted by the LM mode. predC(i,j) may be a final prediction value at position (I, j) in the current chroma block. w0 and w1 may be weight values, and the shift value may be 2. The weights w0 and w1 may be determined based on a fixed ratio. The fixed ratio can be any one of 3:1, 1:3, and 2:2. For example, if both the upper and left neighboring blocks of the current coding block are coded in the LM mode, {w0, w1} can be {1, 3}, if only one of the two is coded in the LM mode, {w0, w1} can be {3, 1}, and if neither of them uses the LM mode, {w0, w1} can be {2, 2}. If there are multiple available LM modes, the video signal processing device can compare the cost values ​​of each template and use the LM mode with a lower cost value. In the non-LM mode, chroma sample(s) predicted using DBV can be used. Pred0(i,j) can be a chroma sample value predicted using DBV. The LM mode can be at least one of CCLM, CCCM, CCP merge mode, CCCM-ND, GL-CCCM, and CCCM-MDF. Alternatively, the video signal processing device can configure a new set by combining MM-CCLM and MM-CCCM, and use the mode determined based on the TM cost as the LM mode.

[0283] A video signal processing device can predict a final chroma prediction sample value using a weighted average combination of a downsampled reconstructed luma sample in an LM mode (e.g., MMLM or CCLM) and a chroma sample in the same location. At this time, the final chroma prediction sample value can be predicted using Equation 20.

[0284]

[0285] Pred0(i,j) in Equation 20 may be the value at position (i, j) of the chroma block predicted in non-LM mode, or may be the value predicted in the LM mode. rec' L (i,j) may be the downsampled restored luma sample value at the same location as the chroma and pred C (i,j) can be the final predicted value of the current chroma block. The ß value can be an intermediate value of the image bit depth. The bit depth can be 512 for 10-bit images. a0, a1, and a2 can be derived using the LDL decomposition method used in CCCM. Pred0(i,j) can be chroma samples / blocks predicted using the DBV method. rec' L (i,j) may be the value of a downsampled restored luma sample at the same location as the chroma sample within the reference block pointed to by DBV. Or, rec' L (i,j) may be a down-sampled restored luma sample value within a luma block corresponding to the current coding chroma block.

[0286] The method for predicting chroma samples derived from DBV described herein, the prediction method using a weighted average combination of samples predicted by a linear / nonlinear (LM / non-LM) model, and the method using coefficients of the CCCM method can also be applied to I(intra) slices. The sizes of coding units / coding blocks / prediction blocks / transform blocks for which these methods are applied can be determined based on a reference value.

[0287] Figures 21 and 22 show additional locations for application of DBV according to one embodiment of the present invention.

[0288] Referring to Fig. 21, a basic block vector position may be added for applying DBV to a chroma block. That is, a position for a video signal processing device to check whether a block vector exists may be added. The position of the added block vector may be determined based on the size of a Luma A block (including TL, TR, BR, and BL), which is a luma block corresponding to a Chroma A block, referring to Fig. 21. The position indicated by x in Fig. 21 may be an added position for applying DBV. If the width of the Luma A block is W1, the height is H1, and the coordinates of the upper left corner of Luma A are (0, 0), the added positions may be (-W1, 0), (-W1, -H1), (0, -H1), (-W1 / 2, H1+H1 / 2), (-W1-W1 / 2, H1+H1 / 2), (-W1-W1 / 2, H1 / 2). The block vector at the added location can be scaled according to the chroma format. If the luma-chroma sample ratio is not 1:1 (e.g., the chroma format is 4:4:4), the block vector needs to be converted. For example, if the chroma format is 4:2:2 or 4:2:0, the block vector must be resized to match the ratio.

[0289] Fig. 22 shows the positions added for searching block vectors in the chroma current block and the restored chroma sample area. If the width of the current chroma current block is W and the height is H, positions corresponding to (-W, 0), (-W, -H), (0, -H), (-W / 2, -HH / 2), (-WW / 2, -HH / 2), (-WW / 2, -H / 2) can be added. The block vectors of the added positions can be used as positions set in the restored chroma area without separate block vector conversion. In addition to the positions described through Figs. 21 and 22, positions for searching block vectors in the restored area and the search area can be set.

[0290] FIG. 23 is a diagram showing a type of conversion kernel according to one embodiment of the present invention.

[0291] Specifically, Fig. 23 shows the definition of the transform kernel used in MTS, and shows the formulas (basis functions) of the DCT-II, DCT-V, DCT-VIII, DST-I, DST-VII, and DST-IV kernels. In this specification, DCT-II can be described as DCT-2 (DCT2), DCT-V as DCT-5 (DCT5), DCT-VIII as DCT-8 (DCT8), DST-I as DST-1 (DST1), DST-VII as DST-7 (DST7), and DST-IV as DST-4 (DST4).

[0292] DCT and DST can be expressed as functions of cosine and sine, respectively, and when the basis function of the transform kernel for the number of samples N is expressed as Ti(j), the index i represents the index in the frequency domain, and the index j represents the index within the basis function. That is, as i becomes smaller, it represents a low-frequency basis function, and as i becomes larger, it represents a high-frequency basis function. When the basis function Ti(j) is expressed as a two-dimensional matrix, it can represent the j-th element of the i-th row, and since the transform kernels illustrated in Fig. 23 all have separable characteristics, they can perform transformations in the horizontal and vertical directions for the residual signal X, respectively. That is, when the residual signal block is X and the transform kernel matrix is ​​T, the transformation for the residual signal X can be expressed as TXT'. At this time, T' means the transpose matrix of the transform kernel matrix T.

[0293] The values ​​of the transform matrix defined by the basis functions illustrated in Fig. 23 may be in decimal form rather than integer form. Therefore, it may be difficult to implement decimal form values ​​in hardware in a video encoding device and a decoding device. Therefore, an integer-approximated transform kernel from an original transform kernel containing decimal form values ​​may be used for encoding and decoding a video signal. The approximated transform kernel containing integer values ​​may be generated through scaling and rounding of the original transform kernel. The integer values ​​contained in the approximated transform kernel may be values ​​within a range that can be expressed by a preset number of bits. The preset number of bits may be 8 or 10 bits. Depending on the approximation, the orthonormal properties of the DCT and DST may not be maintained. However, since the loss in encoding efficiency resulting from this is not significant, approximating the transform kernel to an integer form may be advantageous in terms of hardware implementation.

[0294] IDTR (Identity Transform) is a transformation whose result is the original original. It's called an identity transformation. Typically, an identity transformation constructs a transformation matrix by setting "1" in positions where rows and columns have the same value. However, here, the identity transformation uses a fixed, arbitrary value, not "1," to uniformly increase or decrease the value of the input residual signal.

[0295] FIG. 24 is a diagram showing the 0th (lowest frequency component of the corresponding transform kernel) basis function of DCT-II, DCT-V, DCT-VIII, DST-I, and DST-VII transforms according to one embodiment of the present invention.

[0296] Specifically, FIG. 24 is a graph of Ti(j), which is a transformation basis function of DCT / DST defined in FIG. 23, when N is 8 and i is 0. The horizontal axis represents the index j (j=0, 1, …, N-1) within the transformation basis function, and the vertical axis represents the magnitude value of the signal.

[0297] As shown in Fig. 24, DST-VII shows a tendency for the signal to increase as the index j increases, so it can be effective for a pattern of residual signals in which the energy of the residual signal increases as the distance in the horizontal and vertical directions from the upper left coordinate of the block within the residual signal block increases, as in the prediction within the screen.

[0298] On the other hand, in the case of DCT-VIII, since the signal size shows a pattern in which the size of the signal decreases as the index j increases, it can be efficient for the pattern of the residual signal in which the energy of the residual signal decreases as the distance in the horizontal and vertical directions from the upper left coordinate of the block within the residual signal block increases.

[0299] In the case of DST-I, the signal increases as the index j within the basis function increases, and then decreases in magnitude after a certain index. Therefore, it can be effective for residual signal patterns in which the energy of the residual signal increases as one moves toward the center of the residual block.

[0300] For DCT-II, the 0th basis function represents DC, and it can be efficient for patterns of residual signals with uniform pixel value distribution within the residual block, such as inter-screen prediction.

[0301] For DCT-V, it is similar to DCT-II, but since the value when j is 0 is smaller than the value when j is not 0, it has a signal model in the form of a straight line that is broken when j is 1.

[0302] Conventional video codecs that primarily use only DCT-II cannot adaptively perform transformation on the pattern of the residual signal that varies depending on the prediction mode and the characteristics of the original signal, and thus cannot achieve optimal encoding efficiency. However, adaptive multiple transform (AMT) performs transform encoding by selecting a transform kernel optimized for the pattern of the residual signal by using various transform kernels differently depending on the prediction mode, and high compression efficiency can be expected. Similar to AMT, multiple transform selection (MTS) technology is a transform encoding method that can improve encoding efficiency by adaptively selecting a transform kernel according to the prediction mode.

[0303] Below, a combination of transformation kernels according to one embodiment of the present specification is described.

[0304] DCT2 can be used as the basic transform kernel for restoring the current block. Meanwhile, if DCT2 is not used, the remaining kernels (e.g., DCT8, DST7, DCT5, DST4, DST1) can be used. If DCT2 is not used, some of the preset combinations for the remaining kernels can be used. Table 3 shows combinations of kernels excluding DCT2 and IDTR among the transform kernels disclosed in FIG. 23. That is, Table 3 shows combinations of five types of kernels (DCT8, DST7, DCT5, DST4, DST1). Specifically, Table 3 shows 25 combinations that can be configured by making one pair (combination) of two transform kernels. A video signal processing device (e.g., decoder, encoder) can use any one of the 25 combinations in Table 3 as a transform kernel for the horizontal or vertical direction of the current block. Meanwhile, IDTR can only be used when certain conditions are met.

[0305]

[0306] FIG. 25 and FIG. 26 are diagrams showing a set of transformation kernels according to one embodiment of the present invention.

[0307] Referring to FIG. 25, a set of transformation kernels determined based on the intra prediction mode of the current block and the size of the current block is described.

[0308] Referring to FIG. 25, the transform kernel used in the intra prediction mode can be determined based on the intra prediction mode and the size of the current block (e.g., coding block, transform block). In addition, FIG. 25 can represent a transform kernel set for MIP. 2501 of FIG. 25 represents the type of intra prediction mode (i.e., an index indicating the intra prediction mode), and 2502 represents the size (i.e., width x height) of the current block (e.g., coding block, transform block). For example, when the size of the current block is 4x4 and the type of intra prediction mode is 1, the transform kernel for the current block can be determined based on the transform kernel set corresponding to index 0. The transform kernel set corresponding to index 0 can be T0 {18, 24, 17, 23, 8, 12} with reference to FIG. 26. And, the video signal processing device can restore the current block based on the transformation kernel (sub-transform kernel set) included in the transformation kernel set corresponding to any one index of T0. There are six indices corresponding to the transformation kernel (sub-transform kernel set) included in the transformation kernel set of Fig. 26, but this is only an example and there may be four.

[0309] Fig. 26(a) shows a transform kernel set composed of indices corresponding to six transform kernels. Fig. 26(a) is a diagram showing some of 80 transform kernel sets composed of indices corresponding to six transform kernels. Similarly, each index constituting the transform kernel set of Fig. 26(a) may correspond to any one of the transform kernel combinations (sub-transform kernel sets) of Table 3. For example, the 25 combinations of Table 3 may be indexed from 0 to 24, respectively, and an index included in any one of the transform kernel sets of Fig. 26 may correspond to a combination of Table 3. The transform kernel set of Fig. 26(a) may be determined based on the size of the current block (coding block, transform block) and the intra prediction mode of the current block. Fig. 26(b) is a diagram showing the first transform kernel set (T0) among the transform kernel sets of Fig. 26(a). The indices of the transformation kernel set can be grouped into multiple groups based on a preset convention. That is, the number of indices of the transformation kernel set to be grouped can be adaptively set. At this time, the group can be composed of three or more. Referring to Fig. 26(b), the indices of the transformation kernel set can be grouped based on multiple reference values ​​(e.g., a first reference value, a second reference value). For example, if the grouping value is less than or equal to the first reference value, a group including one index (18) is selected. If the grouping value is greater than the first reference value and less than or equal to the second reference value, a group including four indices (18, 2, 17, 23) is selected. If the grouping value is greater than the second reference value, a group including six indices (18, 24, 17, 23, 8, 12) can be selected. If the grouping value is compared to the reference value and the corresponding group is selected as per the pre-promised settings, there may be a complexity reduction effect compared to signaling / parsing one of the six indices (transformation kernels) for the current block.At this time, the reference value may be the sum of the conversion coefficients, and the first reference value and the second reference value may be determined based on the sum of the conversion coefficients. For example, the first reference value may be 6, and the second reference value may be 32.

[0310] Separate signaling may be required to indicate the indices included in the group selected by comparing the reference value and the grouping value. For example, as shown in Fig. 26(b), if the grouping value is greater than the first reference value and less than or equal to the second reference value, the selected group may be the group consisting of indices (18, 24, 17, 23). In this case, separate signaling may be required to indicate each of the four indices. Similarly, if the grouping value is greater than the second reference value, the selected group may be the group consisting of indices (18, 24, 17, 23, 8, 12). In this case, separate signaling (mts_idx) may be required to indicate each of the six indices. The indices within the transform kernel set may be indicated by the mts_idx described above. In this case, mts_idx may have a fixed bit size. For example, mts_idx for six indices may have a 3-bit size. Alternatively, mts_idx can be signaled using the Truncated Unary Binarization (TB) method. mts_idx can be coded using the TB method, with context-model-based CABAC coding applied to the first bin and TB-based CABAC coding applied to the remaining bins.

[0311] Meanwhile, if the grouping value is equal to or less than the first reference value, the selected group may be a group composed of the indices of (18). In this case, since the group is composed of only one index, separate signaling to indicate one index may not be necessary.

[0312] FIG. 27 is a diagram illustrating a process for restoring a residual signal according to one embodiment of the present invention.

[0313] The residual signal, which is the difference between the original signal and the predicted signal, has the characteristic that the energy distribution of the signal changes depending on the prediction method. Therefore, if the transform kernel is adaptively selected according to the prediction method, such as MTS, the encoding efficiency can be improved. In addition, if the transform using only MTS or DCT2 kernels is called a first transform, the video signal processing device can also improve the encoding efficiency by additionally performing a second transform on the first-transformed coefficient block. The second transform is particularly effective in terms of energy compaction for predicted residual signal blocks within the screen, where strong energy is likely to exist in directions other than the horizontal or vertical directions of the residual signal block.

[0314] Referring to FIG. 27, a video signal processing device can parse syntax elements related to a residual signal included in a bitstream and restore quantization coefficients through inverse binarization based on the parsing result. The video signal processing device can obtain transform coefficients by performing inverse quantization on the restored quantization coefficients. The video signal processing device can restore a residual signal block by performing inverse transform on the transform coefficients. At this time, the inverse transform can be applied to a block to which transform skip (TS) is not applied. The video signal processing device can perform inverse transform in the order of a second inverse transform and a first inverse transform. At this time, the second inverse transform may be omitted. For example, if the current block is encoded in an inter prediction mode, the second inverse transform may be omitted. In addition, the second inverse transform may be omitted depending on the size of the current block. The restored residual signal includes a quantization error, and the second transform can reduce the quantization error compared to when only the first transform is performed by changing the energy distribution of the residual signal.

[0315] FIG. 28 is a diagram showing a region-of-interest (ROI) of a block to which a secondary transformation is applied according to one embodiment of the present invention.

[0316] According to one embodiment of the present specification, the numbers indicated in the sub-blocks in FIG. 28 may be sub-block indexes, and the sub-block indexes may be scanned in order from small numbers to large numbers.

[0317] Fig. 28(a) shows the ROI of LFNST4. The ROI of LFNST4 can be an ROI for a transform block of size 4 x N or N x 4. In this case, N can be an integer between 4 and 128. Referring to Fig. 28(a), the ROI of LFNST4 can be an ROI in a 16 x 4 block composed of 4 subblocks (subblocks 0 to 3). In this case, the ROI is one subblock of size 4 x 4, and referring to Fig. 28(a), the ROI corresponds to subblock 0. The number of input samples of the ROI can be 16. The forward transform matrix of LFNST4 can be R x 16. In this case, R can be 4, 8, 16, etc. For example, if R is 16, the number of transform coefficients generated after transformation can be 16.

[0318] Fig. 28(b) shows the ROI of LFNST8. The ROI of LFNST8 can be an ROI for a transform block of size 8 x N or N x 8. In this case, N can be an integer between 8 and 128. Referring to Fig. 28(b), the ROI of LFNST8 can be an ROI in a 16 x 8 block composed of 8 subblocks (subblocks 0 to 7). In this case, the ROI can be an area corresponding to 4 subblocks of size 4 x 4, and referring to Fig. 28(b), the ROI corresponds to subblocks 0, 1, 2, and 3. The number of input samples of the ROI can be 64. The forward transform matrix of LFNST8 can be R x 64. In this case, R can be 8, 16, 32, 64, etc. For example, if R is 32, the number of conversion coefficients generated after conversion can be 32.

[0319] Fig. 28(c) shows the ROI of LFNST16. The ROI of LFNST8 can be an ROI for a transform block of size 16 x N or N x 16. In this case, N can be an integer between 16 and 128. Referring to Fig. 28(c), the ROI of LFNST16 can be an ROI in a 16 x 16 block composed of 16 subblocks (subblocks 0 to 15). In this case, the ROI can be an area corresponding to six subblocks of size 4 x 4, and referring to Fig. 28(b), the ROI corresponds to subblocks 0, 1, 2, 3, 4, and 5. The number of input samples of the ROI can be 96. The forward transform matrix of LFNST16 can be R x 96. In this case, R can be 8, 16, 32, 64, 96, etc. For example, if R is 32, the number of conversion coefficients generated after conversion can be 32.

[0320] FIG. 29 is a diagram showing a method of applying a second-order transform (LFNST) according to one embodiment of the present invention.

[0321] The secondary transform can be expressed as the product of the matrix of the secondary transform kernel and the first-transformed coefficient vector. In other words, it can be interpreted as mapping the first-transformed coefficients to another space. At this time, if the number of coefficients subjected to the secondary transform is reduced, that is, the number of basis vectors constituting the second-transform kernel is reduced, the amount of computation required for the second transform and the memory capacity required to store the transform kernel can be reduced. For example, when a video signal processing device performs the second transform for an area corresponding to the upper left ROI of a transform block, if the number of coefficients subjected to the second transform is reduced to 32, a second-transform kernel of size 32 x 96 can be applied, and an inverse second-transform kernel of size 96x32 can be applied.

[0322] Referring to FIG. 29, the encoder can perform a forward primary transform on a residual signal block to obtain a primary transformed coefficient block. At this time, the residual signal may be a signal obtained by intra prediction. The size of the primary transformed coefficient block may be M x N. The encoder can perform a forward primary transform on a residual signal block where the value of min(M,N) is 16 to obtain a primary transformed coefficient block. Then, the encoder can perform a 32 x 96 secondary transform (LFNST) on samples of the upper left ROI area of ​​the primary transformed coefficient block. In addition, the encoder can perform a forward primary transform on a residual signal block where the value of min(M,N) is 8 to obtain a primary transformed coefficient block. And, the encoder can perform a secondary transformation on samples in the upper left ROI area of ​​the first transformed coefficient block.

[0323] Referring to FIG. 29, the transform coefficients of the entire transform block size including the secondary transform coefficients can be quantized, and information about the quantized transform coefficients can be included in the bitstream. In addition, the bitstream can include a syntax element (lfnst_idx) related to the secondary transform. Specifically, the bitstream can include information indicating whether the secondary transform is applied to the current block and the transform kernel.

[0324] Referring to FIG. 29, the decoder can parse quantized transform coefficients from a bitstream and obtain transform coefficients through de-quantization. The decoder can determine whether to perform an inverse secondary transform (Inverse LFNST) on the current transform block based on syntax elements related to the secondary transform. If the inverse secondary transform is applied to the current transform block, 16 or 32 transform coefficients can be input to the inverse secondary transform depending on the size of the transform block. The number of transform coefficients that become the input of the inverse secondary transform can be the same as the number of transform coefficients obtained by the encoder performing the secondary transform. The decoder can obtain the first-transformed coefficients through the product of the vectorized transform coefficients and the inverse secondary transform kernel matrix. The inverse secondary transform kernel can be determined based on the size of the transform block, the intra prediction mode, and the syntax elements indicating the transform kernel. The inverse secondary transform kernel matrix can be a transpose matrix of the secondary transform kernel matrix, and considering the complexity of the implementation, the elements of the kernel matrix can be integers expressed in 10-bit or 8-bit precision. The primary transform coefficients obtained through the inverse secondary transform are in the form of vectors, so they can be expressed as two-dimensional data. The primary transform coefficients can depend on the intra prediction mode. The mapping relationship based on the intra prediction mode applied by the encoder can be applied in the same manner. The decoder can obtain a residual signal by performing an inverse primary transform on a transform coefficient block of the entire transform block size including the transform coefficients obtained by performing the inverse secondary transform. The process described through Fig. 29 may include a scaling process using a bit shift operation.

[0325] FIG. 30 is a diagram showing a mapping relationship between an intra prediction mode and a set of transformation kernels for secondary transformation according to one embodiment of the present invention.

[0326] A set of transform kernels for LFNST applied to a transform block can be determined for each intra prediction mode of the transform block. One set of transform kernels can be composed of multiple LFNST kernels. For example, one set of transform kernels can be composed of three or four LFNST kernels. There can be 35 sets of transform kernels, and each set of transform kernels can be indexed with an index from 0 to 34. Intra prediction mode indices -14 to -1 and 67 to 80 corresponding to the extended angle mode can be mapped to the set of transform kernels with index 2.

[0327] FIG. 31 is a diagram showing the relationship between the input vector of the secondary transformation and the intra prediction mode according to one embodiment of the present invention.

[0328] The secondary transform can be calculated by multiplying the secondary transform kernel matrix by the input vector. The video signal processing device can organize the coefficients in the upper left sub-block of the first transformed coefficient block in the form of a vector. The vector can be organized depending on the intra prediction mode. For example, if the intra prediction mode is a prediction mode corresponding to an index less than or equal to index 34 of FIG. 6, or an INTRA_LT_CCLM, INTRA_T_CCLM, or INTRA_L_CCLM mode that predicts chroma samples using the linear relationship of chroma, the video signal processing device can organize the coefficients in the form of a vector by horizontally scanning the upper left sub-block of the first transformed coefficient block. The element of the i-th row and j-th column of the upper left nxn block of the first transformed coefficient block can be described as x_ij. At this time, the vectorized coefficients are [x_00, x_01, ..., x_0n-1, x_10, x_11, …]. , x_1n-1, ..., x_n-10, x_n-11, ..., x_n-1n-1]. On the other hand, if the intra prediction mode is a prediction mode corresponding to an index greater than index 34 of Fig. 6, the video signal processing device can vertically scan the upper left sub-block of the first transformed coefficient block to configure the coefficients in vector form. The vectorized coefficients can be expressed as [x_00, x_10, ..., x_n-10, x_01, x_11, ..., x_n-11, ..., x_0n-1, x_1n-1, ..., x_n-1n-1].

[0329] Since the secondary transformed coefficients are in vector form, they can be expressed as two-dimensional data. The secondary transformed coefficients can be assigned to the upper-left sub-block of the transform block according to a preset scanning order. The preset scanning order may be an up-right diagonal scanning order.

[0330] Figure 32 shows the application block and kernel size of NSPT (Non-Separable Primary Transform) according to one embodiment of the present invention.

[0331] NSPT is a method that transforms the transformation that previously applied DCT-II and LFNST separately to a block of a specific size in a single step. Similar to LFNST, NSPT is a non-separable transformation. For NSPT, 35 transformation sets based on intra prediction modes can be set, and each transformation set can be composed of three kernel candidates. The encoder can derive a transformation kernel set according to the intra prediction mode, and generate and signal a bitstream that includes an index indicating the optimal candidate among the three candidates in the derived transformation kernel set. The decoder can inversely transform the current transformation coefficients based on the optimal candidate determined by parsing the index, and obtain a residual block. Referring to Figure 32, NSPT WxH can mean a case where NSPT is applied to a block of size WxH. NSPT 4x4 has a 16x16 kernel size, NSPT 4x8 / NSPT 8x4 has a 32x20 kernel size, NSPT 4x16 / NSPT 16x4 has a 64x24 kernel size, NSPT 8x8 has a 64x32 kernel size, NSPT 8x16 / NSPT 16x8 has a 128x40 kernel size, NSPT 4x32 / NSPT 32x4 has a 128x20 kernel size, and NSPT 8x32 / NSPT 32x8 can have a 256x24 kernel size.

[0332] The coefficients of a 4x4 block to which NSPT is applied may not be zeroed out. 12 coefficients of a 4x8 or 8x4 block may be zeroed out, 32 coefficients of an 8x8 block may be zeroed out, 40 coefficients of a 4x16 or 16x4 block may be zeroed out, and 88 coefficients of an 8x16 or 16x8 block may be zeroed out. The number of coefficients that are zeroed out may vary depending on the size of the NSPT kernel. Zeroing out coefficients may mean that the values ​​of the coefficients are set to 0.

[0333] Fig. 33 illustrates an intra prediction mode derivation method for applying MTS / LFNST / NSPT according to one embodiment of the present invention.

[0334] At least one of MTS / LFNST / NSPT can be applied to a chroma block to which DBV is applied. Intra prediction mode information may be required for the transformation method of MTS / LFNST / NSPT to be applied. A video signal processing device can determine a transformation kernel set based on the intra prediction mode information, select (decide) a transformation kernel within the transformation kernel set, and perform transformation based on the selected kernel. At this time, the transformation kernel within the transformation kernel set can be indicated by information (syntax element) within the bitstream. The following describes a method for determining the intra prediction mode of a chroma block to which DBV is applied.

[0335] i) The intra prediction mode of the chroma block may be a preset intra prediction mode. For example, the preset intra prediction mode may be a planar mode or a DC mode.

[0336] ii) The intra prediction mode of the chroma block may be an intra prediction mode set in a reference block indicated by a block vector at a specific position of a luma block corresponding to the chroma block. At this time, the intra prediction mode of the chroma block may be an intra prediction mode stored at a specific position of the reference block. For example, the luma block corresponding to the chroma A block of FIG. 33(b) may be the luma A block of FIG. 33(a). The reference block indicated by the block vector at the TL position of the luma A block may be the block of FIG. 33(c). At this time, the intra prediction mode of the chroma block may be an intra prediction mode stored at one of {TL, TR, C, BL, BR, 1, 2, 3, 4} of the preset positions of the reference block. The search order in which the video signal searches for the preset positions may be TL-TR-C-BL-BR-1-2-3-4. In Fig. 33, the block vector of the TL position of the luma A block is used, but the present invention is not limited thereto, and a block vector determined based on a cost among the block vector list may be used. Alternatively, the block vector of the position where the block vector is first stored among C-TR-TL-BL-BR of the luma A block may be used. Alternatively, the block vector of the position to which the basic position and the basic position-based distance offset are applied among C-TR-TL-BL-BR of the luma A block may be used. The video signal processing device may determine and apply a set of MTS / LFNST / NSPT kernels based on the intra prediction mode of the chroma block.

[0337] iii) The intra prediction mode of the chroma block can be determined based on intra prediction mode information of a preset position of a luma block corresponding to the chroma block. The preset position may be a position for searching a block vector within the luma block. An intra prediction mode may not exist in a position where a block vector is stored, and if the block vector is not stored, an intra prediction mode may be stored. For example, referring to FIG. 33(a), the preset positions may be C, TL, TR, BL, and BR of FIG. 33(a). The video signal processing device may perform a search for only one of the preset positions, and if there is no intra prediction mode in the searched position, the default intra prediction mode may be used. The default intra prediction mode may be a planar mode or a DC mode. Alternatively, the video signal processing device may search for a plurality of positions among the preset positions and use the intra prediction mode of the first searched position. Meanwhile, if the video signal processing device searches multiple locations and no intra prediction mode exists, the default intra prediction mode can be used.

[0338] Figure 34 illustrates syntax elements related to DBV according to one embodiment of the present invention.

[0339] Figure 34 illustrates syntax elements related to DBV included in the SPS RBSP syntax. sps_chroma_format_idc may be a syntax element indicating the relationship between chroma sampling and luma sampling. The relationship between chroma sampling and luma sampling may be as shown in Table 4. If the value of sps_chroma_format_idc is 0, it may indicate that the current block only has a luma component without a chroma component.

[0340]

[0341] sps_ibc_enabled_flag can indicate whether IBC mode is enabled at the sequence level. A value of sps_ibc_enabled_flag of 1 indicates that IBC mode is enabled, while a value of 0 indicates that IBC mode is not enabled. If sps_ibc_enabled_flag is not present in the syntax, the value of sps_ibc_enabled_flag is inferred to be 0. (sps_ibc_enabled_flag equal to 1 specifies that the IBC prediction mode is enabled for the CLVS. sps_ibc_enabled_flag equal to 0 specifies that the IBC prediction mode is disabled for the CLVS. When sps_ibc_enabled_flag is not present, it is inferred to be equal to 0.)

[0342] sps_intra_tmp_enabled_flag can indicate whether Intra TMP is enabled or not at the sequence level. If the value of sps_intra_tmp_enabled_flag is 1, it indicates that Intra TMP is enabled, and if the value of sps_intra_tmp_enabled_flag is 0, it indicates that Intra TMP is not enabled. If sps_intra_tmp_enabled_flag does not exist in the syntax, the value of sps_intra_tmp_enabled_flag can be inferred to be 0. ( sps_intra_tmp_enabled_flag equal to 1 specifies that the Intra TMP prediction mode is enabled for the CLVS. sps_intra_tmp_enabled_flag equal to 0 specifies that the Intra TMP prediction mode is disabled for the CLVS. When sps_intra_tmp_enabled_flag is not present, it is inferred to be equal to 0.)

[0343] sps_dbv_chroma_enabled_flag can indicate whether DBV of a chroma block is enabled at the sequence level. If the value of sps_dbv_chroma_enabled_flag is 1, it can indicate that DBV of a chroma block is enabled, and if the value of sps_dbv_chroma_enabled_flag is 0, it can indicate that DBV of a chroma block is not enabled. If sps_dbv_chroma_enabled_flag does not exist in the syntax, the value of sps_dbv_chroma_enabled_flag can be inferred to be 0. (sps_dbv_chroma_enabled_flag equal to 1 specifies that the Direct Block Vector for Chroma prediction mode is enabled for the CLVS. sps_dbv_chroma_enabled_flag equal to 0 specifies that the Direct Block Vector for Chroma prediction mode is disabled for the CLVS. When sps_dbv_chroma_enabled_flag is not present, it is inferred to be equal to 0.)

[0344] sps_dbv_chroma_enabled_flag can be signaled / parsed (3401) if i) sps_chroma_format_idc value is not equal to 0, and ii) sps_ibc_enabled_flag value is true (equal to 1) or sps_intra_tmp_enabled_flag value is true (equal to 1). In order for DBV to be applied to a chroma block, a block vector must exist in the luma block corresponding to the chroma block. Therefore, IBC mode or Intra TMP must be enabled, and the current block must also contain a chroma component.

[0345] There may be a syntax element for each of the CCCM-related modes (e.g., CCCM / MM-CCCM / CCLM / GLM / GL-CCCM / CCCM-ND / CCCM-MDF). That is, a flag (syntax element) for each of the CCCM-related modes in the SPS RBSP syntax may be defined, or whether the CCCM-related mode is enabled or disabled may be indicated based on each flag. At this time, in order for the flag for each of the CCCM-related modes to be signaled (parsed), the value of sps_chroma_format_idc may not be 0. In addition, a flag for whether the SPS level is enabled or disabled for the CCP merge mode may be defined, and in order for the flag for the CCP merge mode to be signaled (parsed), the value of any one of the flags for each of the CCCM-related modes may have to be 1 (true) or the value of sps_cclm_enabled_flag may have to be 1 (true).

[0346] Figure 35 illustrates the operation of BVG-CCCM (Block Vector Guided CCCM) according to one embodiment of the present invention.

[0347] The current chroma block can be predicted using the CCCM method (see Fig. 16). At this time, if the corresponding luma block is coded in Intra TMP or IBC mode, the video signal processing device can obtain adjacent surrounding luma samples of a reference luma block indicated by a block vector of a luma block (co-located luma block) corresponding to the current chroma block, and the video signal processing device can obtain adjacent surrounding chroma samples of the reference chroma block indicated by the block vector of the current chroma block. The video signal processing device can derive CCCM parameters based on the obtained surrounding luma samples and luma chroma samples. The video signal processing device can perform chroma prediction using the luma sample corresponding to the current chroma block as an input value based on the derived CCCM parameters. BVG-CCCM can be applied when i) the chroma prediction mode index indicates LM_CHROMA_IDX or MMLM_CHROMA_IDX, ii) the luma block corresponding to the chroma block is coded in Intra TMP or IBC mode, iii) the partitioning methods of the luma and chroma blocks are different, iv) BVG-CCCM is enabled, the CCLM mode is enabled, and the CCCM mode is enabled, and v) a single model for CCCM or a multi-model is applicable. Whether BVG-CCCM in iv) is enabled can be indicated by the higher-level syntax (SPS level syntax) sps_bvg_cccm_enabled_flag, whether the CCLM mode is enabled can be indicated by the higher-level syntax sps_cclm_enabled_flag, and whether the CCCM mode is enabled can be indicated by the higher-level syntax sps_cccm_enabled_flag.ii) LM_CHROMA_IDX may be an index indicating whether a single model of CCLM is applied, and MMLM_CHROMA_IDX may be an index indicating whether a multi-model of CCLM is applied.

[0348] MTS, NSPT, or LFNST can also be applied to chroma blocks predicted using BVG-CCCM. Kernel sets for NSPT and LFNST can be defined based on intra prediction modes and block sizes, and a video signal processing device can perform transformation using specific kernels within the kernel set. The intra prediction mode for determining the transformation kernel can be determined using the method described above in FIG. 33.

[0349] Figure 36 illustrates syntax elements related to BVC-CCCM according to one embodiment of the present invention.

[0350] Figure 36 shows syntax elements related to BVC-CCCM included within the SPS RBSP syntax.

[0351] sps_bvg_cccm_enabled_flag can indicate whether BVG-CCCM is enabled. If the value of sps_bvg_cccm_enabled_flag is 1, it indicates that BVG-CCCM is enabled, and if the value of sps_bvg_cccm_enabled_flag is 0, it indicates that BVG-CCCM is not enabled. If sps_bvg_cccm_enabled_flag does not exist in the syntax, the value of sps_bvg_cccm_enabled_flag can be inferred to be 0. (sps_bvg_cccm_enabled_flag equal to 1 specifies that the Block Vector Guided CCCM prediction mode is enabled for the CLVS. sps_bvg_cccm_enabled_flag equal to 0 specifies that the Block Vector Guided CCCM prediction mode is disabled for the CLVS. When sps_bvg_cccm_enabled_flag is not present, it is inferred to be equal to 0.)

[0352] The conditions for sps_bvg_cccm_enabled_flag to be signaled (parsed) are as follows:

[0353] When i) the value of sps_chroma_format_idc is not 0, ii) the value of sps_cclm_enabled_flag is 1, and iii) the value of sps_ibc_enabled_flag is 1 or the value of sps_intra_tmp_enabled_flag is 1, sps_bvg_cccm_enabled_flag can be signaled (3601).

[0354] In addition, the conditions described below may be added to the above-described 3601 conditions. For example, a condition in which the value of sps_cccm_enabled_flag is 1 may be added. Or, a condition in which the value of the non-inter tool control enabled flag is 1 may be added. The non-inter tool control enabled flag may be a flag that controls on a per-tool basis.

[0355] Figure 37 illustrates TIMD according to one embodiment of the present invention.

[0356] Referring to FIG. 37, a video signal processing device can construct a template of a certain size using neighboring blocks of a current block. The video signal processing device can derive an intra prediction mode with the minimum cost for the template using reference samples of the template and use it as a prediction block of the current block. The derived intra prediction mode can be a prediction mode with the minimum cost calculated using the SATD method within 67 intra prediction modes or 131 extended intra prediction modes.

[0357] An example of how a video signal processing device performs TIMD is described.

[0358] 1) The video signal processing device can set (determine) the size of the template. The horizontal or vertical size of the template can be 4 by default. Additionally, if the horizontal or vertical size of the current block is 8 or less, the horizontal or vertical size of the template can be 2.

[0359] 2) The video signal processing device can set the type of the template. The template type can be divided into three types. For example, it can be divided into i) a type that uses only the left reference sample of the current block, ii) a type that uses only the upper reference sample of the current block, and iii) a type that uses both the left and upper reference samples of the current block. The type of the template can be determined by determining whether the surrounding blocks of the current block can be used as reference samples. If all the surrounding blocks cannot be used as reference samples, the TIMD mode can be set to Planar, and the TIMD blending process may not be performed. In addition, the video signal processing device can set a distortion parameter to be used in cost calculation according to the selected template type.

[0360] 3) The video signal processing device can configure a template for the current block based on the size of the template and the type of the template.

[0361] 4) The video signal processing device can derive the intra prediction directionality of surrounding blocks located to the left, above, upper left, upper right, and lower left of the current block to determine whether the current block has directionality. At this time, the locations of the surrounding blocks may be the locations (L, A, AL, AR, BL) shown in Fig. 38(b).

[0362] 5) If none of the neighboring blocks of the current block have directionality (or all of the neighboring blocks are in DC or planar mode), the video signal processing device may evaluate only the two modes, DC or planar, to select the optimal intra prediction mode with the minimum cost, and terminate the TIMD process. At this time, the TIMD blending process may not be performed.

[0363] 6) If at least one block among the surrounding blocks of the current block has directionality, the video signal processing device may perform the process described below. The process described below may be performed by all video signal processing devices based on the intra prediction modes in the MPM list to reduce the complexity required to evaluate the intra prediction mode.

[0364] Below, a process performed by a video signal processing device is described when at least one block among the surrounding blocks of the current block (6) has directionality.

[0365] 6-1) The video signal processing device can configure an MPM list.

[0366] 6-2) If DC mode, Horizontal mode, and Vertical mode are not in the MPM list, the video signal processing device can add DC mode, Horizontal mode, and Vertical mode to the MPM list. An MPM list with DC mode, Horizontal mode, and Vertical mode added can be described as a modified list.

[0367] 6-3) The video signal processing device can perform an evaluation on all intra prediction modes in the modified list to obtain the most optimal (primary best) mode and the second best (secondary best) mode.

[0368] 6-4) For further performance enhancement, the video signal processing device may additionally evaluate modes whose indices are 1 less than or greater than the indices of the most optimal mode and the second optimal mode, respectively. If the cost of the additionally evaluated mode is less than the cost of the optimal mode and the second optimal mode, the optimal mode and the second optimal mode may be changed to the additionally evaluated mode, respectively.

[0369] 6-5) The video signal processing device can determine whether to perform blending (mode). If the difference between the cost of the most optimal mode and the cost of the second optimal mode is less than the cost of the most optimal mode, the video signal processing device can perform blending. At this time, the weights for the most optimal mode and the second optimal mode can be determined based on the costs of the most optimal mode and the second optimal mode, respectively. If the difference between the cost of the most optimal mode and the cost of the second optimal mode is greater than the cost of the most optimal mode, a prediction block can be generated using only the most optimal mode, and the video signal processing device can not perform blending.

[0370] When the prediction mode of a surrounding block is a TIMD mode during the process of deriving an MPM list by a video signal processing device, the video signal processing device may perform a process of converting the range of the directional mode into the range of a general intra prediction directional mode. This is because TIMD can use one of 131 extended intra prediction modes.

[0371] In addition, in the process of deriving the MPM list, if the surrounding block is a block that has performed inter prediction, if Intra Template Matching Prediction (TMP) is applied to the surrounding block, or if the surrounding block is predicted in Intra Block Copy (IBC) mode, the video signal processing device may add the corresponding intra prediction mode to the MPM list in each case if the intra prediction mode of the corresponding block indicated by the motion vector (or block vector) exists. If the corresponding intra prediction mode does not exist in each case, the video signal processing device may use a default value to perform TIMD. The default value may be a planar mode or a DC mode. The intra prediction modes corresponding to the three cases above may be stored in an IPM (Intra Prediction Mode) buffer, and the video signal processing device may call and use the value stored in the IPM. IPM information of a block coded in an intra prediction mode may also be stored in the IPM buffer and used. The variable indicating the intra prediction mode of a block coded with the intra prediction mode can store the actually used intra prediction mode, but if the prediction mode of the surrounding block is intra TMP or IBC or inter mode, it can be set to the default value, DC mode. Therefore, the intra prediction mode indication value may be different depending on the properties of the coded block. A block coded with the intra mode uses the same value whether it uses the variable indicating the intra prediction mode or the IPM buffer value, but if the prediction mode of the surrounding block is intra TMP or IBC or inter mode, the values ​​may be different. In the three cases above, the default setting mode, DC mode, can be replaced by the value stored in the IPM buffer.

[0372] FIG. 38 illustrates a method for generating a final prediction block using an inter prediction block and an intra prediction block according to one embodiment of the present invention.

[0373] Referring to FIG. 38(a), a video signal processing device can generate a final prediction block by performing a GPM (Geometric partitioning mode) with Inter and Intra method using an inter prediction block and an intra prediction block.

[0374] The intra prediction mode of an intra prediction block can be determined based on the DIMD, TIMD, and intra prediction modes of surrounding neighboring blocks. Referring to Fig. 38(b), a video signal processing device can derive an intra prediction mode from a neighboring block of a current prediction block. At this time, the location of the neighboring block can be a preset location (A, L, AL, AR, BL in Fig. 38(b)). The video signal processing device can derive the intra prediction mode by checking the coding mode of the neighboring block as described with reference to Fig. 37. A block coded with an intra mode can be derived from a variable indicating the intra prediction mode, and a block coded with Inter or IBC can be derived using the intra prediction mode stored in the IPM buffer. The video signal processing device can configure an MPM list for GPM with Inter and Intra in the following order. The order described below takes into account blocks whose neighboring blocks are coded with Intra or Inter.

[0375] <GPM with Inter and Intra에 대한 MPM 리스트 구성 순서1>

[0376] 1) Intra prediction mode of an intra-coded block at L position, 2) Intra prediction mode of an intra-coded block at A position, 3) Intra prediction mode of an Inter-coded block at L position, 4) Intra prediction mode of an Inter-coded block at A position, 5) Intra prediction mode of an intra-coded block at BL position, 6) Intra prediction mode of an intra-coded block at AR position, 7) Intra prediction mode of an intra-coded block at AL position, 8) Intra prediction mode of an inter-coded block at BL position, 9) Intra prediction mode of an inter-coded block at AR position, 10) Intra prediction mode of an inter-coded block at AL position

[0377] <GPM with Inter and Intra에 대한 MPM 리스트 구성 순서1> In the order of 2), 4), 8), 9), and 10), the IBC-coded block can derive the intra prediction mode of the inter or IBC-coded block.

[0378] <GPM with Inter and Intra에 대한 MPM 리스트 구성 순서2>

[0379] <GPM with Inter and Intra에 대한 MPM 리스트 구성 순서2> is configured so that IBC has higher priority than Inter.

[0380] 1) Intra prediction mode of an intra-coded block at L position, 2) Intra prediction mode of an intra-coded block at A position, 3) Intra prediction mode of an IBC-coded block at L position, 4) Intra prediction mode of an IBC-coded block at A position, 5) Intra prediction mode of an Inter-coded block at L position, 6) Intra prediction mode of an Inter-coded block at A position, 7) Intra prediction mode of an intra-coded block at BL position, 8) Intra prediction mode of an intra-coded block at AR position, 9) Intra prediction mode of an intra-coded block at AL position, 10) Intra prediction mode of an IBC-coded block at BL position, 11) Intra prediction mode of an IBC-coded block at AR position, 12) Intra prediction mode of an IBC-coded block at AL position, 13) Intra prediction mode of an inter-coded block at BL position, 14) Intra of an inter-coded block at AR position Prediction mode, 15) Intra prediction mode of inter-coded blocks at AL location

[0381] <GPM with Inter and Intra에 대한 MPM 리스트 구성 순서2> The positions of Inter and IBC can be interchanged.

[0382] When a video signal processing device derives an intra prediction mode of a neighboring block of a current block, if the neighboring block is coded in an intra mode, the video signal processing device can derive the intra prediction mode in the following three ways.

[0383] 1. Planar mode when coded with MIP

[0384] 2. Planar mode when coded with intra TMP

[0385] 3. Intra prediction mode indicator variable value except for the above 2 cases

[0386] Although Intra TMP belongs to the Intra mode, like IBC, a template matching block within the current reference picture can be used as a prediction block, and a block vector can be derived for this purpose. If the intra prediction mode used in the template matching block exists, the video signal processing device stores the value of the intra prediction mode in the IPM buffer. If not, a default value is stored in the IPM buffer, and this can be used for a block coded as Intra TMP. Therefore, the intra prediction mode of a block coded as Intra TMP in Intra mode can be determined using the intra prediction mode stored in the IPM buffer instead of the planar mode or a preset mode. This method can be applied when deriving an intra prediction mode from a TIMD or neighboring block. In the MPM list construction order for GPM with Inter and Intra, a block coded as Intra TMP can be considered separately from the Intra mode. For example, a block coded as Intra TMP can have the same rank as Inter or IBC. Alternatively, the order of MPM list construction for GPM with Inter and Intra can be determined based on multiple combinations of Inter, IBC, and Intra TMP.

[0387] The intra prediction mode stored in the IPM buffer can be determined using the DIMD method. The intra prediction mode with the highest frequency can be stored in the IPM buffer using the DIMD method.

[0388] FIG. 39 illustrates a method for deriving an intra prediction mode from a luma block corresponding to a chroma block according to one embodiment of the present invention.

[0389] Referring to FIG. 39, a video signal processing device can derive an intra prediction mode at a preset position of Luma A (FIG. 39(a)) corresponding to Chroma A (FIG. 39(b)). There are five preset positions, which can be C, TL, TR, BL, and BR. Previously, the intra prediction indicator variable value used in the luma PU at the C position could be used, or if it was MIP or intra TMP, the planar mode value could be used.

[0390] If each luma PU of the five preset positions is an intra mode and is not an intra TMP, the intra prediction indicator variable value used in each PU can be used to derive the intra prediction mode. In addition, if each luma PU of the five preset positions is an intra mode and is an intra TMP, the IPM value of the IPM buffer can be used to derive the intra prediction mode. In addition, if each luma PU of the five preset positions is an IBC coded block, the IPM value of the IPM buffer can be used to derive the intra prediction mode. Alternatively, if each luma PU of the five preset positions is an inter coded block, the IPM value stored in the IPM buffer can be used.

[0391] A planar mode can be used to derive an intra prediction mode when each luma PU of the five preset positions is an intra mode and an intra TMP.

[0392] If each luma PU of the five preset positions is an intra mode, the video signal processing device can use the intra prediction indicator variable value used by each luma PU to derive the intra prediction mode. If each luma PU of the five preset positions is coded with IBC, the video signal processing device can use the IPM value of the IPM buffer to derive the intra prediction mode. Alternatively, if each luma PU of the five preset positions is coded with inter, the video signal processing device can use the IPM value stored in the IPM buffer to derive the intra prediction mode.

[0393] The order in which the video signal processing device searches for each of the five preset luma PUs to induce the intra prediction mode may be, for example, C, TL, TR, BL, and BR. The illustrated order is only an example and is not limited thereto.

[0394] The video signal processing device first checks whether the luma PU at the center position (C) of the corresponding luma block is the intra mode, and if so, can use the intra prediction indicator variable value to derive the intra prediction mode. At this time, the planar mode may be used for the intra TMP or the IPM value stored in the IPM buffer may be used. The video signal processing device first checks whether the luma PU at the center position (C) of the corresponding luma block is the intra mode, and if not, can check whether it is the IBC or inter mode, and if it is the IBC or inter mode, can use the IPM value stored in the IPM buffer to derive the intra prediction mode. The video signal processing device may use the default planar or DC mode to derive the intra prediction mode if the luma PU at the center position (C) of the corresponding luma block is not the intra mode, the IBC, or the inter mode.

[0395] A video signal processing device can, in the operation of DBV (Direct Block Vector) for Chroma (DBV for Chroma) and DBV-CCCM, check whether a block vector exists at a preset position of a luma block corresponding to the chroma of a current prediction block, and perform chroma prediction based on the check result. The preset positions for chroma prediction may be the same as positions (C, TL, TR, BL, BR) of FIG. 39(a). When performing DBV and DBV-CCCM operations for chroma, if there is a block coded with IBC or intra TMP at each of the five positions, the video signal processing device can perform DBV and DBV-CCCM operations for chroma, and if not, search for three additional positions. The three additional positions can be derived by applying offsets to the preset positions. The offsets can be obtained by multiplying coordinates (1, 1), (0, 1), and (1, 0) by a preset distance value (2 or 4 or 8). The three new locations may be locations obtained by subtracting the above-described offset from each of the base locations. In other words, if there is no block coded as IBC or Intra TMP at each location (C, TL, TR, BL, BR) in FIG. 39(a), the video signal processing device may search for additional locations by applying the offset and preset distance values ​​to each location.

[0396] FIG. 40 illustrates a method for determining a reference pixel line based on a template according to one embodiment of the present invention.

[0397] Below, we describe a method for determining the optimal reference pixel line for the current block (for restoration of the current block) based on the template.

[0398] Referring to FIG. 40, a video signal processing device can construct a reference template using reference pixel lines adjacent to a current block. The video signal processing device can generate prediction samples for the positions of the reference templates using reference pixel lines 1, 2, 3, etc. The video signal processing device can calculate a cost between the generated prediction samples and samples of the reference template. At this time, the cost can be calculated using a method such as SAD (Sum of Absolute Differences) or MRSAD (Mean-Removed SAD). The reference pixel corresponding to the minimum cost may be the optimal reference pixel. In addition, the encoder can rearrange the calculated costs in ascending order, construct a list for reference pixel lines, and then generate and signal a bitstream including information on an index for the optimal reference pixel line. The decoder can construct a list for reference pixel lines using the above-described method, parse the index for the optimal reference pixel line included in the bitstream, and generate a prediction sample using the reference pixel line indicated by the index. As described herein, a method for a video signal processing device to determine a reference pixel line based on a template can be described as a Template-based Multiple Reference Line (TMRL) method or a TMRL intra prediction method.

[0399] In order to reduce the complexity of determining the optimal reference pixel line, the optimal reference pixel line may not be used for all intra prediction modes, but may be used only for the intra prediction modes included in the MPM list. That is, the encoder can construct a list of reference pixel lines by combining intra prediction modes included in the MPM list with multiple reference pixel lines, and calculate the cost between the prediction sample generated using each candidate in the list for the reference pixel line and the reference template. Then, the encoder can reconstruct the list in ascending order based on the cost and use only a few combinations with low costs. The encoder can generate and signal a bitstream that includes information about an index for the optimal combination information among the combination information (any intra prediction mode and any reference pixel lines) in the reconstructed list. The decoder can construct a list for the same reference pixel lines using the above-described method, parse the index for the optimal combination information included in the bitstream, and generate a prediction sample using the optimal combination information indicated by the index.

[0400] When prediction samples are generated using multiple reference pixel lines, different weights may be applied to each prediction sample generated through each reference pixel line. At this time, a template-based method may be used to derive optimal weights. That is, the encoder may construct a list for reference pixel lines by combining intra prediction modes included in the MPM list, multiple reference pixel lines, and weights (for example, one of 3:1 and 2:2) for samples predicted by each reference pixel line, and may calculate the cost between the prediction sample generated using each candidate in the list for the reference pixel line and the reference template. Then, the video signal processing device may rearrange the MPM list in ascending order based on the cost, and then reconstruct the MPM list using only a few combinations with low costs. The encoder can generate and signal a bitstream including information about an index for optimal combination information among combination information within the reconstructed list (any intra prediction mode, any reference pixel lines, and weights (e.g., one of 3:1 and 2:2) for samples predicted by each reference pixel line). The decoder can construct a list for the same reference pixel lines using the above-described method, and then parse the index for optimal combination information included in the bitstream to generate a prediction sample using the optimal combination information indicated by the index.

[0401] In the method for determining a reference pixel line based on a template described through FIG. 40, reference pixel lines adjacent to the current block can be used to construct the template. Accordingly, reference pixel lines determined based on the template can be determined using reference pixel lines not adjacent to the current block.

[0402] Figure 41 shows a block vector related to an IBC encoding method according to one embodiment of the present invention.

[0403] The IBC coding method (IBC mode) is a method of finding the part (reference block) that is most similar to the current block in an already reconstructed area within the current picture and using the reference block as a prediction block for the current block. At this time, the encoder can generate a bitstream that includes information related to a block vector, which is the distance between the current block and the reference block. The decoder can parse information related to the block vector included in the BeastStream to calculate or set the block vector for the current block. The IBC coding method can be applied to chrominance blocks. In the chrominance block, the block vector of the luminance block corresponding to the chrominance block can be used as the block vector of the chrominance block without finding a new block vector, and this coding method can be called DBV (Direct Block Vector) mode.

[0404] The RRIBC (Reconstruction-Reordered IBC) encoding mode can be used in IBC blocks (blocks to which the IBC encoding method is applied). RRIBC can be composed of vertical flips and horizontal flips. The reconstructed block of a block to which RRIBC is applied is flipped according to the RRIBC type of the current block. The encoder can flip the original block to be encoded before finding the most similar part to the current block from the reference picture. That is, the most similar part from the reference picture is found using the flipped original block. Therefore, the prediction block uses a block that has not been flipped, and the residual block can also be a block that has not been flipped. The decoder can flip the final reconstructed block according to the RRIBC type of the current block.

[0405] FIG. 42 illustrates a method for predicting a current block using RRIBC in the horizontal direction according to one embodiment of the present invention.

[0406] FIG. 43 illustrates a method for predicting a current block using RRIBC in the vertical direction according to one embodiment of the present invention.

[0407] In Figures 42 and 43, (Xn, Yn) represents the central position of the surrounding blocks, and (Xc, Yc) represents the central position of the current block. BV n h , BV n v represents the horizontal block vector and vertical block vector of the surrounding blocks, respectively, and BV C h , BV C v represent the horizontal block vector and the vertical block vector of the current block, respectively. As shown in Fig. 39, if the RRIBC type of the current block is horizontal, BV C h is 2 * (Xn - Xc) + BV n h It can be calculated as, and as in Fig. 43, if the RRIBC type of the current block is in the vertical direction, BV C v is 2 * (yn - yc) + BV n v can be calculated as . At this time, BV n h , BV n v Since it uses restored blocks, BV n h , BV n v The sign of can only be negative.

[0408] When the current block is encoded with RRIBC, the video signal processing device can determine an optimal block vector by constructing a block vector candidate list. At this time, the video signal processing device can construct the block vector candidate list according to the RRIBC type of the current block. For example, when the RRIBC type of the current block is horizontal, the video signal processing device can construct the block vector candidate list using only the neighboring blocks of the current block encoded with RRIBC in the horizontal direction. In addition, the video signal processing device can construct the block vector candidate list regardless of the RRIBC type of the current block. For example, when the RRIBC type of the current block is horizontal, the block vector candidate list can be constructed using not only the neighboring blocks of the current block encoded with RRIBC in the horizontal direction, but also the neighboring blocks of the current block encoded with RRIBC in the vertical direction, and / or blocks encoded with general motion, and / or blocks encoded with block vectors.

[0409] When a video signal processing device predicts a current block using general motion, the video signal processing device can construct a motion candidate list depending on whether the neighboring blocks of the current block are encoded in the IBC mode or the RRIBC mode. At this time, if the neighboring blocks of the current block are encoded in the RRIBC mode, the video signal processing device can construct the motion candidate list by additionally considering the RRIBC type. For example, when the video signal processing device constructs a motion candidate list for the current block, if the encoding mode of the neighboring blocks of the current block is the RRIBC mode and the RRIBC type is the vertical direction or the horizontal direction, the block vector of the neighboring blocks may not be included in the motion candidate list. Alternatively, when the video signal processing device predicts the current block using general motion, the video signal processing device can construct a motion candidate list regardless of the encoding mode of the neighboring blocks of the current block. In other words, the video signal processing device can construct a motion candidate list regardless of whether the neighboring blocks of the current block are encoded in the IBC mode or the RRIBC mode. For example, when a video signal processing device constructs a motion candidate list for a current block, the video signal processing device may include block vectors of the surrounding blocks in the motion candidate list even if the encoding mode of the surrounding blocks of the current block is the IBC mode and the RRIBC type is the vertical direction or the horizontal direction.

[0410] The RRIBC encoding method is effective for images with symmetrical characteristics. Symmetry can mean perfect horizontal (or vertical) symmetry with equal distances between the current block and the reference block around a central axis. The vertical (or horizontal) direction (symmetry axis) of the current block can be colinear with the vertical (or horizontal) direction (symmetry axis) of the reference block. In this case, the block vector in the vertical (or horizontal) direction can be set to '0', and the block vector in the horizontal direction can be set to any negative value other than '0'. The current block and the reference block can be configured to be symmetric with different distances apart from one central axis between the current block and the reference block, and the current block and the reference block can be located on different vertical lines. In this case, the block vector in the vertical direction can be set to any negative value other than '0'. That is, the video signal processing device can encode or decode the current block using a symmetric block having a block vector value of an arbitrary negative value other than '0' in both the horizontal and vertical directions.

[0411] Figure 44 shows a block vector of a block encoded in Intra TMP mode according to one embodiment of the present invention.

[0412] Referring to FIG. 44, the Intra TMP method (Intra TMP encoding mode) is a method in which a video signal processing device constructs a reference template using pixel values ​​of neighboring blocks adjacent to a current block, then finds the most similar part to the reference template in an already reconstructed area (reference block) within the current picture, and then uses the reference block (Ref. luma block of FIG. 44) as a prediction block for the current block. At this time, there may be more than one block vector used to generate a prediction block for the current luminance block, and FIG. 44 shows a case in which there are two block vectors. The video signal processing device can generate a prediction block for the luminance block by weighting and averaging reference blocks predicted from two block vectors of the current luminance block. In the case of a chrominance block, the video signal processing device can derive a block vector from a luminance block corresponding to the chrominance block, and then generate a chrominance prediction block using the block vector. There may be more than one block vector used to generate a prediction block for the current chrominance block, and Fig. 44 shows a case where there are two block vectors for the chrominance block. In this case, the block vector of the chrominance block may be the same as or different from the block vector derived from the luminance block.

[0413] Figure 45 illustrates a case where a current block is divided by GPM mode according to one embodiment of the present invention and the divided area is encoded in IBC mode.

[0414] The current block can be encoded or decoded in IBC-GPM mode. IBC-GPM mode may mean a mode in which the current block is divided using GPM mode, and the divided region is encoded in intra prediction mode or IBC prediction mode. Referring to Fig. 42, the current block can be divided into two regions based on a dotted line. At this time, one of the divided regions can be encoded in intra prediction mode, and the other can be encoded in IBC prediction mode. For example, the left region among the divided regions can be encoded in intra prediction mode, and the right region can be encoded in IBC prediction mode. At this time, the region encoded in IBC prediction mode can be encoded in Intra TMP prediction mode, and this can be described as IntraTMP-GPM mode.

[0415] Figure 46 illustrates how a current block is encoded in IBC-CIIP mode according to one embodiment of the present invention.

[0416] A video signal processing device can predict the current block using a weighted average between a block predicted in Intra mode and a block predicted in IBC mode, which can be referred to as IBC-CIIP mode. In this case, a block predicted in Intra TMP mode can be used instead of a block predicted in IBC mode, which can be referred to as Intra TMP-CIIP mode.

[0417] A video signal processing device can obtain a reconstructed luminance block by adding a residual signal for a luminance prediction block of a current block and a luminance block, and can configure a CCP model by using a correlation between the reconstructed luminance block and the luminance prediction block. At this time, the CCP model can be one of CCLM, MMLM, GLM, CCCM, MM-CCCM, GL-CCCM, CCCM-ND, and CCCM-MDF. The CCP model derived from the luminance block can be applied to a chrominance prediction block to generate a first chrominance prediction block to which the CCP model is applied. A final chrominance block can be generated by adding an error signal for the first chrominance prediction block and the chrominance block.

[0418] Figure 47 illustrates a method for deriving a multi-transform set and a LFNST / NSPT set according to one embodiment of the present invention.

[0419] Referring to Fig. 47(a), the encoder can select which one of MTS, DCT2 + LFNST, and NSPT to apply to the residual block, and can transform the residual block and obtain transform coefficients based on the selected transform method. At this time, the encoder can signal information on which transform method was applied by including it in the bitstream. If the MTS transform is applied to the residual block, the transform coefficients of the residual block can be obtained by applying the MTS transform, and LFNST and NSPT transforms may not be applied. If the DCT2 + LFNST transform is applied to the residual block, the encoder can apply the DCT2 transform to the residual block to obtain the first transform coefficients, and apply the LFNST transform to the first transform coefficients to obtain the second transform coefficients. At this time, the MTS and NSPT transforms may not be applied. When the NSPT transform is applied to the residual block, the video encoder can output transform coefficients by applying the NSPT transform to the residual block, and at this time, the MTS and DCT2 + LFNST transforms may not be applied to the residual block.

[0420] Referring to Fig. 47(b), the decoder can parse information on which transform method was applied from the bitstream, and determine whether to apply one of the inverse transform methods among MTS, DCT2 + LFNST, and NSPT to the current transform coefficient based on the parsed information. Then, the decoder can perform an inverse transform on the transform coefficient based on the determined transform method and obtain a residual block. If the MTS transform is applied to the current transform coefficient, the decoder can perform an inverse MTS transform on the transform coefficient to obtain a residual block, and at this time, the LFNST and NSPT inverse transforms may not be applied. If the DCT2 + LFNST transform is applied to the secondary transform coefficient, the decoder can perform an inverse LFNST transform on the secondary transform coefficient to output a primary transform coefficient, and perform an inverse DCT2 transform on the primary transform coefficient to obtain a residual block, and at this time, the MTS and NSPT transforms may not be applied. If the NSPT transform is applied to the current transform coefficients, the decoder can obtain a residual block by performing the NSPT inverse transform on the current transform coefficients, and at this time, the MTS and DCT2 + LFNST transforms may not be applied.

[0421] A video signal processing device can derive a transform kernel for each of MTS, LFNST, and NSPT transforms (or inverse transforms) using an intra prediction mode. In addition, the video signal processing device can determine which transform (or inverse transform) is applied among MTS, DCT2 + LFNST, and NSPT. At this time, in order to determine the transform, at least one or more of the horizontal and vertical sizes of the current block, whether the components of the current block are luminance components or chrominance components, whether the current block is a single tree or a dual tree, information on whether the current block is encoded in intra mode or inter mode, information on the encoding mode of the current block (e.g., IBC, Intra TMP, Merge, AMVP, GPM, SGPM, CCLM, CCCM), and information on the quantization parameters of the current block may be used.

[0422] Figure 48 shows a mapping table according to one embodiment of the present invention.

[0423] Figure 49 illustrates a conversion type set table according to one embodiment of the present invention.

[0424] Figure 50 shows a conversion type combination table according to one embodiment of the present invention.

[0425] Figure 51 shows a threshold value table for an IDT conversion type according to one embodiment of the present invention.

[0426] A method for selecting a set of multiple transforms available for a current block in a video signal processing device is described when a prediction block of a current block is generated using a vertical planar mode, a horizontal planar mode, or a conventional planar mode.

[0427] 1) First, the video signal processing device can derive the nSzIdxW and nSzIdxH values ​​based on the size of the current block to map the horizontal and vertical sizes of the current block into a single variable. nSzIdxW can be the minimum value among the logarithm of 2 calculated for the width of the current block, the difference of 2 with the decimal places discarded, and the three. nSzIdxH can be the minimum value among the logarithm of 2 calculated for the height of the current block, the difference of 2 with the decimal places discarded, and the three.

[0428] 2) Next, the video signal processing device can derive the intra-directional mode (predMode) of the current block. In the case of TIMD mode, the number of intra-prediction modes expanded from the existing 67 to 131 can be used, reducing the precision of the existing 67 modes.

[0429] 3) Next, the video signal processing device can derive the ucMode, nMdIdx, and isTrTransposed values.

[0430] A. If the current block is encoded in MIP mode, ucMode can be set to '0', nMdIdx can be set to '35', and isTrTransposed can be set to a value derived from MIP.

[0431] B. If the current block is not encoded in MIP mode, ucMode may be set to the intra directional mode (predMode) of the current block. predMode may mean an index value of the intra directional mode. predMode may be determined through an extended angle mode according to the aspect ratio of the current block. The video signal processing device may clip predMode to a value between 2 and 66. If the prediction block of the current block is generated using the vertical plane mode, the video signal processing device may reset predMode to the horizontal prediction mode (angle mode 18 in FIG. 6). If the prediction block of the current block is generated using the horizontal plane mode, the video signal processing device may reset predMode to the vertical prediction mode (angle mode 50 in FIG. 6). When predMode is greater than the 34th angular mode, which is a diagonal mode, the isTrTransposed value can be set to 1, and when predMode is equal to or less than 34, the isTrTransposed value can be set to 0. When predMode is greater than 34, the video signal processing device resets the value of predMode to a value that is a difference of 67 (the maximum value of the intra directional mode index) plus 1. For example, when predMode is 35, the video signal processing device can reset the 35th angular mode to the 33rd angular mode (67+1-35(predMode)). When predMode is 66, the video signal processing device can reset the 66th angular mode to the 2nd angular mode (67+1-66(predMode)). That is, by making it symmetrical with respect to the 34th angular mode, which is a diagonal mode, the size of the transformation mapping table of FIG. 45 is reduced by about half.

[0432] 4) The video signal processing device can derive the nSzIdx value through nSzIdxW, nSzIdxH, and isTrTransposed values. If the isTrTransposed value is '1', the value obtained by multiplying nSzIdxH by 4 and then adding nSzIdxW can be set to nSzIdx. If the isTrTransposed value is '0', the value obtained by multiplying nSzIdxW by 4 and then adding nSzIdxH can be set to nSzIdx.

[0433] 5) The video signal processing device can derive nTrSet, which is an index of an available transformation type set, according to the predefined table of FIG. 48 using nSzIdx, which is the size information of the current block, and nMdIdx, which is the intra-directional mode information of the current block. FIG. 48 defines an index of a transformation type set according to the intra-screen directional mode (0 to 34 and MIP) of the current block and the size index (0 to 15) of the current block. Referring to FIG. 48, nTrSet can be 80, and if the size of the current block is 4x8 and the intra-screen directional mode of the current block is 13, nTrSet can be '7'.

[0434] 6) The video signal processing device can derive a set of transformation types corresponding to nTrSet from the table of FIG. 49 by parsing mts_idx included in the bitstream. The transformation types in the vertical and horizontal directions are set differently depending on whether predMode is a value greater than the 34th angle mode, which is a diagonal mode. 0 to 79 in the gray-shaded vertical columns of FIG. 49 may correspond to nTrSet, and 0 to 3 in the gray-shaded horizontal columns may correspond to mts_idx. Referring to FIG. 49, if nTrSet is 7 and the value of mts_idx is 3, 22 may be selected from (2, 17, 18, 22). In addition, DST1 and DCT5 corresponding to index 22 of the transformation type combination table of FIG. 50 are selected, and the vertical transformation type of the current block may be set to DST1 and the horizontal transformation type may be set to DCT5. The gray-shaded horizontal rows 0 to 24 in FIG. 50 represent indices selected through FIG. 49, and the gray-shaded vertical rows 0 to 1 may represent vertical and horizontal transformation types, respectively. If the predicted directionality mode within the screen of the current block is greater than 34, which is a diagonal mode, the vertical and horizontal transformation types may be interchanged.

[0435] If the mts_idx value is '3' and the width and height of the current block are both 16 or less, the vertical or horizontal transformation type can be reset to the IDT transformation type through the process described below.

[0436] If the absolute difference between the index of the predicted directionality mode within the screen of the current block and the index of the horizontal direction mode, 18, is less than a predetermined value, the vertical transformation type may be reset to the IDT transformation type. If the absolute difference between the index of the predicted directionality mode within the screen of the current block and the index of the horizontal direction mode, 50, is less than a predetermined value, the horizontal transformation type may be reset to the IDT transformation type. In this case, the predetermined value may be an integer and may be determined based on the horizontal or vertical size of the current block. For example, the predetermined value may be determined through the table of Fig. 48. The table of Fig. 51(a) shows a case where the threshold value is set differently whenever the horizontal or vertical size differs by 4, and the table of Fig. 51(b) shows a case where the threshold value is set differently whenever the horizontal or vertical size differs by 2 times. If the size of the current block is 16x16, the vertical transformation type may not be reset to the IDT transformation type, but the existing transformation type may be maintained as is.

[0437] To reduce the bit count for encoding mts_idx, the number of transform type sets can be adaptively changed for each block. The number of transform type sets can be adaptively determined based on the sum of the absolute values ​​of the transform coefficients of the current transform block. For example, the number of transform type sets can be adaptively determined to be 1, 4, or 6. When the number of transform type sets is different, the maximum number of bins for signaling mts_idx may be different. If the sum of the absolute values ​​of the transform coefficients is less than or equal to 6, the number of transform type sets may be one. Therefore, if the sum of the absolute values ​​of the transform coefficients is less than or equal to 6, the encoder may not include (i.e., may not signal) mts_idx in the bitstream, and the decoder may not parse mts_idx. If the sum of the absolute values ​​of the transform coefficients is greater than 6 and less than or equal to 32, the number of transform type sets may be four, and if the sum of the absolute values ​​of the transform coefficients is greater than 32, the number of transform type sets may be six. Transform Kernel In this case, when the number of transform type sets is multiple (e.g., 4 or 6), the encoder may generate and signal a bitstream including mts_idx based on the maximum number of transform type sets, The decoder can parse mts_idx based on the maximum number of transformation type sets, where the transformation type sets can be used in the same sense as transformation kernel candidates.

[0438] If the current block is encoded in inter mode, the transform kernel set for the current block can be any one of four combinations of transform kernel sets. The four combinations can be {(DST7, DST7), (DST7, DCT8), (DCT8, DST7), (DCT8, DCT8)}. The encoder can signal an index indicating the transform kernel set used among the four combinations by including it in the bitstream. The decoder can parse the index to determine the transform kernel set for the current block. For example, if the determined transform kernel set is (DST7, DCT8), DST7 can be applied to the current block in the horizontal direction and DCT8 can be applied to the vertical direction to perform transform / inverse transform. In addition, for complexity optimization, for images larger than 1920x1080 resolution, the maximum CU size to which inter MTS can be applied can be set to 32x32. For resolutions other than 1920x1080, the maximum CU size for which inter-MTS can be applied can be 16x16. Therefore, if the current block size is larger than 32x32 or 16x16, inter-MTS may not be applied to the current block, and DCT2 transform may be applied in the horizontal and vertical directions. In addition, the transform kernels DST7 and DCT8 for the current block size equal to or smaller than 16x16 may be replaced with a separable KLT (Karhunen-Loeve Transform).

[0439] Figure 52 shows block boundaries and samples around the boundaries in a deblocking filtering process according to one embodiment of the present invention.

[0440] Referring to Figure 52(a), the part indicated by the dotted line between the P block and the Q block may indicate a block boundary. The block boundary may exist at any fixed size, and a block boundary may exist at every size of 4.

[0441] Figure 52(b) shows samples for which filtering is performed based on block boundaries. The video signal processing device can perform a process of alleviating blocking artifacts occurring at block boundaries by performing deblocking filtering on the currently restored block. The deblocking filtering process can include a process of determining a transform block boundary, determining a sub-block boundary, determining a length of a filter to perform filtering, determining a filtering strength (bS), determining a filtering parameter, determining whether to perform filtering, and determining a filtering type.

[0442] Figures 53 and 54 illustrate the concept of matrix-based intra-screen prediction according to one embodiment of the present invention.

[0443] bdry_top in Fig. 53 represents the upper reference samples of the current prediction block and may correspond to the width (W) of the current prediction block. bdry_left represents the left reference samples of the current prediction block and may correspond to the height (H) of the current prediction block. The reference sample line may be the one closest to the current prediction block. Based on the reference samples, the video signal processing device can predict the current prediction block using the MIP (Matrix-based Intra prediction) method that applies matrix values ​​and offsets.

[0444] Referring to FIG. 54, a method for performing matrix-based intra-picture prediction will be described. FIG. 54 is a diagram illustrating a method for performing matrix-based intra-picture prediction when the size of the current block is 8 x 8. First, i) the video signal processing device can perform an averaging process on neighboring reference samples of the current block. In the averaging process, the upper reference samples (bdry_top) of the current block can be re-expressed as an average value in a certain group unit. For example, the video signal processing device can generate four new reference samples (bdry^top_red) by taking the average of two reference samples each among the eight reference samples (bdry_top) on the top of the current block. Similarly, the video signal processing device can obtain four new reference samples (bdry^left_red) by taking the average of two reference samples each among the eight reference samples (bdry_left) on the left of the current block. Therefore, the video signal processing device can obtain a total of eight new reference samples. The number of newly generated reference samples after the averaging process can be determined according to the size of the current block. Next, ii) the video signal processing device can obtain a prediction block (pred) of the current block by performing a matrix vector multiplication process. The video signal processing device can determine the matrix and offset values ​​required for the matrix vector multiplication process for obtaining the prediction block of the current block based on the result of step i) (i.e., each of the four new samples (bdry^top_red, bdry^left_red) and / or mode k. Mode k will be described later. iii) the video signal processing device can perform interpolation. The result value (the prediction block (pred) of the current block) in step ii) can be mapped to a designated location of the current block.If the current block is larger than the reference sample (e.g., the averaging process of the downsampling step i), the video signal processing device can generate the final prediction block by applying single-step linear interpolation in the vertical or horizontal direction, or in both the vertical and horizontal directions. The single-step linear interpolation can be applied to a block of size W x H where max(W, H)>=8. W can be the width (horizontal) of the current block, H can be the height (vertical) of the current block, and max(W, H) is a function that outputs the larger value of W and H. Specifically, if the size of the current block is 8 x 8 (see FIG. 54), the video signal processing device can obtain 8 new reference samples. Then, the video signal processing device can perform a matrix vector multiplication process to obtain 16 new samples. The 16 new samples can be mapped to specific locations of the prediction block of the current block (gray shaded portion of 5401 in FIG. 54). In the process of linear interpolation (e.g., single-step linear interpolation), the reference samples corresponding to the width of the current block may be reference samples generated in the averaging process mapped to a specific location (the gray shaded portion of 5402 in FIG. 54). Referring to 5402 in FIG. 54, the specific location may be an odd number (the 1st, 3rd, 5th, and 7th of 5402 in FIG. 54). The reference samples corresponding to the height of the current block may be the same as the reference samples used in the averaging process. At this time, the video signal processing device may perform filtering on the reference samples used in the linear interpolation process. For example, the video signal processing device may add the upper and lower weights [1, 2, 1] based on the current sample and perform filtering with the bit shift operator 2.

[0445]

[0446] In Equation 21, A is a matrix, and if the width (W) and the height (H) of the current block are 4, A can be a 4 x 4 matrix with 4 rows and 4 columns. If at least one of the width (W) and the height (H) of the current block is not 4, A can be an 8 x 8 matrix with 8 rows and 8 columns. b is an offset value in vector form, which can have a size of Wred x Hred. Matrix A and vector b can be one of three sets S0, S1, and S2. S0 can be composed of 18 matrices, each matrix can be composed of 16 rows and 4 columns, and can have 18 offset vectors. S1 can be composed of 10 matrices, each of which can have 16 rows and 8 columns, and can have 10 offset vectors. S2 can be composed of 6 matrices, each of which can have 64 rows and 8 columns, and can have 6 offset vectors. S0, S1, and S2 can be indicated by separate indices (idx). The indices can be determined as in Equation 5.

[0447]

[0448] In mathematical expression 22, W can be the width (horizontal) of the current block, and H can be the height (vertical) of the current block. max(W, H) is a function that outputs the larger value between W and H.

[0449] The method for determining the above mode k may be as follows.

[0450] Mode k may be a set of transform kernels for MIP. Mode k may be used to determine a matrix for MIP. Mode k may be determined based on an intra prediction mode (e.g., an angular mode) and / or the size of the current block. For example, if the intra prediction mode is an angular mode with an index lower than 18, and the width and height of the current block are 4, mode k may be the same mode as the intra prediction mode. If the intra prediction mode is an angular mode with an index greater than or equal to 18, and the width and height of the current block are 4, mode k may be an angular mode corresponding to an index that subtracts 17 from the index of the intra prediction mode. If the intra prediction mode is an angular mode with an index lower than 10, and the larger of the width and height of the current block is 8, mode k may be the same as the intra prediction mode. If the intra prediction mode is an angular mode with an index greater than or equal to 10, and the larger value between the width and the height of the current block is 8, then mode k may be an angular mode corresponding to an index obtained by subtracting 9 from the index of the intra prediction mode. If the intra prediction mode is an angular mode with an index less than 6, and the larger value between the width and the height of the current block is greater than 8, then mode k may be the same as the intra prediction mode. If the intra prediction mode is an angular mode with an index greater than or equal to 6, and the larger value between the width and the height of the current block is greater than 8, then mode k may be an angular mode corresponding to an index obtained by subtracting 5 from the index of the intra prediction mode. This can be organized into a mathematical equation as shown in Equation 23.

[0451]

[0452] FIG. 55 is a diagram showing the locations of surrounding pixels used to derive directional information according to one embodiment of the present invention.

[0453] Figure 55(a) shows when all neighboring blocks of the current block are available for deriving directional information, Figure 55(b) shows when the upper boundary of the current block is a sub-picture, slice, tile, or CTU boundary, and Figure 55(c) shows when the left boundary of the current block is a sub-picture, slice, tile, or CTU boundary. Meanwhile, if the neighboring blocks and the current block do not belong to the same sub-picture, slice, tile, or CTU, the neighboring blocks may not be used for deriving directional information. The gray dots in Figure 55 indicate the positions of pixels used to actually derive directional information, and the dotted lines indicate sub-picture, slice, tile, and CTU boundaries. In addition, referring to Figures 55(d) to (f), pixels located at the boundary may be padded by one pixel outside the boundary to derive directional information. This padding may allow for more accurate directional information to be derived.

[0454] In order to derive direction information for a pixel at a specific location, a 3x3 Sobel filter of Equation 24 can be applied in the horizontal and vertical directions, respectively. A in Equation 1 can denote pixel information (values) of the reconstructed neighboring blocks of the current block of 3x3 size. And the direction information (θ) can be determined using Equation 2. In order to reduce the computational complexity for deriving direction information, the decoder can derive the direction information (θ) only by calculating Gy / Gx in Equation 1 without calculating the atan function in Equation 25.

[0455]

[0456]

[0457] Referring to FIG. 55, directional information can be calculated for each gray dot shown in FIG. 55, and the directional information can be mapped to an angle of an intra prediction mode. The intra prediction mode set can include a planar mode, a DC mode, and a plurality of (e.g., 65) angular modes (i.e., directional modes). The intra prediction modes can be 67 modes, and the directional information (angle, θ) calculated through mathematical expression 2 can be a value in real units. Therefore, a process of mapping directional information to a specific intra prediction directional mode is required. The intra prediction directional mode described herein can be the same as the angular mode illustrated in FIG. 6. In addition, a method of deriving intra prediction directional information and mapping (determining) an intra prediction directional mode in the present specification can be described as a DIMD method.

[0458] FIG. 56 is a diagram illustrating a method for mapping directional modes according to one embodiment of the present invention.

[0459] Referring to FIG. 56, the intra prediction directional mode can be divided into four sections based on 0 degrees (index 18), 45 degrees (index 34), 90 degrees (index 50), and 135 degrees (index 66) (see FIG. 6). Referring to FIG. 10, the sections for determining the intra prediction directional mode can be divided into four sections from section 0 to section 3. Section 0 can be from -45 degrees to 0 degrees, section 1 can be from 0 degrees to 45 degrees, section 2 can be from 45 degrees to 90 degrees, and section 3 can be from 90 degrees to 135 degrees. In this case, each section can include 16 intra prediction directional modes. The directional mode can be determined as one of the four sections by comparing the signs and magnitudes of Gx and Gy calculated through mathematical expression 24. For example, if Gx and Gy are positive and the absolute value of Gx is greater than the absolute value of Gy, interval 1 can be selected. The intra prediction directional mode mapped to each interval can be determined through the directional information (θ) calculated from mathematical expression 2. Specifically, the decoder extends the value by multiplying the directional information (θ) by 2^16. Then, the decoder can compare the extended value with the numbers in the predefined table to find the value closest to the extended value and determine the intra prediction directional mode based on the closest value. At this time, there can be 17 values ​​in the predefined table. Specifically, the values ​​of the predefined table can be {0, 2048, 4096, 6144, 8192, 12288, 16384, 20480, 24576, 28672, 32768, 36864, 40960, 47104, 53248, 59392, 65536}. At this time, the difference between the predefined table values ​​can be set differently depending on the difference between the angles of the intra prediction directional mode.

[0460] Meanwhile, if the directional angle is obtained using only Gy / Gx without performing atan calculation to reduce the computational complexity, the difference between the predefined table values ​​may not match the distance between the angles of the intra prediction directional mode. atan has a characteristic that the slope gradually decreases as the input value increases. Therefore, the above-defined table should also be set in numerical value by considering not only the difference between the angles of the intra prediction directional mode but also the nonlinear characteristic of atan. For example, the difference between the above-defined table values ​​may be set to gradually decrease. Conversely, the difference between the above-defined table values ​​may be set to gradually increase.

[0461] If the width and height of the current block are different, the available intra prediction directional modes may vary. That is, if the width and height of the current block are different, the interval for deriving the intra prediction directional mode may vary. In other words, the interval for deriving the intra prediction directional mode may change based on the width and height of the current block (e.g., the ratio of the width and height). For example, if the width of the current block is longer than the height, the intra prediction mode may be remapped to 67 to 80, and the intra prediction mode in the opposite direction may be excluded to 2 to 15. For example, if the width of the current block is n (an integer) times longer than the height (e.g., 2), the intra prediction modes {3, 4, 5, 6, 7, 8} may be remapped (mapped) to {67, 68, 69, 70, 71, 72}, respectively. Additionally, if the horizontal length of the current block is longer than the vertical length, the intra prediction mode may be reset to a value obtained by adding '65' to the intra prediction mode. On the other hand, if the horizontal length of the current block is shorter than the vertical length, the intra prediction mode may be reset to a value obtained by subtracting '67' from the intra prediction mode.

[0462] A histogram can be used to derive an intra-prediction directional mode for restoring the current block. If, after obtaining directional information for surrounding blocks, there are more non-directional blocks than directional blocks, the prediction mode for the non-directional block may have the highest cumulative value in the histogram. However, since a directional mode must be derived for restoring the current block, the prediction mode for the non-directional block may be excluded even if it has the highest cumulative value in the histogram. That is, a smooth region with no gradient or directionality between surrounding pixels may not be used to derive an intra-prediction directional mode. For example, the prediction mode for a block without directional information may be planar or DC. If the left neighboring block is planar or DC, the left neighboring block may not be used to derive directional information, and directional information may be derived using only the upper neighboring block. When the surrounding blocks of the current block contain a mixture of smooth regions and directional regions, the decoder can generate a histogram using the G value calculated as in Equation 26 to emphasize the directionality. In this case, the histogram may be a cumulative value in which the calculated G value is added to each generated intra prediction directional mode, rather than a frequency-based one in which '1' is added to each generated intra prediction directional mode.

[0463]

[0464] FIG. 57 is a diagram showing a histogram for deriving an intra prediction directional mode according to one embodiment of the present invention.

[0465] The X-axis of Fig. 57 represents the intra prediction directional mode, and the Y-axis represents the accumulated value of G values. The decoder can select the intra prediction directional mode having the largest accumulated value of G values ​​among the intra prediction directional modes. In other words, the decoder can select the intra prediction directional mode for the current block based on the accumulated value. Referring to Fig. 19, modeA having the largest accumulated value and modeB having the second largest accumulated value can be selected as the intra prediction directional mode. The decoder can generate a final prediction block by weighting the prediction block generated with modeA, the prediction block generated with modeB, and finally the prediction block generated with the planar mode to generate a prediction block for the current block. At this time, the weight of each prediction block can be determined using the accumulated value of modeA and modeB. For example, the weight for the prediction block generated with the planar mode can be set to 1 / 3 of the total weight. The weight for the prediction block generated by modeA can be set to a weight corresponding to the value obtained by dividing the modeA accumulated value by the sum of the accumulated values ​​of modeA and modeB. The weight for the prediction block generated by modeB can be determined as a difference value between the modeA weight and 1 / 3 of the total weight. To make the calculation of the weight more accurate, the decoder can expand the range of the weight by multiplying the weight for the prediction block generated by modeA by an arbitrary value. The weight for the prediction block generated by modeB and the weight for the prediction block generated in the planar mode can also be expanded in the same way.

[0466] Figure 58 shows samples used when an intra prediction mode is derived in a DIMD manner for a block to which MIP is applied according to one embodiment of the present invention.

[0467] A video signal processing device can derive an intra prediction mode using the DIMD method for a block to which MIP is applied. The video signal processing device can derive an MTS set (e.g., see FIG. 48) based on the derived intra prediction mode and apply the MTS. One of the transformation kernel candidates (e.g., see FIG. 49) based on the derived MTS set can be applied.

[0468] The method for deriving the intra prediction mode using DIMD for the block to which MIP is applied is as follows.

[0469] Method 1) A video signal processing device can derive the most frequent intra prediction mode using the DIMD method by using neighboring samples of the current prediction block (e.g., see FIGS. 55 to 57). Referring to FIG. 58(a), the video signal processing device can use neighboring samples of the current prediction block to derive the intra prediction mode. Referring to FIG. 58(b), the size of the neighboring samples can be a size to which a Sobel filter can be applied.

[0470] Method 2) A video signal processing device can derive the most frequent intra prediction mode using the DIMD method by using internal samples of a prediction block generated by applying MIP to the current prediction block. The prediction block generated by applying MIP may be a block before interpolation is applied. Whether or not interpolation is applied to the block generated by applying MIP can be determined based on its size. The required internal sample format is based on the format as shown in Fig. 58(b), but some or all of the predicted samples may be considered.

[0471] Method 3) The video signal processing device can obtain a final MIP prediction block by applying interpolation to the prediction block generated in Method 2. The video signal processing device can derive the most frequent intra prediction mode using the internal samples of the final MIP prediction block using the DIMD method. Meanwhile, since whether to apply interpolation is determined according to the block size, if the final MIP prediction block is generated without interpolation, the video signal processing device can derive the intra prediction mode using the DIMD method using the final MIP prediction block generated without interpolation.

[0472] FIG. 59 shows a positional relationship for sample-based blending when performing a DIMD prediction method according to one embodiment of the present invention.

[0473] Below, we describe the prediction method using DIMD.

[0474] (i) A video signal processing device can derive up to N intra prediction modes. N may be 5. At this time, the video signal processing device can derive location dependency for each intra prediction mode. The location dependency can be determined based on the histogram sizes of the three location areas (ABOVE, ABOVE-LEFT, LEFT) of FIG. 59. For example, if the histogram size of the LEFT of the derived intra prediction mode is larger than twice the histogram size of ABOVE, the location dependency of LEFT can be determined to be high.

[0475] (ii) If the two modes having the largest histogram sizes are greater than DC_IDX, blending may be performed. If the two modes having the largest histogram sizes are less than DC_IDX, blending may not be performed. When blending is performed, the video signal processing device may derive a weight value based on the size of the histogram for each derived intra prediction mode. At this time, the derived intra mode may be an intra mode derived through method 1) of deriving an intra prediction mode using the DIMD method for a block to which the above-described MIP is applied. In addition, blending may be possible if the index values ​​of the derived intra prediction modes are not equal to the index (PLANAR_IDX) value of the planar mode.

[0476] (iii) The vertical-based weight (weightVer) and horizontal-based weight (weightHor) for each intra prediction mode derived in (i) can be derived based on position dependence. Alternatively, position-independent weights (weightNonLocDep) can be derived.

[0477] (iv) If the weightVer or weightHor value is greater than 0, the video signal processing device can perform blending using position-based weights. At this time, the video signal processing device can apply blending on a sample-by-sample basis. If the weightVer or weightHor value is not greater than 0, weight-based blending independent of position can be performed.

[0478] (v) In the process (i) of deriving the maximum N intra prediction modes described above, the video signal processing device can derive new weights by considering the derived intra prediction modes, whether position-dependent blending is applied, and the weights derived based on the size of the histogram. The blending method may vary based on the number of intra prediction modes whose derived weight values ​​are greater than 0. The video signal processing device can derive weight values ​​to be applied to the entire block based on the newly derived weights. At this time, the planar mode can be used as the basic blending mode.

[0479] (vi) After blending in step (v), the video signal processing device can additionally perform sample-by-sample blending in step (iv). The blending mode can be determined based on the size of the value of weightVer or weightHor in step (v). The blending weight value can vary on a sample-by-sample basis. At this time, the blending mode can include a diagonal mode, a vertical mode, and a horizontal mode.

[0480] This paper describes how to apply directional planar modes in the DIMD prediction method. Directional planar modes can be divided into vertical planar (VerPlanar) and horizontal planar (HorPlanar) planar.

[0481] Method a) The video signal processing device can fuse the directional planar by comparing whether the first mode with the highest frequency among the five modes derived by DIMD is not the same as the planar mode and is greater than or equal to the intra prediction mode index 34. For example, if the index of the first mode with the highest frequency is greater than or equal to the index 34, VerPlanar is used, and if it is less than the index 34, HorPlanar can be used instead of the planar mode. Conversely, if it is less than the index 34, HorPlanar is used, and if it is greater than the index 34, VerPlanar can be used instead of the planar mode.

[0482] Method b) The video signal processing device may use VerPlanar instead of the planar mode if the index of the first mode having the highest frequency among the five modes derived by DIMD is outside the first section, HorPlanar may be used instead of the planar mode if the index of the first mode having the highest frequency is outside the second section, and in other cases, the planar mode may be used. Conversely, if the index of the first mode having the highest frequency is outside the first section, HorPlanar may be used instead of the planar mode, and if the index of the first mode having the highest frequency is outside the second section, VerPlanar may be used instead of the planar mode.

[0483] Method c) The video signal processing device can perform cost calculations for HorPlanr-based DIMD prediction and VerPlanr-based DIMD prediction in addition to the planar mode-based DIMD prediction method, and select the prediction method with the lowest cost. The encoder can generate information indicating the selected prediction method and signal it by including it in the bitstream, and the decoder can parse the information to determine the selected prediction method.

[0484] Figures 60 to 62 illustrate an EIP (Extrapolation filter-based intra prediction mode) according to one embodiment of the present invention.

[0485] EIP can be applied to a luminance block larger than 32x32. The method for deriving filter coefficients for EIP is similar to the CCCM method and filter coefficient derivation method described through FIG. 16. The video signal processing device can generate filter coefficients for EIP using reconstructed samples of a certain size or larger in neighboring blocks of the current prediction block. The reconstructed sample area can be determined according to the width and height of the current block and the selected filter shape. The filter for EIP can be one of three shapes as illustrated in FIG. 60. Referring to FIG. 60, the filter shape can be 4x4, 2x8, or 8x2, and can have a structure of 15 input samples and 1 output sample. FIG. 61 illustrates a method for deriving filter coefficients from a template of a current block using one of the three filter shapes of FIG. 60. Referring to FIG. 61, the template for the current block can be configured as an L-shape, an upper-only shape, or a left-only shape based on the current block. The width and height of the template can be determined as multiples of the sample size. Referring to FIG. 62, the video signal processing device can generate an EIP prediction block using a diagonal prediction order from the upper left to the lower right of the current block using the coefficients of the filter for the determined EIP.

[0486]

[0487] pred in mathematical expression 27 (x,y) means the value of the (x, y) position sample of the current block, and c imeans the i-th coefficient of the determined EIP filter. At this time, the index of the coefficient can be a value from 0 to 14. t(x-offsetXi, y-offsetYi) means the restored or predicted value for the current position prediction. offsetX i and offsetY i represents the offset from the current position along the x and y directions.

[0488] In the above EIP prediction block, an intra prediction mode-based LFNST / NSPT, MTS transformation set derived using the DIMD method can be derived, and each transformation kernel can be applied.

[0489] A video signal processing device can store an intra prediction mode derived by the DIMD method used in a block predicted by the EIP prediction mode as the intra prediction mode of a current block. The intra prediction mode derived by the DIMD method may be an intra prediction mode with the highest frequency. The intra prediction mode may be stored in an IPM buffer or the like in units of 4x4 pixels. In this specification, a buffer may have the same meaning as a memory or a storage device. In addition, the intra prediction mode may be stored after prediction or restoration is completed in units of CUs. The intra prediction mode may be used when deriving an intra prediction mode of a spatially neighboring block of a next prediction block, and may be used when deriving an intra prediction mode of a luma block corresponding to a chroma block. When the video signal processing device derives an intra prediction mode from a neighboring block of a current block, if the neighboring block is a block coded by EIP, the intra prediction information stored in the IPM buffer (information related to the intra prediction mode stored in the IPM buffer) may be used. When the DIMD method is applied, the samples used may be samples within a neighboring block of the current block or a predicted block of the current block, as shown in FIG. 58.

[0490] The video signal processing device can store the intra prediction mode (and related information) derived by the DIMD method used to apply MTS to a block to which MIP is applied in a 4x4 pixel unit in an IPM buffer when prediction block generation or restoration is completed. The video signal processing device can use the information related to the derived intra prediction mode stored in the IPM buffer when deriving the intra prediction mode of a spatial neighboring block of a next prediction block, and when deriving the intra prediction mode of a corresponding luma block for a chroma block. When the video signal processing device derives the intra prediction mode from a neighboring block of a current block, if the neighboring block is a block coded with MIP, the information related to the derived intra prediction mode stored in the IPM buffer can be used. The unit stored in the buffer may not be limited to a 4x4 pixel unit.

[0491] A video signal processing device can derive an intra prediction mode by the DIMD method for deriving a transform set when LFNST, NSPT, and MTS transforms are applied in MIP prediction and EIP prediction, and the derived intra prediction mode (and related information) can be stored. However, when LFNST, NSPT, and MTS transforms are not applied in MIP prediction and EIP prediction, a preset value can be stored. The preset value may be a planar or DC mode. The video signal processing device derives an intra prediction mode by the DIMD method to derive a LFNST, NSPT, and MTS transform set in MIP prediction and EIP prediction, but can always derive and store the intra prediction mode by the DIMD method regardless of the transform set derivation. That is, an intra prediction mode derived by the DIMD method can be stored regardless of whether a transform is applied to prediction modes.

[0492] FIG. 63 illustrates a method for deriving an intra prediction mode for inter prediction according to one embodiment of the present invention.

[0493] A video signal processing device can store an intra prediction mode for inter prediction. The video signal processing device can derive and store the intra prediction mode based on a reference block of a reference picture indicated by motion information. The intra prediction mode can be stored in an IPM buffer in 4x4 units. Additionally, the intra prediction mode can be stored after prediction or restoration is completed on a CU basis. Information related to the stored intra prediction mode can be used when deriving the intra prediction mode of a spatial neighboring block of the next prediction block, and can be used when deriving the intra prediction mode of a corresponding luma block for a chroma block. The video signal processing device can use the intra prediction mode based on whether intra prediction mode information exists at a specific location relative to the reference block. For example, if intra prediction mode information exists, the video signal processing device can use the intra prediction mode, and if no intra prediction mode exists, it can use a preset mode. The preset default mode can be planar or DC mode. The specific location can be the center, upper left, upper right, lower left, or lower right of the reference block. Referring to Fig. 63(b), specific locations may be the center (0), upper left (1), upper right (3), lower left (2), and lower right (4) of the reference block. In addition, specific locations may further include locations corresponding to 5, 6, 7, and 8 of Fig. 63(b). The video signal processing device may search for information related to the intra prediction mode for specific locations.

[0494] A video signal processing device can derive an intra prediction mode using the DIMD method, store information related to the derived intra prediction mode, and use it in an inter prediction method. The video signal processing device can confirm that the predicted sample has directionality even in the inter prediction mode. The video signal processing device can derive a transform set such as MTS, LFNST, NSPT, etc. based on the directionality information of the predicted sample. The samples used when deriving the intra prediction mode using the DIMD method may be predicted samples. Referring to FIG. 63(c), the video signal processing device can predict the current block using samples of a reference block for the current block, and then derive the intra prediction mode using some or all of the internal samples of the predicted current block using the DIMD method. When the video signal processing device uses some of the internal samples of the predicted current block, some of the internal samples may be gray samples of FIG. 63(c). Alternatively, the video signal processing device may use surrounding samples of the current block to derive the intra prediction mode. However, in the case of inter-prediction mode, the video signal processing device can perform parallel processing on a per-CU basis, and if it must wait until the surrounding samples of the current block are restored to derive the intra-prediction mode, unnecessary delay may occur. Therefore, the video signal processing device can derive and use the intra-prediction mode stored in the inter-block from a spatially neighboring block of the current block as needed. For example, the video signal processing device can use the intra-prediction mode stored in the inter-block when deriving a spatial candidate of the MPM list from the current prediction block.

[0495] A method for deriving an intra prediction mode using DIMD using samples of the current prediction block can be as follows.

[0496] i) The video signal processing device can derive and store an intra prediction mode using the DIMD method when a surrounding block of the current block is an inter prediction block.

[0497] ii) The video signal processing device may derive an intra prediction mode using the DIMD method only when a specific tool is applied to a neighboring block of the current block. If the specific tool is not applied, the video signal processing device may derive and store the intra prediction mode using a conventional method. The specific tool may be a tool that requires an intra prediction mode. For example, the video signal processing device may derive a transform set using an intra prediction mode.

[0498] The method for deriving the intra prediction mode described above can also be applied to an inter-combined mode that combines one or more identical or different prediction modes based on an inter-combined mode. Alternatively, if the surrounding blocks of the current block are in an inter-combined mode, the video signal processing device may use a preset value (e.g., DC or planar mode) without deriving an intra prediction mode.

[0499] Figure 64 is a flowchart illustrating a method for predicting a current block according to one embodiment of the present invention.

[0500] A method for predicting the current block described through FIGS. 1 to 63 is described with reference to FIG. 64.

[0501] The video signal processing device can derive an intra prediction mode for prediction of the current block (S6410).

[0502] The intra prediction mode may be derived from one or more neighboring blocks of the current block. If the encoding mode of the one or more neighboring blocks is not an intra mode, the intra prediction mode may be derived from a prediction mode stored in an intra prediction mode (IPM) buffer.

[0503] The video signal processing device can predict the current block based on the derived intra prediction mode (S6420).

[0504] The encoding mode of the one or more surrounding blocks may be any one of intra TMP (template matching prediction), EIP (extrapolation filter-based intra prediction) mode, inter mode, and IBC (intra block copy) mode.

[0505] The one or more surrounding blocks may be blocks located at least on the left, lower left, upper right, upper left, or upper right of the current block.

[0506] The mode stored in the IPM buffer may be an intra prediction mode related to one or more surrounding blocks.

[0507] The intra prediction mode associated with the one or more surrounding blocks may be a prediction mode derived by applying decoder side intra mode derivation (DIMD) to surrounding blocks of the one or more surrounding blocks.

[0508] The intra prediction mode associated with the one or more surrounding blocks may be a prediction mode having the highest frequency among prediction modes derived by applying DIMD (decoder side intra mode derivation) to the surrounding blocks of the one or more surrounding blocks.

[0509] The methods described herein can be performed by a processor of a video signal processing device (decoder or encoder). Furthermore, the encoder can generate a bitstream decoded by the methods described above. Furthermore, the bitstream generated by the encoder can be stored on a computer-readable, non-transitory storage medium (recording medium).

[0510] While this specification primarily addresses the decoder's perspective, the same principles apply to encoders. While the term "parsing" in this specification focuses on the process of obtaining information from a bitstream, it can also be interpreted as configuring that information into a bitstream from an encoder's perspective. Therefore, the term "parsing" is not limited to decoder operations; it can also be interpreted as the act of configuring a bitstream in an encoder. Furthermore, such a bitstream can be stored and configured on a computer-readable recording medium.

[0511] The embodiments of the present invention described above may be implemented through various means. For example, the embodiments of the present invention may be implemented through hardware, firmware, software, or a combination thereof.

[0512] In the case of hardware implementation, the method according to embodiments of the present invention may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), processors, controllers, microcontrollers, microprocessors, etc.

[0513] When implemented via firmware or software, the methods according to embodiments of the present invention may be implemented in the form of modules, procedures, or functions that perform the functions or operations described above. The software code may be stored in memory and executed by the processor. The memory may be located internally or externally to the processor and may exchange data with the processor via various known means.

[0514] Some embodiments may also be implemented in the form of a recording medium containing computer-executable instructions, such as program modules, executed by a computer. Computer-readable media can be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Furthermore, computer-readable media can include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. Communication media typically contains computer-readable instructions, data structures, or other data in a modulated data signal, such as program modules, or other transport mechanisms, and includes any information delivery media.

[0515] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will readily appreciate that the present invention can be readily modified into other specific forms without altering the technical spirit or essential characteristics of the present invention. Therefore, the embodiments described above should be construed as illustrative and not restrictive in all respects. For example, components described as being single may be implemented in a distributed manner, and similarly, components described as being distributed may be implemented in a combined manner.

[0516] The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention.

Claims

1. In a video signal decoding device, Includes a processor, The above processor, Derive an intra prediction mode for the prediction of the current block, The intra prediction mode is derived from one or more surrounding blocks of the current block, If the encoding mode of one or more of the surrounding blocks is not an intra mode, the intra prediction mode is derived from a prediction mode stored in an IPM (intra prediction mode) buffer, A video signal decoding device that predicts the current block based on the derived intra prediction mode.

2. In paragraph 1, A video signal decoding device, wherein the encoding mode of the one or more surrounding blocks is any one of intra TMP (template matching prediction), EIP (extrapolation filter-based intra prediction) mode, inter mode, and IBC (intra block copy) mode.

3. In paragraph 1, A video signal decoding device, wherein the one or more surrounding blocks are blocks located at least on the left, lower left, upper right, upper left, or upper right of the current block.

4. In paragraph 1, A video signal decoding device, wherein the mode stored in the IPM buffer is an intra prediction mode related to one or more surrounding blocks.

5. In paragraph 4, A video signal decoding device, wherein the intra prediction mode associated with the one or more surrounding blocks is a prediction mode derived by applying DIMD (decoder side intra mode derivation) to surrounding blocks of the one or more surrounding blocks.

6. In paragraph 5, A video signal decoding device, wherein the intra prediction mode associated with the one or more surrounding blocks is a prediction mode having the highest frequency among prediction modes derived by applying DIMD (decoder side intra mode derivation) to the surrounding blocks of the one or more surrounding blocks.

7. In a video signal encoding device, Includes a processor, The above processor, Obtaining a bitstream decoded by a decoding method, wherein the decoding method comprises: A step of deriving an intra prediction mode for prediction of the current block, The intra prediction mode is derived from one or more surrounding blocks of the current block, If the encoding mode of one or more of the surrounding blocks is not an intra mode, the intra prediction mode is derived from a prediction mode stored in an IPM (intra prediction mode) buffer; and A video signal encoding device, comprising a step of predicting the current block based on the derived intra prediction mode.

8. In paragraph 7, A video signal encoding device, wherein the encoding mode of the one or more surrounding blocks is any one of intra TMP (template matching prediction), EIP (extrapolation filter-based intra prediction) mode, inter mode, and IBC (intra block copy) mode.

9. In paragraph 7, A video signal encoding device, wherein the one or more surrounding blocks are blocks located at least on the left, lower left, upper right, upper left, or upper right of the current block.

10. In paragraph 7, A video signal encoding device, wherein the mode stored in the IPM buffer is an intra prediction mode related to one or more surrounding blocks.

11. In paragraph 10, A video signal encoding device, wherein the intra prediction mode associated with the one or more surrounding blocks is a prediction mode derived by applying DIMD (decoder side intra mode derivation) to surrounding blocks of the one or more surrounding blocks.

12. In paragraph 11, A video signal encoding device, wherein the intra prediction mode associated with the one or more surrounding blocks is a prediction mode having the highest frequency among prediction modes derived by applying DIMD (decoder side intra mode derivation) to surrounding blocks of the one or more surrounding blocks.

13. In a computer-readable non-transitory storage medium storing a bitstream, the bitstream is decoded by a decoding method, and the decoding method comprises: A step of deriving an intra prediction mode for prediction of the current block, The intra prediction mode is derived from one or more surrounding blocks of the current block, If the encoding mode of one or more of the surrounding blocks is not an intra mode, the intra prediction mode is derived from a prediction mode stored in an IPM (intra prediction mode) buffer; and A non-transitory storage medium comprising a step of predicting the current block based on the derived intra prediction mode.

14. In paragraph 13, A non-transitory storage medium, wherein the encoding mode of the one or more surrounding blocks is any one of intra TMP (template matching prediction), EIP (extrapolation filter-based intra prediction) mode, inter mode, and IBC (intra block copy) mode.

15. In paragraph 13, A non-transitory storage medium, wherein the one or more surrounding blocks are blocks located at least on the left, lower left, upper right, upper left, or upper right of the current block.

16. In paragraph 13, A non-temporary storage medium, wherein the mode stored in the IPM buffer is an intra prediction mode associated with one or more surrounding blocks.

17. In paragraph 16, A non-temporary storage medium, wherein the intra prediction mode associated with the one or more surrounding blocks is a prediction mode derived by applying DIMD (decoder side intra mode derivation) to surrounding blocks of the one or more surrounding blocks.

18. In paragraph 17, A non-temporary storage medium, wherein the intra prediction mode associated with the one or more surrounding blocks is a prediction mode having the highest frequency among prediction modes derived by applying DIMD (decoder side intra mode derivation) to the surrounding blocks of the one or more surrounding blocks.

Images