Video signal processing method and apparatus therefor

Abstract

A processor of the video signal decoding apparatus may: determine a first prediction mode of a current block; generating a prediction block of the current block based on the first prediction mode; generating a residual block of the current block based on a transformation matrix set determined according to the second prediction mode; and reconstructing the current block based on the prediction block and the residual block.

Description

Technical Field

[0001] The present disclosure relates to a video signal processing method and device, and more particularly, to a video signal processing method and device for encoding or decoding a video signal. Background Art

[0002] Compression coding refers to a series of signal processing techniques for sending digitized information through communication lines or storing information in a form suitable for storage media. Objects of compression coding include objects such as voice, video, and text, and in particular, the technology for performing compression coding on images is called video compression. Compression coding of video signals is performed by removing excess information in consideration of spatial correlation, temporal correlation, and random correlation. However, with the recent development of various media and data transmission media, more efficient video signal processing methods and devices are needed. Summary of the invention

[0003] Technical issues

[0004] One aspect of the present disclosure is to provide a video signal processing method and device thereof to increase the coding efficiency of a video signal.

[0005] Technical Solution

[0006] The present disclosure provides a video signal processing method and a device thereof.

[0007] In the present disclosure, a video signal decoding device may include a processor, and the processor may determine a transform core set for transforming a current block to which intra-frame template matching is applied, and predict the current block based on the transform cores included in the transform core set, and the transform core set may be determined based on an intra-frame prediction mode related to the current block.

[0008] In the present disclosure, a video signal encoding device may include a processor, and the processor may obtain a bit stream decoded by a decoding method, and the decoding method may include determining a transform core set for transforming a current block to which intra-frame template matching is applied; and predicting the current block based on the transform core included in the transform core set, and the transform core set may be determined based on an intra-frame prediction mode related to the current block.

[0009] In the present disclosure, in a computer-readable non-temporary storage medium configured to store a bitstream, the bitstream can be decoded by a decoding method, and the decoding method may include determining a transform core set for transforming a current block to which intra-frame template matching is applied; and predicting the current block based on the transform core included in the transform core set, and the transform core set can be determined based on an intra-frame prediction mode related to the current block.

[0010] In addition, in the present disclosure, the transform kernel set may be at least one of a set of transform matrices of a multiple transform set (MTS), a set of transform matrices of a low frequency non-separable transform (LFNST), and / or a set of transform matrices of a non-separable main transform.

[0011] Additionally, in the present disclosure, intra prediction modes may be derived based on decoder-side intra mode derivation (DIMD).

[0012] In addition, in the present disclosure, the intra prediction mode may be derived based on the intra prediction mode of a preconfigured position within a reference block of the current block and the intra prediction modes of neighboring blocks of the reference block.

[0013] In addition, in the present disclosure, the preconfigured positions within the reference block may be the upper left, lower right, and center of the current block, and the neighboring blocks of the reference block may be blocks adjacent to the center block of the blocks at the upper boundary of the current block among the neighboring blocks of the reference block adjacent to the upper boundary of the reference block and blocks adjacent to the center block of the blocks at the left boundary of the current block among the neighboring blocks adjacent to the left boundary of the reference block.

[0014] In addition, in the present disclosure, the intra prediction mode may be derived based on the intra prediction modes of neighboring blocks of the current block.

[0015] In addition, in the present disclosure, the neighboring blocks of the current block may be blocks located at (-1, H-1), (W-1, -1), (-1, H), (W, -1), (-1, 0), and H may be the height of the current block, W may be the width of the current block, and the position of the upper left block of the current block may be (0, 0).

[0016] In addition, in the present disclosure, when the intra prediction mode of a preconfigured position within a reference block of a current block and the intra prediction mode of a neighboring block of the reference block do not exist, the intra prediction mode can be derived based on decoder-side intra mode derivation (DIMD).

[0017] Advantageous Effects of the Invention

[0018] The present disclosure provides a method for efficiently processing a video signal.

[0019] Effects obtainable from the present specification are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those skilled in the art to which the present disclosure belongs through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a schematic block diagram of a video signal encoding apparatus according to an embodiment of the present invention.

[0021] Figure 2is a schematic block diagram of a video signal decoding apparatus according to an embodiment of the present invention.

[0022] Figure 3 An embodiment in which a coding tree unit is divided into coding units in a picture is shown.

[0023] Figure 4 An embodiment of a method for signaling partitioning of quadtrees and multi-type trees is shown.

[0024] Figure 5 and Figure 6 The intra prediction method is illustrated in more detail according to an embodiment of the present disclosure.

[0025] Figure 7 The diagram illustrates positions of neighboring blocks used to construct a motion candidate list in inter-frame prediction.

[0026] Figure 8 is a diagram illustrating a transformation kernel according to an embodiment of the present disclosure.

[0027] Fig. 9 is a diagram illustrating the 0th (lowest frequency component of the corresponding transform kernel) basis functions of DCT-II, DCT-V, DCT-VIII, DST-I, and DST-VII transforms according to an embodiment of the present disclosure.

[0028] Fig.10 and Fig.11 is a diagram illustrating a transformation kernel set according to an embodiment of the present disclosure.

[0029] Fig.12 is a diagram illustrating a process of reconstructing a residual signal according to an embodiment of the present disclosure.

[0030] Fig.13 is a diagram illustrating a region of interest (ROI) of a block to which a secondary transform is applied according to an embodiment of the present disclosure.

[0031] Fig.14 is a diagram illustrating a method of applying a secondary transform (LFNST) according to an embodiment of the present disclosure.

[0032] Fig.15 is a diagram illustrating a mapping relationship between an intra prediction mode and a transform kernel set for auxiliary transform according to an embodiment of the present disclosure.

[0033] Fig.16 is a diagram illustrating a process of generating a prediction block by using a DIMD according to an embodiment of the present disclosure.

[0034] Fig.17 is a diagram illustrating positions of surrounding pixels used to derive orientation information according to an embodiment of the present disclosure.

[0035] Fig.18 is a diagram illustrating a method of mapping a directional pattern according to an embodiment of the present disclosure.

[0036] Fig.19 is a diagram illustrating a histogram for deriving an intra-prediction directional mode according to an embodiment of the present disclosure.

[0037] Figures 20 to 22 Matrix-based intra prediction according to an embodiment of the present disclosure is illustrated.

[0038] Fig.23 is a diagram illustrating intra-frame template matching according to an embodiment of the present disclosure.

[0039] Fig.24 is a diagram illustrating an encoding / decoding process of a signal related to a DIMD method according to an embodiment of the present disclosure.

[0040] Fig.25 is a diagram illustrating an encoding / decoding process of a signal related to a MIP and / or intra-frame TMP method according to an embodiment of the present disclosure.

[0041] Fig.26 is a diagram illustrating a relationship between an input vector of auxiliary transformation and an intra prediction mode according to an embodiment of the present disclosure.

[0042] Fig. 27 is a diagram illustrating a method of configuring an input vector of an auxiliary transformation according to an embodiment of the present disclosure.

[0043] Fig.28 is a diagram illustrating a process of deriving orientation information of a template of a current block for intra template matching according to an embodiment of the present disclosure.

[0044] Fig.29 is a diagram illustrating a template form for deriving intra-prediction directional information according to an embodiment of the present disclosure.

[0045] Fig.30 is a diagram illustrating an MTS set applied to an intra template matching block according to an embodiment of the present disclosure.

[0046] Fig.31 and Fig.32 is a diagram illustrating a syntax structure including a flag indicating whether intra template matching is applied according to an embodiment of the present disclosure.

[0047] Fig.33 is a diagram illustrating a syntax structure showing a method of parsing a syntax element indicating whether LFNST is applied.

[0048] Fig.34 is a diagram illustrating intra-propagation of an intra-template matching block according to an embodiment of the present disclosure.

[0049] Fig.35 is a diagram illustrating a method of applying a hash key according to a method of searching for an intra-template matching block according to an embodiment of the present disclosure.

[0050] Fig.36 is a diagram illustrating pre-configured positions for searching for intra-frame template matching blocks according to an embodiment of the present disclosure.

[0051] Fig.37 is a diagram illustrating a compilation unit syntax structure according to an embodiment of the present disclosure.

[0052] Fig.38 is a diagram illustrating a method of selecting a transform set of a block to which intra TMP is applied according to an embodiment of the present disclosure.

[0053] Fig.39 is a diagram illustrating a plurality of block vectors for an intra TMP block according to an embodiment of the present disclosure.

[0054] Fig.40 is a diagram illustrating a method of deriving an intra prediction mode of a block to which an intra TMP is applied according to an embodiment of the present disclosure.

[0055] Fig.41 is a diagram illustrating a method for determining an MTS set or a LFNST set according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0056] Considering the functions in the present disclosure, the terms used in this specification may be general terms that are currently widely used, but may be changed according to the intention of those skilled in the art, customs, or the emergence of new technologies. In addition, in some cases, there may be terms arbitrarily selected by the applicant, and in this case, their meanings are described in the corresponding description section of the present disclosure. Therefore, the terms used in this specification should be interpreted based on the substantial meaning of the terms and contents throughout the specification.

[0057] In the present specification, “A and / or B” may be interpreted as “including at least one of A or B”.

[0058] In this specification, some terms may be explained as follows. In some cases, coding may be explained as encoding or decoding. In this specification, a device that generates a video signal bit stream by performing coding (coding) of a video signal is referred to as a coding device or encoder, and a device that performs decoding (decoding) of a video signal bit stream to reconstruct a video signal is referred to as a decoding device or decoder. In addition, in this specification, a video signal processing device is used as a term that includes the concepts of both encoders and decoders. Information is a term that includes all values, parameters, coefficients, elements, etc. In some cases, the meaning is interpreted differently, so the present disclosure is not limited thereto. "Unit" is used as a meaning that refers to a basic unit of image processing or a specific position of a picture, and refers to an image area that includes both luminance components and chrominance components. In addition, "block" refers to an area of ​​a special component in an image that includes luminance components and chrominance components (ie, Cb and Cr). However, depending on the embodiment, the terms "unit", "block", "partition", "signal" and "area" may be used interchangeably. In addition, in this specification, the term "current block" refers to a block currently scheduled to be encoded, and the term "reference block" refers to a block that has been encoded or decoded and is used as a reference in the current block. In addition, in this specification, the terms "luma", "luminance", "Y", etc. can be used interchangeably. In addition, in this specification, the terms "chroma", "chrominance", "Cb or Cr", etc. can be used interchangeably, and the chroma component is classified into two components, Cb and Cr, and thus each chroma component can be distinguished and used. In addition, in this specification, the term "unit" can be used as a concept including a coding unit, a prediction unit, and a transform unit. "Picture" refers to a field or a frame, and these terms can be used interchangeably depending on the embodiment. Specifically, when the captured video is an interlaced video, a single frame can be separated into an odd (or base or top) field and an even (or even number or bottom) field, and each field can be configured in a picture unit and encoded or decoded. If the captured video is a progressively scanned video, a single frame can be configured as a picture and encoded or decoded. In addition, in this specification, the terms "error signal", "residual signal", "residue signal", "residual signal" and "difference signal" can be used interchangeably. In addition, in this specification, the terms "intra-frame prediction mode", "intra-frame prediction directional mode", "intra-frame picture prediction mode" and "intra-frame picture prediction directional mode" can be used interchangeably. In addition, in this specification, the terms "motion", "movement" and the like can be used interchangeably.In addition, in the present specification, the terms "left", "upper left", "above", "upper right", "right", "lower right", "below", and "lower left" may be used interchangeably with "leftmost", "upper left", "above", "upper right", "right", "lower right", "below", and "lower left". In addition, the terms "element" and "member" may be used interchangeably. A picture order count (POC) represents time position information of a picture (or frame) and may be a playback order in which display is performed on a screen, and each picture may have a unique POC.

[0059] Figure 1 is a schematic block diagram of a video signal encoding apparatus according to an embodiment of the present invention. Figure 1 , the encoding device 100 of the present invention includes a transform unit 110, a quantization unit 115, an inverse quantization unit 120, an inverse transform unit 125, a filtering unit 130, a prediction unit 150 and an entropy coding unit 160.

[0060] The transform unit 110 obtains the value of the transform coefficient by transforming the residual signal, which is the difference between the input video signal and the prediction signal generated by the prediction unit 150. For example, a discrete cosine transform (DCT), a discrete sine transform (DST), or a wavelet transform may be used. DCT and DST perform the transform by dividing the input picture signal into a plurality of blocks. In the transform, the coding efficiency may vary according to the distribution and characteristics of the values ​​in the transform area. The transform kernel used for the transform of the residual block may have a characteristic that allows the vertical transform and the horizontal transform to be separable. In this case, the transform of the residual block may be performed separately as a vertical transform and a horizontal transform. For example, the encoder may perform a vertical transform by applying the transform kernel in the vertical direction of the residual block. In addition, the encoder may perform a horizontal transform by applying the transform kernel in the horizontal direction of the residual block. In the present disclosure, the transform kernel may be used to refer to a set of parameters used for the transform of the residual signal, such as a transform matrix, a transform array, a transform function, or a transform. For example, the transform kernel may be any one of a plurality of available kernels. In addition, transform kernels based on different transform types may be used for vertical transform and horizontal transform, respectively.

[0061] The transform coefficients are distributed with higher coefficients toward the upper left corner of the block and coefficients closer to "0" toward the lower right corner of the block. As the size of the current block increases, there may be many coefficients of "0" in the lower right area of ​​the block. In order to reduce the transform complexity of large-sized blocks, only the random upper left area may be retained and the remaining area may be reset to "0".

[0062] In addition, the error signal may only exist in some areas of the coding block. In this case, the transform process may be performed only on some random areas. In an embodiment, in a block with a size of 2Nx2N, the error signal may only exist in the first 2NxN block, and the transform process may be performed on the first 2NxN block. However, the second 2NxN block may not be transformed and may not be encoded or decoded. Here, N may be any positive integer.

[0063] The encoder may perform an additional transform before quantizing the transform coefficients. The above-mentioned transform method may be referred to as a main transform, and the additional transform may be referred to as an auxiliary transform. The auxiliary transform may be selectable for each residual block. According to an embodiment, the encoder may improve coding efficiency by performing an auxiliary transform on an area where it is difficult to concentrate energy in a low-frequency area by using only the main transform. For example, an auxiliary transform may be additionally performed on a block where a large residual value appears in a direction other than the horizontal or vertical direction of the residual block. Unlike the main transform, the auxiliary transform cannot be performed separately as a vertical transform and a horizontal transform. This auxiliary transform may be referred to as a low-frequency inseparable transform (LFNST).

[0064] The quantization unit 115 quantizes the value of the transform coefficient value output from the transform unit 110 .

[0065] In order to improve the coding efficiency, instead of the method of coding the picture signal as it is, a method is used to predict the picture using the area that has been encoded by the prediction unit 150, and to obtain the reconstructed picture by adding the residual value between the original picture and the predicted picture to the predicted picture. In order to prevent the mismatch between the encoder and the decoder, when the prediction is performed in the encoder, information that can be used in the decoder should be used. To this end, the encoder performs the process of reconstructing the encoded current block again. The inverse quantization unit 120 inversely quantizes the value of the transform coefficient, and the inverse transform unit 125 reconstructs the residual value using the inversely quantized transform coefficient value. At the same time, the filtering unit 130 performs a filtering operation to improve the quality of the reconstructed picture and improve the coding efficiency. For example, a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter may be included. The filtered picture is output or stored in a decoded picture buffer (DPB) 156 to be used as a reference picture.

[0066] A deblocking filter is a filter for removing intra-block distortion generated at the boundary between blocks in a reconstructed picture. By distributing pixels included in several columns or rows based on random edges in the block, the encoder can determine whether to apply a deblocking filter to the edge. When applying a deblocking filter to a block, the encoder can apply a long filter, a strong filter, or a weak filter depending on the strength of the deblocking filter. In addition, horizontal filtering and vertical filtering can be processed in parallel. Sample adaptive offset (SAO) can be used to correct the offset from the original video on a pixel-by-pixel basis relative to the residual block to which the deblocking filter has been applied. In order to correct the offset of a special picture, the encoder can use a technique of dividing the pixels included in the picture into a predetermined number of regions, determining the region in which the offset correction is to be performed, and applying the offset to the region (with offset). Alternatively, the encoder can use a method for applying an offset considering the edge information of each pixel (edge ​​offset). An adaptive loop filter (ALF) is a technique of dividing the pixels included in a video into predetermined groups and then determining a filter to be applied to each group so as to perform filtering differently for each group. Information about whether to apply ALF can be signaled on a per-coding unit basis, and the shape and filter coefficients of the ALF to be applied can vary for each block. In addition, regardless of the characteristics of the target block to which the ALF filter is to be applied, an ALF filter having the same shape (fixed shape) can be applied.

[0067] The prediction unit 150 includes an intra prediction unit 152 and an inter prediction unit 154. The intra prediction unit 152 performs intra prediction within the current picture, and the inter prediction unit 154 performs inter prediction to predict the current picture by using the reference picture stored in the decoded picture buffer 156. The intra prediction unit 152 performs intra prediction from the reconstructed area in the current picture, and sends intra coding information to the entropy coding unit 160. The intra coding information may include at least one of an intra prediction mode, a most probable mode (MPM) flag, an MPM index, and information about a reference sample. The inter prediction unit 154 may also include a motion estimation unit 154a and a motion compensation unit 154b. The motion estimation unit 154a finds the most similar part to the current area with reference to a specific area of ​​the reconstructed reference picture, and obtains a motion vector value as a distance between the areas. The reference area related motion information (reference direction indication information (L0 prediction, L1 prediction or bidirectional prediction), reference picture index, motion vector information, etc.) obtained by the motion estimation unit 154a is sent to the entropy coding unit 160 so as to be included in the bitstream. The motion compensation unit 154B performs inter motion compensation by using the motion information transmitted from the motion estimation unit 154a to generate a prediction block for the current block. The inter prediction unit 154 transmits inter encoding information including the motion information related to the reference region to the entropy coding unit 160.

[0068] According to an additional embodiment, the prediction unit 150 may include an intra block copy (IBC) prediction unit (not shown). The IBC prediction unit performs IBC prediction based on the reconstructed samples in the current picture and sends IBC encoding information to the entropy coding unit 160. The IBC prediction unit refers to a specific area within the current picture to obtain a block vector value indicating a reference area used to predict the current area. The IBC prediction unit may perform IBC prediction by using the obtained block vector value. The IBC prediction unit sends the IBC encoding information to the entropy coding unit 160. The IBC encoding information may include at least one of reference area size information and block vector information (index information and block vector difference information of the block vector for predicting the current block in the motion candidate list).

[0069] When performing the above-mentioned picture prediction, the transform unit 110 transforms the residual value between the original picture and the predicted picture to obtain a transform coefficient value. At this time, the transform can be performed based on a specific block in the picture, and the size of the specific block can vary within a predetermined range. The quantization unit 115 quantizes the transform coefficient value generated by the transform unit 110 and sends the quantized transform coefficient to the entropy coding unit 160.

[0070] The quantized transform coefficients in the form of a two-dimensional array can be rearranged into a one-dimensional array for entropy coding. Regarding the method for scanning the quantized transform coefficients, the size of the transform block and the intra-picture prediction mode can determine which scanning method is used. In an embodiment, diagonal, vertical and horizontal scanning can be applied. This scanning information can be signaled on a block-by-block basis and can be derived based on predetermined rules.

[0071] The entropy coding unit 160 generates a video signal bit stream by indicating entropy coding information of quantized transform coefficients, intra-frame coding information, and inter-frame coding information. The entropy coding unit 160 can use variable length coding (VLC) and arithmetic coding. Variable length coding (VLC) is a technique for transforming input symbols into continuous codewords, wherein the length of the codeword is variable. For example, frequently occurring symbols are represented by shorter codewords, while less frequently occurring symbols are represented by longer codewords. As variable length coding, context-based adaptive variable length coding (CAVLC) can be used. Arithmetic coding uses the probability distribution of each data symbol to transform continuous data symbols into a single decimal number. Arithmetic coding allows the optimal decimal places required to represent each symbol to be obtained. As arithmetic coding, context-based adaptive binary arithmetic coding (CABAC) can be used.

[0072] CABAC is a binary arithmetic coding technique that uses multiple context models generated based on probabilities obtained from experiments. First, when the symbol is not in binary form, the encoder binarizes each symbol by using exponential-Golomb (exp-Golomb) and the like. The binarized value 0 or 1 can be described as bin. The CABAC initialization process is divided into context initialization and arithmetic coding initialization. Context initialization is the process of initializing the probability of occurrence of each symbol, and is determined by the type of symbol, quantization parameter (QP) and slice type (I, P or B). The context model with initialization information can use probability-based values ​​obtained by experiment. The context model provides information about the probability of occurrence of the minimum possible symbol (LPS) or the maximum possible symbol (MPS) for the symbol currently to be coded and about which of the bin values ​​0 and 1 corresponds to the MPS (valMPS). One of the multiple context models is selected via a context index (ctxIdx), and the context index can be derived from information in the current block to be encoded or information about neighboring blocks. Initialization for binary arithmetic coding is performed based on the probability model selected from the context model. In binary arithmetic compilation, encoding is performed by dividing the probability interval by the probability of occurrence of 0 and 1, and then the probability interval corresponding to the bin to be processed becomes the entire probability interval of the next bin to be processed. Information about the position within the previous bin where the previous bin has been processed is output. However, the probability interval cannot be divided infinitely, and therefore, when the probability interval is reduced to a certain size, a renormalization process is performed to widen the probability interval and output the corresponding position information. In addition, after each bin is processed, a probability update process can be performed, in which information about the processed bin is used to set a new probability for the next one to be processed.

[0073] The generated bitstream is encapsulated in a network abstraction layer (NAL) unit as a basic unit. NAL units are classified into video coding layer (VCL) NAL units including video data and non-VCL NAL units including parameter information for decoding video data. There are multiple types of VCL or non-VCL NAL units. The NAL unit includes NAL header information and a raw byte sequence payload (RBSP) as data. The NAL header information includes summary information about the RBSP. The RBSP of the VCL NAL unit includes an integer number of encoded coding tree units. In order to decode the bitstream in a video decoder, it is necessary to separate the bitstream into NAL units and then decode each individual NAL unit. The information required for decoding the video signal bitstream can be included in a picture parameter set (PPS), a sequence parameter set (SPS), a video parameter set (VPS), etc. and sent.

[0074] Figure 1The block diagram of the encoding device 100 according to an embodiment of the present disclosure is illustrated, wherein the blocks shown separately logically distinguish the elements of the encoding device 100. Therefore, depending on the design of the device, the above elements of the encoding device 100 may be installed as a single chip or multiple chips. According to an embodiment, the above operations of each element of the encoding device 100 may be performed by a processor (not shown).

[0075] Figure 2 2 is a schematic block diagram of a video signal decoding apparatus 200 according to an embodiment of the present invention. Figure 2 The decoding device 200 of the present invention includes an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 225, a filtering unit 230 and a prediction unit 250.

[0076] The entropy decoding unit 210 performs entropy decoding on the video signal bit stream to extract the transform coefficient information, intra-frame coding information, inter-frame coding information, etc. of each region. For example, the entropy decoding unit 210 can obtain the binarization code of the transform coefficient information for a specific region from the video signal bit stream. The entropy decoding unit 210 obtains the quantized transform coefficient by debinarizing the binary code. The inverse quantization unit 220 dequantizes the quantized transform coefficient, and the inverse transform unit 225 reconstructs the residual value by using the dequantized transform coefficient. The video signal processing device 200 reconstructs the original pixel value by adding the residual value obtained by the inverse transform unit 225 to the predicted value obtained by the prediction unit 250.

[0077] At the same time, the filtering unit 230 performs filtering on the picture to improve the image quality. This may include a deblocking filter for reducing block distortion and / or an adaptive loop filter for removing distortion of the entire picture. The filtered picture is output or stored in the DPB 256 to be used as a reference picture for the next picture.

[0078] The prediction unit 250 includes an intra prediction unit 252 and an inter prediction unit 254. The prediction unit 250 generates a predicted picture by using the encoding type, the transform coefficient of each region, and the intra / inter encoding information decoded by the above-mentioned entropy decoding unit 210. In order to reconstruct the current block in which decoding is performed, the decoded region of the current picture or other pictures including the current block may be used. In reconstruction, only the current picture, that is, the picture (or tile / slice) in which only intra prediction or intra BC prediction is performed is referred to as an intra picture or an I picture (or tile / slice), and the picture (or tile / slice) in which all intra prediction, inter prediction, and intra BC prediction can be performed is referred to as an inter picture (or tile / slice). In order to predict the sample values ​​of each block in an inter-frame picture (or patch / slice), a picture (or patch / slice) using at most one motion vector and a reference picture index is called a predicted picture or a P picture (or patch / slice), and a picture (or patch / slice) using at most two motion vectors and a reference picture index is called a bidirectional predicted picture or a B picture (or patch / slice). In other words, a P picture (or patch / slice) uses at most one motion information set to predict each block, and a B picture (or patch / slice) uses at most two motion information sets to predict each block. Here, a motion information set includes one or more motion vectors and a reference picture index.

[0079] The intra prediction unit 252 generates a prediction block using the intra coding information and the reconstructed samples in the current picture. As described above, the intra coding information may include at least one of an intra prediction mode, a most probable mode (MPM) flag, and an MPM index. The intra prediction unit 252 predicts the sample value of the current block by using the reconstructed samples located to the left and / or above the current block as reference samples. In the present disclosure, the reconstructed samples, the reference samples, and the samples of the current block may represent pixels. Moreover, the sample value may represent a pixel value.

[0080] According to an embodiment, the reference sample may be a sample included in a neighboring block of the current block. For example, the reference sample may be a sample adjacent to the left boundary of the current block and / or the sample may be a sample adjacent to the upper boundary. Moreover, the reference sample may be a sample located on a line within a predetermined distance from the left boundary of the current block and / or a sample located on a line within a predetermined distance from the upper boundary of the current block among the samples of the neighboring blocks of the current block. In this case, the neighboring blocks of the current block may include a left (L) block, an upper (A) block, a lower left (BL) block, an upper right (AR) block, or an upper left (AL) block.

[0081] The inter prediction unit 254 generates a prediction block using the reference picture and inter coding information stored in the DPB 256. The inter coding information may include a motion information set (reference picture index, motion vector information, etc.) for the current block of the reference block. Inter prediction may include L0 prediction, L1 prediction, and bidirectional prediction. L0 prediction means prediction using a reference picture included in the L0 picture list, and L1 prediction means prediction using a reference picture included in the L1 picture list. To this end, a motion information set (e.g., motion vector and reference picture index) may be required. In the bidirectional prediction method, up to two reference areas may be used, and the two reference areas may exist in the same reference picture or may exist in different pictures. That is, in the bidirectional prediction method, up to two motion information sets (e.g., motion vector and reference picture index) may be used, and the two motion vectors may correspond to the same reference picture index or different reference picture indexes. In this case, the reference picture is a picture that is temporally located before or after the current picture, and may be a picture that has been reconstructed. According to an embodiment, the two reference areas used in the bidirectional prediction scheme may be areas selected from the picture list L0 and the picture list L1, respectively.

[0082] The inter-frame prediction unit 254 can use the motion vector and the reference picture index to obtain the reference block of the current block. The reference block is in the reference picture corresponding to the reference picture index. Moreover, the sample value of the block specified by the motion vector or its interpolated value can be used as a predictor of the current block. For motion prediction with sub-pel unit pixel accuracy, for example, an 8-tap interpolation filter for luminance signal and a 4-tap interpolation filter for chrominance signal can be used. However, the interpolation filter for motion prediction in sub-pixel units is not limited to this. In this way, the inter-frame prediction unit 254 performs motion compensation to predict the texture of the current unit based on the motion picture previously reconstructed using motion information. In such a case, the inter-frame prediction unit can use a motion information set.

[0083] According to another embodiment, the prediction unit 250 may include an IBC prediction unit (not shown). The IBC prediction unit may reconstruct the current region by referring to a specific region including the reconstructed samples in the current picture. The IBC prediction unit obtains IBC encoding information for the current region from the entropy decoding unit 210. The IBC prediction unit obtains a block vector value of the current region indicating a specific region in the current picture. The IBC prediction unit may perform IBC prediction by using the obtained block vector value. The IBC encoding information may include block vector information.

[0084] The reconstructed video picture is generated by adding the prediction value output from the intra prediction unit 252 or the inter prediction unit 254 to the residual value output from the inverse transform unit 225. That is, the video signal decoding apparatus 200 reconstructs the current block using the prediction block generated by the prediction unit 250 and the residual obtained from the inverse transform unit 225.

[0085] at the same time, Figure 2 The block diagram of the decoding device 200 according to an embodiment of the present invention is shown, and the blocks shown separately logically distinguish and show the elements of the decoding device 200. Therefore, the elements of the above-mentioned decoding device 200 may be installed as one chip or multiple chips depending on the design of the device. According to the embodiment, the operation of each element of the above-mentioned decoding device 200 may be performed by a processor (not shown).

[0086] The techniques presented in this specification can be applied to methods and apparatus for both encoders and decoders, and the wording signaling and parsing can be used for convenience of description. In general, signaling can be described as encoding each type of syntax from the perspective of the encoder, and parsing can be described as interpreting each type of syntax from the perspective of the decoder. In other words, each type of syntax can be included in a bitstream and signaled by the encoder, and the decoder can parse the syntax and use it in the reconstruction process. In this case, the bit sequence of each type of syntax arranged according to a specified hierarchical configuration can be referred to as a bitstream.

[0087] A picture can be partitioned into sub-pictures, slices, tiles, etc. and encoded. A sub-picture may include one or more slices or tiles. When a picture is partitioned into multiple slices or tiles and encoded, all slices or tiles within the picture must be decoded before the picture can be output on the screen. On the other hand, when a picture is encoded into multiple sub-pictures, only random sub-pictures can be decoded and output on the screen. A slice may include multiple tiles or sub-pictures. Alternatively, a tile may include multiple sub-pictures or slices. Sub-pictures, slices, and tiles can be encoded or decoded independently of each other, and therefore are conducive to parallel processing and improvements in processing speed. However, there is a disadvantage of increased bit rate because the encoding information of other adjacent sub-pictures, slices, and tiles is not available. Sub-pictures, slices, and tiles can be partitioned into multiple coding tree units (CTUs) and encoded.

[0088] Figure 3An embodiment in which a coding tree unit (CTU) is divided into coding units (CUs) within a picture is illustrated. In the process of coding a video signal, a picture may be divided into a series of coding tree units (CTUs). A coding tree unit may include a luma coding tree block (CTB), two chroma coding tree blocks, and syntax information encoded therein. A coding tree unit may include one coding unit, or a coding tree unit may be divided into multiple coding units. A coding unit may include a luma coding block (CB), two chroma coding blocks, and syntax information encoded therein. A coding block may be partitioned into multiple sub-coding blocks. A coding unit may include one transform unit (TU), or a coding unit may be partitioned into multiple transform units. A transform unit may include a luma transform block (TB), two chroma transform blocks, and syntax information encoded therein. A coding tree unit may be partitioned into multiple coding units. A coding tree unit may become a leaf node without being partitioned. In this case, the coding tree unit itself may be a coding unit.

[0089] A coding unit refers to a basic unit for processing a picture in the processing of the above-mentioned video signal, i.e., intra / inter prediction, transform, quantization and / or entropy coding. The size and shape of a coding unit in a picture may not be constant. A coding unit may have a square or rectangular shape. A rectangular coding unit (or rectangular block) includes a vertical coding unit (or vertical block) and a horizontal coding unit (or horizontal block). In this specification, a vertical block is a block whose height is greater than its width, and a horizontal block is a block whose width is greater than its height. In addition, in this specification, a non-square block may refer to a rectangular block, but the present disclosure is not limited thereto.

[0090] refer to Figure 3 , firstly, the compilation tree unit is split into a quadtree (QT) structure. That is, a node having a size of 2NX2N in the quadtree structure can be split into four nodes having a size of NXN. In this specification, a quadtree may also be referred to as a quadtree. Quadtree splitting may be performed recursively, and not all nodes need to be split at the same depth.

[0091] At the same time, the leaf nodes of the above-mentioned quadtree can be further divided into a multi-type tree (MTT) structure. According to an embodiment of the present invention, in a multi-type tree structure, a node can be divided into a binary or ternary tree structure divided horizontally or vertically. That is, in a multi-type tree structure, there are four segmentation structures, such as vertical binary segmentation, horizontal binary segmentation, vertical ternary segmentation and horizontal ternary segmentation. According to an embodiment of the present invention, in each tree structure, the width and height of the node can have a power of 2. For example, in a binary tree (BT) structure, a node of 2NX2N size can be divided into two NX2N nodes by vertical binary segmentation, and divided into two 2NXN nodes by horizontal binary segmentation. In addition, in a ternary tree (TT) structure, a node of 2NX2N size is divided into (N / 2)X2N, NX2N and (N / 2)X2N nodes by vertical ternary segmentation, and is divided into 2NX(N / 2), 2NXN and 2NX(N / 2) nodes by horizontal ternary segmentation. This multi-type tree splitting can be performed recursively.

[0092] The leaf nodes of the multi-type tree can be coding units. When the coding unit is not larger than the maximum transform length, the coding unit can be used as a unit of prediction and / or transformation without further segmentation. As an embodiment, when the width or height of the current coding unit is larger than the maximum transform length, the current coding unit can be segmented into multiple transform units without explicit signaling about the segmentation. On the other hand, at least one of the following parameters in the above-mentioned quadtree and multi-type tree can be predefined or sent through RBSP of a higher level set such as PPS, SPS, VPS, etc. 1) CTU size: the root node size of the quadtree, 2) Minimum QT size MinQtSize: the minimum allowed QT leaf node size, 3) Maximum BT size MaxBtSize: the maximum allowed BT root node size, 4) Maximum TT size MaxTtSize: the maximum allowed TT root node size, 5) Maximum MTT depth MaxMttDepth: the maximum allowed depth of the MTT split from the leaf node of the QT, 6) Minimum BT size MinBtSize: the minimum allowed BT leaf node size, 7) Minimum TT size MinTtSize: the minimum allowed TT leaf node size.

[0093] Figure 4 An embodiment of a method for signaling a partition of a quadtree and a multi-type tree is shown. A preset flag may be used to signal the partition of the quadtree and the multi-type tree. Figure 4, at least one of the flag "split_cu_flag" indicating whether to split a node, the flag "split_qt_flag" indicating whether to split a quadtree node, the flag "mtt_split_cu_vertical_flag" indicating the split direction of a multi-type tree node, or the flag "mtt_split_cu_binary_flag" indicating the split shape of a multi-type tree node may be used.

[0094] According to an embodiment of the present invention, a "split_cu_flag" as a flag indicating whether to split the current node can be first sent with a signal. When the value of "split_cu_flag" is 0, it indicates that the current node is not split and the current node becomes a compilation unit. When the current node is a compilation tree unit, the compilation tree unit includes an unsplit compilation unit. When the current node is a quadtree node "QT node", the current node is a leaf node "QT leaf node" of the quadtree and becomes a compilation unit. When the current node is a multi-class tree node "MTT node", the current node is a leaf node "MTT leaf node" of the multi-class tree and becomes a compilation unit.

[0095] When the value of "split_cu_flag" is 1, the current node can be split into nodes of a quadtree or a multi-type tree according to the value of "split_qt_flag". The compilation tree unit is the root node of the quadtree and can be first split into a quadtree structure. In the quadtree structure, "split_qt_flag" is sent with a signal for each node "QT node". When the value of "split_qt_flag" is 1, the corresponding node is split into 4 square nodes, and when the value of "qt_split_flag" is 0, the corresponding node becomes a "QT leaf node" of the quadtree, and the corresponding node is split into multiple types of nodes. According to an embodiment of the present invention, the splitting of the quadtree can be limited according to the type of the current node. Quadtree splitting can be allowed when the current node is a compilation tree unit (the root node of the quadtree) or a quadtree node, and quadtree splitting is not allowed when the current node is a multi-type tree node. Each quadtree leaf node "QT leaf node" can be further split into a multi-type tree structure. As described above, when "split_qt_flag" is 0, the current node can be split into multiple types of nodes. In order to indicate the split direction and split shape, "mtt_split_cu_vertical_flag" and "mtt_split_cu_binary_flag" can be sent by signal. When the value of "mtt_split_cu_vertical_flag" is 1, it indicates the vertical splitting of the node "MTT node", and when the value of "mtt_split_cu_vertical_flag" is 0, it indicates the horizontal splitting of the node "MTT node". In addition, when the value of "mtt_split_cu_binary_flag" is 1, the node "MTT node" is split into two rectangular nodes, and when the value of "mtt_split_cu_binary_flag" is 0, the node "MTT node" is split into three rectangular nodes.

[0096] In the tree partition structure, the luminance block and the chrominance block can be partitioned in the same form. That is, the chrominance block can be partitioned with reference to the partition form of the luminance block. When the current chrominance block is smaller than a predetermined size, even if the luminance block is partitioned, the chrominance block may not be partitioned.

[0097] In the tree partition structure, the luminance block and the chrominance block may have different forms. In this case, the luminance block partition information and the chrominance block partition information may be separately signaled. In addition, in addition to the partition information, the luminance block encoding information and the chrominance block encoding information may also be different from each other. In one example, the luminance block and the chrominance block may be different in at least one aspect among the intra-frame coding mode, the encoding information for motion information, and the like.

[0098] The node to be divided into the smallest unit can be regarded as a coding block. When the current block is a coding block, the coding block can be partitioned into several sub-blocks (sub-coding blocks), and the sub-blocks can have the same prediction information or different prediction information. In one example, when the coding unit is in intra mode, the intra prediction modes of the sub-blocks can be the same or different from each other. In addition, when the coding unit is in inter mode, the sub-blocks can have the same motion information or different motion information. In addition, the sub-blocks can be encoded or decoded independently of each other. Each sub-block can be distinguished by a sub-block index (sbIdx). In addition, when the coding unit is partitioned into sub-blocks, the coding unit can be partitioned horizontally, vertically or diagonally. In intra mode, a mode in which the current coding unit is partitioned into two or four sub-blocks horizontally or vertically is called intra sub-partitioning (ISP). In inter mode, a mode in which the current coding block is partitioned diagonally is called a geometric partitioning mode (GPM). In GPM mode, a predetermined angle table is used to derive the position and direction of the diagonal line, and the index information of the angle table is sent with a signal.

[0099] Picture prediction (motion compensation) for encoding is performed on a coding unit that is no longer divided (ie, a leaf node of a coding unit tree). Hereinafter, a basic unit for performing prediction will be referred to as a "prediction unit" or a "prediction block".

[0100] Hereinafter, the term "unit" used herein may replace a prediction unit, which is a basic unit for performing prediction. However, the present disclosure is not limited thereto, and the "unit" may be understood as broadly covering the concept of a coding unit.

[0101] Figure 5 and Figure 6 The intra prediction method according to an embodiment of the present invention is illustrated in more detail. As described above, the intra prediction unit predicts sample values ​​of the current block by using reconstructed samples located on the left and / or above the current block as reference samples.

[0102] first, Figure 5 An embodiment of a reference sample used for prediction of a current block in an intra prediction mode is shown. According to an embodiment, the reference sample may be a sample adjacent to the left boundary of the current block and / or a sample adjacent to the upper boundary. Figure 5 As shown in , when the size of the current block is WXH and samples of a single reference line adjacent to the current block are used for intra prediction, a maximum of 2W+2H+1 neighboring samples located on the left and top of the current block can be used to configure reference samples.

[0103] Pixels from multiple reference lines may be used for intra prediction of the current block. The multiple reference lines may include n lines located within a predetermined range from the current block. According to an embodiment, when pixels from multiple reference lines are used for intra prediction, separate index information indicating lines to be set as reference pixels may be signaled and may be named reference line index.

[0104] When at least some samples to be used as reference samples have not been reconstructed, the intra prediction unit may obtain reference samples by performing a reference sample filling process. The intra prediction unit may perform a reference sample filtering process to reduce errors in intra prediction. That is, filtering may be performed on adjacent samples and / or reference samples obtained by the reference sample filling process to obtain filtered reference samples. The intra prediction unit predicts samples of the current block by using the reference samples obtained as above. The intra prediction unit predicts samples of the current block by using unfiltered reference samples or filtered reference samples. In the present disclosure, adjacent samples may include samples on at least one reference line. For example, adjacent samples may include adjacent samples on a line adjacent to a boundary of the current block.

[0105] Next, Figure 6 An embodiment of a prediction mode for intra prediction is shown. For intra prediction, intra prediction mode information indicating an intra prediction direction may be signaled. The intra prediction mode information indicates one of a plurality of intra prediction modes included in an intra prediction mode set. When a current block is an intra prediction block, a decoder receives the intra prediction mode information of the current block from a bitstream. An intra prediction unit of the decoder performs intra prediction on the current block based on the extracted intra prediction mode information.

[0106] According to an embodiment of the present invention, the intra prediction mode set may include all intra prediction modes used in intra prediction (e.g., a total of 67 intra prediction modes). More specifically, the intra prediction mode set may include a plane mode, a DC mode, and a plurality of (e.g., 65) angle modes (i.e., directional modes). Each intra prediction mode may be indicated by a preset index (i.e., an intra prediction mode index). For example, Figure 6As shown, intra-frame prediction mode index 0 indicates plane mode, and intra-frame prediction mode index 1 indicates DC mode. In addition, intra-frame prediction mode indexes 2 to 66 may indicate different angle modes, respectively. The angle modes respectively indicate angles that are different from each other within a preset angle range. For example, the angle mode may indicate an angle within an angle range (i.e., a first angle range) between 45 degrees and -135 degrees in a clockwise direction. The angle mode may be defined based on the 12 o'clock direction. In this case, intra-frame prediction mode index 2 indicates a horizontal diagonal (HDIA) mode, intra-frame prediction mode index 18 indicates a horizontal (horizontal, HOR) mode, intra-frame prediction mode index 34 indicates a diagonal (DIA) mode, intra-frame prediction mode index 50 indicates a vertical (VER) mode, and intra-frame prediction mode index 66 indicates a vertical diagonal (VDIA) mode.

[0107] Meanwhile, the preset angle range may be set differently depending on the shape of the current block. For example, if the current block is a rectangular block, a wide angle mode indicating that the angle in the clockwise direction exceeds 45 degrees or is less than -135 degrees may be additionally used. When the current block is a horizontal block, the angle mode may indicate an angle within an angle range (i.e., a second angle range) between (45+offset1) degrees and (-135+offset1) degrees in the clockwise direction. In this case, angle modes 67 to 76 outside the first angle range may be additionally used. In addition, if the current block is a vertical block, the angle mode may indicate an angle within an angle range (i.e., a third angle range) between (45-offset2) degrees and (-135-offset2) degrees in the clockwise direction. In this case, angle modes -10 to -1 outside the first angle range may be additionally used. According to an embodiment of the present disclosure, the values ​​of offset1 and offset2 may be determined differently depending on the ratio between the width and height of the rectangular block. In addition, offset1 and offset2 may be positive numbers.

[0108] According to yet another embodiment of the present invention, the plurality of angle modes configuring the intra prediction mode set may include a basic angle mode and an extended angle mode. In this case, the extended angle mode may be determined according to the basic angle mode.

[0109] According to an embodiment, the basic angle mode is a mode corresponding to the angle used in the intra-frame prediction of the existing high-efficiency video coding (HEVC) standard, and the extended angle mode may be a mode corresponding to the angle newly added in the intra-frame prediction of the next-generation video coding standard. More specifically, the basic angle mode may be an angle mode corresponding to any one of the intra-frame prediction modes {2, 4, 6, ..., 66}, and the extended angle mode may be an angle mode corresponding to any one of the intra-frame prediction modes {3, 5, 7, ..., 65}. That is, the extended angle mode may be an angle mode between the basic angle modes within the first angle range. Therefore, the angle indicated by the extended angle mode may be determined based on the angle indicated by the basic angle mode.

[0110] According to another embodiment, the basic angle mode may be a mode corresponding to an angle within a preset first angle range, and the extended angle mode may be a wide-angle mode outside the first angle range. That is, the basic angle mode may be an angle mode corresponding to any one of the intra-frame prediction modes {2,3,4,...,66}, and the extended angle mode may be an angle mode corresponding to any one of the intra-frame prediction modes {-14,-12,-12...,-1} and {67,68,...,80}. The angle indicated by the extended angle mode may be determined as an angle on the opposite side of the angle indicated by the corresponding basic angle mode. Therefore, the angle indicated by the extended angle mode may be determined based on the angle indicated by the basic angle mode. At the same time, the number of extended angle modes is not limited thereto, and additional extended angles may be defined according to the size and / or shape of the current block. At the same time, the total number of intra-frame prediction modes included in the intra-frame prediction mode set may vary depending on the configuration of the above-mentioned basic angle mode and the extended angle mode.

[0111] In the above embodiment, the intervals between the extended angle modes can be set based on the intervals between the corresponding basic angle modes. For example, the intervals between the extended angle modes {3, 5, 7, ..., 65} can be determined based on the intervals between the corresponding basic angle modes {2, 4, 6, ..., 66}. In addition, the intervals between the extended angle modes {-14, -12, ..., -1} can be determined based on the intervals between the corresponding basic angle modes {53, 54, ..., 66} on the opposite side, and the intervals of the extended angle modes {67, 68, ..., 80} can be determined based on the intervals between the corresponding basic angle modes {2, 3, 4, ..., 15} on the opposite side. The angular intervals between the extended angle modes can be set to be the same as the angular intervals between the corresponding basic angle modes. In addition, the number of extended angle modes in the intra-frame prediction mode set can be set to be less than or equal to the number of basic angle modes.

[0112] According to an embodiment of the present invention, an extended angle mode may be signaled based on a basic angle mode. For example, a wide angle mode (i.e., an extended angle mode) may replace at least one angle mode (i.e., a basic angle mode) within a first angle range. The basic angle mode to be replaced may be a corresponding angle mode on a side opposite to the wide angle mode. That is, the basic angle mode to be replaced is an angle mode corresponding to an angle in the opposite direction to the angle indicated by the wide angle mode, or an angle mode corresponding to an angle in the opposite direction that differs by a preset offset index. According to one embodiment of the present invention, the preset offset index is 1. The intra-frame prediction mode index corresponding to the basic angle mode to be replaced may be remapped to the wide angle mode to signal the corresponding wide angle mode. For example, the wide angle mode {-14, -12, ..., -1} may be signaled by the intra-frame prediction mode index {52, 53, ..., 66}, respectively, and the wide angle mode {67, 68, ..., 80} may be signaled by the intra-frame prediction mode index {2, 3, ..., 15}, respectively. In this way, the intra-frame prediction mode index for the basic angle mode is used to signal the extended angle mode, and thus even if the configuration of the angle mode for intra-frame prediction of each block is different from each other, the intra-frame prediction mode can be signaled using the same intra-frame prediction mode index set. Therefore, the signaling overhead caused by the change of the intra-frame prediction mode configuration can be minimized.

[0113] Meanwhile, whether to use the extended angle mode may be determined based on at least one of the shape and size of the current block. According to an embodiment, when the size of the current block is greater than a preset size, the extended angle mode may be used for intra-frame prediction of the current block, otherwise the basic angle mode can be used only for intra-frame prediction of the current block. According to another embodiment, when the current block is a block other than a square, the extended angle mode may be used for intra-frame prediction of the current block, and when the current block is a square block, only the basic angle mode may be used for intra-frame prediction of the current block.

[0114] The intra prediction unit determines the reference samples and / or interpolated reference samples to be used for intra prediction of the current block based on the intra prediction mode information of the current block. When the intra prediction mode index indicates a specific angle mode, the reference samples corresponding to the specific angle or the interpolated reference samples from the current samples in the current block are used for the prediction of the current pixel. Therefore, depending on the intra prediction mode, different sets of reference samples and / or interpolated reference samples can be used for intra prediction. After performing intra prediction of the current block using the reference samples and the intra prediction mode information, the decoder reconstructs the sample values ​​of the current block by adding the residual signal of the current block obtained from the inverse transform unit to the intra prediction value of the current block.

[0115] The motion information used for inter prediction may include reference direction indication information (inter_pred_idc), reference picture indexes (ref_idx_l0, ref_idx_l1), and motion vectors (mvL0, mvL1). Reference picture list utilization information (predFlagL0, predFlagL1) may be set based on the reference direction indication information. In one example, for unidirectional prediction using L0 reference pictures, predFlagL0=1 and predFlagL1=0 may be set. For unidirectional prediction using L1 reference pictures, predFlagL0=0 and predFlagL1=1 may be set. For bidirectional prediction using both L0 and L1 reference pictures, predFlagL0=1 and predFlagL1=1 may be set.

[0116] When the current block is a coding unit, the coding unit may be partitioned into a plurality of sub-blocks, and the sub-blocks may have the same prediction information or different prediction information. In one example, when the coding unit is in intra mode, the intra prediction modes of the sub-blocks may be the same or different from each other. In addition, when the coding unit is in inter mode, the sub-blocks may have the same motion information or different motion information. In addition, the sub-blocks may be encoded or decoded independently of each other. Each sub-block may be distinguished by a sub-block index (sbIdx).

[0117] The motion vector of the current block is likely to be similar to the motion vectors of neighboring blocks. Therefore, the motion vectors of neighboring blocks can be used as motion vector predictors (MVPs), and the motion vectors of neighboring blocks can be used to derive the motion vector of the current block. In addition, to improve the accuracy of the motion vector, the motion vector difference (MVD) between the best motion vector of the current block and the motion vector predictor found by the encoder from the original video can be signaled.

[0118] Motion vectors can have various resolutions, and the resolution of motion vectors can vary on a block-by-block basis. Motion vector resolution can be expressed in integer units, half-pixel units, 1 / 4 pixel units, 1 / 16 pixel units, 4 integer pixel units, etc. Videos such as screen content have simple graphic forms such as text, and do not require the application of interpolation filters. Therefore, integer units and 4 integer pixel units can be selectively applied on a block-by-block basis. Blocks encoded using affine modes representing rotation and scaling show significant changes in form, so integer units, 1 / 4 pixel units, and 1 / 16 pixel units can be selectively applied on a block-by-block basis. Information about whether motion vector resolution is selectively applied on a block-by-block basis is sent by amvr_flag with a signal. If applied, information about the motion vector resolution to be applied to the current block is sent by amvr_precision_idx with a signal.

[0119] In the case of a block to which bidirectional prediction is applied, weights applied between two prediction blocks when a weighted average is applied may be equal or different, and information on the weights is signaled via BCW_IDX.

[0120] In order to improve the accuracy of the motion vector predictor, a merge or AMVP (Advanced Motion Vector Prediction) method can be selectively used on a block-by-block basis. The merge method is a method of configuring the motion information of the current block to be the same as the motion information of the neighboring blocks adjacent to the current block, and its advantage is that the motion information is propagated spatially without changes in motion areas with homogeneity, and thus the encoding efficiency of the motion information is increased. On the other hand, the AMVP method is a method for predicting motion information in the L0 and L1 prediction directions, respectively, and signaling the optimal motion information in order to represent accurate motion information. The decoder derives the motion information for the current block by using the AMVP or merge method, and then uses the reference block in the motion information located in the reference picture as the prediction block for the current block.

[0121] A method for deriving motion information in a merge or AMVP relates to a method for constructing a motion candidate list using a motion vector predictor derived from a neighboring block of a current block, and then signaling index information for the best motion candidate. In the case of AMVP, motion candidate lists are derived for L0 and L1, respectively, so the best motion candidate indexes (mvp_l0_flag, mvp_l1_flag) for L0 and L1 are signaled, respectively. In the case of a merge, a single motion candidate list is derived, so a single merge index (merge_idx) is signaled. There may be various motion candidate lists derived from a single coding unit, and a motion candidate index or a merge index may be signaled for each motion candidate list. In this case, a mode in which there is no information about a residual block in a block encoded using a merge mode may be referred to as a merge skip (MergeSkip) mode.

[0122] The motion candidate and the motion information candidate in this specification may have the same meaning. In addition, the motion candidate list and the motion information candidate list in this specification may have the same meaning.

[0123] Symmetric MVD (SMVD) is a method that makes the motion vector difference (MVD) values ​​in the L0 and L1 directions symmetrical in the case of bidirectional prediction, thereby reducing the bit rate of the transmitted motion information. The MVD information in the L1 direction, which is symmetrical with the L0 direction, is not transmitted, and the reference picture information in the L0 and L1 directions is not transmitted, but is derived during decoding.

[0124] Overlapped Block Motion Compensation (OBMC) is a method that generates a prediction block for a current block by using the motion information of neighboring blocks when the blocks have different motion information, and then weighted averaging the prediction blocks to generate a final prediction block for the current block. This has the effect of reducing the blocking phenomenon that occurs at the edges of blocks in motion compensated video.

[0125] Typically, the merged motion candidate has lower motion accuracy. In order to improve the accuracy of the merged motion candidate, a merge mode with MVD (MMVD) method can be used. The MMVD method is a method for correcting motion information by using a candidate selected from several motion difference value candidates. Information about the correction value of the motion information obtained by the MMVD method (for example, an index indicating a candidate selected from the motion difference value candidates, etc.) can be included in the bitstream and sent to the decoder. By including information about the correction value of the motion information in the bitstream, bit rate can be saved compared to including the existing motion information difference value in the bitstream.

[0126] The template matching (TM) method is a method of configuring a template by neighboring pixels of a current block, searching for a matching area most similar to the template, and correcting motion information. Template matching (TM) is a method of reducing the size of the encoded bitstream by performing motion prediction by a decoder without including motion information in the bitstream. The decoder does not have the original image, and therefore can schematically derive motion information of the current block by using pre-reconstructed neighboring blocks.

[0127] The decoder-side motion vector refinement (DMVR) method is a method for correcting motion information by correlation with an already reconstructed reference video in order to find more accurate motion information. The DMVR method is a method of using the bidirectional motion information of the current block to use the point with the best match between the reference blocks in the reference picture as a new bidirectional motion within a predetermined area of ​​the two reference pictures. When performing the DMVR method, the encoder may perform DMVR on one block to correct the motion information, and then partition the block into sub-blocks, and perform DMVR on each sub-block to correct the motion information of the sub-block again, and this may be referred to as multi-pass DMVR (MP-DMVR).

[0128] The local illumination compensation (LIC) method is a method for compensating for brightness variation between blocks, and is a method of deriving a linear model by using neighboring pixels adjacent to a current block, and then compensating for brightness information of the current block by using the linear model.

[0129] Existing video coding methods perform motion compensation by only considering parallel motion in the upward, downward, leftward and rightward directions, thus reducing coding efficiency when encoding videos including motions such as scaling, zooming and rotation that are common in real life. In order to express motions such as scaling, zooming and rotation, an affine model-based motion prediction technique using four (rotation) or six (scaling, scaling, rotation) parameter models can be applied.

[0130] Bidirectional optical flow (BDOF) is used to correct the predicted block by estimating the amount of change in pixels based on optical flow from a reference block of a block with bidirectional motion. The motion information derived by BDOF of VVC can be used to correct the motion of the current block.

[0131] Prediction refinement with optical flow (PROF) is a technique for improving the affine motion prediction accuracy for each sub-block to make it similar to the motion prediction accuracy for each pixel. Similar to BDOF, PROF is a technique for obtaining the final prediction signal by calculating a correction value for each pixel relative to the pixel value of affine motion compensation for each sub-block based on optical flow.

[0132] The combined inter / intra prediction (CIIP) method is a method for generating a final prediction block by performing weighted averaging of a prediction block generated by an intra prediction method and a prediction block generated by an inter prediction method when generating a prediction block for a current block.

[0133] The intra block copy (IBC) method is a method for finding a portion most similar to a current block in an already reconstructed area within a current picture and using a reference block as a prediction block for the current block. In this case, information related to a block vector, which is a distance between the current block and the reference block, may be included in a bitstream. The decoder is able to parse the information related to the block vector contained in the bitstream to calculate or set the block vector for the current block.

[0134] The bi-prediction with CU level weight (BCW) method is a method of performing weighted averaging of two prediction blocks by adaptively applying weights on a block-by-block basis without generating the prediction blocks using an average value, with respect to two motion compensated prediction blocks from different reference pictures.

[0135] The intra-frame TMP (template matching prediction) method is a method in which a video signal processing device constructs a reference template by using pixel values ​​of surrounding blocks adjacent to a current block, finds a portion most similar to the constructed reference template in an already reconstructed area within a current picture, and then uses the corresponding reference block (the portion found within the reconstructed area) as a prediction block for the current block.

[0136] The Multiple Hypothesis Prediction (MHP) method is a method for performing weighted prediction through various prediction signals by transmitting additional motion information in addition to unidirectional and bidirectional motion information during inter-picture prediction.

[0137] Cross-component linear model (CCLM) is a method of constructing a linear model by using the high correlation between a luminance signal and a chrominance signal at the same position as the luminance signal, and then predicting the chrominance signal by using the linear model. A template is constructed using a completely reconstructed block among neighboring blocks adjacent to the current block, and the parameters for the linear model are derived through the template. Next, the current luminance block is downsampled, and the current luminance block is selectively reconstructed based on the video format to fit the size of the chrominance block. Finally, the downsampled luminance block and the corresponding linear model are used to predict the chrominance block of the current block. In this case, the method of using two or more linear models is called a multi-model linear model (MMLM).

[0138] In independent scalar quantization, the reconstructed coefficient t'k of the input coefficient tk depends only on the associated quantization index qk. That is, the quantization index of the random reconstructed coefficient has a different value from the quantization index of other reconstructed coefficients. Here, t'k can be a value including the quantization error in tk, and can be different or the same depending on the quantization parameter. Among them, t'k can be called a reconstructed transform coefficient or an inverse quantized transform coefficient, and the quantization index can be called a quantized transform coefficient.

[0139] In uniform reconstruction quantization (URQ), the reconstruction coefficients have the property of being arranged at equal intervals. The distance between two adjacent reconstruction values ​​may be referred to as the quantization step size. The reconstruction value may include 0, and the entire set of available reconstruction values ​​may be uniquely defined based on the quantization step size. The quantization step size may vary depending on the quantization parameter.

[0140] In existing methods, quantization reduces the set of acceptable reconstructed transform coefficients, and the elements of this set may be limited. Therefore, there is a limitation in minimizing the average error between the original video and the reconstructed video. Vector quantization can be used as a method for minimizing the average error.

[0141] A simple form of vector quantization used in video coding is sign data hiding. This is a method in which the encoder does not encode the sign for a non-zero coefficient, and the decoder determines the sign for the coefficient based on whether the sum of the absolute values ​​of all coefficients is even or odd. To this end, in the encoder, at least one coefficient may be incremented or decremented by "1", and at least one coefficient may be selected and have a value adjusted to be optimal from a rate-distortion cost perspective. In one example, a coefficient having a value close to a boundary between quantization intervals may be selected.

[0142] Another vector quantization method is grid-coded quantization, and in video coding, it is used as an optimal path search technique to obtain optimized quantization values ​​in dependent quantization. On a block-by-block basis, the quantization candidates for all coefficients in the block are placed in a grid map, and the optimal grid path between the optimized quantization candidates is found by considering the rate-distortion cost. Specifically, the dependent quantization applied to video coding can be designed so that the set of acceptable reconstructed transform coefficients relative to the transform coefficients depends on the value of the transform coefficients that precede the current transform coefficient in the reconstruction order. At this time, by selectively using multiple quantizers according to the transform coefficients, the average error between the original video and the reconstructed video is minimized, thereby increasing the coding efficiency.

[0143] In the intra prediction coding technology, the matrix intra prediction (MIP) method is a matrix-based intra prediction method, and obtains a prediction signal by using a predefined matrix and an offset value of pixels on the left and top of a neighboring block, which is different from a prediction method having directionality of pixels from neighboring blocks adjacent to a current block. In the MIP method, the matrix can be a matrix vector.

[0144] In order to derive an intra prediction mode for a current block, based on a template that is a random reconstruction area adjacent to the current block, the current block may be reconstructed using an intra prediction mode for a template derived from neighboring pixels of the template. First, the decoder may generate a prediction template for the template by using neighboring pixels (references) adjacent to the template, and may use an intra prediction mode that generates a prediction template that is most similar to the reconstructed template to reconstruct the current block. This method may be referred to as template intra mode derivation (TIMD).

[0145] Typically, the encoder can determine the prediction mode for generating a prediction block and generate a bitstream including information about the determined prediction mode. The decoder can parse the received bitstream to set the intra-frame prediction mode. In this case, the bit rate of the information about the prediction mode can be approximately 10% of the total bitstream size. In order to reduce the bit rate of the information about the prediction mode, the encoder may not include the information about the intra-frame prediction mode in the bitstream. Therefore, the decoder can use the characteristics of the neighboring blocks to derive (determine) the intra-frame prediction mode for reconstruction of the current block, and can use the derived intra-frame prediction mode to reconstruct the current block. In this case, in order to derive the intra-frame prediction mode, the decoder can apply the Sobel filter horizontally and vertically to each neighboring pixel adjacent to the current block to infer the directional information, and then map the directional information to the intra-frame prediction mode. The method by which the decoder derives the intra-frame prediction mode using the neighboring blocks can be described as decoder-side intra-frame mode derivation (DIMD).

[0146] Figure 7The diagram illustrates positions of neighboring blocks used to construct a motion candidate list in inter-frame prediction.

[0147] The neighboring block may be a spatially positioned block or a temporally positioned block. The neighboring block spatially adjacent to the current block may be at least one of a left (A1) block, a lower left (A0) block, an upper (B1) block, an upper right (B0) block, or an upper left (B2) block. The neighboring block temporally adjacent to the current block may be a block in a collocated picture, which includes the position of the upper left pixel of the lower right (BR) block of the current block. When the neighboring block temporally adjacent to the current block is encoded using an intra-frame mode, or when the neighboring block temporally adjacent to the current block is positioned as not being used, a block including the horizontal and vertical center (Ctr) pixel positions in the current block in the collocated picture corresponding to the current picture may be used as a temporal neighboring block. The motion candidate information derived from the collocated picture may be referred to as a temporal motion vector predictor (TMVP). Only one TMVP can be derived from a block. A block may be partitioned into a plurality of sub-blocks, and a TMVP candidate may be derived for each sub-block. The method for deriving TMVP based on sub-blocks may be referred to as sub-block temporal motion vector predictor (sbTMVP).

[0148] It can be determined whether to apply the method described in this specification based on at least one of the information related to slice type information (e.g., whether the slice is an I slice, a P slice, or a B slice), whether the current block is a tile, whether the current block is a sub-picture, the size of the current block, the depth of the coding unit, whether the current block is a luminance block or a chrominance block, whether the frame is a reference frame or a non-reference frame, and the temporal layer corresponding to the reference sequence and layer. The multiple pieces of information used to determine whether to apply the method described in this specification can be multiple pieces of information pre-agreed between the decoder and the encoder. In addition, such multiple pieces of information can be determined according to the profile and level. Such multiple pieces of information can be expressed by variable values, and the bitstream can include information about the variable values. That is, the decoder can parse the information about the variable values ​​included in the bitstream to determine whether to apply the above method. For example, it can be determined whether to apply the above method based on the width length or height length of the coding unit. If the width length or height length is equal to or greater than 32 (e.g., 32, 64, or 128), the above method can be applied. If the width length or height length is less than 32 (e.g., 2, 4, 8, or 16), the above method can be applied. If width length or height length is equal to 4 or 8, the above method can be applied.

[0149] Figure 8 is a diagram illustrating one type of transform kernel according to an embodiment of the present disclosure.

[0150] Specifically, Figure 8The definition of the transform kernel used in MTS is illustrated, and the formulas (basis functions) of the DCT-II, DCT-V, DCT-VIII, DST-I, DST-VII, and DST-IV kernels applied to MTS are illustrated. In the present disclosure, DCT-II may be described as DCT-2 (DCT2), DCT-V may be described as DCT-5 (DCT5), DCT-VIII may be described as DCT-8 (DCT8), DST-I may be described as DST-1 (DST1), DST-VII may be described as DST-7 (DST7), and DST-IV may be described as DST-4 (DST4).

[0151] DCT and DST can be expressed as functions of cosine and sine, respectively, and when the basis function of the transform kernel for the number of samples N is expressed as T i (j), the index i represents the index in the frequency domain, and the index j represents the index within the basis function. That is, as i decreases, it represents a low-frequency basis function, and as i increases, it represents a high-frequency basis function. When expressed as a two-dimensional matrix, the basis function T i (j) represents the jth element in the i-th row, and because Figure 8 All the transform kernels illustrated in have separable characteristics, so the transform can be performed on the residual signal X in the horizontal and vertical directions respectively. When the residual signal block is X and the transform kernel matrix is ​​T, the transform for the residual signal X can be expressed as TXT'. In this case, T' refers to the transpose of the transform kernel matrix T.

[0152] Depend on Figure 8 The values ​​of the transformation matrix defined by the basis functions illustrated in the figure may be in decimal form rather than integer form. Therefore, in the video encoding device and the decoding device, it may be difficult to implement the values ​​in decimal form in hardware. Therefore, the video signal can be encoded and decoded using an integer approximate transformation kernel from the original transformation kernel including the values ​​in decimal form. The approximate transformation kernel including the values ​​in integer form can be generated by scaling and rounding the original transformation kernel. The integer values ​​included in the approximate transformation kernel may be values ​​within the range that can be expressed by a preconfigured number of bits. The preconfigured number of bits may be 8 or 10 bits. Depending on the approximation, the orthogonality of DCT and DST may not be maintained. However, because the loss of coding efficiency caused by this is not significant, it may be advantageous to approximate the transformation kernel to integer form in terms of hardware implementation.

[0153] The identity transform (IDTR) is a transform in which the result of the transform is the same as before the transform, and is called an identity transform. Typically, an identity transform constructs a transform matrix by configuring "1" at positions where rows and columns have the same value. However, the identity transform uses any fixed value other than 1 to equally increase or decrease the value of the input residual signal.

[0154] Fig. 9 is a diagram illustrating the 0th (lowest frequency component of the corresponding transform kernel) basis functions of DCT-II, DCT-V, DCT-VIII, DST-I, and DST-VII transforms according to an embodiment of the present disclosure.

[0155] Specifically, Fig. 9 It is T i (j) is a graph of Figure 8 The transformation basis function of DCT / DST defined in , when N is 8 and i is 0, and the horizontal axis represents the index j (j=0, 1, ..., N-1) in the transformation basis function, and the vertical axis represents the signal amplitude value.

[0156] like Fig. 9 As illustrated in , since DST-VII shows a trend that the signal increases with increasing index j, it may be effective for a pattern of the residual signal in which the energy of the residual signal increases with increasing horizontal and vertical distances from the upper left coordinates of the block within the residual signal block, such as intra-screen prediction.

[0157] On the other hand, since DCT-VIII shows a pattern in which the signal amplitude decreases as the index j increases, it may be effective for a pattern of the residual signal in which the energy of the residual signal increases as the horizontal and vertical distances from the upper left coordinate of the block decrease within the residual signal block.

[0158] DCT-I shows a shape in which the signal amplitude increases as the index j in the basis function increases but decreases from a specific index. Therefore, it may be effective for a pattern of a residual signal in which the energy of the residual signal increases as the index moves toward the center of the residual block.

[0159] In the case of DCT-II, the 0th basis function represents DC, and it may be effective for a pattern of a residual signal in which pixel values ​​in a residual block are uniformly distributed, such as inter-screen prediction.

[0160] DCT-V is similar to DCT-II, but the value when j is 0 is smaller than the value when j is not 0, so it has a signal model in the form of a straight line that is broken when j is 1.

[0161] In the case of a conventional video codec that mainly uses only DCT-II, since it is not possible to adaptively perform transformation on the pattern of the residual signal depending on the prediction mode and the characteristics of the original signal, optimal coding efficiency cannot be achieved. However, high compression efficiency can be expected for Adaptive Multiple Transform (AMT), which performs transform coding by differently selecting a transform kernel optimized for the pattern of the residual signal by using different transform kernels depending on the prediction mode. Similar to AMT, the Multiple Transform Selection (MTS) technology is a transform coding method that can improve coding efficiency by adaptively selecting a transform kernel depending on the prediction mode.

[0162] Hereinafter, a combination of transformation kernels according to an embodiment of the present disclosure is described.

[0163] DCT2 can be used as the basic transform kernel for reconstructing the current block. Meanwhile, when DCT2 is not used, the remaining kernels (e.g., DCT8, DST7, DCT5, DST4, and DST1) can be used. When DCT2 is not used, some preconfigured combinations for the remaining kernels can be used. Table 1 shows Figure 8 Combinations of cores other than DCT2 and IDTR among the transform cores disclosed in Table 1. Table 1 shows combinations of five types of cores DCT8, DST7, DCT5, DST4 and DST1. Specifically, Table 1 shows 25 combinations that can be configured with two transform cores as a pair (combination). A video signal processing device (e.g., a decoder, an encoder) can use any of the 25 combinations in Table 1 as a transform core for the horizontal or vertical direction of the current block. At the same time, IDTR can be used only when specific conditions are met.

[0164] [Table 1]

[0165] Transform kernel

[25] [2]=

[0166] {

[0167] {DCT8,DCT8},{DCT8,DST7},{DCT8,DCT5},{DCT8,DST4},{DCT8,DST1},

[0168] {DST7,DCT8},{DST7,DST7},{DST7,DCT5},{DST7,DST4},{DST7,DST1},

[0169] {DCT5,DCT8},{DCT5,DST7},{DCT5,DCT5},{DCT5,DST4},{DCT5,DST1},

[0170] {DST4,DCT8},{DST4,DST7},{DST4,DCT5},{DST4,DST4},{DST4,DST1},

[0171] {DST1,DCT8},{DST1,DST7},{DST1,DCT5},{DST1,DST4},{DST1,DST1},

[0172] };

[0173] Fig.10 and Fig.11 is a diagram illustrating a transformation kernel set according to an embodiment of the present disclosure.

[0174] refer to Fig.10 A transform kernel set determined based on the intra prediction mode of the current block and the size of the current block is described.

[0175] refer to Fig.10 , the transform kernel used in the intra prediction mode can be determined based on the intra prediction mode and the size of the current block (eg, coding block, transform block). In addition, Fig.10 It can represent the set of transformation kernels used for MIP. Fig.10 101 in represents the type of intra prediction mode (i.e., an index indicating the intra prediction mode), and 102 represents the size (i.e., width x height) of the current block (e.g., coding block, transform block). For example, if the size of the current block is 4x4 and the type of intra prediction mode is 1, the transform core for the current block can be determined based on the transform core set corresponding to index 0. Fig.11 , the transform core set corresponding to index 0 may be T0{18, 24, 17, 23, 8, 12}. In addition, the video signal processing device may reconstruct the current block based on the transform core (sub-transform core set) included in the transform core set corresponding to an index T0. Fig.11 , but this is only an example and the number of indexes may be 4.

[0176] Fig.11 A illustrates a transform kernel set consisting of indices corresponding to six transform kernels. Fig.11 A is a diagram illustrating some of the 80 transform core sets consisting of indices corresponding to six transform cores. Fig.11 Each index of the transformation core set of A can correspond to a combination in the transformation core combination (sub-transformation core set) of Table 1. For example, the 25 combinations of Table 1 can be indexed from 0 to 24 respectively, and Fig.11 A transformation kernel set and a combination of indexes included in Table 1 may correspond. Fig.11The transform kernel set of A may be determined based on the size of the current block (coding block, transform block) and the intra prediction mode of the current block. Fig.11 B is a diagram Fig.11 A diagram of the first transformation core set T0 among the transformation core sets of A. The indexes of the transformation core sets can be grouped into multiple groups based on a preconfigured convention. That is, the number of indexes of the transformation core sets to be grouped can be adaptively configured. In this case, the group can consist of three or more. Reference Fig.11 B. The indexes of the transform core set may be grouped based on a plurality of reference values ​​(e.g., a first reference value and a second reference value). For example, if the grouping value is less than or equal to the first reference value, a group including one index (18) may be selected; if the grouping value is greater than the first reference value and less than or equal to the second reference value, a group including four indexes (18, 2, 17, 23) may be selected; and if the grouping value is greater than the second reference value, a group including six indexes (18, 24, 17, 23, 8, 12) may be selected. If the grouping value is compared with the reference value and the corresponding group is selected according to a predefined configuration, there may be an effect of reducing complexity compared to signaling / parsing one of the six indexes (transform cores) for the current block. In this case, the reference value may be the sum of the transform coefficients, and the first reference value and the second reference value may be determined based on the sum of the transform coefficients. For example, the first reference value may be 6, and the second reference value may be 32.

[0177] Separate signaling may be required to indicate the indexes included in the group selected by comparing the reference value and the grouping value. Fig.11 As shown in FIG. 1B, if the grouping value is greater than the first reference value and less than or equal to the second reference value, the selected group may be a group consisting of indexes of (18, 24, 17, 23). In this case, separate signaling may be required to indicate each of the four indexes. Similarly, if the grouping value is greater than the second reference value, the selected group may be a group consisting of indexes of (18, 24, 17, 23, 8, 12). In this case, separate signaling (mts_idx) may be required to indicate each of the six indexes. The index within the transform core set may be indicated by the above-mentioned mts_idx. In this case, mts_idx may have a fixed bit size. For example, mts_idx for 6 indexes may have a size of 3 bits. Alternatively, mts_idx may be signaled by a truncated unary binarization (TB) method. Because mts_idx is coded by the TB method, CABAC coding based on the context model may be applied to the first bin, and CABAC coding based on the TB may be applied to the remaining bins.

[0178] Meanwhile, if the grouping value is equal to or less than the first reference value, the selected group may be a group consisting of an index of 18. In this case, since the group consists of only one index, separate signaling for indicating one index may not be required.

[0179] Fig.12 is a diagram illustrating a process of reconstructing a residual signal according to an embodiment of the present disclosure.

[0180] The residual signal is the difference between the original signal and the predicted signal, and has the characteristic that the energy distribution of the signal changes depending on the prediction method. Therefore, if a transform kernel, such as MTS, is adaptively selected depending on the prediction method, the coding efficiency can be improved. In addition, if a transform using only the MTS or DCT2 kernel is called a main transform, the video signal processing device can also improve the coding efficiency by additionally performing an auxiliary transform on the coefficient block of the main transform. The auxiliary transform is effective in energy compression, especially for the predicted residual signal block in the screen, where there may be strong energy in directions other than the horizontal or vertical direction of the residual signal block.

[0181] refer to Fig.12 , the video signal processing device may parse the syntax elements related to the residual signal included in the bitstream, and reconstruct the quantization coefficients by inverse binarization based on the parsing result. The video signal processing device may perform inverse quantization on the reconstructed quantization coefficients to obtain transform coefficients. The video signal processing device may perform an inverse transform on the transform coefficients to reconstruct the residual signal block. In this case, the inverse transform may be applied to the block to which the transform skip (TS) is not applied. The video signal processing device may perform the inverse transform in the order of the auxiliary inverse transform and the main inverse transform. In this case, the auxiliary inverse transform may be omitted. For example, if the current block is encoded in the inter-frame prediction mode, the auxiliary inverse transform may be omitted. In addition, the auxiliary inverse transform may be omitted depending on the size of the current block. The reconstructed residual signal includes a quantization error, and the auxiliary transform may reduce the quantization error by changing the energy distribution of the residual signal compared to performing only the main transform.

[0182] Fig.13 is a diagram illustrating a region of interest (ROI) of a block to which a secondary transform is applied according to an embodiment of the present disclosure.

[0183] According to an embodiment of the present disclosure, Fig.13 The number indicated in the sub-block in may be a sub-block index, and the sub-block index may be a scanning order and may be scanned in order from a small number to a large number.

[0184] Fig.13 A illustrates the ROI of LFNST4. The ROI of LFNST4 may be an ROI for a transform block of size 4×N or N×4. In this case, N may be an integer between 4 and 128. Fig.13 A, the ROI of LFNST4 can be an ROI in a 16×4 block composed of four sub-blocks (sub-block 0 to sub-block 3). In this case, the ROI is a sub-block with a size of 4×4, and refers to Fig.13 A, ROI corresponds to sub-block 0. The number of input samples of ROI may be 16. The forward transformation matrix of LFNST4 may be Rx16. In this case, R may be 4, 8, 16, etc. For example, if R is 16, 16 transformation coefficients may be generated after transformation.

[0185] Fig.13 B illustrates the ROI of LFNST8. The ROI of LFNST8 may be an ROI for a transform block of size 8×N or N×8. In this case, N may be an integer between 8 and 128. Fig.13 B, the ROI of LFNST8 can be an ROI in a 16×8 block consisting of 8 sub-blocks (sub-block 0 to sub-block 7). In this case, the ROI can be a region corresponding to 4 4×4 size sub-blocks, and refer to Fig.13 B, ROI corresponds to sub-blocks 0, 1, 2, and 3. The number of input samples of ROI may be 64. The forward transform matrix of LFNST8 may be R x 64. In this case, R may be 8, 16, 32, 64, etc. For example, if R is 32, 32 transform coefficients may be generated after the transform.

[0186] Fig.13 C illustrates the ROI of LFNST16. The ROI of LFNST8 may be an ROI for a transform block of size 16×N or N×16. In this case, N may be an integer between 16 and 128. Fig.13 C, the ROI of LFNST16 can be an ROI in a 16×16 block consisting of 16 sub-blocks (sub-block 0 to sub-block 15). In this case, the ROI can be a region corresponding to 6 4×4 size sub-blocks, and refer to Fig.13 B, ROI corresponds to sub-blocks 0, 1, 2, 3, 4, and 5. The number of input samples of the ROI may be 96. The forward transform matrix of LFNST16 may be R×96. In this case, R may be 8, 16, 32, 64, 96, etc. For example, if R is 32, 32 transform coefficients may be generated after the transform.

[0187] Fig.14 is a diagram illustrating a method of applying a secondary transform (LFNST) according to an embodiment of the present disclosure.

[0188] The auxiliary transform can be expressed as the product of the matrix of the auxiliary transform kernel and the coefficient vector of the main transform. In other words, this can be interpreted as mapping the coefficients of the main transform to another space. In this case, if the number of auxiliary transform coefficients is reduced, that is, if the number of basis vectors constituting the auxiliary transform kernel is reduced, the amount of calculation required for the auxiliary transform and the memory capacity required to store the transform kernel can be reduced. For example, when the video signal processing device performs an auxiliary transform on the area corresponding to the upper left ROI of the transform block, if the number of auxiliary transform coefficients is reduced to 32, an auxiliary transform kernel of size 32×96 can be applied, and an inverse auxiliary transform kernel of size 96×32 can be applied.

[0189] refer to Fig.14 , the encoder may perform a forward main transform on the residual signal block to obtain a coefficient block of the main transform. In this case, the residual signal may be a signal obtained by intra-frame prediction. The size of the coefficient block of the main transform may be M×N. The encoder may perform a forward main transform on the residual signal block having a value of min(M,N) of 16 to obtain a coefficient block of the main transform. In addition, the encoder may perform a forward main transform on the samples of the upper left ROI area of ​​the coefficient block of the main transform ( Fig.23 The encoder may perform a 32×96 auxiliary transform (LFNST) on the residual signal block having a value of min(M,N) of 8 to obtain a coefficient block of the main transform. In addition, the encoder may perform an auxiliary transform on samples of the upper left ROI region of the coefficient block of the main transform.

[0190] refer to Fig.14 , the transform coefficients of the entire transform block size including the coefficients of the auxiliary transform may be quantized, and information about the quantized transform coefficients may be included in the bitstream. In addition, the bitstream may include a syntax element (lfnst_idx) related to the auxiliary transform. Specifically, the bitstream may include information on whether the auxiliary transform is applied to the current block and information indicating the transform kernel.

[0191] refer to Fig.14, the decoder can parse the transform coefficients quantized from the bit stream and obtain the transform coefficients by inverse quantization. The decoder can determine whether to perform an inverse auxiliary transform (inverse LFNST) on the current transform block based on the syntax elements related to the auxiliary transform. If the inverse auxiliary transform is applied to the current transform block, 16 or 32 transform coefficients can be input to the inverse auxiliary transform depending on the size of the transform block. The number of transform coefficients input to the inverse auxiliary transform can be the same as the number of transform coefficients obtained by the encoder performing the auxiliary transform. The decoder can obtain the coefficients of the main transform by multiplying the vectorized transform coefficients with the inverse auxiliary transform kernel matrix. The inverse auxiliary transform kernel can be determined based on the size of the transform block, the intra-frame prediction mode, and the syntax elements indicating the transform kernel. The inverse auxiliary transform kernel matrix can be a transposed matrix of the auxiliary transform kernel matrix, and considering the complexity of the implementation, the elements of the kernel matrix can be integers expressed in 10-bit or 8-bit precision. Because the main transform coefficients obtained by the inverse auxiliary transform are in the form of vectors, they can be expressed as data in two-dimensional form. The main transform coefficients can depend on the intra-frame prediction mode. The mapping relationship based on the intra-frame prediction mode applied by the encoder can be applied in the same way. The decoder may obtain a residual signal by performing an inverse main transform on a transform coefficient block of the entire transform block size including the transform coefficient obtained by performing the inverse auxiliary transform. Fig.14 The described process may include a scaling process using a bit shift operation.

[0192] Fig.15 is a diagram illustrating a mapping relationship between an intra prediction mode and a transform kernel set for auxiliary transform according to an embodiment of the present disclosure.

[0193] A transform core set for LFNST applied to a transform block may be determined for each intra prediction mode of the transform block. A transform core set may consist of a plurality of LFNST cores. For example, a transform core set may consist of three or four LFNST cores. The transform core set may be 35, and each transform core set may be indexed with indexes 0 to 34. Intra prediction mode indexes -14 to -1 and 67 to 80 corresponding to the extended angle mode may be mapped to a transform core set with index 2.

[0194] Fig.16 is a diagram illustrating a process of generating a prediction block by using a DIMD according to an embodiment of the present disclosure.

[0195] refer to Fig.16 , the decoder can derive the prediction block by using surrounding samples (blocks, pixels). In this case, the surrounding samples can be the surrounding blocks (pixels) of the current block. Specifically, the decoder can determine the intra prediction mode and weight information for reconstructing the current block by using the surrounding samples as input through a histogram for directional information (angle information).

[0196] Fig.17 is a diagram illustrating positions of surrounding pixels used to derive orientation information according to an embodiment of the present disclosure.

[0197] Fig.17 A shows the situation where all surrounding blocks of the current block can be used to derive directional information. Fig.17 B illustrates the case when the upper boundary of the current block is a sub-picture, slice, tile or CTU boundary, and Fig.17 C illustrates the case when the left boundary of the current block is a sub-picture, slice, tile or CTU boundary. Meanwhile, if the surrounding blocks and the current block do not belong to the same sub-picture, slice, tile and CTU, the surrounding blocks may not be used to derive the orientation information. Fig.17 The grey dots in represent the locations of pixels used to derive the actual orientation information, and the dashed lines represent sub-picture, slice, tile, and CTU boundaries. Fig.17 D to Fig.17 F, pixels located at the boundary can be filled outside the boundary by one pixel to derive orientation information. Through this filling, more accurate orientation information can be derived.

[0198] In order to derive the directional information about the pixel at a specific position, the Sobel filter with a size of 3×3 of Formula 1 can be applied in the horizontal and vertical directions respectively. A in Formula 1 can mean the pixel information (value) of the reconstructed surrounding blocks of the current block size of 3×3. In addition, the directional information θ can be determined using Formula 2. In order to reduce the computational complexity for deriving the directional information, the decoder can derive the directional information θ only by calculating Gy / Gx for Formula 1 without calculating the atan function in Formula 2.

[0199] [Formula 1]

[0200]

[0201] [Formula 2]

[0202]

[0203] refer to Fig.17 , can be targeted Fig.17 The directional information is calculated for each gray point displayed in , and the directional information can be mapped to the angle of the intra-frame prediction mode. The intra-frame prediction mode set may include a planar mode, a DC mode, and multiple (e.g., 65) angle modes (i.e., directional modes). The intra-frame prediction mode can be 67 modes, and the directional information (angle, θ) calculated by Formula 2 can be a value in real units. Therefore, a process of mapping the directional information to a specific intra-frame prediction direction mode is required. The intra-frame prediction directional mode described in the present disclosure can be Figure 6The angle mode illustrated in is the same as that in In addition, in the present disclosure, a method of mapping (determining) an intra prediction directional mode by deriving intra prediction directional information may be described by a DIMD method.

[0204] Fig.18 is a diagram illustrating a method of mapping a directional pattern according to an embodiment of the present disclosure.

[0205] refer to Fig.18 , the intra prediction directional mode can be divided into four segments based on 0 degrees (index 18), 45 degrees (index 34), 90 degrees (index 50) and 135 degrees (index 66) (refer to Figure 6 ). refer to Fig.10 , the segment for determining the intra-frame prediction directional mode can be divided into four segments from segment 0 to segment 3. Segment 0 can be from -45 degrees to 0 degrees, segment 1 can be from 0 degrees to 45 degrees, segment 2 can be from 45 degrees to 90 degrees, and segment 3 can be from 90 degrees to 135 degrees. In this case, each segment can include 16 intra-frame prediction directional modes. The directional mode can be determined by comparing the signs and magnitudes of Gx and Gy calculated by formula 1, and one of the four segments can be determined. For example, if Gx and Gy are positive and the absolute value of Gx is greater than the absolute value of Gy, segment 1 can be selected. The intra-frame prediction directional mode mapped to each segment can be determined by the directional information θ calculated according to formula 2. Specifically, the decoder extends the directional information θ by multiplying the value by 2^16. In addition, the decoder can compare the extended value with the value of the predefined table, find the value closest to the extended value, and determine the intra-frame prediction directional mode based on the closest value. In such a case, the value of the predefined table can be 17. Specifically, the value of the predefined table may be {0, 2048, 4096, 6144, 8192, 12288, 16384, 20480, 24576, 28672, 32768, 36864, 40960, 47104, 53248, 59392, 65536}. In this case, the difference between the predefined table values ​​may be configured differently depending on the difference between the angles of the intra prediction directional modes.

[0206] On the other hand, if atan calculation is not performed to reduce computational complexity and only Gy / Gx is used to obtain the orientation angle, the difference between the predefined table values ​​may not be consistent with the distance between the angles of the intra-frame prediction orientation mode. atan has the characteristic that the slope gradually decreases as the input value increases. Therefore, the above-defined table should also be configured by considering not only the difference between the angles of the intra-frame prediction orientation mode, but also the nonlinear characteristics of atan. For example, the difference between the above-defined table values ​​can be configured to gradually decrease. On the contrary, the difference between the above-defined table values ​​can be configured to gradually increase.

[0207] If the width and height of the current block are different, the available intra-prediction directional modes may be different. That is, if the width and height of the current block are different, the segments used to derive the intra-prediction directional mode may be different. In other words, the segments used to derive the intra-prediction directional mode can be changed based on the width and height of the current block (e.g., the ratio of width to height). For example, if the width of the current block is longer than the height, the intra-prediction mode can be remapped from 67 to 80, and the intra-prediction mode in the opposite direction can be excluded from 2 to 15. For example, if the width of the current block is n (integer) times longer than the height (e.g., 2 times), the intra-prediction modes {3, 4, 5, 6, 7, 8} can be reconfigured (mapped to) {67, 68, 69, 70, 71, 72}, respectively. In addition, if the width of the current block is longer than the height, the intra-prediction mode can be reconfigured to add "65" to the value of the intra-prediction mode. On the other hand, if the width of the current block is shorter than the height, the intra prediction mode may be reconfigured to a value of "67" subtracted from the intra prediction mode.

[0208] The histogram can be used to derive the intra-frame prediction directional mode for reconstructing the current block. As a result of obtaining the directional information of the surrounding blocks, if there are more blocks without directionality than blocks with directionality, the prediction mode for the block without directionality may have the highest cumulative value in the histogram. However, because the directional mode must be derived in order to reconstruct the current block, even if the prediction mode has the highest cumulative value in the histogram, the prediction mode for the block without directionality can be excluded. That is, the smooth area without gradient or directionality between the surrounding pixels may not be used to derive the intra-frame prediction directional mode. For example, the prediction mode for the block without directionality may be a plane mode or a DC mode. If the left neighboring block is a plane mode or a DC mode, the left neighboring block may not be used to derive the directional information, and only the upper neighboring block can be used to derive the directional information. If the neighboring blocks of the current block include a mixture of smooth areas and areas with directionality, the decoder can generate a histogram to emphasize the directionality by using the G value calculated as in Formula 3. In this case, the histogram may be a cumulative value in which the calculated G value is added to each generated intra-prediction directional mode, rather than a frequency-based histogram in which "1" is added to each generated intra-prediction directional mode.

[0209] [Formula 3]

[0210] G=|G x |+|G y |

[0211] Fig.19 is a diagram illustrating a histogram for deriving an intra-prediction directional mode according to an embodiment of the present disclosure.

[0212] Fig.19 The X-axis represents the intra-prediction directional mode, and the Y-axis represents the cumulative value of the G value. The decoder can select the intra-prediction directional mode with the largest cumulative value of the G value among the intra-prediction directional modes. In other words, the decoder can select the intra-prediction directional mode for the current block based on the cumulative value. Fig.19 , modeA with the largest cumulative value and modeB with the second largest cumulative value can be selected as intra-frame prediction directional modes. In order to generate a prediction block for the current block, the decoder can generate a final prediction block by weighted averaging the prediction block generated by modeA, the prediction block generated by modeB, and the prediction block finally generated by the plane mode. In this case, the weight of each prediction block can be determined by using the cumulative values ​​of modeA and modeB. For example, the weight of the prediction block generated for the plane mode can be set to 1 / 3 of the total weight. The weight for the prediction block generated by modeA can be configured as a weight corresponding to the value obtained by dividing the modeA cumulative value by the sum of the cumulative values ​​of modeA and modeB. The weight for the prediction block generated by modeB can be determined as the value of the difference between the modeA weight and 1 / 3 of the total weight. In order to make the calculation of the weight more accurate, the decoder can expand the range of the weight by multiplying the weight for the prediction block generated by modeA by an arbitrary value. The weight for the prediction block generated by modeB and the weight for the prediction block generated for the plane mode can also be expanded in the same way.

[0213] Figure 20 to Figure 22 Matrix-based intra prediction according to an embodiment of the present disclosure is illustrated.

[0214] refer to Fig. 20 , video signal processing equipment can be used by using (input) bdry top and bdry left Predict the current block to obtain (output) the predicted block (pred). top represents the upper reference sample of the current block and can be Fig. 20 W. Bdry left represents the left reference sample of the current block and can be Fig. 20 The reference sample line may be a reference sample line closest to the current block. The video signal processing device may obtain a prediction block of the current block based on the reference sample, the matrix and the offset value. Fig. 20 The method of obtaining the prediction block shown in can be described as matrix-based intra prediction (MIP). The MIP of the present disclosure can be described as affine linear weighted intra prediction (ALWIP).

[0215] refer to Fig.21A method of performing matrix-based intra prediction is described. Fig.21 is a diagram illustrating a method for performing matrix-based intra prediction when the current block size is 8×8. First, i) the video signal processing device may perform an averaging process on neighboring reference samples of the current block. In the averaging process, the upper reference sample bdry of the current block top can be re-expressed as an average value of a predetermined set of units. For example, the video signal processing device can obtain the average value of the eight upper reference samples bdry from the current block. top Take the average of the two reference samples to generate four new reference samples bdry top red Similarly, the video signal processing device can obtain the eight left reference samples bdry from the current block. left Take the average of the two reference samples to generate four new reference samples bdry left red . Therefore, the video signal processing device can obtain a total of eight new reference samples. The number of newly generated reference samples after the averaging process can be determined according to the size of the current block. Next, ii) the video signal processing device can obtain a prediction block (pred) of the current block by performing a matrix-vector multiplication process. The video signal processing device can obtain a prediction block (pred) of the current block based on the result of step i) (i.e., the four new samples (bdry top red , bdry left red ) and / or mode k determine the matrix and offset values ​​required in the matrix-vector multiplication process for obtaining the prediction block of the current block. Mode k will be described later. iii) The video signal processing device can perform interpolation. The result value in step ii) (the prediction block (pred) of the current block) can be mapped to a specified position of the current block. If the current block is larger than the reference sample that has been downsampled (for example, the averaging process of step i), the video signal processing device can generate a final prediction block by applying a single-step linear interpolation in the vertical or horizontal direction or in the vertical and horizontal directions. Single-step linear interpolation can be applied to blocks with max(W, H)>=8 among blocks with a size of W×H. W can be the width of the current block (horizontally), H can be the height of the current block (vertically), and max(W, H) is a function of the larger value between the outputs W and H. Specifically, when the size of the current block is 8×8 (see Fig.21 ), the video signal processing device can obtain 8 new reference samples. In addition, the video signal processing device can obtain 16 new samples by performing a matrix-vector multiplication process. The 16 new samples can be mapped to a specific position of the prediction block of the current block ( Fig.21In the linear interpolation process (eg, single-step linear interpolation), the reference sample corresponding to the width of the current block may be mapped to a specific position ( Fig.21 The reference sample is generated by the averaging process (the gray shaded area in 2102). Fig.21 2102, a specific position can be an odd number ( Fig.21 The reference samples corresponding to the height of the current block may be the same as the reference samples used in the averaging process. In this case, the video signal processing device may perform filtering on the reference samples used in the linear interpolation process. For example, the video signal processing device may perform filtering based on the current sample by using a shift operator 2 after applying and adding the upper and lower weights [1, 2, 1].

[0216] [Formula 4]

[0217] pred=Ax bdry red +b

[0218] A in Formula 1 is a matrix, and if the width W and height H of the current block are 4, A may be a 4×4 matrix with 4 rows and 4 columns. If at least one of the width W and height H of the current block is not 4, A may be an 8×8 matrix with 8 rows and 8 columns. b is an offset value in the form of a vector, which may have W red ×H red The size of . Matrix A and vector b can be one of three sets S0, S1 and S2. S0 can be composed of 18 matrices, and each matrix can be composed of 16 rows and 4 columns and can have 18 offset vectors. S1 can be composed of 10 matrices, and each matrix can be composed of 16 rows and 8 columns and can have 10 offset vectors. S2 can be composed of 6 matrices, and each matrix can be composed of 64 rows and 8 columns and can have 6 offset vectors. S0, S1 and S2 can be indicated by a separate index (idx). The index can be determined as illustrated in Formula 5.

[0219] [Formula 5]

[0220]

[0221] In Formula 5, W may be the width (horizontally) of the current block, and H may be the height (vertically) of the current block. Max(W, H) is a function that outputs the larger of W and H.

[0222] A method of determining the mode k may be as follows.

[0223] Mode k can be used for the transform kernel set for MIP. Mode k can be used to determine the matrix for MIP. Mode k can be determined based on the intra prediction mode (e.g., angle mode) and / or the size of the current block. For example, if the intra prediction mode is an angle mode with an index less than 18 and the width and height of the current block are 4, mode k can be the same as the intra prediction mode. If the intra prediction mode is an angle mode with an index greater than or equal to 18 and the width and height of the current block are 4, mode k can be the angle mode corresponding to the index of the intra prediction mode minus 17. If the intra prediction mode is an angle mode with an index less than 10 and the larger value of the width and height of the current block is 8, mode k can be the same as the intra prediction mode. If the intra prediction mode is an angle mode with an index greater than or equal to 10 and the larger value of the width and height of the current block is 8, mode k can be the angle mode corresponding to the index of the intra prediction mode minus 9. If the intra prediction mode is an angular mode with an index less than 6 and the larger value of the width and height of the current block is greater than 8, mode k may be the same as the intra prediction mode. If the intra prediction mode is an angular mode with an index greater than or equal to 6 and the larger value of the width and height of the current block is greater than 8, mode k may be an angular mode corresponding to the index of the intra prediction mode minus 5. If mode k is organized into a formula, it may be illustrated as Formula 6.

[0224] [Formula 6]

[0225] For W=H=4 and intra prediction mode <18, mode (mode k)

[0226] For W=H=4 and intra prediction mode>=18, mode -17 (mode k)

[0227] For max(W, H) = 8 and intra prediction mode < 10, mode (mode k)

[0228] For max(W,H)=8 and intra prediction mode>=10, mode-9 (mode k)

[0229] For max(W, H)>8 and intra prediction mode<6, mode (mode k)

[0230] For max(W,H)>8 and intra prediction mode>=6, mode-5 (mode k)

[0231] Fig. 22 A method in which a video signal processing apparatus performs matrix-based intra prediction when a current block size is 4×4 is illustrated.

[0232] refer to Fig. 22, if the size of the current block is 4×4, the video signal processing device may generate four new reference samples after the averaging process. For example, the video signal processing device may generate four new reference samples from the four upper reference samples bdry of the current block. top Take the average of the two reference samples to generate two new reference samples bdry top red Similarly, the video signal processing device can obtain the four left reference samples bdry from the current block. left Take the average of the two reference samples to generate two new reference samples bdry left red . Therefore, the video signal processing device can obtain a total of four new reference samples. The video signal processing device can perform a matrix-vector multiplication process based on the four new reference samples. In this case, the video signal processing device can obtain a prediction block of the current block without performing additional linear interpolation (e.g., single-step linear interpolation). If the size of the current block is 4×4, the number of samples of the prediction block of the current block may be 16, which may be the same as the number of samples used for the matrix-vector multiplication process.

[0233] Fig.23 is a diagram illustrating intra-frame template matching according to an embodiment of the present disclosure.

[0234] When performing intra-frame template matching, the video signal processing device can find a template with high similarity or lowest cost with the template of the current coding / prediction block in a predetermined search area within the current frame / slice, and use the block corresponding to the found template as the prediction block of the current block. Fig.23 , the predetermined search area may be four areas R1, R2, R3, R4 including the current CTU. R1 may be the CTU including the current coding / prediction block, and may be the CTU adjacent to R2, R3, R4. The size of the CTU may be 32, 64, 128, or 256, and may be a square. Fig.23 ,template( Fig.23 2301 and 2302) may be configured in an L shape, and the size of the template may be 4. However, the size is not limited to 4. The video signal processing device may find a template with the lowest cost within the determined search area using a sum of absolute transform differences (SATD) method. In addition, the video signal processing device may use Hadamard transform for intra-frame template matching. In the present disclosure, for ease of description, a predetermined search area, CTU size, shape, template shape, and size have been illustrated, but are not limited thereto.

[0235] Fig.24 is a diagram illustrating an encoding / decoding process of a signal related to a DIMD method according to an embodiment of the present disclosure.

[0236] To perform prediction using the DIMD method, the video signal processing device may derive Figures 16 to 19 In the DIMD prediction method, the first mode and the second mode can be Figures 16 to 19 The method described in is derived. The video signal processing device can generate a final prediction block of the current block by superimposing ModeA, ModeB and plane mode. ModeA and ModeB can be described as DIMD mode. If ModeA and ModeB are angle modes, ModeA and ModeB can be transformed into wide-angle modes based on the size of the current block (e.g., coding block, transform block and prediction block). Based on the transformed wide-angle mode, the video signal processing device can generate a prediction block of the current block. The video signal processing device can perform transform processing on the residual signal of the predicted prediction block by using the DIMD method. In this case, the transform method used for the transform process can be MTS or LFNST. MTS and LFNST are based on intra-frame prediction modes, and the video signal processing device can derive a transform core set for each of MTS and LFNST. Each core set can be composed of multiple core candidates. The intra-frame prediction mode derived by the DIMD method can be a mode derived based on the surrounding sample information of the current block, and can be an angle mode 0 to 67. Therefore, for the intra-frame prediction mode derived by the DIMD mode, the transform process to the extended angle mode can be omitted. The video signal processing device may generate a prediction block of the current block for the intra prediction mode derived through the DIMD mode without transforming to the extended angle mode. In addition, when deriving a transform kernel set of MTS or LFNST, the video signal processing device may also generate an MTS or LFNST transform kernel set for the intra prediction mode derived through the DIMD mode without transforming to the extended angle mode.

[0237] Fig.25 is a diagram illustrating an encoding / decoding process of a signal related to a MIP and / or intra-frame TMP method according to an embodiment of the present disclosure.

[0238] The video signal processing device can be based on Figure 20 to Figure 22 The MIP method described in the present invention generates a prediction block for the current block. The video signal processing device can generate a prediction block for the current block based on the MIP method described in the present invention. Fig.23The described intra-frame TMP method generates a prediction block for the current block. The video signal processing device may perform transform processing on the residual signal of the prediction block generated based on the MIP method or the intra-frame TMP method. The transform method used for the transform process may be MTS or LFNST. The video signal processing device may derive a transform core set for each of MTS and LFNST. Each core set may consist of multiple core candidates. When the MIP method and the intra-frame TMP method are used, the intra-frame prediction mode may not exist. Therefore, in the transform process, the video signal processing device may omit MTS or LFNST, or apply a limited transform core set.

[0239] If a prediction block of a current block is generated without using an intra prediction mode that is a MIP method and an intra TMP method, the video signal processing device may perform a process of deriving a DIMD mode to apply MTS or LFNST to the block to which the MIP method or the TMP method is applied. The video signal processing device may derive a transform core set of MTS or LFNST based on the derived DIMD mode. In order to derive a transform core set of MTS or LFNST, the video signal processing device may perform a process of transforming to an extended angle mode. The intra prediction mode derived by the DIMD method may be a mode derived based on surrounding sample information of the current block, and may be an angle mode 0 to 67. Therefore, for the intra prediction mode derived by the DIMD mode, the transform process to the extended angle mode may be omitted.

[0240] During the decoding process, the video signal processing device may perform MTS or LFNST transformation on the signal that has been inverse quantized. The video signal processing device may derive a DIMD mode before performing MTS or LFNST transformation on a block to which the MIP method or the intra-frame TMP method is applied, and may perform MTS or LFNST transformation based on the intra-frame prediction mode derived in the DIMD mode. In this case, the transformation process to the extended angle mode may be omitted.

[0241] The video signal processing device may perform MTS or LFNST transformation on a signal that has been inversely quantized. The video signal processing device may derive a DIMD mode before performing MTS or LFNST transformation on a block to which the DNID method is applied, and may perform MTS or LFNST transformation based on the intra-frame prediction mode derived in the DIMD mode. In this case, the transformation process to the extended angle mode may be omitted.

[0242] The operations related to the MTS or LFNST transform according to the above disclosure can also be applied to a non-separable primary transform (NSPT) other than LFNST. For example, the video signal processing device can derive a transform kernel set for the NSPT based on the intra prediction mode derived from the DIMD mode. In addition, the video signal processing device can perform the NSPT transform based on the transform kernel set for the NSPT.

[0243] Fig.26 is a diagram illustrating a relationship between an input vector of auxiliary transformation and an intra prediction mode according to an embodiment of the present disclosure.

[0244] The auxiliary transform may be calculated by multiplying the auxiliary transform kernel matrix with the input vector. The video signal processing device may configure the coefficients in the upper left subblock of the coefficient block of the main transform in the form of a vector. The vector may be configured depending on the intra-frame prediction mode. For example, if the intra-frame prediction mode is corresponding to less than or equal to Figure 6 If the intra prediction mode is an INTRA_LT_CCLM, INTRA_T_CCLM, or INTRA_L_CCLM mode that predicts chroma samples by a linear relationship of chroma, the video signal processing device may horizontally scan the upper left sub-block of the coefficient block of the main transform to configure the coefficients in vector form. The elements of the I-th row and the J-th column of the upper left n×n block of the coefficient block of the main transform may be described as x_ij. In this case, the vectorized coefficients may be expressed as [x_00, x_01, ..., x_0n-1, x_10, x_11, ..., x_1n-1, ..., x_n-10, x_n-11, ..., x_n-1n-1]. On the other hand, if the intra prediction mode is an INTRA_LT_CCLM, INTRA_T_CCLM, or INTRA_L_CCLM mode that predicts chroma samples by a linear relationship of chroma, the video signal processing device may horizontally scan the upper left sub-block of the coefficient block of the main transform to configure the coefficients in vector form. Figure 6 The video signal processing device may vertically scan the upper left subblock of the coefficient block of the main transform for the prediction mode with an index greater than index 34, and configure the coefficients in vector form. The vectorized coefficients may be expressed as [x_00, x_10, ..., x_n-10, x_01, x_11, ..., x_n-11, ..., x_0n-1, x_1n-1, ..., x_n-1n-1].

[0245] Since the coefficients of the auxiliary transform are in vector form, they can be expressed as two-dimensional data. The coefficients of the auxiliary transform can be assigned to the upper left sub-block of the transform block according to the preconfigured scanning order. The preconfigured scanning order can be an upper right diagonal scanning order.

[0246] Fig. 27 is a diagram illustrating a method of configuring an input vector of an auxiliary transformation according to an embodiment of the present disclosure.

[0247] Fig. 27The method for using the forward main transform coefficients as the input vector for the forward LFNST is shown in FIG. Fig.14 As described in Fig.26 to use the forward main transform coefficients as the input vector for the forward LFNST. Fig. 27 , the ROI of LFNST16 can correspond to six 4×4 sub-blocks (ref. Fig. 27 B and Fig. 27 C) A total of 96 main transform coefficients may be used, and the matrix of the transform kernel of LFNST16 may be 32×96. A total of 96 transform coefficients may be configured in the form of a 96×1 input vector.

[0248] refer to Fig. 27 A, the current block may consist of 16 4×4 sub-blocks, and each sub-block may be respectively mapped to an index of 0 to 15. In this case, the ROI of LFNST16 may be a region corresponding to sub-blocks corresponding to indexes 0, 4, 8, 1, 5, and 2.

[0249] refer to Fig. 27 B. The scanning order in the horizontal (lateral) direction can be used for input vector configuration. That is, the video signal processing device can scan the transformation coefficients in the order of sub-blocks corresponding to indexes 0, 1, 2, 4, 5 and 8. In other words, 12 samples of the first row can be scanned in the horizontal direction of sub-blocks with consecutive indexes 0, 1 and 2, 12 samples of the second row can be scanned, 12 samples of the third row can be scanned, and then 12 samples of the fourth row can be scanned. In addition, 4 samples of each of the fifth to eighth rows of sub-blocks with consecutive indexes 4 and 5 can be scanned in the horizontal direction. In addition, 4 samples of each of the ninth to twelfth rows of the sub-block with index 8 can be scanned in the horizontal direction. If the encoding mode of the current block is an intra-frame prediction mode corresponding to an index greater than an intra-frame prediction mode of index 34, the input vector can be configured according to the following. Fig. 27 The vertical (longitudinal) scanning order shown in C is configured. That is, the video signal processing apparatus can configure the input vector by scanning the transformation coefficients in the order corresponding to 0, 4, 8, 1, 5 and 2 sub-blocks.

[0250] Fig.28 is a diagram illustrating a process of deriving orientation information of a template of a current block for intra template matching according to an embodiment of the present disclosure.

[0251] The process of deriving the orientation information of the template of the current block for intra template matching may be applied to each color component (i.e., luminance component and chrominance (Cb, Cr) component) of the current block. Alternatively, the process of deriving the orientation information of the template of the current block for intra template matching may be applied to one of the chrominance components (Cb or Cr), and the result of the deriving process may be used for the remaining chrominance components. The size of the template of the current block for intra template matching may be 4. The video signal processing apparatus may be configured to generate the directional information of the template of the current block for intra template matching by using Fig.17 and Fig.18 The video signal processing device may apply a Sobel filter to a 3x3 unit template. The video signal processing device may represent the derived intra-frame prediction directional information (mode) in the form of a histogram and sort it in order of high frequency. Alternatively, the video signal processing device may derive the intra-frame prediction directional information by using a template of a matching block found by template matching in a search area instead of using the current block.

[0252] Fig.29 is a diagram illustrating a template form for deriving intra-prediction directional information according to an embodiment of the present disclosure.

[0253] The method described herein for deriving intra-prediction directional information may be described as a DIMD method.

[0254] Fig.29 A illustrates a template located above the current block used to derive intra-prediction directional information. The video signal processing apparatus may derive intra-prediction directional information by using only the template located above the current block. Fig.29 B illustrates a template located on the left side of the current block for deriving intra-frame prediction directional information. The video signal processing apparatus may derive intra-frame prediction directional information by using only the template located on the left side of the current block. Fig.28 and Fig.29The intra-frame prediction directional information derived from the template illustrated in the figure can be used as the intra-frame prediction mode. The video signal processing device can apply MTS to the intra-frame template matching block based on the intra-frame prediction mode. In addition, the video signal processing device can also apply LFNST to the intra-frame template matching block based on the intra-frame prediction mode. The derived intra-frame prediction directional information can be obtained for each color component. That is, the intra-frame prediction directional information can be derived for each brightness and chrominance. The video signal processing device can derive MTS and LFNST core sets for each color component, and can also derive multiple core sets. In this case, the video signal processing device can derive intra-frame prediction directional information for only one of the Cb and Cr components. In the block to which the intra-frame block matching is applied, because there is no intra-frame prediction mode information, MTS and LFNST may not be used, or limited MTS and LFNST may be applied. The intra-frame prediction directional information derived by the video signal processing device may be multiple. The video signal processing device can determine the intra-frame prediction directional information with a larger coding gain by applying MTS or LFNST to multiple intra-frame prediction directional information, and can derive the core set based on the determined intra-frame prediction directional information. In addition, the video signal processing device can send the core candidates in the derived core set with a signal. For example, the encoder can generate a bitstream including the syntax elements mts_idx and / or lfnst_idx indicating the core candidates. The decoder can determine the core candidates by parsing the mts_idx and / or lfnst_idx included in the bitstream. The video signal processing device can derive the MTS or LFNST core set for one of the highest frequency modes. Mts_idx and lfnst_idx can be parsed by color components (e.g., Y, Cb, Cr) or by brightness and chrominance components.

[0255] Fig.30 is a diagram illustrating an MTS set applied to an intra template matching block according to an embodiment of the present disclosure.

[0256] In a typical intra template matching block, intra prediction information may not exist. Therefore, the video signal processing device can configure and use the MTS set and kernel type for each set according to the size of the prediction block of the current block. For example, the block size can be 4x4, 4x8, 4x16, 4x32, 8x4, 8x8, 8x16, 8x32, 16x4, 16x8, 16x16, 16x32, 32x4, 32x8, 32x16, and 32x32. In addition, when the block size is greater than 32, it can be expanded to, for example, a width or height of 64. Reference Fig.30 A, a transform set for intra-frame template matching blocks can be configured for each block size. The transform set can be an additional form of the existing intra-frame mode and block size-based transform set. Fig.30B illustrates some core candidates configured for each transform set. The number of transform core candidates configured for each transform set may be 4 or 6. The transform core candidates may be indicated in the transform core combination of Table 1 above. The video signal processing device may find transform core candidates suitable for intra-frame template block matching through experiments.

[0257] Because the intra template matching method has the characteristics of the inter prediction mode, the MTS method based on the inter prediction mode can be applied to the MTS method to be applied to the intra template matching block. For example, the encoder can generate a bitstream including index information about the best transform set for the block to which the intra template matching is applied. In this case, the best transform set can be one of {(DST7, DST7), (DST7, DCT8), (DCT8, DST7), (DCT8, DCT8)}. The decoder can determine the transform set for the current block based on the best transform set determined by parsing the index information included in the bitstream.

[0258] Fig.31 and Fig.32 is a diagram illustrating a syntax structure including a flag indicating whether intra template matching is applied according to an embodiment of the present disclosure.

[0259] refer to Fig.31 , a flag (syntax element) indicating whether intra template matching is applied to the current block may be included in the coding unit syntax structure. The flag indicating the intra prediction method may be signaled and / or parsed in the order of indicating DIMD, BDPCM, intra TMP, MIP, TIMD, MRL, ISP, and MPM. First, the video signal processing device may parse and / or signal a flag (cu_dimd_flag) indicating whether DIMD is applied. When the value of cu_dimd_flag is 0 (when DIMD is not applied), the video signal processing device may parse and / or signal a flag (intra_bdpcm_luma_flag) indicating whether BDPCM is applied to the luma component. When the values ​​of both cu_dimd_flag and intra_bdpcm_luma_flag are 0 (DIMD is not applied and BDPCM is not applied), the video signal processing device may parse and / or signal a flag (intra_tmp_flag) indicating whether intra TMP is applied. When the values ​​of cu_dimd_flag, intra_bdpcm_luma_flag, and intra_tmp_flag are all 0, the flags following intra_mip_flag may be parsed and / or signaled.

[0260] refer to Fig.32, a flag indicating an intra prediction method for the current block may be signaled and / or parsed in the order of TIMD, BDPCM, intra TMP, MIP, DIMD, MRL, ISP, and MPM. The video signal processing device may parse and / or signal a flag (cu_timd_flag) indicating whether TIMD is applied. When the value of cu_timd_flag is 0 (TIMD is not applied), the video signal processing device may parse and / or signal intra_bdpcm_luma_flag. When the values ​​of both cu_timd_flag and intra_bdpcm_luma_flag are 0, the video signal processing device may parse and / or signal intra_tmp_flag. When the values ​​of cu_timd_flag, intra_bdpcm_luma_flag, and intra_tmp_flag are all 0, the flag following intra_mip_flag may be parsed and / or signaled. The video signal processing device can signal and / or parse the syntax element (intra_luma_ref_idx) indicating whether multiple reference lines (MRL) are applied only when the value of the flag (intra_dimd_flag) indicating whether DIMD is applied to the current block (e.g., prediction block) is 0 (when DIMD is not applied).

[0261] Fig.33 is a diagram illustrating a syntax structure showing a method of parsing a syntax element indicating whether LFNST is applied.

[0262] refer to Fig.33 , when IntraTmp is applied (when the value of IntraTmpFlag[x0][y0] is not 0), the syntax element (lfnst_idx) indicating whether LFNST is applied to the current block can be parsed ( Fig.333301 in). The variable IntraTmpFlag[x][y] can be configured as the value of intra_tmp_flag. In IntraTmpFlag[x][y], x can be x0...x0+cbWidth-1, and y can be y0...y0+cbHeight-1. In the coding block where intra template matching is applied, the impact of LFNST may be relatively small. It can be determined not only for the luminance component block but also for the chrominance component block whether LFNST is applied. In this case, when determining whether LFNST is applied to the chrominance component block, Cb and Cr can be determined jointly or individually. When indicating whether LFNST is applied to the chrominance component block, a variable (channel type variable) indicating the color component can be additionally included. For example, IntraTmpFlag[x][y] can be expressed in the form of IntraTmpFlag[channel type variable][x][y], and lfnst_idx can be expressed in the form of lfnst_idx[channel type variable]. Additionally, a syntax element indicating whether LFNST is applied may be parsed based on a block size for each color component.

[0263] Fig.34 is a diagram illustrating intra-propagation of an intra-template matching block according to an embodiment of the present disclosure.

[0264] Because the block to which the intra-frame template block matching is applied does not have intra-frame prediction mode information, one of the preconfigured intra-frame prediction modes can be stored in the intra-frame prediction mode map. The preconfigured intra-frame prediction modes can be a plane mode, a DC mode, and an angle mode. The intra-frame prediction mode can be applied to the intra-frame prediction mode map in units of 4×4. When the video signal processing device configures the MPM list of the current prediction block, the intra-frame prediction mode stored in the intra-frame prediction mode map can be used. The video signal processing device can store the intra-frame prediction mode mapping information of the template matching block in the intra-frame prediction mode map of the current block. If the position of the template matching block and the intra-frame prediction mode map do not match, the video signal processing device can store the intra-frame prediction mode mapping information of the position where the preconfigured position of the template matching block of the 4×4 unit is included in the intra-frame prediction mode map of the current block. The preconfigured position can be one of the corners of the 4×4 unit block or the center of the block.

[0265] Fig.35 is a diagram illustrating a method of applying a hash key according to a method of searching for an intra-template matching block according to an embodiment of the present disclosure.

[0266] The video signal processing device can search for a template matching block based on a hash key. The video signal processing device can perform a hash key (32-bit CRC (cyclic redundancy check)) match between the template of the current block and the template of the template matching blocks of all template sizes used to apply the intra-frame template matching block. The hash key can be calculated in units of 4×4 blocks (sub-blocks). For hash key matching of template blocks larger than 4×4, the video signal processing device can identify whether the hash key of the template of the current block matches the hash key of the template of the template matching block. Specifically, the video signal processing device can identify whether the hash key of the template of the current block matches all 4×4 blocks (sub-blocks, Fig.35 The video signal processing device may calculate the cost of the template of the template matching block in which the template of the current block matches the hash key, and determine the block corresponding to the template corresponding to the lowest cost as the prediction block of the current block. That is, the video signal processing device may calculate the similarity (cost) between the template of one or more template matching blocks in which the template of the current block matches the hash key and the template of the current block, so as to determine the block corresponding to the template with the highest similarity (lowest cost) as the prediction block of the current block. The video signal processing device may Fig.23 The search is performed in 4×4 units within the search segment.

[0267] Fig.36 is a diagram illustrating pre-configured positions for searching for intra-frame template matching blocks according to an embodiment of the present disclosure.

[0268] The search area for intra-frame template matching block search may be 4CTU in size. The video signal processing device may calculate a preconfigured position of the search area (e.g., Fig.36 The cost between the template of the x part of the current block and the template of the current block is determined to determine the block at the position corresponding to the minimum cost as the matching block. The preconfigured positions may be determined equally or unequally. For example, if the preconfigured positions are determined equally, they may be determined in multiples of 2 or 4, and if they are determined unequally, the preconfigured positions may be determined. Reference Fig.36 , the preconfigured location included in R1 corresponding to the current CTU may be a reconstruction area.

[0269] Fig.37 is a diagram illustrating a compilation unit syntax structure according to an embodiment of the present disclosure.

[0270] Fig.37 The grammatical elements in the grammatical structure can be combined with Figure 31 to Figure 33 The syntax elements are the same as those shown in the figure. Fig.37, a flag (intra_tmp_flag) indicating whether intra TMP is applied can be parsed and signaled based on the maximum size of the intra template matching block. The maximum size of the intra template matching block can be configured for each slice type. Alternatively, the maximum size of the intra template matching block can be configured for each color component (Y, Cb, Cr), and can be set for each chroma component and luma component. For example, if the slice type is an I-slice, the width (cbWidth) of the current block (ie, the coding block) is less than or equal to the maximum block size (TMPSize) to which template matching can be applied, and the height (cbHeight) of the current block can be less than or equal to the TMP size (ie, cbWidth <= TMPSize, cbHeight <= TMPSize). If the slice type is not an I-slice, the width of the current block is less than or equal to the TMP maximum size (TMP_MaxSize), and the height of the current block can be less than or equal to the TMP maximum size (cbWidth <= TMP_MaxSize, cbHeight <= TMP_MaxSize). TMP_MaxSize can be 64, or one of 16, 32, 128, and 256. TMPSize can be smaller than TMP_MaxSize. If TMP_MaxSize does not exist, TMPSize can be configured as an integer not exceeding the CTU size. In the following, the parsing conditions of Intra_tmp_flag are described.

[0271] (Condition 1)

[0272] (sps_tmp_enable_flag&&!cu_dimd_flag&&((sh_slice_type!=I&&cbWidth<=TMP_MaxSize&&cbHeight<=TMP_MaxSize)||(sh_slice_type==I&&cbWidth<=TMPSize&&cbHeight<=TMPSize)))

[0273] If condition 1 is met, intra_tmp_flag can be parsed. sps_tmp_enable_flag is a syntax element indicating whether intra-frame TMP is enabled and can be signaled at the SPS level. If the value of sps_tmp_enable_flag is 1, it can indicate that intra-frame TMP is enabled, and if the value of sps_tmp_enable_flag is 0, it can indicate that intra-frame TMP is disabled. According to condition 1, i) sps_tmp_enable_flag indicates that intra-frame TMP is enabled, ii) cu_dimd_flag indicates that DIMD is not applied, and iii) if the slice type is I-slice, the width and height of the current block are less than or equal to the TMP size (TMPSize), or if the slice type is not I-slice, the width of the current block is less than or equal to the TMP maximum size (TMP_MaxSize), then intra_tmp_flag can be parsed.

[0274] Depending on the syntax structure, the !cu_dimd_flag in condition 1 may or may not be included.

[0275] (Condition 2)

[0276] (sps_tmp_enable_flag&&!cu_dimd_flag&&!Intra_bdpcm_luma_flag&&((sh_slice_type!=I&&cbWidth<=TM P_MaxSize&&cbHeight<=TMP_MaxSize)||(sh_slice_type==I&&cbWidth<=TMPSize&&cbHeight<=TMPSize)))

[0277] If condition 2 is satisfied, intra_tmp_flag may be parsed. Condition 2 may be an additional condition for intra_bdpcm_luma_flag to condition 1. According to condition 2, in addition to i) to iii) of condition 1, there is also iv) if BDPCM is not applied, intra_tmp_flag may be parsed.

[0278] intra_tmp_flag, TMP_MaxSize and TMPSize can be configured separately for each color component (Y, Cb, Cr) or for each luma and chroma.

[0279] Because the block to which intra-frame template matching is applied has the characteristics of the inter-frame prediction mode, when determining the filtering strength (bS) of the boundary portion in the deblocking filter, the method for determining the filtering strength in the inter-frame prediction mode can be used. For example, in the process of determining the filtering strength, if the intra-frame template matching is applied to any one of the p-block and the q-block, the filtering strength of the boundary between the p-block and the q-block can be configured to 1. The greater the filtering strength, the stronger the filtering, and a filtering strength of 0 can mean that no filtering is performed. For example, a filtering strength of 1 (weak filtering) can mean a weaker filtering than a filtering strength of 2 (strong filtering). Strong filtering can change any number or more pixel values ​​around the boundary between the p-block and the q-block, while weak filtering can change any number or less pixel values. Any number can be an integer of 6. Alternatively, if the intra-frame template matching is applied to both the p-block and the q-block, the filtering strength can be determined based on the difference between the block vectors of the p-block and the q-block. For example, in determining the filtering strength, if intra template matching is applied to both the p block and the q block, and the difference in block vectors between the p block and the q block is greater than an arbitrary predetermined number, the filtering strength of the boundary between the p block and the q block may be configured to be 1. In this case, the arbitrary predetermined number may be an integer of 1.

[0280] Fig.38 is a diagram illustrating a method of selecting a transform set for a block to which intra TMP is applied according to an embodiment of the present disclosure.

[0281] The width and height of the block to which the intra-frame TMP is applied may be limited. For example, the width and height of the block to which the intra-frame TMP is applied may be 64, respectively. The horizontal direction transform core of the block to which the intra-frame TMP is applied may be DCT2, and the vertical direction transform core may be DCT2. If the width of the block to which the intra-frame TMP is applied meets a specific condition, the horizontal direction transform core may be DST7. If the height of the block to which the intra-frame TMP is applied meets a specific condition, the vertical direction transform core may be DST7. On the other hand, if the width of the block to which the intra-frame TMP is applied does not meet a specific condition, the horizontal direction transform core may be DST2, and if the height of the block to which the intra-frame TMP is applied does not meet a specific condition, the vertical direction transform core may be DST2. The specific condition may be that the width (or height) of the block to which the intra-frame TMP is applied is greater than or equal to 4 and less than or equal to 16. The video signal processing device may determine an MTS set for the block to which the intra-frame TMP is applied. The MTS set may be determined using an intra-frame prediction mode derived by the DIMD method. The video signal processing device may calculate costs for the transform core candidates constituting the determined MTS set, and use the transform core candidate corresponding to the minimum cost. Fig.38 The described method can be applied to blocks except for the case where the width (or height) of the block to which the intra TMP is applied is greater than or equal to 4 and less than or equal to 16. That is, by Fig.38The described method may be applied to a block corresponding to a case where the width (or height) of the block to which the intra TMP is applied is greater than or equal to 4 and less than or equal to 16. In this case, the parsing condition of the MTS index indicating the transform core candidate included in the bitstream is as follows.

[0282] (Condition 1)

[0283] Cu.IntraTmpFlag&&Width>1st reference value&&Height>2nd reference value

[0284] If the flag (Cu.IntraTmpFlag) indicating whether the current block (e.g., coding block, transform block) is applied with intra TMP is true (the value is 1, indicating that intra TMP is applied), the width of the current block is greater than the first reference value, and the height of the current block is greater than the second reference value, the MTS index can be parsed. The first reference value and the second reference value of condition 1 can be a preset value of 16. In addition, the first reference value and the second reference value can be configured as different values, and the first reference value and the second reference value can be any one of 4, 8, 16, 32, etc.

[0285] Meanwhile, the video signal processing apparatus may determine an MTS set for all blocks to which the intra-frame TMP is applied, regardless of Fig.32 The specific conditions of the MTS set can be determined by using the intra-frame prediction mode derived by the DIMD method. The video signal processing device can determine the specific transform core to be used based on the cost of the transform core included in the determined MTS set. In this case, the parsing conditions of the MTS index indicating the specific transform core included in the bitstream are as follows.

[0286] (Condition 2)

[0287] Cu.IntraTmpFlag&&width<=third reference value&&height<=fourth reference value

[0288] If Cu.IntraTmpFlag is true and the width of the current block is less than or equal to the third reference value, and the height of the current block is less than or equal to the fourth reference value, the MTS index may be parsed. The third reference value and the fourth reference value may be 64. In addition, the third reference value and the fourth reference value may be equal to the maximum size to which the intra TMP may be applied. In addition, the third reference value and the fourth reference value may be different values. The condition for parsing the LFNST index representing the core for LFNST may also be the same as condition 2.

[0289] Alternatively, the video signal processing device may parse the MTS index based on a specific core set. In this case, intra prediction mode information may not be required. The specific core set may be 4 to 6 core candidates. In this case, the core candidates may be configured as shown in Table 1.

[0290] Fig.39 is a diagram illustrating a plurality of block vectors for an intra TMP block according to an embodiment of the present disclosure.

[0291] The video signal processing device can calculate the cost between the template of the preconfigured position for the intra-frame template matching block search and the template of the current block, and calculate (obtain) the block vector (motion vector) of the template for the intra-frame template matching block search corresponding to the minimum cost. The video signal processing device can sort and store the block vectors in the order of low cost. The cost can be calculated by using the template of the current block and the template of the reference block (the device is a template of the preconfigured position for the intra-frame template matching block search) samples, using the sum of absolute differences (SAD), the sum of mean absolute differences (MR-SAD) and the sum of absolute transformation differences (SATD). The block vector (motion vector) can be expressed in integer pixel units, 1 / 2, 1 / 4, 1 / 8, 2 and 4 pixel units. Among the obtained block vectors, the block vector actually used can be the first (or last) vector, and the remaining block vectors can be sorted and stored in the order of low cost. The stored block vector can be used as a motion vector candidate for an IBC coding block. If a block to which the intra TMP is applied exists among neighboring blocks of the IBC coding block, the video signal processing apparatus may use the stored block vector.

[0292] An intra TMP block may be predicted by a combination of multiple intra TMP blocks. In this case, the video signal processing device may store a block vector of a reference block used for combination. In this case, the combined reference block may be two or more blocks. The video signal processing device may store as many block vectors as the number of combined reference blocks and block vectors obtained by the above-mentioned cost calculation. If only the block vectors of the reference blocks used for combination are used, the video signal processing device may use weight combination. The video signal processing device may determine the storage order of the block vectors based on the weights. When the block vectors stored for the IBC coding block are used, the video signal processing device may determine the order of motion candidates for the IBC coding block based on the storage order of the block vectors. For example, a block vector with a large weight may have a high priority. When the block vector of the reference block to be combined and the block vector obtained by cost calculation are stored together, the block vector of the reference block to be combined may have a high priority.

[0293] Fig.40 is a diagram illustrating a method of deriving an intra prediction mode of a block to which an intra TMP is applied according to an embodiment of the present disclosure.

[0294] Specifically, Fig.40A method for deriving an intra prediction mode required for applying MTS and LFNST to a block to which intra TMP is applied is shown. A video signal processing device may determine an intra prediction mode for a block to which intra TMP is applied based on an intra prediction mode at a preconfigured position stored in a reference block of a current block and an intra prediction mode at a preconfigured position stored in a surrounding (neighboring) block of the reference block. Meanwhile, if a sample at a preconfigured position is in MIP mode, intra TMP mode, IBC mode, palette mode, or intra mode, the video signal processing device may not use the intra prediction mode stored at the preconfigured position. Fig.40 A can be Fig.40 A magnified image of the reference block (matching block) of B. Fig.40 A, there are five preconfigured positions for the video signal processing device to identify the intra-frame prediction mode of the matching block. For example, there may be three preconfigured positions (P1 to P3) within the reference block, and there may be two preconfigured positions (P4 to P5) within the surrounding blocks of the reference block. Specifically, the preconfigured positions may be P1 (center), P2 (lower right), P3 (upper left), P4 (a block adjacent to the center block among the blocks at the upper boundary of the current block among the neighboring blocks of the reference block adjacent to the upper boundary of the reference block), and P5 (a block adjacent to the center block among the blocks at the left boundary of the current block among the neighboring blocks adjacent to the left boundary of the reference block). Fig.40 In A, for ease of description, five preconfigured positions are illustrated, but the present disclosure is not limited thereto. At the same time, there may be no intra-frame prediction mode at the preconfigured position. In this case, the video signal processing device may configure and use the preconfigured mode as the intra-frame prediction mode. The preconfigured mode may be one of a plane mode, a DC mode, a horizontal angle mode, a vertical angle mode, a diagonal mode, and an angle mode. In addition, if there is no intra-frame prediction mode at the preconfigured position, the video signal processing device may configure and use the preconfigured mode as the intra-frame prediction mode. Fig.28 The template shown is used to derive the intra prediction mode. In addition, the video signal processing device can derive the intra prediction mode by using all or part of the samples in the reference block. The video signal processing device can use the derived intra prediction mode to determine the MTS set or LFNST set of the current block to which the intra TMP is applied.

[0295] In addition, video signal processing equipment can use Fig.40 The intra prediction mode of the preconfigured position of the neighboring blocks of the current block shown in C. Fig.40C, the video signal processing device can use the intra-frame prediction modes of five pre-configured positions to determine the MTS set or the LFNST set. For example, if the position of the upper left block of the current block is (0, 0), the pre-configured positions of the neighboring blocks of the current block can be P1 (-1, H-1), P2 (W-1, -1), P3 (-1, H), P4 (W, -1) and P5 (-1, 0). In this case, H can be the height of the current block and W can be the width of the current block. At the same time, if there is no intra-frame prediction mode at the pre-configured position, the video signal processing device can use one of the above-mentioned pre-configured modes or use a template sample located in the vicinity of the current block to derive the intra-frame prediction mode.

[0296] In addition, the video signal processing device may identify whether there is an intra-frame prediction mode stored in a preconfigured position within a reference block of the current block, and an intra-frame prediction mode stored in a preconfigured position within a surrounding block of the reference block, and if there is no stored intra-frame prediction mode, the video signal processing device may identify whether there is an intra-frame prediction mode in a preconfigured position of a neighboring block of the current block. If there is no intra-frame prediction mode in the preconfigured position of the neighboring block of the current block, the video signal processing device may derive the intra-frame prediction mode (using DIMD) by using one of the above-mentioned preconfigured modes or by using a template sample located in the vicinity of the current block (by using DIMD).

[0297] The video signal processing device can scan the video signal in the order of P1, P2, P3, P4 and P5. Fig.40 A and 40C identify the intra prediction mode. The video signal processing apparatus may determine the MTS set or the LFNST set by using the first identified intra prediction mode while performing scanning on P1 to P5.

[0298] refer to Fig.40 The described method may also be applied to a block to which the IBC mode is applied. That is, the video signal processing apparatus may derive an intra prediction mode required to apply the MTS and LFNST to a block to which the IBC mode is applied.

[0299] Fig.41 A method for determining an MTS set or a LFNST set according to an embodiment of the present disclosure is illustrated.

[0300] Fig.41 The diagram shows the intra prediction mode derived by Figures 1 to 40 The method of determining the MTS set or LFNST set by the intra-frame TMP block is described.

[0301] refer to Fig.41, the video signal decoding apparatus may determine a transform core set for transforming the current block to which intra template matching is applied (S4110). The video signal decoding apparatus may predict the current block based on the transform cores included in the transform core set (S4120). The transform core set may be determined based on an intra prediction mode associated with the current block.

[0302] The transform kernel set may be at least one of a set of transform matrices of a multiple transform set (MTS), a set of transform matrices of a low frequency non-separable transform (LFNST), and / or a set of different types of transform matrices of a non-separable main transform.

[0303] The intra prediction modes may be derived based on decoder-side intra mode derivation (DIMD).

[0304] The intra prediction mode may be derived based on the intra prediction mode of a preconfigured position within a reference block of the current block and the intra prediction modes of neighboring blocks of the reference block.

[0305] The preconfigured positions within the reference block may be the upper left, lower right, and center of the current block, and the neighboring blocks of the reference block may be blocks adjacent to the center block of the blocks at the upper boundary of the current block among the neighboring blocks of the reference block adjacent to the upper boundary of the reference block, and blocks adjacent to the center block of the blocks at the left boundary of the current block among the neighboring blocks adjacent to the left boundary of the reference block.

[0306] The intra prediction mode may be derived based on intra prediction modes of neighboring blocks of the current block.

[0307] The neighboring blocks of the current block may be blocks located at (-1, H-1), (W-1, -1), (-1, H), (W, -1), (-1, 0), and H may be the height of the current block, W may be the width of the current block, and the position of the upper left block of the current block may be (0, 0).

[0308] When the intra prediction mode of the preconfigured position in the reference block of the current block and the intra prediction mode of the neighboring block of the reference block do not exist, the intra prediction mode can be derived based on the decoder side intra mode derivation (DIMD). The above method described in this specification can be performed by a processor in a decoder or an encoder. In addition, the encoder can generate a bit stream decoded by the video signal processing method. In addition, the bit stream generated by the encoder can be stored in a computer-readable non-transient storage medium (recording medium).

[0309] The present specification has been described mainly from the perspective of a decoder, but it may also work in an encoder. The term "parsing" in the present specification has been described in terms of a process of obtaining information from a bitstream, but in terms of an encoder, it may be interpreted as configuring information in a bitstream. Therefore, the term "parsing" is not limited to the operation of a decoder, but may also be interpreted as the act of configuring a bitstream in an encoder. In addition, the bitstream may be configured to be stored in a computer-readable recording medium.

[0310] The above-mentioned embodiments of the present invention may be implemented by various means. For example, the embodiments of the present invention may be implemented by hardware, firmware, software or a combination thereof.

[0311] For the case of implementation through hardware, the method according to the embodiment of the present invention can be implemented by one or more of an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a processor, a controller, a microcontroller, a microprocessor, etc.

[0312] In the case of implementation by firmware or software, the method according to the embodiment of the present invention can be implemented in the form of a module, process or function that performs the above functions or operations. The software code can be stored in a memory and driven by a processor. The memory can be located inside or outside the processor and can exchange data with the processor in various known ways.

[0313] Certain embodiments may also be implemented in the form of a recording medium including computer executable instructions, such as a program module executed by a computer. Computer readable media may be any available media that can be accessed by a computer, and may include all volatile, non-volatile, removable and non-removable media. In addition, computer readable media may include computer storage media and communication media. Computer storage media include all volatile, non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer readable instructions, data structures, program modules or other data. Typically, communication media include computer readable instructions, other data (such as data structures or program modules) or other transmission mechanisms of modulated data signals, and include any information transmission media.

[0314] The above description of the present invention is for illustrative purposes only, and it will be understood that those skilled in the art to which the present invention belongs can change the present invention without changing the technical ideas or basic features of the present invention, and the present invention can be easily modified in other specific forms. Therefore, the above embodiments are illustrative and are not limited in all aspects. For example, each component described as a single entity can be distributed and implemented, and similarly, the components described as distributed can also be implemented in an associated manner.

[0315] The scope of the present invention is defined by the appended claims rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the appended claims and their equivalents should be construed as being included in the scope of the present invention.

Claims

1. A video signal decoding device, comprising a processor, in, The processor is configured to: determining a transform kernel set for a transform of a current block to which intra template matching is applied; and predicting the current block based on the transform kernels included in the transform kernel set, and The transformation kernel set is determined based on an intra-frame prediction mode related to the current block.

2. The video signal decoding device according to claim 1, in, The transform kernel set is at least one of a set of transform matrices of a multiple transform set (MTS), a set of transform matrices of a low frequency non-separable transform (LFNST), and / or a set of transform matrices of a non-separable primary transform.

3. The video signal decoding device according to claim 1, in, The intra prediction modes are derived based on decoder side intra mode derivation (DIMD).

4. The video signal decoding device according to claim 1, in, The intra prediction mode is derived based on an intra prediction mode of a preconfigured position within a reference block of the current block and an intra prediction mode of a neighboring block of the reference block.

5. The video signal decoding device according to claim 4, in, The preconfigured positions within the reference block are the top left, bottom right and center of the current block, and The neighboring blocks of the reference block are blocks adjacent to the center block of the blocks at the upper boundary of the current block among the neighboring blocks of the reference block adjacent to the upper boundary of the reference block, and blocks adjacent to the center block of the blocks at the left boundary of the current block among the neighboring blocks adjacent to the left boundary of the reference block.

6. The video signal decoding device according to claim 1, in, The intra prediction mode is derived based on intra prediction modes of neighboring blocks of the current block.

7. The video signal decoding device according to claim 6, in, The neighboring blocks of the current block are blocks located at (-1, H-1), (W-1, -1), (-1, H), (W, -1), (-1, 0), and Here, H is the height of the current block, W is the width of the current block, and the position of the upper left block of the current block is (0, 0).

8. The video signal decoding device according to claim 4, in, When the intra-frame prediction mode of the pre-configured position within the reference block of the current block and the intra-frame prediction mode of the neighboring block of the reference block do not exist, the intra-frame prediction mode is derived based on the decoder side intra-frame mode derivation [Xia Kai 1] (DIMD).

9. A video signal encoding device, comprising a processor, in, The processor is configured to obtain a bit stream decoded by a decoding method, Wherein, the decoding method comprises: determining a transform kernel set for a transform of a current block to which intra template matching is applied; and predicting the current block based on the transform kernels included in the transform kernel set, and The transform kernel set is determined based on an intra prediction mode associated with the current block.

10. The video signal encoding device according to claim 9, in, The transform kernel set is at least one of a set of transform matrices of a multiple transform set (MTS), a set of transform matrices of a low frequency non-separable transform (LFNST), and / or a set of transform matrices of a non-separable primary transform.

11. The video signal encoding device according to claim 9, in, The intra prediction modes are derived based on decoder side intra mode derivation (DIMD).

12. The video signal encoding device according to claim 9, in, The intra prediction mode is derived based on an intra prediction mode of a preconfigured position within a reference block of the current block and an intra prediction mode of a neighboring block of the reference block.

13. The video signal encoding device according to claim 12, in, The preconfigured positions within the reference block are the top left, bottom right and center of the current block, and The neighboring blocks of the reference block are blocks adjacent to the center block of the blocks at the upper boundary of the current block among the neighboring blocks of the reference block adjacent to the upper boundary of the reference block, and blocks adjacent to the center block of the blocks at the left boundary of the current block among the neighboring blocks adjacent to the left boundary of the reference block.

14. The video signal encoding device according to claim 9, in, The intra prediction mode is derived based on intra prediction modes of neighboring blocks of the current block.

15. The video signal encoding device according to claim 14, in, The neighboring blocks of the current block are blocks located at (-1, H-1), (W-1, -1), (-1, H), (W, -1), (-1, 0), and Here, H is the height of the current block, W is the width of the current block, and the position of the upper left block of the current block is (0, 0).

16. The video signal encoding device according to claim 12, in, When the intra prediction mode of a preconfigured position within a reference block of the current block and the intra prediction mode of a neighboring block of the reference block do not exist, the intra prediction mode is derived based on decoder-side intra mode derivation (DIMD).

17. A computer-readable non-transitory storage medium configured to store a bitstream, in, Decoding the bit stream by a decoding method, Wherein, the decoding method comprises: determining a transform kernel set for a transform of a current block to which intra template matching is applied; and predicting the current block based on the transform kernels included in the transform kernel set, and The transformation kernel set is determined based on an intra-frame prediction mode related to the current block.

18. The non-transitory storage medium according to claim 17, in, The transform kernel set is at least one of a set of transform matrices of a multiple transform set (MTS), a set of transform matrices of a low frequency non-separable transform (LFNST), and / or a set of transform matrices of a non-separable primary transform.

19. The non-transitory storage medium according to claim 17, in, The intra prediction modes are derived based on decoder side intra mode derivation (DIMD).

20. The non-transitory storage medium according to claim 17, in, The intra prediction mode is derived based on an intra prediction mode of a preconfigured position within a reference block of the current block and an intra prediction mode of a neighboring block of the reference block.

Images