Method and apparatus for processing video signal by using sub-block based motion compensation

Abstract

A method and apparatus for processing a video signal by using a sub-block based motion compensation are disclosed. Specifically, the video signal processing method can include the steps of obtaining a first syntax element indicating whether a sub-block based merge mode is applied to a current block; and when the first syntax element indicates that the sub-block based merge mode is not applied to the current block, obtaining a second syntax element indicating whether a merge mode using a motion vector difference is applied to the current block; when the second syntax element indicates that the merge mode using the motion vector difference is applied to the current block, obtaining a third syntax element indicating a candidate used in inter prediction of the current block among candidates included in a merge candidate list for the current block; obtaining information related to the motion vector difference; deriving a motion vector of the current block by adding the motion vector difference to a motion vector of the candidate indicated by the third syntax element; and generating a prediction block of the current block by using the motion vector of the current block.

Description

[0001] This application is a divisional application of patent application No. 201980069820.7 (PCT / KR2019 / 014016), filed with the China Patent Office on April 22, 2021, with an international application date of October 23, 2019, entitled "Method and apparatus for processing video signals by using sub-block-based motion compensation". Technical Field

[0002] This disclosure relates to a method and apparatus for processing video signals, and more specifically, to a video signal processing method and apparatus for encoding and decoding video signals. Background Technology

[0003] Compression encoding refers to a series of signal processing techniques used to transmit digitized information over communication lines or to store information in a form suitable for storage media. The objects of compression encoding include objects such as speech, video, and text, and in particular, techniques used to perform compression encoding on images are called video compression. Compression encoding of video signals is performed by removing excess information, taking into account spatial, temporal, and random correlations. However, with the recent development of various media and data transmission media, there is a need for more efficient video signal processing methods and devices. Summary of the Invention

[0004] Technical issues

[0005] One aspect of this invention is to increase the compilation efficiency of video signals. Another aspect is to provide a method for effectively performing sub-block-based motion compensation. Furthermore, a motion compensation method based on a merging pattern utilizing motion vector difference (MVD) is provided.

[0006] Technical solutions

[0007] To address this problem, the present invention provides a video signal processing apparatus and a video signal processing method.

[0008] According to an embodiment of the present invention, a method for processing a video signal includes: acquiring a first syntax element, the first syntax element indicating whether a sub-block-based merging mode is applied to a current block; when the first syntax element indicates that the sub-block-based merging mode is not applied to the current block, acquiring a second syntax element, the second syntax element indicating whether a motion vector difference-based merging mode is applied to the current block; when the second syntax element indicates that the motion vector difference-based merging mode is applied to the current block, acquiring a third syntax element, the third syntax element indicating candidates included in the merging candidate list of the current block for use in inter-frame prediction of the current block; acquiring information related to the motion vector difference; deriving a motion vector of the current block by adding the motion vector difference to the motion vector of the candidate indicated by the third syntax element; and generating a prediction block of the current block using the motion vector of the current block.

[0009] As an example, the third syntax element may indicate a candidate from the first and second candidates in the merge candidate list of the current block that is used for inter-frame prediction of the current block.

[0010] As an example, when the maximum number of merge candidates is greater than 1, the third syntax element can be parsed from the bitstream, and when the maximum number of merge candidates is not greater than 1, the third syntax element can be inferred to be 0.

[0011] As an example, obtaining information related to the motion vector difference may include: obtaining a fourth syntax element indicating the distance of the motion vector difference; and obtaining a fifth syntax element indicating the direction of the motion vector difference.

[0012] As an embodiment, the method may further include: when the first syntax element indicates that the sub-block-based merging mode is applied to the current block, obtaining a sixth syntax element, the sixth syntax element indicating candidates included in the sub-block merging candidate list of the current block for inter-frame prediction of the current block, wherein, when the first syntax element indicates that the sub-block-based merging mode is applied to the current block, the motion vector of the current block is derived based on the motion vector of the candidate indicated by the sixth syntax element, on a unit basis of at least one sub-block included in the current block.

[0013] As an example, when the first syntax element indicates that the sub-block-based merging pattern is applied to the current block, it can be inferred that the second syntax element is 0.

[0014] As an example, when the maximum number of sub-block merging candidates is greater than 1, the sixth syntax element can be parsed from the bitstream, and when the maximum number of sub-block merging candidates is not greater than 1, the sixth syntax element can be inferred to be 0.

[0015] According to an embodiment of the present invention, an apparatus for processing video signals includes: a processor, wherein the processor is configured to: acquire a first syntax element, the first syntax element indicating whether a sub-block-based merging mode is applied to a current block; when the first syntax element indicates that the sub-block-based merging mode is not applied to the current block; acquire a second syntax element, the second syntax element indicating whether a motion vector difference-based merging mode is applied to the current block; when the second syntax element indicates that the motion vector difference-based merging mode is applied to the current block; acquire a third syntax element, the third syntax element indicating candidates included in the merging candidate list of the current block for use in inter-frame prediction of the current block; acquire information related to the motion vector difference; derive a motion vector of the current block by adding the motion vector difference to the motion vectors of the candidates indicated by the third syntax element; and generate a prediction block of the current block using the motion vector of the current block.

[0016] As an example, the third syntax element may indicate a candidate from the first and second candidates in the merge candidate list of the current block that is used for inter-frame prediction of the current block.

[0017] As an example, when the maximum number of merge candidates is greater than 1, the third syntax element can be parsed from the bitstream, and when the maximum number of merge candidates is not greater than 1, the third syntax element can be inferred to be 0.

[0018] As an example, the processor may be configured to: acquire a fourth syntax element indicating the distance of the motion vector difference; and acquire a fifth syntax element indicating the direction of the motion vector difference.

[0019] As an embodiment, the processor may be configured to: when the first syntax element indicates that the sub-block-based merging mode is applied to the current block, obtain a sixth syntax element, the sixth syntax element indicating candidates included in the sub-block merging candidate list of the current block for inter-frame prediction of the current block, wherein, when the first syntax element indicates that the sub-block-based merging mode is applied to the current block, derive the motion vector of the current block based on the motion vector of the candidate indicated by the sixth syntax element, on a unit basis of at least one sub-block included in the current block.

[0020] As an example, when the first syntax element indicates that the sub-block-based merging pattern is applied to the current block, it can be inferred that the second syntax element is 0.

[0021] As an example, when the maximum number of sub-block merging candidates is greater than 1, the sixth syntax element can be parsed from the bitstream, and when the maximum number of sub-block merging candidates is not greater than 1, the sixth syntax element can be inferred to be 0.

[0022] According to an embodiment of the present invention, a method for processing a video signal includes: encoding a first syntax element indicating whether a sub-block-based merging mode is applied to a current block; when the first syntax element indicates that the sub-block-based merging mode is not applied to the current block, encoding a second syntax element indicating whether a motion vector difference-based merging mode is applied to the current block; when the second syntax element indicates that the motion vector difference-based merging mode is applied to the current block, encoding a third syntax element indicating candidates for inter-frame prediction of the current block among candidates included in a merging candidate list; encoding information related to the motion vector difference; deriving a motion vector of the current block by adding the motion vector difference to the motion vector of the candidate indicated by the third syntax element; and generating a prediction block of the current block using the motion vector of the current block.

[0023] Beneficial effects

[0024] According to embodiments of the present invention, the compilation efficiency of video signals can be improved. In particular, according to embodiments of the present invention, signaling overhead can be reduced and compression efficiency improved by controlling the syntax parsing order. Attached Figure Description

[0025] Figure 1 This is a block diagram schematically illustrating an apparatus for encoding video signals according to an embodiment of the present invention; Figure 2 This is a schematic block diagram illustrating an apparatus for decoding video signals according to an embodiment of the present invention; Figure 3 The illustration shows an example of dividing the compiler tree unit into compiler units within an image; Figure 4 The illustration shows an embodiment of a method for signaling the partitioning of quadtrees and polytrees; Figure 5 The illustration shows inter-frame prediction according to an embodiment of the present invention; Figure 6 The illustration shows a method for signaling motion vectors according to an embodiment of the present invention; Figure 7The illustration shows a method for transmitting adaptive motion vector resolution information by signaling according to an embodiment of the present invention; Figure 8 The illustration shows affine motion compensation according to an embodiment of the present invention; Figure 9 Figure 4 illustrates an embodiment of the parametric affine motion compensation method; Figure 10 The figure illustrates an affine motion prediction method according to an embodiment of the present invention; Figure 11 The illustration shows an embodiment of the affine motion compensation method based on sub-blocks; Figure 12 and 13 The illustration shows an embodiment of the present invention for obtaining the set of control point motion vectors used to predict the current block; Figure 14 The figure illustrates an affine motion compensation method according to an embodiment of the present invention; Figure 15 The figure illustrates an affine motion compensation method according to an embodiment of the present invention; Figure 16 The illustration shows a sub-block-based timed MVP (SbTMVP) according to an embodiment of the present invention. Figure 17 The figure illustrates a planar MVP according to an embodiment of the present invention; Figure 18 The illustration shows a method for determining a temporary motion vector according to an embodiment of the present invention; Figure 19 The illustration shows whether SbTMVP is used according to an embodiment of the present invention; Figure 20 The diagram illustrates a method for deriving SbTMVP according to an embodiment of the present invention; Figure 21 The illustration shows an MMVD application method according to an embodiment of the present invention; Figure 22 The diagram illustrates the syntax structure related to MMVD according to an embodiment of the present invention; Figure 23 The diagram illustrates the syntax structure related to MMVD according to an embodiment of the present invention; Figure 24 The diagram illustrates the syntax structure related to MMVD according to an embodiment of the present invention; Figure 25 The diagram illustrates the MMVD syntax according to an embodiment of the present invention; Figure 26 The diagram illustrates the MMVD syntax according to an embodiment of the present invention; Figure 27 The diagram illustrates the MMVD syntax according to an embodiment of the present invention; Figure 28The diagram illustrates the MMVD syntax according to an embodiment of the present invention; Figure 29 The diagram illustrates the syntax structure of a compilation unit according to an embodiment of the present invention; Figure 30 The diagram illustrates the merged data syntax structure according to an embodiment of the present invention; Figure 31 The diagram illustrates the merged data syntax structure according to an embodiment of the present invention; Figure 32 The diagram illustrates a merged data syntax structure according to an embodiment of the present invention; and Figure 33 The illustration shows a method for processing video signals according to an embodiment of the present invention. Detailed Implementation

[0026] Considering the functions of this disclosure, the terminology used in this specification may be currently widely used and common, but may change according to the intent, practice, or emergence of new technologies of those skilled in the art. Additionally, in some cases, terms may be arbitrarily chosen by the applicant, and in such cases, their meanings are described in the corresponding descriptive sections of this disclosure. Therefore, the terminology used in this specification should be interpreted based on its substantive meaning throughout the specification.

[0027] In this specification, some terms may be interpreted as follows. In some cases, encoding can be interpreted as either encoding or decoding. In this specification, an apparatus for generating a video signal bitstream by performing encoding (encoding) of a video signal is called an encoding apparatus or encoder, and an apparatus for performing decoding (decoding) of a video signal bitstream to reconstruct a video signal is called a decoding apparatus or decoder. Additionally, in this specification, video signal processing apparatus is used as a term encompassing both the concepts of encoder and decoder. Information is a term that includes all values, parameters, coefficients, elements, etc. In some cases, the meaning is interpreted differently, therefore this disclosure is not limited thereto. "Unit" is used to refer to the basic unit of image processing or a specific location in an image, and refers to an image region including both luminance and chrominance components. Additionally, "block" refers to an image region including a specific component among the luminance and chrominance components (i.e., Cb and Cr). However, depending on the embodiment, terms such as "unit," "block," "partition," and "region" may be used interchangeably. Furthermore, in this specification, "unit" can be used as a concept encompassing all of the compilation unit, prediction unit, and transformation unit. The image refers to a field or frame, and according to embodiments, these terms may be used interchangeably.

[0028] Figure 1 This is a schematic block diagram of a video signal encoding apparatus according to an embodiment of the present disclosure. Reference Figure 1The encoding device 100 disclosed herein includes a transformation unit 110, a quantization unit 115, an inverse quantization unit 120, an inverse transformation unit 125, a filtering unit 130, a prediction unit 150, and an entropy compilation unit 160.

[0029] Transform unit 110 obtains the values ​​of transform coefficients by transforming the residual signal, which is the difference between the input video signal and the prediction signal generated by prediction unit 150. For example, discrete cosine transform (DCT), discrete sine transform (DST), or wavelet transform can be used. DCT and DST perform the transform by dividing the input image signal into multiple blocks. During the transform, the coding efficiency can vary depending on the distribution and characteristics of the values ​​in the transform region. Quantization unit 115 quantizes the values ​​of the transform coefficients output from transform unit 110.

[0030] To improve coding efficiency, instead of encoding the image signal as is, a method is used that predicts the image using the region already encoded by prediction unit 150, and obtains the reconstructed image by adding the residual value between the original image and the predicted image to the predicted image. To prevent mismatches in the encoder and decoder, information that can be used in the decoder should be used when performing prediction in the encoder. For this purpose, the encoder performs the processing of the current block of reconstruction encoding again. Inverse quantization unit 120 inverse quantizes the values ​​of the transform coefficients, and inverse transform unit 125 uses the inverse quantized transform coefficient values ​​to reconstruct the residual values. Simultaneously, filtering unit 130 performs filtering operations to improve the quality of the reconstructed image and improve coding efficiency. For example, it may include a deblocking filter, sample adaptive offset (SAO), and adaptive loop filter. The filtered image is output or stored in decoded image buffer (DPB) 156 for use as a reference image.

[0031] To improve compilation efficiency, instead of compiling the image signal as is, a method is used to predict the image via prediction unit 150 by using a compiled region and adding the residual value between the original image and the predicted image to the predicted image, thereby obtaining a reconstructed image. Intra-frame prediction unit 152 performs intra-frame prediction within the current image, and inter-frame prediction unit 154 predicts the current image using a reference image stored in the decoded image buffer 156. Intra-frame prediction unit 152 performs intra-frame prediction from the reconstructed region in the current image and transmits intra-frame compilation information to entropy compilation unit 160. Inter-frame prediction unit 154 may include motion estimation unit 154a and motion compensation unit 154b. Motion estimation unit 154a obtains the motion vector value of the current region by referencing a specific reconstructed region. Motion estimation unit 154a transmits the position information of the reference region (reference frame, motion vector, etc.) to entropy compilation unit 160 so that the position information is included in the bitstream. Motion compensation unit 154b performs inter-frame motion compensation using the motion vector value transmitted from motion estimation unit 154a.

[0032] Prediction unit 150 includes intra-frame prediction unit 152 and inter-frame prediction unit 154. Intra-frame prediction unit 152 performs intra-frame prediction within the current image, and inter-frame prediction unit 154 performs inter-frame prediction to predict the current image using a reference image stored in DBP 156. Intra-frame prediction unit 152 performs intra-frame prediction from reconstructed samples in the current image and transmits intra-frame coding information to entropy compilation unit 160. Intra-frame coding information may include at least one of intra-frame prediction mode, most probable mode (MPM) flag, and MPM index. Intra-frame coding information may include information about the reference sample. Inter-frame prediction unit 154 may include motion estimation unit 154a and motion compensation unit 154b. Motion estimation unit 154a obtains motion vector values ​​for the current region by referencing a specific region of the reconstructed reference image. Motion estimation unit 154a transmits a set of motion information (reference image index, motion vector information, etc.) for the reference region to entropy compilation unit 160. The motion compensation unit 154b performs motion compensation by using motion vector values ​​passed from the motion estimation unit 154a. The inter-frame prediction unit 154 passes inter-frame coding information, including motion information about the reference region, to the entropy compilation unit 160.

[0033] According to another embodiment, prediction unit 150 may include an intra-block copy (BC) prediction unit (not shown). The intra-BC prediction unit performs intra-BC prediction based on reconstructed samples in the current image and transmits intra-BC coding information to entropy compilation unit 160. The intra-BC prediction unit obtains a specific region in the reference current image, indicating the block vector values ​​of the reference region used to predict the current region. The intra-BC prediction unit can use the obtained block vector values ​​to perform intra-BC prediction. The intra-BC prediction unit transmits intra-BC coding information to entropy compilation unit 160. The intra-BC coding information may include block vector information.

[0034] When performing the image prediction described above, the transformation unit 110 transforms the residual values ​​between the original image and the predicted image to obtain transformation coefficient values. In this case, the transformation can be performed on a unit of specific blocks within the image, and the size of the specific blocks can be changed within a preset range. The quantization unit 115 quantizes the transformation coefficient values ​​generated in the transformation unit 110 and sends them to the entropy compilation unit 160.

[0035] Entropy compilation unit 160 performs entropy compilation on information indicating quantized transform coefficients, intra-frame coding information, inter-frame coding information, etc., to generate a video signal bitstream. In entropy compilation unit 160, variable-length compilation (VLC) schemes, arithmetic compilation schemes, etc., can be used. Variable-length compilation (VLC) schemes involve transforming input symbols into continuous codewords, and the length of the codewords can be variable. For example, frequently occurring symbols are represented by short codewords, while rarely occurring symbols are represented by long codewords. Context-based adaptive variable-length compilation (CAVLC) can be used as a variable-length compilation scheme. Arithmetic compilation transforms continuous data symbols into single prime numbers, where arithmetic compilation obtains the optimal number of bits required to represent each symbol. Context-based adaptive binary arithmetic compilation (CABAC) can be used as arithmetic compilation. For example, entropy compilation unit 160 can binarize information indicating quantized transform coefficients. Entropy compilation unit 160 can generate a bitstream by arithmetically compiling binary information.

[0036] The generated bitstream is encapsulated using Network Abstraction Layer (NAL) units as the basic unit. Each NAL unit comprises an integer number of compiled tree units. To decode the bitstream in the video decoder, the bitstream must first be separated into NAL units, and then each separated NAL unit must be decoded. Simultaneously, the information required for decoding the video signal bitstream can be sent via Raw Byte Sequence Payloads (RBSPs) of higher-level sets such as Picture Parameter Sets (PPS), Sequence Parameter Sets (SPS), and Video Parameter Sets (VPS).

[0037] at the same time, Figure 1The block diagram illustrates an encoding device 100 according to an embodiment of the present invention, and the separately displayed blocks logically distinguish and illustrate the elements of the encoding device 100. Therefore, depending on the device design, the elements of the encoding device 100 can be mounted as one or more chips. According to an embodiment, the operation of each element of the encoding device 100 can be performed by a processor (not shown).

[0038] Figure 2 This is a schematic block diagram of a video signal decoding apparatus 200 according to an embodiment of the present invention. (See reference) Figure 2 The decoding device 200 of the present invention includes an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 225, a filtering unit 230, and a prediction unit 250.

[0039] Entropy decoding unit 210 performs entropy decoding on the video signal bitstream to extract transform coefficient information, intra-frame coding information, inter-frame coding information, etc., for each region. For example, entropy decoding unit 210 can obtain binarized codes for transform coefficient information of a specific region from the video signal bitstream. Entropy decoding unit 210 obtains quantized transform coefficients by inverse binarizing the binary code. Inverse quantization unit 220 inverse quantizes the quantized transform coefficients, and inverse transform unit 225 reconstructs the residual value using the inverse quantized transform coefficients. Video signal processing device 200 reconstructs the original pixel value by adding the residual value obtained by inverse transform unit 225 to the predicted value obtained by prediction unit 250.

[0040] Simultaneously, the filtering unit 230 performs filtering on the image to improve image quality. This may include a deblocking filter to reduce block distortion and / or an adaptive loop filter to remove distortion from the entire image. The filtered image is output or stored in the DPB 256 as a reference image for the next image.

[0041] Prediction unit 250 includes intra-frame prediction unit 252 and inter-frame prediction unit 254. Prediction unit 250 generates a prediction picture using the coding type decoded by the entropy decoding unit 210 described above, the transform coefficients of each region, and the intra / inter-frame coding information. To reconstruct the current block in which decoding is performed, the current picture or the decoded region of other pictures including the current block can be used. In reconstruction, only the current picture, i.e., the picture (or tile / slice) that only performs intra-frame prediction or intra-frame BC prediction, is referred to as an intra-frame picture or I picture (or tile / slice), and the picture (or tile / slice) that can perform all intra-frame prediction, inter-frame prediction, and intra-frame BC prediction is referred to as an inter-frame picture (or tile / slice). To predict the sample value of each block in an inter-frame image (or patch / slice), an image (or patch / slice) using at most one motion vector and a reference image index is called a prediction image or P-image (or patch / slice), and an image (or patch / slice) using at most two motion vectors and a reference image index is called a bidirectional prediction image or B-image (or patch / slice). In other words, a P-image (or patch / slice) uses at most one set of motion information to predict each block, and a B-image (or patch / slice) uses at most two sets of motion information to predict each block. Here, the set of motion information includes one or more motion vectors and a reference image index.

[0042] Intra-prediction unit 252 generates prediction blocks using intra-coding information and reconstructed samples in the current image. As described above, the intra-coding information may include at least one of intra-prediction mode, most probable mode (MPM) flag, and MPM index. Intra-prediction unit 252 predicts sample values ​​for the current block by using reconstructed samples located to the left and / or above the current block as reference samples. In this disclosure, the reconstructed samples, reference samples, and samples of the current block can represent pixels. Moreover, sample values ​​can represent pixel values.

[0043] According to an embodiment, the reference sample may be a sample included in the neighboring blocks of the current block. For example, the reference sample may be a sample adjacent to the left boundary of the current block and / or a sample adjacent to the top boundary. Moreover, the reference sample may be a sample among the samples of the neighboring blocks of the current block located on a line within a predetermined distance from the left boundary of the current block and / or a sample located on a line within a predetermined distance from the top boundary of the current block. In this case, the neighboring blocks of the current block may include the left (L) block, the top (A) block, the bottom left (BL) block, the top right (AR) block, or the top left (AL) block.

[0044] Inter-frame prediction unit 254 generates prediction blocks using reference images and inter-frame coding information stored in DPB 256. The inter-frame coding information may include a set of motion information (reference image index, motion vector information, etc.) for the current block used as the reference block. Inter-frame prediction may include L0 prediction, L1 prediction, and bidirectional prediction. L0 prediction means making a prediction using a reference image included in the L0 image list, while L1 prediction means making a prediction using a reference image included in the L1 image list. For this purpose, a set of motion information (e.g., motion vectors and reference image index) may be required. In the bidirectional prediction method, up to two reference regions can be used, and the two reference regions may exist in the same reference image or in different images. That is, in the bidirectional prediction method, up to two sets of motion information (e.g., motion vectors and reference image indexes) can be used, and the two motion vectors may correspond to the same reference image index or different reference image indices. In this case, in terms of time, reference images can be displayed (or output) before and after the current image. According to an embodiment, the two reference regions used in the bidirectional prediction scheme may be regions selected from image list L0 and image list L1, respectively.

[0045] The inter-frame prediction unit 254 can obtain a reference block for the current block using motion vectors and a reference image index. The reference block is located in the reference image corresponding to the reference image index. Furthermore, the sample value of the block specified by the motion vector, or its interpolated value, can be used as a predictor for the current block. For motion prediction with sub-pellet unit pixel accuracy, for example, an 8-tap interpolation filter for the luma signal and a 4-tap interpolation filter for the chroma signal can be used. However, the interpolation filter for motion prediction at the sub-pellet level is not limited to this. In this way, the inter-frame prediction unit 254 performs motion compensation to predict the texture of the current unit based on a motion image previously reconstructed using motion information. In such a case, the inter-frame prediction unit can use a set of motion information.

[0046] According to another embodiment, prediction unit 250 may include an intra-frame BC prediction unit (not shown). The intra-frame BC prediction unit can reconstruct a current region by referencing a specific region including reconstructed samples within the current image. The intra-frame BC prediction unit obtains intra-frame BC coding information for the current region from entropy decoding unit 210. The intra-frame BC prediction unit obtains block vector values ​​indicating a specific region in the current image. The intra-frame BC prediction unit can perform intra-frame BC prediction using the obtained block vector values. The intra-frame BC coding information may include block vector information.

[0047] The reconstructed video image is generated by adding the predicted value output from the intra-frame prediction unit 252 or the inter-frame prediction unit 254 to the residual value output from the inverse transform unit 225. That is, the video signal decoding device 200 uses the predicted block generated by the prediction unit 250 and the residual obtained from the inverse transform unit 225 to reconstruct the current block.

[0048] at the same time, Figure 2 The block diagram illustrates a decoding apparatus 200 according to an embodiment of the present disclosure, and the separately displayed blocks logically distinguish and illustrate the elements of the decoding apparatus 200. Therefore, depending on the device design, the elements of the decoding apparatus 200 can be mounted as one or more chips. According to an embodiment, the operation of each element of the decoding apparatus 200 can be performed by a processor (not shown).

[0049] Figure 3 The illustration shows an embodiment where a Compiler Tree Unit (CTU) in an image is segmented into Compiler Units (CUs). During the encoding of a video signal, an image can be segmented into a series of Compiler Tree Units (CTUs). A Compiler Tree Unit consists of N×N blocks of luminance samples and two blocks of corresponding chrominance samples. A Compiler Tree Unit can be segmented into multiple Compiler Units. A Compiler Tree Unit may not be segmented and may be a leaf node. In this case, the Compiler Tree Unit itself can be a Compiler Unit. A Compiler Unit refers to the basic unit used to process an image during the aforementioned video signal processing, i.e., intra / inter-frame prediction, transform, quantization, and / or entropy compilation. The size and shape of a Compiler Unit in an image may not be constant. A Compiler Unit can have a square or rectangular shape. A rectangular Compiler Unit (or rectangular block) includes vertical Compiler Units (or vertical blocks) and horizontal Compiler Units (or horizontal blocks). In this specification, a vertical block is a block whose height is greater than its width, and a horizontal block is a block whose width is greater than its height. Furthermore, in this specification, a non-square block may refer to a rectangular block, but this disclosure is not limited thereto.

[0050] refer to Figure 3 First, the compiler tree unit is divided into a quadtree (QT) structure. That is, in a quadtree structure, a node of size 2×2N can be divided into four nodes of size N×N. In this specification, a quadtree can also be referred to as a quaternion tree. Quadtree partitioning can be performed recursively, and not all nodes need to be partitioned at the same depth.

[0051] Simultaneously, the leaf nodes of the aforementioned quadtree can be further divided into multi-type tree (MTT) structures. According to embodiments of this disclosure, in a multi-type tree structure, a node can be divided into a horizontally or vertically partitioned binary or ternary tree structure. That is, in a multi-type tree structure, there are four partitioning structures, such as vertical binary partitioning, horizontal binary partitioning, vertical ternary partitioning, and horizontal ternary partitioning. According to embodiments of this disclosure, in each tree structure, the width and height of a node can both be powers of 2. For example, in a binary tree (BT) structure, a node of size 2×2N can be partitioned into two NX2N nodes through vertical binary partitioning, and further partitioned into two 2NXN nodes through horizontal binary partitioning. Additionally, in a ternary tree (TT) structure, a node of size 2NX2N is partitioned into (N / 2)×2N, NX2N, and (N / 2)×2N nodes through vertical ternary partitioning, and further partitioned into 2NX(N / 2), 2NXN, and 2NX(N / 2) nodes through horizontal ternary partitioning. This multi-type tree split can be performed recursively.

[0052] Leaf nodes of multi-type trees can be compilation units. If no partitioning for a compilation unit is specified or the compilation unit is small for the maximum transform length, the compilation unit can be used as the unit for prediction and transformation without further partitioning. Alternatively, at least one of the following parameters for quadtrees and multi-type trees can be predefined or sent through RBSPs of higher-level collections such as PPS, SPS, VPS, etc.: 1) CTU size: the size of the root node of the quadtree, 2) Minimum QT size MinQtSize: the minimum allowed QT leaf node size, 3) Maximum BT size MaxBtSize: the maximum allowed BT root node size, 4) Maximum TT size MaxTtSize: the maximum allowed TT root node size, 5) Maximum MTT depth MaxMttDepth: the maximum allowed depth of the MTT derived from the leaf nodes of the QT, 6) Minimum BT size MinBtSize: the minimum allowed BT leaf node size, 7) Minimum TT size MinTtSize: the minimum allowed TT leaf node size.

[0053] Figure 4 An embodiment of a method for signaling the segmentation of quadtrees and multi-type trees is shown. Preset flags can be used to signal the segmentation of the aforementioned quadtrees and multi-type trees. Reference Figure 4 At least one of the following flags can be used: "qt_split_flag" indicating whether to split a quadtree node, "mtt_split_flag" indicating whether to split a multi-type tree node, "mtt_split_vertical_flag" indicating the split direction of a multi-type tree node, or "mtt_split_binary_flag" indicating the split type of a multi-type tree node.

[0054] According to embodiments of this disclosure, the compiler tree unit is the root node of a quadtree and can be initially partitioned into a quadtree structure. In the quadtree structure, a "qt_split_flag" signal is sent to each node "QT_node". If the value of "qt_split_flag" is 1, the node is partitioned into four square nodes, and if the value of "qt_split_flag" is 0, the corresponding node becomes a leaf node "QT_leaf_node" of the quadtree.

[0055] Each quadtree leaf node "QT_leaf_node" can be further segmented into a multi-type tree structure. In the multi-type tree structure, "mtt_split_flag" is signaled for each node "MTT_node". When the value of "mtt_split_flag" is 1, the corresponding node is segmented into multiple rectangular nodes, and when the value of "mtt_split_flag" is 0, the corresponding node is a leaf node "MTT_leaf_node" in the multi-type tree. When a multi-type tree node "MTT_node" is segmented into multiple rectangular nodes (i.e., when the value of "mtt_split_flag" is 1), the "mtt_split_vertical_flag" and "mtt_split_binary_flag" of the node "MTT_node" can be additionally signaled. When the value of "mtt_split_vertical_flag" is 1, it indicates the vertical split of the node "MTT_node", and when the value of "mtt_split_vertical_flag" is 0, it indicates the horizontal split of the node "MTT_node". Additionally, when the value of "mtt_split_binary_flag" is 1, the node "MTT_node" is split into 2 rectangular nodes, and when the value of "mtt_split_binary_flag" is 0, the node "MTT_node" is split into 3 rectangular nodes.

[0056] Image prediction (motion compensation) for encoding is performed on compilation units that are no longer divided (i.e., leaf nodes of the compilation unit tree). In the following text, the basic unit used to perform the prediction will be referred to as a "prediction unit" or "prediction block".

[0057] In the following text, the term "unit" as used herein may be used in place of a prediction unit, which is the basic unit used to perform prediction. However, this disclosure is not limited thereto, and "unit" may be understood to broadly encompass the concept of a compilation unit.

[0058] Figure 5The illustration depicts inter-frame prediction according to an embodiment of the present invention. As described above, the decoder can predict the current block by referring to a reconstructed sample from another decoded image. (Reference) Figure 5 The decoder obtains a reference block 42 within a reference image based on the motion information of the current block 32. In this case, the motion information may include a reference image index and a motion vector 50. The reference image index indicates the reference image of the current block in the reference image list. Furthermore, the motion vector 50 indicates the offset between the coordinate values ​​of the current block 32 within the current image and the coordinate values ​​of the reference block 42 within the reference image. The decoder obtains a predictor for the current block 32 based on the sampled values ​​of the reference block 42, and uses this predictor to reconstruct the current block 32.

[0059] Furthermore, according to embodiments of the present invention, sub-block-based motion compensation can be used. That is, the current block 32 can be divided into multiple sub-blocks, and independent motion vectors can be used for each sub-block. Therefore, different reference blocks can be used within the current block 32 to predict each sub-block. According to embodiments, sub-blocks can have a preset size such as 4x4 or 8x8. The decoder obtains the predictor for each sub-block of the current block 32 using the motion vector of each sub-block. By combining the predictors of each sub-block, the predictor of the current block 32 can be obtained, and the decoder can use the obtained predictor of the current block 32 to recover the current block 32.

[0060] According to embodiments of the present invention, sub-block-based motion compensation can be performed using various methods. Sub-block-based motion compensation may include affine model-based motion compensation (hereinafter referred to as affine motion compensation or affine motion prediction) and sub-block-based temporal motion vector prediction (SbTMVP). Various embodiments of affine motion compensation and SbTMVP are described below with reference to the accompanying drawings.

[0061] Figure 6 The illustration depicts a method for signaling motion vectors according to an embodiment of the present invention. According to an embodiment of the present invention, motion vectors (MVs) can be generated based on a motion vector prediction (or predictor) (MVP). For example, MVs can be determined using the MVP, as shown in [Equation 1] below. In other words, MVs can be determined (configured or derived) to have the same value as the MVP.

[0062] [Equation 1]

[0063] In another example, the MV can be determined based on the MVP and the motion vector difference (MVD), as shown in [Equation 2] below. The encoder can signal the decoder with MVD information to indicate the MV more accurately, and the decoder can deduce the MV by adding the acquired MVD to the MVP.

[0064] [Equation 2]

[0065] According to embodiments of the present invention, the encoder can transmit the determined motion information to the decoder, and the decoder can generate a prediction vector (MV) from the received motion information and generate (or derive) a prediction block based on it. For example, the motion information may include MVP information and MVD information. In this case, the elements of the motion information may vary depending on the inter-frame prediction mode. For example, in the merge mode, the motion information may or may not include MVP information. In another example, in the Advanced Motion Vector Prediction (AMVP) mode, the motion information may include MVP information and MVD information.

[0066] To determine, send, and receive information about MVPs, the encoder and decoder can generate MVP candidates (or a list of MVP candidates) in the same way. For example, the encoder and decoder can generate the same MVP candidates in the same order. Furthermore, the encoder can send the decoder an index indicating (or representing) the index of the MVP determined (or selected) from the generated MVP candidates, and the decoder can deduce the determined MVP and / or MV based on the received index.

[0067] According to embodiments of the present invention, MVP candidates may include spatial candidates, temporal candidates, etc. When a merge mode is applied, an MVP candidate may be called a merge candidate, and when an AMVP mode is applied, an MVP candidate may be called an AMVP candidate. A spatial candidate may be the MV (or motion information) of a block located at a specific position relative to the current block. For example, a spatial candidate may be the MV of a block that is adjacent to or not adjacent to the current block. A temporal candidate may be the MV corresponding to a block in an image different from the current image. Furthermore, for example, MVP candidates may include affine MV, ATMVP, STMVP, combinations of MVs (or candidates), average MV of MVs (or candidates), zero MV, etc.

[0068] According to an embodiment, the encoder can signal information indicating the reference image to the decoder. As an example, when the reference image of an MVP candidate differs from the reference image of the current block (or the currently processed block), the encoder / decoder can scale the MV (motion vector scaling) of the MVP candidate. In this case, MV scaling can be performed based on the image sequence number (POC) of the current image, the POC of the reference image of the current block, and the POC of the reference image of the MVP candidate.

[0069] The following describes a detailed embodiment of the MVD signaling method. Table 1 below shows the syntax structure for MVD signaling.

[0070] [Table 1]

[0071] Referring to [Table 1], according to embodiments of the present invention, the symbol and absolute value of the MVD can be compiled separately. That is, the symbol and absolute value of the MVD can have different syntax (or syntax elements). Furthermore, as shown in Table 1, the absolute value of the MVD can be compiled directly or progressively based on a flag indicating whether the absolute value is greater than N. If the absolute value is greater than N, a signal can also be sent indicating the value of (absolute value – N). Specifically, in the example in [Table 1], an abs_mvd_greater0_flag indicating whether the absolute value is greater than 0 can be sent. If abs_mvd_greater0_flag indicates that the absolute value is not greater than 0, the absolute value of the MVD can be determined to be 0. Furthermore, if abs_mvd_greater0_flag indicates that the absolute value is greater than 0, additional syntax (or syntax elements) may exist.

[0072] For example, an `abs_mvd_greater1_flag` indicating an absolute value greater than 1 can be sent. If `abs_mvd_greater1_flag` indicates that the absolute value is not greater than 1, then the absolute value of the MVD can be determined to be 1. If `abs_mvd_greater1_flag` indicates that the absolute value is greater than 1, then additional syntax may exist. For example, `abs_mvd_minus2` may exist. `abs_mvd_minus2` could be a value of (absolute value – 2). Since the absolute value is determined to be greater than 1 (i.e., greater than or equal to 2) based on the values ​​of `abs_mvd_greater0_flag` and `abs_mvd_greater1_flag`, a value of (absolute value – 2) can be signaled. Compared to binarization and signaling the absolute value, performing syntax signaling hierarchically on information about the absolute value can use fewer bits.

[0073] According to the embodiments, absolute value-related syntax can be compiled by applying variable-length binary conversion methods (such as exponential Golomb, truncated unary codes, truncated Rice codes, etc.). Furthermore, a flag indicating the sign of the MVD can be signaled via `mvd_sign_flag`.

[0074] Although the method for compiling MVD has been described in the above embodiments, the symbols and absolute values ​​of information other than MVD can also be signaled separately. The absolute value can be compiled as a flag indicating whether the absolute value is greater than a predetermined specific value, or as a value obtained by subtracting a specific value from the absolute value. In [Table 1], [0] and [1] can indicate component indices. For example, [0] and [1] can be the x-component (i.e., the horizontal component) and the y-component (i.e., the vertical component).

[0075] Figure 7 The illustration depicts a method for signaling adaptive motion vector resolution information according to an embodiment of the present invention. According to an embodiment of the present invention, the resolution of the MV or MVD can be changed. For example, the resolution can be expressed based on pixels (or pel) (e.g., 1 / 4, 1 / 2, 1, 2, or 4 pixels). The encoder can signal the resolution information of the MV or MVD to the decoder. Furthermore, for example, 16 can be compiled into 64 (1 / 4) in units of 1 / 4. 64=16), compiled into 16 (1) in units of 1. 16=16), and compiled in units of 4 as 4 (4 4=16). That is, the MV or MVD value can be determined using [Equation 3] below.

[0076] [Equation 3]

[0077] In [Equation 3], `valueDetermined` indicates the MV or MVD value. Additionally, `valuePerResolution` is the value signaled based on the determined resolution. In this case, rounding can be applied when the value signaled by the MV or MVD cannot be divided by the determined resolution. When using high resolution, accuracy can increase, but the compiled value is larger, so more bits may be used. When using low resolution, accuracy may decrease, but the compiled value is smaller, so fewer bits can be used. According to embodiments, the resolution can be configured differently in units such as sequences, images, slices, compilation tree units (CTUs), compilation units (CUs), etc. That is, the encoder / decoder can adaptively determine / apply the resolution based on predetermined units among the above.

[0078] According to the embodiments in the specification, the encoder can signal resolution information to the decoder. In this case, the resolution information can be communicated based on variable-length binarized concurrent signals. In this case, signaling overhead can be reduced when signaling is performed based on the index corresponding to the minimum value (i.e., the foremost value). As an example, the resolution can be mapped to the signaling index in order from high resolution to low resolution.

[0079] According to the embodiments in this specification, Figure 7The diagram illustrates the signaling method when three resolutions are assumed to be used across various resolutions. In this case, the three signaling bits can be 0, 10, and 11, and the three signaling indices can be referred to as the first resolution, second resolution, and third resolution. Since 1 bit is needed to signal the first resolution and 2 bits are needed to signal the remaining resolutions, the signaling overhead can be relatively reduced when signaling the first resolution. Figure 7 In the example, the first resolution, the second resolution, and the third resolution can be defined as 1 / 4, 1, and 4 pixel resolutions, respectively. In the following embodiments, the MV resolution can be the resolution of the MVD.

[0080] Figure 8 The illustration depicts affine motion compensation according to an embodiment of the present invention. According to conventional inter-frame prediction methods, inter-frame prediction is performed using only one motion vector for each of the L0 and L1 predictions of the current block, thus optimizing the prediction of translational motion. However, reference blocks 44 of various shapes and sizes are required to efficiently perform motion compensation for scaling, rotation, and other irregular motions.

[0081] refer to Figure 8 In affine motion compensation, the current block 34 can be predicted using a reference block 44 with a different size, shape, and / or orientation than the current block 34. That is, the reference block 44 can have a non-rectangular shape and can be larger or smaller than the current block 34. The reference block 44 can be obtained by performing an affine transformation on the current block 34. The affine transformation can include a 6-parameter affine transformation using three control point motion vectors (CPMV) and a 4-parameter affine transformation using two control point motion vectors. Detailed embodiments thereof will be described below.

[0082] Figure 9 Figure 4 illustrates an embodiment of the parametric affine motion compensation method. To reduce the computational cost and signaling overhead of the affine transformation, a preset number of control point motion vectors (CPMVs) can be used to perform affine motion compensation. A control point motion vector (CPMV) is a motion vector corresponding to a specific control point (or sample position) of the current block. A specific control point may include at least one corner of the current block. In embodiments of the invention, the CPMV corresponding to the upper left corner of the current block is referred to as v0 (or the first CPMV), the CPMV corresponding to the upper right corner of the current block is referred to as v1 (or the second CPMV), and the CPMV corresponding to the lower left corner of the current block is referred to as v2 (or the third CPMV). A set of CPMVs including at least two CPMVs can be used for affine motion prediction.

[0083] according to Figure 9In this embodiment, v0 and v1 can be used to perform 4-parameter affine motion prediction. The current block 36, represented by the solid line, can be predicted using a reference block 46 located at the position represented by the dashed line. Samples of the current block 36 can be mapped to different reference samples through affine transformations. More specifically, the motion vector (Vx, Vy) at the sample position (x, y) of the current block 36 can be derived from [Equation 4] below.

[0084] [Equation 4]

[0085] Here, (v 0x , v 0y ) indicates the first CPMV corresponding to the top left corner of the current block 36, while (v 1x , v 1y This indicates the second CPMV corresponding to the top right corner of the current block. Additionally, w is the width of the current block, which is 36.

[0086] Figure 10 The illustration shows an affine motion prediction method according to an embodiment of the present invention. According to an embodiment of the present invention, multiple CPMVs or multiple parameters can be used to indicate affine motion.

[0087] refer to Figure 10 The encoder / decoder can use the CPMVs of v0, v1, and v2 to perform affine motion prediction. When using six parameters, the three control point motion vectors (CPMVs) of v0, v1, and v2 can be used for affine motion prediction. The shape of the reference block by which the current block 36 is predicted can be indicated based on the three vectors v0, v1, and v2 or the six parameters. Reference Figure 10 This can be seen from the reference image. Figure 10 The reference block 46, represented by a dashed line, is used to predict the current block 36, represented by a straight line. That is, each control point (or pixel at the corresponding position) of the current block 36 can be mapped to another point (or pixel) through an affine transformation. Equation (5) below shows the motion vector field according to an embodiment of the invention.

[0088] [Equation 5]

[0089] In [Equation 5], the indicator Figure 10The CPMV of the motion vector v0 indicating the top-left control point can be represented by (mv_0^x, mv_0^y). The CPMV of the motion vector v1 indicating the top-right control point can be represented by (mv_1^x, mv_1^y). The CPMV of the motion vector v2 indicating the bottom-left control point can be represented by (mv_2^x, mv_2^y). In this case, the motion vector (mv^x, mv^y) at position (x, y) can be derived (or calculated) using [Equation 5]. Therefore, based on the control point motion vectors v0, v1, and v2, the position of each pixel or the motion vector at a specific position can be derived according to [Equation 5].

[0090] Furthermore, in [Equation 5], (x, y) can be relative coordinates within the block. For example, when the top-left position of the block is (0,0), (x, y) might be a position based on the top-left position. Therefore, assuming the position of v0 is (x0, y0), the position of v1 is (x1, y1), and the position of v2 is (x2, y2), and when the same coordinates as the positions of v0, v1, and v2 are used for (x, y), in [Equation 5], x and y are represented as (x-x0) and (y-y0), respectively. Additionally, in this case, w, which indicates the width of the block, can be (x1-x0), and h, which indicates the height of the block, can be (y2-y0).

[0091] Figure 11 The illustration shows an embodiment of a sub-block-based affine motion compensation method. As described above, affine motion transformations can be used to derive the motion vector (i.e., the motion vector field) at each sample position of the current block. However, to reduce computational complexity, sub-block-based affine motion compensation can be performed according to embodiments of the present invention. Figure 8 As shown, the current block may include multiple sub-blocks, and a representative motion vector for each sub-block is obtained based on the CPMV set. According to an embodiment, the representative motion vector for each sub-block may be a motion vector corresponding to the position of an intermediate sample of the corresponding sub-block. According to an additional embodiment, motion vectors with higher precision than general motion vectors can be used as the motion vectors for sub-blocks. For this purpose, a motion-compensated interpolation filter can be applied.

[0092] The size of the sub-block after affine motion compensation can be configured using various methods. According to embodiments of the invention, the sub-block can have a preset size such as 4x4 or 8x8. According to embodiments of the invention, the size MXN of the sub-block can be determined by the following [Equation 6].

[0093] [Equation 6]

[0094] Here, w represents the width of the current block, and MvPre represents the fractional unit precision of the motion vector. (v2x, v2y) is the third CPMV corresponding to the lower left corner of the current block, and may be calculated according to an embodiment through [Equation 5]. Max(a, b) is a function that returns the larger value between a and b, and abs(x) is a function that returns the absolute value of x. In addition, clip3(x, y, z) is a function that returns x when z < x, returns y when z > y, and returns z in other cases.

[0095] The decoder obtains the motion vector of each sub-block of the current block through the CPMVs in the CPMV set. In addition, the decoder obtains the predictor of each sub-block based on the motion vector of each sub-block, and obtains the predictor of the current block by combining the predictors of the respective sub-blocks. The decoder may use the obtained predictor of the current block to reconstruct the current block.

[0096] Figure 12 and 13 illustrates an embodiment of the present invention for obtaining the control point motion vector set used to predict the current block. According to the embodiment of the present invention, the CPMV set used to predict the current block may be obtained through various methods. More specifically, the CPMV set used to predict the current block may be obtained by referring to the motion vector information set of one or more adjacent blocks. According to the embodiment of the present invention, the motion vector information may indicate the motion vector of the corresponding block or the CPMV of the corresponding block. In addition, the motion vector information set indicates a collection of motion vector information of one or more blocks. The adjacent blocks may be blocks including preset adjacent positions of the current block. In this case, the adjacent block may be a coding unit including a preset adjacent position, or a region in a preset unit (for example, 4x4 or 8x8) including the adjacent position.

[0097] Multiple candidates can exist, which can be referenced to derive the CPMV of the current block. Therefore, information about neighboring blocks to be referenced for deriving the CPMV of the current block can be signaled individually. According to an embodiment of the invention, an indicator indicating a set of motion vector information to be referenced for deriving the motion vectors of each sub-block of the current block can be signaled. The indicator can indicate the set of motion vector information of neighboring blocks to be referenced for deriving the motion vectors of each sub-block of the current block. The decoder can acquire the indicator and, by referencing the set of motion vector information of neighboring blocks indicated by the indicator, acquire each CPMV for the CPMV set of the current block. According to a more detailed embodiment, the decoder can generate a candidate list including one or more motion vector information set candidates. Each motion vector information set candidate included in the candidate list is a set of motion vectors of neighboring blocks that can be used to derive the motion vector information of the current block. In this case, the indicator can be an index indicating a motion vector information set in the candidate list. The CPMV of the current block can be acquired by referring to the motion vector information set selected from the candidate list based on the indicator (i.e., the index). In the following sections, various embodiments of motion vector information set candidates will be described, which may be included in a candidate list for deriving motion vector information (or CPMV set) for the current block.

[0098] Figure 12 The illustration shows an example of obtaining the CPMV set of the current block. Figure 12 In this embodiment, it is assumed that the CPMV set of the current block includes two CPMVs, namely v0 and v1. According to an embodiment of the invention, the CPMV of the current block can be derived from the motion vectors of neighboring blocks adjacent to the corresponding point. (See reference...) Figure 12 v0 can be derived from one of the motion vectors of adjacent blocks A, B, and C adjacent to the corresponding point, and v1 can be derived from one of the motion vectors of adjacent blocks D and E adjacent to the corresponding point. When the motion vectors of adjacent blocks A, B, C, D, and E are vA, vB, vC, vD, and vE, the set of motion vector information that can be included in the candidate list can be derived as shown in [Equation 7] below.

[0099] [Equation 7]

[0100] That is, (v0, v1) pairs can be obtained, including v0 selected from vA, vB, and vC, and v1 selected from vD and vE. In this case, v0 can be derived from the motion vector of the block adjacent to the top-left corner of the current block, and v1 can be derived from the motion vector of the block adjacent to the top-right corner of the current block. According to an additional embodiment, motion vector scaling can be performed based on the picture sequence number (POC) of the current block, the POC of the reference picture of the adjacent block, and the POC of the reference picture of the current block.

[0101] As described above, a candidate list including the acquired motion vector information set candidates can be generated, and an indicator indicating a motion vector information set in the candidate list can be signaled. According to an additional embodiment of the invention, the candidate list may include motion vector information set candidates for another type of inter-frame prediction. For example, the candidate list may include motion vector information set candidates for sub-block-based temporal motion vector prediction (SbTMVP).

[0102] The decoder can derive the CPMV of the current block based on a set of motion vector information obtained from a candidate list. According to one embodiment, the decoder can perform affine merging prediction by using motion vectors from the motion vector information set obtained from the candidate list as the CPMV of the current block without requiring any separate motion vector difference values. According to another embodiment, the decoder can obtain separate motion vector difference values ​​for the CPMV of the current block. The decoder can obtain the CPMV of the current block by adding the motion vectors from the motion vector information set obtained from the candidate list to the motion vector difference values. The decoder can separately signal a flag or index indicating whether to use separate motion vector difference values ​​for affine motion compensation of the current block.

[0103] Figure 13 The illustration shows another embodiment for obtaining the CPMV set of the current block. According to another embodiment of the invention, the CPMV of the current block can be derived from the motion vector information of neighboring blocks to which affine motion compensation has already been performed, i.e., the CPMV or motion vectors of the neighboring blocks. In this case, the neighboring blocks may include the left neighboring block and the upper neighboring block of the current block. (Reference) Figure 13 (a) The left adjacent block can include the block adjacent to the lower-left corner of the current block, i.e., left block A and lower-left block D. Furthermore, the upper adjacent block includes the block adjacent to the upper-left corner of the current block, i.e., upper-left block E, and the block adjacent to the upper-right corner of the current block, i.e., upper block B and upper-right block C. The decoder identifies whether affine motion compensation has been performed on adjacent blocks in a preset order. When an adjacent block for which affine motion compensation has been performed is found, the decoder uses the CPMV set (or motion vector) of the corresponding adjacent block to obtain the CPMV set of the current block. (See reference) Figure 13 (b) The CPMV set of the left block A can be used to deduce the CPMV set of the current block. That is, the CPMV set (v0, v1) of the current block can be obtained based on the CPMV set (v2, v3, v4) of the left block A.

[0104] According to embodiments of the present invention, information about neighboring blocks to be referenced can be individually signaled to derive the CPMV of the current block. In this case, the CPMV set of the current block's neighboring blocks can be transformed into motion vector information set candidates included in the candidate list according to a preset order. More specifically, the motion vector information set candidates may include a first candidate derived from the CPMV (or motion vector) of the current block's left neighboring block and a second candidate derived from the CPMV (or motion vector) of the current block's upper neighboring block. In this case, the left neighboring block is the block adjacent to the lower left corner of the current block, while the upper neighboring block is the block adjacent to the upper left corner or the upper right corner of the current block. As described above, a candidate list including the acquired motion vector information set candidates can be generated, and an indicator indicating a motion vector information set in the candidate list can be signaled. According to an embodiment, the indicator may indicate the position information of neighboring blocks referenced for deriving the motion vector of each sub-block of the current block. The decoder may refer to the CPMV set (or motion vector) of the neighboring blocks indicated by the indicator to obtain the CPMV set of the current block.

[0105] According to an additional embodiment of the invention, the CPMV of the current block can be derived based on the CPMV of neighboring blocks close to the corresponding point. For example, v0 can be obtained by referring to the CPMV of the left neighboring block, and v1 can be obtained by referring to the CPMV of the upper neighboring block. Alternatively, v0 can be obtained by referring to the CPMV of neighboring blocks A, D, or E, and v1 can be obtained by referring to the CPMV of neighboring blocks B or C.

[0106] Figure 14 The illustration depicts an affine motion compensation method according to an embodiment of the present invention. According to an embodiment of the present invention, a CPMV may be needed for affine motion prediction, and a motion vector field, i.e., a motion vector at a sub-block or any location (or pixel), can be calculated based on the CPMV. In this specification, the CPMV may be referred to as a seed vector. In this case, the CPMV can be derived based on a predictor. For example, the predictor can be determined as the CPMV. In another example, the CPMV can be calculated based on the predictor and the difference. Specifically, the CPMV can be calculated by adding the predictor to the difference or subtracting the difference from the predictor.

[0107] According to embodiments of the present invention, a predictor for CPMV can be derived from the CPMV or MV of adjacent affine motion compensation (MC) blocks (i.e., adjacent blocks compiled in affine mode). For example, when a block at a preset position is subjected to affine motion prediction, the encoder / decoder can derive a predictor for affine motion compensation for the current block from the CPMV or MV of the corresponding block. According to an embodiment, the preset position may be... Figure 14The positions A0, A1, B0, B1, or B2 are shown. Alternatively, the preset positions may include positions adjacent to the current block and / or positions not adjacent to the current block. Furthermore, the encoder / decoder may reference spatially adjacent CPMVs or MVs at the preset positions, or temporally adjacent CPMVs or MVs at the preset positions. In this specification, candidates based on affine MC blocks may be referred to as inheritance candidates. Alternatively, candidates based on affine MC blocks may be referred to as merge candidates. Furthermore, according to embodiments, when referencing preset positions, the encoder / decoder may refer to the preset positions according to a preset order.

[0108] Figure 15 The illustration depicts an affine motion compensation method according to an embodiment of the present invention. A CPMV may be needed for affine motion prediction, and the motion vector field, i.e., the motion vector at a sub-block or arbitrary location, can be calculated based on the CPMV. In this specification, the CPMV may be referred to as a seed vector. In this case, the CPMV can be derived based on a predictor. For example, the predictor can be determined as the CPMV. In another example, the CPMV can be calculated based on the predictor and the difference. Specifically, the CPMV can be calculated by adding the predictor to the difference or subtracting the difference from the predictor.

[0109] According to embodiments of the present invention, the encoder / decoder can use the MVs of adjacent blocks to derive a predictor for the CPMV. In this case, the MVs of adjacent blocks may include the MVs of blocks that are not affine motion compensated. For example, when deriving each CPMV of the current block, the encoder / decoder can use the MV at a preset position for each CPMV as the predictor for the CPMV. For example, the preset position may be a portion included in the block adjacent to it.

[0110] refer to Figure 15 The encoder / decoder can determine the CPMV of mv0, mv1, and mv2. Here, mv0 indicates the upper-left CPMV, mv1 indicates the upper-right CPMV, and mv2 indicates the lower-left CPMV. According to an embodiment of the invention, the encoder / decoder can use the MV corresponding to preset positions A, B, or C as the predictor of mv0. Furthermore, the encoder / decoder can use the MV corresponding to preset positions D or E as the predictor of mv1. Additionally, the encoder / decoder can use the MV corresponding to preset positions F or G as the predictor of mv2.

[0111] When according to Figure 15 In embodiments where each predictor of the CPMV for mv0, mv1, or mv2 is determined, the order in which preset positions for each control point are referenced can be predefined. For example, for each control point position, the number of preset positions referenced by the CPMV predictor can be multiple, and combinations of available preset positions can be predefined. Figure 15 The method shown in the embodiments can derive a candidate (or predictor) for affine MC, and this candidate can be referred to as a constructed candidate. Alternatively, the candidate can be referred to as an inter-frame candidate or a virtual candidate. Furthermore, in Figure 15 In this method, when referencing a preset position, the encoder / decoder can refer to the preset position according to a preset order.

[0112] According to embodiments of the present invention, the encoder / decoder can be... Figures 12 to 15 The embodiments or combinations thereof are used to generate a candidate list of affine motion predictors or a candidate list of CPMVs for affine motion predictors. [Equation 8] illustrates a method for deriving an affine motion predictor according to an embodiment of the invention.

[0113] [Equation 8]

[0114] As described above, the CPMV for the current block can be derived from the neighboring blocks of the affine motion prediction. In this case, Equation 8 can be used. In Equation 8, the MVs of the top-left, top-right, and bottom-left control points of the neighboring blocks of the affine motion prediction can be represented as (v_E0x, v_E0y), (v_E1x, v_E1y), and (v_E2x, v_E2y), respectively. Furthermore, the coordinates of the top-left, top-right, and bottom-left control points of the neighboring blocks of the affine motion prediction can be represented as (x_E0, y_E0), (x_E1, y_E1), and (x_E2, y_E2), respectively. In this case, the encoder / decoder can calculate the predictor of the CPMV of the current block or the CPMV corresponding to (v_0x, v_0y) and (v_1x, v_1y) according to Equation 8. Equations 9 and 10 below illustrate a method for deriving the affine motion predictor according to an embodiment of the present invention.

[0115] [Equation 9]

[0116] [Equation 10]

[0117] As described above, multiple CPMVs or multiple CPMV predictors can be used for affine motion compensation. In this case, the encoder / decoder can derive a CPMV or CPMV predictor from another CPMV or CPMV predictor. For example, the encoder / decoder can derive (or generate) two CPMVs or two CPMV predictors using the methods described above, and derive another CPMV or another CPMV predictor based on them.

[0118] According to embodiments of the present invention, the encoder / decoder can use [Equation 9] and [Equation 10] to derive (or generate) mv0, mv1, and mv2 as top-left, top-right, and bottom-left CPMV predictors or CPMVs. In [Equation 9] and [Equation 10], x and y indicate the x-component and y-component, respectively, and the size of the current block can be w. h. The encoder / decoder can create a predictor for the CPMV to perform affine motion compensation on the current block, and determine the CPMV by adding the difference to the predictor. According to an embodiment, this can be achieved through... Figures 14 to 15 The method described herein generates a predictor for CPMV and can signal the difference in CPMV from the encoder to the decoder.

[0119] According to the embodiments, there may be a difference for each CPMV. Furthermore, each difference for each CPMV can be signaled. In a 4-parameter model, two CPMVs can be used, and the MV of each CPMV can be derived by adding the predictor of each CPMV to each of mvd0 and mvd1. In a 6-parameter model, three CPMVs can be used, and the MV of each CPMV can be derived by adding the predictor of each CPMV to each of mvd0, mvd1, and mvd2.

[0120] According to an embodiment of the invention, the encoder / decoder can determine the motion vector difference based on [Equation 11] below.

[0121] [Equation 11]

[0122] In [Equation 11], when the reference list is 0, MvdL0[x0][y0][compIdx] can be configured as lMvd[compIdx]. In this case, compIdx indicates the component index and can have a value of 0 or 1. When the reference list is 1, MvdL1[x0][y0][compIdx] can be configured as lMvd[compIdx].

[0123] According to an embodiment, the motion vector difference can be signaled using the method described in [Table 1], and can be lMvd in [Equation 11]. As described above, the encoder / decoder can determine that the signaled mvd is the difference for the predictor of CPMV, and the determined difference can be MvdL0 and MvdL1 in [Equation 11]. In [Equation 11], L0 can indicate reference list 0, and L1 can indicate reference list 1.

[0124] According to an embodiment of the present invention, in order to perform affine MC on the current block, the encoder / decoder can derive a predictor for the CPMV and determine the CPMV by adding the difference to the predictor. As an example, the encoder / decoder can... Figures 14 to 15 The method described herein generates a predictor for the CPMV. The difference in the CPMV can be signaled from the encoder to the decoder. According to an embodiment, the encoder / decoder can derive the CPMV based on [Equations 12] and [Equations 13] below.

[0125] [Equation 12]

[0126] [Equation 13]

[0127] Referring to [Equations 12] and [Equations 13], there may be a predictor for the difference of each CPMV. According to an embodiment, the difference of a CPMV can be determined based on the difference of another CPMV. This is based on the similarity between the differences of CPMVs. That is, if a predictor for one CPMV is determined, then because the differences of CPMVs are similar, a predictor for another CPMV can be determined using a small amount of information. For example, the difference predictor for the CPMV can be signaled, and the difference with the difference predictor for the CPMV can be signaled from the encoder to the decoder. [Equation 12] assumes the case of using a 4-parameter model, while [Equation 13] assumes the case of using a 6-parameter model.

[0128] Referring to [Equations 12] and [Equations 13], the encoder / decoder can determine the difference and CPMV of each CPMV based on the difference (mvd0) of mv0 corresponding to CPMV 0. The mvd0, mvd1, and mvd2 from [Equations 12] and [Equations 13] can be signaled from the encoder to the decoder. (Refer to the above...) Figure 15 Compared to the described method, even when using the same Figure 15 Using the same predictor, methods [Equation 12] and [Equation 13] can also have different signaling values ​​for mvd1 and mvd2. If the differences between the predictors for mv0, mv1, and mv2 in CPMV are similar, the absolute values ​​of mvd1 and mvd2 in methods [Equation 12] and [Equation 13] may be smaller than... Figure 15 The absolute value in the method is used, therefore, the signaling overhead of mvd1 and mvd2 can be effectively reduced by using the methods in [Equations 12] and [Equations 13]. The encoder / decoder can determine the difference between the predictor of mv1 and the predictor of mv2 as (mvd1+mvd0), and the difference between the predictor of mv2 and the predictor of mv2 as (mvd2+mvd0). In [Equations 12] and [Equations 13], the elements with the superscript can indicate the predictor of CMVP.

[0129] According to an embodiment of the invention, the encoder / decoder can determine the motion vector difference based on [Equation 14] below. In this case, [Equations 12] and [Equations 13] above can be applied.

[0130] [Equation 14]

[0131] According to the embodiment, the motion vector difference can be signaled according to [Table 1] above or [Table 2] below. The motion vector difference signaled can be lMvd in [Equation 14]. Furthermore, the mvds signaled in [Equation 12] and [Equation 13], i.e., mvd0, mvd1, and mvd2, can be lMvd in [Equation 14]. In [Equation 14], MvdLX can indicate the difference with the predictor of each CPMV. That is, MvdLX can be (mv - mvp). In this case, as described in [Equation 12] and [Equation 13], for CPMV 0, the motion vector difference signaled can be directly used as the difference (MvdLX) of the CPMV. For other CPMVs, the motion vector difference informed by the signal (mvd1 and mvd2 in [Equations 12] and [Equations 13]) and the motion vector difference informed by the signal for CPMV 0 (mvd0 in [Equations 12] and [Equations 13]) can be used as MvdLX corresponding to the CPMV difference.

[0132] In [Equation 14], LX can indicate the reference list X. compIdx indicates the component index and can indicate the x and y components. cpIdx represents the control point index. cpIdx can be 0 or 1, or 0, 1, or 2 in [Equation 12] or [Equation 13]. According to embodiments, the encoder / decoder can take into account the resolution of the motion vector differences in [Table 1], [Equation 14], and [Table 2]. For example, when the resolution is R, lMvd The value of R can be used in lMvd in [Equation 14]. Table 2 below shows the motion vector difference syntax structure according to an embodiment of the present invention.

[0133] [Table 2]

[0134] Referring to [Table 2], the motion vector difference can be compiled using essentially the same method as described in [Table 1]. In this case, the motion vector difference can be compiled separately based on cpIdx and / or the control point index.

[0135] Figure 16The illustration depicts a sub-block-based temporal MVP (SbTMVP) according to an embodiment of the present invention. According to an embodiment of the present invention, the encoder / decoder can extract (or derive) motion vectors of temporally adjacent blocks on a sub-block basis. For example, the encoder / decoder can extract motion vectors corresponding to sub-blocks of the current block from another image. Here, a block can be a CU (Cubic Unit), and a sub-block can be a sub-CU. Furthermore, the other image can be a collocated picture. In this invention, SbTMVP can be referred to as a sub-block-based temporal merge candidate or alternative temporal motion vector predictor (ATMVP).

[0136] According to an embodiment of the present invention, when the SbTMVP mode is applied, the encoder / decoder can search for a block (referred to as a corresponding block or reference block) in another image that corresponds to the current block. In this case, the corresponding block can be referred to as a juxtaposed block. According to an embodiment, the juxtaposed block can be determined as a block in another image corresponding to a preset position (such as the lower right or center position of the current block). In SbTMVP, the juxtaposed block can be a motion-compensated block. For example, the encoder / decoder can refer to motion vectors around the current block to find the juxtaposed block.

[0137] refer to Figure 16 A juxtaposed block can be identified (or deduced) as a block within another image whose motion vector moves from the current block's position to position A1 among its neighboring blocks. The motion vector indicating (facing) the juxtaposed block can be deduced based on candidate motion vectors found by scanning preset positions around the current block according to a preset order. As another embodiment, the motion vector indicating the juxtaposed block can be determined based on some merge candidate lists. For example, the encoder / decoder can determine the juxtaposed block based on the first merge candidate in the merge candidate list.

[0138] Furthermore, according to embodiments of the present invention, the encoder / decoder can determine (or derive) the motion information of the sub-blocks of the current block from the motion information of the sub-blocks of the juxtaposed block. In this specification, the motion vector can be motion information, which includes information about whether motion vectors, reference indices, and reference lists are used, regardless of the expression. The sub-blocks of the juxtaposed block, which form the basis for the motion information of the sub-blocks of the current block, can have the same relative position within the block (CU). That is, two sub-blocks can have the same relative position within the block.

[0139] Furthermore, according to embodiments of the present invention, when SbTMVP is used, the sub-block size can be fixed to a predetermined size. For example, the sub-block size can be fixed at 8×8. According to another embodiment, the sub-block size can be variable. In this case, to determine the variable sub-block size, there may be signaling in units of sequence, image, slice, CTU, or CU. For example, the sub-block size can be variably determined to be 4×4 or 8×8.

[0140] Figure 17 The diagram illustrates a planar MVP according to an embodiment of the present invention. According to an embodiment of the present invention, the encoder / decoder can generate a motion vector field on a sub-block basis using the planar MVP method. Furthermore, the encoder / decoder can reference neighboring motion vectors using the planar MVP method. Additionally, the encoder / decoder can determine the sub-block motion vector based on a linear weighted sum of one or more neighboring motion vectors in the planar MVP method. In this case, neighboring motion vectors can include motion information extrapolated from the motion vectors of neighboring blocks. For example, the encoder / decoder can use the motion vectors of neighboring blocks to perform extrapolation processing on the motion vectors below or to the right of the current block.

[0141] refer to Figure 17 The motion vector P(x,y) corresponding to the sub-block of the current block can be determined based on P_h(x,y) or P_v(x,y). Figure 17 In this context, W and H indicate the width and height of the current block, respectively. In this case, P_h(x,y) can be determined based on L(-1,y) or R(W,y). L(-1,y) can be the motion vector corresponding to the adjacent sub-block to the left of the current sub-block. R(W,y) can be the value based on the motion vector corresponding to the upper right side of the current block and the time motion vector corresponding to the lower right side of the current block. P_v(x,y) can be determined based on A(x,-1) or B(x,H). A(x,-1) can be the motion vector corresponding to the adjacent sub-block to the top of the current sub-block. B(x,H) can be the value obtained by weighted summation based on the motion vector corresponding to the lower left side of the current block and the upper right motion vector corresponding to the lower right side of the current block.

[0142] [Table 3] shows the syntax structure related to inter-frame prediction according to an embodiment of the present invention.

[0143] [Table 3]

[0144] According to embodiments of the present invention, the inter-frame prediction method may include skip mode, merge mode, inter-frame mode, etc. As an example, in skip mode, residual signals may not be transmitted. Furthermore, MV determination methods such as merge mode can be applied in skip mode. For example, whether to use skip mode can be determined based on a skip flag (or syntax element). Referring to [Table 3], whether to use skip mode can be determined based on the value of cu_skip_flag. cu_skip_flag indicates whether skip mode is applied to the current CU.

[0145] In this embodiment, motion vector difference may not be used when the merge mode is applied. In this case, the encoder / decoder can determine the motion vector based on the motion candidate index. For example, whether to use the merge mode can be determined based on the merge flag (or syntax element). Referring to [Table 3], whether to apply the merge mode can be determined based on the value of the merge flag (i.e., merge_flag). The merge mode can be used when the skip mode is not used.

[0146] According to embodiments of the present invention, the encoder / decoder may selectively use one or more candidate list sets in a skip mode or a merge mode. For example, the candidate list set may include a candidate list using merge candidates and / or a candidate list using sub-block-based merge candidates. Furthermore, merge candidates may include spatially adjacent candidates, temporally adjacent candidates, etc. Merge candidates may include candidates that apply motion vectors to the entire current block (CU). That is, motion vectors of various sub-blocks belonging to the current block may include the same candidate. Furthermore, sub-block merge candidates may include sub-block-based temporal motion vectors (MV), affine merge candidates, etc. According to embodiments, sub-block merge candidates may include candidates that apply difference motion vectors to each sub-block of the current block (CU). In embodiments, affine merge candidates may indicate merge candidates used to determine the CPMV for affine motion prediction without any motion vector differences. Furthermore, sub-block merge candidates may include methods for determining motion vectors on a sub-block basis within the current block. For example, sub-block merge candidates may include planar MV, regression-based MV, STMVP, etc., as well as sub-block-based temporal MV and affine merge candidates.

[0147] According to embodiments of the present invention, the encoder / decoder can use motion vector difference in inter-frame mode. A motion vector predictor can be determined based on a motion candidate index, and a motion vector can be determined based on the motion vector predictor and the motion vector difference. Whether to use an inter-frame mode can be determined based on information about whether other modes are used. As another embodiment, whether to use an inter-frame mode can be determined by a flag (or syntax element). In [Table 3], it is assumed that an inter-frame mode is used when other modes are not used (i.e., skip mode and merge mode). In embodiments, the inter-frame mode can include AMVP mode, affine inter-frame mode, etc. The inter-frame mode can be a mode used to determine motion vectors based on a motion vector predictor and the motion vector difference. In affine inter-frame mode, the encoder / decoder can use the motion vector difference when determining the CPMV of the affine motion prediction.

[0148] Referring to [Table 3], after determining the skip mode or merge mode, the encoder / decoder can determine whether to use a sub-block merge candidate or a merge candidate. For example, the decoder can parse the `merge_subblock_flag` indicating whether a sub-block merge candidate is used when a specific condition is met. In this case, the specific condition can be a condition related to the block size. For example, the condition can be related to width, height, area, etc., or it can be a condition that includes a combination of these.

[0149] Referring to [Table 3], for example, specific conditions can indicate whether the width and height of the current block (CU) are greater than or equal to specific values. When `merge_subblock_flag` is not resolved, the encoder / decoder can infer that value is 0. If `merge_subblock_flag` is 1, the encoder / decoder can use subblock merge candidates, and if `merge_subblock_flag` is 0, merge candidates can be used. When using subblock merge candidates, `merge_subblock_idx`, representing the index (or syntax element) indicating a specific candidate within the candidate list (subblock merge candidate list), can be resolved. When using merge candidates, `merge_index`, representing the index of a specific candidate within the candidate list (merge candidate list), can be resolved. When the maximum number of candidates in the candidate list is 1, the encoder / decoder may not perform resolution. That is, when the maximum number of candidates in the candidate list is greater than 1, the encoder / decoder can resolve the index. When `merge_subblock_idx` or the merge index is not resolved, the encoder / decoder can infer that value is 0.

[0150] [Table 3] illustrates the functions of the compilation unit, and details related to intra-frame prediction are omitted here. [Table 3] provides an example of the case where inter-frame prediction is applied.

[0151] The following describes a method for configuring a sub-block merging candidate list according to an embodiment of the present invention. [Table 3] has described the sub-block merging candidates, and the method for configuring the sub-block merging candidate list is described below. According to an embodiment of the present invention, the encoder / decoder can determine whether to add a candidate to the candidate list by considering the current block size or the sub-block size. The size can be defined based on the width, height, area, etc. of the sub-block.

[0152] In an embodiment, when multiple candidates exist that can belong to the sub-block merging candidate list, if the sub-block sizes of the multiple candidates are different, the size of the current block can be considered to determine whether to add a candidate to the candidate list. Furthermore, the condition for considering the size of the current block for any candidate can be based on the size of the candidate's sub-blocks. A candidate (before being added to the candidate list) can refer to an MVP method or pattern, or can include all methods that make an MVP or MV. A candidate added to the candidate list can refer to a candidate, MVP method, pattern, etc., that can be used with the current block.

[0153] According to embodiments of the present invention, the encoder / decoder can determine candidates to be added to the candidate list based on the block size condition of the current block. For example, the encoder / decoder can determine whether to add a candidate to the candidate list based on whether the size of the current block is greater than or equal to a threshold of any block size. Furthermore, the block size condition or threshold can be determined based on the sub-block size of any candidate. This is because in blocks larger than or sufficiently larger than their sub-blocks, compression efficiency can be improved through sub-block-based prediction. For example, in cases where the block is equal to or smaller than its sub-blocks, motion information may not be obtainable on a sub-block basis, and even if sub-block-based prediction is applied, it may not contribute to compression efficiency.

[0154] In an embodiment, when the size of the current block is equal to or less than a preset size, the encoder / decoder can configure a first candidate list, and when the size of the current block is greater than or equal to the preset size, it can configure a second candidate list. Furthermore, the types of candidates belonging to the second candidate list can include the types of candidates belonging to the first candidate list. For example, when the sub-block size of the first mode is configured to the first sub-block size and the sub-block size of the second mode is configured to the second sub-block size, if the current block is equal to the preset size, the encoder / decoder can add only the second mode to the candidate list, and if the current block is greater than the preset size, it can add both the first mode and the second mode to the candidate list. For example, the first sub-block size can be 8×8, and the second sub-block size can be 4×4. Additionally, the preset size can be 8×8. Alternatively, the preset size can have a width (or height) of 8. For example, the first mode can be SbTMVP, and the second mode can be an affine merge mode.

[0155] In the embodiment, the subblock-based merge candidate list may be configured (or generated) by the following method. In the following, first, i is configured to 0 (i = 0).

[0156] - If the value of (availableFlagSbCol && blockSizeCondition) is true, subblockMergeCandList[i++] is configured as SbCol. That is, SbCol may be added to the candidate list. subblockMergeCandList[i++] indicates the i++-th subblock merge candidate in the subblock merge candidate list, and SbCol indicates the subblock-based temporal merge candidate (or SbTMVP).

[0157] - If the value of (availableFlagA && i<MaxNumSubblockMergeCand) is true, subblockMergeCandList[i++] is configured as A. That is, candidate A (or the candidate at position A) may be added as the next candidate.

[0158] - If the value of (availableFlagB && i<MaxNumSubblockMergeCand) is true, subblockMergeCandList[i++] is configured as B. That is, candidate B may be added as the next candidate.

[0159] - If the value of (availableFlagConst1 && i <MaxNumSubblockMergeCand) is true, subblockMergeCandList[i++] is configured as Const1. That is, candidate Const1 may be added as the next candidate. Const1 indicates the first candidate among the configured candidates.

[0160] - If the value of (availableFlagConst2 && i <MaxNumSubblockMergeCand) is true, subblockMergeCandList[i++] is configured as Const2. That is, candidate Const2 may be added as the next candidate. Const2 indicates the second candidate among the configured candidates.

[0161] - If the value of (availableFlagConst3 && i < MaxNumSubblockMergeCand) is true, subblockMergeCandList[i++] is configured as Const3. That is, the candidate Const3 can be added as the next candidate. Const3 indicates the third candidate among the configured candidates.

[0162] - If the value of (availableFlagConst4 && i < MaxNumSubblockMergeCand) is true, subblockMergeCandList[i++] is configured as Const4. That is, the candidate Const4 can be added as the next candidate. Const4 indicates the fourth candidate among the configured candidates.

[0163] - If the value of (availableFlagConst5 && i < MaxNumSubblockMergeCand) is true, subblockMergeCandList[i++] is configured as Const5. That is, the candidate Const5 can be added as the next candidate. Const5 indicates the fifth candidate among the configured candidates.

[0164] - If the value of (availableFlagConst6 && i < MaxNumSubblockMergeCand) is true, subblockMergeCandList[i++] is configured as Const6. That is, the candidate Const1 can be added as the next candidate. Const6 indicates the sixth candidate among the configured candidates.

[0165] According to an embodiment of the present invention, the encoder / decoder adds the candidate SbCol to the candidate list subblockMergeCandList only when a predetermined block size condition is satisfied. In an embodiment, the block size condition for determining whether to add SbTMVP to the candidate list may be determined according to the subblock size of SbTMVP. For example, specifically, the block size condition may be (cbWidth >= 16 && cbHeight >= 16). Here, cbWidth denotes the width of the current coded block, and cbHeight denotes the height of the current coded block. In addition, A, B, Const1, Const2, Const3, Const4, Const5 and Const6 may be affine merge candidates. A and B may be affine inherited candidates, and ConstX (X is one of 1 to 6) may be affine configured candidates.

[0166] In another embodiment, the encoder / decoder can determine whether to parse candidate indices based on the block size condition of the current block. For example, when multiple candidates exist, if all candidates other than one of the multiple candidates are not added to the candidate list based on the block size condition and another condition, the encoder / decoder may not parse the candidate index. For example, when the number of candidates that can be added to the candidate list is 2, the encoder / decoder can determine whether to parse the candidate index based on the block size condition. For example, when one of the two candidates that can be added to the candidate list does not meet the block size condition, the encoder / decoder may not parse the candidate index.

[0167] Table 4 below illustrates the syntax related to inter-frame prediction according to an embodiment of the present invention.

[0168] [Table 4]

[0169] According to embodiments of the present invention, a sub-block mode can be used when the size of the current block is greater than the size of the sub-blocks. For example, when there are multiple candidates that can be added to the sub-block merge candidate list and the sub-block sizes of the multiple candidates are the same, a sub-block merge mode can be used if the size of the current block is greater than the size of the sub-blocks. In a detailed example, a sub-block mode can be used when the sub-block size is 8x8 and the size of the current block is greater than or equal to 16x16 (or the width or height is greater than or equal to 16). The use of a sub-block mode can indicate whether a flag indicating whether the mode corresponding to the sub-block mode is used is resolved. Furthermore, the flag indicating whether the mode corresponding to the sub-block mode is used can be merge_subblock_flag, inter_affine_flag, etc. In [Table 4], descriptions that are repeated in [Table 3] are omitted.

[0170] According to an embodiment of the present invention, referring to [Table 4], the decoder can only parse merge_subblock_flag and inter_affine_flag when cbWidth>=16 and cbHeight>=16. As another embodiment, the encoder / decoder can use subblock mode even when the current block size is equal to the subblock size, and when the current block size is greater than the subblock size. This is because the motion compensation method in subblock mode may differ from other modes. In this specification, the subblock-based merging mode may be referred to as subblock mode.

[0171] Table 5 below illustrates the syntax related to inter-frame prediction according to an embodiment of the present invention.

[0172] [Table 5]

[0173] According to embodiments of the present invention, a sub-block mode can be used when the width of the current block meets a specific condition or the height of the current block meets a specific condition. For example, this is because if the width or height is greater than the size of one side (or the other side) of the sub-block size, multiple sub-blocks may exist within the current block even when the width or height of the current block is equal to the size of one side of the sub-block. Therefore, a sub-block mode can be used when the width of the current block is greater than the width of the sub-block, or when the height of the current block is greater than the height of the sub-block. Alternatively, a sub-block mode can be used when the width of the current block is greater than or equal to the width of the sub-block, or when the height of the current block is greater than or equal to the height of the sub-block. Descriptions repeated in [Table 5] from [Table 3] and [Table 4] are omitted. Referring to [Table 5], the encoder / decoder can use a sub-block merging mode when the width of the current block is greater than or equal to 8 or the height of the current block is greater than or equal to 8. The decoder can use an affine inter-frame mode when the width of the current block is greater than or equal to 16 or the height of the current block is greater than or equal to 16.

[0174] Tables 6 and 7 below illustrate the syntax related to sub-block merging according to embodiments of the present invention.

[0175] [Table 6]

[0176] [Table 7]

[0177] According to embodiments of the present invention, a signal can be sent indicating whether a specific mode is available in the first unit. When availability is indicated by signaling, the encoder can send information about which mode is actually used in the second unit belonging to the first unit to the decoder. In this case, it can be determined whether to parse syntax elements in the second unit based on the value signaled in the first unit. For example, the first unit can be a sequence, image, slice, tile, etc., while the second unit can be a CUT, CU, etc. In embodiments, the specific mode can be a sub-block mode. The first unit can be referred to as the higher unit, and the second unit can be referred to as the lower unit.

[0178] [Table 6] illustrates the signaling parsing process, which indicates whether to use a sub-block mode when all sub-block sizes of a mode belonging to a sub-block mode are the same or when differences between sub-block sizes are not considered. According to an embodiment of the invention, when there are multiple modes belonging to a sub-block mode and multiple signals indicating whether higher units corresponding to the sub-block mode are available, the sub-block mode can be used in lower units if at least one of the signals indicating whether higher units are available indicates availability. Furthermore, the signal indicating whether higher units are available can correspond to multiple modes. For example, when different modes exist based on the location of referenced adjacent blocks, information indicating whether higher units for these modes are available can be signaled. In this case, this can be combined with conditions such as block size conditions. In this specification, the availability of a sub-block mode can refer to parsing the signal indicating whether a sub-block mode is available, and reference can be made to the embodiments described above.

[0179] Referring to [Table 6], the decoder can only parse merge_subblock_flag if at least one of sps_affine_enabled_flag and sps_SbTMVP_enabled_flag, which indicate whether the affine merge mode and SbTMVP are available, is 1.

[0180] Referring to [Table 7], when the sub-block sizes of patterns belonging to the sub-block mode are different, the decoder can take this into account when parsing the syntax element indicating whether to use the sub-block mode. According to an embodiment of the invention, the encoder / decoder can use the sub-block mode when both the signaling indicating whether a higher unit of any pattern belonging to the sub-block mode is available and the block size condition of that pattern are satisfied. For example, when there are a first mode and a second mode belonging to the sub-block mode, using a first signaling indicating whether a higher unit corresponding to the first mode is available and a second signaling indicating whether a higher unit corresponding to the second mode is available, and given a first block size condition for the first mode and a second block size condition for the second mode, if the first signaling indicating whether a higher unit is available indicates that a higher unit is available and the first block size condition is satisfied, the encoder / decoder can use the sub-block mode. When the second signaling indicating whether a higher unit is available indicates that a higher unit is available and the second block size condition is satisfied, the encoder / decoder can use the sub-block mode.

[0181] As an example, referring to [Table 7], the syntax elements sps_affine_enabled_flag and sps_SbTMVP_enabled_flag, indicating whether the affine merge mode and SbTMVP are available, can be signaled from the encoder to the decoder. In this case, the decoder can parse merge_subblock_flag when the following [Equation 15] is satisfied. Similarly, the encoder can perform syntax signaling on merge_subblock_flag when [Equation 15] below is satisfied.

[0182] [Equation 15]

[0183] ((sps_affine_enabled_flag&& Affine merge block size condition) || (sps_SbTMVP_enabled_flag&&SbTMVP block size condition))

[0184] [Table 7] shows the cases where the block size condition for the affine merge mode is (cbWidth>= 8 &&cbHeight>= 8) and the block size condition for SbTMVP is (cbWidth>= 16 &&cbHeight>= 16).

[0185] Table 8 below shows another example of the syntax for indicating the use of the conditions in Table 7.

[0186] [Table 8]

[0187] When there is a first mode and a second mode belonging to the sub-block mode, there is a first signaling indicating whether the higher unit corresponding to the first mode is available and a second signaling indicating whether the higher unit corresponding to the second mode is available, as well as a first block size condition for the first mode and a second block size condition for the second mode. If the first block size condition belongs to the second block size condition (i.e., the first block size condition is satisfied and the second block size condition is satisfied), the encoder / decoder can use the sub-block mode, as shown in [Equation 16] below.

[0188] [Equation 16]

[0189] If ((first signaling indicating whether a higher cell is available || second signaling indicating whether a higher cell is available) && (first block size condition))

[0190] Sub-block mode is available

[0191] Otherwise, if ((second signaling indicating whether the higher unit is available) && (second block size condition))

[0192] Sub-block mode is available

[0193] In another embodiment, there may be signaling indicating whether higher units corresponding to all sub-block modes are available. Alternatively, there may be signaling indicating whether higher units corresponding to all sub-block merge modes are available. In this case, the decoder can only parse lower unit signaling if the higher unit is indicated to be available. According to embodiments of the invention, the encoder / decoder can configure block size conditions using sub-block modes based on the sub-block size. In this case, when the sub-block size is variable, the encoder / decoder can configure the block size conditions based on the larger of the available sub-block sizes.

[0194] Tables 9 and 10 below illustrate the syntax related to sub-block merging according to embodiments of the present invention.

[0195] [Table 9]

[0196] [Table 10]

[0197] According to embodiments of the present invention, a signal can be sent indicating whether a specific mode is available. When availability is indicated by signaling, the encoder can signal the decoder with information about which mode is actually used in the second unit belonging to the first unit. In this case, it can be determined whether to parse syntax elements in the second unit based on the value signaled in the first unit. For example, the first unit can be a sequence, image, slice, patch, etc., while the second unit can be a CUT, CU, etc. In embodiments, the specific mode can be a sub-block mode. The first unit can be referred to as the higher unit, and the second unit can be referred to as the lower unit. Furthermore, the signaling indicating the availability of the higher unit can correspond to multiple modes. For example, when different modes exist depending on the position of the adjacent block to be referenced, there can be signaling indicating whether the higher unit used for multiple modes is available.

[0198] According to embodiments of the present invention, signaling indicating whether a higher unit is available can be referenced to determine whether a candidate index should be parsed. For example, a candidate index can be parsed when one or more of the signaling indicating whether a plurality of higher units are available is available. According to embodiments, there may be signaling indicating whether higher units corresponding to a plurality of patterns (in this case, the pattern may indicate units included as candidates in a candidate list) are available. In this specification, signaling for indicating whether higher units corresponding to a plurality of patterns are available can be indicated by a first flag, a second flag, a third flag, etc. Furthermore, there may be signaling indicating whether a higher unit (candidate; a unit that maintains a position in the candidate list) corresponding to a single pattern is available. In this specification, the signaling indicating whether a higher unit corresponding to a single pattern is available is referred to as flag A, flag B, or flag C.

[0199] According to an embodiment of the invention, if at least one of the signaling indicating whether a higher unit corresponding to a plurality of modes is available indicates availability, the encoder / decoder can parse the candidate index. That is, in the case of (first flag || second flag || third flag, ...) (i.e., flag value 1 indicates availability), the encoder / decoder can parse the candidate index.

[0200] When the number of signaling messages indicating whether a higher unit is available for a given mode is 0 or 1, or when the number of modes without signaling indicating whether a higher unit is available is 0 or 1, the encoder / decoder can resolve a candidate index if at least one indication of signaling indicating whether a higher unit is available for multiple modes is available. That is, in the case of (first flag || second flag || third flag, ...) (flag value 1 indicates availability), the encoder / decoder can resolve a candidate index.

[0201] When the number of signaling messages indicating whether a higher unit corresponding to a mode is available is 1, and the number of modes without signaling indicating whether a higher unit is available is 1, the encoder / decoder can resolve a candidate index if the signaling indication of a higher unit corresponding to a mode is available. In this case, this can be independent of the value of the signaling messages indicating whether higher units corresponding to multiple modes are available. Therefore, in this case, the encoder / decoder can resolve a candidate index when at least one of the multiple signaling messages indicating whether higher units corresponding to multiple modes are available and the signaling message indicating whether a higher unit corresponding to a single mode is available is available. That is, in the case of (first flag || second flag || third flag, ...) (flag value 1 indicates availability), the encoder / decoder can resolve a candidate index.

[0202] When no candidate index is resolved, the encoder / decoder can use the only candidate in the candidate list. Alternatively, when no candidate index is resolved, the encoder / decoder can infer that the candidate index is 0.

[0203] Referring to [Table 9], affine merge mode and SbTMVP can correspond to subblock merge mode, and affine merge mode can be configured as multiple candidates using this method. The availability of the higher unit corresponding to the affine merge mode can be signaled via the syntax element of sps_affine_enabled_flag. In this case, when sps_affine_enabled_flag indicates that the higher unit is available, the encoder / decoder can resolve the merge_subblock_idx corresponding to the candidate index.

[0204] According to an embodiment of the invention, if the number of signaling messages indicating whether a higher unit corresponding to a mode is available is two or more, and at least two of the multiple signaling messages indicating whether a higher unit corresponding to a mode is available indicate that a higher unit is available, then the encoder / decoder can resolve the candidate index. That is, in the case of (flag A + flag B + flag C + ... >= 2) (flag value 1 indicates availability), the encoder / decoder can resolve the candidate index. In this case, there may be modes that do not have signaling indicating whether a higher unit can be used.

[0205] When there are two or more signaling messages indicating whether a higher unit corresponding to a mode is available, and there are signaling messages indicating whether a higher unit corresponding to multiple modes is available, the encoder / decoder can resolve the candidate index if at least two of the signaling messages indicating whether a higher unit corresponding to a mode is available indicate that a higher unit is available, or if at least one of the signaling messages indicating whether a higher unit corresponding to multiple modes is available indicates that a higher unit is available. That is, in the case of (((first flag || second flag || third flag,...) || (flag A + flag B + flag C +....>=2)) (flag value 1 indicates availability), the encoder / decoder can resolve the candidate index. In this case, there may be modes that do not have signaling indicating whether a higher unit is available.

[0206] In the case where the number of modes indicating whether a higher unit is available is 1, a candidate index can be resolved when at least one of the multiple signaling signals indicating whether a higher unit corresponding to a mode is available indicates that a higher unit is available, rather than when at least two of the multiple signaling signals indicating whether a higher unit corresponding to a mode is available indicate that a higher unit is available. That is, the encoder / decoder can resolve a candidate index in the case of (flag A || flag B || flag C + ...), rather than in the case of (flag A + flag B + flag C + ... >= 2).

[0207] Referring to Table 10, subblock merging modes can include affine merging mode, SbTMVP, and planar MVP. Affine merging mode can be configured as multiple candidates. SbTMVP can be configured as one candidate. Planar MVP can be configured as one candidate. The syntax elements of sps_affine_enabled_flag, sps_SbTMVP_enabled_flag, and sps_planar_enabled_flag can be used to signal whether the higher unit corresponding to the affine merging mode, SbTMVP, and planar MVP is available. In this case, when sps_affine_enabled_flag indicates that the higher unit is available, or when both sps_SbTMVP_enabled_flag and sps_planar_enabled_flag indicate that the higher unit is available (or two or more of sps_SbTMVP_enabled_flag and sps_planar_enabled_flag indicate that the higher unit is available), the encoder / decoder can resolve the merge_subblock_idx corresponding to the candidate index. In this invention, when both sps_SbTMVP_enabled_flag and sps_planar_enabled_flag indicate that a higher unit is available, it can be represented as (sps_SbTMVP_enabled_flag + sps_planar_enabled_flag >= 2).

[0208] The availability conditions of SbTMVP according to embodiments of the present invention can be derived by the following method. In the following text, availableFlagSbCol indicates a parameter (or flag) indicating whether SbTMVP is available.

[0209] - If one or more of the following conditions are true, availableFlagSbCol can be configured to 0.

[0210] 1) tile_group_temporal_mvp_enable_flag is 0

[0211] 2) sps_SbTMVP_flag is 0

[0212] 3) cbWidth is less than 8

[0213] 4) cbHeight is less than 8

[0214] According to embodiments of the present invention, the availability of SbTMVP can be indicated by an availability flag. The availability flag indicating the availability of SbTMVP can be availableFlagSbCol. Furthermore, SbTMVP can also be referred to as SbCol. Additionally, an availability flag with a value of 1 indicates that SbTMVP is available, while an availability flag with a value of 0 indicates that SbTMVP is unavailable.

[0215] According to embodiments of the present invention, there may be conditions under which SbTMVP is not used. As an example, SbTMVP is unavailable based on the aforementioned higher unit signaling. The higher unit used for the current block, CU, or PU can be a slice, patch, patch group, sequence, image, CTU, etc. Higher unit signaling may include `tile_group_temporal_mvp_enable_flag`. For example, `tile_group_temporal_mvp_enable_flag` indicates whether temporal motion vectors (or temporal motion vector predictions) are available. Since SbTMVP can be a technique using temporal motion vectors, SbTMVP can be not used if temporal MV is not configured to be available. Furthermore, higher unit signaling may include `sps_SbTMVP_flag`. For example, `sps_SbTMVP_flag` can be a flag indicating whether SbTMVP is available.

[0216] In another embodiment, SbTMVP may not be used based on block size. Block size can include block width, block height, or a value based on block width or height. Furthermore, the value based on block width or height can include the area of ​​the block. SbTMVP may not be used when the block width or height is less than a threshold. As an example, the threshold can be the width or height of a sub-block. For example, the threshold can be configured to 8.

[0217] When the conditions preventing the use of SbTMVP are met, the encoder / decoder can configure the availability flag corresponding to SbTMVP to 0. Otherwise, the availability flag can be determined through an append operation.

[0218] Figure 18The illustration depicts a method for determining a temporary motion vector according to an embodiment of the present invention. In this specification, the motion vector indicating a juxtaposed block is referred to as a temporary motion vector. The temporary motion vector is not limited to its name and may be referred to as an initial motion vector, a temporal motion vector, a sub-block-based merging basic motion vector (sub-block-based temporal merging basic MV), or a basic motion vector. That is, the encoder / decoder may determine the motion vector of a specified juxtaposed block based on the initial motion vector, the temporal motion vector, the sub-block-based merging basic motion vector, or the basic motion vector.

[0219] According to embodiments of the present invention, the encoder / decoder can determine temporary motion vectors based on motion information of the locations surrounding the current block. Furthermore, the locations surrounding the current block can be preset locations. The locations surrounding the current block can include spatial candidate locations for merging candidates. Alternatively, the locations surrounding the current block can include spatial candidate locations for AMVP candidates.

[0220] refer to Figure 18 The positions corresponding to A0, A1, B0, B1, and B2 can be preset. Furthermore, these positions can be brightness positions. When the top-left position of the current block is (xCb, yCb), A0, A1, B0, B1, and B2 can be (xCb - 1, yCb + cbHeight), (xCb - 1, yCb + cbHeight - 1), (xCb + cbWidth, yCb - 1), (xCb + cbWidth - 1, yCb - 1), and (xCb - 1, yCb - 1), respectively. In this case, cbWidth and cbHeight can be the width and height of the current block, respectively. (xCb, yCb) can be the top-left position within a shared merge list area. When multiple blocks use the same merge candidate list, the areas of the multiple blocks can be a shared merge list area. Additionally, a preset reference order corresponding to the preset positions can exist. [Table 11] shows the preset reference order.

[0221] [Table 11]

[0222] Referring to [Table 11], the preset positions can be predefinedPosition1, predefinedPosition2, ..., predefinedPositionN, etc. The reference order can be preset as predefinedPosition1, predefinedPosition2, ..., and predefinedPositionN. The reference order can be the forward or reverse of the reference order constructed from the merged candidate list. Alternatively, the reference order can be the forward or reverse of the reference order constructed from the AMVP candidate list. For example, the reference positions and reference order can be A1, B1, B0, and A0. Alternatively, predefinedPosition1, predefinedPosition2, ..., and predefinedPositionN can indicate motion information corresponding to the preset positions.

[0223] In [Table 11], temporary motion vectors can be configured as zero motion vectors. If motion information for a preset position is unavailable, the encoder / decoder can configure the temporary motion vector as a zero motion vector. When motion information for a preset position is available, the motion information for that position can be configured as a temporary motion vector. Availability can include cases where the corresponding position is not intra-frame predicted. Alternatively, availability can include cases where the corresponding position is inter-frame predicted. Alternatively, availability can include cases where the reference image for the motion information at the corresponding position is the same as the juxtaposed image corresponding to the current block. Alternatively, availability can include cases where the reference image for the motion information at the corresponding position is the same as the reference image corresponding to the current block.

[0224] In [Table 11], it can be identified whether the motion information of the first corresponding position in the preset positions that meets the first availability condition meets the second availability condition. When the second availability condition is met, the encoder / decoder can configure the motion information of the corresponding position as a temporary motion vector. In other cases, the encoder / decoder can configure the temporary motion vector as a zero motion vector. This can be represented by pseudocode, as shown in [Table 12] below.

[0225] [Table 12]

[0226] In this embodiment, to allow only the first position among preset positions that satisfy the first availability condition to be used as a temporary motion vector, the else if statement in [Table 12] can be used. Furthermore, the first availability condition and the second availability condition can be the availability conditions described above. For example, the first availability condition could mean that the corresponding position uses inter-frame prediction. The second availability condition could mean that the reference image for the motion information of the corresponding position is the same as the juxtaposed image corresponding to the current block. In this embodiment, when any position satisfies the first availability condition but not the second availability condition, the referenced position after that position may not be identified.

[0227] In another embodiment, the encoder / decoder can configure motion information of a first corresponding position that satisfies both a first availability condition and a second availability condition within a preset position as a temporary motion vector. In other cases, the encoder / decoder can configure the temporary motion vector as a zero motion vector. This can be represented in pseudocode, as shown in [Table 13] below.

[0228] [Table 13]

[0229] In [Table 13], if any position satisfies the first availability condition but not the second availability condition, the encoder / decoder can also identify the referenced position after that position.

[0230] According to an embodiment of the present invention, the number of preset positions can be as low as one. As shown in [Table 14] below, this can be represented by pseudocode.

[0231] [Table 14]

[0232] That is, when motion information corresponding to a preset position is available, the encoder / decoder can configure a temporary motion vector as the corresponding motion information. Otherwise, the encoder / decoder can configure the temporary motion vector as a zero motion vector. As an example, a preset position can be the left position of the current block. That is, the x-coordinate of a preset position can be less than the left coordinate within the current block. For example, a preset position can be position A1.

[0233] Although examples of using both a first availability condition and a second availability condition have been described in the embodiments, the invention is not limited thereto and may include cases where more availability conditions exist. Motion information may include motion vectors (mvLX), reference indices (refIdxLX), predicted utilization flags (predFlagLX), etc.

[0234] In this embodiment, the process of identifying the second availability condition and configuring a temporary motion vector can be performed as follows: That is, if availableFlagN is true, the following processing can be applied.

[0235] A. The temporary motion vector can be configured as mvL1N if all of the following conditions are true.

[0236] a. The case where predFlagL1N is 1

[0237] b. The case where DiffPicOrderCnt(ColPic, RefPcList1[refIdxL1N]) is 0

[0238] C. If DiffPicOrderCnt(aPic, currPic) is equal to or less than 0, then aPic indicates all images in the list of all reference images for the slice (or patch group).

[0239] d. Case where tile_group_type is B

[0240] e. Case where collocated_from_10_flag is 0

[0241] B. Otherwise, if all of the following conditions are true, the temporary motion vector can be configured as mvL0N.

[0242] a. predFlagL0N is 1

[0243] b. The case where DiffPicOrderCnt(ColPic, RefPicList0[refIdxL0N]) is 0

[0244] When availableFlagN is true, it may indicate that the first availability condition is met. In embodiments where only a preset location is used, the parameter availableFlagN may be availableFlag corresponding to that preset location. For example, when only location A1 is used, availableFlagN may be availableFlagA1, which may be a value indicating whether motion information for location A1 is available.

[0245] Figure 19 The illustration shows whether SbTMVP is used according to an embodiment of the present invention. According to an embodiment of the present invention, the encoder / decoder can determine whether to use a specific mode based on the position of the current block 1901. According to another embodiment of the present invention, the encoder / decoder can determine whether to use a specific mode based on the position of the merge list shared region to which the current block 1901 belongs.

[0246] According to an embodiment of the present invention, the encoder / decoder can determine whether SbTMVP is used based on the position of the current block 1901. According to another embodiment of the present invention, the determination of whether SbTMVP is used can be based on the position of the merge list shared region to which the current block 1901 belongs. As an example, the position of the current block 1901 or the position of the merge list shared region to which the current block belongs can be a top-left coordinate or can be represented by a top-left coordinate. Furthermore, the position of the current block 1901 or the position of the merge list shared region to which the current block belongs can be a relative position within a higher group. Higher groups can include images, patches, patch groups, CTUs, slices, etc. Furthermore, a higher group can be the group to which the current block belongs. That is, for example, the encoder / decoder can determine whether SbTMVP is used based on the position within image 1902 of the current block 1901. In this specification, the current block or the merge list shared region of the current block can be referred to as the current region.

[0247] According to embodiments of the present invention, the encoder / decoder can determine whether to use SbTMVP based on whether the current region is adjacent to the boundary of a higher group. For example, when the current region is adjacent to the boundary of a higher group, it can be determined or determined that SbTMVP is not used. Alternatively, it can be based on the determination of whether to use SbTMVP in the context of a higher group. Figure 19 The use of SbTMVP is determined by whether the preset positions referenced for the temporary motion vector corresponding to the current region are outside the higher group. For example, when all preset positions referenced for determining the temporary motion vector corresponding to the current region are outside the higher group, the encoder / decoder can determine or decide not to use SbTMVP. Furthermore, the determination of whether all preset positions referenced for determining the temporary motion vector corresponding to the current region are outside the higher group can be based on the coordinates of the current region. This is because when all preset positions referenced for determining the temporary motion vector corresponding to the current region are outside the higher group, a zero motion vector can be used as the temporary motion vector, thus potentially degrading the prediction performance of SbTMVP. In an embodiment, in this case, by determining that SbTMVP is unavailable, a smaller number of bits can be used to signal another mode signaled via the same syntax elements as SbTMVP. For example, as mentioned above, SbTMVP and affine merging modes can be included in a sub-block merging mode, and when it is determined that SbTMVP is unavailable, a smaller number of indices can be used to signal the affine merging mode. Therefore, in this case, index 0 can always indicate the affine merging mode.

[0248] refer to Figure 19As described above, the temporary motion vector for the current region can be determined by referring only to position A1 1903. When motion information at position A1 1903 is unavailable, a zero motion vector can be used as the temporary motion vector. If the current region is adjacent to the left boundary of image 1902, the temporary motion vector can always be a zero motion vector. Therefore, SbTMVP is not used when the current region is adjacent to the left boundary of image 1902. Furthermore, SbTMVP is not used when the current region is adjacent not only to the left boundary of image 1902 but also to the left boundary of a unit that can be processed in parallel. Units that can be processed in parallel can be patches, patch groups, slices, etc. Therefore, when the top-left coordinate of the current region is (xCb, yCb) and xCb is 0, the encoder / decoder can be adjacent to the left boundary of image 1902, and in this case, SbTMVP is determined not to be used. Alternatively, SbTMVP is not used when (xCb - (left coordinate of the unit that can be processed in parallel)) is 0.

[0249] In an embodiment, if the temporary motion vector of the current region is determined by referring only to the upper side of the current region, the encoder / decoder can be configured not to use SbTMVP if the current region is adjacent to the upper boundary of image 1902 or a unit that can be processed in parallel.

[0250] In another additional embodiment, although the conditions for determining or deciding not to use SbTMVP have been described, in addition to these conditions, SbTMVP is only not used when conditions related to block size are met. That is, for example, SbTMVP is not used when the coordinates of the current region meet both a preset condition and a condition related to block size. As an example, the condition related to block size may correspond to the case where the block size of the current block is equal to or less than the sub-block size. When the current block size is equal to or less than the sub-block size, the number of sub-blocks of the current block is only 1, and in this case, the advantages and performance of SbTMVP may be reduced. For example, the sub-block size may be 8×8. Therefore, the following embodiment can be applied to the above embodiment. If the current region is adjacent to the left boundary of the image, and the current block is equal to or less than 8x8, the encoder / decoder can determine or decide not to use SbTMVP. However, there may be embodiments where the sub-block merging mode can only be used in blocks greater than or equal to 8×8. In this case, when the current region is adjacent to the left boundary of image 1902 and the current block 1901 is 8×8, it can be determined and decided not to use SbTMVP.

[0251] Furthermore, in another embodiment of the invention, the availability flag indicating the availability conditions of SbTMVP may include whether xCb is 0, whether the width and height of the current block are both 8, and the aforementioned conditions. That is, when xCb is 0 and the width and height of the current block are both 8, the availability flag can be configured to 0. When this condition or all conditions are met, or when one or more of these conditions are met, the encoder / decoder can configure the availability flag to 0. As described above, the availability flag can be used to indicate whether SbTMVP is available. Furthermore, when any condition is met, SbTMVP can be indicated as unavailable. That is, when any condition is met, the encoder / decoder can configure availableFlagSbCol to 0. Therefore, in reference... Figure 9 In the illustrated embodiment, the encoder / decoder can configure availableFlagSbCol based on the location of the current region. That is, the encoder / decoder can configure availableFlagSbCol based on the coordinates (xCb, yCb) indicating the current region. For example, (xCb, yCb) could be the top-left coordinates of the current region.

[0252] As in Figure 19 As described above, SbTMVP may be unavailable when the current region or block is adjacent to the left edge of the image. Therefore, if xCb is 0, the encoder / decoder can configure affordableFlagSbCol to 0. For example, if the current region or block is adjacent to the left edge of the image and the current block is 8x8, SbTMVP may be unavailable. Therefore, if xCb is 0 and the width and height of the current block are both 8, the encoder / decoder can configure affordableFlagSbCol to 0.

[0253] Table 15 below illustrates the sub-block merging pattern syntax structure according to an embodiment of the present invention.

[0254] [Table 15]

[0255] As described above, sub-block merging modes (or sub-block-based merging modes) can include SbTMVP and / or affine merging modes. The encoder can signal the decoder with a higher-level syntax, indicating the sps_affine_enabled_flag corresponding to the syntax element (or flag) indicating whether the affine mode is available. As an example, the higher level can be at least one of the sequence, image, slice, or CTU level. Furthermore, in an embodiment, it can be as follows... Figure 19 As described above, predefined conditions for SbTMVP to be unavailable (or inapplicable) are provided.

[0256] According to an embodiment of the invention, when the SbTMVP unavailability condition and affine mode are unavailable, the encoder / decoder may not parse merge_subblock_flag. For example, merge_subblock_flag may be a signaling indication of using subblock merging mode. If merge_subblock_flag does not exist, the encoder / decoder can infer that its value is 0. The case where affine mode is unavailable can be when sps_affine_enabled_flag is 0. Therefore, when SbTMVP is available or affine mode is available, merge_subblock_flag can be parsed. The conditions for parsing merge_subblock_flag may include additional conditions as well as the conditions described above. For example, referring to [Table 15], in order to determine the parsing of merge_subblock_flag, conditions related to MaxNumSubblockMergeCand indicating the maximum number of subblock merging candidates and conditions related to block size may also be considered.

[0257] Referring to [Table 15], merge_subblock_flag can be skipped when x0, which is the x-value of the top-left coordinate of the current block, is 0 and sps_affine_enabled_flag, which indicates whether affine mode is enabled, is 0. Merge_subblock_flag can be skipped when x0 is not 0 or sps_affine_enabled_flag is 1.

[0258] In the above embodiments, the description has been based on the assumption that SbTMVP and affine merging modes are included in the sub-block merging mode. However, the invention is not limited to this, and another mode may be included in the sub-block merging mode. For example, in addition to SbTMVP, the sub-block merging mode may also include a first mode and a second mode. Furthermore, the syntax elements sps_mode1_enabled_flag and sps_mode2_enabled_flag, indicating whether the first mode and the second mode are available, can be signaled. In this case, when the SbTMVP unavailability condition is met and both the first mode and the second mode are unavailable, the encoder / decoder may not parse merge_subblock_flag. In this case, the encoder / decoder can infer that merge_subblock_flag is 0. As an example, if the SbTMVP unavailability condition is not met, the first mode is available, or the second mode is available, the encoder / decoder may parse merge_subblock_flag. The conditions for parsing merge_subblock_flag may include additional conditions as well as the conditions described above. Referring to [Table 15], in order to determine the resolution of merge_subblock_flag, conditions related to MaxNumSubblockMergeCand, which indicates the maximum number of candidate subblocks to be merged, and conditions related to block size can also be considered.

[0259] According to an embodiment of the invention, when the affine mode is not configured to be available, the encoder / decoder can configure MaxNumSubblockMergeCand to 1. Alternatively, when the affine mode is not configured to be available and SbTMVP is configured to be available, the encoder / decoder can configure MaxNumSubblockMergeCand to 1. Furthermore, when the affine mode is not configured to be available and SbTMVP is not configured to be available, the encoder / decoder can configure MaxNumSubblockMergeCand to 0.

[0260] Figure 20 The illustration shows a method for deriving SbTMVP according to an embodiment of the present invention. According to an embodiment of the present invention, the preset positions referenced for determining the temporary motion vector can be variable. Alternatively, the number of preset positions referenced for determining the temporary motion vector of a particular block can be configured to 1, and these positions can be variable.

[0261] In an embodiment, the reference position used to determine the temporary motion vector based on the position of the current block 2001 or the current region can be determined to be variable. Current region indication. Figure 19The region described in the text. For example, motion information at a specific location is always unavailable when the current region is adjacent to the boundary of image 2002 or the boundary of a unit that can be processed in parallel. Therefore, the encoder / decoder can configure a location that is not always unavailable as a preset location referenced for determining temporary motion vectors. That is, the preset location referenced for determining temporary motion vectors can be configured as a location that is not always unavailable based on the location of the current region. A location that is always unavailable can be a location outside of image 2002 or the range that can be processed in parallel.

[0262] according to Figure 18 In the described embodiment, the preset position referenced for determining the temporary motion vector can be defined as the left position of the current block 2001 or the current region. In this case, as... Figure 19 As described above, when the current block 2001 or the current region is located at a specific position, the motion information of the preset position is unavailable. Therefore, the preset position can be configured (or determined) to be a position that is not the left position. For example, the preset position can be configured to the upper position 2003 of the current block 2001 or the current region. For example, the preset position could be... Figure 18 Position B1.

[0263] Furthermore, in the embodiments, reference is made to Figure 20 When the current block 2001 or the current region is adjacent to the left boundary of image 2002 or the left boundary of a unit that can be processed in parallel, the preset position referenced for determining the temporary motion vector can be determined as the upper position of the current block 2001 or the current region. The upper position of the current region can be a position with a y-coordinate smaller than the y-coordinate of the current region (i.e., the vertical coordinate). For example, when the current block 2001 or the current region is adjacent to the left boundary of image 2002 or the left boundary of a unit that can be processed in parallel, the preset position referenced for determining the temporary motion vector can be... Figure 18 Position B1.

[0264] Alternatively, in an embodiment, when the current block 2001 or the current region is adjacent to the upper boundary of the image 2002 or the upper boundary of a unit that can be processed in parallel, the preset position referenced for determining the temporary motion vector can be determined as the left position of the current block 2001 or the current region. The left position of the current region can be a position with an x-coordinate smaller than the x-coordinate of the current region (i.e., the horizontal coordinate). For example, when the current block 2001 or the current region is adjacent to the upper boundary of the image 2002 or the upper boundary of a unit that can be processed in parallel, the preset position referenced for determining the temporary motion vector can be... Figure 18 Position A1.

[0265] Sub-block-based motion compensation methods have been described. Below, a method combining an embodiment of motion compensation based on merging patterns using motion vector differences (merging patterns using MVD (MMVD)) (or merging MVD) and sub-block-based motion compensation is described.

[0266] Figure 21 The illustration depicts an MMVD application method according to an embodiment of the present invention. According to an embodiment of the present invention, the encoder / decoder can determine the motion vector (MV) based on the motion vector predictor (MVP) and the motion vector difference (MVD). In this specification, the MVP can be referred to as the base motion vector (baseMV). That is, the encoder / decoder can derive the motion vector (i.e., the final motion vector) by adding the base motion vector and the motion vector difference. However, the present invention is not limited to these names, and the MVP can be referred to as the base motion vector, temporary motion vector, initial motion vector, MMVD candidate motion vector, etc. The MVD can be represented by values ​​used to refine the MVP and can be referred to as refined motion vector (refineMV) or merged motion vector difference.

[0267] According to embodiments of the present invention, when MMVD is applied, i.e., in MMVD mode, the motion vector (MV) can be determined based on a basic motion vector, a distance parameter (or variable), or a direction parameter (or variable). Furthermore, according to embodiments of the present invention, the basic motion vector can be determined from a candidate list. For example, the basic motion vector can be determined from a merged candidate list. The encoder / decoder can determine the basic motion vector from some candidates in another candidate list. Some candidates in the candidate list can be the beginning of the candidate list (the portion with a smaller index). For example, the encoder / decoder can use the first and second candidates from the merged candidate list to determine the basic motion vector. For this purpose, the candidate index indicating a specific candidate among the two candidates can be signaled from the encoder to the decoder. (See reference...) Figure 21 A basic candidate index can be defined, which is used to signal the basic motion vector. The encoder / decoder can determine, based on the basic candidate index, which candidate from the candidate list is applicable to the current block, and then designates the motion vector of the determined candidate as the basic motion vector. In this invention, the basic candidate index is not limited to this name and may also be referred to as basic candidate index, candidate index, candidate flag, MMVD index, MMVD candidate index, MMVD candidate flag, etc.

[0268] According to embodiments of the present invention, it is possible to exist with Figure 6 and 7 The MVD described in [the document] is different from the MVD described elsewhere. For example, the MVD in MMVD can be compared with [other MVDs]. Figure 6 and 7The MVD described in the specification is defined differently. In this specification, MMVD may indicate a merging mode (i.e., motion compensation mode or method) using motion vector difference, or it may indicate motion vector difference when MMVD is applied. For example, the encoder / decoder may determine whether to apply (or use) MMVD. If MMVD is applied, the encoder / decoder can derive merging candidates for inter-frame prediction of the current block from the merging candidate list, and determine the motion vector of the current block by deriving MMVD and applying (or adding) MMVD to the motion vectors of the merging candidates.

[0269] In embodiments, different MVDs can be simplified MVDs, MVDs with different (or smaller) resolutions, MVDs with a smaller number of available MVDs, MVDs with different signaling methods, etc. For example, although used in Figure 6 and 7 The MVD used in conventional AMVP, affine inter-frame modes, etc., as described in the text can indicate all regions at uniform intervals on the x and y axes (i.e., horizontal and vertical directions) for a specific signaling unit (e.g., x-pel). This could be a picture-based region (e.g., a picture region or a region including the picture or its surrounding area). However, the MMVD can have a relatively limited number of units for a specific signaling. Furthermore, the regions (or units) used to signal the MMVD may not have uniform intervals. The MMVD may only indicate a specific direction for a specific signaling unit.

[0270] According to embodiments of the present invention, MMVD can be determined based on distance and orientation. (Reference) Figure 21 The distance and direction of the MMVD can be preset based on a distance index indicating the distance of the MMVD and a direction index indicating the direction of the MMVD. In an embodiment, the distance can indicate the size of the MMVD in specific pixels (e.g., an absolute value), while the direction can indicate the orientation of the MMVD. The encoder / decoder can signal a relatively small distance using a relatively small index. That is, if the signaling is not via fixed-length binarized signaling, the encoder / decoder can signal a relatively short distance using relatively small bits.

[0271] According to embodiments of the present invention, an MVD can use a signaled MV or an MV based on a signaled MV. For example, an MV based on a signaled MV can have a sign opposite to that of the signaled MV. For example, MVD signaling can be performed based on a value corresponding to any reference list, and a value corresponding to another reference list different from the reference list can be the same as or have the opposite sign to the value corresponding to the reference list (i.e., the signaled MVD). Whether the reference list is the same as or has the opposite sign to the value can be determined by the Picture Order Number (POC) relationship between the current picture and the reference pictures in the reference list, and the POC relationship between the current picture and the reference pictures in another reference list different from the reference list. More specifically, even if both reference lists L0 and L1 are used, only one MVD can be signaled. For example, the MVD corresponding to L0 can be signaled. Furthermore, the MVD corresponding to L1 can be determined based on the MVD corresponding to L0.

[0272] For example, the MVD corresponding to L1 can be the same value as the MVD corresponding to L0, or it can have a sign that has changed from the sign of the MVD corresponding to L0. Furthermore, this can be determined through the POC relationship between the current image and the L0 reference image, and the POC relationship between the current image and the L1 reference image. For example, it can be determined based on (RefPicList0[refIdxLN0],currPic). The value of `DiffPicOrderCnt(currPic, RefPicList1[refIdxLN1])` is greater than or less than 0 to determine whether to use the same MVD corresponding to L1 as the MVD corresponding to L0, or an MVD corresponding to L0 with a changed sign. `DiffPicOrderCnt(RefPicList0[refIdxLN0], currPic)` A value greater than 0 for DiffPicOrderCnt(currPic, RefPicList1[ refIdxLN1]) can mean that both the L0 reference image and the L1 reference image are either before or after the current image in time.

[0273] Therefore, in this case, L0 MVD and L1 MVD can have the same notation. Furthermore, DiffPicOrderCnt(RefPicList0[refIdxLN0], currPic) A value less than 0 for `DiffPicOrderCnt(currPic,RefPicList1[refIdxLN1])` can mean that one of the L0 reference image and the L1 reference image is chronologically earlier than the current image, while the other is chronologically later than the current image. Therefore, in this case, the L0 MVD and L1 MVD can have different signs. The earlier position in time can refer to a smaller POC, while the later position in time can refer to a larger POC. In the above embodiment, an MV scaling process can be added. That is, a process can be added to scale the MV that is signaled or the MV modified from the signaled MV (e.g., an MV with the opposite sign).

[0274] Table 16 below illustrates the MMVD syntax structure according to an embodiment of the present invention.

[0275] [Table 16]

[0276] According to embodiments of the present invention, there may be an indication of whether to use (or apply) in Figure 21 The signaling for MMVD as described in [Table 16] refers to the MMVD flag (i.e., mmvd_flag), which indicates a syntax element that indicates whether MMVD is used. For example, MMVD can be applied to skip mode or merge mode. In an embodiment, when MMVD is used, the decoder can parse the syntax related to MMVD. When MMVD is used, MMVD-related information can be additionally parsed in the MMVD index compilation (i.e., mmvd_idx_coding) syntax.

[0277] As an example, when MMVD is not used, the decoder can parse the merge index. Referring to [Table 16], the merge index (i.e., merge_idx) indicates a syntax element that indicates a specific merge candidate in the merge candidate list. Furthermore, in merge modes that are not skipped, the MMVD flag can be parsed when the merge flag (i.e., merge flag) is 1 after parsing the merge flag. The merge flag can indicate whether a merge mode is used (or applied) or a sub-block merge mode. The merge flag can indicate that AMVP, inter-frame mode, affine inter-frame mode, etc., are not used. In this embodiment, the merge index can be... Figure 21 The basic candidate indexes described in the text are different from the indexes.

[0278] According to embodiments of the present invention, there may be signaling indicating the maximum number of merge candidates. The signaling indicating the maximum number of merge candidates can be executed in units greater than CUs or prediction units. For example, the signaling indicating the maximum number of merge candidates can be executed in slices or patches. When the conditions based on the signaling indicating the maximum number of merge candidates are met, the decoder can parse the merge index. As an example, when the value of the syntax element indicating the maximum number of merge candidates indicates that the maximum number of merge candidates is 1, the encoder / decoder can infer that the merge index is 0. That is, when the signaling indicating the maximum number of merge candidates indicates that the maximum number of merge candidates is 1, candidates can be determined without any parsing of the merge index.

[0279] Referring to [Table 16], a MaxNumMergeCand corresponding to a value (variable or parameter) based on the signaling indicating the maximum number of merge candidates can be defined. MaxNumMergeCand can refer to the maximum number of merge candidates. MaxNumMergeCand can have a value greater than or equal to 1. The maximum number of merge candidates for the signaling unit indicating the maximum number of merge candidates can be determined based on the maximum number of supported merge candidates and the signaling indicating the maximum number of merge candidates. For example, the encoder / decoder can determine the maximum number of merge candidates for the signaling unit indicating the maximum number of merge candidates by subtracting the signaling value indicating the maximum number of merge candidates from the maximum number of supported merge candidates. Although only the maximum number of merge candidates is described, the maximum number of sub-block merge candidates and the maximum number signaling can exist substantially the same.

[0280] In an embodiment, the merge index can be resolved when using either skip mode or merge mode. The decoder can resolve the merge index when the conditions for using either skip mode or merge mode are met. For example, the encoder / decoder can use either skip mode or merge mode when subblock merge mode is not used. In Table 16 above, the merge index can be resolved after resolving the MMVD flags. This can take precedence over regular skip mode or merge mode considerations for MMVD. Furthermore, the subblock merge flag (i.e., subblock_merge_flag) or merge subblock flag (i.e., merge_subblock_flag) syntax elements indicating whether to use subblock merge mode can be signaled.

[0281] Figure 22 The illustration shows the syntax structure related to MMVD according to an embodiment of the present invention. The embodiments described above [Table 16] can be equivalently applied to... Figure 22 Furthermore, related repetitive descriptions have been omitted. Compared to [Table 16], Figure 22 The syntax illustrated in the diagram also includes processes related to sub-block merging. Figure 22 and the following figures (i.e., Figures 23 to 31 In this document, descriptions related to intra-frame prediction processing, inter-frame prediction modes, AMVP mode, and affine inter-frame modes can be omitted. Although for ease of description, the main focus is on... Figure 22 The decoder is described in the document, but the compilation process according to this embodiment can be applied to the encoder in essentially the same way.

[0282] According to embodiments of the invention, the decoder may parse the sub-block merge flag earlier than the MMVD flag and / or merge index. This is to prioritize the sub-block merge mode over at least one of skip mode, merge mode, and / or MMVD. When the sub-block merge mode is not used, the decoder may parse the MMVD flag and / or merge index. For example, the decoder may parse the sub-block merge flag before the MMVD flag, and may parse the MMVD flag even when the sub-block merge mode is not applied. Since motion compensation is performed on a per-sub-block basis at relatively small sizes in the sub-block merge mode, the accuracy of motion prediction may be high due to the characteristics of sub-block-based motion compensation, and improving MV using an MVD may be ineffective. According to embodiments of the invention, compression efficiency can be improved by identifying whether MMVD is applied when the sub-block merge mode is not applied, and this can be achieved by configuring the parsing order of the syntax (or syntax elements) (i.e., parsing the sub-block merge flag before the MMVD flag).

[0283] Figure 23 The diagram illustrates the syntax structure related to MMVD according to an embodiment of the present invention. Figure 23 The syntax (or function) shown can indicate the method used to parse the data in [Table 16]. Figure 22 The syntax for MMVD information indicated in the syntax related to MMVD described below. See reference... Figure 21 The MMVD can be determined based on basic motion vectors, distance, and direction. Indexes (or syntax elements) for signaling information can be defined. (See reference.) Figure 23 The syntax elements used to signal the basic motion vector, distance, and direction can be defined as base_mv_idx, distance_idx, and direction_idx. As shown in [Table 16] and Figure 22 As described above, when determining to use MMVD, the instruction can be placed in... Figure 23 The detailed syntax elements of the MMVD values ​​illustrated in the diagram are signaled from the encoder to the decoder. Although for ease of description, the main focus is on... Figure 23 The decoder is described in the document, but the compilation process according to this embodiment can be applied to the encoder in essentially the same way.

[0284] Figure 24The diagram illustrates the syntax structure related to MMVD according to an embodiment of the present invention. Figure 24 The syntax (or function) shown can indicate the method used to parse the data in [Table 16]. Figure 22 The syntax for MMVD information indicated in the syntax related to MMVD described below. According to embodiments of the invention, some MMVD-related syntax can be parsed when certain predefined conditions are met. For example, the decoder can parse some MMVD-related syntax when certain predefined conditions are met after determining that MMVD is used. In embodiments, some MMVD-related syntax may be signaling related to the basic motion vector. The determination of using MMVD may be based on the MMVD flag value. When the MMVD flag is parsed or inferred and indicates a specific value (e.g., 1), it can be determined that MMVD is used. Although for ease of description, mainly in Figure 24 The decoder is described in the document, and the compilation process according to this embodiment can be applied to the encoder in essentially the same way.

[0285] According to embodiments of the invention, specific conditions may be related to the maximum number of available basic motion vector candidates. For example, when the maximum number of available basic motion vector candidates is greater than or equal to 2, the decoder may parse the syntax related to basic motion vectors, while when the maximum number of available basic motion vector candidates is 1, the decoder may not parse the syntax related to basic motion vectors. When no syntax related to basic motion vectors exists, the decoder may infer this value. In this case, the syntax related to basic motion vectors may be inferred to be 0. When a basic motion vector candidate is a merge candidate, the decoder may use a condition based on the maximum number of available merge candidates to determine whether to parse the syntax related to basic motion vectors.

[0286] As mentioned above, the maximum number of available basic motion vector candidates can be a value indicated by signaling that indicates the maximum number of merged candidates as described in [Table 16], and this value can be represented as MaxNumMergeCand.

[0287] refer to Figure 24 The decoder can only resolve the basic motion vector index if MaxNumMergeCand, indicating the maximum number of merged candidates, is greater than 1. If the basic motion vector index does not exist, for example, if it has not been resolved and therefore does not exist, the decoder can infer that the value is 0. This is because the determination can be performed even if signaling is not executed when the maximum number of available candidates is 1.

[0288] Figure 25 The illustration depicts the MMVD syntax according to an embodiment of the present invention. Figure 22In the case of the syntax structure shown, the MMVD flags are always parsed even when using skip mode or merge mode without using MMVD. Furthermore, the basic motion vector index or merge index is parsed in both cases with and without MMVD. Meanwhile, in Figure 25 In some embodiments, when using skip mode or merge mode without using MMVD, the MMVD flag may not be parsed. Although for ease of description, primarily in... Figure 25 The decoder is described in the document, but the compilation process according to this embodiment can be applied to the encoder in essentially the same way.

[0289] According to embodiments of the present invention, the syntax and merge index related to the basic motion vector can be defined to be identical to each other. For example, the basic motion vector index and the merge index can be identical to each other. That is, signaling for indicating the corresponding candidate through a syntax element can be executed. Furthermore, according to embodiments of the present invention, the decoder can parse the syntax related to the basic motion vector and parse the MMVD flag based on conditions. When the MMVD flag indicates the use of MMVD, the syntax related to MMVD and the syntax related to the basic motion vector can be parsed.

[0290] For example, after parsing the merge index, it can be identified whether the corresponding candidate is one that can be used for MMVD. When a candidate is usable for MMVD, the MMVD flag can be parsed; when a candidate is not usable for MMVD, the MMVD flag can be left unparsed. When the MMVD flag is absent, it can be inferred that MMVD will not be used. For example, when the number of basic motion vectors in MMVD is num_mmvd_baseCand (the first few in the merge candidate list, in the direction of the smallest index), the MMVD flag can be parsed if the parsed index is less than num_mmvd_baseCand. Otherwise, it can be determined that MMVD will not be used without parsing the MMVD flag. In this case, using num_mmvd_baseCand or more candidate indices has the advantage of reducing the number of MMVD flag bits.

[0291] refer to Figure 25 After resolving the merge index, the MMVD flags are resolved if the merge index is less than num_mmvd_baseCand. num_mmvd_baseCand can be the number of candidates for the basic motion vectors available for the MMVD. Furthermore, the basic motion vector index of the MMVD can be determined based on the merge index.

[0292] Figure 26 The illustration shows the MMVD syntax according to an embodiment of the present invention. (Reference) Figure 26The MMVD flag can be positioned higher than the merge flag. For example, the decoder can determine whether to parse the merge flag based on the MMVD flag. This can be applied, for example, to cases where it is not a skip mode. Although for ease of description, in Figure 26 The decoder is mainly described, but the compilation process according to this embodiment can be applied to the encoder in essentially the same way.

[0293] According to embodiments of the present invention, when the MMVD flag indicates that MMVD is used, the merge flag may not need to be parsed. For example, the basic motion vector candidates for MMVD can be determined from the merge candidates, and the decoder may not need to parse the merge flag when the MMVD flag indicates that MMVD is used. When MMVD is used (MMVD flag is 1), the use of a merge mode can be inferred from the merge flag. (See reference...) Figure 26 When using the skip mode, i.e., when cu_skip_flag is 1, the decoder can infer that the merge flag is 1, regardless of the use of MMVD. Furthermore, when the merge flag is absent and the skip mode is not used, it can be inferred that the merge flag is 1 if MMVD is used, and 0 if MMVD is not used. Alternatively, in the embodiment, when the merge flag is absent and the skip mode is not used, the decoder can infer that the merge flag is 1. Furthermore, refer to... Figure 26 In non-skip mode, if the MMVD flag is 1, the decoder can parse the syntax related to MMVD, while if the MMVD flag is 0, it parses the merge flag.

[0294] According to an embodiment of the present invention, when the sub-block merge flag is positioned higher than the merge flag and the sub-block merge flag is 1, the merge flag does not need to be parsed and it can be inferred that the merge flag is 1. When the sub-block merge flag is positioned higher than the MMVD flag and the sub-block merge flag is 1, the MMVD flag does not need to be parsed and it can be inferred that the MMVD flag is 0. In this specification, a modeX_flag of 1 can indicate that modeX is used, while a modeX_flag of 0 can indicate that modeX is not used.

[0295] Figure 27The illustration illustrates the MMVD syntax according to an embodiment of the present invention. According to an embodiment of the present invention, MMVD can always be used for some candidates in a candidate list that serve as the basic motion vector candidates for MMVD. For example, when determining the basic motion vector candidates for MMVD from a merged candidate list, MMVD can always be used for some candidates in the merged candidate list. For example, MMVD can always be used for a predetermined candidate index. For example, when the candidate index is less than a preset value, MMVD can always be used. In this case, whether to use MMVD can be determined based on the candidate index. Furthermore, MMVD flag resolution may not be available.

[0296] For example, after parsing the merge index, when the merge index corresponds to a value indicating the use of MMVD, it can be determined that MMVD is used. Furthermore, in this case, syntax related to MMVD (e.g., distance, direction, signaling, etc.) can be parsed. Although for ease of description... Figure 27 The decoder is primarily described here, but the compilation process according to this embodiment can be applied to the encoder using essentially the same method. (Reference) Figure 27 MMVD can always be used until there are num_mmvd_baseCand candidates in the first few candidates of the merge candidate list. In this case, when the candidate index is less than num_mmvd_baseCand, it can be determined that MMVD should be used, and the syntax related to MMVD can be parsed.

[0297] Figure 28 The illustration depicts the MMVD syntax according to an embodiment of the present invention. According to an embodiment of the present invention, the decoder can always use MMVD for some candidates in a candidate list used as basic motion vector candidates for MMVD. For example, when basic motion vector candidates for MMVD are determined from a merged candidate list, the decoder can always use MMVD for some candidates in the merged candidate list. For example, the decoder can always use MMVD for a predetermined candidate index. For example, when the candidate index is less than a preset value, MMVD can always be used. Although for ease of description, Figure 27 The decoder is mainly described, but the compilation process according to this embodiment can be applied to the encoder in essentially the same way.

[0298] According to an embodiment of the invention, the MMVD flag can be positioned above the merge flag in the grammar structure (or in the order of grammar parsing). In this case, when the MMVD flag is 1, the decoder can parse the grammar related to MMVD. Furthermore, when the MMVD flag is 1, the merge flag can be inferred to be 1. When the MMVD flag is 0, the decoder can parse the merge flag. When the merge flag is 1, the decoder can parse the merge index. In this case, there may be other conditions for parsing the merge index. As an embodiment, the decoder can determine the merge index to be actually used after modifying the merge index based on the parsed merge index. For example, the merge index to be actually used can be determined as the value obtained by adding the parsed merge index to the number of candidates that always use MMVD among values ​​less than the parsed merge index. For example, when it is determined that MMVD is always used for the first num_mmvd_baseCand candidates in the candidate list, the value obtained by adding the parsed merge index to num_mmvd_baseCand can be used as the merge index. When the MMVD flag precedes the merge flag and the MMVD flag is 0, candidates that use MMVD can be excluded from the candidate list.

[0299] refer to Figure 27 The MMVD flag precedes the merge flag. Furthermore, when the MMVD flag is 1, the decoder can parse the syntax related to MMVD. When the MMVD flag is 0, the decoder can parse the merge flag. When the MMVD flag is 0 and the merge flag is 1, the decoder can parse the merge index. In this case, there may be additional conditions for parsing the index. The actual merge index to be used can be determined by adding the parsed merge index to num_mmvd_baseCand, which corresponds to the number of candidates that can be used as the basic motion vector for MMVD.

[0300] Figure 28The MMVD syntax according to an embodiment of the present invention is illustrated. According to an embodiment of the present invention, the candidate index may exist before the flag indicating whether the candidate index uses a mode. For example, the candidate index may exist before the MMVD flag or the merge flag. In this case, after parsing the candidate index, whether to parse the MMVD flag may be determined according to whether the parsed index is a candidate that can use MMVD. For example, when the parsed candidate index is a candidate that can use MMVD, the MMVD flag is parsed, and when the parsed candidate index is a candidate that cannot use MMVD, the MMVD flag is not parsed, and the MMVD flag can be inferred to be 0. In addition, when the MMVD flag is 0, the merge flag can be parsed. Through this merge flag, it can be determined whether to use the merge mode, the sub-block merge mode, the inter mode, the AMVP mode, or the affine inter mode. Although for the convenience of description, in Figure 28 the decoder is mainly described, the encoding process according to the present embodiment can be applied to the encoder through substantially the same method.

[0301] For example, when the number of candidates available for the first mode is numb1 and the number of candidates available for the second mode is num2, and num1<num2, if a candidate is parsed and the parsed index is greater than or equal to num1, the syntax for determining one of the first mode and the second mode (for example, the syntax indicating whether to use the first mode) may not be parsed, and it can be inferred that the first mode is not used. When the parsed index is less than num1, the syntax for determining one of the first mode and the second mode can be parsed.

[0302] Reference Figure 28 after parsing the merge index, when the merge index is less than num_mmvd_baseCand corresponding to the number of candidates that can use MMVD, the MMVD flag can be parsed. When the merge index is greater than or equal to num_mmvd_baseCand, the MMVD flag may not be parsed, and in this case, the MMVD flag can be inferred to be 0. When the MMVD flag is 1, the distance index (distance_idx) and direction index (direction_idx) indicating the syntax elements related to MMVD can be parsed. When the MMVD flag is 0, the merge flag is parsed, so that the merge mode, sub-block merge mode, inter mode, AMVP mode or affine mode can be identified.

[0303] Figure 29 The coding unit syntax structure according to an embodiment of the present invention is illustrated. As shown in Figure 29 as shown, when the merge flag is 1, the merge data (i.e., merge_data) parsing process can be executed. Although for the convenience of description, in Figure 29The document primarily describes the decoder, but the compilation process according to this embodiment can be applied to the encoder using essentially the same method. In this embodiment, the merged data may include a portion of the syntax related to merging. Furthermore, the merged data may be referred to as merged data syntax. For example, the merged data may include, as described above... Figures 21 to 28 In the syntax parsing, the process is executed when the merge flag is set to 1. Furthermore, a merge flag of 1 can indicate that a merge pattern is used. A merge flag of 1 can also indicate that no merge pattern is used. Figure 6 Inter-frame prediction of mvd_coding as indicated in [Table 1].

[0304] Figure 30 The diagram illustrates the merged data syntax structure according to an embodiment of the present invention. For example... Figure 29 As stated above, when the merge flag is 1, the decoder can parse (or invoke) the merged data syntax. Although for ease of description, in Figure 30 The decoder is primarily described, but the compilation process according to this embodiment can be applied to the encoder using essentially the same method. Furthermore, as mentioned above, the merge flag is configured to 1 when using merge mode or skip mode.

[0305] refer to Figure 30 In the merged data syntax, the MMVD flag can be parsed first among the syntax elements. The MMVD flag can be the first syntax element to be parsed after the merge flag is identified as 1. The MMVD flag can be parsed before signaling indicating whether to use another prediction mode with the merge flag configured as 1. The MMVD flag can be signaling indicating whether to use MMVD. This can be applied to [Table 16], and... Figures 22 to 29 The same method as the embodiments described herein. Figure 30 As shown, the merged data syntax can include decoding processing for sub-block merging modes, multiple hypothesis prediction (or intra-frame and inter-frame combined prediction), triangle prediction, etc.

[0306] According to embodiments of the present invention, when generating prediction blocks, multi-hypothesis prediction can be a method of generating and combining two or more prediction blocks. Alternatively, when generating prediction blocks, multi-hypothesis prediction can be a method using both inter-frame prediction and intra-frame prediction. Furthermore, when performing prediction, inter-frame prediction and intra-frame prediction can be methods that use images that are different from or the same as those included in the current block. (See reference) Figure 30 The multi-hypothesis prediction flag indicates whether the syntax element for multi-hypothesis prediction is used.

[0307] According to embodiments of the present invention, when predicting the current block (e.g., a compilation unit or a prediction unit), the sub-block merging mode can be a method of performing motion compensation on a sub-block basis (i.e., determining motion vectors on a sub-block basis). According to embodiments, the sub-block merging mode may include methods such as sub-block-based temporal MVP, affine motion prediction, etc. Reference Figure 30 The sub-block merge flag is a syntax element that indicates whether the sub-block merge mode is used.

[0308] Furthermore, according to embodiments of the present invention, triangle prediction is a method for performing motion compensation on regions within the current block that are not rectangular. That is, in triangle prediction, units within the current block that have the same motion vector may not be rectangular. (See reference...) Figure 30 The triangle prediction flag (i.e., merge_triangle_flag) indicates whether the syntax element for triangle prediction is used.

[0309] According to embodiments of the present invention, the MMVD flag can be resolved before signaling (or syntax elements) indicating the use of another merge besides MMVD. Signaling indicating the use of another merge besides MMVD may include a multiple hypothesis prediction flag, a merge subblock flag, a merge triangle flag, etc.

[0310] refer to Figure 30 The MMVD merge index (i.e., mmvd_merge_idx) (or MMVD index) is a syntax element that indicates which is used as the base motion vector of the MMVD. If MMVD is used, it can be inferred that the merge index is the MMVD merge flag (i.e., mmvd_merge_flag) (or MMVD flag).

[0311] Figure 31 The diagram illustrates the merged data syntax structure according to an embodiment of the present invention. Figure 31 An example could be part of a merge data syntax executed when the merge flag is 1. Figure 29 In, although, Figure 31 The decoder is primarily described, but the compilation process according to this embodiment can be applied to the encoder using essentially the same method. According to embodiments of the invention, the MMVD flags can be parsed later than the merge index. For example, the MMVD flags can be parsed immediately after the merge index. For example, the MMVD flags may not be parsed at positions that do not precede merge data syntax. That is, after recognizing the merge flag as 1, the MMVD flags can be parsed after parsing other syntax that is not an MMVD flag.

[0312] Furthermore, according to embodiments of the present invention, such as Figure 21As shown, the fundamental motion vectors of MMVD can be determined from another candidate list. As an example, the fundamental motion vectors can be determined from a portion of another candidate list. Therefore, as... Figure 25 As shown, according to an embodiment, whether to parse the MMVD flag can be determined based on an index associated with another candidate list, which can be used as the basic motion vector of the MMVD. For example, the index associated with another candidate list that can be used as the basic motion vector of the MMVD can be parsed earlier than the MMVD flag. When the index associated with the other candidate list indicates that the other candidate list can be used as the basic motion vector of the MMVD, the MMVD flag is parsed. Otherwise, the MMVD flag may not be parsed.

[0313] refer to Figure 31 The basic motion vector of the MMVD can be determined from the merge candidates. Therefore, as an example, the MMVD flag is resolved when the merge index indicates its availability as the basic motion vector of the MMVD. Otherwise, the MMVD flag may not be resolved. Alternatively, when the basic motion vector of the MMVD can be selected from the first n candidates in the merge candidate list, the MMVD flag may be resolved if the merge index is less than n (the merge index can start from 0), and may not be resolved if the merge index is not less than n. More specifically, the basic motion vector of the MMVD can be the first or second candidate in the merge candidate list. (See reference...) Figure 31 When the merge index is less than 2 (i.e., 0 or 1), the MMVD flag can be resolved. The merge index can indicate the merge candidate index. Therefore, regardless of the merge mode, when the merge index indicates that MMVD is unavailable, the MMVD flag can be left unresolved, thus improving compilation efficiency.

[0314] Furthermore, the basic motion vector of MMVD can be determined from a candidate list of any mode. According to embodiments of the invention, when multiple signalings indicating whether each mode is used exist, the MMVD flag can be resolved after determining that the mode is to be used. For example, when assuming the existence of a first mode, a second mode, and a third mode, and determining MMVD based on the third mode, or determining the basic motion vector of MMVD from candidates of the third mode, the MMVD flag can be resolved after determining that the third mode is to be used. For example, the use of the third mode can be determined by signaling indicating whether the third mode is to be used. Alternatively, for example, when it is determined that an available mode other than the third mode, such as the first and second modes, is not to be used, the use of the third mode can be determined. For example, when sub-block merging mode, multiple hypothesis prediction, triangle prediction, regular merging mode, etc., exist, and MMVD can be applied to the regular merging mode, the MMVD flag can be resolved after determining that the regular merging mode and MMVD among the modes listed above are to be used. The regular merging mode is a mode used to perform inter-frame prediction using motion information of adjacent intra-frame compiled blocks, and can be a merging mode of a conventional video compression technique (e.g., High Efficiency Video Compilation (HEVC)).

[0315] Furthermore, when resolving MMVD flags based on an index related to another candidate list of possible fundamental motion vectors for MMVD, a separate signaling indicating the fundamental motion vectors of the MMVD is not required. For example, in Figure 23 or Figure 24 In this context, signaling such as the basic motion vector index exists independently of the merged index. Furthermore, reference... Figure 30 The MMVD merging flag can be a signaling indication of the basic motion vector of the MMVD. According to embodiments of the present invention, the MMVD merging flag may not be like... Figure 31 It exists as described in the example. For instance, the syntax related to MMVD could include only the MMVD flag and signaling indicating the MMVD distance. Figures 30 to 31 mmvd_distance_idx in the code, and the signaling indicating the direction of MMVD ( Figures 30 to 31 (mmvd_direction_idx in the original text). Furthermore, the fundamental motion vector of the MMVD can be determined by merging indices. Therefore, in Figure 31 In some embodiments, there may be no indication. Figure 30 Signaling and context model of the basic motion vectors of MMVD.

[0316] According to an embodiment of the invention, the MMVD flag can be parsed after signaling indicating whether any merging mode other than MMVD is used. (See reference) Figure 31The MMVD flags can be resolved after the merge subblock flag, the multiple hypothesis prediction flag (i.e., `mh_intra_flag`), and the merge triangle flag (i.e., `merge_triangle_flag`). Furthermore, it's possible to use a specific merge mode without MMVD. In this case, the MMVD flags can be resolved when a flag indicating whether any merge mode is used indicates that no merge mode is used. (See reference) Figure 31 MMVD can be used without triangle prediction, and the MMVD flag can be resolved when the merge triangle flag is 0. Alternatively, MMVD can be used without MH intra-frame, and the MMVD flag can be resolved when mh_intra_flag is 0. Alternatively, MMVD can be used without sub-block merging mode, and the MMVD flag can be resolved when the merge sub-block flag is 0.

[0317] Figure 32 The diagram illustrates the merged data syntax structure according to an embodiment of the present invention. Figure 32 An embodiment may be when Figure 29 The merge flag in the code is set to 1, which is part of the merge data syntax performed. Although for ease of description, in Figure 32 The decoder is mainly described, but the compilation process according to this embodiment can be applied to the encoder in essentially the same way.

[0318] According to embodiments of the invention, the MMVD flag can be resolved after signaling indicating whether another mode is used. Alternatively, the MMVD flag can be resolved after signaling indicating whether another mode of MMVD, where the merge flag is not configured as 1, is used. For example, the MMVD flag can be resolved after the merge subblock flag, the multiple hypothesis prediction flag (i.e., mh_intra_flag), and the merge triangle flag (i.e., merge_triangle_flag).

[0319] Furthermore, in the embodiments, the MMVD flag can be resolved when no mode other than MMVD is used. For example, the MMVD flag can be resolved when the merge subblock flag is 0. The MMVD flag can be resolved when the multiple hypothesis prediction flag is 0. The MMVD flag can be resolved when the merge triangle flag is 0. A value of 0 for a flag indicating which mode is used can mean that the mode is not used. When MMVD can be applied to the regular merge mode, the MMVD flag can be resolved after determining whether to use the regular merge mode or MMVD.

[0320] Furthermore, in the embodiments, when MMVD is used, the syntax related to MMVD can be parsed, while when MMVD is not used, the merge index can be parsed. Alternatively, when MMVD is used, the decoder can parse the MMVD merge flag (i.e., mmvd_merge_flag), the MMVD distance index (mmvd_distance_idx), and the MMVD direction index (mmvd_direction_idx), and can parse the merge index when MMVD is not used. Furthermore, when such... Figure 32 As shown, when resolving the MMVD flags after a flag indicating whether to use another mode, the signaling indicating the basic motion vector of the MMVD can exist as a syntax element separate from the merge index. (See reference) Figure 32 The signaling indicating the basic motion vector of MMVD can be the MMVD merging flag. For example, the MMVD flag can be parsed after determining whether to use regular merging mode or MMVD.

[0321] According to embodiments of the present invention, the basic motion vectors of an MMVD can be determined from any candidate list, and the maximum number of candidates in the candidate list can be variable. For example, the maximum number of candidates can be determined from a higher-level syntax. A higher-level syntax can be a syntax at a level higher than the current compilation unit. For example, a higher-level syntax can be a sequence, image, slice, or patch level syntax. According to an embodiment, in this case, the maximum number of candidates for the basic motion vectors of an MMVD can follow the maximum number of candidates that can become the basic motion vectors of an MMVD in the candidate list. Therefore, when the maximum number of candidates that can become the basic motion vectors of an MMVD decreases in the candidate list, the maximum number of candidates that can become the basic motion vectors of an MMVD also decreases.

[0322] In an embodiment, the basic motion vector of the MMVD can be determined from the list of merging candidates, and the maximum number of merging candidates can be MaxNumMergeCand. Furthermore, MaxNumMergeCand can be determined from a higher-level syntax. In this case, the maximum number of merging candidates that can become the basic motion vector of the MMVD can be equal to or less than MaxNumMergeCand. Accordingly, as... Figure 32 As shown, MaxNumMergeCand can be used to determine whether to parse the signal indicating the basic motion vector of the MMVD. For example, when MaxNumMergeCand is 1, the signal indicating the basic motion vector of the MMVD can be left unparsed. When there is no signal indicating the basic motion vector of the MMVD, it can be inferred that MaxNumMergeCand is 0.

[0323] In another embodiment, MMVD can be omitted when MaxNumMergeCand is 1. This is to reduce the signaling overhead of the syntax related to MMVD. Therefore, when MaxNumMergeCand is 1, the MMVD flag does not need to be parsed. Furthermore, when the MMVD flag is absent, MaxNumMergeCand can be inferred to be 0. (See reference) Figure 32 The merged triangle flag can only be resolved when multi-hypothesis intra-frame prediction is not used. If the merged triangle flag does not exist, MaxNumMergeCand can be inferred to be 0.

[0324] According to embodiments of the present invention, multiple modes that cannot be used together may exist. For example, sub-block merging mode, triangle prediction, multiple hypothesis intra-frame prediction, and MMVD may not be used together. Signaling indicating whether each of the multiple modes that cannot be used together is used can be parsed according to a preset order. In this case, signaling indicating whether any of the multiple modes is used can only be parsed if it is determined that the signaling of the other modes among the multiple modes parsed before the use indication all indicate that the other modes are not used.

[0325] Figure 33 The illustration shows a method for processing video signals according to an embodiment of the present invention. (Reference) Figure 33 Although the decoder has been described primarily for ease of description, the present invention is not limited thereto, and the video signal processing method based on multiple hypothesis prediction according to this embodiment can be applied to the encoder in essentially the same manner.

[0326] In S3301, the decoder obtains a first syntax element indicating whether a sub-block merge mode is applied to the current block. As described above, as an example, the first syntax element may be a sub-block merge flag (or a merge sub-block flag).

[0327] In S3302, when the first syntax element indicates that the sub-block-based merge mode is not applied to the current block, the decoder obtains a second syntax element indicating whether the motion vector difference merge mode is applied to the current block. As described above, as an embodiment, the second syntax element may be an MMVD flag (or an MMVD merge flag).

[0328] In S3303, when the second syntax element indicates that a merging mode using motion vector difference is applied to the current block, the decoder obtains a third syntax element that indicates a candidate among the candidates included in the merging candidate list for the current block that is used for inter-frame prediction of the current block. As described above, as an embodiment, the third syntax element may be an MMVD index (or an MMVD candidate index or an MMVD candidate flag).

[0329] In S3304, the decoder acquires information related to the motion vector difference. As described above, as an embodiment, the step of acquiring information related to the motion vector difference may include acquiring a fourth syntax element indicating the distance of the motion vector difference and a fifth syntax element indicating the direction of the motion vector difference. As described above, as an embodiment, the fourth syntax element may be a distance index (or an MMVD distance index), and the fifth syntax element may be a direction index (or an MMVD direction index).

[0330] In S3305, the decoder can deduce the motion vector of the current block by adding the candidate motion vector indicated by the third syntax element to the motion vector difference.

[0331] In S3306, the decoder uses the motion vector of the current block to generate the prediction block for the current block.

[0332] As described above, as an example, the third syntax element can indicate the candidate used for inter-frame prediction of the current block from the first and second candidates in the merge candidate list of the current block.

[0333] As described above, as an example, when the maximum number of merge candidates is greater than 1, the third syntax element can be parsed from the bitstream, while when the maximum number of merge candidates is not greater than 1, the third syntax element can be inferred as 0.

[0334] As described above, as an embodiment, when the first syntax element indicates that a sub-block-based merging mode is applied to the current block, the step of obtaining a sixth syntax element may be further included. The sixth syntax element indicates candidates included in the sub-block merging candidate list of the current block that are used for inter-frame prediction of the current block. Furthermore, the motion vector of the current block can be derived based on the motion vectors of the candidates indicated by the sixth syntax element when the first syntax element indicates that a sub-block-based merging mode is applied to the current block, on a unit basis, at least one sub-block included in the current block. As described above, as an embodiment, the sixth syntax element may be a sub-block merging index (or a merged sub-block index).

[0335] As described above, as an example, when the first syntax element indicates that a sub-block-based merging pattern is applied to the current block, it can be inferred that the second syntax element is 0.

[0336] As described above, as an example, when the maximum number of sub-block merging candidates is greater than 1, the sixth syntax element can be parsed from the bitstream, while when the maximum number of sub-block merging candidates is not greater than 1, the sixth syntax element can be inferred to be 0.

[0337] Embodiments of the present invention can be implemented in various ways. For example, embodiments of the present invention can be implemented in hardware, firmware, software, or a combination thereof.

[0338] When implemented in hardware, embodiments of the present invention may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, etc.

[0339] In firmware or software implementations, the methods according to embodiments of the present invention can be implemented in the form of modules, processes, functions, etc., for performing the above-described functions or operations. Software code can be stored in memory and executed by a processor. The memory can be located inside or outside the processor and can exchange data with the processor in various known ways.

[0340] Some embodiments may be implemented in the form of a recording medium that includes computer-executable instructions, such as computer-executed program modules. Computer-readable media can be randomly available media accessible by a computer and includes all volatile and non-volatile media, as well as separable and non-separable media. Furthermore, computer-readable media can include all computer storage media and communication media. Computer storage media includes all volatile, non-volatile, separable, and non-separable media implemented using random methods or techniques for storing information, such as computer-readable instructions, data structures, program modules, or other data. Communication media includes computer-readable instructions, data structures, modulated data signals such as program modules, other data or other transmission mechanisms, and includes random information transmission media.

[0341] The descriptions of the present invention are merely exemplary, and those skilled in the art will understand that embodiments of the invention can be readily modified into other detailed forms without departing from the technical spirit or essential features of the invention. Therefore, the above embodiments should be interpreted in all respects as examples, not limitations. For example, each element mentioned in the singular can be implemented in a distributed manner, and elements mentioned in a distributed manner can also be implemented in a combined manner.

[0342] The scope of this invention is defined by the appended claims rather than the detailed description. The meaning and scope of the claims, as well as all modifications or variations derived from their equivalents, should be interpreted as including within the scope of this invention.

Claims

1. A method for processing video signals, the method comprising: Get the first syntax element, which indicates whether a sub-block-based merge pattern is applied to the current block; When the first syntax element indicates that the sub-block-based merging mode is not applied to the current block, a second syntax element is obtained, the second syntax element indicating whether the merging mode using motion vector difference is applied to the current block; When the second syntax element indicates that the merging mode using motion vector difference is applied to the current block, a third syntax element is obtained, which indicates a candidate included in the merging candidate list of the current block that is used for inter-frame prediction of the current block; information related to the motion vector difference is obtained; and the motion vector of the current block is derived by adding the motion vector difference to the motion vector of the candidate indicated by the third syntax element. When the first syntax element indicates that the sub-block-based merging pattern is applied to the current block, the motion vector of the current block is derived without resolving the second syntax element; as well as The prediction block for the current block is generated using the motion vector of the current block. The sub-block-based merging mode uses one or more sub-block merging candidates related to the motion vectors determined on a sub-block basis in the current block.

2. The method according to claim 1, wherein, The third syntax element indicates the candidate from the first and second candidates in the merge candidate list of the current block that is used for inter-frame prediction of the current block.

3. The method according to claim 2, wherein, When the maximum number of merge candidates is greater than 1, the third syntax element is parsed from the bitstream, and when the maximum number of merge candidates is not greater than 1, the third syntax element is inferred to be 0.

4. The method according to claim 1, wherein, The acquisition of information related to the motion vector difference includes: Obtain a fourth syntax element indicating the distance of the motion vector difference; and Obtain the fifth syntax element that indicates the direction of the difference in motion vectors.

5. The method of claim 1, further comprising: When the first syntax element indicates that the sub-block-based merging mode is applied to the current block, a sixth syntax element is obtained, the sixth syntax element indicating a candidate included in the sub-block merging candidate list of the current block for inter-frame prediction of the current block, wherein, when the first syntax element indicates that the sub-block-based merging mode is applied to the current block, the motion vector of the current block is derived based on the motion vector of the candidate indicated by the sixth syntax element, on a unit basis of at least one sub-block included in the current block.

6. The method according to claim 5, wherein, When the first syntax element indicates that the sub-block-based merging pattern is applied to the current block, the second syntax element is inferred to be 0.

7. The method according to claim 5, wherein, When the maximum number of sub-block merging candidates is greater than 1, the sixth syntax element is parsed from the bitstream, and when the maximum number of sub-block merging candidates is not greater than 1, the sixth syntax element is inferred to be 0.

8. A method for processing a video signal, the method comprising: The encoding indicates whether the sub-block merging pattern is applied to the first syntax element of the current block; When the first syntax element indicates that the sub-block-based merging mode is not applied to the current block, a second syntax element is encoded, the second syntax element indicating whether the motion vector difference merging mode is applied to the current block; When the second syntax element indicates that the merging mode using motion vector difference is applied to the current block, a third syntax element is encoded, which indicates a candidate included in the merging candidate list of the current block for inter-frame prediction of the current block, and encodes information related to the motion vector difference for deriving the motion vector of the current block by adding the motion vector difference to the motion vector of the candidate indicated by the third syntax element. When the first syntax element indicates that the sub-block-based merging pattern is applied to the current block, the motion vector of the current block is derived without encoding the second syntax element; as well as The prediction block for the current block is generated using the motion vector of the current block. The sub-block-based merging mode uses one or more sub-block merging candidates related to the motion vectors determined on a sub-block basis in the current block.

9. A method for transmitting a bit stream, the method comprising: Encoding bitstreams through encoding methods, and Send the bit stream, The encoding method includes: The encoding indicates whether the sub-block merging pattern is applied to the first syntax element of the current block; When the first syntax element indicates that the sub-block-based merging mode is not applied to the current block, a second syntax element is encoded, the second syntax element indicating whether the motion vector difference merging mode is applied to the current block; When the second syntax element indicates that the merging mode using motion vector difference is applied to the current block, a third syntax element is encoded, which indicates a candidate included in the merging candidate list of the current block for inter-frame prediction of the current block, and encodes information related to the motion vector difference for deriving the motion vector of the current block by adding the motion vector difference to the motion vector of the candidate indicated by the third syntax element. When the first syntax element indicates that the sub-block-based merging pattern is applied to the current block, the motion vector of the current block is derived without encoding the second syntax element; and The prediction block for the current block is generated using the motion vector of the current block. The sub-block-based merging mode uses one or more sub-block merging candidates related to the motion vectors determined on a sub-block basis in the current block.

Images