Video signal processing method and apparatus using sub-block based motion compensation

Abstract

To provide a video signal processing method and device for encoding and decoding a video signal, capable of enhancing a compression efficiency.SOLUTION: In the case where a first syntax element indicating whether or not a subblock-based merging mode is adopted to the current block indicates that the subblock-based merging mode is not adopted to the current block, a method acquires a second syntax element indicating whether or not a merging mode using a motion vector difference is adopted to a current block. In the case where the second syntax element indicates that the merging mode using the motion vector difference to the current block is adopted, the method: derives a motion vector for the current block by adding the motion vector difference to a candidate motion vector indicated by a third syntax element that indicates a candidate to be used in inter-prediction for the current block, among candidates contained in a merge candidate list for the current block; and generates a prediction block of the current block.SELECTED DRAWING: Figure 33

Description

[Technical Field]

[0001] The present invention relates to a method and apparatus for processing a video signal, and more particularly to a method and apparatus for processing a video signal that encodes or decodes a video signal using sub-block based motion compensation. [Background technology]

[0002] Compression coding refers to a series of signal processing techniques for transmitting digitized information over a communication line or storing it in a form suitable for a storage medium. Compression coding can be used to encode audio, video, text, and other data, but video compression is the technology that specifically encodes video. Video signal compression is performed by removing redundant information by taking into account spatial correlation, temporal correlation, and stochastic correlation. However, with the recent development of various media and data transmission media, more efficient video signal processing methods and devices are needed. Summary of the Invention [Problem to be solved by the invention]

[0003] The present invention aims to improve the coding efficiency of video signals, provide a method for efficiently performing sub-block-based motion compensation, and provide a motion compensation method based on a merge mode with motion vector difference (MVD). [Means for solving the problem]

[0004] In order to solve the above problems, the present invention provides a video signal processing device and a video signal processing method as follows.

[0005] According to one embodiment of the present invention, there is provided a video signal processing method, including: obtaining a first syntax element indicating whether a sub-block based merge mode is applied to a current block; if the first syntax element indicates that the sub-block based merge mode is not applied to the current block, obtaining a second syntax element indicating whether a merge mode using a motion vector differential is applied to the current block; if the second syntax element indicates that the merge mode using the motion vector differential is applied to the current block, obtaining a third syntax element indicating a candidate to be used for inter prediction of the current block among candidates included in a merge candidate list of the current block; obtaining information related to the motion vector differential; deriving a motion vector of the current block by adding the motion vector differential to a motion vector of a candidate indicated by the third syntax element; and generating a prediction block of the current block using the motion vector of the current block.

[0006] For example, the third syntax element may indicate a candidate to be used for inter prediction of the current block from among a first candidate and a second candidate in a merge candidate list of the current block.

[0007] As an example, the third syntax element may be parsed from the bitstream if the maximum number of merging candidates is greater than one, and may be inferred to be 0 if the maximum number of merging candidates is not greater than one.

[0008] In an embodiment, the step of obtaining information related to the motion vector difference may include the step of obtaining a fourth syntax element indicating the distance of the motion vector difference; and the step of obtaining a fifth syntax element indicating the direction of the motion vector difference.

[0009] As an embodiment, if the first syntax element indicates that a sub-block based merge mode is applied to the current block, the method may further include obtaining a sixth syntax element indicating a candidate to be used for inter-prediction of the current block among candidates included in a sub-block merge candidate list of the current block, and if the first syntax element indicates that a sub-block based merge mode is applied to the current block, a motion vector of the current block may be derived in units of at least one sub-block included in the current block based on the motion vector of the candidate indicated by the sixth syntax element.

[0010] As an example, if the first syntax element indicates that a sub-block based merge mode is applied to the current block, the second syntax element may be inferred as 0.

[0011] As an example, the sixth syntax element may be parsed from the bitstream if the maximum number of sub-block merging candidates is greater than 1, and may be inferred to be 0 if the maximum number of sub-block merging candidates is not greater than 1.

[0012] According to one embodiment of the present invention, there is provided a video signal processing device including a processor, wherein the processor obtains a first syntax element indicating whether a sub-block based merge mode is applied to a current block, and if the first syntax element indicates that the sub-block based merge mode is not applied to the current block, obtains a second syntax element indicating whether a merge mode using a motion vector differential is applied to the current block, and if the second syntax element indicates that the merge mode using the motion vector differential is applied to the current block, obtains a third syntax element indicating a candidate to be used for inter prediction of the current block from among candidates included in a merge candidate list of the current block, obtains information related to the motion vector differential, derives a motion vector of the current block by adding the motion vector differential to a motion vector of a candidate indicated by the third syntax element, and generates a prediction block of the current block using the motion vector of the current block.

[0013] For example, the third syntax element may indicate a candidate to be used for inter prediction of the current block from among a first candidate and a second candidate in a merge candidate list of the current block.

[0014] As an example, the third syntax element may be parsed from the bitstream if the maximum number of merging candidates is greater than one, and may be inferred to be 0 if the maximum number of merging candidates is not greater than one.

[0015] As an example, the processor may obtain a fourth syntax element indicating a distance of the motion vector difference, and obtain a fifth syntax element indicating a direction of the motion vector difference.

[0016] In an embodiment, when the first syntax element indicates that a sub-block-based merge mode is applied to the current block, the processor may obtain a sixth syntax element indicating a candidate to be used for inter-prediction of the current block among candidates included in a sub-block merge candidate list of the current block, and when the first syntax element indicates that a sub-block-based merge mode is applied to the current block, the processor may derive a motion vector of the current block for at least one sub-block included in the current block based on the motion vector of the candidate indicated by the sixth syntax element.

[0017] As an example, if the first syntax element indicates that a sub-block based merge mode is applied to the current block, the second syntax element may be inferred as 0.

[0018] As an example, the sixth syntax element may be parsed from the bitstream if the maximum number of sub-block merging candidates is greater than 1, and may be inferred to be 0 if the maximum number of sub-block merging candidates is not greater than 1.

[0019] According to one embodiment of the present invention, there is provided a video signal processing method, including: encoding a first syntax element indicating whether a sub-block based merge mode is applied to a current block; if the sub-block based merge mode is not applied to the current block, encoding a second syntax element indicating whether a merge mode using a motion vector differential is applied to the current block; if the merge mode using the motion vector differential is applied to the current block, encoding a third syntax element indicating a candidate to be used for inter prediction of the current block among candidates included in a merge candidate list of the current block; encoding information related to the motion vector differential; deriving a motion vector of the current block by adding the motion vector differential to the motion vector of the candidate indicated by the third syntax element; and generating a prediction block of the current block using the motion vector of the current block. [Effects of the Invention]

[0020] According to an embodiment of the present invention, it is possible to improve the coding efficiency of a video signal. In particular, according to an embodiment of the present invention, it is possible to reduce signaling overhead and improve compression efficiency by adjusting the syntax parsing order. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a schematic block diagram of a video signal encoding apparatus according to an embodiment of the present invention; [Figure 2] 1 is a schematic block diagram of a video signal decoding device according to an embodiment of the present invention; [Figure 3] FIG. 1 illustrates an example of how coding tree units are divided into coding units within a picture. [Figure 4] FIG. 1 illustrates an embodiment of a method for signaling the splitting of quadtrees and multi-type trees. [Figure 5] FIG. 1 is a diagram illustrating inter prediction according to an embodiment of the present invention. [Figure 6] 1 is a diagram illustrating a motion vector signaling method according to an embodiment of the present invention. [Figure 7] 1 is a diagram illustrating a method for signaling adaptive motion vector resolution information according to one embodiment of the present invention. [Figure 8] FIG. 1 illustrates affine motion compensation according to an embodiment of the present invention. [Figure 9] FIG. 1 illustrates an embodiment of a four-parameter affine motion compensation method. [Figure 10] 1 is a diagram illustrating an affine motion prediction method according to an embodiment of the present invention; [Figure 11] FIG. 1 illustrates an embodiment of a sub-block-based affine motion compensation method. [Figure 12] FIG. 10 illustrates an embodiment of the present invention for obtaining a set of control point motion vectors for prediction of a current block. [Figure 13] FIG. 10 illustrates an embodiment of the present invention for obtaining a set of control point motion vectors for prediction of a current block. [Figure 14] 1 is a diagram illustrating an affine motion compensation method according to an embodiment of the present invention; [Figure 15] 1 is a diagram illustrating an affine motion compensation method according to an embodiment of the present invention; [Figure 16] FIG. 1 illustrates a subblock-based temporal MVP (SbTMVP) according to one embodiment of the present invention. [Figure 17] FIG. 1 illustrates a planar MVP according to one embodiment of the present invention. [Figure 18] 10 is a diagram illustrating a method for determining a temporary motion vector according to an embodiment of the present invention; [Figure 19]FIG. 10 is a diagram illustrating the use or non-use of SbTMVP in one embodiment of the present invention. [Figure 20] FIG. 1 is a diagram illustrating a method for inducing SbTMVP according to one embodiment of the present invention. [Figure 21] FIG. 10 illustrates a method for applying MMVD according to one embodiment of the present invention. [Figure 22] A figure illustrating an MMVD-related syntax structure in one embodiment of the present invention. [Figure 23] A figure illustrating an MMVD-related syntax structure in one embodiment of the present invention. [Figure 24] A figure illustrating an MMVD-related syntax structure in one embodiment of the present invention. [Figure 25] A figure illustrating MMVD syntax for one embodiment of the present invention. [Figure 26] A figure illustrating MMVD syntax for one embodiment of the present invention. [Figure 27] A figure illustrating MMVD syntax for one embodiment of the present invention. [Figure 28] A figure illustrating MMVD syntax for one embodiment of the present invention. [Figure 29] FIG. 1 illustrates a coding unit syntax structure according to one embodiment of the present invention. [Figure 30] FIG. 10 illustrates a merge data syntax structure according to one embodiment of the present invention. [Figure 31] FIG. 10 illustrates a merge data syntax structure according to one embodiment of the present invention. [Figure 32] FIG. 10 illustrates a merge data syntax structure according to one embodiment of the present invention. [Figure 33] 1 is a diagram illustrating a video signal processing method according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0022] The terms used in this specification have been selected as widely used and general terms as possible, taking into consideration the functions of the present invention, but these may vary depending on the intentions of engineers in the field, customs, or the emergence of new technologies. In addition, in certain cases, the applicant may have arbitrarily selected terms, and in such cases, the meanings of these terms will be described in the relevant section on the mode for carrying out the invention. Therefore, it is made clear that the terms used in this specification should be interpreted not simply as terms, but based on the substantive meanings of the terms and the overall content of this specification.

[0023] In this specification, some terms may be interpreted as follows: "Coding" may be interpreted as "Encoding" or "Decoding" in some cases. In this specification, an apparatus that encodes a video signal to generate a video signal bitstream is referred to as an encoding apparatus or encoder, and an apparatus that decodes a video signal bitstream to restore a video signal is referred to as a decoding apparatus or decoder. In this specification, "video signal processing apparatus" is used as a conceptual term that includes both an encoder and a decoder. "Information" is a term that includes values, parameters, coefficients, elements, etc., and may be interpreted differently in some cases, so the present invention is not limited thereto. "Unit" is used interchangeably to refer to a basic unit of image processing or a specific position in a picture, and refers to an image area including both luma and chroma components. "Block" refers to an image area including specific components of luma and chroma components (i.e., Cb and Cr). However, depending on the embodiment, terms such as "unit," "block," "partition," and "area" may be used interchangeably. In this specification, the term "unit" is used as a concept including a coding unit, a prediction unit, and a transform unit, and the term "picture" refers to a field or a frame, and these terms may be used interchangeably depending on the embodiment.

[0024] 1 is a schematic block diagram of a video signal encoding apparatus according to an embodiment of the present invention. Referring to FIG. 1, the encoding apparatus 100 includes a transform unit 110, a quantization unit 115, an inverse quantization unit 120, an inverse transform unit 125, a filtering unit 130, a prediction unit 150, and an entropy coding unit 160.

[0025] The transform unit 110 transforms a residual signal, which is the difference between the input video signal and the prediction signal generated by the prediction unit 150, to obtain transform coefficient values. For example, a discrete cosine transform (DCT), a discrete sine transform (DST), or a wavelet transform may be used. The discrete cosine transform and discrete sine transform divide the input picture signal into blocks and then perform the transform. During the transform, coding efficiency may vary depending on the distribution and characteristics within the transform domain. The quantization unit 115 quantizes the transform coefficient values ​​output from the transform unit 110.

[0026] To improve coding efficiency, instead of directly coding the picture signal, the prediction unit 150 predicts a picture using a pre-coded region and adds the residual value between the original picture and the predicted picture to obtain a reconstructed picture. To avoid mismatch between the encoder and decoder, the encoder should use information available to the decoder when making predictions. To achieve this, the encoder performs a process of further reconstructing the coded current block. The inverse quantization unit 120 inversely quantizes the transform coefficient values, and the inverse transform unit 125 reconstructs the residual value using the inversely quantized transform coefficient values. Meanwhile, the filtering unit 130 performs filtering operations to improve the quality of the reconstructed picture and the coding efficiency. Examples of filtering operations include a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter. The filtered picture is stored in a decoded picture buffer (DPB) 156 for output or use as a reference picture.

[0027] To improve coding efficiency, instead of directly coding a picture signal, the prediction unit 150 predicts a picture using an already coded region and adds a residual value between the original picture and the predicted picture to the predicted picture to obtain a reconstructed picture. The intra prediction unit 152 performs intra prediction within the current picture, and the inter prediction unit 154 predicts the current picture using a reference picture stored in the decoded picture buffer 156. The intra prediction unit 152 performs intra prediction from a reconstructed region within the current picture and transmits the intra coding information to the entropy coding unit 160. The inter prediction unit 154 may further include a motion estimation unit 154a and a motion compensation unit 154b. The motion estimation unit 154a obtains a motion vector value for the current region by referring to a specific reconstructed region. The motion estimation unit 154a transmits position information of the reference region (e.g., reference frame, motion vector) to the entropy coding unit 160 so that it can be included in the bitstream. Using the motion vector values ​​transmitted from the motion estimation unit 154a, the motion compensation unit 154b performs inter-frame motion compensation.

[0028] The prediction unit 150 includes an intra prediction unit 152 and an inter prediction unit 154. The intra prediction unit 152 performs intra prediction within the current picture, and the inter prediction unit 154 performs inter prediction to predict the current picture using a reference buffer stored in the decoded picture buffer 156. The intra prediction unit 152 performs intra prediction from reconstructed samples within the current picture and transmits intra coding information to the entropy coding unit 160. The intra coding information includes at least one of an intra prediction mode, an MPM (Most Probable Mode) flag, and an MPM index. The intra coding information may include information about reference samples. The inter prediction unit 154 includes a motion estimation unit 154a and a motion compensation unit 154b. The motion estimation unit 154a obtains a motion vector value for the current region by referring to a specific region of the reconstructed reference signal picture. The motion estimation unit 154a transmits a motion information set (reference picture index, motion vector information) for the reference region to the entropy coding unit 160. The motion compensation unit 154b performs motion compensation using the motion vector values ​​transmitted from the motion compensation unit 154a. The inter prediction unit 154 transmits inter coding information including the motion information for the reference region to the entropy coding unit 160.

[0029] According to a further embodiment, the prediction unit 150 includes an intra block copy (BC) prediction unit (not shown). The intra BC prediction unit performs intra BC prediction from reconstructed samples in the current picture and transmits intra BC coding information to the entropy coding unit 160. The intra BC prediction unit obtains block vector values ​​indicating a reference region to be used for predicting the current region by referring to a specific region in the current picture. The intra BC prediction unit performs intra BC prediction using the obtained block vector values. The intra BC prediction unit transmits the intra BC coding information to the entropy coding unit 160. The intra BC prediction unit includes the block vector information.

[0030] After the picture prediction is performed, the transform unit 110 converts residual values ​​between the original picture and the predicted picture to obtain transform coefficient values. The transform is performed in units of specific blocks within the picture, and the size of the specific blocks varies within a predetermined range. The quantization unit 115 quantizes the transform coefficient values ​​generated by the transform unit 110 and transmits the quantized values ​​to the entropy coding unit 160.

[0031] The entropy coding unit 160 generates a video signal bitstream by entropy coding information indicating quantized transform coefficients, intra-coding information, and inter-coding information. The entropy coding unit 160 uses a variable length coding (VLC) scheme and an arithmetic coding scheme. The VLC scheme converts input symbols into successive codewords, where the length of the codewords is variable. For example, frequently occurring symbols are represented by short codewords, and infrequently occurring symbols are represented by long codewords. The variable length coding scheme is a context-based adaptive variable length coding (CAVLC) scheme. The arithmetic coding scheme converts successive data symbols into a single prime number, and obtains the optimal prime number bits required to represent each symbol. The arithmetic coding scheme is a context-based adaptive binary arithmetic coding (CABAC) scheme. For example, the entropy coding unit 160 may binarize information indicating quantized transform coefficients and may arithmetically code the binarized information to generate a bitstream.

[0032] The generated bitstream is encapsulated in Network Abstraction Layer (NAL) units as basic units. An NAL unit includes an integer number of coded coding tree units. In order for a video decoder to decode the bitstream, the bitstream must first be separated into NAL units and then each separated NAL unit must be decoded. Meanwhile, information required for decoding the video signal bitstream is transmitted via Raw Byte Sequence Payload (RBSP) of higher level sets such as a Picture Parameter Set (PPS), a Sequence Parameter Set (SPS), and a Video Parameter Set (VPS).

[0033] 1 illustrates an encoding device 100 according to one embodiment of the present invention, with separate blocks illustrating logically distinct elements of encoding device 100. Therefore, the elements of encoding device 100 described above may be implemented on a single chip or multiple chips depending on the device design. According to one embodiment, the operation of each element of encoding device 100 described above is performed by a processor (not shown).

[0034] 2 is a schematic block diagram of a video signal decoding apparatus 200 according to an embodiment of the present invention. Referring to FIG. 2, the decoding apparatus 200 of the present invention includes an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 225, a filtering unit 230, and a prediction unit 250.

[0035] The entropy decoding unit 210 entropy decodes the video signal bitstream to extract transform coefficient information, intra-coding information, inter-coding information, etc. for each region. For example, the entropy decoding unit 210 may obtain a binary code for transform coefficient information of a specific region from the video signal bitstream. The entropy decoding unit 210 also de-binarizes the binary code to obtain quantized transform coefficients. The inverse quantization unit 220 de-quantizes the quantized transform coefficients, and the inverse transform unit 225 restores residual values ​​using the de-quantized transform coefficients. The video signal processing device 200 restores original pixel values ​​by combining the residual values ​​obtained from the inverse transform unit 225 with predicted values ​​obtained from the prediction unit 250.

[0036] Meanwhile, the filtering unit 230 performs filtering on the picture to improve image quality. This includes a deblocking filter to reduce block distortion and / or an adaptive loop filter to remove distortion from the entire picture. The filtered picture is output or stored in the decoded picture buffer (DPB) 256 to be used as a reference picture for the next picture.

[0037] The prediction unit 250 includes an intra prediction unit 252 and an inter prediction unit 254. The prediction unit 250 generates a predicted picture using the coding type, transform coefficients for each region, intra / inter coding information, etc. decoded by the entropy decoding unit 210. To reconstruct the current block to be decoded, the current picture including the current block or a decoded region of another picture can be used. A picture (or tile / slice) that uses only the current picture for reconstruction, i.e., performs intra prediction or intra BC prediction, is called an intra picture or I picture (or tile / slice), and a picture (or tile / slice) that can perform all of intra prediction, inter prediction, and intra BC prediction is called an inter picture (or tile / slice). Among interpictures (or tiles / slices), a picture (or tile / slice) that uses at most one motion vector and reference picture index to predict sample values ​​for each block is called a predictive picture or P picture (or tile / slice), and a picture (or tile / slice) that uses at most two motion vectors and reference picture indexes is called a bi-predictive picture or B picture (or tile / slice). In other words, a P picture (or tile / slice) uses at most one motion information set to predict each block, and a B picture (or tile / slice) uses at most two motion information sets to predict each block. Here, a motion information set includes one or more motion vectors and one reference picture index.

[0038] The intra prediction unit 252 generates a prediction block using intra coding information and reconstructed samples in the current picture. As described above, the intra coding information includes at least one of an intra prediction mode, a Most Probable Mode (MPM) flag, and an MPM index. The intra prediction unit 252 predicts sample values ​​of the current block using reconstructed samples located to the left and / or above the current block as reference samples. In the present disclosure, the reconstructed samples, reference samples, and samples of the current block refer to pixels. Furthermore, sample values ​​refer to pixel values.

[0039] In one embodiment, the reference samples are samples included in neighboring blocks of the current block. For example, the reference samples are samples adjacent to the left boundary and / or the top boundary of the current block. Furthermore, the reference samples are samples located on a line within a predetermined distance from the left boundary of the current block and / or samples located on a line within a predetermined distance from the top boundary of the current block, among samples in neighboring blocks of the current block. In this case, the neighboring blocks of the current block include at least one of the left (L) block, the top (A) block, the below left (BL) block, the above right (AR) block, and the above left (AL) block adjacent to the current block.

[0040] The inter prediction unit 254 generates a prediction block using reference pictures and inter coding information stored in the decoded picture buffer 256. The inter coding information includes a motion information set (e.g., reference picture index, motion vector, etc.) of the current block relative to the reference block. Inter prediction includes L0 prediction, L1 prediction, and bi-prediction. L0 prediction is prediction using one reference picture included in the L0 picture list, and L1 prediction is prediction using one reference picture included in the L1 picture list. This requires one set of motion information (e.g., motion vector and reference picture index). The bi-prediction method uses up to two reference regions, and these two reference regions may exist in the same reference picture or in different pictures. That is, the bi-prediction method uses up to two sets of motion information (e.g., motion vector and reference picture index), and two motion vectors may correspond to the same reference picture index or different reference picture indexes. In this case, the reference picture may be displayed (or output) either temporally before or after the current picture. According to one embodiment, in a bi-predictive scheme, the two reference regions used may be regions selected from the L0 picture list and the L1 picture list, respectively.

[0041] The inter prediction unit 254 obtains a current reference block using a motion vector and a reference picture index. The reference block exists in a reference picture corresponding to the reference picture index. Furthermore, sample values ​​of a block identified by the motion vector or their interpolated values ​​are used as a predictor for the current block. For motion prediction with sub-pel pixel accuracy, for example, an 8-tab interpolation filter is used for the luma signal and a 4-tab interpolation filter is used for the chroma signal. However, the interpolation filters for sub-pel motion prediction are not limited thereto. In this way, the inter prediction unit 254 performs motion compensation, which predicts the texture of the current unit from a previously reconstructed picture. In this case, the inter prediction unit uses a motion information set.

[0042] According to a further embodiment, the predictor 250 may include an intra BC predictor (not shown). The intra BC predictor may reconstruct the current region by referring to a specific region including reconstructed samples in the current picture. The intra BC predictor obtains intra BC coding information for the current region from the entropy decoding unit 210. The intra BC predictor obtains block vector values ​​of the current region indicating the specific region in the current picture. The intra BC predictor may perform intra BC prediction using the obtained block vector values. The intra BC coding information may include block vector information.

[0043] According to a further embodiment, the predictor 250 may include an intra BC predictor (not shown). The intra BC predictor may reconstruct the current region by referring to a specific region including reconstructed samples in the current picture. The intra BC predictor obtains intra BC coding information for the current region from the entropy decoding unit 210. The intra BC predictor obtains block vector values ​​of the current region indicating the specific region in the current picture. The intra BC predictor may perform intra BC prediction using the obtained block vector values. The intra BC coding information may include block vector information.

[0044] 2 shows a decoding device 200 according to one embodiment of the present invention, with separate blocks logically separating elements of the decoding device 200. Thus, the elements of the decoding device 200 described above may be implemented on a single chip or multiple chips depending on the device design. According to one embodiment, the operation of each element of the decoding device 200 described above is performed by a processor (not shown).

[0045] FIG. 3 illustrates an example in which a coding tree unit (CTU) is divided into coding units (CUs) within a picture. During video signal coding, a picture is divided into a sequence of coding tree units (CTUs). A coding tree unit consists of an NXN block of luma samples and two blocks of corresponding chroma samples. A coding tree unit is divided into multiple coding units. A coding tree unit may be a leaf node without being divided. In this case, the coding tree unit itself may be a coding unit. A coding unit refers to a basic unit for processing a picture during the above-mentioned video signal processing, i.e., intra / inter prediction, transform, quantization, and / or entropy coding. Within a picture, the size and shape of coding units are not constant. Coding units have a square or rectangular shape. A rectangular coding unit (or rectangular block) includes a vertical coding unit (or vertical block) and a horizontal coding unit (or horizontal block). In this specification, a vertical block is a block whose height is greater than its width, and a horizontal block is a block whose width is greater than its height. In addition, in this specification, non-square blocks refer to rectangular blocks, but the present invention is not limited to this.

[0046] Referring to Figure 3, a coding tree unit is first divided into a quad tree (QT) structure. That is, in the quad tree structure, one node having a size of 2N x 2N is divided into four nodes having a size of N x N. In this specification, a quad tree is also referred to as a quaternary tree. The quad tree division is performed recursively, and all nodes do not need to be divided to the same depth.

[0047] Meanwhile, the leaf node of the above-mentioned quad tree is further divided into a multi-type tree (MTT) structure. According to an embodiment of the present invention, in the multi-type tree structure, one node is divided into a horizontally or vertically divided binary or ternary tree structure. That is, there are four division structures in the multi-type tree structure: vertical binary division, horizontal binary division, vertical ternary division, and horizontal ternary division. According to an embodiment of the present invention, in each of the tree structures, the width and height of the node are both powers of 2. For example, in a binary tree (BT) structure, a node of size 2N×2N is divided into two N×2N nodes by vertical binary division and into two 2N×N nodes by horizontal binary division. In addition, in a ternary tree (TT) structure, a node of size 2Nx2N is divided into (N / 2)x2N, Nx2N, and (N / 2)x2N nodes by vertical ternary division, and into 2Nx(N / 2), 2NxN, and 2Nx(N / 2) nodes by horizontal ternary division. Such multi-type tree division is performed recursively.

[0048] The leaf nodes of a multi-type tree can be coding units. If no division for a coding unit is specified or the coding unit is not larger than the maximum transform length, the coding unit is used as the unit of prediction and transformation without further division. Meanwhile, in the above-mentioned quad trees and multi-type trees, at least one of the following parameters is predefined or transmitted via the RBSP of a higher-level set such as a PPS, SPS, or VPS: 1) CTU size: the size of the root node of the quad tree; 2) minimum QT size (MinQtSize): the size of the minimum QT leaf node allowed; 3) maximum BT size (MaxBtSize): the size of the maximum BT root node allowed; 4) maximum TT size (MaxTtSize): the size of the maximum TT root node allowed; 5) maximum MTT depth (MaxMttDepth): the maximum allowed depth of MTT division from the QT leaf node; 6) minimum BT size (MinBtSize): the size of the minimum BT leaf node allowed; 7) minimum TT size: the size of the minimum TT leaf node allowed.

[0049] 4 illustrates an embodiment of a method for signaling the splitting of a quadtree and a multi-type tree. Pre-set flags can be used to signal the splitting of the quadtree and multi-type tree. Referring to FIG. 4, at least one of a flag 'qt_split_flag' indicating whether to split a quadtree node, a flag 'mtt_split_flag' indicating whether to split a multi-type tree node, a flag 'mtt_split_vertical_flag' indicating the split direction of a multi-type tree node, and a flag 'mtt_split_binary_flag' indicating the split type of a multi-type tree node can be used.

[0050] According to an embodiment of the present invention, a coding tree unit is the root node of a quad tree and can be split into a quad tree structure first. In the quad tree structure, a 'qt_split_flag' is signaled for each node 'QT_node'. If the value of 'qt_split_flag' is 1, the corresponding node is split into four regular rectangular nodes, and if the value of 'qt_split_flag' is 0, the corresponding node becomes a leaf node 'QT_leaf_node' of the quad tree.

[0051] Each quadtree leaf node 'QT_leaf_node' can be further split into a multi-type tree structure. In a multi-type tree structure, 'mtt_split_flag' is signaled for each node 'MTT_node'. If 'mtt_split_flag' is set to 1, the node is split into multiple rectangular nodes, and if 'mtt_split_flag' is set to 0, the node becomes a leaf node 'MTT_leaf_node' of the multi-type tree. If a multi-type tree node 'MTT_node' is split into multiple rectangular nodes (i.e., if 'mtt_split_flag' is set to 1), 'mtt_split_vertical_flag' and 'mtt_split_binary_flag' can be additionally signaled for the node 'MTT_node'. If the value of 'mtt_split_vertical_flag' is 1, vertical split of node 'MTT_node' is indicated, and if the value of 'mtt_split_vertical_flag' is 0, horizontal split of node 'MTT_node' is indicated. Also, if the value of 'mtt_split_binary_flag' is 1, node 'MTT_node' is split into two rectangular nodes, and if the value of 'mtt_split_binary_flag' is 0, node 'MTT_node' is split into three rectangular nodes.

[0052] Picture prediction (motion compensation) for coding is performed on coding units that cannot be further divided (i.e., leaf nodes of the coding unit tree). Such a basic unit for prediction is hereinafter referred to as a prediction unit or a prediction block.

[0053] Hereinafter, the term "unit" used in this specification is used as an alternative term to the prediction unit, which is a basic unit for performing prediction, but the present invention is not limited thereto and can be understood as a concept including the coding unit in a broader sense.

[0054] FIG. 5 illustrates inter-prediction according to an embodiment of the present invention. As described above, a decoder predicts a current block by referring to reconstructed samples of other decoded pictures. Referring to FIG. 5, the decoder obtains a reference block 42 in a reference picture based on motion information of a current block 32. The motion information may include a reference picture index and a motion vector 50. The reference picture index indicates a reference picture of the current block in a reference picture list. The motion vector 50 represents an offset between the coordinate values ​​of the current block 32 in the current picture and the coordinate values ​​of the reference block 42 in the reference picture. The decoder obtains a predictor for the current block 32 based on sample values ​​of the reference block 42 and reconstructs the current block 32 using the predictor.

[0055] Meanwhile, according to an embodiment of the present invention, sub-block-based motion compensation may be used. That is, the current block 32 may be divided into a plurality of sub-blocks, and an independent motion vector may be used for each sub-block. Therefore, each sub-block within the current block 32 may be predicted using a different reference block. According to one embodiment, the sub-blocks may have a predetermined size, such as 4x4 or 8x8. The decoder obtains a predictor for each sub-block of the current block 32 using the motion vector of each sub-block. The predictor for each sub-block may be combined to obtain a predictor for the current block 32, and the decoder may reconstruct the current block 32 using the predictor for the current block 32 obtained in this manner.

[0056] According to an embodiment of the present invention, various methods of subblock-based motion compensation may be performed. Subblock-based motion compensation may include affine model-based motion compensation (hereinafter referred to as affine motion compensation or affine motion prediction) and subblock-based temporal motion vector prediction (Subblock-based Temporal Motion Vector Prediction, SbTMVP). Various embodiments of affine motion compensation and SbTMVP will be described below with reference to the drawings.

[0057] 6 is a diagram illustrating a motion vector signaling method according to an embodiment of the present invention. According to an embodiment of the present invention, a motion vector (MV) may be generated based on a motion vector prediction (or predictor) (MVP). As an example, the MV may be determined as the MVP according to the following Equation 1. In other words, the MV may be determined (or set or induced) to be the same value as the MVP.

[0058]

number

[0059] As another example, the MV may be determined based on the MVP and a motion vector difference (MVD), as shown in the following mathematical formula 2. The encoder can signal MVD information to the decoder to represent a more accurate MV, and the decoder can derive the MV by adding the obtained MVD to the MVP.

[0060]

number

[0061] According to one embodiment of the present invention, the encoder transmits the determined motion information to the decoder, and the decoder generates (or induces) a motion vector (MVP) from the received motion information and generates a prediction block based on the motion vector (MVD). For example, the motion information may include MVP information and MVD information. In this case, components of the motion information may differ depending on the inter-prediction mode. For example, in a merge mode, the motion information may include MVP information but not MVD information. For another example, in an advanced motion vector prediction (AMVP) mode, the motion information may include MVP information and MVD information.

[0062] To determine, transmit, and receive information about MVPs, the encoder and decoder can generate MVP candidates (or MVP candidate lists) in the same manner. For example, the encoder and decoder can generate the same MVP candidates in the same order. Then, the encoder transmits an index indicating (or pointing to) the determined (or selected) MVP from the generated MVP candidates to the decoder, and the decoder can derive the determined MVP and / or MV based on the received index.

[0063] According to an embodiment of the present invention, MVP candidates may include spatial candidates, temporal candidates, etc. The MVP candidates may be referred to as merge candidates when a merge mode is applied, and as AMVP candidates when an AMVP mode is applied. The spatial candidate may be an MV (or motion information) for a block at a specific position relative to the current block. For example, the spatial candidate may be an MV for a block at a position adjacent or non-adjacent to the current block. The temporal candidate may be an MV corresponding to a block in a picture different from the current picture. In addition, for example, the MVP candidate may include an affine MV, ATMVP, STMVP, a combination of the above-mentioned MVs (or candidates), an average MV of the above-mentioned MVs (or candidates), a zero MV, etc.

[0064] In one embodiment, the encoder may signal information indicating the reference picture to the decoder. For example, if the reference picture of the MVP candidate is different from the reference picture of the current block (or the currently processed block), the encoder / decoder may perform motion vector scaling (MV scaling) of the MVP candidate. In this case, the MV scaling may be performed based on the picture order count (POC) of the current picture, the POC of the reference picture of the current block, and the POC of the reference picture of the MVP candidate.

[0065] A specific embodiment of the MVD signaling method is described below. Table 1 below illustrates a syntax structure for MVD signaling.

[0066] [Table 1]

[0067] Referring to Table 1, according to one embodiment of the present invention, the sign and absolute value of the MVD may be coded separately. That is, the sign and absolute value of the MVD may each be coded using different syntax (or syntax elements). The absolute value of the MVD may be coded directly or may be coded stepwise based on a flag indicating whether the absolute value is greater than N, as shown in Table 1. If the absolute value is greater than N, the value of (absolute value - N) may also be signaled. Specifically, in the example of Table 1, abs_mvd_greater0_flag may be transmitted to indicate whether the absolute value is greater than 0. If abs_mvd_greater0_flag indicates (or indicates) that the absolute value is not greater than 0, the absolute value of the MVD may be determined to be 0. If abs_mvd_greater0_flag indicates that the absolute value is greater than 0, an additional syntax (or syntax element) may be present.

[0068] For example, abs_mvd_greater1_flag may be transmitted to indicate whether the absolute value is greater than 1. If abs_mvd_greater1_flag indicates (or indicates) that the absolute value is not greater than 1, the absolute value of the MVD may be determined to be 1. If abs_mvd_greater1_flag indicates that the absolute value is greater than 1, additional syntax may be present. For example, abs_mvd_minus2 may be present. abs_mvd_minus2 may be a value of (absolute value - 2). Because the abs_mvd_greater0_flag and abs_mvd_greater1_flag values ​​determine that the absolute value is greater than 1 (i.e., 2 or greater), a value of (absolute value - 2) may be signaled. In this way, by hierarchically syntactically signaling information about the absolute value, fewer bits can be used than when the absolute value is directly binarized and signaled.

[0069] In one embodiment, the absolute value-related syntax described above may be coded using a variable length binarization method such as Exponential-Golomb, truncated unary, truncated Rice, etc. Also, a flag indicating the sign of the MVD may be signaled by mvd_sign_flag.

[0070] Although the coding method for MVD has been described in the above embodiment, information other than MVD can also be signaled by separating the sign and absolute value. The absolute value may be coded as a flag indicating whether the absolute value is greater than a predefined specific value and a value obtained by subtracting the specific value from the absolute value. In Table 1, [0] and [1] may represent component indexes. For example, they may represent the x-component (i.e., horizontal component) and the y-component (i.e., vertical component).

[0071] FIG. 7 is a diagram illustrating a method for signaling adaptive motion vector resolution information according to an embodiment of the present invention. According to an embodiment of the present invention, the resolution for indicating MV or MVD may vary. For example, the resolution may be expressed based on pixels (or pels). For example, MV or MVD may be signaled in units of 1 / 4, 1 / 2, 1 (integer), 2, or 4 pixels. The encoder may then signal the MV or MVD resolution information to the decoder. For example, 16 may be coded as 64 in 1 / 4 units (1 / 4*64=16), 16 in 1 unit (1*16=16), and 4 in 4 units (4*.4=16). That is, the MV or MVD value may be determined by the following Equation 3:

[0072]

number

[0073] In Equation 3, valueDetermined represents an MV or MVD value. Also, valuePerResolution represents a value signaled based on the determined resolution. If the value signaled by MV or MVD is not divisible by the determined resolution, a rounding process may be applied. Using a high resolution may improve accuracy but consume more bits because the coded values ​​are large. Using a low resolution may reduce accuracy but consume fewer bits because the coded values ​​are small. In one embodiment, the resolution may be individually set for each unit, such as a sequence, a picture, a slice, a coding tree unit (CTU), or a coding unit (CU). That is, the encoder / decoder may adaptively determine / apply the resolution according to a predefined unit among the above units.

[0074] According to one embodiment of the present specification, the above-mentioned resolution information may be signaled from the encoder to the decoder. At this time, the resolution information may be binarized and signaled based on the above-mentioned variable length. In this case, if the resolution information is signaled based on the index corresponding to the smallest value (i.e., the earliest value), signaling overhead can be reduced. As one embodiment, the resolution information may be mapped to the signaling index in order from highest to lowest resolution.

[0075] According to one embodiment of the present specification, Figure 7 illustrates a signaling method assuming that three resolutions are used among a variety of resolutions. In this case, three signaling bits may be 0, 10, and 11, and the three signaling indexes may represent a first resolution, a second resolution, and a third resolution, respectively. Since one bit is required to signal the first resolution and two bits are required to signal the remaining resolutions, signaling overhead can be relatively reduced when signaling the first resolution. In the example of Figure 7, the first resolution, the second resolution, and the third resolution may be defined as 1 / 4, 1, and 4 pixel resolutions, respectively. In the following embodiments, MV resolution may refer to the resolution of MVD.

[0076] 8 shows affine motion compensation according to an embodiment of the present invention. Conventional inter prediction methods are optimized for predicting translational motion because they use a single motion vector for L0 prediction and L1 prediction of a current block. However, to efficiently perform motion compensation for zoom-in / out, rotation, and other irregular motions, reference blocks 44 of various shapes and sizes must be used.

[0077] 8, in affine motion compensation, a current block 34 may be predicted using a reference block 44 having a different size, shape, and / or orientation than the current block 34. That is, the reference block 44 may have a non-rectangular shape and may be larger or smaller in size than the current block 34. The reference block 44 may be obtained by performing an affine transformation on the current block 34. The affine transformation may include a six-parameter affine transformation using three control point motion vectors (CPMVs) and a four-parameter affine transformation using two control point motion vectors. Specific examples of this will be described later.

[0078] FIG. 9 illustrates an embodiment of a four-parameter affine motion compensation method. To reduce the computational complexity and signaling overhead of affine transformation, affine motion prediction can be performed using a predefined number of control point motion vectors (CPMVs). A control point motion vector (CPMV) is a motion vector corresponding to a specific control point (or sample position) of the current block. The specific control point may include at least one of the vertices of the current block. In this embodiment, the CPMV corresponding to the top-left vertex of the current block is referred to as v0 (or the first CPMV), the CPMV corresponding to the top-right vertex of the current block is referred to as v1 (or the second CPMV), and the CPMV corresponding to the bottom-left vertex of the current block is referred to as v2 (or the third CPMV). A CPMV set including at least two CPMVs can be used for affine motion prediction.

[0079] According to the embodiment of Figure 9, four-parameter affine motion prediction can be performed using v0 and v1. The current block 36, represented by a solid line, can be predicted using a reference block 46 at a position represented by a dotted line. Each sample of the current block 36 can be mapped to a different reference sample by an affine transformation. More specifically, the motion vector (vx, vy) at a sample position (x, y) of the current block 36 can be derived by the following Equation 4:

[0080]

number

[0081] Here, (v0x, v0y) is the first CPMV corresponding to the top left vertex of the current block 36, (v1x, v1y) is the second CPMV corresponding to the top right vertex of the current block 36, and w is the width of the current block 36.

[0082] 10 is a diagram illustrating an affine motion prediction method according to an embodiment of the present invention. According to an embodiment of the present invention, affine motion can be represented using multiple CPMVs or multiple parameters.

[0083] Referring to FIG. 10, the encoder / decoder may perform affine motion prediction using CPMVs of v0, v1, and v2. When six parameters are used, three control point motion vectors (CPMVs) of v0, v1, and v2 may be used for affine motion prediction. The shape of the reference block from which the current block 36 is predicted may be indicated based on the three vectors of v0, v1, and v2 or the six parameters. Referring to FIG. 10, the current block 36 represented by a solid line may be predicted from a reference block 46 represented by a dotted line in FIG. 10 in the reference picture. That is, each control point (or pixel at the corresponding position) of the current block 36 may be mapped to another point (or pixel) through affine transformation. The following mathematical formula 5 represents a motion vector field according to one embodiment of the present invention.

[0084]

number

[0085] In Equation 5, CPMV v0 representing the motion vector of the top-left corner control point in FIG. 10 may be expressed as (mv_0^x, mv_0^y). CPMV v1 representing the motion vector of the top-right corner control point may be expressed as (mv_1^x, mv_1^y). CPMV v2 representing the motion vector of the bottom-left corner control point may be expressed as (mv_2^x, mv_2^y). In this case, the motion vector (mv^x, mv^y) at the (x, y) position can be derived (or calculated) using Equation 5. Therefore, the motion vector for each pixel position or a specific position can be derived using Equation 5 based on the control point motion vectors v0, v1, and v2.

[0086] Furthermore, in Equation 5, (x, y) may be relative coordinates within the block. For example, if the top left corner of the block is located at (0, 0), (x, y) may be a position relative to the top left corner. Therefore, assuming that the position of v0 is (x0, y0), the position of v1 is (x1, y1), and the position of v2 is (x2, y2), and if (x, y) is the same coordinate as the positions of v0, v1, and v2, then x and y in Equation 5 can be expressed as (x-x0) and (y-y0), respectively. In this case, w, representing the width of the block, may be (x1-x0), and h, representing the height of the block, may be (y2-y0).

[0087] FIG. 11 shows an example of a sub-block-based affine motion compensation method. As described above, when using an affine motion transformation, a motion vector (i.e., a motion vector field) at each sample position of the current block can be derived. However, according to an embodiment of the present invention, in order to reduce the amount of computation, sub-block-based affine motion compensation can be performed. As shown in FIG. 8, the current block can include a plurality of sub-blocks, and the representative motion vector of each sub-block is obtained based on the CPMV set. According to one embodiment, the representative motion vector of each sub-block may be the motion vector corresponding to the sample position at the center of the sub-block. According to an additional embodiment, a motion vector with higher accuracy than a general motion vector may be used as the motion vector of the sub-block. For this purpose, a motion compensation interpolation filter may be applied.

[0088] The size of the sub-block for which affine motion compensation is performed may be set in various ways. According to one embodiment of the present invention, the sub-block can have a size that has already been set, such as 4X4 or 8X8. According to another embodiment of the present invention, the size MXN of the sub-block may be determined by the following mathematical formula 6.

[0089]

Equation

[0090] Here, w is the width of the current block, and MvPre is the fractional unit accuracy of the motion vector. (v2x, v2y) is the third CPMV corresponding to the lower left vertex of the current block, and according to one embodiment, it may be calculated by mathematical formula 5. max(a, b) is a function that returns the relatively larger value between a and b, and abs(x) is a function that returns the absolute value of x. Also, clip3(x, y, z) is a function that returns x when z < x, returns y when z > y, and returns z in other cases.

[0091] The decoder obtains a motion vector for each sub-block in the current block using the CPMV of the CPMV set. The decoder also obtains a predictor for each sub-block using the motion vector for each sub-block, and combines the predictors for each sub-block to obtain a predictor for the current block. The decoder can reconstruct the current block using the predictor for the current block obtained in this way.

[0092] 12 and 13 illustrate embodiments of the present invention for obtaining a control point motion vector set for predicting a current block. According to embodiments of the present invention, the CPMV set for predicting a current block may be obtained in various ways. More specifically, the CPMV set for predicting a current block may be obtained by referring to motion vector information sets of one or more neighboring blocks. In embodiments of the present invention, the motion vector information may represent the motion vector of a corresponding block or the CPMV of the corresponding block. Furthermore, the motion vector information set represents a collection of motion vector information of one or more blocks. The neighboring blocks may represent blocks including a predetermined neighboring position of the current block. In this case, the neighboring blocks may be coding units including the predetermined neighboring positions, or may be areas of a predetermined unit (e.g., 4x4, 8x8) including the neighboring positions.

[0093] There may be multiple candidates that can be referenced to derive the CPMV of the current block. Therefore, information about neighboring blocks to be referenced to derive the CPMV of the current block may be signaled separately. According to one embodiment of the present invention, an indicator indicating a motion vector information set to be referenced to derive the motion vector of each sub-block of the current block may be signaled. The indicator may indicate the motion vector information set of a neighboring block to be referenced to derive the motion vector of each sub-block of the current block. A decoder may obtain the indicator and acquire each CPMV of the CPMV set for the current block by referring to the motion vector information set of the neighboring block indicated by the indicator. According to a more specific embodiment, the decoder may generate a candidate list consisting of one or more motion vector information set candidates. Each motion vector information set candidate constituting the candidate list is a motion vector set of a neighboring block that can be used to derive the motion vector information of the current block. In this case, the indicator may be an index indicating one of the motion vector information sets in the candidate list. The CPMV of the current block may be acquired by referring to a motion vector information set selected from the candidate list based on the indicator (i.e., index). Various examples of motion vector information set candidates that can be included in the candidate list for deriving motion vector information (or CPMV set) of the current block are described below.

[0094] FIG. 12 illustrates an embodiment of obtaining a CPMV set for a current block. In the embodiment of FIG. 12, it is assumed that the CPMV set for the current block includes two CPMVs, i.e., v0 and v1. According to an embodiment of the present invention, the CPMVs for the current block may be derived from motion vectors of neighboring blocks adjacent to the corresponding point. Referring to FIG. 12, v0 may be derived from the motion vector of any one of neighboring blocks A, B, and C adjacent to the corresponding point, and v1 may be derived from the motion vector of any one of neighboring blocks D and E adjacent to the corresponding point. When the motion vectors of neighboring blocks A, B, C, D, and E are vA, vB, vC, vD, and vE, respectively, a motion vector information set that can be included in a candidate list may be derived as shown in Equation 7 below.

[0095]

number

[0096] That is, a (v0, v1) pair consisting of v0 selected from vA, vB, and vC and v1 selected from vD and vE may be obtained. In this case, v0 is derived from the motion vector of the block adjacent to the top-left vertex of the current block, and v1 is derived from the motion vector of the block adjacent to the top-right vertex of the current block. According to an additional embodiment, motion vector scaling may be performed based on the POC (Picture Order Count) of the current block, the POCs of reference pictures of neighboring blocks, and the POC of the reference picture of the current block.

[0097] A candidate list including the motion vector information set candidates obtained in this manner may be generated, and an indicator indicating one of the motion vector information sets in the candidate list may be signaled. According to additional embodiments of the present invention, the candidate list may include motion vector information set candidates for other inter prediction methods. For example, the candidate list may include motion vector information set candidates for sub-block-based temporal motion vector prediction (SbTMVP).

[0098] The decoder may derive the CPMV of the current block based on the motion vector information set obtained from the candidate list. According to one embodiment, the decoder may perform affine merging prediction by using the motion vectors of the motion vector information set obtained from the candidate list as the CPMV of the current block without a separate motion vector differential value. According to another embodiment, the decoder may obtain a separate motion vector differential value for the CPMV of the current block. The decoder may obtain the CPMV of the current block by summing the motion vectors of the motion vector information set obtained from the candidate list with the motion vector differential value. A flag or index indicating whether the decoder uses a separate motion vector differential value for affine motion compensation of the current block may be separately signaled.

[0099] FIG. 13 illustrates another embodiment for obtaining a CPMV set for a current block. According to another embodiment of the present invention, the CPMV of the current block may be derived from motion vector information of neighboring blocks that have undergone affine motion compensation, i.e., the CPMV or motion vectors of the neighboring blocks. In this case, the neighboring blocks may include a left neighboring block of the current block and an upper neighboring block of the current block. Referring to FIG. 13(a), the left neighboring block includes a block adjacent to the lower left vertex of the current block, i.e., left block A, and lower left block D. Also, the upper neighboring blocks include a block adjacent to the upper left vertex of the current block, i.e., upper left block E, and a block adjacent to the upper right vertex of the current block, i.e., upper block B, and upper right block C. The decoder checks whether the neighboring blocks are affine motion compensated in a predetermined order. If a neighboring block that has undergone affine motion compensation is found, the decoder obtains a CPMV set for the current block using the CPMV set (or motion vector) of the neighboring block. Referring to the embodiment of FIG. 13(b), the CPMV set of left block A may be used to derive a CPMV set for the current block. That is, the CPMV set (v0, v1) of the current block may be obtained based on the CPMV set (v2, v3, v4) of the left block A.

[0100] According to an embodiment of the present invention, information about neighboring blocks referenced to derive the CPMV of the current block may be separately signaled. In this case, the CPMV sets of neighboring blocks of the current block may become motion vector information set candidates constituting the candidate list in a predetermined order. More specifically, the motion vector information set candidates may include a first candidate derived from the CPMV (or motion vector) of a neighboring block to the left of the current block and a second candidate derived from the CPMV (or motion vector) of a neighboring block above the current block. Here, the left neighboring block is a block adjacent to the bottom left vertex of the current block, and the upper neighboring block is a block adjacent to the top left vertex or the top right vertex of the current block. A candidate list including the motion vector information set candidates obtained in this manner may be generated, and an indicator indicating one of the motion vector information sets in the candidate list may be signaled. According to an embodiment, the indicator may indicate position information of a neighboring block referenced to derive a motion vector of each sub-block of the current block. The decoder can obtain the CPMV set of the current block by referring to the CPMV sets (or motion vectors) of the neighboring blocks indicated by the pointer.

[0101] According to an additional embodiment of the present invention, the CPMV of the current block may be derived based on the CPMVs of neighboring blocks close to the current point. For example, v0 may be obtained by referring to the CPMV of the left neighboring block, and v1 may be obtained by referring to the CPMV of the upper neighboring block. Alternatively, v0 may be obtained by referring to the CPMV of neighboring block A, D, or E, and v1 may be obtained by referring to the CPMV of neighboring block B or C.

[0102] FIG. 14 is a diagram illustrating an affine motion compensation method according to an embodiment of the present invention. According to an embodiment of the present invention, CPMV may be required for affine motion prediction, and a motion vector field, i.e., a motion vector for a sub-block or a certain position (or pixel), may be calculated based on the CPMV. In this specification, the CPMV may be referred to as a seed vector. In this case, the CPMV may be derived based on a predictor. For example, the predictor may be determined as the CPMV. As another example, the CPMV may be calculated based on a predictor and a difference. Specifically, the CPMV may be calculated by adding or subtracting the difference from the predictor.

[0103] According to one embodiment of the present invention, a predictor for CPMV may be derived from the CPMV or MV of a neighboring affine motion compensation (MC) block (i.e., a neighboring block coded with affine mode). For example, if a block at a previously set position is affine motion predicted, the encoder / decoder can derive a predictor for affine motion compensation of the current block from the CPMV or MV of the previously set block. In one embodiment, the previously set position may be positions A0, A1, B0, B1, or B2 shown in FIG. 14. Alternatively, the previously set position may include positions adjacent to and / or non-adjacent to the current block. In addition, the encoder / decoder may refer to the CPMV or MV of a previously set position that is spatially adjacent, or may refer to the CPMV or MV of a previously set position that is temporally adjacent. In this specification, the candidate based on the affine MC block described above may be referred to as an inherited candidate. Alternatively, the candidate based on the affine MC block described above may be referred to as a merge candidate. In addition, in one embodiment, when the encoder / decoder refers to a pre-established position, it can refer to it in a pre-established order.

[0104] FIG. 15 is a diagram illustrating an affine motion compensation method according to an embodiment of the present invention. CPMV may be required for affine motion prediction, and a motion vector field, i.e., a motion vector for a sub-block or a certain position, may be calculated based on the CPMV. In this specification, the CPMV may be referred to as a seed vector. In this case, the CPMV may be derived based on a predictor. For example, the predictor may be determined as the CPMV. As another example, the CPMV may be calculated based on a predictor and a difference. Specifically, the CPMV may be calculated by adding or subtracting the difference from the predictor.

[0105] According to one embodiment of the present invention, an encoder / decoder may derive a predictor of CPMV using MVs of neighboring blocks. In this case, the MVs of neighboring blocks may include MVs other than MVs of affinely MC-enhanced blocks. For example, when deriving each CPMV of a current block, the encoder / decoder may use an MV of a previously set position for each CPMV as a predictor of the CPMV. For example, the previously set position may be a portion included in a block adjacent to the portion.

[0106] Referring to FIG. 15, the encoder / decoder can determine CPMVs mv0, mv1, and mv2. Here, mv0 represents the upper-left CPMV, mv1 represents the upper-right CPMV, and mv2 represents the lower-left CPMV. According to an embodiment of the present invention, the encoder / decoder can use MVs corresponding to previously set positions A, B, and C as predictors for mv0. In addition, the encoder / decoder can use MVs corresponding to previously set positions D and E as predictors for mv1. In addition, the encoder / decoder can use MVs corresponding to previously set positions F and G as predictors for mv2.

[0107] When determining each predictor of CPMV mv0, mv1, and mv2 according to the embodiment of FIG. 15 , the order of referencing predefined positions for each control point position may be predefined. For example, there may be multiple predefined positions referenced as predictors of CPMV for each control point position, and possible combinations of predefined positions may be predefined. Candidates (or predictors) for affine MC can be derived using the method according to the embodiment of FIG. 15 , and such candidates may be referred to as constructed candidates. Alternatively, such candidates may be referred to as inter candidates or virtual candidates. Furthermore, in the method of FIG. 15 , when referencing predefined positions, the encoder / decoder may refer to them in a predefined order.

[0108] According to an embodiment of the present invention, the encoder / decoder may generate an affine MC candidate list or an affine MC CPMV candidate list using the embodiments described in Figures 12 to 15 or a combination thereof. The following Equation 8 illustrates a method for deriving an affine motion predictor according to an embodiment of the present invention.

[0109]

number

[0110] As described above, the CPMV for affine motion prediction of the current block can be derived from neighboring affine motion predicted blocks. Here, Equation 8 may be used. In Equation 8, the MVs of the top-left, top-right, and bottom-left control points of neighboring affine motion predicted blocks may be expressed as (v_E0x, v_E0y), (v_E1x, v_E1y), and (v_E2x, v_E2y), respectively. Furthermore, the coordinates of the top-left, top-right, and bottom-left of neighboring affine motion predicted blocks may be expressed as (x_E0, y_E0), (x_E1, y_E1), and (x_E2, y_E2), respectively. In this case, the encoder / decoder can calculate (v_0x, v_0y), (v_1x, v_1y), which are the predictors or CPMVs of the current block, according to Equation 8. Equations 9 and 10 below illustrate a method for deriving an affine motion predictor according to one embodiment of the present invention.

[0111]

number

[0112]

number

[0113] As described above, multiple CPMVs or multiple CPMV predictors may be used for affine motion compensation. In this case, the encoder / decoder can derive another CPMV or CPMV predictor from one CPMV or CPMV predictor. For example, the encoder / decoder can derive (or generate) two CPMVs or two CPMV predictors using the above-described method, and derive another CPMV or another CPMV predictor based on the two CPMVs or two CPMV predictors.

[0114] According to one embodiment of the present invention, the encoder / decoder may derive (or generate) mv0, mv1, and mv2, which are the CPMV predictors or CPMVs for the upper left, upper right, and lower left corners, using Equations 9 and 10. In Equations 9 and 10, x and y represent the x-component and y-component, respectively, and the size of the current block may be w*h. The encoder / decoder may generate a predictor for CPMV to perform affine MC on the current block and add a difference to the predictor to determine the CPMV. According to one embodiment, the CPMV predictor may be generated using the methods described in FIGS. 14 and 15, and the CPMV difference may be signaled from the encoder to the decoder.

[0115] Also, in one embodiment, a difference may exist for each CPMV, and each difference may be signaled. In a four-parameter model, two CPMVs may be used, and the MV for each CPMV may be derived by adding mvd0 and mvd1 to the predictor for each CMPV, respectively. In a six-parameter model, three CPMVs may be used, and the MV for each CPMV may be derived by adding mvd0, mvd1, and mvd2 to the predictor for each CMPV, respectively.

[0116] According to one embodiment of the present invention, the encoder / decoder can determine the motion vector differential based on the following mathematical formula 11.

[0117]

number

[0118] In Equation 11, if the reference list is 0, MvdL0[x0][y0][compIdx] may be set to lMvd[compIdx]. In this case, compIdx represents a component index and may have a value of 0 or 1. If the reference list is 1, MvdL1[x0][y0][compIdx] may be set to lMvd[compIdx].

[0119] In one embodiment, a motion vector difference may be signaled by the method described in Table 1 above, and the motion vector difference may be lMvd in Equation 11. As described above, the encoder / decoder may determine the signaled mvd as a difference for the CPMV predictor, and the determined difference may be MvdL0 and MvdL1 in Equation 11. In Equation 11, L0 represents reference list 0, and L1 represents reference list 1.

[0120] According to one embodiment of the present invention, the encoder / decoder may derive a predictor for CPMV and add a difference to the CPMV to perform affine MC on the current block. As one example, the encoder / decoder may generate a predictor for CPMV using the methods described with reference to FIGS. 14 and 15. The difference in CPMV may be signaled from the encoder to the decoder. In one embodiment, the encoder / decoder may derive CPMV based on the following Equations 12 and 13.

[0121]

number

[0122]

number

[0123] Referring to Equations 12 and 13, a predictor for the difference for each CPMV may exist. In one embodiment, a difference for another CPMV may be determined based on the difference for one CPMV. This may be based on the similarity between the differences for the CPMV. That is, since the differences for the CPMV are similar, once a predictor for one CPMV is determined, a predictor for the CPMV may be determined using relatively little information. As an example, a difference predictor for the CPMV may be signaled, and the difference between the difference predictor for the CPMV and the difference may be signaled from the encoder to the decoder. Equation 12 assumes the use of a four-parameter model, and Equation 13 assumes the use of a six-parameter model.

[0124] Referring to Equations 12 and 13, the encoder / decoder can determine the difference and CPMV for each CPMV based on the difference (mvd0) of mv0, which is CPMV 0. mvd0, mvd1, and mvd2 in Equations 12 and 13 may be signaled from the encoder to the decoder. Compared to the method described in FIG. 15 above, the methods of Equations 12 and 13 may signal different values ​​of mvd1 and mvd2, even though they use the same predictors as the method described in FIG. 15 above. If the differences between the predictors of CPMVs mv0, mv1, and mv2 are similar, the absolute values ​​of mvd1 and mvd2 may be smaller when using the methods of Equations 12 and 13 than when using the method described in FIG. 15 above, thereby effectively reducing the signaling overhead of mvd1 and mvd2. The encoder / decoder can determine the difference between the predictor of mv1 as (mvd1 + mvd0), and the difference between the predictor of mv2 as (mvd2 + mvd0). In Equations 12 and 13, the letters with a bar above them represent the predictors of CPMV.

[0125] According to an embodiment of the present invention, the encoder / decoder may determine the motion vector differential based on the following Equation 14. In this case, the above-described Equations 12 and 13 may be applied.

[0126]

number

[0127] In one embodiment, a motion vector difference may be signaled according to Table 1 described above or Table 2 described below. The signaled motion vector difference may be lMvd in Equation 14. Furthermore, mvd signaled in Equations 12 and 13, i.e., mvd0, mvd1, and mvd2, may be lMvd in Equation 14. In Equation 14, MvdLX may represent the difference between each CPMV and the predictor, i.e., (mv-mvp). In this case, as described in Equations 12 and 13, for CPMV 0, the signaled motion vector difference may be directly used as the difference (MvdLX) for the CPMV. For other CPMVs, the signaled motion vector difference (mvd1, mvd2 in Equations 12 and 13) and the motion vector difference signaled for CPMV 0 (mvd0 in Equations 12 and 13) may be used as the CPMV difference MvdLX.

[0128] In Equation 14, LX may represent reference list X. compIdx represents a component index and may indicate x and y components. cpIdx may represent a control point index. cpIdx may mean 0, 1 or 0, 1, 2 as represented in Equations 12 and 13. In one embodiment, in Table 1, Equation 14, and Table 2, the encoder / decoder may take into account the resolution of the motion vector difference. For example, when the resolution is R, the value of lMvd*R may be used as lMvd in Equation 14. Table 2 below illustrates an example of a motion vector difference syntax structure according to one embodiment of the present invention.

[0129] [Table 2]

[0130] Referring to Table 2, the motion vector differential may be coded in substantially the same manner as described above in Table 1. In this case, coding of the motion vector differential may be performed individually according to cpIdx and / or control point index.

[0131] FIG. 16 illustrates a subblock-based temporal MVP (SbTMVP) according to an embodiment of the present invention. According to an embodiment of the present invention, an encoder / decoder can incorporate (or induce) a motion vector of a temporal neighboring block on a subblock-by-subblock basis. For example, the encoder / decoder can incorporate a motion vector corresponding to a subblock of a current block from another picture. Here, the block may be a CU, and the subblock may be a sub-CU. Furthermore, the other picture may be a collocated picture. In the present invention, the SbTMVP may also be referred to as a subblock-based temporal merge candidate, or alternative temporal motion vector prediction (ATMVP).

[0132] According to one embodiment of the present invention, when the SbTMVP mode is applied, the encoder / decoder can search for a block corresponding to the current block (which can also be called a corresponding block or a reference block) from another picture. In this case, the corresponding block can be called a collocated block. As an example, the collocated block may be determined as a block in another picture corresponding to a predetermined position, such as the bottom right corner or center position of the current block. In SbTMVP, the collocated block may be a motion-compensated block. For example, the encoder / decoder can search for the collocated block by referring to motion vectors around the current block.

[0133] Referring to FIG. 16, a co-located block may be determined (or guided) as a block located in another picture, which is moved from the current block position by the motion vector of position A1 among neighboring blocks of the current block. In one embodiment, the motion vector pointing (or heading) to the co-located block may be guided based on the motion vector of a candidate searched by scanning predetermined positions around the current block according to a predetermined order. In yet another embodiment, the motion vector pointing to the co-located block may be determined based on a part of a merge candidate list. For example, the encoder / decoder may determine the co-located block using the first merge candidate in the merge candidate list.

[0134] According to an embodiment of the present invention, an encoder / decoder may determine (or derive) motion information of a sub-block of a current block from motion information of a sub-block of a co-located block. In this specification, a motion vector may refer to motion information including a motion vector, a reference index, information on whether a reference list is used, etc., regardless of its expression. Furthermore, the sub-blocks of the co-located block that are the basis for the motion information of the sub-blocks of the current block may have the same relative positions within a block (CU). That is, both sub-blocks may have the same relative positions within the block.

[0135] Furthermore, according to one embodiment of the present invention, when SbTMVP is used, the sub-block size may be fixed to a predefined size. For example, it may be fixed to a size of 8x8. In another embodiment, the sub-block size may be variable. In this case, sequence, picture, slice, CTU, or CU unit signaling may be present to determine the variable sub-block size. For example, the sub-block size may be variably determined to be a size of 4x4 or 8x8.

[0136] 17 is a diagram illustrating planar MVP according to an embodiment of the present invention. According to an embodiment of the present invention, an encoder / decoder may generate a sub-block-based motion vector field of a current block using the planar MVP method. The encoder / decoder may also refer to neighboring motion vectors using the planar MVP method. In the planar MVP method, the encoder / decoder may determine a sub-block motion vector based on a linear weighted summation using one or more neighboring motion vectors. In this case, the neighboring motion vector may include motion information extrapolated from the motion vectors of neighboring blocks. For example, the encoder / decoder may extrapolate a motion vector below or to the right of the current block using the motion vectors of neighboring blocks.

[0137] Referring to FIG. 17, a motion vector P(x,y) corresponding to a sub-block of a current block may be determined based on P_h(x,y) or P_v(x,y). In FIG. 17, W and H represent the width and height of the current block, respectively. In this case, P_h(x,y) may be determined based on L(-1,y) or R(W,y). L(-1,y) may be a motion vector corresponding to a sub-block located adjacent to the current block on the left side of the current sub-block. R(W,y) may be a value based on a motion vector corresponding to the upper right position of the current block and a temporal motion vector corresponding to the lower right position of the current block. P_v(x,y) may be determined based on A(x,-1) or B(x,H). A(x,-1) may be a motion vector corresponding to a sub-block located above the current sub-block and adjacent to the current block. Also, B(x,H) may be a weighted sum value based on a motion vector corresponding to the bottom left position of the current block and a top right motion vector corresponding to the bottom right position of the current block.

[0138] Table 3 below illustrates an example of an inter prediction related syntax structure according to an embodiment of the present invention.

[0139] [Table 3A]

[0140] [Table 3B]

[0141] According to an embodiment of the present invention, the inter prediction method may include skip mode, merge mode, inter mode, etc. In one embodiment, a residual signal may not be transmitted in skip mode. Also, the same MV determination method as in merge mode may be applied to skip mode. For example, whether or not skip mode is used may be determined by a skip flag (or syntax element). Referring to Table 3, whether or not skip mode is used may be determined by a value of cu_skip_flag. The cu_skip_flag indicates whether or not skip mode is applied to the current CU.

[0142] In one embodiment, when the merge mode is applied, the motion vector difference may not be used. In this case, the encoder / decoder may determine the motion vector based on the motion candidate index. For example, whether the merge mode is used may be determined by a merge flag (or a syntax element). Referring to Table 3, whether the merge mode is applied may be determined by the value of the merge flag (i.e., merge_flag). Also, the merge mode may be used when the skip mode is not used.

[0143] According to an embodiment of the present invention, an encoder / decoder selectively uses one or more candidate list sets in skip mode or merge mode. For example, the candidate list set may include a candidate list using merge candidates and / or a candidate list using sub-block-based merge candidates. The merge candidates may include spatial neighboring candidates, temporal neighboring candidates, etc. The merge candidates may include candidates using a motion vector for the entire current block (CU). That is, the merge candidates may include candidates in which the motion vectors of each sub-block belonging to the current block are the same. The sub-block merge candidates may include sub-block-based temporal motion vectors, affine merge candidates, etc. In one embodiment, the sub-block merge candidates may include candidates using different motion vectors for each sub-block of the current block (CU). In one embodiment, the affine merge candidates may represent merge candidates that determine the CPMV of affine motion prediction without motion vector difference. The sub-block merge candidates may include a method of determining a motion vector for each sub-block of the current block. For example, the sub-block merging candidates may include planar MVs, regression-based MVs, STMVPs, etc., in addition to the sub-block-based temporal MVs and affine merging candidates mentioned above.

[0144] According to one embodiment of the present invention, in inter mode, the encoder / decoder can use a motion vector differential. A motion vector predictor can be determined based on a motion candidate index, and a motion vector can be determined based on the motion vector predictor and the motion vector differential. Whether or not inter mode is used can be determined based on whether other modes are used. In another embodiment, whether or not inter mode is used can be determined by a flag (or a syntax element). Table 3 assumes an example in which inter mode is used when other modes, such as skip mode and merge mode, are not used. In one embodiment, inter mode can include AMVP mode, affine inter mode, etc. Inter mode can be a mode in which a motion vector is determined based on a motion vector predictor and a motion vector differential. In affine inter mode, the encoder / decoder can use the motion vector differential when determining CPMV for affine motion prediction.

[0145] Referring to Table 3, the encoder / decoder may determine whether to use a sub-block merging candidate or a merge candidate after determining the skip mode or the merge mode. For example, the decoder may parse merge_subblock_flag, which indicates whether to use a sub-block merging candidate if a specific condition is met. In this case, the specific condition may be a condition related to block size. For example, the specific condition may be a condition related to width, height, area, etc., or a combination of these conditions.

[0146] Referring to Table 3, for example, the specific condition may be whether the width and height of the current block (CU) are equal to or greater than a specific value. If merge_subblock_flag is not parsed, the encoder / decoder can infer (infer) its value as 0. If merge_subblock_flag is 1, the encoder / decoder can use a subblock merge candidate, and if merge_subblock_flag is 0, the encoder / decoder can use a merge candidate. If a subblock merge candidate is used, the encoder / decoder can parse merge_subblock_idx, which indicates an index (or syntax element) indicating a specific candidate in a candidate list (subblock merge candidate list). If a merge candidate is used, the encoder / decoder can parse merge_index, which indicates an index indicating a specific candidate in a candidate list (merge candidate list). In this case, the encoder / decoder does not need to parse if the maximum number of candidates in the candidate list is 1. That is, the encoder / decoder can parse the index if the maximum number of candidates in the candidate list is greater than 1. If merge_subblock_idx or merge index is not parsed, the encoder / decoder can infer its value as 0.

[0147] The above Table 3 illustrates coding unit functions, and the contents related to intra prediction may be omitted. Table 3 may be an example of a case where it is determined that inter prediction is applied.

[0148] A method for constructing a sub-block merging candidate list according to an embodiment of the present invention will be described below. Sub-block merging candidates are described in Table 3 above, and a method for constructing a sub-block merging candidate list will be described below. According to an embodiment of the present invention, an encoder / decoder may determine whether to add a candidate to a candidate list by taking into account the size of a current block or a sub-block size. The above-mentioned size may be defined based on the width, height, or area of ​​a block or sub-block.

[0149] In one embodiment, when there are multiple candidates that can belong to the subblock merging candidate set and the multiple candidates have different subblock sizes, it can be determined whether to add a candidate to the candidate list taking into account the size of the current block. Furthermore, the condition for considering the size of the current block for a candidate can be based on the subblock size of the candidate. Here, a candidate (before being added to the candidate list) can represent any MVP method, mode, etc., and can include any method of creating an MVP or MV. A candidate added to the candidate list can represent a candidate, MVP method, mode, etc. that can be used for the current block.

[0150] According to an embodiment of the present invention, an encoder / decoder may determine whether to add a candidate to a candidate list according to a block size condition of a current block. For example, the encoder / decoder may determine whether to add a candidate to a candidate list according to whether the size of the current block is equal to or greater than a certain block size threshold. The block size condition or threshold may be determined according to the sub-block size of the candidate. This is because the sub-block-based prediction method can improve compression efficiency by using sub-block-based prediction for blocks that are larger than the sub-block size or significantly larger than the sub-block size. For example, even if sub-block-based prediction is applied to a block that is the same size as or smaller than the sub-block size, sub-block-based motion information cannot be obtained, which may result in poor compression efficiency.

[0151] In one embodiment, the encoder / decoder may construct a first candidate list when the size of the current block is equal to or less than the preset size, and may construct a second candidate list when the size of the current block is equal to or greater than the preset size. Furthermore, the candidate types belonging to candidate list 2 may include candidate types belonging to candidate list 1. For example, if the subblock size of the first mode is set to the first subblock size and the subblock size of the second mode is set to the second subblock size, the encoder / decoder may add only the second mode to the candidate list when the current block is equal to the preset size, and may add both the first mode and the second mode to the candidate list when the current block is larger than the preset size. For example, the first subblock size may be 8x8 and the second subblock size may be 4x4. The preset size may be 8x8. Alternatively, the width (or height) of the preset size may be 8. For example, the first mode may be SbTMVP and the second mode may be affine merge mode.

[0152] In one embodiment, a sub-block based merging candidate list may be constructed (or generated) in the following manner: In the following, first, i=0 may be set.

[0153] If the (availableFlagSbCol && blockSizeCondition) value is true, then subblockMergeCandList[i++] is set to SbCol, i.e., SbCol may be added to the candidate list, where subblockMergeCandList[i++] represents the i++th subblock merge candidate in the subblock merge candidate list, and SbCol represents the subblock-based temporal merge candidate (or SbTMVP).

[0154] - If the value of (availableFlagA && i<MaxNumSubblockMergeCand) is true, then subblockMergeCandList[i++] is set to A. That is, the A candidate (or the candidate at the A position) may be added as the next candidate.

[0155] - If the value of (availableFlagB && i<MaxNumSubblockMergeCand) is true, then subblockMergeCandList[i++] is set to B. That is, the B candidate may be added as the next candidate.

[0156] - If the value of (availableFlagConst1 && i<MaxNumSubblockMergeCand) is true, then subblockMergeCandList[i++] is set to Const1. That is, the Const1 candidate may be added as the next candidate. Const1 represents the first candidate among the configured candidates described above.

[0157] - If the value of (availableFlagConst2 && i<MaxNumSubblockMergeCand) is true, then subblockMergeCandList[i++] is set to Const2. That is, the Const2 candidate may be added as the next candidate. Const2 represents the second candidate among the configured candidates described above.

[0158] - If the value of (availableFlagConst3 && i<MaxNumSubblockMergeCand) is true, then subblockMergeCandList[i++] is set to Const3. That is, the Const3 candidate may be added as the next candidate. Const3 represents the third candidate among the configured candidates described above.

[0159] - If the value of (availableFlagConst4 && i<MaxNumSubblockMergeCand) is true, then subblockMergeCandList[i++] is set to Const4. That is, the Const4 candidate may be added as the next candidate. Const4 represents the fourth candidate among the configured candidates described above.

[0160] - If the value of (availableFlagConst5 && i<MaxNumSubblockMergeCand) is true, then subblockMergeCandList[i++] is set to Const5. That is, the Const5 candidate may be added as the next candidate. Const5 represents the fifth candidate among the configured candidates described above.

[0161] - If the value of (availableFlagConst6 && i<MaxNumSubblockMergeCand) is true, then subblockMergeCandList[i++] is set to Const6. That is, the Const6 candidate may be added as the next candidate. Const6 represents the sixth candidate among the configured candidates described above.

[0162] According to one embodiment of the present invention, the encoder / decoder adds an SbCol candidate to a candidate list, subblockMergeCandList, only if the candidate satisfies a predefined block size condition. In one embodiment, the block size condition for determining whether to add SbTMVP to the candidate list may be determined by the subblock size of SbTMVP. For example, the block size condition may be (cbWidth>=16 && cbHeight>=16). Here, cbWidth represents the width of the current coding block, and cbHeight represents the height of the current coding block. Also, A, B, Const1, Const2, Const3, Const4, Const5, and Const6 may be affine merge candidates. A and B may be affine inheritance candidates, and ConstX (X is one of 1 to 6) may be an affine constructed candidate.

[0163] In another embodiment, the encoder / decoder may determine whether to parse the candidate index according to the block size condition of the current block. For example, when there are multiple candidates, if all but one of the multiple candidates cannot be added to the candidate list according to the block size condition and other conditions, the encoder / decoder may not parse the candidate index. For example, when there are two candidates that can be added to the candidate list, the encoder / decoder may determine whether to parse the candidate index according to the block size condition. For example, if one of the two candidates that can be added to the candidate list does not satisfy the block size condition, the encoder / decoder may not parse the candidate index.

[0164] Table 4 below illustrates an example of inter prediction related syntax according to an embodiment of the present invention.

[0165] [Table 4A]

[0166] [Table 4B]

[0167] According to an embodiment of the present invention, the sub-block mode may be used when the size of the current block is larger than the sub-block size. For example, when there are multiple candidates that can be added to the sub-block merge candidate list and the multiple candidates have the same sub-block size, the sub-block merge mode may be used if the size of the current block is larger than the sub-block size. As a specific example, if the sub-block size is 8x8 and the current block size is 16x16 or larger (or the width or height is 16 or larger), the sub-block mode may be used. The use of the sub-block mode may indicate the presence or absence of a parsing flag indicating whether a mode corresponding to the sub-block mode is used. In addition, the flag indicating whether a mode corresponding to the sub-block mode is used may be merge_subblock_flag, inter_affine_flag, etc. In Table 4, descriptions that overlap with those in Table 3 above will be omitted.

[0168] According to one embodiment of the present invention, referring to Table 4, the decoder can parse merge_subblock_flag and inter_affine_flag only if cbWidth>=16 and cbHeight>=16. As another embodiment, the encoder / decoder can use the sub-block mode when the current block size is equal to the sub-block size as well as when it is greater than the sub-block size. This is because the motion compensation method of the sub-block mode may differ from that of modes other than the sub-block mode. In this specification, the sub-block-based merge mode may be referred to as the sub-block mode.

[0169] Table 5 below illustrates an example of inter prediction related syntax according to an embodiment of the present invention.

[0170] [Table 5A]

[0171] [Table 5B]

[0172] According to one embodiment of the present invention, the sub-block mode can be used when the width or height of the current block satisfies a specific condition. For example, even if the width or height of the current block is equal to the size of one side of the sub-block size, if the unequal width or height is larger than the size of one side (or the other side) of the sub-block size, multiple sub-blocks may exist within the current block. Therefore, the sub-block mode can be used when the width or height of the current block is larger than the width or height of the sub-block. Alternatively, the sub-block mode can be used when the width or height of the current block is larger than or equal to the width or height of the sub-block. In Table 5, descriptions that overlap with Tables 3 and 4 above are omitted. Referring to Table 5, if the width or height of the current block is 8 or greater, the encoder / decoder can use the sub-block merge mode. Furthermore, if the width or height of the current block is 16 or greater, the decoder can use the affine inter mode.

[0173] Tables 6 and 7 below illustrate sub-block merging related syntax according to an embodiment of the present invention.

[0174] [Table 6]

[0175] [Table 7]

[0176] According to one embodiment of the present invention, a first unit may signal whether or not a specific mode is to be used. If the signaling indicates that a specific mode is to be used, the encoder may signal to a decoder which mode is to be actually used in a second unit belonging to the first unit. In this case, whether or not to parse a syntax element in the second unit may be determined based on the signaling value in the first unit. For example, the first unit may be a sequence, picture, slice, tile, etc., and the second unit may be a CTU, CU, etc. In one embodiment, the specific mode may be a sub-block mode. The first unit may be referred to as an upper unit, and the second unit may be referred to as a lower unit.

[0177] Table 6 shows a signaling parsing process for determining whether or not a sub-block mode is used when the sub-block sizes of modes belonging to the sub-block mode are all the same or when differences in sub-block size are not taken into consideration. According to an embodiment of the present invention, when there are multiple modes belonging to the sub-block mode and multiple upper unit usability signaling for a mode corresponding to the sub-block mode also exist, if at least one of the multiple upper unit usability signaling indicates that the sub-block mode is usable, the lower unit can use the sub-block mode. In addition, the upper unit usability signaling may correspond to multiple modes. For example, when different modes exist depending on the position of a reference neighbor block, the usability of the upper unit for multiple of these modes may be signaled. This may be applied in combination with conditions such as a block size condition. In this specification, whether or not the sub-block mode is usable may mean parsing signaling indicating whether or not the sub-block mode is used, and for this, see the above embodiment.

[0178] Referring to Table 6, the decoder can parse merge_subblock_flag only if at least one of sps_affine_enabled_flag and sps_sbtmvp_enabled_flag, which indicate whether the affine merge mode belonging to the subblock merge mode and SbTMVP are available, is 1.

[0179] Referring to Table 7, if the sub-block sizes of modes belonging to the sub-block mode may differ, the decoder can parse the syntax element indicating whether the sub-block mode is to be used taking this into consideration. According to one embodiment of the present invention, if the signaling indicating whether an upper unit of a mode belonging to the sub-block mode is usable and the block size condition of the mode are all satisfied, the encoder / decoder can use the sub-block mode. For example, when there are first and second modes belonging to the sub-block mode, first signaling indicating whether an upper unit corresponding to the first mode is usable and second signaling indicating whether an upper unit corresponding to the second mode is usable are used, and there are a first block size condition for the first mode and a second block size condition for the second mode, the encoder / decoder can use the sub-block mode if the first signaling indicating whether the upper unit is usable indicates that it is usable and the first block size condition is satisfied. Furthermore, if the second signaling indicating whether the upper unit is usable indicates that it is usable and the second block size condition is satisfied, the encoder / decoder can use the sub-block mode.

[0180] As an example, referring to Table 7, syntax elements sps_affine_enabled_flag and sps_sbtmvp_enabled_flag, which respectively indicate whether the affine merge mode and SbTMVP included in the subblock merge mode are enabled, may be signaled from the encoder to the decoder. In this case, the decoder parses merge_subblock_flag if the condition of the following mathematical formula 15 is satisfied. Similarly, the encoder can syntax-signal merge_subblock_flag if the following mathematical formula 15 is satisfied.

[0181]

number

[0182] Table 7 also illustrates a case where the block size condition for the affine merge mode is (cbWidth>=8 && cbHeight>=8) and the block size condition for SbTMVP is (cbWidth>=16 && cbHeight>=16).

[0183] Table 8 below shows syntax that represents other examples in which the conditions in Table 7 are used.

[0184] [Table 8]

[0185] There are a first mode and a second mode belonging to the sub-block mode, there is a first signaling indicating whether an upper unit corresponding to the first mode is available, and there is a second signaling indicating whether an upper unit corresponding to the second mode is available, and there is a first block size condition 1 for the first mode and a block size condition 2 for the second mode. If the first block size condition is included in the second block size condition (i.e., if the first block size condition is satisfied, the second block size condition is satisfied), the encoder / decoder can use the sub-block mode as shown in the following mathematical formula 16.

[0186]

number

[0187] In another embodiment, there may be a higher unit usability signaling corresponding to all sub-block modes. Alternatively, there may be a higher unit usability signaling corresponding to all sub-block merging modes. In this case, the decoder may parse the lower unit signaling only if the higher unit usability signaling indicates that the lower unit is usable. According to one embodiment of the present invention, the encoder / decoder may set a block size condition for using the sub-block mode according to the sub-block size. In this case, if the sub-block size is variable, the encoder / decoder may set a block size condition based on the largest size among the available sub-block sizes.

[0188] Tables 9 and 10 below show sub-block merging related syntax according to an embodiment of the present invention.

[0189] [Table 9]

[0190] [Table 10]

[0191] According to one embodiment of the present invention, a first unit may signal whether a specific mode is available. If the signaling indicates that a specific mode is available, the encoder may signal to a decoder which mode the second unit belonging to the first unit will actually use. In this case, whether or not to parse a syntax element in the second unit may be determined based on the signaling value in the first unit. For example, the first unit may be a sequence, picture, slice, tile, etc., and the second unit may be a CTU, CU, etc. In one embodiment, the specific mode may be a sub-block mode. The first unit may be referred to as an upper unit, and the second unit may be referred to as a lower unit. The upper unit availability signaling may correspond to multiple modes. For example, when different modes exist depending on the position of a neighboring block to be referenced, upper unit availability signaling may exist for multiple of these modes.

[0192] According to an embodiment of the present invention, whether or not to parse a candidate index may be determined by referring to a higher unit usability signaling. For example, if one or more of multiple higher unit usability signalings indicate that a candidate index is usable, the candidate index may be parsed. As an example, there may be higher unit usability signaling corresponding to multiple modes (wherein the mode may represent a unit included as a candidate in a candidate list). In this specification, the higher unit usability signaling corresponding to the multiple modes may be indicated by a first flag, a second flag, a third flag, etc. Furthermore, there may be higher unit usability signaling corresponding to one mode (a candidate; a unit that can occupy one position in a candidate list). In this specification, the higher unit usability signaling corresponding to one mode may be referred to as an A flag, a B flag, or a C flag.

[0193] According to one embodiment of the present invention, if at least one of the upper unit usability signalings corresponding to multiple modes indicates usability, the encoder / decoder can parse the candidate index. That is, if (first flag || second flag || third flag, ...) (i.e., if the flag value 1 indicates usability), the encoder / decoder can parse the candidate index.

[0194] Furthermore, if there is zero or one upper unit usability signaling corresponding to one mode, or if there is zero or one mode without upper unit usability signaling, and at least one of the upper unit usability signaling corresponding to multiple modes indicates that the mode is usable, the encoder / decoder can parse the candidate index. That is, when (first flag || second flag || third flag, ...) (when the flag value 1 indicates usable), the encoder / decoder can parse the candidate index.

[0195] If there is one upper unit usability signaling corresponding to one mode and one mode without an upper unit usability signaling, the encoder / decoder can parse the candidate index if the upper unit usability signaling corresponding to the one mode indicates usability. This may be regardless of the value of the upper unit usability signaling corresponding to the other modes. Therefore, in this case, the encoder / decoder can parse the candidate index if at least one of the upper unit usability signaling corresponding to the other modes and the upper unit usability signaling corresponding to one mode indicates usability. That is, when (first flag || second flag || third flag, ... || A flag) (when the flag value 1 indicates usability), the encoder / decoder can parse the candidate index.

[0196] If the candidate index is not parsed, the encoder / decoder can use the only candidate in the candidate list, or if the candidate index is not parsed, the encoder / decoder can infer the candidate index to be 0.

[0197] Referring to Table 9, the affine merge mode and SbTMVP may correspond to the subblock merge mode, and the affine merge mode may be configured as multiple candidates from this method. Also, the availability of the upper unit corresponding to the affine merge mode may be signaled by the sps_affine_enabled_flag syntax element. In this case, if sps_affine_enabled_flag indicates availability, the encoder / decoder can parse the candidate index, merge_subblock_idx.

[0198] According to one embodiment of the present invention, if there are two or more upper unit usability signalings corresponding to one mode, the encoder / decoder can parse a candidate index if at least two of the upper unit usability signalings corresponding to one mode indicate that the mode is usable. That is, when (A flag + B flag + C flag + ... >= 2) (when flag value 1 indicates usable), the encoder / decoder can parse a candidate index. In this case, there may not be a mode without upper unit usability signaling.

[0199] If there are two or more upper unit usability signalings corresponding to one mode and there are upper unit usability signalings corresponding to multiple modes, the encoder / decoder can parse the candidate index if at least two of the multiple upper unit usability signalings corresponding to one mode indicate that they are usable, or if at least one of the multiple upper unit usability signalings corresponding to multiple modes indicates that they are usable. That is, when ((first flag || second flag || third flag ...) || (A flag + B flag + C flag + ... >= 2)), (when flag value 1 indicates usable), the encoder / decoder can parse the candidate index. In this case, there may not be a mode without upper unit usability signaling.

[0200] If there is one mode without higher unit usability signaling, the candidate index can be parsed when at least one of the higher unit usability signalings corresponding to one mode is usable, rather than when at least two of the higher unit usability signalings corresponding to one mode are usable. That is, the encoder / decoder can parse the candidate index when (A flag || B flag || C flag + ...) is satisfied, rather than (A flag + B flag + C flag + ... >= 2).

[0201] Referring to Table 10, the sub-block merge mode can include affine merge mode, SbTMVP, and planar MVP. The affine merge mode may be configured as multiple candidates. SbTMVP may be configured as one candidate. Planar MVP may be configured as one candidate. In addition, the sps_affine_enabled_flag, sps_sbtmvp_enabled_flag, and sps_planar_enabled_flag syntax elements may be used to signal whether or not the upper units corresponding to affine merge mode, SbTMVP, and planar MVP are available, respectively. In this case, the encoder / decoder can parse merge_subblock_idx, which is a candidate index, when sps_affine_enabled_flag indicates that it is enabled, or when sps_sbtmvp_enabled_flag and sps_planar_enabled_flag both indicate that it is enabled (or when two or more of sps_sbtmvp_enabled_flag and sps_planar_enabled_flag are enabled). In the present invention, the indication that both sps_sbtmvp_enabled_flag and sps_planar_enabled_flag are enabled may be expressed as (sps_sbtmvp_enabled_flag+sps_planar_enabled_flag>=2).

[0202] The SbTMVP usable condition according to one embodiment of the present invention may be derived in the following manner: Hereinafter, availableFlagSbCol represents a parameter (or a flag) indicating whether SbTMVP is usable.

[0203] - availableFlagSbCol may be set to 0 if one or more of the following conditions are true:

[0204] 1) tile_group_temporal_mvp_enable_flag is 0.

[0205] 2) When sps_sbtmvp_flag is 0.

[0206] 3) If cbWidth is less than 8.

[0207] 4) If cbHeight is less than 8.

[0208] According to one embodiment of the present invention, whether SbTMVP is available or not may be expressed by an availability flag. The availability flag indicating whether SbTMVP is available or not may be availableFlagSbCol. SbTMVP may also be referred to as SbCol. Furthermore, when the availability flag is 1, it indicates that it is available, and when the availability flag is 0, it indicates that it is not available.

[0209] According to one embodiment of the present invention, a condition may exist under which SbTMVP cannot be used. As one embodiment, SbTMVP may be disabled based on the above-described higher unit signaling. The higher unit for the current block, CU, or PU may be a slice, tile, tile group, sequence, picture, CTU, etc. The higher unit signaling may include tile_group_temporal_mvp_enable_flag. For example, tile_group_temporal_mvp_enable_flag represents a flag indicating whether temporal motion vectors (or temporal motion vector predictions) are enabled. Since SbTMVP may be a technology that uses temporal motion vectors, SbTMVP may be disabled if temporal MV usage is not set as enabled. In addition, the higher unit signaling may include sps_sbtmvp_flag. For example, sps_sbtmvp_flag may be a flag indicating whether SbTMVP is enabled.

[0210] In another embodiment, SbTMVP may be disabled based on block size. The block size may include a block width, a block height, or a value based on the block width or height. The value based on the block width or height may include the area of ​​the block. If the block width or height is smaller than a threshold value, SbTMVP may be disabled. In one embodiment, the threshold value may be the width or height of a sub-block. For example, the threshold value may be set to 8.

[0211] If the above-mentioned conditions for not being able to use SbTMVP are met, the encoder / decoder can set the availability flag corresponding to SbTMVP to 0. If not, the encoder / decoder can perform additional operations to determine the availability flag.

[0212] 18 is a diagram illustrating a method for determining a temporary motion vector according to an embodiment of the present invention. In this specification, a motion vector representing the above-mentioned collocated block is referred to as a temporary motion vector. The temporary motion vector may be referred to as an initial motion vector, a temporal motion vector, a subblock-based temporal merging base motion vector, or a base motion vector, without being limited to the names thereof. That is, an encoder / decoder may determine a motion vector identifying the above-mentioned collocated block based on the initial motion vector, the temporal motion vector, the subblock-based temporal merging base motion vector, or the base motion vector.

[0213] According to an embodiment of the present invention, an encoder / decoder may determine a temporary motion vector based on motion information of a position around a current block. The position around the current block may be a previously set position. The position around the current block may include a spatial candidate position of a merge candidate. Alternatively, the position around the current block may include a spatial candidate position of an AMVP candidate.

[0214] Referring to FIG. 18, positions corresponding to A0, A1, B0, B1, and B2 may be pre-set. The positions here may refer to luminance positions. When the top left corner of the current block is (xCb, yCb), A0, A1, B0, B1, and B2 may be (xCb-1, yCb+cbHeight), (xCb-1, yCb+cbHeight-1), (xCb+cbWidth, yCb-1), (xCb+cbWidth-1, yCb-1), and (xCb-1, yCb-1), respectively. Here, cbWidth and cbHeight may be the width and height of the current block, respectively. Furthermore, (xCb, yCb) may be the top left corner of the shared merge list region. When multiple blocks use the same merge candidate list, the multiple block regions may be the shared merge list region. Also, there may be a pre-defined lookup order corresponding to a pre-defined position. Table 11 below shows examples of the pre-defined lookup order.

[0215] [Table 11]

[0216] Referring to Table 11, the predefined positions may be predefinedPosition1, predefinedPosition2, ..., predefinedPositionN, etc. Also, the reference order may be predefined in the order of predefinedPosition1, predefinedPosition2, ..., predefinedPositionN. The reference order may be forward or reverse to the reference order of the merge candidate list construction. Or, the reference order may be forward or reverse to the reference order of the AMVP candidate list construction. For example, the reference positions and reference order may be A1, B1, B0, A0. Or, predefinedPosition1, predefinedPosition2, ..., predefinedPositionN may represent motion information corresponding to the predefined positions.

[0217] In Table 11, the temporary motion vector may be set to a zero motion vector. Also, if motion information at a previously set position is unavailable, the encoder / decoder may set the temporary motion vector to a zero motion vector. Also, if motion information at a previously set position is available, the encoder / decoder may set the motion information at the previously set position to the temporary motion vector. Also, "available" may mean that the corresponding position is not intra-predicted. Or, "available" may mean that the corresponding position is inter-predicted. Or, "available" may mean that the reference picture of the motion information at the corresponding position is the same as the co-located picture corresponding to the current block. Or, "available" may mean that the reference picture of the motion information at the corresponding position is the same as the reference picture corresponding to the current block.

[0218] In Table 11, it is checked whether the motion information of the first corresponding position that satisfies the first usable condition 1 among the already set positions satisfies the second usable condition. If the second usable condition is met, the encoder / decoder can set the motion information of the corresponding position to a temporary motion vector. Otherwise, the encoder / decoder can set the temporary motion vector to a zero motion vector. This can be expressed in pseudo code as shown in Table 12 below.

[0219] [Table 12]

[0220] In the above embodiment, to enable only the first position among the already set positions that satisfy the first usability condition to be used as a temporary motion vector, an else if statement may be used in Table 12 above. The first usability condition and the second usability condition may indicate usability as described above. For example, the first usability condition may indicate that the corresponding position uses inter prediction. The second usability condition may indicate that the reference picture for the motion information of the corresponding position is the same as the co-located picture corresponding to the current block. In the above embodiment, if a certain position satisfies the first usability condition but does not satisfy the second usability condition, it is not necessary to check positions referenced after the certain position.

[0221] In another embodiment, the encoder / decoder may set the motion information of the first corresponding position among the previously set positions that satisfies both the first and second usable conditions as a temporary motion vector. Otherwise, the encoder / decoder may set the temporary motion vector to a zero motion vector. This may be expressed in pseudocode as shown in Table 13 below.

[0222] [Table 13]

[0223] In Table 13, if a certain position satisfies the first usable condition but does not satisfy the second usable condition, the encoder / decoder can also check positions that are referenced after the certain position.

[0224] Also, according to one embodiment of the present invention, there may be only one pre-set position. This case can be represented in pseudo code as shown in Table 14 below.

[0225] [Table 14]

[0226] That is, if motion information corresponding to one pre-defined position is available, the encoder / decoder may set a temporary motion vector to the motion information; otherwise, the encoder / decoder may set the temporary motion vector to a zero motion vector. As an example, the one pre-defined position may be a position on the left side of the current block. That is, the x-coordinate of the one pre-defined position may be smaller than the left coordinate within the current block. For example, the one pre-defined position may be position A1.

[0227] In the above embodiment, an example is shown in which two types of availability conditions, i.e., the first availability condition and the second availability condition, are used, but the present invention is not limited thereto and may include cases in which more availability conditions exist. In addition, the motion information may include a motion vector (mvLX), a reference index (refIdxLX), a prediction utilization flag (predFlagLX), etc.

[0228] In one embodiment, the process of checking the second availability condition and setting the temporary motion vector may be performed as follows: If availableFlagN is true, the following process may be applied.

[0229] A. If all of the following conditions are true, the temporary motion vector may be set to mvL1N:

[0230] a.predFlagL1N is 1

[0231] b. DiffPicOrderCnt(ColPic,RefPicList1[refIdxL1N]) is 0

[0232] If c.DiffPicOrderCnt(aPic, currPic) is less than or equal to 0, aPic represents all pictures in all reference picture lists of the slice (or tile group).

[0233] d.tile_group_type is equal to B

[0234] If e.collocated_from_l0_flag is equal to 0

[0235] B. Otherwise, if both of the following conditions are true, the temporary motion vector may be set to mvL0N:

[0236] a.predFlagL0N is 1

[0237] b. DiffPicOrderCnt(ColPic,RefPicList0[refIdxL0N]) is 0

[0238] Here, availableFlagN being true may indicate that the first availability condition is satisfied. In the above-described embodiment in which only one pre-defined position is used, the parameter that can become availableFlagN may be the availableFlag corresponding to the pre-defined position. For example, if only the A1 position is used, availableFlagN may be availableFlagA1, which may be a value indicating whether motion information for the A1 position is available.

[0239] 19 is a diagram illustrating whether or not SbTMVP is used according to one embodiment of the present invention. According to one embodiment of the present invention, the encoder / decoder can determine whether or not to use a particular mode based on the position of the current block 1901. According to yet another embodiment of the present invention, the encoder / decoder can determine whether or not to use a particular mode based on the position of the merge list sharing area to which the current block 1901 belongs.

[0240] According to one embodiment of the present invention, an encoder / decoder may determine whether to use SbTMVP based on the position of the current block 1901. According to another embodiment of the present invention, whether to use SbTMVP may be determined based on the position of the merge list sharing region to which the current block 1901 belongs. As an example, the position of the current block 1901 or the position of the merge list sharing region to which the current block belongs may be expressed as the coordinates of the top left corner or based on the coordinates of the top left corner. Furthermore, the position of the current block 1901 or the position of the merge list sharing region to which the current block 1901 belongs may be a relative position within a higher group. The higher group may include a picture, a tile, a tile group, a CTU, a slice, etc. Furthermore, the higher group may be a group to which the current block belongs. That is, for example, the encoder / decoder may determine whether to use SbTMVP based on the position of the current block 1901 within the picture 1902. In this specification, the current block or the merge list sharing region of the current block may be referred to as the current region.

[0241] According to one embodiment of the present invention, an encoder / decoder may determine whether to use SbTMVP based on whether the current region borders a boundary of a higher group. For example, if the current region borders a boundary of a higher group, it may determine or decide not to use SbTMVP. Alternatively, it may determine whether to use SbTMVP based on whether pre-defined positions referenced to determine temporary motion vectors corresponding to the current region described with reference to FIG. 19 are outside the higher group. For example, if all pre-defined positions referenced to determine temporary motion vectors corresponding to the current region are outside the higher group, the encoder / decoder may determine or decide not to use SbTMVP. Furthermore, whether all pre-defined positions referenced to determine temporary motion vectors corresponding to the current region are outside the higher group may be determined based on the coordinates of the current region. This is because, according to the above-described embodiment, if all pre-defined positions referenced to determine temporary motion vectors corresponding to the current region are outside the higher group, a zero motion vector is used as the temporary motion vector, which may result in a deterioration in the prediction performance of SbTMVP. Also, in one embodiment, by determining that SbTMVP cannot be used in this case, other modes signaled with the same syntax element as SbTMVP can be signaled using fewer bits. For example, as described above, SbTMVP and affine merge mode may be included in the sub-block merge mode, and if it is determined that SbTMVP cannot be used, affine merge mode can be signaled using fewer indices. Therefore, in such a case, index 0 can always indicate affine merge mode.

[0242] Referring to FIG. 19, as described above, only the A1 position 1903 can be referenced to determine the temporary motion vector for the current region. If motion information for the A1 position 1903 is unavailable, a zero motion vector can be used as the temporary motion vector. If the current region abuts the left boundary of the picture 1902, the temporary motion vector may always be a zero motion vector. Therefore, if the current region abuts the left boundary of the picture 1902, SbTMVP may not be used. Furthermore, if the current region abuts not only the left boundary of the picture 1902 but also the left boundary of a unit that can be processed in parallel, SbTMVP may not be used. The unit that can be processed in parallel may be a tile, a tile group, a slice, etc. Therefore, if the coordinates of the top left corner of the current region are (xCb, yCb), and xCb is 0, the encoder / decoder may determine that the current region abuts the left boundary of the picture 1902, and in this case, SbTMVP may not be used. Alternatively, if (xCb-(left coordinate of the parallel processable unit)) is 0, SbTMVP may not be used.

[0243] Also, in one embodiment, if only the top edge of the current region is referenced to determine the temporary motion vector for the current region, when the current region contacts the top boundary of the picture 1902 or a unit that can be processed in parallel, the encoder / decoder can be configured not to use SbTMVP.

[0244] In another additional embodiment, the conditions for determining not to use SbTMVP have been described. However, in addition to these conditions, SbTMVP may not be used only when a condition related to block size is met. That is, for example, SbTMVP may not be used when the coordinates of the current region satisfy a previously set condition and a condition related to block size. In one embodiment, the condition related to block size may be when the block size of the current block is equal to or smaller than the sub-block size. This is because, if the current block size is equal to or smaller than the sub-block size, the current block has only one sub-block, which may reduce the advantages and performance of SbTMVP. For example, the sub-block size may be 8x8. Therefore, the following embodiment may be derived from the described embodiment. If the current region abuts the left boundary of the picture and the current block is 8x8 or smaller, the encoder / decoder may determine not to use SbTMVP. However, there may be an embodiment in which the sub-block merge mode is only available for blocks 8x8 or larger. In this case, if the current region is adjacent to the left boundary of the picture 1902 and the current block 1901 is 8x8, it can be determined and judged that SbTMVP is not to be used.

[0245] In another embodiment of the present invention, the availability flag indicating the conditions for using SbTMVP may include, in addition to the above-mentioned conditions, whether xCb is 0 and the width and height of the current block are 8. That is, if xCb is 0 and the width and height of the current block are 8, the availability flag may be set to 0. If both the above-mentioned conditions and the above-mentioned conditions are met, or if one or more of them are met, the encoder / decoder can set the availability flag to 0. As described above, the availability flag may indicate whether SbTMVP is available. Furthermore, if certain conditions are met, it may be indicated that SbTMVP is unavailable. That is, if certain conditions are met, the encoder / decoder can set availableFlagSbCol to 0. Therefore, according to the embodiment described in FIG. 19, the encoder / decoder can set availableFlagSbCol based on the position of the current region. That is, the encoder / decoder can set availableFlagSbCol based on the coordinates (xCb, yCb) representing the current region. For example, (xCb, yCb) may be the top left coordinate of the current region.

[0246] As described in Figure 19, if the current region or current block abuts the left boundary of the picture, SbTMVP may be unavailable. Therefore, if xCb is 0, the encoder / decoder can set availableFlagSbCol to 0. For example, if the current region or current block abuts the left boundary of the picture and the current block is 8x8, SbTMVP may be unavailable. Therefore, if xCb is 0 and the width and height of the current block are both 8, the encoder / decoder can set availableFlagSbCol to 0.

[0247] Table 15 below illustrates a sub-block merge mode syntax structure according to one embodiment of the present invention.

[0248] [Table 15]

[0249] As described above, the sub-block merge mode (or sub-block-based merge mode) may include SbTMVP and / or affine merge mode. Furthermore, a syntax element (or flag) sps_affine_enabled_flag indicating whether the affine mode is enabled or disabled may be signaled from the encoder to the decoder using higher-level syntax. In one embodiment, the higher level may be at least one of the sequence, picture, slice, or CTU level. In one embodiment, conditions under which SbTMVP cannot be used (or applied) may be predefined, as described above in FIG. 19.

[0250] According to one embodiment of the present invention, if the SbTMVP unusable condition is met and affine mode is not enabled, the encoder / decoder may not parse merge_subblock_flag. For example, merge_subblock_flag may be signaling indicating the use of subblock merging mode. If merge_subblock_flag is not present, the encoder / decoder may infer its value to be 0. If affine mode is not enabled, this may be the case when sps_affine_enabled_flag is 0. Therefore, merge_subblock_flag can be parsed when SbTMVP is enabled or when affine mode is enabled. Furthermore, the conditions for parsing merge_subblock_flag may include additional conditions in addition to the above-mentioned conditions. For example, referring to Table 15, to determine the parsing of merge_subblock_flag, conditions related to MaxNumSubblockMergeCand, which indicates the maximum number of subblock merging candidates, and conditions related to the block size may both be considered.

[0251] Referring to Table 15, if x0, which is the x value of the top left coordinate of the current block, is 0 and sps_affine_enabled_flag, which indicates whether affine mode is enabled, is 0, merge_subblock_flag does not need to be parsed. Also, if x0 is not 0 or sps_affine_enabled_flag is 1, merge_subblock_flag can be parsed.

[0252] Furthermore, in the above-described embodiment, it is assumed that the subblock merge mode includes SbTMVP and affine merge mode. However, the present invention is not limited to this, and the subblock merge mode may include other modes. For example, the subblock merge mode may include a first mode and a second mode in addition to SbTMVP. Furthermore, syntax elements sps_mode1_enabled_flag and sps_mode2_enabled_flag may be signaled, indicating whether the first mode and the second mode are enabled, respectively. In this case, if the SbTMVP unusable condition is met but neither the first mode nor the second mode is enabled, the encoder / decoder may not parse merge_subblock_flag. In this case, the encoder / decoder may infer that merge_subblock_flag is 0. As an example, if the SbTMVP unusable condition is not met, or if the first mode is enabled or the second mode is enabled, the encoder / decoder may parse merge_subblock_flag. Furthermore, the conditions for parsing merge_subblock_flag may include other conditions in addition to the above conditions. Referring to Table 15, to determine the parsing of merge_subblock_flag, conditions related to MaxNumSubblockMergeCand, which indicates the maximum number of subblock merge candidates, and conditions related to the block size may both be considered.

[0253] Also, according to one embodiment of the present invention, if affine mode is not enabled, the encoder / decoder can set MaxNumSubblockMergeCand to 1. Alternatively, if affine mode is not enabled and SbTMVP is enabled, the encoder / decoder can set MaxNumSubblockMergeCand to 1. Alternatively, if affine mode is not enabled and SbTMVP is not enabled, the encoder / decoder can set MaxNumSubblockMergeCand to 0.

[0254] 20 is a diagram illustrating an SbTMVP deriving method according to an embodiment of the present invention. According to an embodiment of the present invention, a pre-defined position referenced to determine a temporary motion vector may be variable. Alternatively, a pre-defined position referenced to determine a temporary motion vector for a specific block may be set to one, and the position may be variable.

[0255] In one embodiment, a reference position for determining a temporary motion vector may be variably determined based on the position of the current block 2001 or the current region. The current region refers to the region described in FIG. 19 above. For example, if the current region borders a boundary of the picture 2002 or a boundary of a parallel-processable unit, motion information at a specific position may always be unavailable. Therefore, the encoder / decoder may set a position other than a position that is always unavailable as a pre-defined position to be referenced for determining a temporary motion vector. That is, the pre-defined position to be referenced for determining a temporary motion vector may be set to a position that is not always unavailable based on the position of the current region. The position that is always unavailable may be a position outside the picture 2002 or a range that can be processed in parallel.

[0256] According to the embodiment described in FIG. 18 above, the previously set position referenced to determine the temporary motion vector may be defined as the left position of the current block 2001 or current region. In this case, as described in FIG. 19 above, when the current block 2001 or current region is located in a specific position, motion information for the previously set position is unavailable, so the previously set position can be set (or determined) to a position other than the left position. For example, the previously set position may be set to the upper position 2003 of the current block 2001 or current region. For example, the previously set position may be position B1 in FIG. 18(a).

[0257] 20, in one embodiment, when a current block 2001 or a current region abuts the left boundary of a picture 2002 or the left boundary of a parallel processable unit, a previously set position referenced to determine a temporary motion vector may be determined to be the upper position of the current block 2001 or the current region. The upper position of the current region may refer to a position whose y coordinate is relatively smaller than the y coordinate (i.e., vertical coordinate) of the current region. For example, when the current block 2001 or the current region abuts the left boundary of a picture 2002 or the left boundary of a parallel processable unit, a previously set position referenced to determine a temporary motion vector may be position B1 in FIG. 18(a).

[0258] Alternatively, in one embodiment, if the current block 2001 or the current region is adjacent to the top boundary of the picture 2002 or the top boundary of the parallel processable unit, the previously set position referenced to determine the temporary motion vector may be determined to be the left position of the current block 2001 or the current region. The left position of the current region may refer to a position whose x coordinate is relatively smaller than the x coordinate (i.e., horizontal coordinate) of the current region. For example, if the current block 2001 or the current region is adjacent to the top boundary of the picture 2002 or the top boundary of the parallel processable unit, the previously set position referenced to determine the temporary motion vector may be position A1 in FIG. 18(a).

[0259] The sub-block based motion compensation method has been described above. In the following, we will describe a motion compensation method based on merge mode with motion vector difference (MMVD) (or merge MVD) and a method for combining and applying the above-mentioned sub-block based motion compensation embodiment.

[0260] FIG. 21 is a diagram illustrating a method for applying MMVD according to an embodiment of the present invention. According to an embodiment of the present invention, an encoder / decoder may determine a motion vector (MV) based on a motion vector predictor (MVP) and a motion vector difference (MVD). In this specification, the MVP may be referred to as a base motion vector (baseMV). That is, the encoder / decoder may derive a motion vector (i.e., a final motion vector) by adding the motion vector difference to the base motion vector. However, the present invention is not limited to these names, and the MVP may also be referred to as a base motion vector, a provisional motion vector, an initial motion vector, an MMVD candidate motion vector, etc. The MVD may be expressed as a value that refines the MVP and may be referred to as an improved motion vector (refineMV) or a merged motion vector difference.

[0261] According to an embodiment of the present invention, when MMVD is applied, i.e., in the MMVD mode, the motion vector may be determined based on a base motion vector, a distance parameter (or variable), and a direction parameter (or variable). According to another embodiment of the present invention, the base motion vector may be determined from a candidate list. For example, the base motion vector may be determined from a merge candidate list. The encoder / decoder may also determine the base motion vector from a portion of another candidate list. The portion of the candidate list may be a front portion (smaller index) of the candidate list. For example, the encoder / decoder may determine the base motion vector using the first and second candidates in the merge candidate list. To this end, a candidate index indicating a specific candidate from the two candidates may be signaled from the encoder to the decoder. Referring to FIG. 21, a base candidate index may be defined as an index for signaling the base motion vector. The encoder / decoder may determine a candidate to be applied to the current block from among the candidates in the candidate list based on the base candidate index, and determine the motion vector of the determined candidate as the base motion vector. In the present invention, the base candidate index is not limited to its name, and may be called a base candidate flag, a candidate index, a candidate flag, an MMVD index, an MMVD candidate index, an MMVD candidate flag, etc.

[0262] Furthermore, according to an embodiment of the present invention, an MVD different from the MVD described in FIGS. 6 and 7 may exist. For example, the MVD in the MMVD may be defined differently from the MVD described in FIGS. 6 and 7. In this specification, the MMVD may represent a merge mode (i.e., a motion compensation mode or method) using a motion vector difference, or may represent a motion vector difference when the MMVD is applied. For example, the encoder / decoder may determine whether or not to apply (or use) the MMVD. If the MMVD is applied, the encoder / decoder may determine a merge candidate to be used for inter-prediction of the current block from the merge candidate list, and determine the motion vector of the current block by deriving the MMVD and applying (or adding) it to the motion vector of the merge candidate.

[0263] In one embodiment, the other MVD may refer to a simplified MVD, an MVD with a different (or smaller) resolution, a MVD with a smaller number of available MVDs, an MVD with a different signaling method, etc. For example, the MVD used in the existing AMVP, affine inter mode, etc. described in FIGS. 6 and 7 can represent all regions in the x and y axes (i.e., horizontal and vertical directions) for a specific signaling unit (e.g., x-pel), e.g., a picture-based region (e.g., a picture region or a region including a picture and its surrounding regions) at uniform intervals, whereas the MMVD may have relatively limited units for representing a specific signaling unit. Also, the regions (or units) signaling the MMVD may not have uniform intervals. Also, the MMVD can indicate only a specific direction for a specific signaling unit.

[0264] Furthermore, according to one embodiment of the present invention, an MMVD may be determined based on distance and direction. Referring to FIG. 21, the distance and direction of the MMVD may be pre-set using a distance index indicating the distance of the MMVD and a direction index indicating the direction of the MMVD. In one embodiment, the distance may indicate the MMVD size (e.g., absolute value) in specific pixel units, and the direction may indicate the direction of the MMVD. Furthermore, the encoder / decoder may signal a relatively small distance with a relatively small index. That is, in the case of signaling that does not use fixed-length binarization, the encoder / decoder may signal a relatively small distance with relatively few bits.

[0265] Furthermore, according to an embodiment of the present invention, the MVD may use a signaled MV or an MV based on the signaled MV. For example, the MV based on the signaled MV may be the signaled MV with its sign reversed. For example, if MVD signaling is based on a value corresponding to a reference list, a value corresponding to a reference list different from the reference list may use the value corresponding to the reference list (i.e., the signaled MVD) as is or with its sign reversed for that value. Whether to use the signaled MVD as is or with its sign reversed may be determined based on a picture order count (POC) relationship between the current picture and a reference picture in a reference list, and a POC relationship between the current picture and a reference picture in a reference list different from the reference list. More specifically, even when both reference lists L0 and L1 are used, only one MVD may be signaled. For example, an MVD corresponding to L0 may be signaled. Then, an MVD corresponding to L1 may be determined based on the MVD corresponding to L0.

[0266] For example, the MVD corresponding to L1 may remain the same as the MVD corresponding to L0, or may be a value obtained by changing the sign of the MVD corresponding to L0. This may be determined based on the POC relationship between the current picture and the L0 reference picture and the POC relationship between the current picture and the L1 reference picture. For example, whether the MVD corresponding to L0 is used for L1 as is or after modification can be determined based on whether the value of DiffPicOrderCnt(RefPicList0[refIdxLN0], currPic) * DiffPicOrderCnt(currPic, RefPicList1[refIdxLN1]) is greater than or less than 0. Also, DiffPicOrderCnt(RefPicList0[refIdxLN0],currPic)*DiffPicOrderCnt(currPic,RefPicList1[refIdxLN1]) being greater than 0 may mean that both the L0 reference picture and the L1 reference picture are temporally earlier than the current picture or both are temporally later than the current picture.

[0267] Therefore, in this case, the signs of L0MVD and L1MVD may be the same. Also, DiffPicOrderCnt(RefPicList0[refIdxLN0], currPic) * DiffPicOrderCnt(currPic, RefPicList1[refIdxLN1]) being less than 0 may mean that one of the L0 reference picture and the L1 reference picture is temporally earlier than the current picture, and the other is temporally later than the current picture. Therefore, in this case, the signs of L0MVD and L1MVD may be different. Being temporally earlier may mean that the POC is smaller, and being temporally later may mean that the POC is larger. Furthermore, an MV scaling process may be added to the above-described embodiment. That is, a process of MV scaling a signaled MV or an MV obtained by modifying the signaled MV (e.g., an MV with the opposite sign) may be added.

[0268] Table 16 below illustrates an MMVD syntax structure according to one embodiment of the present invention.

[0269] [Table 16]

[0270] According to one embodiment of the present invention, there may be signaling indicating whether or not the MMVD is used (or whether or not it is applied), as described in FIG. 21 above. Referring to Table 16, the MMVD flag (i.e., mmvd_flag) represents a syntax element indicating whether or not the MMVD is used. As an example, the MMVD may be applied to skip mode or merge mode. In one embodiment, when the MMVD is used, the decoder can parse MMVD-related syntax. When the MMVD is used, the MMVD-related information may be further parsed from the MMVD index coding (i.e., mmvd_idx_coding) syntax.

[0271] In one embodiment, if the MMVD is not used, the decoder can parse the merge index. Referring to Table 16, the merge index (i.e., merge_idx) represents a syntax element indicating a specific merge candidate in the merge candidate list. In addition, in the case of a merge mode other than skip mode, after parsing the merge flag (i.e., merge_flag), if the merge flag is 1, the decoder can parse the MMVD flag. The merge flag may indicate whether or not the merge mode or the sub-block merge mode is used (or whether or not it is applied). The merge flag may also indicate that AMVP, inter mode, affine inter mode, etc. are not used. In this embodiment, the merge index may be an index different from the base candidate index described above in FIG. 21.

[0272] According to an embodiment of the present invention, signaling indicating the maximum number of merge candidates may be present. The signaling indicating the maximum number of merge candidates may be performed in units larger than a CU or a prediction unit. For example, signaling indicating the maximum number of merge candidates may be performed in slice or tile units. If a condition based on the signaling indicating the maximum number of merge candidates is satisfied, the decoder may parse the merge index. As an example, if a value of a syntax element indicating the maximum number of merge candidates indicates that the maximum number of merge candidates is 1, the encoder / decoder may infer that the merge index is 0. That is, if the signaling indicating the maximum number of merge candidates indicates that the maximum number of merge candidates is 1, candidates may be determined without parsing the merge index.

[0273] Also, referring to Table 16, MaxNumMergeCand, which is a signaling-based value (variable, parameter) indicating the maximum number of merging candidates, may be defined. MaxNumMergeCand may mean the maximum number of merging candidates. MaxNumMergeCand may have a value greater than or equal to 1. Furthermore, the maximum number of merging candidates for a signaling unit indicating the maximum number of merging candidates may be determined based on the maximum number of supportable merging candidates and signaling indicating the maximum number of merging candidates. For example, the encoder / decoder may determine the maximum number of merging candidates for a signaling unit indicating the maximum number of merging candidates by subtracting the signaling value indicating the maximum number of merging candidates from the maximum number of supportable merging candidates. Furthermore, although the maximum number of merging candidates has been described, the maximum number of sub-block merging candidates and maximum number signaling may exist in a substantially identical manner.

[0274] Also, in one embodiment, the merge index may be parsed when using skip mode or merge mode. The decoder can parse the merge index when the conditions for using skip mode or merge mode are met. For example, if the subblock merge mode is not used, the encoder / decoder can use skip mode or merge mode. In Table 16 above, the merge index may be parsed after MMVD flag parsing. This may be to consider MMVD preferentially over the existing skip mode or merge mode. Also, a subblock merge flag (i.e., subblock_merge_flag) (or merge subblock flag (i.e., merge_subblock_flag)) syntax element may be signaled to indicate whether or not to use the subblock merge mode.

[0275] FIG. 22 is a diagram illustrating an MMVD-related syntax structure according to an embodiment of the present invention. The embodiment described in Table 16 above may also be applied to FIG. 22, and related duplicated descriptions will be omitted. The syntax shown in FIG. 22 further includes a process related to sub-block merging in addition to Table 16. In FIG. 22 and the drawings described below (i.e., FIGS. 23 to 31), portions related to the intra prediction process and inter prediction, inter mode, AMVP mode, affine inter mode, etc. may be omitted. For convenience of explanation, FIG. 22 will be described focusing on a decoder, but the coding process according to this embodiment may also be applied to an encoder in substantially the same manner.

[0276] According to one embodiment of the present invention, a decoder may parse the sub-block merge flag before the MMVD flag and / or merge index. This is for the purpose of considering the sub-block merge mode preferentially over at least one of the skip mode, the merge mode, and / or the MMVD. The decoder may parse the MMVD flag and / or the merge index when the sub-block merge mode is not used. For example, the decoder may parse the sub-block merge flag before the MMVD flag, and may parse the MMVD flag when the sub-block merge mode is not applied. The sub-block merge mode performs motion compensation in units of relatively small sub-blocks, which may increase the accuracy of motion prediction. However, due to the characteristics of sub-block-based motion compensation, improving the MV using a single MVD may be ineffective. Therefore, according to an embodiment of the present invention, compression efficiency can be improved by checking whether the MMVD is applied only when the sub-block merge mode is not applied. This can be achieved by setting the parsing order of syntax (or syntax elements) (i.e., by parsing the sub-block merge flag before the MMVD flag).

[0277] FIG. 23 illustrates an example of an MMVD-related syntax structure according to an embodiment of the present invention. The syntax (or function) illustrated in FIG. 23 may represent syntax for parsing MMVD information shown in Table 16, FIG. 22, and the MMVD-related syntax described below. As described in FIG. 21, the MMVD may be determined based on a base motion vector, distance, and direction. Indices (or syntax elements) for signaling such information may be defined. Referring to FIG. 23, syntax elements for signaling the base motion vector, distance, and direction may be defined as base_mv_idx, distance_idx, and direction_idx. As described in Table 16 and FIG. 22, if it is determined that the MMVD is to be used, the syntax elements of FIG. 23 indicating specific MMVD values ​​may be signaled from the encoder to the decoder. Although FIG. 23 focuses on the decoder for convenience of explanation, the coding process according to this embodiment may also be applied to the encoder in substantially the same manner.

[0278] FIG. 24 is a diagram illustrating an example of an MMVD-related syntax structure according to an embodiment of the present invention. The syntax (or function) illustrated in FIG. 24 may represent syntax for parsing MMVD information shown in Table 16, FIG. 22, and the MMVD-related syntax described below. According to an embodiment of the present invention, some of the MMVD-related syntax may be parsed if a predefined specific condition is met. For example, after determining to use the MMVD, a decoder may parse some of the MMVD-related syntax if the predefined specific condition is met. In one embodiment, some of the MMVD-related syntax may be base motion vector-related signaling. The determination to use the MMVD may be based on an MMVD flag value. The determination to use the MMVD may be made by parsing the MMVD flag or when the MMVD flag is inferred to indicate a specific value (e.g., 1). Although FIG. 24 focuses on a decoder for convenience of explanation, the coding process according to this embodiment may be applied to an encoder in substantially the same manner.

[0279] According to another embodiment of the present invention, the specific condition may relate to the maximum number of possible base motion vector candidates. For example, the decoder may parse the base motion vector-related syntax if the maximum number of possible base motion vector candidates is two or more, and may not parse the base motion vector-related syntax if the maximum number of possible base motion vector candidates is one. When the base motion vector-related syntax does not exist, the decoder can infer its value. In this case, the base motion vector-related syntax may be inferred as 0. If the base motion vector candidate is a merge candidate, the decoder may determine whether to parse the base motion vector-related syntax according to a condition based on the maximum number of possible merge candidates.

[0280] As mentioned above, the maximum number of possible base motion vector candidates may be a value indicated by the signaling indicating the maximum number of merge candidates described in Table 16, and this value may be expressed as MaxNumMergeCand.

[0281] Referring to Figure 24, the decoder can parse the base motion vector index only if MaxNumMergeCand, which represents the maximum number of merge candidates, is greater than 1. If the base motion vector index does not exist, for example, because it has not been parsed, the decoder can infer its value as 0. If the maximum number of possible candidates is 1, the index can be determined without signaling it.

[0282] FIG. 25 is a diagram illustrating an MMVD syntax according to an embodiment of the present invention. In the syntax structure described in FIG. 22, the MMVD flag is always parsed even when the MMVD is not used and the skip mode or merge mode is used. In addition, the base motion vector index or merge index is parsed both when the MMVD is used and when it is not used. Meanwhile, in the embodiment of FIG. 25, the MMVD flag may not be parsed even when the skip mode or merge mode is used and the MMVD is not used. Although FIG. 25 mainly describes a decoder for convenience of explanation, the coding process according to this embodiment may be applied to an encoder in substantially the same manner.

[0283] According to one embodiment of the present invention, the base motion vector-related syntax and the merge index may be defined as the same value. For example, the base motion vector index and the merge index may be the same. That is, signaling for indicating a corresponding candidate may be performed using one syntax element. Also, according to one embodiment of the present invention, a decoder may parse the base motion vector-related syntax and parse the MMVD flag conditionally. Then, if the MMVD flag indicates the use of MMVD, the decoder may parse MMVD-related syntax other than the base motion vector-related syntax.

[0284] For example, after parsing a merge index, it is possible to distinguish whether the corresponding candidate is a candidate usable for MMVD. If the candidate is usable for MMVD, the MMVD flag is parsed. If the candidate is not usable for MMVD, the MMVD flag does not need to be parsed. Also, if the MMVD flag is not present, it can be inferred that the MMVD is not used. For example, when the base motion vector of the MMVD can match up to num_mmvd_baseCand candidates at the front (smaller index) of the merge candidate list, if the parsed index is smaller than num_mmvd_baseCand, the MMVD flag is parsed. Otherwise, it can be determined that the MMVD is not used without parsing the MMVD flag. In this case, when using candidate indexes equal to or greater than num_mmvd_baseCand, it is advantageous to save bits for the MMVD flag.

[0285] 25, after parsing the merge index, if the merge index is smaller than num_mmvd_baseCand, the MMVD flag is parsed. num_mmvd_baseCand may be the number of possible candidates for the base motion vector of the MMVD. Furthermore, the base motion vector index of the MMVD may be determined based on the merge index.

[0286] FIG. 26 is a diagram illustrating an MMVD syntax according to an embodiment of the present invention. Referring to FIG. 26, in the syntax structure, the MMVD flag may be higher than the merge flag. For example, a decoder may determine whether to parse the merge flag based on the MMVD flag. For example, this is applicable when not in skip mode. Although FIG. 26 focuses on a decoder for convenience of explanation, the coding process according to this embodiment may also be applied to an encoder in substantially the same manner.

[0287] According to one embodiment of the present invention, if the MMVD flag indicates that the MMVD is to be used, the decoder does not need to parse the merge flag. For example, if a base motion vector candidate for the MMVD is determined from among merge candidates and the MMVD flag indicates that the MMVD is to be used, the decoder does not need to parse the merge flag. If the MMVD is to be used (the MMVD flag is 1), the decoder can infer that the merge mode is to be used from the merge flag. Referring to FIG. 26, if skip mode is to be used, i.e., if cu_skip_flag is 1, the decoder can infer that the merge flag is 1 regardless of whether the MMVD is used. Also, if the merge flag is not present and skip mode is not to be used, the decoder can infer that the merge flag is 1 if the MMVD is to be used, and can infer that the merge flag is 0 if the MMVD is not to be used. Alternatively, in one embodiment, the decoder can infer that the merge flag is 1 if the merge flag is not present and skip mode is not to be used. Also, referring to FIG. 26, when not in skip mode, the decoder parses the MMVD-related syntax if the MMVD flag is 1, and parses the merge flag if the MMVD flag is not 0.

[0288] According to one embodiment of the present invention, when the sub-block merge flag is higher than the merge flag, if the sub-block merge flag is 1, the merge flag does not need to be parsed and the merge flag can be inferred to be 1. Also, when the sub-block merge flag is higher than the MMVD flag, if the sub-block merge flag is 1, the MMVD flag does not need to be parsed and the MMVD flag can be inferred to be 0. Also, in this specification, modeX_flag being 1 means that modeX is used, and modeX_flag being 0 means that modeX is not used.

[0289] FIG. 27 is a diagram illustrating an example of MMVD syntax according to an embodiment of the present invention. According to an embodiment of the present invention, in a candidate list used as a base motion vector candidate for the MMVD, the MMVD can always be used for some candidates. For example, when a base motion vector candidate for the MMVD is determined from a merge candidate list, the MMVD can always be used for some of the merge candidate list. For example, the MMVD can always be used for a determined candidate index. For example, when a candidate index is smaller than a pre-set value, the MMVD can always be used. In this case, whether or not the MMVD is to be used can be determined from the candidate index. Also, MMVD flag parsing may not be required.

[0290] For example, after parsing a merge index, if the merge index corresponds to a value that indicates the use of the MMVD, it can be determined that the MMVD is to be used. In this case, MMVD-related syntax (e.g., distance, direction signaling, etc.) can be parsed. Although FIG. 27 focuses on a decoder for convenience of explanation, the coding process according to this embodiment can be applied to an encoder in substantially the same manner. Referring to FIG. 27, the MMVD can always be used up to the num_mmvd_baseCand candidates at the front of the merge candidate list. In this case, if the candidate index is smaller than num_mmvd_baseCand, it can be determined that the MMVD is to be used, and the MMVD-related syntax can be parsed.

[0291] FIG. 28 illustrates an example of MMVD syntax according to an embodiment of the present invention. According to an embodiment of the present invention, a decoder can always use the MMVD for some candidates in a candidate list used as base motion vector candidates for the MMVD. For example, when a base motion vector candidate for the MMVD is determined from a merge candidate list, the decoder can always use the MMVD for some of the merge candidate list. For example, the decoder can always use the MMVD for a determined candidate index. For example, when a candidate index is smaller than a pre-set value, the decoder can always use the MMVD. Although FIG. 27 focuses on a decoder for convenience of explanation, the coding process according to this embodiment may also be applied to an encoder in substantially the same manner.

[0292] According to one embodiment of the present invention, the MMVD flag may be higher than the merge flag in the syntax structure (or syntax parsing order). In this case, the decoder can parse the MMVD-related syntax if the MMVD flag is 1. Also, if the MMVD flag is 1, the decoder can infer that the merge flag is 1. Also, if the MMVD flag is 0, the decoder can parse the merge flag. If the merge flag is 1, the decoder can parse the merge index. In this case, additional conditions may exist for parsing the merge index. In this case, as one embodiment, the decoder can determine the merge index to be actually used by modifying it based on the parsed merge index. For example, the merge index to be actually used may be determined by adding the parsed merge index to the number of candidates that are smaller than the parsed merge index and are determined to always use the MMVD. For example, if it is specified that MMVD should always be used for the first num_mmvd_baseCand of the candidate list, the value obtained by adding num_mmvd_baseCand to the parsed merge index can be used as the merge index. If the MMVD flag is before the merge flag, candidates that use MMVD can be excluded from the candidate list if the MMVD flag is 0.

[0293] Referring to FIG. 27, the MMVD flag precedes the merge flag. If the MMVD flag is 1, the decoder can parse the MMVD-related syntax. If the MMVD flag is 0, the decoder can parse the merge flag. If the MMVD flag is 0 and the merge flag is 1, the decoder can parse the merge index. At this time, there may be additional conditions for parsing the merge index. The number of candidates that can be used as the base motion vector of the MMVD, num_mmvd_baseCand, can be added to the parsed merge index to determine the merge index to be actually used.

[0294] FIG. 28 is a diagram illustrating the MMVD syntax according to an embodiment of the present invention. According to an embodiment of the present invention, a candidate index may exist before a flag indicating the presence or absence of mode use. For example, the candidate index may exist before the MMVD flag or the merge flag. In such a case, after parsing the candidate index, it is possible to determine whether to parse the MMVD flag depending on whether the parsed index is a candidate that can use MMVD. For example, if the parsed candidate index is a candidate that can use MMVD, parse the MMVD flag, and if it is a candidate that cannot use MMVD, do not parse the MMVD flag and infer it as 0. Also, when the MMVD flag is 0, the merge flag can be parsed. The merge flag can determine whether the merge mode or the sub-block merge mode is used, or whether the inter mode or the AMVP mode or the affine inter mode is used. In FIG. 28, the decoder is mainly described for convenience of explanation, but the coding process according to this embodiment may be applied to the encoder in substantially the same manner. [[ID=*]] [[ID=*]]

[0295] [[ID=*]] For example, if the number of candidates possible as the first mode is num1, the number of candidates possible as the second mode is num2, and num1 < num2, when parsing the candidate index, if the parsed index is greater than or equal to num1, it is not necessary to parse the syntax for determining from the first mode and the second mode (for example, the syntax indicating whether to use the first mode), and it can be inferred that the first mode is not used. And when the parsed index is smaller than num1, the syntax for determining from the first mode and the second mode can be parsed.

[0296] 28, after parsing the merge index, if the merge index is smaller than num_mmvd_baseCand, which is the number of candidates for which the MMVD can be used, the MMVD flag can be parsed. Also, if the merge index is equal to or greater than num_mmvd_baseCand, the MMVD flag does not need to be parsed, and in this case, the MMVD flag can be inferred to be 0. Also, if the MMVD flag is 1, the distance index (distance_idx) and direction index (direction_idx), which represent MMVD-related syntax elements, can be parsed. Also, if the MMVD flag is 0, the merge flag is parsed, and it can be determined whether the mode is merge mode, sub-block merge mode, inter mode, AMVP mode, or affine mode.

[0297] FIG. 29 illustrates a coding unit syntax structure according to an embodiment of the present invention. As shown in FIG. 29, when the merge flag is 1, a merge data (i.e., merge_data) parsing process can be performed. For convenience of explanation, FIG. 29 mainly describes a decoder, but the coding process according to this embodiment may be applied to an encoder in substantially the same manner. In one embodiment, the merge data may include a portion of merge-related syntax. Furthermore, the merge data may be referred to as merge data syntax. For example, the merge data may include the syntax parsing process performed when the merge flag is 1 in the above-described FIGS. 21 to 28. Furthermore, the merge flag being 1 may indicate the use of a merge mode. Furthermore, the merge flag being 1 may indicate the use of inter prediction without using mvd_coding as shown in FIG. 6 and Table 1 above.

[0298] Figure 30 illustrates a merge data syntax structure according to an embodiment of the present invention. As described in Figure 29, if the merge flag is 1, a decoder can parse (or call) the merge data syntax. For convenience of explanation, Figure 30 focuses on a decoder, but the coding process according to this embodiment may be applied to an encoder in substantially the same manner. Also, as described above, if merge mode or skip mode is used, the merge flag may be set to 1.

[0299] Referring to FIG. 30, in the merge data syntax, the MMVD flag may be parsed first among the syntax elements. The MMVD flag may be syntax that can be parsed first after it is confirmed that the merge flag is 1. Furthermore, the MMVD flag may be parsed before signaling indicating whether or not another prediction mode that sets the merge flag to 1 is used. Furthermore, the MMVD flag may be signaling indicating whether or not MMVD is used. Furthermore, the same method as in the embodiments described with reference to Table 16 and FIGS. 22 to 29 may be applied. As shown in FIG. 30, the merge data syntax may include decoding processes for a sub-block merge mode, multi-hypothesis prediction (or intra and inter combined prediction), triangle prediction, etc.

[0300] According to an embodiment of the present invention, multi-hypothesis prediction may be a method of generating and combining two or more prediction blocks when generating a prediction block. Alternatively, multi-hypothesis prediction may be a method of using both inter-prediction and intra-prediction when generating a prediction block. Furthermore, inter-prediction and intra-prediction may be methods of using the same picture as the picture containing the current block or a different picture when making a prediction. Referring to Figure 30, a multi-hypothesis prediction flag represents a syntax element indicating whether to use multi-hypothesis prediction.

[0301] According to one embodiment of the present invention, the sub-block merging mode may be a method of performing motion compensation on a sub-block basis (i.e., determining a motion vector on a sub-block basis) when predicting a current block (e.g., a coding unit or a prediction unit). In one embodiment, the sub-block merging mode may include methods such as sub-block-based temporal MVP and affine motion prediction. Referring to Figure 30, the merge sub-block flag represents a syntax element indicating whether to use the sub-block merging mode.

[0302] According to an embodiment of the present invention, triangle prediction represents a method of performing motion compensation on a non-rectangular region within a current block. That is, in triangle prediction, a unit having the same motion vector within the current block may not be a rectangle. Referring to Figure 30, a triangle prediction flag (i.e., merge_triangle_flag) represents a syntax element indicating whether triangle prediction is used.

[0303] According to an embodiment of the present invention, the MMVD flag may be parsed before signaling (or syntax elements) indicating the use of a merge mode other than MMVD. The signaling indicating the use of a merge mode other than MMVD may include a multiple hypothesis prediction flag, a merge sub-block flag, a merge triangle flag, etc.

[0304] 30, the MMVD merge index (i.e., mmvd_merge_idx) (or MMVD index) represents a syntax element that indicates which of the MMVD base motion vectors to use. If the MMVD is used, the merge index may be inferred as the MMVD merge flag (i.e., mmvd_merge_flag) (or MMVD flag).

[0305] FIG. 31 is a diagram illustrating a merge data syntax structure according to an embodiment of the present invention. The embodiment of FIG. 31 may be a portion of the merge data syntax performed when the merge flag is 1 in FIG. 29. For convenience of explanation, FIG. 31 focuses on a decoder, but the coding process according to this embodiment may be applied to an encoder in substantially the same manner. According to an embodiment of the present invention, the MMVD flag may be parsed after the merge index. For example, the MMVD flag may be parsed immediately after the merge index. For example, the MMVD flag may not be parsed at a position other than the very front in the merge data syntax. That is, after confirming that the merge flag is 1, the MMVD flag may be parsed after parsing other syntax instead of the MMVD flag.

[0306] Also, according to one embodiment of the present invention, as described in FIG. 21, the base motion vector of the MMVD may be determined from another candidate list, and in one embodiment, the base motion vector may be determined from a portion of the other candidate list. Therefore, according to this embodiment, as described in FIG. 25, it is possible to determine whether to parse the MMVD flag based on an index associated with the other candidate list that can be used as the base motion vector of the MMVD. For example, an index associated with the other candidate list that can be used as the base motion vector of the MMVD may be parsed before the MMVD flag. Then, if the index associated with the other candidate list indicates that it can be used as the base motion vector of the MMVD, the MMVD flag may be parsed, and if not, the MMVD flag may not be parsed.

[0307] Referring to FIG. 31, the base motion vector of the MMVD may be determined from a merge candidate. Therefore, in one embodiment, if the merge index indicates that the vector can be used as the base motion vector of the MMVD, the MMVD flag may be parsed; otherwise, the MMVD flag may not be parsed. Alternatively, if the base motion vector of the MMVD may be selected from the first n vectors in the merge candidate list, the MMVD flag may be parsed if the merge index is smaller than n (the merge index may start from 0), and the MMVD flag may not be parsed if the merge index is not smaller than n. More specifically, the base motion vector of the MMVD may be the first or second candidate in the merge candidate list. Referring to FIG. 31, the MMVD flag may be parsed if the merge index is smaller than 2, i.e., 0 or 1. The merge index may represent a merge candidate index. Thus, if the merge index indicates that the MMVD cannot be used in the merge mode, the MMVD flag may not be parsed, thereby improving coding efficiency.

[0308] Furthermore, the base motion vector of the MMVD may be determined from a candidate list of a certain mode. Therefore, according to one embodiment of the present invention, when there are multiple signalings indicating whether or not various modes are used, the MMVD flag can be parsed after it is determined that a certain mode is to be used. For example, assuming that first, second, and third modes exist, if the MMVD is determined based on the third mode, or if the base motion vector of the MMVD is determined from a candidate for the third mode, the MMVD flag can be parsed after it is determined that the third mode is to be used. For example, the use of the third mode may be determined based on signaling indicating whether or not the third mode is to be used. Alternatively, for example, if it is determined that a mode other than the third mode, such as the first and second modes, is not to be used, the use of the third mode may be determined. For example, when sub-block merge mode, multiple hypothesis prediction, triangle prediction, conventional merge mode, etc. exist, and if the MMVD can be applied to conventional merge mode, the MMVD flag can be parsed after it is determined that the conventional merge mode or the MMVD is to be used among the listed modes. The conventional merge mode is a mode in which inter prediction is performed using motion information of neighboring intra-coded blocks, and may refer to the merge mode of a conventional video compression technology (e.g., high efficiency video coding (HEVC)).

[0309] Furthermore, when parsing the MMVD flag based on an index associated with another candidate list that can be used as the base motion vector of the MMVD, there is no need for separate signaling indicating the base motion vector of the MMVD. For example, in FIG. 23 or 24, signaling such as a base motion vector index exists separately from the merge index. Also, referring to FIG. 30, the MMVD merge flag may be signaling indicating the base motion vector of the MMVD. According to one embodiment of the present invention, as shown in FIG. 31, the MMVD merge flag may not be present. For example, the MMVD-related syntax may include only the MMVD flag, signaling indicating the MMVD distance (mmvd_distance_idx in FIG. 30 and FIG. 31), and signaling indicating the MMVD direction (mmvd_direction_idx in FIG. 30 and FIG. 31). Also, the base motion vector of the MMVD may be determined by the merge index. As a result, the signaling indicating the base motion vector of the MMVD in FIG. 30 and its context model may not be present in the embodiment of FIG.

[0310] Also, according to one embodiment of the present invention, the MMVD flag may be parsed after signaling indicating the use of a merge mode other than the above-mentioned MMVD. Referring to FIG. 31, the MMVD flag may be parsed after the merge sub-block flag, the multiple hypothesis prediction flag (i.e., mh_intra_flag), and the merge triangle flag (i.e., merge_triangle_flag). A specific merge mode may not be used with the MMVD. In this case, if the flag indicating whether or not to use the specific merge mode indicates no use, the MMVD flag may be parsed. Referring to FIG. 31, the MMVD may be disabled for use with triangle prediction, and the MMVD flag may be parsed if the merge triangle flag is 0. Alternatively, the MMVD may be disabled for use with MH intra, and the MMVD flag may be parsed if the mh_intra_flag is 0. Alternatively, the MMVD may be disabled for use with a sub-block merge mode, and the MMVD flag may be parsed if the merge sub-block flag is 0.

[0311] Figure 32 is a diagram illustrating a merge data syntax structure according to an embodiment of the present invention. The embodiment of Figure 32 may be part of the merge data syntax performed when the merge flag is 1 in Figure 29. For convenience of explanation, Figure 32 focuses on a decoder, but the coding process according to this embodiment may also be applied to an encoder in substantially the same way.

[0312] According to one embodiment of the present invention, the MMVD flag may be parsed after signaling indicating whether or not other modes are used, or the MMVD flag may be parsed after signaling indicating whether or not other modes other than MMVD are used in which the merge flag is set to 1. For example, the MMVD flag may be parsed after the merge sub-block flag, the multiple hypothesis prediction flag (i.e., mh_intra_flag), and the merge triangle flag (i.e., merge_triangle_flag).

[0313] In one embodiment, the MMVD flag may be parsed when a mode other than MMVD is not used. For example, the MMVD flag may be parsed when the merge sub-block flag is 0. The MMVD flag may be parsed when the multiple hypothesis prediction flag is 0. The MMVD flag may be parsed when the merge triangle flag is 0. A flag indicating whether a certain mode is used or not being 0 may indicate that the certain mode is not used. In addition, when MMVD can be applied to conventional merge mode, the MMVD flag may be parsed after it is determined that conventional merge mode or MMVD is to be used.

[0314] Also, in one embodiment, if an MMVD is used, the decoder can parse MMVD-related syntax, and if an MMVD is not used, the decoder can parse the merge index. Alternatively, if an MMVD is used, the decoder can parse the MMVD merge flag (i.e., mmvd_merge_flag), MMVD distance index (mmvd_distance_idx), and MMVD direction index (mmvd_direction_idx), and if an MMVD is not used, the decoder can parse the merge index. Also, as shown in FIG. 32, if the MMVD flag is parsed after the flag indicating whether or not another mode is used, signaling indicating the base motion vector of the MMVD may exist as a syntax element separate from the merge index. Referring to FIG. 32, the signaling indicating the base motion vector of the MMVD may be the MMVD merge flag. For example, the MMVD flag can be parsed after it is determined to use conventional merge mode or MMVD.

[0315] According to one embodiment of the present invention, a base motion vector of an MMVD is determined from a candidate list, and the maximum number of candidates in the candidate list may be variable. For example, the maximum number of candidates may be determined from a higher-level syntax. The higher-level syntax may be a syntax at a higher level than the current coding unit. For example, the higher-level syntax may be a syntax at a sequence, picture, slice, or tile level. According to one embodiment, in this case, the maximum number of candidates for the base motion vector of the MMVD may follow the maximum number of candidates in the candidate list that can be the base motion vector of the MMVD. Therefore, if the maximum number of candidates in the candidate list that can be the base motion vector of the MMVD decreases, the maximum number of candidates that can be the base motion vector of the MMVD may also decrease.

[0316] In one embodiment, the base motion vector of the MMVD is determined from the merge candidate list, and the maximum number of merge candidates may be MaxNumMergeCand. Alternatively, MaxNumMergeCand may be determined from higher-level syntax. In such a case, the maximum number of merge candidates that can become the base motion vector of the MMVD may be less than or equal to MaxNumMergeCand. Therefore, as shown in FIG. 32, MaxNumMergeCand may determine whether to parse signaling indicating the base motion vector of the MMVD. For example, if MaxNumMergeCand is 1, it is not necessary to parse signaling indicating the base motion vector of the MMVD. If there is no signaling indicating the base motion vector of the MMVD, it can be inferred to be 0.

[0317] In another embodiment, if MaxNumMergeCand is 1, MMVD may not be used. This may be to reduce signaling overhead for MMVD-related syntax. Therefore, if MaxNumMergeCand is 1, the MMVD flag may not be parsed. Also, if the MMVD flag is not present, it can be inferred as 0. Also, referring to FIG. 32, the merge triangle flag can be parsed only when multiple hypothesis intra prediction is not used. If the merge triangle flag is not present, it can be inferred as 0.

[0318] According to an embodiment of the present invention, there may be multiple modes that cannot be used together. For example, subblock merging mode, triangle prediction, multiple hypothesis intra prediction, and MMVD may be mutually incompatible. Furthermore, signaling indicating whether to use each of the multiple modes that cannot be used together may be parsed in a predetermined order. In this case, signaling indicating whether to use a certain mode among the multiple modes may be parsed only if all previously parsed signaling indicating whether to use another certain mode among the multiple modes indicates that the mode will not be used.

[0319] 33 is a diagram illustrating a video signal processing method according to an embodiment to which the present invention is applied. For convenience of explanation, the present invention will be described focusing on a decoder, but the present invention is not limited thereto, and the multiple hypothesis prediction-based video signal processing method according to this embodiment may be applied to an encoder in substantially the same manner.

[0320] The decoder obtains a first syntax element indicating whether a sub-block based merge mode is applied to a current block (S3301). As described above, in one embodiment, the first syntax element may be a sub-block merge flag (or a merge sub-block flag).

[0321] If the first syntax element indicates that the sub-block based merge mode is not applied to the current block, the decoder obtains a second syntax element indicating whether the merge mode using the motion vector difference is applied to the current block (S3302). As described above, in one embodiment, the second syntax element may be an MMVD flag (or an MMVD merge flag).

[0322] If the second syntax element indicates that the merge mode using the motion vector difference is to be applied to the current block, the decoder obtains a third syntax element indicating a candidate to be used for inter prediction of the current block among candidates included in the merge candidate list of the current block (S3303). As described above, in one embodiment, the third syntax element may be an MMVD index (or an MMVD candidate index or an MMVD candidate flag).

[0323] The decoder obtains information related to the motion vector difference (S3304). As described above, in one embodiment, obtaining the information related to the motion vector difference may include obtaining a fourth syntax element indicating the distance of the motion vector difference; and obtaining a fifth syntax element indicating the direction of the motion vector difference. As described above, in one embodiment, the fourth syntax element may be a distance index (or an MMVD distance index), and the fifth syntax element may be a direction index (or an MMVD direction index).

[0324] The decoder derives the motion vector of the current block by adding the motion vector differential to the motion vector of the candidate indicated by the third syntax element (S3305).

[0325] The decoder generates a prediction block for the current block using the motion vector of the current block (S3306).

[0326] As described above, as an example, the third syntax element may indicate a candidate to be used for inter prediction of the current block among the first and second candidates in the merge candidate list of the current block.

[0327] As mentioned above, as an example, the third syntax element may be parsed from the bitstream if the maximum number of merging candidates is greater than 1, and may be inferred to be 0 if the maximum number of merging candidates is not greater than 1.

[0328] As described above, in one embodiment, if the first syntax element indicates that a sub-block based merge mode is to be applied to the current block, the method may further include obtaining a sixth syntax element indicating a candidate to be used for inter prediction of the current block from among candidates included in a sub-block merge candidate list of the current block, and if the first syntax element indicates that a sub-block based merge mode is to be applied to the current block, a motion vector of the current block may be derived in units of at least one sub-block included in the current block based on the motion vector of the candidate indicated by the sixth syntax element. As described above, in one embodiment, the sixth syntax element may be a sub-block merge index (or a merge sub-block index).

[0329] As mentioned above, as an example, if the first syntax element indicates that a sub-block based merge mode is applied to the current block, the second syntax element may be inferred as 0.

[0330] As mentioned above, as an example, the sixth syntax element may be parsed from the bitstream if the maximum number of sub-block merging candidates is greater than 1, and may be inferred to be 0 if the maximum number of sub-block merging candidates is not greater than 1.

[0331] The above-described embodiments of the present invention may be implemented by various means, such as hardware, firmware, software, or a combination thereof.

[0332] In the case of a hardware implementation, the method according to an embodiment of the present invention may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), processors, controllers, microcontrollers, microprocessors, etc.

[0333] In the case of implementation by firmware or software, the methods according to the embodiments of the present invention may be implemented in the form of modules, procedures, or functions that perform the functions or operations described above. The software code may be stored in a memory and driven by a processor. The memory may be located inside or outside the processor, and data may be exchanged with the processor by various means known in the art.

[0334] Some embodiments may be embodied in the form of a recording medium containing computer-executable instructions, such as program modules, executed by a computer. Computer-readable media may be any available medium accessible by a computer, including volatile and nonvolatile media, detachable and non-detachable media. Computer-readable media may also include any computer storage media and communication media. Computer storage media includes any volatile and non-volatile, detachable and non-detachable media embodied in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. Communication media typically include computer-readable instructions, data structures, or other data in a modulated data signal, such as a program module, or other transmission mechanism, and includes any information delivery media.

[0335] The above description of the present invention is for illustrative purposes only, and those skilled in the art will understand that the present invention can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. Therefore, the above-described embodiments should be construed as illustrative in all respects and not restrictive. For example, each component described as a single component may be implemented in a distributed form, and similarly, each component described as a distributed component may be implemented in a combined form.

[0336] The scope of the present invention is indicated by the claims below rather than the above detailed description, and any modifications or variations derived from the meaning and scope of the claims and their equivalents should be construed as being within the scope of the present invention. [Explanation of symbols]

[0337] 100 Encoding Device 110 Conversion unit 115 Quantization section 120 Inverse quantization section 125 Inverse conversion unit 130 Filtering section 150 Prediction Department 160 Entropy Coding Unit

Claims

1. 1. A method for processing a video signal, comprising: obtaining a first syntax element indicating whether a sub-block based merge mode is applied to a current block; obtaining a second syntax element indicating whether a merge mode using a motion vector difference is to be applied to the current block, when the first syntax element indicates that a sub-block based merge mode is not to be applied to the current block; if the second syntax element indicates that a merge mode using the motion vector difference is to be applied to the current block, obtaining a third syntax element indicating a candidate to be used for inter prediction of the current block from among candidates included in a merge candidate list of the current block, obtaining a fourth syntax element indicating a distance of the motion vector difference, obtaining a fifth syntax element indicating a direction of the motion vector difference, and deriving a motion vector of the current block by adding the motion vector difference to a motion vector of the candidate indicated by the third syntax element, wherein the third syntax element indicates a candidate to be used for the inter prediction of the current block from among a first candidate and a second candidate in the merge candidate list of the current block; deriving the motion vector of the current block without parsing the second syntax element if the first syntax element indicates that the sub-block based merge mode is applied to the current block; and generating a prediction block for the current block using a motion vector of the current block; The video signal processing method, wherein the sub-block based merging mode uses a sub-block merging candidate list associated with a motion vector derived on a per-sub-block basis of at least one sub-block included in the current block.

2. 2. The video signal processing method of claim 1, wherein the third syntax element is parsed from the bitstream if the maximum number of merging candidates is greater than 1, and is inferred to be 0 if the maximum number of merging candidates is not greater than 1.

3. 2. The video signal processing method of claim 1, further comprising: obtaining a sixth syntax element indicating a candidate to be used for inter prediction of the current block among candidates included in a sub-block merge candidate list of the current block when the first syntax element indicates that a sub-block based merge mode is applied to the current block; and when the first syntax element indicates that a sub-block based merge mode is applied to the current block, the motion vector of the current block is derived in units of at least one sub-block included in the current block based on a motion vector of a candidate indicated by the sixth syntax element.

4. 4. The method of claim 3, wherein the second syntax element is inferred to be 0 if the first syntax element indicates that a sub-block based merge mode is applied to the current block.

5. 4. The video signal processing method of claim 3, wherein the sixth syntax element is parsed from the bitstream if the maximum number of sub-block merging candidates is greater than 1, and is inferred to be 0 if the maximum number of sub-block merging candidates is not greater than 1.

6. 1. A method for generating a video signal, comprising: encoding a first syntax element indicating whether a sub-block based merge mode is applied to a current block; encoding a second syntax element indicating whether a merge mode using a motion vector differential is to be applied to the current block, when the first syntax element indicates that a sub-block based merge mode is not to be applied to the current block; encoding a third syntax element indicating a candidate to be used for inter prediction of the current block among candidates included in a merge candidate list of the current block when the second syntax element indicates that a merge mode using the motion vector difference is applied to the current block, and encoding information related to the motion vector difference to derive a motion vector of the current block by adding the motion vector difference to a motion vector of the candidate indicated by the third syntax element, wherein the third syntax element indicates the candidate to be used for the inter prediction of the current block among a first candidate and a second candidate of merge candidates for the current block, and encoding the information related to the motion vector difference includes encoding a fourth syntax element indicating a distance of the motion vector difference and a fifth syntax element indicating a direction of the motion vector difference; deriving the motion vector of the current block without encoding the second syntax element if the first syntax element indicates that the sub-block based merge mode is applied to the current block; and generating a prediction block for the current block using a motion vector of the current block; The method for generating a video signal, wherein the sub-block based merging mode uses sub-block merging candidates associated with motion vectors derived on at least one sub-block basis included in the current block.

7. A method for transmitting a bitstream, the bitstream being decoded by a decoding method, The decoding method comprises: obtaining a first syntax element indicating whether a sub-block based merge mode is applied to a current block; obtaining a second syntax element indicating whether a merge mode using a motion vector difference is to be applied to the current block, when the first syntax element indicates that the sub-block based merge mode is not to be applied to the current block; if the second syntax element indicates that the merge mode using the motion vector differential is to be applied to the current block, obtaining a third syntax element indicating a candidate to be used for inter prediction of the current block from among candidates included in a merge candidate list of the current block, obtaining information related to the motion vector differential, obtaining a fourth syntax element indicating a distance of the motion vector differential, obtaining a fifth syntax element indicating a direction of the motion vector differential, and deriving a motion vector of the current block by adding the motion vector differential to a motion vector of the candidate indicated by the third syntax element, wherein the third syntax element indicates the candidate to be used for the inter prediction of the current block from a first candidate and a second candidate in the merge candidate list of the current block; deriving the motion vector of the current block without parsing the second syntax element if the first syntax element indicates that the sub-block based merge mode is applied to the current block; and generating a predicted block for the current block using the motion vector of the current block; The method for transmitting a bitstream, wherein the sub-block based merging mode uses a sub-block merging candidate list associated with a motion vector derived on a per-sub-block basis of at least one sub-block included in the current block.

Images