Video signal processing method and device using subblock-based motion compensation

The method enhances video signal processing by employing subblock-based motion compensation and motion vector differences to improve coding efficiency and compression, addressing inefficiencies in existing methods.

JP2024161597AActive Publication Date: 2024-11-19WILUS INSTITUTE OF STANDARDS & TECHNOLOGY INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2024145683
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2019-02-13
Filing Date
2024-08-27
Publication Date
2024-11-19
Estimated Expiration
2039-10-23

AI Technical Summary

Technical Problem

Existing video signal processing methods lack efficiency in coding and require improved techniques for subblock-based motion compensation and motion vector difference utilization.

Method used

A method and apparatus for video signal processing that employs subblock-based motion compensation using motion vector differences, including syntax elements to determine merging modes and motion vector information for inter prediction, thereby optimizing coding efficiency.

Benefits of technology

The approach increases coding efficiency and reduces signaling overhead by effectively utilizing subblock-based motion compensation and motion vector differences, enhancing compression efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024161597000001_ABST
    Figure 2024161597000001_ABST
Patent Text Reader

Abstract

To provide a video signal processing method and device for encoding and decoding a video signal, capable of enhancing a compression efficiency.SOLUTION: In the case where a first syntax element indicating whether or not a subblock-based merging mode is adopted to the current block indicates that the subblock-based merging mode is not adopted to the current block, a method acquires a second syntax element indicating whether or not a merging mode using a motion vector difference is adopted to a current block. In the case where the second syntax element indicates that the merging mode using the motion vector difference to the current block is adopted, the method: derives a motion vector for the current block by adding the motion vector difference to a candidate motion vector indicated by a third syntax element that indicates a candidate to be used in inter-prediction for the current block, among candidates contained in a merge candidate list for the current block; and generates a prediction block of the current block.SELECTED DRAWING: Figure 33
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a method and apparatus for processing a video signal, and more particularly to a method and apparatus for processing a video signal for encoding or decoding a video signal using sub-block based motion compensation. [Background technology]

[0002] Compression coding refers to a series of signal processing techniques for transmitting digitalized information through a communication line or storing it in a form suitable for a storage medium. The objects of compression coding include voice, video, text, etc., and the technology for compressing and coding video in particular is called video compression. Compression coding of video signals is performed by removing redundant information by considering spatial correlation, temporal correlation, probabilistic correlation, etc. However, with the recent development of various media and data transmission media, there is a demand for more efficient video signal processing methods and devices. Summary of the Invention [Problem to be solved by the invention]

[0003] An object of the present invention is to improve the coding efficiency of a video signal. An object of the present invention is to provide a method for efficiently performing sub-block based motion compensation. Also, an object of the present invention is to provide a motion compensation method based on a merge mode with motion vector difference (MVD). [Means for solving the problem]

[0004] In order to solve the above problems, the present invention provides a video signal processing device and a video signal processing method as follows.

[0005] According to one embodiment of the present invention, there is provided a video signal processing method, comprising: obtaining a first syntax element indicating whether a sub-block based merge mode is applied to a current block; obtaining a second syntax element indicating whether a merge mode using a motion vector differential is applied to the current block if the first syntax element indicates that a sub-block based merge mode is not applied to the current block; obtaining a third syntax element indicating a candidate to be used for inter prediction of the current block among candidates included in a merge candidate list of the current block if the second syntax element indicates that a merge mode using the motion vector differential is applied to the current block; obtaining information related to the motion vector differential; deriving a motion vector of the current block by adding the motion vector differential to a motion vector of a candidate indicated by the third syntax element; and generating a prediction block of the current block using the motion vector of the current block.

[0006] As an example, the third syntax element may indicate a candidate to be used for inter prediction of the current block among a first candidate and a second candidate in a merge candidate list of the current block.

[0007] As an example, the third syntax element may be parsed from the bitstream if the maximum number of merging candidates is greater than one, and may be inferred to be 0 if the maximum number of merging candidates is not greater than one.

[0008] As an embodiment, the step of obtaining information related to the motion vector difference may include the step of obtaining a fourth syntax element indicating a distance of the motion vector difference; and obtaining a fifth syntax element indicating a direction of the motion vector difference.

[0009] As an embodiment, when the first syntax element indicates that a sub-block based merge mode is applied to the current block, the method further includes obtaining a sixth syntax element indicating a candidate to be used for inter prediction of the current block among candidates included in a sub-block merge candidate list of the current block, and when the first syntax element indicates that a sub-block based merge mode is applied to the current block, a motion vector of the current block may be derived on a per at least one sub-block basis included in the current block based on a motion vector of a candidate indicated by the sixth syntax element.

[0010] As an example, if the first syntax element indicates that a sub-block based merging mode is applied to the current block, the second syntax element may be inferred to be 0.

[0011] As an example, the sixth syntax element may be parsed from the bitstream if the maximum number of sub-block merging candidates is greater than one, and may be inferred to be 0 if the maximum number of sub-block merging candidates is not greater than one.

[0012] According to one embodiment of the present invention, there is provided a video signal processing device, the video signal processing device including a processor, the processor obtaining a first syntax element indicating whether a sub-block based merge mode is applied to a current block, and if the first syntax element indicates that a sub-block based merge mode is not applied to the current block, obtaining a second syntax element indicating whether a merge mode using a motion vector differential is applied to the current block, and if the second syntax element indicates that a merge mode using the motion vector differential is applied to the current block, obtaining a third syntax element indicating a candidate to be used for inter prediction of the current block among candidates included in a merge candidate list of the current block, obtaining information related to the motion vector differential, deriving a motion vector of the current block by adding the motion vector differential to a motion vector of a candidate indicated by the third syntax element, and generating a prediction block of the current block using the motion vector of the current block.

[0013] As an example, the third syntax element may indicate a candidate to be used for inter prediction of the current block among a first candidate and a second candidate in a merge candidate list of the current block.

[0014] As an example, the third syntax element may be parsed from the bitstream if the maximum number of merging candidates is greater than one, and may be inferred to be 0 if the maximum number of merging candidates is not greater than one.

[0015] As an example, the processor may obtain a fourth syntax element indicating a distance of the motion vector difference, and obtain a fifth syntax element indicating a direction of the motion vector difference.

[0016] As an example, when the first syntax element indicates that a sub-block based merge mode is applied to the current block, the processor may obtain a sixth syntax element indicating a candidate to be used for inter prediction of the current block among candidates included in a sub-block merge candidate list of the current block, and when the first syntax element indicates that a sub-block based merge mode is applied to the current block, a motion vector of the current block may be derived on a per at least one sub-block basis included in the current block based on a motion vector of a candidate indicated by the sixth syntax element.

[0017] As an example, if the first syntax element indicates that a sub-block based merging mode is applied to the current block, the second syntax element may be inferred to be 0.

[0018] As an example, the sixth syntax element may be parsed from the bitstream if the maximum number of sub-block merging candidates is greater than one, and may be inferred to be 0 if the maximum number of sub-block merging candidates is not greater than one.

[0019] According to one embodiment of the present invention, a video signal processing method is provided, comprising: encoding a first syntax element indicating whether a sub-block based merge mode is applied to a current block; if the sub-block based merge mode is not applied to the current block, encoding a second syntax element indicating whether a merge mode using a motion vector differential is applied to the current block; if the merge mode using the motion vector differential is applied to the current block, encoding a third syntax element indicating a candidate to be used for inter prediction of the current block among candidates included in a merge candidate list of the current block; encoding information related to the motion vector differential; deriving a motion vector of the current block by adding the motion vector differential to a motion vector of a candidate indicated by the third syntax element; and generating a prediction block of the current block using the motion vector of the current block. Effect of the Invention

[0020] According to an embodiment of the present invention, it is possible to improve the coding efficiency of a video signal. In particular, according to an embodiment of the present invention, it is possible to reduce signaling overhead and improve compression efficiency by adjusting a syntax parsing order. [Brief description of the drawings]

[0021] [Figure 1] 1 is a schematic block diagram of a video signal encoding apparatus according to an embodiment of the present invention; [Diagram 2] 1 is a schematic block diagram of a video signal decoding device according to an embodiment of the present invention; [Diagram 3] FIG. 2 shows an example of how coding tree units are divided into coding units within a picture. [Figure 4] A diagram showing an embodiment of a method for signaling the splitting of quadtrees and multi-type trees. [Diagram 5] FIG. 2 is a diagram illustrating inter-prediction according to an embodiment of the present invention. [Figure 6] 1 is a diagram illustrating a motion vector signaling method according to one embodiment of the present invention. [Figure 7] 1 is a diagram illustrating a method of signaling adaptive motion vector resolution information according to one embodiment of the present invention. [Figure 8] FIG. 2 illustrates affine motion compensation according to one embodiment of the present invention. [Figure 9] FIG. 1 illustrates an embodiment of a four-parameter affine motion compensation method. [Figure 10] 1 is a diagram illustrating an affine motion prediction method according to an embodiment of the present invention. [Figure 11] FIG. 2 illustrates an embodiment of a sub-block based affine motion compensation method. [Figure 12] FIG. 2 illustrates an embodiment of the present invention for obtaining a set of control point motion vectors for prediction of a current block. [Figure 13] FIG. 2 illustrates an embodiment of the present invention for obtaining a set of control point motion vectors for prediction of a current block. [Figure 14] 1 is a diagram illustrating an affine motion compensation method according to an embodiment of the present invention; [Figure 15] 1 is a diagram illustrating an affine motion compensation method according to an embodiment of the present invention; [Figure 16] FIG. 1 illustrates a subblock-based temporal MVP (SbTMVP) according to one embodiment of the present invention. [Figure 17] FIG. 2 illustrates a planar MVP according to one embodiment of the present invention. [Figure 18] 4 is a diagram illustrating a method of determining a temporary motion vector according to an embodiment of the present invention. [Figure 19]FIG. 13 is a diagram illustrating the use or non-use of SbTMVP in one embodiment of the present invention. [Figure 20] FIG. 2 is a diagram illustrating a method for inducing SbTMVP according to one embodiment of the present invention. [Figure 21] 1 is a diagram illustrating a method of applying MMVD according to one embodiment of the present invention. [Figure 22] A figure illustrating an MMVD-related syntax structure in one embodiment of the present invention. [Figure 23] A figure illustrating an MMVD-related syntax structure in one embodiment of the present invention. [Figure 24] A figure illustrating an MMVD-related syntax structure in one embodiment of the present invention. [Diagram 25] A figure illustrating MMVD syntax for one embodiment of the present invention. [Figure 26] A figure illustrating MMVD syntax for one embodiment of the present invention. [Figure 27] A figure illustrating MMVD syntax for one embodiment of the present invention. [Figure 28] A figure illustrating MMVD syntax for one embodiment of the present invention. [Figure 29] A diagram illustrating a coding unit syntax structure according to one embodiment of the present invention. [Diagram 30] A diagram illustrating a merge data syntax structure in one embodiment of the present invention. [Diagram 31] A diagram illustrating a merge data syntax structure in one embodiment of the present invention. [Diagram 32] A diagram illustrating a merge data syntax structure in one embodiment of the present invention. [Diagram 33] 1 is a diagram illustrating a video signal processing method according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] The terms used in this specification are selected as general terms currently in wide use as much as possible while taking into consideration the functions in the present invention, but this may vary depending on the intentions of the engineers in this field, customs, or the emergence of new technologies. In addition, in certain cases, the applicant may arbitrarily select terms, and in such cases, the meanings of these terms will be described in the corresponding embodiment of the invention. Therefore, it is made clear that the terms used in this specification should be interpreted based on the substantial meanings of the terms and the contents of this specification as a whole, rather than simply the names of the terms.

[0023] In this specification, some terms are interpreted as follows. Coding may be interpreted as encoding or decoding in some cases. In this specification, an apparatus for encoding a video signal to generate a video signal bitstream is referred to as an encoding apparatus or encoder, and an apparatus for decoding a video signal bitstream to restore a video signal is referred to as a decoding apparatus or decoder. In this specification, a video signal processing apparatus is used as a term including both an encoder and a decoder. Information is a term including values, parameters, coefficients, elements, etc., and may be interpreted differently in some cases, so the present invention is not limited thereto. A "unit" is used to mean a basic unit of image processing or a specific position of a picture, and refers to an image area including both luma components and chroma components. In addition, a "block" refers to an image area including a specific component of a luma component and a chroma component (i.e., Cb and Cr). However, depending on the embodiment, the terms "unit", "block", "partition", and "area" may be used interchangeably. In this specification, a unit is used as a concept including a coding unit, a prediction unit, and a transform unit, and a picture refers to a field or a frame, and the terms are used interchangeably depending on the embodiment.

[0024] 1 is a schematic block diagram of a video signal encoding apparatus according to an embodiment of the present invention. Referring to FIG. 1, the encoding apparatus 100 of the present invention includes a transform unit 110, a quantization unit 115, an inverse quantization unit 120, an inverse transform unit 125, a filtering unit 130, a prediction unit 150, and an entropy coding unit 160.

[0025] The transform unit 110 transforms a residual signal, which is a difference between an input video signal and a prediction signal generated by the prediction unit 150, to obtain transform coefficient values. For example, a discrete cosine transform (DCT), a discrete sine transform (DST), or a wavelet transform is used. The discrete cosine transform and the discrete sine transform divide the input picture signal into blocks and perform the transform. In the transform, the coding efficiency may vary depending on the distribution and characteristics within the transform domain. The quantizer 115 quantizes the transform coefficient values ​​output from the transform unit 110.

[0026] In order to improve coding efficiency, a method is used in which a picture signal is not coded as it is, but a picture is predicted using a previously coded area through a prediction unit 150, and a residual value between an original picture and a predicted picture is added to the predicted picture to obtain a reconstructed picture. In order to prevent mismatch between the encoder and the decoder, when the encoder performs prediction, information that can also be used by the decoder should be used. For this purpose, the encoder performs a process of further reconstructing the current block that has been coded. The inverse quantization unit 120 inversely quantizes the transform coefficient values, and the inverse transform unit 125 reconstructs the residual values ​​using the inversely quantized transform coefficient values. Meanwhile, the filtering unit 130 performs a filtering operation to improve the quality of the reconstructed picture and the coding efficiency. For example, the filtering unit 130 includes a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter. The filtered picture is stored in a decoded picture buffer (DPB) 156 to be output or used as a reference picture.

[0027] In order to improve coding efficiency, a method is used in which a picture signal is not coded as it is, but a picture is predicted using an already coded area in the prediction unit 150, and a residual value between the original picture and the predicted picture is added to the predicted picture to obtain a restored picture. The intra prediction unit 152 performs intra prediction within the current picture, and the inter prediction unit 154 predicts the current picture using a reference picture stored in the decoded picture buffer 156. The intra prediction unit 152 performs intra prediction from a restored area in the current picture, and transmits intra coding information to the entropy coding unit 160. The inter prediction unit 154 may further include a motion estimation unit 154a and a motion compensation unit 154b. The motion estimation unit 154a obtains a motion vector value of the current area by referring to a specific restored area. The motion estimation unit 154a transmits position information (reference frame, motion vector, etc.) of the reference area to the entropy coding unit 160 so that it can be included in the bitstream. Using the motion vector values ​​transmitted from the motion estimation unit 154a, the motion compensation unit 154b performs inter-frame motion compensation.

[0028] The prediction unit 150 includes an intra prediction unit 152 and an inter prediction unit 154. The intra prediction unit 152 performs intra prediction within a current picture, and the inter prediction unit 154 performs inter prediction to predict the current picture using a reference buffer stored in a decoded picture buffer 156. The intra prediction unit 152 performs intra prediction from reconstructed samples within the current picture, and transmits intra coding information to the entropy coding unit 160. The intra coding information includes at least one of an intra prediction mode, a Most Probable Mode (MPM) flag, and an MPM index. The intra coding information may include information about a reference sample. The intra coding information may include information about a reference sample. The inter prediction unit 154 includes a motion estimation unit 154a and a motion compensation unit 154b. The motion estimation unit 154a obtains a motion vector value of a current region by referring to a specific region of a reconstructed reference signal picture. The motion estimation unit 154a transmits a motion information set (reference picture index, motion vector information) for the reference region to the entropy coding unit 160. The motion compensation unit 154b performs motion compensation using the motion vector value transmitted from the motion compensation unit 154a. The inter prediction unit 154 transmits inter coding information including the motion information for the reference region to the entropy coding unit 160.

[0029] According to a further embodiment, the prediction unit 150 includes an intra block copy (BC) prediction unit (not shown). The intra BC prediction unit performs intra BC prediction from reconstructed samples in the current picture and transmits intra BC coding information to the entropy coding unit 160. The intra BC prediction unit obtains block vector values ​​indicating a reference region to be used for predicting the current region by referring to a specific region in the current picture. The intra BC prediction unit performs intra BC prediction using the obtained block vector values. The intra BC prediction unit transmits the intra BC coding information to the entropy coding unit 160. The intra BC prediction unit includes the block vector information.

[0030] When the above-mentioned picture prediction is performed, the transform unit 110 converts residual values ​​between the original picture and the predicted picture to obtain transform coefficient values. At this time, the transform is performed in units of specific blocks within the picture, and the size of the specific blocks varies within a preset range. The quantization unit 115 quantizes the transform coefficient values ​​generated by the transform unit 110 and transfers the quantized values ​​to the entropy coding unit 160.

[0031] The entropy coding unit 160 generates a video signal bit stream by entropy coding information indicating quantized transform coefficients, intra-coding information, and inter-coding information. The entropy coding unit 160 uses a variable length coding (VLC) method, an arithmetic coding method, and the like. The variable length coding (VLC) method converts input symbols into continuous code words, and the length of the code words is variable. For example, frequently occurring symbols are represented by short code words, and infrequently occurring symbols are represented by long code words. As the variable length coding method, a context-based adaptive variable length coding (CAVLC) method is used. In arithmetic coding, continuous data symbols are converted into one prime number, and arithmetic coding obtains optimal prime number bits required to represent each symbol. As the arithmetic coding, a context-based adaptive binary arithmetic coding (CABAC) method is used. For example, the entropy coding unit 160 may binarize information indicating the quantized transform coefficients, and may arithmetically code the binarized information to generate a bitstream.

[0032] The generated bitstream is encapsulated in a basic unit of Network Abstraction Layer (NAL) units. An NAL unit includes an integer number of coded coding tree units. In order for a video decoder to decode the bitstream, the bitstream must first be separated into NAL units and then each separated NAL unit must be decoded. Meanwhile, information required for decoding the video signal bitstream is transmitted via Raw Byte Sequence Payload (RBSP) of higher level sets such as a Picture Parameter Set (PPS), a Sequence Parameter Set (SPS), a Video Parameter Set (VPS), etc.

[0033] 1 illustrates an encoding apparatus 100 according to one embodiment of the present invention, with separate blocks illustrating logically distinct elements of encoding apparatus 100. Thus, the elements of encoding apparatus 100 described above may be implemented on a single chip or multiple chips depending on the device design. According to one embodiment, the operation of each element of encoding apparatus 100 described above is performed by a processor (not shown).

[0034] 2 is a schematic block diagram of a video signal decoding device 200 according to an embodiment of the present invention. Referring to FIG. 2, the decoding device 200 of the present invention includes an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 225, a filtering unit 230, and a prediction unit 250.

[0035] The entropy decoding unit 210 entropy decodes the video signal bitstream to extract transform coefficient information, intra-coding information, inter-coding information, etc. for each region. For example, the entropy decoding unit 210 may obtain a binary code for transform coefficient information of a specific region from the video signal bitstream. The entropy decoding unit 210 also de-binarizes the binary code to obtain quantized transform coefficients. The inverse quantization unit 220 inverse quantizes the quantized transform coefficients, and the inverse transform unit 225 restores residual values ​​using the inverse quantized transform coefficients. The video signal processing device 200 combines the residual values ​​obtained from the inverse transform unit 225 with the predicted values ​​obtained from the prediction unit 250 to restore original pixel values.

[0036] Meanwhile, the filtering unit 230 performs filtering on the picture to improve image quality, including a deblocking filter for reducing block distortion and / or an adaptive loop filter for removing distortion of the entire picture, etc. The filtered picture is output or stored in a decoded picture buffer (DPB) 256 to be used as a reference picture for the next picture.

[0037] The prediction unit 250 includes an intra prediction unit 252 and an inter prediction unit 254. The prediction unit 250 generates a predicted picture using the coding type, the transform coefficient for each region, the intra / inter coding information, etc., decoded through the entropy decoding unit 210. To reconstruct the current block to be decoded, the current picture including the current block or a decoded region of another picture can be used. A picture (or tile / slice) that uses only the current picture for reconstruction, i.e., performs intra prediction or intra BC prediction, is called an intra picture or I picture (or tile / slice), and a picture (or tile / slice) that can perform all of intra prediction, inter prediction, and intra BC prediction is called an inter picture (or tile / slice). Among interpictures (or tiles / slices), a picture (or tile / slice) that uses at most one motion vector and reference picture index to predict sample values ​​of each block is called a predictive picture or P picture (or tile / slice), and a picture (or tile / slice) that uses at most two motion vectors and reference picture indexes is called a bi-predictive picture or B picture (or tile / slice). In other words, a P picture (or tile / slice) uses at most one motion information set to predict each block, and a B picture (or tile / slice) uses at most two motion information sets to predict each block. Here, a motion information set includes one or more motion vectors and one reference picture index.

[0038] The intra prediction unit 252 generates a prediction block using intra coding information and reconstructed samples in the current picture. As described above, the intra coding information includes at least one of an intra prediction mode, a MOST Probable Mode (MPM) flag, and an MPM index. The intra prediction unit 252 predicts sample values ​​of the current block using reconstructed samples located to the left and / or above the current block as reference samples. In the present disclosure, the reconstructed samples, the reference samples, and the samples of the current block refer to pixels. Also, a sample value refers to a pixel value.

[0039] In one embodiment, the reference samples are samples included in neighboring blocks of the current block. For example, the reference samples are samples adjacent to the left boundary and / or the top boundary of the current block. Also, the reference samples are samples located on a line within a preset distance from the left boundary of the current block and / or samples located on a line within a preset distance from the top boundary of the current block among samples of neighboring blocks of the current block. In this case, the neighboring blocks of the current block include at least one of a left (L) block, an upper (A) block, a below left (BL) block, an above right (AR) block, or an above left (AL) block adjacent to the current block.

[0040] The inter prediction unit 254 generates a prediction block using the reference picture and inter coding information stored in the decoded picture buffer 256. The inter coding information includes a motion information set (reference picture index, motion vector, etc.) of a current block with respect to a reference block. Inter prediction includes L0 prediction, L1 prediction, and bi-prediction. L0 prediction means a prediction using one reference picture included in an L0 picture list, and L1 prediction means a prediction using one reference picture included in an L1 picture list. For this purpose, one set of motion information (e.g., motion vector and reference picture index) is required. In the bi-prediction method, a maximum of two reference regions are used, and the two reference regions may exist in the same reference picture or may exist in different pictures. That is, in the bi-prediction method, a maximum of two sets of motion information (e.g., motion vector and reference picture index) are used, and the two motion vectors may correspond to the same reference picture index or may correspond to different reference picture indexes. In this case, the reference picture may be displayed (or output) either before or after the current picture in terms of time. According to one embodiment, in a bi-predictive scheme, the two reference regions used may be regions selected from the L0 picture list and the L1 picture list, respectively.

[0041] The inter prediction unit 254 obtains a current reference block using a motion vector and a reference picture index. The reference block exists in a reference picture corresponding to the reference picture index. In addition, a sample value of a block identified by the motion vector or an interpolated value thereof is used as a predictor of the current block. For motion prediction with pixel accuracy in sub-pel units, for example, an 8-tab interpolation filter is used for a luminance signal, and a 4-tab interpolation filter is used for a chrominance signal. However, the interpolation filter for sub-pel unit motion prediction is not limited thereto. In this way, the inter prediction unit 254 performs motion compensation to predict the texture of the current unit from a previously restored picture. In this case, the inter prediction unit uses a motion information set.

[0042] According to a further embodiment, the prediction unit 250 may include an intra BC prediction unit (not shown). The intra BC prediction unit may reconstruct the current region by referring to a specific region including reconstructed samples in the current picture. The intra BC prediction unit obtains intra BC coding information for the current region from the entropy decoding unit 210. The intra BC prediction unit obtains block vector values ​​of the current region indicating the specific region in the current picture. The intra BC prediction unit may perform intra BC prediction using the obtained block vector values. The intra BC coding information may include block vector information.

[0043] According to a further embodiment, the prediction unit 250 may include an intra BC prediction unit (not shown). The intra BC prediction unit may reconstruct the current region by referring to a specific region including reconstructed samples in the current picture. The intra BC prediction unit obtains intra BC coding information for the current region from the entropy decoding unit 210. The intra BC prediction unit obtains block vector values ​​of the current region indicating the specific region in the current picture. The intra BC prediction unit may perform intra BC prediction using the obtained block vector values. The intra BC coding information may include block vector information.

[0044] 2 shows a block diagram of a decoding apparatus 200 according to an embodiment of the present invention, with separate blocks logically separating elements of the decoding apparatus 200. Thus, the elements of the decoding apparatus 200 described above may be implemented on a single chip or multiple chips depending on the device design. According to an embodiment, the operation of each element of the decoding apparatus 200 described above is performed by a processor (not shown).

[0045] FIG. 3 illustrates an embodiment in which a coding tree unit (CTU) is divided into coding units (CUs) in a picture. In the process of coding a video signal, a picture is divided into a sequence of coding tree units (CTUs). A coding tree unit consists of two blocks, an NXN block of luma samples and a corresponding chroma sample. A coding tree unit is divided into a plurality of coding units. A coding tree unit may be a leaf node without being divided. In this case, the coding tree unit itself may be a coding unit. A coding unit refers to a basic unit for processing a picture in the above-mentioned video signal processing process, i.e., in the process of intra / inter prediction, transformation, quantization, and / or entropy coding. In one picture, the size and pattern of the coding units are not constant. The coding units have a square or rectangular pattern. A rectangular coding unit (or rectangular block) includes a vertical coding unit (or vertical block) and a horizontal coding unit (or horizontal block). In this specification, a vertical block is a block whose height is greater than its width, and a horizontal block is a block whose width is greater than its height. In addition, in this specification, a non-square block refers to a rectangular block, but the present invention is not limited to this.

[0046] Referring to Figure 3, the coding tree unit is first divided into a quad tree (QT) structure. That is, in the quad tree structure, one node having a size of 2N x 2N is divided into four nodes having a size of N x N. In this specification, the quad tree is also referred to as a quaternary tree. The quad tree division is performed recursively, and it is not necessary that all nodes are divided to the same depth.

[0047] Meanwhile, the leaf node of the above-mentioned quad tree is further divided into a Multi-Type Tree (MTT) structure. According to an embodiment of the present invention, in the Multi-Type Tree structure, one node is divided into a horizontally or vertically divided binary or ternary tree structure. That is, the Multi-Type Tree structure has four division structures, namely, vertical binary division, horizontal binary division, vertical ternary division, and horizontal ternary division. According to an embodiment of the present invention, in each of the tree structures, the width and height of the node are both powers of 2. For example, in a Binary Tree (BT) structure, a node of size 2N×2N is divided into two N×2N nodes by vertical binary division and divided into two 2N×N nodes by horizontal binary division. In addition, in a Ternary Tree (TT) structure, a node of size 2N x 2N is divided into (N / 2) x 2N, N x 2N, and (N / 2) x 2N nodes by vertical ternary division, and into 2N x (N / 2), 2N x N, and 2N x (N / 2) nodes by horizontal ternary division. Such multi-type tree division is performed recursively.

[0048] A leaf node of a multi-type tree can be a coding unit. If no division is indicated for a coding unit or the coding unit is not larger than the maximum transform length, the coding unit is used as a unit of prediction and transformation without further division. Meanwhile, in the above-mentioned quad tree and multi-type tree, at least one of the following parameters is predefined or transmitted via the RBSP of a higher level set such as PPS, SPS, VPS, etc. 1) CTU size: size of the root node of the quad tree, 2) MinQT size (MinQtSize): size of the minimum QT leaf node allowed, 3) MaxBt size (MaxBtSize): size of the maximum BT root node allowed, 4) MaxTt size (MaxTtSize): size of the maximum TT root node allowed, 5) MaxMttDepth (MaxMttDepth): maximum allowed depth of MTT division from the QT leaf node, 6) MinBt size (MinBtSize): size of the minimum BT leaf node allowed, 7) MinTT size: size of the minimum TT leaf node allowed.

[0049] 4 illustrates an embodiment of a method for signaling the split of a quad tree and a multi-type tree. A preset flag may be used to signal the split of the quad tree and the multi-type tree. Referring to FIG. 4, at least one of a flag 'qt_split_flag' indicating whether to split a quad tree node, a flag 'mtt_split_flag' indicating whether to split a multi-type tree node, a flag 'mtt_split_vertical_flag' indicating the split direction of a multi-type tree node, or a flag 'mtt_split_binary_flag' indicating the split type of a multi-type tree node may be used.

[0050] According to an embodiment of the present invention, the coding tree unit is the root node of a quad tree and can be split into a quad tree structure in advance. In the quad tree structure, 'qt_split_flag' is signaled for each node 'QT_node'. If the value of 'qt_split_flag' is 1, the corresponding node is split into four regular square nodes, and if the value of 'qt_split_flag' is 0, the corresponding node becomes a leaf node 'QT_leaf_node' of the quad tree.

[0051] Each quad tree leaf node 'QT_leaf_node' can be further split into a multi-type tree structure. In the multi-type tree structure, 'mtt_split_flag' is signaled for each node 'MTT_node'. If the value of 'mtt_split_flag' is 1, the corresponding node is split into multiple rectangular nodes, and if the value of 'mtt_split_flag' is 0, the corresponding node becomes the leaf node 'MTT_leaf_node' of the multi-type tree. If the multi-type tree node 'MTT_node' is split into multiple rectangular nodes (i.e., if the value of 'mtt_split_flag' is 1), 'mtt_split_vertical_flag' and 'mtt_split_binary_flag' can be additionally signaled for the node 'MTT_node'. If the value of 'mtt_split_vertical_flag' is 1, the node 'MTT_node' is split vertically, and if the value of 'mtt_split_vertical_flag' is 0, the node 'MTT_node' is split horizontally. If the value of 'mtt_split_binary_flag' is 1, the node 'MTT_node' is split into two rectangular nodes, and if the value of 'mtt_split_binary_flag' is 0, the node 'MTT_node' is split into three rectangular nodes.

[0052] Picture prediction (motion compensation) for coding is performed on coding units that cannot be further divided (i.e., leaf nodes of a coding unit tree). The basic unit for performing such prediction is hereinafter referred to as a prediction unit or a prediction block.

[0053] Hereinafter, the term "unit" used in this specification is used as an alternative term to the prediction unit, which is a basic unit for performing prediction, but the present invention is not limited thereto, and in a broader sense, the term "unit" is understood as a concept including the coding unit.

[0054] FIG. 5 illustrates inter-prediction according to an embodiment of the present invention. As described above, the decoder predicts a current block by referring to reconstructed samples of other decoded pictures. Referring to FIG. 5, the decoder obtains a reference block 42 in a reference picture based on motion information of a current block 32. At this time, the motion information may include a reference picture index and a motion vector 50. The reference picture index indicates a reference picture of the current block in a reference picture list. Also, the motion vector 50 represents an offset between coordinate values ​​of the current block 32 in the current picture and coordinate values ​​of the reference block 42 in the reference picture. The decoder obtains a predictor of the current block 32 based on sample values ​​of the reference block 42, and reconstructs the current block 32 using the predictor.

[0055] Meanwhile, according to an embodiment of the present invention, sub-block based motion compensation may be used. That is, the current block 32 may be divided into a plurality of sub-blocks, and an independent motion vector may be used for each sub-block. Therefore, each sub-block in the current block 32 may be predicted using a different reference block. According to an embodiment, the sub-block may have a preset size such as 4x4 or 8x8. The decoder obtains a predictor for each sub-block of the current block 32 using the motion vector of each sub-block. The predictor of the current block 32 may be obtained by combining the predictors of each sub-block, and the decoder may reconstruct the current block 32 using the predictor of the current block 32 obtained in this manner.

[0056] According to an embodiment of the present invention, various methods of subblock-based motion compensation may be performed. Subblock-based motion compensation may include affine model-based motion compensation (hereinafter, affine motion compensation or affine motion prediction) and subblock-based temporal motion vector prediction (Subblock-based Temporal Motion Vector Prediction, SbTMVP). Various embodiments of affine motion compensation and SbTMVP will be described below with reference to the drawings.

[0057] 6 is a diagram illustrating a motion vector signaling method according to an embodiment of the present invention. According to an embodiment of the present invention, a motion vector (MV) may be generated based on a motion vector prediction (or predictor) (MVP). As an example, the MV may be determined to be the MVP as shown in the following mathematical formula 1. In other words, the MV may be determined (or set, induced) to be the same value as the MVP.

[0058]

number

[0059] As another example, the MV may be determined based on the MVP and a motion vector difference (MVD), as shown in the following Mathematical Formula 2. The encoder can signal MVD information to the decoder to represent a more accurate MV, and the decoder can derive the MV by adding the obtained MVD to the MVP.

[0060]

number

[0061] According to an embodiment of the present invention, an encoder transmits the determined motion information to a decoder, and the decoder generates (or induces) an MV from the received motion information and generates a prediction block based on the MV. For example, the motion information may include MVP information and MVD information. In this case, components of the motion information may differ depending on an inter prediction mode. As an example, in a merge mode, the motion information may include MVP information but may not include MVD information. As another example, in an advanced motion vector prediction (AMVP) mode, the motion information may include MVP information and MVD information.

[0062] In order to determine, transmit, and receive information about MVPs, the encoder and the decoder can generate MVP candidates (or MVP candidate lists) in the same manner. For example, the encoder and the decoder can generate the same MVP candidates in the same order. Then, the encoder transmits an index indicating (or pointing to) the determined (or selected) MVP from the generated MVP candidates to the decoder, and the decoder can derive the determined MVP and / or MV based on the received index.

[0063] According to an embodiment of the present invention, the MVP candidate may include a spatial candidate, a temporal candidate, etc. The MVP candidate may be called a merge candidate when a merge mode is applied, and may be called an AMVP candidate when an AMVP mode is applied. The spatial candidate may be an MV (or motion information) for a block at a specific position based on the current block. For example, the spatial candidate may be an MV of a block at a position adjacent or non-adjacent to the current block. The temporal candidate may be an MV corresponding to a block in a picture different from the current picture. In addition, for example, the MVP candidate may include an affine MV, ATMVP, STMVP, a combination of the above-mentioned MVs (or candidates), an average MV of the above-mentioned MVs (or candidates), a zero MV, etc.

[0064] In one embodiment, the encoder may signal information indicating the reference picture to the decoder. As an example, if the reference picture of the MVP candidate is different from the reference picture of the current block (or the currently processed block), the encoder / decoder may perform motion vector scaling (MV scaling) of the MVP candidate. In this case, the MV scaling may be performed based on the picture order count (POC) of the current picture, the POC of the reference picture of the current block, and the POC of the reference picture of the MVP candidate.

[0065] A specific embodiment of the MVD signaling method is described below. Table 1 below illustrates a syntax structure for MVD signaling.

[0066] [Table 1]

[0067] Referring to Table 1, according to an embodiment of the present invention, the MVD may be coded with the sign and absolute value of the MVD separately. That is, the sign and absolute value of the MVD may be different syntax (or syntax elements). The absolute value of the MVD may be directly coded or may be coded stepwise based on a flag indicating whether the absolute value is greater than N, as in Table 1. If the absolute value is greater than N, the value of (absolute value-N) may be signaled as well. Specifically, in the example of Table 1, abs_mvd_greater0_flag indicating whether the absolute value is greater than 0 may be transmitted. If abs_mvd_greater0_flag indicates (or indicates) that the absolute value is not greater than 0, the absolute value of the MVD may be determined to be 0. If abs_mvd_greater0_flag indicates that the absolute value is greater than 0, an additional syntax (or syntax element) may be present.

[0068] For example, abs_mvd_greater1_flag may be transmitted to indicate whether the absolute value is greater than 1. If abs_mvd_greater1_flag indicates (or indicates) that the absolute value is not greater than 1, the absolute value of the MVD may be determined to be 1. If abs_mvd_greater1_flag indicates that the absolute value is greater than 1, additional syntax may be present. For example, abs_mvd_minus2 may be present. abs_mvd_minus2 may be a value of (absolute value-2). Because the abs_mvd_greater0_flag and abs_mvd_greater1_flag values ​​determine that the absolute value is greater than 1 (i.e., 2 or greater), the value of (absolute value-2) may be signaled. In this way, by hierarchically syntactically signaling information regarding the absolute value, fewer bits can be used compared to the case where the absolute value is directly binarized and signaled.

[0069] In one embodiment, the absolute value related syntax described above may be coded using a variable length binarization method such as Exponential-Golomb, truncated unary, truncated Rice, etc. Also, a flag indicating the sign of the MVD may be signaled by mvd_sign_flag.

[0070] In the above embodiment, a coding method for MVD has been described, but information other than MVD can also be signaled by dividing the sign and absolute value. The absolute value can be coded as a flag indicating whether the absolute value is greater than a predefined specific value and a value obtained by subtracting the specific value from the absolute value. In Table 1, [0] and [1] can represent component indexes. For example, they can represent x-component (i.e., horizontal component) and y-component (i.e., vertical component).

[0071] FIG. 7 is a diagram illustrating a method of signaling adaptive motion vector resolution information according to an embodiment of the present invention. According to an embodiment of the present invention, the resolution for indicating MV or MVD may be various. For example, the resolution may be expressed based on pixels (or pels). For example, MV or MVD may be signaled in units of 1 / 4, 1 / 2, 1 (integer), 2, 4 pixels, etc. Then, the encoder may signal the resolution information of MV or MVD to the decoder. Also, for example, 16 may be coded to 64 in 1 / 4 units (1 / 4*64=16), coded to 16 in 1 units (1*16=16), and coded to 4 in 4 units (4*.4=16). That is, the MV or MVD value may be determined by the following mathematical formula 3.

[0072]

number

[0073] In Equation 3, valueDetermined represents an MV or MVD value. Also, valuePerResolution represents a value signaled based on the determined resolution. In this case, if the value signaled by MV or MVD is not divisible by the determined resolution, a rounding process or the like may be applied. Using a high resolution may improve accuracy, but many bits may be used because the coded value is large, and using a low resolution may reduce accuracy, but fewer bits may be used because the coded value is small. In one embodiment, the above-mentioned resolution may be set individually in units such as a sequence, a picture, a slice, a coding tree unit (CTU), a coding unit (CU), etc. That is, the encoder / decoder may adaptively determine / apply the resolution according to a predefined unit among the above-mentioned units.

[0074] According to an embodiment of the present specification, the above-mentioned resolution information may be signaled from the encoder to the decoder. At this time, the resolution information may be binarized and signaled based on the above-mentioned variable length. In this case, if the resolution information is signaled based on an index corresponding to the smallest value (i.e., the first value), signaling overhead can be reduced. As an embodiment, the resolution information may be mapped to the signaling index in order from high resolution to low resolution.

[0075] According to an embodiment of the present specification, FIG. 7 illustrates a signaling method assuming that three resolutions are used among various resolutions. In this case, the three signaling bits may be 0, 10, and 11, and the three signaling indexes may represent a first resolution, a second resolution, and a third resolution, respectively. Since one bit is required to signal the first resolution and two bits are required to signal the remaining resolutions, the signaling overhead can be relatively reduced when the first resolution is signaled. In the example of FIG. 7, the first resolution, the second resolution, and the third resolution may be defined as 1 / 4, 1, and 4 pixel resolutions, respectively. In the following embodiment, MV resolution may refer to the resolution of the MVD.

[0076] 8 shows affine motion compensation according to an embodiment of the present invention. According to the existing inter prediction method, inter prediction is performed using a single motion vector for L0 prediction and L1 prediction for the current block, and is therefore optimized for predicting translation motion. However, in order to efficiently perform motion compensation for zoom in / out, rotation, and other irregular motions, reference blocks 44 of various shapes and sizes must be used.

[0077] 8, in affine motion compensation, a prediction of a current block 34 may be performed using a reference block 44 having a size, shape, and / or orientation different from that of the current block 34. That is, the reference block 44 may have a non-rectangular shape and may be larger or smaller in size than the current block 34. The reference block 44 may be obtained by performing an affine transformation on the current block 34. The affine transformation may include a 6-parameter affine transformation using three control point motion vectors (CPMV) and a 4-parameter affine transformation using two control point motion vectors. A specific embodiment of this will be described later.

[0078] FIG. 9 illustrates an embodiment of a 4-parameter affine motion compensation method. In order to reduce the amount of computation and signaling overhead of affine transformation, affine motion prediction can be performed using a pre-set number of control point motion vectors (CPMVs). A control point motion vector (CPMV) is a motion vector corresponding to a specific control point (or sample position) of a current block. The specific control point may include at least one of the vertices of the current block. In an embodiment of the present invention, a CPMV corresponding to the top left vertex of the current block is called v0 (or a first CPMV), a CPMV corresponding to the top right vertex of the current block is called v1 (or a second CPMV), and a CPMV corresponding to the bottom left vertex of the current block is called v2 (or a third CPMV). A CPMV set including at least two CPMVs can be used for affine motion prediction.

[0079] According to the embodiment of FIG. 9, four-parameter affine motion prediction can be performed using v0 and v1. The current block 36, represented by a solid line, can be predicted using a reference block 46 at a position represented by a dotted line. Each sample of the current block 36 can be mapped to a different reference sample by an affine transformation. More specifically, a motion vector (vx, vy) at a sample position (x, y) of the current block 36 can be derived by the following Equation 4.

[0080]

number

[0081] Here, (v0x, v0y) is a first CPMV corresponding to the top-left vertex of the current block 36, (v1x, v1y) is a second CPMV corresponding to the top-right vertex of the current block 36, and w is the width of the current block 36.

[0082] 10 is a diagram illustrating an affine motion prediction method according to an embodiment of the present invention. According to an embodiment of the present invention, affine motion can be represented using multiple CPMVs or multiple parameters.

[0083] Referring to FIG. 10, the encoder / decoder may perform affine motion prediction using CPMVs of v0, v1, and v2. When six parameters are used, three control point motion vectors (CPMVs) of v0, v1, and v2 may be used for affine motion prediction. Based on the three vectors of v0, v1, and v2 or the six parameters, it may be possible to indicate what shape of reference block a current block 36 is predicted from. Referring to FIG. 10, the current block 36 represented by a solid line may be predicted from a reference block 46 represented by a dotted line in FIG. 10 in a reference picture. That is, each control point (or pixel at the corresponding position) of the current block 36 may be mapped to another point (or pixel) by affine transformation. The following mathematical formula 5 represents a motion vector field according to an embodiment of the present invention.

[0084]

number

[0085] In Equation 5, CPMV v0 representing the motion vector of the top-left corner control point in FIG. 10 may be expressed as (mv_0^x, mv_0^y). CPMV v1 representing the motion vector of the top-right corner control point may be expressed as (mv_1^x, mv_1^y). CPMV v2 representing the motion vector of the bottom-left corner control point may be expressed as (mv_2^x, mv_2^y). In this case, the motion vector (mv^x, mv^y) at the (x, y) position may be derived (or calculated) using Equation 5. Therefore, the motion vector for the position of each pixel or a specific position may be derived by Equation 5 based on the control point motion vectors v0, v1, and v2.

[0086] Furthermore, in the mathematical formula 5, (x, y) may be relative coordinates within the block. For example, if the top left corner position of the block is (0, 0), (x, y) may be a position based on the top left corner position. Therefore, assuming that the position of v0 is (x0, y0), the position of v1 is (x1, y1), and the position of v2 is (x2, y2), and (x, y) is the same coordinate as the positions of v0, v1, and v2, then x and y in the mathematical formula 5 can be expressed as (x-x0) and (y-y0), respectively. In this case, w representing the width of the block may be (x1-x0), and h representing the height of the block may be (y2-y0).

[0087] FIG. 11 shows an example of a sub-block based affine motion compensation method. As described above, when using an affine motion transformation, a motion vector (i.e., a motion vector field) at each sample position of the current block can be derived. However, according to an embodiment of the present invention, in order to reduce the amount of computation, sub-block based affine motion compensation can be performed. As shown in FIG. 8, the current block can include a plurality of sub-blocks, and a representative motion vector of each sub-block is obtained based on the CPMV set. According to one embodiment, the representative motion vector of each sub-block may be a motion vector corresponding to the sample position at the center of the sub-block. According to an additional embodiment, a motion vector with higher accuracy compared to a general motion vector may be used as the motion vector of the sub-block. For this purpose, a motion compensation interpolation filter may be applied.

[0088] The size of the sub-block for which affine motion compensation is performed may be set in various ways. According to one embodiment of the present invention, the sub-block can have a size that is already set, such as 4X4 or 8X8. According to another embodiment of the present invention, the size MXN of the sub-block may be determined by the following mathematical formula 6.

[0089]

Equation

[0090] Here, w is the width of the current block, MvPre is the fractional unit accuracy of the motion vector. (v2x, v2y) is the third CPMV corresponding to the lower left vertex of the current block, and according to one embodiment, it may be calculated by mathematical formula 5. max(a, b) is a function that returns the relatively larger value between a and b, and abs(x) is a function that returns the absolute value of x. Also, clip3(x, y, z) is a function that returns x when z < x, returns y when z > y, and returns z in other cases.

[0091] The decoder obtains a motion vector for each subblock in the current block using the CPMV of the CPMV set. The decoder also obtains a predictor for each subblock using the motion vector for each subblock, and combines the predictors for each subblock to obtain a predictor for the current block. The decoder can reconstruct the current block using the predictor for the current block obtained in this way.

[0092] 12 and 13 show an embodiment of the present invention for obtaining a control point motion vector set for prediction of a current block. According to an embodiment of the present invention, a CPMV set for prediction of a current block may be obtained in various ways. More specifically, a CPMV set for prediction of a current block may be obtained by referring to a motion vector information set of one or more neighboring blocks. In an embodiment of the present invention, the motion vector information may represent a motion vector of a corresponding block or a CPMV of the corresponding block. Also, the motion vector information set represents a collection of motion vector information of one or more blocks. The neighboring block may represent a block including a previously set neighboring position of the current block. In this case, the neighboring block may be a coding unit including the previously set neighboring position, or may be an area of ​​a previously set unit (e.g., 4x4, 8x8) including the neighboring position.

[0093] There may be a plurality of candidates that can be referred to in deriving the CPMV of the current block. Therefore, information on the neighboring blocks to be referred to in deriving the CPMV of the current block may be signaled separately. According to an embodiment of the present invention, an indicator indicating a motion vector information set to be referred to in deriving a motion vector of each sub-block of the current block may be signaled. The indicator may indicate a motion vector information set of a neighboring block to be referred to in deriving a motion vector of each sub-block of the current block. A decoder may obtain the indicator and obtain each CPMV of the CPMV set for the current block by referring to the motion vector information set of the neighboring block indicated by the indicator. According to a more specific embodiment, the decoder may generate a candidate list consisting of one or more motion vector information set candidates. Each motion vector information set candidate constituting the candidate list is a motion vector set of a neighboring block that can be used to derive the motion vector information of the current block. In this case, the indicator may be an index indicating any one of the motion vector information sets in the candidate list. The CPMV of the current block may be obtained by referring to a motion vector information set selected from the candidate list based on the indicator (i.e., index). Various embodiments of motion vector information set candidates that can be included in a candidate list for deriving the motion vector information (or CPMV set) of a current block are described below.

[0094] FIG. 12 illustrates an embodiment of obtaining a CPMV set of a current block. In the embodiment of FIG. 12, it is assumed that the CPMV set of the current block includes two CPMVs, i.e., v0 and v1. According to an embodiment of the present invention, the CPMV of the current block may be derived from the motion vectors of the neighboring blocks adjacent to the corresponding point. Referring to FIG. 12, v0 may be derived from one of the motion vectors of the neighboring blocks A, B, and C adjacent to the corresponding point, and v1 may be derived from one of the motion vectors of the neighboring blocks D and E adjacent to the corresponding point. When the motion vectors of the neighboring blocks A, B, C, D, and E are vA, vB, vC, vD, and vE, respectively, a motion vector information set that may be included in a candidate list may be derived as shown in the following Mathematical Formula 7.

[0095]

number

[0096] That is, a (v0, v1) pair consisting of v0 selected from vA, vB, and vC and v1 selected from vD and vE may be obtained. In this case, v0 is derived from a motion vector of a block adjacent to an upper left vertex of the current block, and v1 is derived from a motion vector of a block adjacent to an upper right vertex of the current block. According to an additional embodiment, motion vector scaling may be performed based on a POC (Picture Order Count) of the current block, POCs of reference pictures of neighboring blocks, and POCs of reference pictures of the current block.

[0097] A candidate list including the motion vector information set candidates obtained in this manner may be generated, and an indicator indicating any one of the motion vector information sets in the candidate list may be signaled. According to an additional embodiment of the present invention, the candidate list may include motion vector information set candidates for inter prediction of other methods. For example, the candidate list may include motion vector information set candidates for sub-block-based temporal motion vector prediction (SbTMVP).

[0098] The decoder may derive the CPMV of the current block based on the motion vector information set obtained from the candidate list. According to one embodiment, the decoder may perform affine merging prediction by using the motion vectors of the motion vector information set obtained from the candidate list as the CPMV of the current block without a separate motion vector difference value. According to another embodiment, the decoder may obtain a separate motion vector difference value for the CPMV of the current block. The decoder may obtain the CPMV of the current block by summing the motion vectors of the motion vector information set obtained from the candidate list with the motion vector difference value. A flag or index indicating whether the decoder uses a separate motion vector difference value for affine motion compensation of the current block may be separately signaled.

[0099] FIG. 13 illustrates another embodiment of obtaining a CPMV set for a current block. According to another embodiment of the present invention, the CPMV of the current block may be derived from motion vector information of a neighboring block that has undergone affine motion compensation, i.e., the CPMV or motion vector of the neighboring block. In this case, the neighboring blocks may include a left neighboring block of the current block and an upper neighboring block of the current block. Referring to FIG. 13(a), the left neighboring block includes a block adjacent to the lower left vertex of the current block, i.e., a left block A and a lower left block D. Also, the upper neighboring blocks include a block adjacent to the upper left vertex of the current block, i.e., an upper left block E, and a block adjacent to the upper right vertex of the current block, i.e., an upper block B and an upper right block C. The decoder checks whether the neighboring blocks are affine motion compensated in a pre-set order. When a neighboring block that has undergone affine motion compensation is found, the decoder obtains a CPMV set for the current block using the CPMV set (or motion vector) of the neighboring block. Referring to the embodiment of FIG. 13(b), the CPMV set of the left block A may be used to derive a CPMV set for the current block. That is, the CPMV set (v0, v1) of the current block may be obtained based on the CPMV set (v2, v3, v4) of the left block A.

[0100] According to an embodiment of the present invention, information on neighboring blocks to be referenced for deriving the CPMV of the current block may be signaled separately. In this case, the CPMV sets of the neighboring blocks of the current block may become motion vector information set candidates constituting the above-mentioned candidate list according to a previously set order. More specifically, the motion vector information set candidates may include a first candidate derived from the CPMV (or motion vector) of the left neighboring block of the current block and a second candidate derived from the CPMV (or motion vector) of the upper neighboring block of the current block. In this case, the left neighboring block is a block adjacent to the lower left vertex of the current block, and the upper neighboring block is a block adjacent to the upper left vertex of the current block or a block adjacent to the upper right vertex of the current block. A candidate list including the motion vector information set candidates obtained in this manner may be generated, and an indicator indicating any one of the motion vector information sets in the candidate list may be signaled. According to an embodiment, the indicator may indicate position information of the neighboring block to be referenced for deriving the motion vector of each sub-block of the current block. The decoder can obtain the CPMV set of the current block by referring to the CPMV set (or motion vector) of the neighboring block indicated by the pointer.

[0101] According to an additional embodiment of the present invention, the CPMV of the current block may be derived based on the CPMV of the neighboring blocks close to the point in question. For example, v0 may be obtained by referring to the CPMV of the left neighboring block, and v1 may be obtained by referring to the CPMV of the upper neighboring block. Or, v0 may be obtained by referring to the CPMV of the neighboring block A, D, or E, and v1 may be obtained by referring to the CPMV of the neighboring block B or C.

[0102] FIG. 14 is a diagram for explaining an affine motion compensation method according to an embodiment of the present invention. According to an embodiment of the present invention, a CPMV may be required for affine motion prediction, and a motion vector field, i.e., a motion vector for a subblock or a certain position (or pixel), may be calculated based on the CPMV. In this specification, the CPMV may be referred to as a seed vector. In this case, the CPMV may be derived based on a predictor. For example, the predictor may be determined as the CPMV. As another example, the CPMV may be calculated based on a predictor and a difference. Specifically, the CPMV may be calculated by adding or subtracting the difference to the predictor.

[0103] According to an embodiment of the present invention, a predictor of CPMV may be derived from the CPMV or MV of a neighboring affine motion compensation (MC) block (i.e., a neighboring block coded with affine mode). For example, if a block at a previously set position is affine motion predicted, the encoder / decoder may derive a predictor for affine motion compensation of a current block from the CPMV or MV of the block. In one embodiment, the previously set position may be the A0, A1, B0, B1, or B2 position shown in FIG. 14. Alternatively, the previously set position may include a position adjacent to the current block and / or a position not adjacent to the current block. Also, the encoder / decoder may refer to the CPMV or MV of a previously set position adjacent in space, or may refer to the CPMV or MV of a previously set position adjacent in time. In this specification, the candidate based on the above-mentioned affine MC block may be referred to as an inherited candidate. Alternatively, the candidate based on the above-mentioned affine MC block may be referred to as a merge candidate. Also, in one embodiment, when the encoder / decoder refers to a pre-established position, it may do so in a pre-established order.

[0104] FIG. 15 is a diagram for explaining an affine motion compensation method according to an embodiment of the present invention. A CPMV may be required for affine motion prediction, and a motion vector field, i.e., a motion vector for a subblock or a certain position, may be calculated based on the CPMV. In this specification, the CPMV may be referred to as a seed vector. In this case, the CPMV may be derived based on a predictor. For example, the predictor may be determined as the CPMV. As another example, the CPMV may be calculated based on a predictor and a difference. Specifically, the CPMV may be calculated by adding or subtracting the difference to the predictor.

[0105] According to an embodiment of the present invention, the encoder / decoder may derive a predictor of the CPMV using the MV of the neighboring block. At this time, the MV of the neighboring block may include an MV other than the MV of the affine MC block. For example, when the encoder / decoder derives each CPMV of the current block, the encoder / decoder may use the MV of the position already set for each CPMV as a predictor of the CPMV. For example, the already set position may be a portion included in a block adjacent to the portion.

[0106] Referring to FIG. 15, the encoder / decoder can determine CPMVs mv0, mv1, and mv2. Here, mv0 represents the upper left CPMV, mv1 represents the upper right CPMV, and mv2 represents the lower left CPMV. According to an embodiment of the present invention, the encoder / decoder can use MVs corresponding to previously set positions A, B, and C as predictors for mv0. The encoder / decoder can also use MVs corresponding to previously set positions D and E as predictors for mv1. The encoder / decoder can also use MVs corresponding to previously set positions F and G as predictors for mv2.

[0107] When determining each predictor of CPMV mv0, mv1, and mv2 according to an embodiment of FIG. 15, an order of referring to the already-set positions for each control point position may be predefined. As an example, there may be a number of already-set positions for each control point position to be referred to as predictors of CPMV, and a combination of possible already-set positions may be predefined. A candidate (or predictor) for affine MC may be derived according to the method of the embodiment of FIG. 15, and such a candidate may be called a constructed candidate. Alternatively, such a candidate may be called an inter candidate or a virtual candidate. In addition, in the method of FIG. 15, when referring to the already-set positions, the encoder / decoder may refer to them according to a predefined order.

[0108] According to an embodiment of the present invention, an encoder / decoder may generate an affine MC candidate list or an affine MC CPMV candidate list using the embodiments described in Figures 12 to 15 or a combination thereof. The following Equation 8 illustrates a method for deriving an affine motion predictor according to an embodiment of the present invention.

[0109]

number

[0110] As described above, a CPMV for affine motion prediction of a current block may be derived from neighboring affine motion predicted blocks. In this case, the above-mentioned mathematical formula 8 may be used. In the mathematical formula 8, the MVs of the top-left, top-right, and bottom-left control points of neighboring affine motion predicted blocks may be expressed as (v_E0x, v_E0y), (v_E1x, v_E1y), and (v_E2x, v_E2y), respectively. Also, the coordinates of the top-left, top-right, and bottom-left of neighboring affine motion predicted blocks may be expressed as (x_E0, y_E0), (x_E1, y_E1), and (x_E2, y_E2), respectively. In this case, the encoder / decoder can calculate (v_0x, v_0y), (v_1x, v_1y), which are predictors or CPMVs of the current block, according to Equation 8. Equations 9 and 10 below illustrate a method of deriving an affine motion predictor according to an embodiment of the present invention.

[0111]

number

[0112]

number

[0113] As described above, multiple CPMVs or multiple CPMV predictors may be used for affine motion compensation. In this case, the encoder / decoder can derive another CPMV or CPMV predictor from a CPMV or CPMV predictor. For example, the encoder / decoder can derive (or generate) two CPMVs or two CPMV predictors by the above-mentioned method, and derive another CPMV or another CPMV predictor based on the two CPMVs or two CPMV predictors.

[0114] According to an embodiment of the present invention, the encoder / decoder can derive (or generate) mv0, mv1, and mv2, which are the CPMV predictors or CPMVs of the top left, top right, and bottom left, using Mathematical Formulas 9 and 10. In Mathematical Formulas 9 and 10, x and y represent the x-component and y-component, respectively, and the size of the current block may be w*h. The encoder / decoder can create a predictor for CPMV to perform affine MC on the current block, and add a difference to the predictor to determine the CPMV. According to an embodiment, the CPMV predictor can be generated by the method described in FIG. 14 and FIG. 15, and the CPMV difference can be signaled from the encoder to the decoder.

[0115] Also, in one embodiment, there may be a difference for each CPMV, and the difference for each CPMV may be signaled respectively. In a four-parameter model, two CPMVs may be used, and the MV for each CPMV may be derived by adding mvd0, mvd1, respectively, to the predictor for each CMPV. For a six-parameter model, three CPMVs may be used, and the MV for each CPMV may be derived by adding mvd0, mvd1, mvd2, respectively, to the predictor for each CMPV.

[0116] According to an embodiment of the present invention, the encoder / decoder may determine the motion vector differential based on the following Equation 11.

[0117]

number

[0118] In Equation 11, if the reference list is 0, MvdL0[x0][y0][compIdx] may be set to lMvd[compIdx]. At this time, compIdx represents a component index and may have a value of 0 or 1. If the reference list is 1, MvdL1[x0][y0][compIdx] may be set to lMvd[compIdx].

[0119] In one embodiment, a motion vector difference may be signaled by the method described in Table 1 above, and the motion vector difference may be lMvd in the above Mathematical Formula 11. As described above, the encoder / decoder may determine the signaled mvd as a difference for a CPMV predictor, and the determined difference may be MvdL0 and MvdL1 in Mathematical Formula 11. In Mathematical Formula 11, L0 represents reference list 0 and L1 represents reference list 1.

[0120] According to an embodiment of the present invention, the encoder / decoder may derive a predictor for CPMV and add a difference thereto to determine CPMV in order to perform affine MC on the current block. As an embodiment, the encoder / decoder may generate a predictor for CPMV by the method described in FIG. 14 and FIG. 15. The difference of CPMV may be signaled from the encoder to the decoder. In an embodiment, the encoder / decoder may derive CPMV based on the following mathematical formulas 12 and 13.

[0121]

number

[0122]

number

[0123] Referring to Equations 12 and 13, a predictor for the difference for each CPMV may exist. In one embodiment, a difference for another CPMV may be determined based on a difference for a CPMV. This may be based on the similarity between the differences for the CPMV. That is, since the differences for the CPMV are similar to each other, if a predictor for one CPMV is determined, a predictor for the CPMV may be determined using relatively little information. As an example, a difference predictor for the CPMV may be signaled, and a difference between the difference predictor for the CPMV and the difference may be signaled from the encoder to the decoder. Equation 12 assumes that a 4-parameter model is used, and Equation 13 assumes that a 6-parameter model is used.

[0124] Referring to Equations 12 and 13, the encoder / decoder can determine the difference and CPMV for each CPMV based on the difference (mvd0) of mv0, which is CPMV 0. mvd0, mvd1, and mvd2 in Equations 12 and 13 may be signaled from the encoder to the decoder. Compared to the method described in FIG. 15 above, the method in Equations 12 and 13 may have different values ​​of mvd1 and mvd2 signaled, even though the method in FIG. 15 above uses the same predictor. If the differences between the predictors of CPMV mv0, mv1, and mv2 are similar, the absolute values ​​of mvd1 and mvd2 may be smaller when using the method in Equations 12 and 13 than when using the method described in FIG. 15 above, thereby effectively reducing the signaling overhead of mvd1 and mvd2. The encoder / decoder can determine the difference between the predictor of mv1 as (mvd1+mvd0) and the difference between the predictor of mv2 as (mvd2+mvd0). In Equations 12 and 13, the letters with a bar above them represent the predictors of CPMV.

[0125] According to an embodiment of the present invention, the encoder / decoder may determine the motion vector differential based on the following Equation 14. In this case, the above-mentioned Equations 12 and 13 may be applied.

[0126]

number

[0127] In one embodiment, the motion vector difference may be signaled according to Table 1 described above or Table 2 described below. The signaled motion vector difference may be lMvd in Equation 14. Also, mvd signaled in Equations 12 and 13, i.e., mvd0, mvd1, and mvd2, may be lMvd in Equation 14. In Equation 14, MvdLX may represent the difference between each CPMV and the predictor. That is, it may be (mv-mvp). In this case, as described in Equations 12 and 13, for CPMV 0, the signaled motion vector difference may be directly used as the difference (MvdLX) for CPMV, and for other CPMVs, it may be used as MvdLX, which is the difference of CPMV, based on the signaled motion vector difference (mvd1, mvd2 in Equations 12 and 13) and the motion vector difference signaled for CPMV 0 (mvd0 in Equations 12 and 13).

[0128] In Equation 14, LX may represent a reference list X. compIdx represents a component index and may indicate x and y components. cpIdx may represent a control point index. cpIdx may mean 0, 1 or 0, 1, 2 as represented in Equations 12 and 13. In one embodiment, in Table 1, Equation 14, and Table 2, the encoder / decoder may take into account the resolution of the motion vector difference. For example, when the resolution is R, the value of lMvd*R may be used as lMvd in Equation 14. Table 2 below illustrates a motion vector difference syntax structure according to one embodiment of the present invention.

[0129] [Table 2]

[0130] Referring to Table 2, the motion vector differential may be coded in substantially the same manner as described above in Table 1. In this case, coding of the motion vector differential may be performed individually according to cpIdx and / or control point index.

[0131] FIG. 16 is a diagram illustrating a subblock-based temporal MVP (SbTMVP) according to an embodiment of the present invention. According to an embodiment of the present invention, an encoder / decoder can incorporate (or induce) a motion vector of a temporal neighboring block on a subblock basis. For example, the encoder / decoder can incorporate a motion vector corresponding to a subblock of a current block from another picture. Here, the block may be a CU, and the subblock may be a sub-CU. Also, the other picture may be a collocated picture. In the present invention, the SbTMVP may be called a subblock-based temporal merge candidate, or alternative temporal motion vector prediction (ATMVP).

[0132] According to an embodiment of the present invention, when the SbTMVP mode is applied, the encoder / decoder can search for a block (which can be called a corresponding block or a reference block) corresponding to the current block from another picture. In this case, the corresponding block can be called a collocated block. In one embodiment, the collocated block can be determined as a block of another picture corresponding to a pre-defined position, such as the lower right corner or center position, of the current block. In SbTMVP, the collocated block can be a motion compensated block. For example, the encoder / decoder can search for the collocated block by referring to a motion vector around the current block.

[0133] Referring to FIG. 16, a co-located block may be determined (or induced) as a co-located block by a block that is moved from the current block position in another picture by the motion vector of the A1 position among neighboring blocks of the current block. In one embodiment, the motion vector pointing (or heading) to the co-located block may be induced based on the motion vector of a candidate that is searched by scanning previously set positions around the current block according to a previously set order. In yet another embodiment, the motion vector pointing to the co-located block may be determined based on a part of the merge candidate list. For example, the encoder / decoder may determine the co-located block using the first merge candidate in the merge candidate list.

[0134] Also, according to an embodiment of the present invention, the encoder / decoder may determine (or derive) motion information of a sub-block of a current block from motion information of a sub-block of a co-located block. In this specification, a motion vector may mean motion information including a motion vector, a reference index, information on whether a reference list is used, etc., regardless of its expression. Also, the sub-blocks of the co-located block that are the basis of the motion information of the sub-block of the current block may have the same relative position within a block (CU). That is, both sub-blocks may have the same relative position within a block.

[0135] Also, according to one embodiment of the present invention, when SbTMVP is used, the subblock size may be fixed to a predefined size. For example, it may be fixed to a size of 8×8. In another embodiment, the subblock size may be variable. In this case, sequence, picture, slice, CTU or CU unit signaling may be present to determine the variable subblock size. For example, the subblock size may be variably determined to a size of 4×4 or 8×8.

[0136] FIG. 17 is a diagram illustrating a planar MVP according to an embodiment of the present invention. According to an embodiment of the present invention, an encoder / decoder may generate a motion vector field for each sub-block of a current block using a planar MVP method. Also, the encoder / decoder may refer to neighboring motion vectors using the planar MVP method. Also, in the planar MVP method, the encoder / decoder may determine a sub-block motion vector based on a linear weighted summation using one or more neighboring motion vectors. In this case, the neighboring motion vector may include motion information extrapolated from the motion vectors of neighboring blocks. For example, the encoder / decoder may perform an extrapolation process for a motion vector at a lower or right position of the current block using the motion vectors of neighboring blocks.

[0137] Referring to FIG. 17, a motion vector P(x,y) corresponding to a sub-block of a current block may be determined based on P_h(x,y) or P_v(x,y). In FIG. 17, W and H respectively represent the width and height of the current block. In this case, P_h(x,y) may be determined based on L(-1,y) or R(W,y). L(-1,y) may be a motion vector corresponding to a sub-block adjacent to the current block on the left side of the current sub-block. Also, R(W,y) may be a value based on a motion vector corresponding to a position on the upper right side of the current block and a temporal motion vector corresponding to a position on the lower right side of the current block. P_v(x,y) may be determined based on A(x,-1) or B(x,H). A(x,-1) may be a motion vector corresponding to a sub-block adjacent to the current block on the upper side of the current sub-block. Also, B(x,H) may be a weighted sum value based on a motion vector corresponding to the bottom left position of the current block and a top right motion vector corresponding to the bottom right position of the current block.

[0138] Table 3 below illustrates an inter prediction related syntax structure according to an embodiment of the present invention.

[0139] [Table 3A]

[0140] [Table 3B]

[0141] According to an embodiment of the present invention, the inter prediction method may include skip mode, merge mode, inter mode, etc. As an embodiment, in skip mode, a residual signal may not be transmitted. Also, the same MV determination method as in merge mode may be applied to skip mode. As an example, whether or not skip mode is used may be determined by a skip flag (or a syntax element). With reference to Table 3, whether or not skip mode is used may be determined by a cu_skip_flag value. The cu_skip_flag indicates whether or not skip mode is applied to the current CU.

[0142] In one embodiment, when the merge mode is applied, the motion vector difference may not be used. In this case, the encoder / decoder may determine the motion vector based on a motion candidate index. For example, whether or not the merge mode is used may be determined by a merge flag (or a syntax element). With reference to Table 3, whether or not the merge mode is applied may be determined by a merge flag (i.e., merge_flag) value. Also, the merge mode may be used when the skip mode is not used.

[0143] According to an embodiment of the present invention, the encoder / decoder may selectively use one of one or more candidate list sets in a skip mode or a merge mode. For example, the candidate list set may include a candidate list using a merge candidate and / or a candidate list using a subblock-based merge candidate. The merge candidates may include a spatial neighboring candidate, a temporal neighboring candidate, etc. The merge candidates may include a candidate using a motion vector for the entire current block (CU). That is, the merge candidates may include candidates in which the motion vectors of each subblock belonging to the current block are the same. The subblock merge candidates may include a subblock-based temporal motion vector, an affine merge candidate, etc. In one embodiment, the subblock merge candidates may include candidates using different motion vectors for each subblock of the current block (CU). In one embodiment, the affine merge candidates may represent merge candidates that determine the CPMV of affine motion prediction without a motion vector difference. The subblock merge candidates may include a method of determining a motion vector in a subblock unit in the current block. For example, the sub-block merging candidates may include planar MV, regression based MV, STMVP, etc. in addition to the sub-block based temporal MV and affine merging candidates mentioned above.

[0144] According to an embodiment of the present invention, in an inter mode, an encoder / decoder can use a motion vector differential. A motion vector predictor can be determined based on a motion candidate index, and a motion vector can be determined based on the motion vector predictor and the motion vector differential. Whether or not an inter mode is used can be determined based on whether or not another mode is used. As another embodiment, whether or not an inter mode is used can be determined by a flag (or a syntax element). In Table 3, an example is assumed in which an inter mode is used when other modes, such as a skip mode and a merge mode, are not used. In an embodiment, the inter mode can include an AMVP mode, an affine inter mode, etc. The inter mode can be a mode in which a motion vector is determined based on a motion vector predictor and a motion vector differential. In an affine inter mode, an encoder / decoder can use a motion vector differential when determining a CPMV of an affine motion prediction.

[0145] Referring to Table 3, the encoder / decoder may determine whether to use a subblock merging candidate or a merge candidate after determining the skip mode or the merge mode. For example, the decoder may parse merge_subblock_flag, which indicates whether to use a subblock merging candidate when a specific condition is satisfied. In this case, the specific condition may be a condition related to a block size. For example, the specific condition may be a condition related to width, height, area, etc., or a combination of these conditions.

[0146] Referring to Table 3, for example, the specific condition may be whether the width and height of the current block (CU) are equal to or greater than a specific value. If merge_subblock_flag is not parsed, the encoder / decoder can infer (infer) its value as 0. If merge_subblock_flag is 1, the encoder / decoder can use the subblock merge candidate, and if merge_subblock_flag is 0, the encoder / decoder can use the merge candidate. If a subblock merge candidate is used, the encoder / decoder can parse merge_subblock_idx indicating an index (or syntax element) indicating a specific candidate in a candidate list (subblock merge candidate list), and if a merge candidate is used, the encoder / decoder can parse merge index indicating an index indicating a specific candidate in a candidate list (merge candidate list). In this case, the encoder / decoder may not parse if the maximum number of the candidate list is 1. That is, the encoder / decoder can parse the index if the maximum number of the candidate list is greater than 1. If merge_subblock_idx or merge index is not parsed, the encoder / decoder can infer its value as 0.

[0147] The above Table 3 illustrates coding unit functions, and intra prediction-related content may be omitted. Table 3 may be an example of a case where it is determined that inter prediction is applied.

[0148] A method for constructing a sub-block merging candidate list according to an embodiment of the present invention will be described below. Sub-block merging candidates are described in Table 3 above, and a method for constructing a sub-block merging candidate list will be described below. According to an embodiment of the present invention, an encoder / decoder may determine whether to add a candidate to a candidate list in consideration of a size of a current block or a size of a sub-block. The above-mentioned size may be defined based on a width, height, or area of ​​a block or a sub-block.

[0149] In one embodiment, when there are multiple candidates that may belong to the subblock merging candidates and the multiple candidates have different subblock sizes, it may be determined whether to add a candidate to the candidate list by considering the size of the current block. Also, the condition for considering the size of the current block for a certain candidate may be based on the subblock size of the certain candidate. Here, a candidate (before being added to the candidate list) may represent any MVP method, mode, etc., and may include any method of making an MVP or MV, and a candidate added to the candidate list may represent a candidate, MVP method, mode, etc. that can be used for the current block.

[0150] According to an embodiment of the present invention, an encoder / decoder may determine whether to add a candidate to a candidate list according to a block size condition of a current block. For example, the encoder / decoder may determine whether to add a candidate to a candidate list according to whether the size of the current block is equal to or larger than a threshold of a certain block size. In addition, the block size condition or threshold may be determined according to a sub-block size of the certain candidate. This is because the sub-block based prediction method can improve compression efficiency by using sub-block based prediction in a block that is larger than the sub-block size or sufficiently larger than the sub-block size. For example, in the case of a block that is the same as or smaller than the sub-block size, even if the sub-block based prediction is applied, motion information on a sub-block basis cannot be obtained, and as a result, it may not contribute to compression efficiency.

[0151] In one embodiment, the encoder / decoder may configure a first candidate list when the size of the current block is equal to or less than the pre-set size, and may configure a second candidate list when the current block is equal to or greater than the pre-set size. Also, the candidate types belonging to the candidate list 2 may include the candidate types belonging to the candidate list 1. For example, if the sub-block size of the first mode is set to the first sub-block size and the sub-block size of the second mode is set to the second sub-block size, the encoder / decoder may add only the second mode to the candidate list when the current block is equal to the pre-set size, and may add both the first and second modes to the candidate list when the current block is larger than the pre-set size. For example, the first sub-block size may be 8×8 and the second sub-block size may be 4×4. Also, the pre-set size may be 8×8. Or, the pre-set size may have a width (or height) of 8. Also, as an example, the first mode may be SbTMVP and the second mode may be affine merge mode.

[0152] In one embodiment, a sub-block based merging candidate list may be constructed (or generated) in the following manner: In the following, first, i=0 may be set.

[0153] If the (availableFlagSbCol && blockSizeCondition) value is true, then subblockMergeCandList[i++] is set to SbCol, i.e., SbCol may be added to the candidate list, where subblockMergeCandList[i++] represents the i++th subblock merging candidate in the subblock merging candidate list, and SbCol represents the subblock-based temporal merging candidate (or SbTMVP).

[0154] - If the value of (availableFlagA && i<MaxNumSubblockMergeCand) is true, then subblockMergeCandList[i++] is set to A. That is, the A candidate (or the candidate at the A position) may be added as the next candidate.

[0155] - If the value of (availableFlagB && i<MaxNumSubblockMergeCand) is true, then subblockMergeCandList[i++] is set to B. That is, the B candidate may be added as the next candidate.

[0156] - If the value of (availableFlagConst1 && i<MaxNumSubblockMergeCand) is true, then subblockMergeCandList[i++] is set to Const1. That is, the Const1 candidate may be added as the next candidate. Const1 represents the first candidate among the configured candidates described above.

[0157] - If the value of (availableFlagConst2 && i<MaxNumSubblockMergeCand) is true, then subblockMergeCandList[i++] is set to Const2. That is, the Const2 candidate may be added as the next candidate. Const2 represents the second candidate among the configured candidates described above.

[0158] - If the value of (availableFlagConst3 && i<MaxNumSubblockMergeCand) is true, then subblockMergeCandList[i++] is set to Const3. That is, the Const3 candidate may be added as the next candidate. Const3 represents the third candidate among the configured candidates described above.

[0159] - If the value of (availableFlagConst4 && i<MaxNumSubblockMergeCand) is true, then subblockMergeCandList[i++] is set to Const4. That is, the Const4 candidate may be added as the next candidate. Const4 represents the fourth candidate among the configured candidates described above.

[0160] - If the value of (availableFlagConst5 && i<MaxNumSubblockMergeCand) is true, then subblockMergeCandList[i++] is set to Const5. That is, the Const5 candidate may be added as the next candidate. Const5 represents the fifth candidate among the configured candidates described above.

[0161] - If the value of (availableFlagConst6 && i<MaxNumSubblockMergeCand) is true, then subblockMergeCandList[i++] is set to Const6. That is, the Const6 candidate may be added as the next candidate. Const6 represents the sixth candidate among the configured candidates described above.

[0162] According to an embodiment of the present invention, the encoder / decoder adds the SbCol candidate to the candidate list, subblockMergeCandList, only if the candidate satisfies a predefined block size condition. In one embodiment, the block size condition for determining whether to add SbTMVP to the candidate list may be determined by the subblock size of SbTMVP. For example, the block size condition may be (cbWidth>=16 && cbHeight>=16), where cbWidth represents the width of the current coding block, and cbHeight represents the height of the current coding block. Also, A, B, Const1, Const2, Const3, Const4, Const5, and Const6 may be affine merge candidates. A and B may be affine inheritance candidates, and ConstX (X is any one of 1 to 6) may be an affine constructed candidate.

[0163] In another embodiment, the encoder / decoder may determine whether to parse the candidate index according to the block size condition of the current block. For example, when there are multiple candidates, if all but one of the multiple candidates are not added to the candidate list according to the block size condition and other conditions, the encoder / decoder may not parse the candidate index. For example, when there are two candidates that can be added to the candidate list, the encoder / decoder may determine whether to parse the candidate index according to the block size condition. For example, if one of the two candidates that can be added to the candidate list does not satisfy the block size condition, the encoder / decoder may not parse the candidate index.

[0164] Table 4 below illustrates an example of inter prediction related syntax according to an embodiment of the present invention.

[0165] [Table 4A]

[0166] [Table 4B]

[0167] According to an embodiment of the present invention, the subblock mode may be used when the size of the current block is larger than the subblock size. For example, when there are many candidates that can be added to the subblock merge candidate list and the subblock sizes of the many candidates are equal, the subblock merge mode may be used when the size of the current block is larger than the subblock size. As a specific example, when the subblock size is 8×8 and the current block size is 16×16 or more (or the width or height is 16 or more), the subblock mode may be used. The use of the subblock mode may indicate the presence or absence of flag parsing indicating the use or absence of the mode corresponding to the subblock mode. In addition, the flag indicating the use or absence of the mode corresponding to the subblock mode may be merge_subblock_flag, inter_affine_flag, etc. In Table 4, the description overlapping with Table 3 above will be omitted.

[0168] According to one embodiment of the present invention, referring to Table 4, the decoder can parse merge_subblock_flag and inter_affine_flag only if they satisfy cbWidth>=16, cbHeight>=16. As another embodiment, the encoder / decoder can use the subblock mode when the current block size is equal to the subblock size as well as when it is greater than the subblock size. This is because the motion compensation method of the subblock mode may be different from that of the non-subblock modes. In this specification, the subblock-based merge mode may be referred to as the subblock mode.

[0169] Table 5 below illustrates an example of inter prediction related syntax according to an embodiment of the present invention.

[0170] [Table 5A]

[0171] [Table 5B]

[0172] According to an embodiment of the present invention, the sub-block mode can be used when the width or height of the current block satisfies a specific condition. For example, even if the width or height of the current block is equal to the size of one side of the sub-block size, if the unequal width or height is larger than the size of one side (or the size of the other side) of the sub-block size, multiple sub-blocks may exist in the current block. Therefore, the sub-block mode can be used when the width of the current block is larger than the width of the sub-block or the height of the current block is larger than the height of the sub-block. Alternatively, the sub-block mode can be used when the width of the current block is larger than or equal to the width of the sub-block or the height of the current block is larger than or equal to the height of the sub-block. In Table 5, descriptions that overlap with Tables 3 and 4 above are omitted. Referring to Table 5, when the width of the current block is 8 or more or the height of the current block is 8 or more, the encoder / decoder can use the sub-block merge mode. Also, when the width of the current block is 16 or more or the height of the current block is 16 or more, the decoder can use the affine inter mode.

[0173] Tables 6 and 7 below illustrate sub-block merging related syntax according to an embodiment of the present invention.

[0174] [Table 6]

[0175] [Table 7]

[0176] According to an embodiment of the present invention, it may be signaled whether a particular mode is to be used in a first unit. Also, if the signaling indicates that it is to be used, the encoder may signal to the decoder which mode is to be actually used in a second unit belonging to the first unit. In this case, the signaling value in the first unit may determine whether or not to parse a syntax element in the second unit. For example, the first unit may be a sequence, a picture, a slice, a tile, etc., and the second unit may be a CTU, a CU, etc. In one embodiment, the particular mode may be a sub-block mode. Also, the first unit may be referred to as a higher unit, and the second unit may be referred to as a lower unit.

[0177] Table 6 shows a signaling parsing process for whether or not the sub-block mode is used when the sub-block sizes of the modes belonging to the sub-block mode are all the same or the difference in the sub-block size is not taken into consideration. According to an embodiment of the present invention, when there are a number of modes belonging to the sub-block mode and there are a number of use / non-use signalings of the upper units of the modes corresponding to the sub-block mode, if at least one of the use / non-use signalings of the upper units indicates that the sub-block mode is usable, the sub-block mode can be used in the lower unit. Also, the use / non-use signaling of the upper unit may correspond to a number of modes. For example, when there are different modes depending on the position of the neighbor block to be referred to, the use / non-use of the upper unit for a number of these modes may be signaled. Also, at this time, it may be applied in combination with conditions such as a block size condition. In this specification, the use of the sub-block mode may mean parsing the signaling indicating the use / non-use of the sub-block mode, and for this, refer to the above embodiment.

[0178] Referring to Table 6, the decoder can parse merge_subblock_flag only if at least one of sps_affine_enabled_flag and sps_sbtmvp_enabled_flag, which indicate whether the affine merge mode belonging to the subblock merge mode and SbTMVP can be used, is 1.

[0179] Referring to Table 7, if the sub-block sizes of the modes belonging to the sub-block mode may be different, the decoder may parse the syntax element indicating whether the sub-block mode is used or not, taking that into consideration. According to an embodiment of the present invention, when the signaling of whether or not the upper unit of a certain mode belonging to the sub-block mode is usable and the block size condition of the mode are all satisfied, the encoder / decoder may use the sub-block mode. For example, when there are a first mode and a second mode belonging to the sub-block mode, a first signaling indicating whether or not the upper unit corresponding to the first mode is usable and a second signaling indicating whether or not the upper unit corresponding to the second mode is usable are used, and there are a first block size condition of the first mode and a second block size condition of the second mode, if the first signaling indicating whether or not the upper unit is usable indicates that it is usable and the first block size condition is satisfied, the encoder / decoder may use the sub-block mode. And, if the second signaling indicating whether or not the upper unit is usable indicates that it is usable and the second block size condition is satisfied, the encoder / decoder may use the sub-block mode.

[0180] As an example, referring to Table 7, sps_affine_enabled_flag and sps_sbtmvp_enabled_flag, which are syntax elements indicating whether or not the affine merge mode and SbTMVP included in the subblock merge mode can be used, may be signaled from the encoder to the decoder. At this time, the decoder is configured to parse merge_subblock_flag if the condition of the following mathematical formula 15 is satisfied. Similarly, the encoder can syntax-signal merge_subblock_flag if the following mathematical formula 15 is satisfied.

[0181]

number

[0182] Moreover, Table 7 illustrates a case where the block size condition for the affine merge mode is (cbWidth>=8 && cbHeight>=8) and the block size condition for SbTMVP is (cbWidth>=16 && cbHeight>=16).

[0183] Table 8 below shows syntax that represents other examples in which the conditions in Table 7 are used.

[0184] [Table 8]

[0185] There are a first mode and a second mode belonging to the sub-block mode, there is a first signaling indicating whether an upper unit corresponding to the first mode is usable or not, and a second signaling indicating whether an upper unit corresponding to the second mode is usable or not, and there is a first block size condition 1 for the first mode and a block size condition 2 for the second mode. If the first block size condition is included in the second block size condition (i.e., if the first block size condition is satisfied, the second block size condition is satisfied), the encoder / decoder can use the sub-block mode as shown in the following mathematical formula 16.

[0186]

number

[0187] In another embodiment, there may be a higher unit usability signaling corresponding to all sub-block modes. Or, there may be a higher unit usability signaling corresponding to all sub-block merging modes. In this case, the decoder may parse the lower unit signaling only if the higher unit usability signaling indicates that it is usable. According to an embodiment of the present invention, the encoder / decoder may set a block size condition for using the sub-block mode according to the sub-block size. In this case, if the sub-block size is variable, the encoder / decoder may set a block size condition based on a larger size among possible sub-block sizes.

[0188] Tables 9 and 10 below show sub-block merging related syntax according to an embodiment of the present invention.

[0189] [Table 9]

[0190] [Table 10]

[0191] According to an embodiment of the present invention, whether or not a particular mode is available in a first unit may be signaled. Also, if the signaling indicates that a particular mode is available, the encoder may signal to a decoder which mode is actually used in a second unit belonging to the first unit. In this case, whether or not parsing for a syntax element in the second unit may be determined according to a signaling value in the first unit. For example, the first unit may be a sequence, a picture, a slice, a tile, etc., and the second unit may be a CTU, a CU, etc. In one embodiment, the particular mode may be a sub-block mode. Also, the first unit may be called an upper unit, and the second unit may be called a lower unit. Also, the upper unit availability signaling may correspond to a number of modes. For example, when there are different modes depending on the position of a neighboring block to be referred to, there may be upper unit availability signaling for a number of these modes.

[0192] According to an embodiment of the present invention, it is possible to determine whether or not to parse the candidate index by referring to the higher unit usability signaling. For example, if one or more of a plurality of higher unit usability signalings indicate that the index is usable, the candidate index can be parsed. As an embodiment, there may be higher unit usability signaling corresponding to a plurality of modes (wherein the mode may represent a unit included as a candidate in a candidate list). In this specification, the higher unit usability signaling corresponding to the plurality of modes may be indicated by a first flag, a second flag, a third flag, etc. Also, there may be higher unit usability signaling corresponding to one mode (candidate; a unit that may occupy one position in a candidate list). In this specification, the higher unit usability signaling corresponding to one mode may be referred to as an A flag, a B flag, or a C flag.

[0193] According to an embodiment of the present invention, when at least one of the upper unit usability signalings corresponding to a number of modes indicates that the candidate index is usable, the encoder / decoder can parse the candidate index. That is, when (first flag || second flag || third flag, ...) (i.e., when the flag value 1 indicates that the candidate index is usable), the encoder / decoder can parse the candidate index.

[0194] Also, if there is zero or one upper unit usability signaling corresponding to one mode, or there is zero or one mode without upper unit usability signaling, if at least one of the upper unit usability signaling corresponding to multiple modes indicates that it is usable, the encoder / decoder can parse the candidate index. That is, when (first flag || second flag || third flag, ...) (when flag value 1 indicates usable), the encoder / decoder can parse the candidate index.

[0195] If there is one upper unit usability signaling corresponding to one mode and one mode without an upper unit usability signaling, the encoder / decoder can parse the candidate index if the upper unit usability signaling corresponding to the one mode indicates that it is usable. At this time, it may not be related to the value of the upper unit usability signaling corresponding to the multiple modes. Therefore, in this case, if at least one of the multiple upper unit usability signaling corresponding to the multiple modes and the upper unit usability signaling corresponding to one mode indicates that it is usable, the encoder / decoder can parse the candidate index. That is, when (first flag || second flag || third flag, ... || A flag) (when flag value 1 indicates usable), the encoder / decoder can parse the candidate index.

[0196] If the candidate index is not parsed, the encoder / decoder can use the only candidate in the candidate list, or if the candidate index is not parsed, the encoder / decoder can infer the candidate index as 0.

[0197] Referring to Table 9, the affine merge mode and SbTMVP may correspond to the subblock merge mode, and the affine merge mode may be configured as multiple candidates from this method. In addition, the availability of the upper unit corresponding to the affine merge mode may be signaled by the sps_affine_enabled_flag syntax element. In this case, if sps_affine_enabled_flag indicates that it is available, the encoder / decoder can parse the candidate index merge_subblock_idx.

[0198] According to an embodiment of the present invention, if there are two or more upper unit usability signalings corresponding to one mode, if at least two of the upper unit usability signalings corresponding to one mode indicate that the mode is usable, the encoder / decoder can parse the candidate index. That is, when (A flag + B flag + C flag + ... >= 2) (if flag value 1 indicates usable), the encoder / decoder can parse the candidate index. In this case, there may not be a mode without upper unit usability signaling.

[0199] If there are two or more upper unit usability signalings corresponding to one mode and there are upper unit usability signalings corresponding to multiple modes, the encoder / decoder can parse the candidate index if at least two of the multiple upper unit usability signalings corresponding to one mode indicate that they are usable, or at least one of the multiple upper unit usability signalings corresponding to multiple modes indicates that they are usable. That is, when ((first flag || second flag || third flag ...) || (A flag + B flag + C flag + ... >= 2)) (when flag value 1 indicates usable), the encoder / decoder can parse the candidate index. In this case, there may not be a mode without upper unit usability signaling.

[0200] In the case where there is one mode without higher unit usability signaling, the candidate index can be parsed when at least one of the higher unit usability signalings corresponding to one mode is usable, not when at least two of the higher unit usability signalings corresponding to one mode are usable. That is, the encoder / decoder can parse the candidate index when (A flag || B flag || C flag + ...) instead of (A flag + B flag + C flag + ... >= 2).

[0201] Referring to Table 10, the sub-block merge mode may include affine merge mode, SbTMVP, and planar MVP. The affine merge mode may be configured as multiple candidates. SbTMVP may be configured as one candidate. Planar MVP may be configured as one candidate. In addition, the sps_affine_enabled_flag, sps_sbtmvp_enabled_flag, and sps_planar_enabled_flag syntax elements may be used to signal whether or not a higher-level unit corresponding to the affine merge mode, SbTMVP, or planar MVP is available. In this case, the encoder / decoder can parse the candidate index merge_subblock_idx when sps_affine_enabled_flag indicates that it is enabled, or when sps_sbtmvp_enabled_flag and sps_planar_enabled_flag both indicate that it is enabled (or when two or more of sps_sbtmvp_enabled_flag and sps_planar_enabled_flag are enabled). In the present invention, the indication that both sps_sbtmvp_enabled_flag and sps_planar_enabled_flag are enabled may be expressed as (sps_sbtmvp_enabled_flag+sps_planar_enabled_flag>=2).

[0202] The SbTMVP usable condition according to one embodiment of the present invention may be derived by the following method: Hereinafter, availableFlagSbCol represents a parameter (or a flag) indicating whether SbTMVP is usable.

[0203] - availableFlagSbCol may be set to 0 if one or more of the following conditions are true:

[0204] 1) When tile_group_temporal_mvp_enable_flag is 0.

[0205] 2) When sps_sbtmvp_flag is 0.

[0206] 3) If cbWidth is less than 8.

[0207] 4) If cbHeight is less than 8.

[0208] According to an embodiment of the present invention, whether SbTMVP is available may be represented by an availability flag. The availability flag indicating whether SbTMVP is available may be availableFlagSbCol. SbTMVP may also be called SbCol. When the availability flag is 1, it indicates that it is available, and when the availability flag is 0, it indicates that it is not available.

[0209] According to an embodiment of the present invention, there may be a condition in which SbTMVP cannot be used. As an embodiment, SbTMVP may be unusable based on the above-mentioned higher unit signaling. The higher unit for the current block or CU or PU may be a slice, a tile, a tile group, a sequence, a picture, a CTU, etc. The higher unit signaling may include tile_group_temporal_mvp_enable_flag. For example, tile_group_temporal_mvp_enable_flag represents a flag indicating whether a temporal motion vector (or a temporal motion vector prediction) is usable. Since SbTMVP may be a technology that uses a temporal motion vector, if the temporal MV usage is not set as usable, SbTMVP may be unusable. In addition, the higher unit signaling may include sps_sbtmvp_flag. For example, sps_sbtmvp_flag may be a flag indicating whether SbTMVP can be used.

[0210] In another embodiment, SbTMVP may be disabled based on a block size. The block size may include a block width, a block height, or a value based on a block width or height. The value based on a block width or height may include an area of ​​a block. If the block width or height is smaller than a threshold value, SbTMVP may be disabled. In one embodiment, the threshold value may be the width or height of a sub-block. For example, the threshold value may be set to 8.

[0211] If the above-mentioned conditions for not being able to use SbTMVP are met, the encoder / decoder can set the availability flag corresponding to SbTMVP to 0. If not, the encoder / decoder can perform additional operations to determine the availability flag.

[0212] 18 is a diagram illustrating a method of determining a temporary motion vector according to an embodiment of the present invention. In this specification, the motion vector representing the above-mentioned collocated block is called a temporary motion vector. The temporary motion vector is not limited to the name and can be called an initial motion vector, a temporal motion vector, a subblock-based merging base motion vector, or a base motion vector. That is, the encoder / decoder can determine a motion vector specifying the above-mentioned collocated block based on the initial motion vector, the temporal motion vector, the subblock-based merging base motion vector, or the base motion vector.

[0213] According to an embodiment of the present invention, an encoder / decoder may determine a temporary motion vector based on motion information of a position around a current block. The position around the current block may be a previously set position. The position around the current block may include a spatial candidate position of a merging candidate. Or, the position around the current block may include a spatial candidate position of an AMVP candidate.

[0214] Referring to FIG. 18, the positions corresponding to A0, A1, B0, B1, and B2 may be already set. Here, the positions may refer to luminance positions. When the top left corner position of the current block is (xCb, yCb), A0, A1, B0, B1, and B2 may be (xCb-1, yCb+cbHeight), (xCb-1, yCb+cbHeight-1), (xCb+cbWidth, yCb-1), (xCb+cbWidth-1, yCb-1), and (xCb-1, yCb-1), respectively. In this case, cbWidth and cbHeight may be the width and height of the current block, respectively. Also, (xCb, yCb) may be the top left corner position of the shared merge list area. When multiple blocks use the same merge candidate list, the multiple block areas may be the shared merge list area. Also, there may be a pre-defined lookup order corresponding to the pre-defined position. Table 11 below shows examples of the pre-defined lookup order.

[0215] [Table 11]

[0216] Referring to Table 11, the already set positions may be predefinedPosition1, predefinedPosition2, ..., predefinedPositionN, etc. Also, the reference order may already be set in the order of predefinedPosition1, predefinedPosition2, ..., predefinedPositionN. The reference order may be a forward or reverse direction of the reference order of the merge candidate list construction. Or, the reference order may be a forward or reverse direction of the reference order of the AMVP candidate list construction. For example, the reference positions and the reference order may be A1, B1, B0, A0. Or, predefinedPosition1, predefinedPosition2, ..., predefinedPositionN may represent motion information corresponding to the already set positions.

[0217] In Table 11, the temporary motion vector may be set to a zero motion vector. Also, if the motion information of the already set position is not available, the encoder / decoder may set the temporary motion vector to a zero motion vector. Also, if the motion information of the already set position is available, the motion information of the already set position may be set to the temporary motion vector. Also, the meaning of "available" may include a case where the corresponding position is not intra prediction. Or, the meaning of "available" may include a case where the corresponding position is inter prediction. Or, the meaning of "available" may include a case where a reference picture of the motion information of the corresponding position is the same as a co-located picture corresponding to the current block. Or, the meaning of "available" may include a case where a reference picture of the motion information of the corresponding position is the same as a reference picture corresponding to the current block.

[0218] In Table 11, it is checked whether the motion information of the first corresponding position that satisfies the first usability condition 1 among the already set positions satisfies the second usability condition, and if the second usability condition is satisfied, the encoder / decoder can set the motion information of the corresponding position to a temporary motion vector. Otherwise, the encoder / decoder can set the temporary motion vector to a zero motion vector. This can be expressed in pseudo code as shown in Table 12 below.

[0219] [Table 12]

[0220] In the above embodiment, in order to enable only the first position among the already set positions that satisfy the first usability condition as a temporary motion vector, an else if statement may be used in Table 12 above. Also, the first usability condition and the second usability condition may mean that the position is usable as described above. For example, the first usability condition may mean that the corresponding position uses inter prediction. Also, the second usability condition may mean that the reference picture of the motion information of the corresponding position is the same as the co-located picture corresponding to the current block. In the above embodiment, if a certain position satisfies the first usability condition but does not satisfy the second usability condition, it is not necessary to check the positions referenced after the certain position.

[0221] In another embodiment, the encoder / decoder may set the motion information of the first corresponding position among the already set positions that satisfies both the first and second usable conditions as a temporary motion vector. Otherwise, the encoder / decoder may set the temporary motion vector to a zero motion vector. This may be represented in pseudo code as shown in Table 13 below.

[0222] [Table 13]

[0223] In Table 13, if a certain position satisfies the first usable condition but does not satisfy the second usable condition, the encoder / decoder can also check positions referenced after the certain position.

[0224] Also, according to an embodiment of the present invention, there may be only one previously set position. This case may be represented in pseudo code as shown in Table 14 below.

[0225] [Table 14]

[0226] That is, if motion information corresponding to one pre-defined position is available, the encoder / decoder may set a temporary motion vector to the motion information, and if not, may set the temporary motion vector to a zero motion vector. As an example, the one pre-defined position may be a left position of the current block. That is, the x coordinate of the one pre-defined position may be smaller than the left coordinate within the current block. For example, the one pre-defined position may be position A1.

[0227] In the above embodiment, the first and second available conditions are used as the available conditions, but the present invention is not limited thereto and may include a case where more available conditions exist. In addition, the motion information may include a motion vector (mvLX), a reference index (refIdxLX), a prediction utilization flag (predFlagLX), etc.

[0228] Also, in one embodiment, the process of checking the second availability condition and setting the temporary motion vector may be performed as follows: That is, if availableFlagN is true, the following process may be applied.

[0229] A. If all of the following conditions are true, then the temporary motion vector may be set to mvL1N:

[0230] a.predFlagL1N is 1

[0231] b. DiffPicOrderCnt(ColPic, RefPicList1[refIdxL1N]) is 0

[0232] c.DiffPicOrderCnt(aPic, currPic) is less than or equal to 0, then aPic represents all pictures in all reference picture lists of the slice (or tile group).

[0233] d.tile_group_type is equal to B

[0234] If e.collocated_from_l0_flag is equal to 0

[0235] B. Otherwise, the temporary motion vector may be set to mvL0N if both of the following conditions are true:

[0236] a.predFlagL0N is 1

[0237] b. DiffPicOrderCnt(ColPic,RefPicList0[refIdxL0N]) is 0

[0238] Here, availableFlagN being true may indicate that the first availability condition is satisfied. If, for example, only one pre-defined position is used as described above, a parameter that can be availableFlagN may be the availableFlag corresponding to the pre-defined position. For example, if only the A1 position is used, availableFlagN may be availableFlagA1, which may be a value indicating whether motion information of the A1 position is available.

[0239] 19 is a diagram illustrating whether or not SbTMVP is used according to an embodiment of the present invention. According to an embodiment of the present invention, the encoder / decoder can determine whether or not to use a specific mode based on the position of the current block 1901. According to yet another embodiment of the present invention, the encoder / decoder can determine whether or not to use a specific mode based on the position of the merge list sharing area to which the current block 1901 belongs.

[0240] According to an embodiment of the present invention, the encoder / decoder may determine whether to use SbTMVP based on the position of the current block 1901. According to yet another embodiment of the present invention, the encoder / decoder may determine whether to use SbTMVP based on the position of the merge list sharing area to which the current block 1901 belongs. As an embodiment, the position of the current block 1901 or the position of the merge list sharing area to which the current block 1901 belongs may be expressed as the upper left corner coordinate or based on the upper left corner coordinate. Also, the position of the current block 1901 or the position of the merge list sharing area to which the current block 1901 belongs may be a relative position within a higher group. The higher group may include a picture, a tile, a tile group, a CTU, a slice, etc. Also, the higher group may be a group to which the current block belongs. That is, for example, the encoder / decoder may determine whether to use SbTMVP based on the position of the current block 1901 in the picture 1902. In this specification, the current block or the merge list sharing area of ​​the current block may be referred to as a current area.

[0241] According to an embodiment of the present invention, the encoder / decoder may determine whether to use SbTMVP based on whether the current region is adjacent to the boundary of the upper group. For example, if the current region is adjacent to the boundary of the upper group, it may be determined and determined not to use SbTMVP. Alternatively, it may be determined whether to use SbTMVP based on whether the already set positions referenced to determine the temporary motion vector corresponding to the current region described in FIG. 19 are out of the upper group. For example, if all the already set positions referenced to determine the temporary motion vector corresponding to the current region are out of the upper group, the encoder / decoder may determine and determine not to use SbTMVP. Also, whether all the already set positions referenced to determine the temporary motion vector corresponding to the current region are out of the upper group may be determined based on the coordinates of the current region. This is because, according to the above-mentioned embodiment, if all the already set positions referenced to determine the temporary motion vector corresponding to the current region are out of the upper group, a zero motion vector is used as the temporary motion vector, which may cause a decrease in the prediction performance of SbTMVP. Also, in one embodiment, by determining that SbTMVP cannot be used in this case, other modes signaled in the same syntax element as SbTMVP can be signaled using fewer bits. For example, as described above, SbTMVP and affine merge mode may be included in the sub-block merge mode, and if it is determined that SbTMVP cannot be used, the affine merge mode can be signaled using fewer indices. Thus, in such a case, index 0 can always indicate the affine merge mode.

[0242] Referring to FIG. 19, as described above, only the A1 position 1903 can be referenced to determine the temporary motion vector of the current region. If the motion information of the A1 position 1903 is not available, a zero motion vector can be used as the temporary motion vector. If the current region abuts the left boundary of the picture 1902, the temporary motion vector may always be a zero motion vector. Therefore, if the current region abuts the left boundary of the picture 1902, it may be determined that SbTMVP is not used. Also, if the current region abuts not only the left boundary of the picture 1902 but also the left boundary of a unit that can be processed in parallel, it may be determined that SbTMVP is not used. The unit that can be processed in parallel may be a tile, a tile group, a slice, etc. Therefore, if the top left corner coordinate of the current region is (xCb, yCb), if xCb is 0, the encoder / decoder may abut the left boundary of the picture 1902, and in this case, it may be determined that SbTMVP is not used. Alternatively, if (xCb-(left coordinate of the parallel processable unit)) is 0, SbTMVP may not be used.

[0243] Also, in one embodiment, if only the top edge of the current region is referenced to determine the temporary motion vector for the current region, the encoder / decoder can be configured not to use SbTMVP when the current region contacts the upper boundary of the picture 1902 or a unit that can be processed in parallel.

[0244] In another additional embodiment, the conditions for determining and judging not to use SbTMVP have been described. In addition to these conditions, SbTMVP may not be used only when a condition related to the block size is satisfied. That is, for example, SbTMVP may not be used when the coordinates of the current region satisfy the already set conditions and the condition related to the block size is satisfied. In one embodiment, the condition related to the block size may be when the block size of the current block is equal to or smaller than the subblock size. If the current block size is equal to or smaller than the subblock size, the current block has only one subblock, and in this case, the advantages and performance of SbTMVP may be reduced. For example, the subblock size may be 8×8. Therefore, the following embodiment may be derived from the described embodiment. If the current region is adjacent to the left boundary of the picture and the current block is equal to or smaller than 8×8, the encoder / decoder may determine and judge not to use SbTMVP. However, there may be an embodiment in which the subblock merge mode can be used only for blocks equal to or larger than 8×8. In this case, if the current region abuts the left boundary of the picture 1902 and the current block 1901 is 8x8, it can be determined and decided not to use SbTMVP.

[0245] Also, in another embodiment of the present invention, the availability flag indicating the condition for SbTMVP use may include, in addition to the above-mentioned conditions, whether xCb is 0 and the width and height of the current block are 8. That is, if xCb is 0 and the width and height of the current block are 8, the availability flag may be set to 0. If the above condition and the above-mentioned conditions are all satisfied, or if any one or more of them are satisfied, the encoder / decoder may set the availability flag to 0. As described above, the availability flag may indicate whether SbTMVP is available. Also, when a certain condition is satisfied, it may be indicated that SbTMVP is unavailable. That is, when a certain condition is satisfied, the encoder / decoder may set availableFlagSbCol to 0. Therefore, according to the embodiment described in FIG. 19, the encoder / decoder may set availableFlagSbCol based on the position of the current region. That is, the encoder / decoder may set availableFlagSbCol based on the coordinates (xCb, yCb) representing the current region. For example, (xCb, yCb) may be the top-left coordinates of the current region.

[0246] As described in FIG. 19, if the current region or current block borders the left border of the picture, SbTMVP may be unavailable. Thus, if xCb is 0, the encoder / decoder can set ableFlagSbCol to 0. For example, if the current region or current block borders the left border of the picture and the current block is 8×8, SbTMVP may be unavailable. Thus, if xCb is 0 and the width and height of the current block are both 8, the encoder / decoder can set ableFlagSbCol to 0.

[0247] Table 15 below illustrates a sub-block merge mode syntax structure according to one embodiment of the present invention.

[0248] [Table 15]

[0249] As described above, the sub-block merging mode (or sub-block-based merging mode) may include SbTMVP and / or affine merging mode. Also, a syntax element (or flag) sps_affine_enabled_flag indicating whether the affine mode is enabled or disabled may be signaled from the encoder to the decoder using higher level syntax. In one embodiment, the higher level may be at least one of the sequence, picture, slice, or CTU level. Also, in one embodiment, a condition under which SbTMVP cannot be used (or applied) may be predefined, as described in FIG. 19 above.

[0250] According to an embodiment of the present invention, if the SbTMVP unusable condition is met and the affine mode is not available, the encoder / decoder may not parse merge_subblock_flag. For example, merge_subblock_flag may be signaling indicating the use of the subblock merging mode. If merge_subblock_flag is not present, the encoder / decoder may infer its value as 0. If the affine mode is not available, sps_affine_enabled_flag may be 0. Thus, merge_subblock_flag can be parsed when SbTMVP is available or when the affine mode is available. In addition, the conditions for parsing merge_subblock_flag may include further conditions in addition to the above conditions. For example, referring to Table 15, in order to determine the parsing of merge_subblock_flag, a condition related to MaxNumSubblockMergeCand indicating the maximum number of subblock merging candidates and a condition related to the block size may both be considered.

[0251] Referring to Table 15, if x0, which is the x value of the top-left coordinate of the current block, is 0 and sps_affine_enabled_flag, which indicates whether the affine mode is enabled, is 0, merge_subblock_flag does not need to be parsed. Also, if x0 is not 0 or sps_affine_enabled_flag is 1, merge_subblock_flag can be parsed.

[0252] In the above embodiment, the subblock merge mode includes SbTMVP and affine merge mode. However, the present invention is not limited to this, and the subblock merge mode may include other modes. For example, the subblock merge mode may include the first mode and the second mode in addition to SbTMVP. Also, syntax elements sps_mode1_enabled_flag and sps_mode2_enabled_flag indicating whether the first mode and the second mode are available may be signaled. In this case, if the SbTMVP unusable condition is met and neither the first mode nor the second mode is available, the encoder / decoder may not parse merge_subblock_flag. In this case, the encoder / decoder may infer that merge_subblock_flag is 0. As an example, if the SbTMVP unusable condition is not met, the first mode is available, or the second mode is available, the encoder / decoder may parse merge_subblock_flag. In addition, the conditions for parsing merge_subblock_flag may include other conditions in addition to the above conditions. Referring to Table 15, in order to determine the parsing of merge_subblock_flag, conditions related to MaxNumSubblockMergeCand indicating the maximum number of subblock merge candidates and conditions related to block size may both be considered.

[0253] Also, according to one embodiment of the present invention, if affine mode is not enabled, the encoder / decoder can set MaxNumSubblockMergeCand to 1. Alternatively, if affine mode is not enabled and SbTMVP is enabled, the encoder / decoder can set MaxNumSubblockMergeCand to 1. Alternatively, if affine mode is not enabled and SbTMVP is not enabled, the encoder / decoder can set MaxNumSubblockMergeCand to 0.

[0254] 20 is a diagram illustrating an SbTMVP induction method according to an embodiment of the present invention. According to an embodiment of the present invention, a previously set position to be referenced for determining a temporary motion vector may be variable. Alternatively, a previously set position to be referenced for determining a temporary motion vector for a specific block may be set to one, and the position may be variable.

[0255] In one embodiment, a reference position for determining a temporary motion vector may be variably determined based on the position of the current block 2001 or the current region. The current region represents the region described in FIG. 19 above. For example, if the current region borders on a boundary of the picture 2002 or a boundary of a unit that can be processed in parallel, motion information at a specific position may be always unavailable. Therefore, the encoder / decoder may set a position other than the always unavailable position as a previously set position to be referenced for determining a temporary motion vector. That is, the previously set position to be referenced for determining a temporary motion vector may be set to a position that is not always unavailable based on the position of the current region. The always unavailable position may be a position outside the picture 2002 or a range that can be processed in parallel.

[0256] According to the embodiment described in FIG. 18 above, the already-set position referenced to determine the temporary motion vector may be defined as the left position of the current block 2001 or the current region. In this case, as described in FIG. 19 above, when the current block 2001 or the current region is in a specific position, the motion information of the already-set position is unavailable, so such already-set position can be set (or determined) to a position other than the left position. For example, the already-set position may be set to the upper position 2003 of the current block 2001 or the current region. For example, the already-set position may be the B1 position in FIG. 18(a).

[0257] 20, in one embodiment, when a current block 2001 or a current region is adjacent to the left boundary of a picture 2002 or the left boundary of a parallel processable unit, a previously set position referenced to determine a temporary motion vector may be determined to be an upper position of the current block 2001 or the current region. The upper position of the current region may refer to a position whose y coordinate is relatively smaller than the y coordinate (i.e., vertical coordinate) of the current region. For example, when a current block 2001 or a current region is adjacent to the left boundary of a picture 2002 or the left boundary of a parallel processable unit, a previously set position referenced to determine a temporary motion vector may be position B1 of FIG. 18(a).

[0258] Alternatively, in one embodiment, if the current block 2001 or the current region is adjacent to the upper boundary of the picture 2002 or the upper boundary of the parallel processable unit, the previously set position referenced to determine the temporary motion vector may be determined to be the left position of the current block 2001 or the current region. The left position of the current region may refer to a position whose x coordinate is relatively smaller than the x coordinate (i.e., horizontal coordinate) of the current region. For example, if the current block 2001 or the current region is adjacent to the upper boundary of the picture 2002 or the upper boundary of the parallel processable unit, the previously set position referenced to determine the temporary motion vector may be position A1 in FIG. 18(a).

[0259] The sub-block based motion compensation method has been described above. In the following, a motion compensation method based on a merge mode with motion vector difference (MMVD) (or merge MVD) and a method of combining and applying the above-mentioned embodiment of the sub-block based motion compensation will be described.

[0260] FIG. 21 is a diagram illustrating a method of applying MMVD according to an embodiment of the present invention. According to an embodiment of the present invention, an encoder / decoder may determine a motion vector (MV) based on a motion vector predictor (MVP) and a motion vector difference (MVD). In this specification, the MVP may be referred to as a base motion vector (baseMV). That is, the encoder / decoder may derive a motion vector (i.e., a final motion vector) by adding a motion vector difference to a base motion vector. However, the present invention is not limited to such names, and the MVP may be referred to as a base motion vector, a temporary motion vector, an initial motion vector, an MMVD candidate motion vector, etc. The MVD may be expressed as a value that refines the MVP, and may be referred to as an improved motion vector (refineMV) or a merged motion vector difference.

[0261] According to an embodiment of the present invention, when MMVD is applied, i.e., in the MMVD mode, the MV may be determined based on a base motion vector, a distance parameter (or variable), and a direction parameter (or variable). According to an embodiment of the present invention, the base motion vector may be determined from a candidate list. For example, the base motion vector may be determined from a merge candidate list. The encoder / decoder may determine the base motion vector from a part of another candidate list. The part of the candidate list may be a front part (a part with a smaller index) of the candidate list. For example, the encoder / decoder may determine the base motion vector using the first and second candidates of the merge candidate list. To this end, a candidate index indicating a specific candidate of the two candidates may be signaled from the encoder to the decoder. Referring to FIG. 21, a base candidate index, which is an index for signaling the base motion vector, may be defined. The encoder / decoder may determine a candidate to be applied to the current block from the candidates of the candidate list according to the base candidate index, and determine the motion vector of the determined candidate as the base motion vector. In the present invention, the base candidate index is not limited to the name, and the base candidate index can be called a base candidate flag, a candidate index, a candidate flag, an MMVD index, an MMVD candidate index, an MMVD candidate flag, etc.

[0262] Also, according to an embodiment of the present invention, an MVD different from the MVD described in FIG. 6 and FIG. 7 may exist. For example, the MVD in the MMVD may be defined differently from the MVD described in FIG. 6 and FIG. 7. In this specification, the MMVD may represent a merge mode (i.e., a motion compensation mode or method) using a motion vector difference, or may represent a motion vector difference when the MMVD is applied. For example, the encoder / decoder may determine whether or not the MMVD is applied (or whether or not it is used). If the MMVD is applied, the encoder / decoder may determine a motion vector of the current block by deriving a merge candidate used for inter prediction of the current block from a merge candidate list, and deriving the MMVD and applying (or adding) it to the motion vector of the merge candidate.

[0263] In one embodiment, the other MVD may refer to a simplified MVD, an MVD with another (or smaller) resolution, a MVD with a smaller number of available MVDs, an MVD with a different signaling method, etc. For example, the MVD used in the existing AMVP, affine inter mode, etc. described in FIG. 6 and FIG. 7 can represent all regions in the x and y axes (i.e., horizontal and vertical directions) for a specific signaling unit (e.g., x-pel), for example, a region based on a picture (e.g., a picture region or a region including a picture and a surrounding region) at uniform intervals, whereas the unit representing the specific signaling unit of the MMVD may be relatively limited. Also, the region (or unit) signaling the MMVD may not have uniform intervals. Also, the MMVD can only indicate a specific direction for a specific signaling unit.

[0264] Also, according to an embodiment of the present invention, the MMVD may be determined based on the distance and the direction. Referring to FIG. 21, the distance and the direction of the MMVD may be set in advance according to a distance index indicating the distance of the MMVD and a direction index indicating the direction of the MMVD. In one embodiment, the distance may indicate the MMVD size in a specific pixel unit (e.g., absolute value), and the direction may indicate the direction of the MMVD. Also, the encoder / decoder may signal a relatively small distance with a relatively small index. That is, in the case of not using signaling with fixed length binarization, the encoder / decoder may signal a relatively small distance with relatively few bits.

[0265] Also, according to an embodiment of the present invention, the MVD may use a signaled MV or an MV based on the signaled MV. For example, the MV based on the signaled MV may be the signaled MV with its sign reversed. For example, the MVD signaling is based on a value corresponding to a reference list, and a value corresponding to a reference list different from the certain reference list may use the value corresponding to the certain reference list (i.e., the signaled MVD) as is or with its sign changed. Whether to use the MVD as is or with its sign changed may be determined according to a picture order count (POC) relationship between the current picture and a reference picture in a reference list and a POC relationship between the current picture and a reference picture in a reference list different from the certain reference list. More specifically, even when both reference lists L0 and L1 are used, only one MVD may be signaled. For example, the MVD corresponding to L0 may be signaled. And the MVD corresponding to L1 may be determined based on the MVD corresponding to L0.

[0266] For example, the MVD corresponding to L1 may remain the MVD corresponding to L0, or may be a value obtained by changing the sign of the MVD corresponding to L0. This may also be determined according to the POC relationship between the current picture and the L0 reference picture and the POC relationship between the current picture and the L1 reference picture. For example, depending on whether the value of DiffPicOrderCnt(RefPicList0[refIdxLN0], currPic)*DiffPicOrderCnt(currPic,RefPicList1[refIdxLN1]) is greater than or less than 0, it can be determined whether the MVD corresponding to L0 is used as is for L1 or is modified. Also, DiffPicOrderCnt(RefPicList0[refIdxLN0], currPic) * DiffPicOrderCnt(currPic, RefPicList1[refIdxLN1]) being greater than 0 may mean that both the L0 reference picture and the L1 reference picture are temporally earlier than the current picture or both are temporally later than the current picture.

[0267] Therefore, in this case, the signs of L0MVD and L1MVD may be the same. Also, DiffPicOrderCnt(RefPicList0[refIdxLN0], currPic)*DiffPicOrderCnt(currPic,RefPicList1[refIdxLN1]) being smaller than 0 may mean that one of the L0 reference picture and the L1 reference picture is temporally earlier than the current picture, and the remaining one is temporally later than the current picture. Therefore, in this case, the signs of L0MVD and L1MVD may be different. Also, being temporally earlier may mean that the POC is smaller, and being temporally later may mean that the POC is larger. Also, in the above-described embodiment, an MV scaling process may be added. That is, a process of MV scaling a signaled MV or an MV obtained by modifying the signaled MV (e.g., an MV with an inverted sign) may be added.

[0268] Table 16 below illustrates an example MMVD syntax structure according to one embodiment of the present invention.

[0269] [Table 16]

[0270] According to one embodiment of the present invention, there may be signaling indicating whether or not the MMVD is used (or applied) as described in FIG. 21 above. With reference to Table 16, the MMVD flag (i.e., mmvd_flag) represents a syntax element indicating whether or not the MMVD is used. As an example, the MMVD may be applied to skip mode or merge mode. In one embodiment, when the MMVD is used, the decoder may parse the MMVD-related syntax. When the MMVD is used, the MMVD-related information may be further parsed from the MMVD index coding (i.e., mmvd_idx_coding) syntax.

[0271] In one embodiment, if the MMVD is not used, the decoder may parse the merge index. Referring to Table 16, the merge index (i.e., merge_idx) represents a syntax element that indicates a particular merge candidate in the merge candidate list. In addition, in the case of a merge mode other than the skip mode, after parsing the merge flag (i.e., the merge flag), if the merge flag is 1, the MMVD flag may be parsed. The merge flag may indicate whether or not the merge mode or the sub-block merge mode is used (or whether or not it is applied). In addition, the merge flag may indicate that the AMVP, inter mode, affine inter mode, etc. are not used. In this embodiment, the merge index may be an index different from the base candidate index described in FIG. 21 above.

[0272] Also, according to an embodiment of the present invention, there may be signaling indicating the maximum number of merge candidates. Also, the signaling indicating the maximum number of merge candidates may be performed in a unit larger than a CU or a prediction unit. For example, there may be signaling indicating the maximum number of merge candidates in slice or tile units. Also, if a condition based on the signaling indicating the maximum number of merge candidates is satisfied, the decoder may parse the merge index. As an embodiment, if a value of a syntax element indicating the maximum number of merge candidates indicates that the maximum number of merge candidates is 1, the encoder / decoder may infer that the merge index is 0. That is, if the signaling indicating the maximum number of merge candidates indicates that the maximum number of merge candidates is 1, a candidate may be determined without parsing the merge index.

[0273] Also, referring to Table 16, MaxNumMergeCand, which is a value (variable, parameter) based on signaling indicating the maximum number of merging candidates, may be defined. MaxNumMergeCand may mean the maximum number of merging candidates. MaxNumMergeCand may have a value of 1 or more. Also, based on the maximum number of supportable merging candidates and signaling indicating the maximum number of merging candidates, the maximum number of merging candidates for the signaling unit indicating the maximum number of merging candidates may be determined. For example, the encoder / decoder may determine the maximum number of merging candidates for the signaling unit indicating the maximum number of merging candidates by subtracting the signaling value indicating the maximum number of merging candidates from the maximum number of supportable merging candidates. Also, although the maximum number of merging candidates has been described, the maximum number of sub-block merging candidates and maximum number signaling may exist in a substantially identical manner.

[0274] Also, in one embodiment, the merge index may be parsed when using skip mode or merge mode. The decoder may parse the merge index when the condition for using skip mode or merge mode is met. For example, when not using subblock merge mode, the encoder / decoder may use skip mode or merge mode. In Table 16 above, the merge index may be parsed after MMVD flag parsing. This may be to consider MMVD in priority over the existing skip mode or merge mode. Also, a subblock merge flag (i.e., subblock_merge_flag) (or a merge subblock flag (i.e., merge_subblock_flag) syntax element may be signaled to indicate whether or not to use the subblock merge mode.

[0275] FIG. 22 is a diagram illustrating an MMVD-related syntax structure according to an embodiment of the present invention. The embodiment described in Table 16 above may be applied to FIG. 22 in the same manner, and the related overlapping description will be omitted. The syntax shown in FIG. 22 further includes a process related to sub-block merging in addition to Table 16. In FIG. 22 and the drawings described below (i.e., FIG. 23 to FIG. 31), the intra prediction process, and the inter mode, AMVP mode, affine inter mode, etc. of the inter prediction process may be omitted. Although FIG. 22 mainly describes a decoder for convenience of description, the coding process according to this embodiment may be applied to an encoder in substantially the same manner.

[0276] According to an embodiment of the present invention, the decoder may parse the sub-block merge flag before the MMVD flag and / or the merge index. This is for the purpose of considering the sub-block merge mode preferentially over at least one of the skip mode, the merge mode, and / or the MMVD. The decoder may parse the MMVD flag and / or the merge index, etc., when the sub-block merge mode is not used. As an example, the decoder may parse the sub-block merge flag before the MMVD flag, and may parse the MMVD flag when the sub-block merge mode is not applied. In the sub-block merge mode, motion compensation is performed in units of sub-blocks having a relatively small size, so that the accuracy of motion prediction may be high, and improvement of the MV using one MVD may not be effective due to the characteristics of the motion compensation based on the sub-block. Therefore, according to an embodiment of the present invention, compression efficiency may be improved by checking whether or not the MMVD is applied only when the sub-block merge mode is not applied, and such an object may be achieved by setting the parsing order of the syntax (or syntax elements) (i.e., by parsing the sub-block merge flag before the MMVD flag).

[0277] FIG. 23 is a diagram illustrating an example of an MMVD-related syntax structure according to an embodiment of the present invention. The syntax (or function) illustrated in FIG. 23 may represent a syntax for parsing MMVD information shown in Table 16, FIG. 22, and the MMVD-related syntax described below. As described in FIG. 21 above, the MMVD may be determined based on a base motion vector, a distance, and a direction. Then, an index (or syntax element) for signaling such information may be defined. Referring to FIG. 23, syntax elements for signaling the base motion vector, the distance, and the direction may be defined as base_mv_idx, distance_idx, and direction_idx. As described in Table 16 and FIG. 22, when it is determined to use the MMVD, the syntax element of FIG. 23 indicating a specific MMVD value may be signaled from the encoder to the decoder. Although FIG. 23 focuses on the decoder for convenience of explanation, the coding process according to this embodiment may be applied to the encoder in substantially the same manner.

[0278] FIG. 24 is a diagram illustrating an example of an MMVD-related syntax structure according to an embodiment of the present invention. The syntax (or function) illustrated in FIG. 24 may represent syntax for parsing MMVD information shown in Table 16, FIG. 22, and the MMVD-related syntax described below. According to an embodiment of the present invention, some of the MMVD-related syntax may be parsed if a predefined specific condition is met. For example, after it is determined that the MMVD is to be used, if the predefined specific condition is met, the decoder may parse some of the MMVD-related syntax. In one embodiment, some of the MMVD-related syntax may be base motion vector-related signaling. The determination to use the MMVD may be based on an MMVD flag value. It may be determined to use the MMVD by parsing the MMVD flag or when the MMVD flag is inferred to indicate a specific value (e.g., 1). Although FIG. 24 focuses on a decoder for convenience of explanation, the coding process according to this embodiment may be applied to an encoder in substantially the same manner.

[0279] Also, according to an embodiment of the present invention, the specific condition may be related to the maximum number of possible base motion vector candidates. For example, the decoder may parse the base motion vector related syntax if the maximum number of possible base motion vector candidates is 2 or more, and may not parse the base motion vector related syntax if the maximum number of possible base motion vector candidates is 1. When the base motion vector related syntax does not exist, the decoder may infer its value. In this case, the base motion vector related syntax may be inferred as 0. If the base motion vector candidate is a merge candidate, the decoder may determine whether to parse the base motion vector related syntax according to a condition based on the maximum number of possible merge candidates.

[0280] As described above, the maximum number of possible base motion vector candidates may be a value indicated by the signaling indicating the maximum number of merge candidates described in Table 16, which may be expressed as MaxNumMergeCand.

[0281] Referring to Figure 24, the decoder can parse the base motion vector index only if MaxNumMergeCand, which represents the maximum number of merge candidates, is greater than 1. If the base motion vector index does not exist, e.g., because it has not been parsed, the decoder can infer its value as 0. If the maximum number of possible candidates is 1, the index can be determined without signaling it.

[0282] FIG. 25 is a diagram illustrating an example of an MMVD syntax according to an embodiment of the present invention. In the case of the syntax structure described in FIG. 22, even if the MMVD is not used and the skip mode or merge mode is used, the MMVD flag is always parsed. In addition, the base motion vector index or the merge index is parsed both when the MMVD is used and when it is not used. Meanwhile, in the embodiment of FIG. 25, even if the MMVD is not used and the skip mode or merge mode is used, the MMVD flag may not be parsed. Although FIG. 25 mainly describes a decoder for convenience of explanation, the coding process according to this embodiment may be applied to an encoder in substantially the same manner.

[0283] According to an embodiment of the present invention, the base motion vector related syntax and the merge index may be defined as the same value. For example, the base motion vector index and the merge index may be the same. That is, signaling for indicating a corresponding candidate may be performed using one syntax element. Also, according to an embodiment of the present invention, the decoder may parse the base motion vector related syntax and parse the MMVD flag conditionally. Then, when the MMVD flag indicates the use of MMVD, the decoder may parse MMVD related syntax other than the base motion vector related syntax.

[0284] For example, after parsing the merge index, it is possible to distinguish whether the corresponding candidate is a candidate available for MMVD. If the candidate is available for MMVD, the MMVD flag may be parsed, and if the candidate is unavailable for MMVD, the MMVD flag may not be parsed. Also, if the MMVD flag does not exist, it may be inferred that the MMVD is not used. For example, when the base motion vector of the MMVD can match up to num_mmvd_baseCand of the candidates at the front side (smaller index) of the merge candidate list, if the parsed index is smaller than num_mmvd_baseCand, the MMVD flag is parsed, and if not, it is determined that the MMVD flag is not parsed and the MMVD is not used. In this case, when using a candidate index equal to or larger than num_mmvd_baseCand, there is an advantage that bits for the MMVD flag can be saved.

[0285] 25, after parsing the merge index, if the merge index is smaller than num_mmvd_baseCand, the MMVD flag is parsed. The num_mmvd_baseCand may be the number of possible candidates for the base motion vector of the MMVD. The base motion vector index of the MMVD may be determined based on the merge index.

[0286] FIG. 26 is a diagram illustrating an MMVD syntax according to an embodiment of the present invention. Referring to FIG. 26, in the syntax structure, the MMVD flag may be higher than the merge flag. For example, the decoder may determine whether to parse the merge flag based on the MMVD flag. As an example, this is applicable when not in skip mode. Although FIG. 26 focuses on the decoder for convenience of explanation, the coding process according to this embodiment may be applied to the encoder in substantially the same manner.

[0287] According to an embodiment of the present invention, if the MMVD flag indicates that the MMVD is used, the merge flag may not be parsed. For example, if a base motion vector candidate for the MMVD is determined from among the merge candidates and the MMVD flag indicates that the MMVD is used, the decoder may not need to parse the merge flag. If the MMVD is used (if the MMVD flag is 1), the merge flag can be inferred to indicate that the merge mode is used. Referring to FIG. 26, if the skip mode is used, i.e., if cu_skip_flag is 1, the decoder can infer that the merge flag is 1 regardless of whether the MMVD is used or not. Also, if the merge flag does not exist and the skip mode is not used, the decoder can infer that the merge flag is 1 when the MMVD is used, and the merge flag is 0 when the MMVD is not used. Or, in one embodiment, the decoder can infer that the merge flag is 1 when the merge flag does not exist and the skip mode is not used. Also, referring to FIG. 26, when not in skip mode, the decoder parses the MMVD-related syntax if the MMVD flag is 1, and parses the merge flag if the MMVD flag is not 0.

[0288] According to an embodiment of the present invention, when the sub-block merge flag is higher than the merge flag, if the sub-block merge flag is 1, the merge flag does not need to be parsed and the merge flag can be inferred to be 1. Also, when the sub-block merge flag is higher than the MMVD flag, if the sub-block merge flag is 1, the MMVD flag does not need to be parsed and the MMVD flag can be inferred to be 0. Also, in this specification, modeX_flag being 1 can mean that modeX is used, and modeX_flag being 0 can mean that modeX is not used.

[0289] FIG. 27 is a diagram illustrating an example of MMVD syntax according to an embodiment of the present invention. According to an embodiment of the present invention, in a candidate list used as a base motion vector candidate of the MMVD, the MMVD can always be used for some candidates. For example, when a base motion vector candidate of the MMVD is determined from a merging candidate list, the MMVD can always be used for some of the merging candidate list. For example, the MMVD can always be used for a determined candidate index. For example, when a candidate index is smaller than a previously set value, the MMVD can always be used. In this case, it is possible to determine whether or not the MMVD is used from the candidate index. Also, the MMVD flag parsing may not exist.

[0290] For example, after parsing the merge index, if the merge index corresponds to a value that is determined to use the MMVD, it can be determined to use the MMVD. In this case, MMVD-related syntax (e.g., distance, direction signaling, etc.) can be parsed. For convenience of explanation, FIG. 27 mainly describes a decoder, but the coding process according to this embodiment can be applied to an encoder in substantially the same manner. Referring to FIG. 27, the MMVD can always be used up to the num_mmvd_baseCand number of candidates at the front of the merge candidate list. In this case, if the candidate index is smaller than num_mmvd_baseCand, it can be determined to use the MMVD, and the MMVD-related syntax can be parsed.

[0291] FIG. 28 is a diagram illustrating an MMVD syntax according to an embodiment of the present invention. According to an embodiment of the present invention, a decoder can always use the MMVD for some candidates in a candidate list used as a base motion vector candidate of the MMVD. For example, when a base motion vector candidate of the MMVD is determined from a merge candidate list, the decoder can always use the MMVD for some of the merge candidate list. For example, the decoder can always use the MMVD for a determined candidate index. For example, when a candidate index is smaller than a previously set value, the decoder can always use the MMVD. Although FIG. 27 focuses on a decoder for convenience of explanation, the coding process according to this embodiment may be applied to an encoder in substantially the same manner.

[0292] According to an embodiment of the present invention, in the syntax structure (or in the syntax parsing order), the MMVD flag may be higher than the merge flag. In this case, if the MMVD flag is 1, the decoder can parse the MMVD-related syntax. Also, if the MMVD flag is 1, the decoder can infer that the merge flag is 1. Also, if the MMVD flag is 0, the decoder can parse the merge flag. If the merge flag is 1, the decoder can parse the merge index. At this time, there may be an additional condition for parsing the merge index. At this time, as an embodiment, the decoder can determine the merge index to be actually used by modifying it based on the parsed merge index. For example, the merge index to be actually used can be determined by adding the number of candidates that are smaller than the parsed merge index and are determined to always use the MMVD, to the parsed merge index. For example, if it is specified that MMVD should always be used for the first num_mmvd_baseCand of the candidate list, the parsed merge index plus num_mmvd_baseCand can be used as the merge index. If the MMVD flag is before the merge flag, candidates that use MMVD can be excluded from the candidate list if the MMVD flag is 0.

[0293] Referring to FIG. 27, the MMVD flag precedes the merge flag. If the MMVD flag is 1, the decoder can parse the MMVD-related syntax. If the MMVD flag is 0, the decoder can parse the merge flag. If the MMVD flag is 0 and the merge flag is 1, the decoder can parse the merge index. At this time, there may be additional conditions for parsing the merge index. In addition, the number of candidates that can be used as the base motion vector of the MMVD, num_mmvd_baseCand, can be added to the parsed merge index to determine the merge index to be actually used.

[0294] FIG. 28 is a diagram illustrating the MMVD syntax according to an embodiment of the present invention. According to an embodiment of the present invention, a candidate index may exist before a flag indicating the presence or absence of mode usage. For example, the candidate index may exist before the MMVD flag or the merge flag. In such a case, after parsing the candidate index, it is possible to determine whether to parse the MMVD flag depending on whether the parsed index is a candidate that can use MMVD. For example, if the parsed candidate index is a candidate that can use MMVD, the MMVD flag is parsed, and if it is a candidate that cannot use MMVD, the MMVD flag is not parsed and can be inferred as 0. Also, when the MMVD flag is 0, the merge flag can be parsed. The merge flag can determine whether the merge mode or the sub-block merge mode is used, or whether the inter mode or the AMVP mode or the affine inter mode is used. Although FIG. 28 mainly describes the decoder for convenience of explanation, the coding process according to this embodiment may be applied to the encoder in substantially the same manner.

[0295] For example, if the number of candidates possible as the first mode is num1, the number of candidates possible as the second mode is num2, and num1 < num2, when parsing the candidate index, if the parsed index is num1 or more, it is not necessary to parse the syntax for determining from among the first mode and the second mode (for example, the syntax indicating whether to use the first mode), and it can be inferred that the first mode is not used. And when the parsed index is smaller than num1, the syntax for determining from among the first mode and the second mode can be parsed.

[0296] Referring to FIG. 28, after parsing the merge index, if the merge index is smaller than num_mmvd_baseCand, which is the number of candidates for which the MMVD can be used, the MMVD flag can be parsed. If the merge index is equal to or larger than num_mmvd_baseCand, the MMVD flag does not need to be parsed, and in this case, the MMVD flag can be inferred to be 0. If the MMVD flag is 1, a distance index (distance_idx) and a direction index (direction_idx) representing MMVD-related syntax elements can be parsed. If the MMVD flag is 0, the merge flag is parsed, and it can be determined whether the mode is a merge mode or a sub-block merge mode, or an inter mode, an AMVP mode, or an affine mode.

[0297] FIG. 29 is a diagram illustrating a coding unit syntax structure according to an embodiment of the present invention. As shown in FIG. 29, when the merge flag is 1, a merge data (i.e., merge_data) parsing process can be performed. For convenience of explanation, FIG. 29 mainly describes a decoder, but the coding process according to this embodiment may be applied to an encoder in substantially the same manner. In one embodiment, the merge data may include a part of a merge-related syntax. Also, the merge data may be referred to as a merge data syntax. For example, the merge data may include a syntax parsing process performed when the merge flag is 1 in the above-mentioned FIG. 21 to FIG. 28. Also, the merge flag being 1 may mean that a merge mode is used. Also, the merge flag being 1 may mean that an inter prediction without using mvd_coding shown in FIG. 6 and Table 1 above is used.

[0298] FIG. 30 illustrates an example of a merge data syntax structure according to an embodiment of the present invention. As described in FIG. 29, if the merge flag is 1, the decoder may parse (or call) the merge data syntax. For convenience of explanation, FIG. 30 focuses on a decoder, but the coding process according to this embodiment may be applied in substantially the same manner to an encoder. Also, as described above, the merge flag may be set to 1 when using merge mode or skip mode.

[0299] Referring to FIG. 30, in the merge data syntax, the MMVD flag may be parsed first among the syntax elements. The MMVD flag may be a syntax that can be parsed first after confirming that the merge flag is 1. The MMVD flag may be parsed before signaling indicating whether or not another prediction mode that sets the merge flag to 1 is used. The MMVD flag may be signaling indicating whether or not the MMVD is used. The same method as the embodiment described in Table 16 and FIGS. 22 to 29 may be applied. As shown in FIG. 30, the merge data syntax may include a decoding process for a sub-block merge mode, multi-hypothesis prediction (or intra and inter combined prediction), triangle prediction, etc.

[0300] According to an embodiment of the present invention, the multiple hypothesis prediction may be a method of generating and combining two or more prediction blocks when generating a prediction block. Alternatively, the multiple hypothesis prediction may be a method of using both inter prediction and intra prediction when generating a prediction block. In addition, the inter prediction and intra prediction may be methods of using the same picture as the picture including the current block and a different picture when making a prediction. Referring to FIG. 30, the multiple hypothesis prediction flag represents a syntax element indicating whether or not the multiple hypothesis prediction is used.

[0301] According to an embodiment of the present invention, the subblock merging mode may be a method of performing motion compensation on a subblock basis (i.e., a motion vector is determined on a subblock basis) when predicting a current block (e.g., a coding unit or a prediction unit). In one embodiment, the subblock merging mode may include a method such as subblock-based temporal MVP, affine motion prediction, etc. Referring to FIG. 30, the merge subblock flag represents a syntax element indicating whether or not to use the subblock merging mode.

[0302] According to an embodiment of the present invention, the triangle prediction represents a method of performing motion compensation for a non-rectangular area in the current block. That is, in the triangle prediction, a unit having the same motion vector in the current block may not be a rectangle. Referring to Figure 30, the triangle prediction flag (i.e., merge_triangle_flag) represents a syntax element indicating whether or not the triangle prediction is used.

[0303] Also, according to an embodiment of the present invention, the MMVD flag may be parsed before signaling (or syntax elements) indicating the use of a merge mode other than MMVD, such as a multiple hypothesis prediction flag, a merge sub-block flag, a merge triangle flag, etc.

[0304] 30, the MMVD merge index (i.e., mmvd_merge_idx) (or MMVD index) represents a syntax element that indicates which one to use as the base motion vector of the MMVD. If the MMVD is used, the merge index may be inferred as the MMVD merge flag (i.e., mmvd_merge_flag) (or MMVD flag).

[0305] FIG. 31 is a diagram illustrating a merge data syntax structure according to an embodiment of the present invention. The embodiment of FIG. 31 may be a part of the merge data syntax performed when the merge flag is 1 in FIG. 29. Although FIG. 31 mainly describes a decoder for convenience of explanation, the coding process according to this embodiment may be applied to an encoder in substantially the same manner. According to an embodiment of the present invention, the MMVD flag may be parsed after the merge index. For example, the MMVD flag may be parsed immediately after the merge index. For example, the MMVD flag may not be parsed at a position other than the frontmost position in the merge data syntax. That is, after confirming that the merge flag is 1, the MMVD flag may be parsed after parsing other syntax instead of the MMVD flag.

[0306] Also, according to an embodiment of the present invention, as described in FIG. 21, the base motion vector of the MMVD may be determined from the other candidate list, and in one embodiment, the base motion vector may be determined from a part of the other candidate list. Therefore, according to this embodiment, as described in FIG. 25, it is possible to determine whether to parse the MMVD flag based on an index associated with the other candidate list that can be used as the base motion vector of the MMVD. For example, an index associated with the other candidate list that can be used as the base motion vector of the MMVD may be parsed before the MMVD flag. Then, if the index associated with the other candidate list indicates that it can be used as the base motion vector of the MMVD, the MMVD flag may be parsed, and if not, the MMVD flag may not be parsed.

[0307] Referring to FIG. 31, the base motion vector of the MMVD may be determined from the merge candidate. Thus, in one embodiment, if the merge index indicates that it can be used as the base motion vector of the MMVD, the MMVD flag may be parsed, otherwise the MMVD flag may not be parsed. Alternatively, if the base motion vector of the MMVD may be selected from among the first n of the merge candidate list, the MMVD flag may be parsed if the merge index is smaller than n (the merge index may start from 0), and the MMVD flag may not be parsed if the merge index is not smaller than n. More specifically, the base motion vector of the MMVD may be the first or second candidate of the merge candidate list, and referring to FIG. 31, if the merge index is smaller than 2, i.e., 0 or 1, the MMVD flag may be parsed. The merge index may represent a merge candidate index. Thus, if the merge mode is in effect but the merge index indicates that the MMVD cannot be used, the MMVD flag may not be parsed, thereby improving coding efficiency.

[0308] Also, the base motion vector of the MMVD may be determined from a candidate list of a certain mode. Thus, according to an embodiment of the present invention, when there are a number of signalings indicating whether or not various modes are used, the MMVD flag may be parsed after it is determined that the certain mode is used. For example, assuming that a first mode, a second mode, and a third mode exist, and the MMVD is determined based on the third mode, or the base motion vector of the MMVD is determined from a candidate of the third mode, the MMVD flag may be parsed after it is determined that the third mode is used. For example, it may be determined that the third mode is used according to signaling indicating whether or not the third mode is used. Or, for example, it may be determined that the third mode is used when it is determined that a mode other than the third mode, for example, the first mode and the second mode, is not used. For example, when a subblock merge mode, multiple hypothesis prediction, triangle prediction, conventional merge mode, etc. exist, and the MMVD can be applied to the conventional merge mode, it may be determined that the conventional merge mode or the MMVD is used among the listed modes, and the MMVD flag may be parsed. The conventional merge mode is a mode in which inter prediction is performed using motion information of neighboring intra-coded blocks, and may refer to a merge mode of a conventional video compression technology (eg, high efficiency video coding (HEVC)).

[0309] Also, when parsing the MMVD flag based on an index associated with another candidate list that can be used as the base motion vector of the MMVD, there is no need for a separate signaling indicating the base motion vector of the MMVD. For example, in FIG. 23 or FIG. 24, a signaling such as a base motion vector index exists separately from the merge index. Also, referring to FIG. 30, the MMVD merge flag may be a signaling indicating the base motion vector of the MMVD. According to an embodiment of the present invention, as shown in FIG. 31, the MMVD merge flag may not exist. For example, the MMVD-related syntax may include only the MMVD flag, signaling indicating the MMVD distance (mmvd_distance_idx in FIG. 30 and FIG. 31), and signaling indicating the MMVD direction (mmvd_direction_idx in FIG. 30 and FIG. 31). Also, the base motion vector of the MMVD may be determined by the merge index. As a result, the signaling indicating the base motion vector of the MMVD in FIG. 30 and its context model do not need to be present in the embodiment of FIG.

[0310] Also, according to an embodiment of the present invention, the MMVD flag may be parsed after signaling indicating the use of a certain merge mode other than the above-mentioned MMVD. With reference to FIG. 31, the MMVD flag may be parsed after the merge sub-block flag, the multiple hypothesis prediction flag (i.e., mh_intra_flag), and the merge triangle flag (i.e., merge_triangle_flag). Also, a certain merge mode may not be used with the MMVD. In that case, if the flag indicating whether or not the certain merge mode is used indicates that it is not used, the MMVD flag may be parsed. With reference to FIG. 31, the MMVD may be unusable with the triangle prediction, and the MMVD flag may be parsed if the merge triangle flag is 0. Or, the MMVD may be unusable with the MH intra, and the MMVD flag may be parsed if the mh_intra_flag is 0. Or, the MMVD may be unusable with the sub-block merge mode, and the MMVD flag may be parsed if the merge sub-block flag is 0.

[0311] Figure 32 is a diagram illustrating a merge data syntax structure according to an embodiment of the present invention. The embodiment of Figure 32 may be a part of the merge data syntax performed when the merge flag is 1 in Figure 29. For convenience of explanation, Figure 32 focuses on a decoder, but the coding process according to this embodiment may be applied in substantially the same manner to an encoder.

[0312] According to an embodiment of the present invention, the MMVD flag may be parsed after signaling indicating whether or not other modes are used, or the MMVD flag may be parsed after signaling indicating whether or not other modes are used other than MMVD in which the merge flag is set to 1. For example, the MMVD flag may be parsed after the merge sub-block flag, the multiple hypothesis prediction flag (i.e., mh_intra_flag), and the merge triangle flag (i.e., merge_triangle_flag).

[0313] Also, in one embodiment, the MMVD flag may be parsed when a mode other than MMVD is not used. For example, the MMVD flag may be parsed when the merge sub-block flag is 0. Also, the MMVD flag may be parsed when the multiple hypothesis prediction flag is 0. Also, the MMVD flag may be parsed when the merge triangle flag is 0. The flag indicating whether or not a certain mode is used may indicate that the certain mode is not used when it is 0. Also, when the MMVD can be applied to a conventional merge mode, the MMVD flag may be parsed after it is determined that the conventional merge mode or the MMVD is used.

[0314] Also, in one embodiment, if the MMVD is used, the decoder may parse the MMVD-related syntax, and if the MMVD is not used, the decoder may parse the merge index. Alternatively, if the MMVD is used, the decoder may parse the MMVD merge flag (i.e., mmvd_merge_flag), the MMVD distance index (mmvd_distance_idx), and the MMVD direction index (mmvd_direction_idx), and if the MMVD is not used, the decoder may parse the merge index. Also, as shown in FIG. 32, if the MMVD flag is parsed after the flag indicating whether or not another mode is used, the signaling indicating the base motion vector of the MMVD may be present as a syntax element separate from the merge index. Referring to FIG. 32, the signaling indicating the base motion vector of the MMVD may be the MMVD merge flag. For example, the MMVD flag may be parsed after it is determined to use the conventional merge mode or the MMVD.

[0315] According to an embodiment of the present invention, the base motion vector of the MMVD is determined from a candidate list, and the maximum number of candidates of the candidate list may be variable. For example, the maximum number of candidates may be determined from a higher level syntax. The higher level syntax may be a syntax at a higher level than the current coding unit. For example, the higher level syntax may be a syntax at a sequence, picture, slice, or tile level. According to an embodiment, in this case, the maximum number of candidates of the base motion vector of the MMVD may follow the maximum number of candidates in the candidate list that can be the base motion vector of the MMVD. Therefore, if the maximum number of candidates in the candidate list that can be the base motion vector of the MMVD is reduced, the maximum number of candidates that can be the base motion vector of the MMVD may also be reduced.

[0316] In one embodiment, the base motion vector of the MMVD may be determined from the merge candidate list, and the maximum number of merge candidates may be MaxNumMergeCand. Also, MaxNumMergeCand may be determined from a higher level syntax. In such a case, the maximum number of merge candidates that can be the base motion vector of the MMVD may be less than or equal to MaxNumMergeCand. Therefore, as shown in FIG. 32, MaxNumMergeCand may determine whether to parse signaling indicating the base motion vector of the MMVD. For example, if MaxNumMergeCand is 1, it is not necessary to parse signaling indicating the base motion vector of the MMVD, and it can be inferred to be 0 if there is no signaling indicating the base motion vector of the MMVD.

[0317] In another embodiment, if MaxNumMergeCand is 1, MMVD may not be used. This may be to reduce signaling overhead for MMVD-related syntax. Therefore, if MaxNumMergeCand is 1, the MMVD flag may not be parsed. Also, if the MMVD flag is not present, it can be inferred as 0. Also, referring to FIG. 32, the merge triangle flag can be parsed only if multiple hypothesis intra prediction is not used. If the merge triangle flag is not present, it can be inferred as 0.

[0318] According to an embodiment of the present invention, there may be a number of modes that cannot be used together. For example, subblock merging mode, triangle prediction, multiple hypothesis intra prediction, and MMVD may not be used together. Also, signaling indicating whether to use each of the multiple modes that cannot be used together may be parsed according to a pre-set order. In this case, signaling indicating whether to use a certain mode among the multiple modes may be parsed only if all of the signaling indicating whether to use another certain mode among the multiple modes parsed earlier indicates not to use it.

[0319] Fig. 33 is a diagram illustrating a video signal processing method according to an embodiment to which the present invention is applied. For convenience of explanation, a decoder will be described with reference to Fig. 33, but the present invention is not limited thereto, and the multiple hypothesis prediction-based video signal processing method according to this embodiment may be applied to an encoder in substantially the same manner.

[0320] The decoder obtains a first syntax element indicating whether a sub-block based merging mode is applied to a current block (S3301). As described above, in one embodiment, the first syntax element may be a sub-block merging flag (or a merge sub-block flag).

[0321] If the first syntax element indicates that the sub-block based merge mode is not applied to the current block, the decoder obtains a second syntax element indicating whether or not a merge mode using a motion vector difference is applied to the current block (S3302). As described above, in one embodiment, the second syntax element may be an MMVD flag (or an MMVD merge flag).

[0322] When the second syntax element indicates that a merge mode using the motion vector difference is applied to the current block, the decoder obtains a third syntax element indicating a candidate to be used for inter prediction of the current block among candidates included in a merge candidate list of the current block (S3303). As described above, in one embodiment, the third syntax element may be an MMVD index (or an MMVD candidate index or an MMVD candidate flag).

[0323] The decoder obtains information related to the motion vector difference (S3304). As described above, in one embodiment, obtaining the information related to the motion vector difference may include obtaining a fourth syntax element indicating a distance of the motion vector difference; and obtaining a fifth syntax element indicating a direction of the motion vector difference. As described above, in one embodiment, the fourth syntax element may be a distance index (or an MMVD distance index), and the fifth syntax element may be a direction index (or an MMVD direction index).

[0324] The decoder derives a motion vector for the current block by adding the motion vector differential to a candidate motion vector indicated by the third syntax element (S3305).

[0325] The decoder generates a prediction block of the current block using the motion vector of the current block (S3306).

[0326] As described above, as an example, the third syntax element may indicate a candidate to be used for inter prediction of the current block among the first candidate and the second candidate in the merge candidate list of the current block.

[0327] As mentioned above, as an example, the third syntax element may be parsed from the bitstream if the maximum number of merging candidates is greater than one, and may be inferred to be 0 if the maximum number of merging candidates is not greater than one.

[0328] As described above, in one embodiment, when the first syntax element indicates that a sub-block based merge mode is applied to the current block, the method further includes obtaining a sixth syntax element indicating a candidate to be used for inter prediction of the current block among candidates included in a sub-block merge candidate list of the current block, and when the first syntax element indicates that a sub-block based merge mode is applied to the current block, a motion vector of the current block may be derived in units of at least one sub-block included in the current block based on a motion vector of a candidate indicated by the sixth syntax element. As described above, in one embodiment, the sixth syntax element may be a sub-block merge index (or a merge sub-block index).

[0329] As mentioned above, as an example, if the first syntax element indicates that a sub-block based merge mode is applied to the current block, the second syntax element may be inferred to be 0.

[0330] As mentioned above, as an example, the sixth syntax element may be parsed from the bitstream if the maximum number of sub-block merging candidates is greater than 1, and may be inferred to be 0 if the maximum number of sub-block merging candidates is not greater than 1.

[0331] The above-described embodiments of the present invention may be implemented in various ways. For example, the embodiments of the present invention may be implemented in hardware, firmware, software, or a combination thereof.

[0332] In the case of a hardware implementation, the method according to the embodiment of the present invention may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, etc.

[0333] In the case of implementation by firmware or software, the method according to the embodiment of the present invention may be implemented in the form of a module, procedure, or function that performs the functions or operations described above. The software code may be stored in a memory and driven by a processor. The memory may be located inside or outside the processor, and data may be exchanged with the processor by various means known in the art.

[0334] Some embodiments may be embodied in the form of a recording medium including computer executable instructions such as a program module executed by the computer. A computer readable medium may be any available medium accessible by a computer, including volatile and non-volatile media, separate and non-separate media. Also, a computer readable medium may include any computer storage medium and communication media. A computer storage medium includes any volatile and non-volatile, separate and non-separate media embodied in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. A communication medium typically includes computer readable instructions, data structures or other data in a modulated data signal such as a program module, or other transmission mechanism, and includes any information delivery medium.

[0335] The above description of the present invention is for illustrative purposes only, and those skilled in the art will understand that the present invention can be easily modified into other specific forms without changing the technical idea or essential features of the present invention. Therefore, the above-described embodiments should be interpreted as illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed form, and similarly, each component described as distributed may be implemented in a combined form.

[0336] The scope of the present invention is indicated by the claims set forth below rather than by the above detailed description, and all modifications and variations derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included within the scope of the present invention. [Explanation of symbols]

[0337] 100 Encoding Device 110 Conversion unit 115 Quantization section 120 Inverse quantization section 125 Reverse conversion unit 130 Filtering section 150 Prediction Department 160 Entropy coding unit

Claims

1. 1. A method for processing a video signal, comprising the steps of: obtaining a first syntax element indicating whether a sub-block based merging mode is applied to a current block; obtaining a second syntax element indicating whether a merge mode using a motion vector difference is to be applied to the current block, when the first syntax element indicates that a sub-block based merge mode is not to be applied to the current block; obtaining a third syntax element indicating a candidate to be used for inter prediction of the current block among candidates included in a merge candidate list of the current block, when the second syntax element indicates that a merge mode using the motion vector difference is to be applied to the current block; obtaining information related to the motion vector difference; deriving a motion vector for the current block by adding the motion vector differential to a candidate motion vector indicated by the third syntax element; and A method for processing a video signal, comprising: generating a prediction block of the current block using a motion vector of the current block.

2. The method of claim 1 , wherein the third syntax element indicates a candidate to be used for inter prediction of the current block among a first candidate and a second candidate in a merge candidate list of the current block.

3. 3. The method of claim 2, wherein the third syntax element is parsed from the bitstream if the maximum number of merging candidates is greater than one, and is inferred to be 0 if the maximum number of merging candidates is not greater than one.

4. The step of obtaining information related to the motion vector difference comprises: obtaining a fourth syntax element indicating a distance of the motion vector difference; and The method of claim 1 , further comprising obtaining a fifth syntax element indicating a direction of the motion vector difference.

5. When the first syntax element indicates that a sub-block based merge mode is applied to the current block, the method further includes obtaining a sixth syntax element indicating a candidate to be used for inter prediction of the current block among candidates included in a sub-block merging candidate list of the current block, 2. The video signal processing method of claim 1, wherein when the first syntax element indicates that a sub-block based merge mode is applied to the current block, a motion vector of the current block is derived based on a candidate motion vector indicated by the sixth syntax element for at least one sub-block included in the current block.

6. The method of claim 5, wherein the second syntax element is inferred to be 0 if the first syntax element indicates that a sub-block based merge mode is applied to the current block.

7. 6. The method of claim 5, wherein the sixth syntax element is parsed from the bitstream if the maximum number of sub-block merging candidates is greater than one, and is inferred to be 0 if the maximum number of sub-block merging candidates is not greater than one.

8. 1. A video signal processing device, comprising: A processor is included. The processor, Obtain a first syntax element indicating whether a sub-block based merging mode is applied to the current block; If the first syntax element indicates that a sub-block based merge mode is not applied to the current block, obtain a second syntax element indicating whether a merge mode using a motion vector difference is applied to the current block; If the second syntax element indicates that a merge mode using the motion vector difference is applied to the current block, a third syntax element is obtained, the third syntax element indicating a candidate to be used for inter prediction of the current block among candidates included in a merge candidate list of the current block; obtaining information related to the motion vector difference; deriving a motion vector for the current block by adding the motion vector differential to a candidate motion vector indicated by the third syntax element; A video signal processing apparatus that generates a predicted block of the current block using the motion vector of the current block.

9. The video signal processing device of claim 8 , wherein the third syntax element indicates a candidate to be used for inter prediction of the current block among a first candidate and a second candidate in a merge candidate list of the current block.

10. The video signal processing device of claim 9, wherein the third syntax element is parsed from the bitstream if the maximum number of merging candidates is greater than one, and is inferred to be 0 if the maximum number of merging candidates is not greater than one.

11. The processor, Obtain a fourth syntax element indicating a distance of the motion vector difference; The apparatus of claim 8 , further comprising: obtaining a fifth syntax element indicating a direction of the motion vector difference.

12. The processor, If the first syntax element indicates that a sub-block based merge mode is applied to the current block, a sixth syntax element is obtained, which indicates a candidate to be used for inter prediction of the current block among candidates included in a sub-block merging candidate list of the current block; 9. The video signal processing device of claim 8, wherein when the first syntax element indicates that a sub-block based merge mode is applied to the current block, a motion vector of the current block is derived on the basis of a candidate motion vector indicated by the sixth syntax element for at least one sub-block included in the current block.

13. The apparatus of claim 12, wherein the second syntax element is inferred to be 0 if the first syntax element indicates that a sub-block based merge mode is applied to the current block.

14. 13. The video signal processing apparatus of claim 12, wherein the sixth syntax element is parsed from a bitstream if a maximum number of sub-block merging candidates is greater than one, and is inferred to be 0 if the maximum number of sub-block merging candidates is not greater than one.

15. 1. A method for processing a video signal, comprising the steps of: encoding a first syntax element indicating whether a sub-block based merging mode is applied to a current block; encoding a second syntax element indicating whether a merge mode using a motion vector difference is to be applied to the current block if a sub-block based merge mode is not to be applied to the current block; encoding a third syntax element indicating a candidate to be used for inter prediction of the current block among candidates included in a merge candidate list of the current block, when a merge mode using the motion vector difference is applied to the current block; encoding information related to the motion vector differentials; deriving a motion vector for the current block by adding the motion vector differential to a candidate motion vector indicated by the third syntax element; and A method for processing a video signal, comprising: generating a prediction block of the current block using a motion vector of the current block.