Video signal encoding / decoding method and apparatus for the method

Abstract

The present invention provides a video signal encoding / decoding method and an apparatus for the method. The video decoding method of the present invention includes the following steps: determining whether to apply a merged motion difference encoding method to a current block; generating a merged candidate list for the current block; determining a merged candidate for the current block based on the merged candidate list; and deriving a motion vector for the current block based on the merged candidate.

Description

Technical Field

[0001] The present invention relates to a method for encoding / decoding a video signal and an apparatus for the method. Background Art

[0002] With the trend of increasingly large display panels, there is a growing need for video services with higher image quality. The biggest problem with high-definition video services is the significant increase in data volume. To solve this problem, research on improving video compression rates is actively underway. As a representative example, in 2009, the Motion Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) under the International Telecommunication Union - Telecommunication (ITU-T) established the Joint Collaborative Team on Video Coding (JCT-VC). JCT-VC proposed the video compression standard HEVC (High Efficiency Video Coding) and it was approved on January 25, 2013. Its compression performance is approximately twice that of H.264 / AVC. With the rapid development of high-definition video services, the limitations of HEVC's performance have gradually emerged. Summary of the Invention

[0003] Technical Problem to be Solved

[0004] An object of the present invention is to provide a method for refining a motion vector derived from merge candidates based on an offset vector when encoding / decoding a video signal and an apparatus for performing the method.

[0005] An object of the present invention is to provide a method for signaling an offset vector when encoding / decoding a video signal and an apparatus for performing the method.

[0006] The technical problems to be achieved by the present invention are not limited to the above-mentioned technical problems, and other technical problems not mentioned will be clearly understood by those of ordinary skill in the technical field to which the present invention pertains through the following description.

[0007] Technical Solution

[0008] The video signal decoding method according to the present invention includes the following steps: determining whether to apply the merged motion difference coding method to a current block; generating a merged candidate list for the current block; determining a merged candidate for the current block based on the merged candidate list; and deriving a motion vector for the current block based on the merged candidate. Wherein, when the merged motion difference coding method is applied to the current block, the motion vector for the current block is derived by adding an offset vector to the motion vector derived based on the merged candidate, and when the maximum number of merged candidates included in the merged candidate list is 1, the merged candidate is determined without decoding index information indicating one of the merged candidates in the merged candidate list decoded from the bitstream. The magnitude of the offset vector is determined based on first index information indicating one of a plurality of motion offset magnitude candidates; wherein, at least one of the maximum value or the minimum value of the motion offset magnitude candidates is set differently according to the value of a flag indicating the range of the motion offset magnitude candidates, or at least one of the maximum value or the minimum value of the motion offset magnitude candidates is set differently according to the motion vector precision of the current block.

[0009] In the video signal decoding method according to the present invention, the flag may be signaled at the picture level.

[0010] In the video signal decoding method according to the present invention, the magnitude of the offset vector may be obtained by performing a shift operation on the value represented by the motion offset magnitude candidate specified by the first index information.

[0011] In the video signal decoding method according to the present invention, the direction of the offset vector may be determined based on second index information for indicating one of a plurality of vector direction candidates.

[0012] The video signal encoding method of the present invention includes the following steps: determining whether to apply the merged motion difference encoding method to a current block; generating a merged candidate list for the current block; determining a merged candidate for the current block based on the merged candidate list; and deriving a motion vector for the current block based on the merged candidate, wherein when the merged motion difference encoding method is applied to the current block, the motion vector for the current block is derived by adding an offset vector to the motion vector derived based on the merged candidate, and when the maximum number of merged candidates included in the merged candidate list is 1, encoding of index information indicating the merged candidate for the current block in the merged candidate list is omitted. Encoding first index information for specifying a motion offset size candidate indicating the size of the offset vector among a plurality of motion offset size candidates; wherein encoding a flag indicating a range of the motion offset size candidate, wherein at least one of a maximum value or a minimum value of the motion offset size candidate is set differently according to a value of the flag, or at least one of a maximum value or a minimum value of the motion offset size candidate is set differently according to a motion vector precision of the current block.

[0013] The video decoding device of the present invention includes an inter prediction unit configured to: determine whether to apply the merged motion difference encoding method to a current block; generate a merged candidate list for the current block; determine a merged candidate for the current block based on the merged candidate list; and derive a motion vector for the current block based on the merged candidate, wherein when the merged motion difference encoding method is applied to the current block, the motion vector for the current block is derived by adding an offset vector to the motion vector derived based on the merged candidate, and when the maximum number of merged candidates included in the merged candidate list is 1, the merged candidate is determined without decoding index information indicating one merged candidate in the merged candidate list decoded from a bitstream. The inter prediction unit is further configured to determine a size of the offset vector based on first index information indicating one of a plurality of motion offset size candidates; the inter prediction unit is further configured to set at least one of a maximum value or a minimum value of the motion offset size candidate differently according to a value of a flag indicating a range of the motion offset size candidate, or set at least one of a maximum value or a minimum value of the motion offset size candidate differently according to a motion vector precision of the current block.

[0014] The video encoding device of the present invention includes an inter prediction unit, which is configured to: determine whether to apply a merged motion difference encoding method to a current block; generate a merged candidate list for the current block; determine a merged candidate for the current block based on the merged candidate list; and derive a motion vector for the current block based on the merged candidate, wherein when the merged motion difference encoding method is applied to the current block, the motion vector for the current block is derived by adding an offset vector to the motion vector derived based on the merged candidate, and when the maximum number of merged candidates included in the merged candidate list is 1, encoding of index information indicating the merged candidate for the current block in the merged candidate list is omitted. The inter prediction unit is further configured to encode first index information for specifying a motion offset size candidate indicating the size of the offset vector among a plurality of motion offset size candidates, and the inter prediction unit is further configured to encode a flag indicating a range of the motion offset size candidate, wherein at least one of a maximum value or a minimum value of the motion offset size candidate is set differently according to the value of the flag, or the inter prediction unit is further configured to set at least one of a maximum value or a minimum value of the motion offset size candidate differently according to the motion vector precision of the current block.

[0015] The video decoder of the present invention includes a processor and a memory storing a computer program. The processor is configured to execute the computer program to perform the above video decoding method.

[0016] The video encoder of the present invention includes a processor and a memory storing a computer program. The processor is configured to execute the computer program to perform the above video encoding method.

[0017] The computer-readable storage medium of the present invention stores an executable program, which, when executed by a processor, implements the above video decoding method or video encoding method.

[0018] The features briefly outlined above for the present invention are merely exemplary embodiments of the detailed description of the present invention to be described later and do not limit the scope of the present invention.

[0019] Advantages of the Invention

[0020] According to the present invention, the inter prediction efficiency can be improved by refining the motion vector of the merged candidate based on the offset vector.

[0021] According to the present invention, the inter prediction efficiency can be improved by adaptively determining the size and direction of the offset vector.

[0022] The effects achievable in the present invention are not limited to the above effects, and other effects not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention pertains from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a block diagram of a video encoder according to an embodiment of the present invention.

[0024] Figure 2 is a block diagram of a video decoder according to an embodiment of the present invention.

[0025] Figure 3 is a diagram showing a basic coding tree unit according to an embodiment of the present invention.

[0026] Figure 4 is a diagram showing various partitioning types of coding blocks.

[0027] Figure 5 is a diagram showing an example of partitioning of a coding tree unit.

[0028] Figure 6 is a diagram showing the shape of a basic data unit.

[0029] Figure 7 and Figure 8 is a diagram showing an example of partitioning a coding block into multiple sub-blocks.

[0030] Figure 9 is a flowchart showing an inter-frame prediction method according to an embodiment of the present invention.

[0031] Figure 10 is a diagram showing the non-linear motion of an object.

[0032] Figure 11 is a flowchart showing an inter-frame prediction method based on affine motion according to an embodiment of the present invention.

[0033] Figure 12 is a diagram showing an example of an affine seed vector for each affine motion model.

[0034] Figure 13 is a diagram showing an example of an affine vector of a sub-block under a 4-parameter motion model.

[0035] Figure 14 is a flowchart showing a process of deriving motion information of a current block using a merge mode.

[0036] Figure 15 is a diagram showing candidate blocks for deriving merge candidates.

[0037] Figure 16 is a diagram showing the position of a reference sample.

[0038] Figure 17 A diagram showing an example of a candidate block for deriving merge candidates.

[0039] Figure 18 A diagram showing an example of changing the position of a reference sample.

[0040] Figure 19 A diagram showing an example of changing the position of a reference sample.

[0041] Figure 20 A flowchart showing the process for updating the inter-frame motion information list.

[0042] Figure 21 A diagram showing an embodiment of updating the inter-frame merge candidate list.

[0043] Figure 22 A diagram showing an example of updating the index of the stored inter-frame merge candidates.

[0044] Figure 23 A diagram showing the position representing a sub-block.

[0045] Figure 24 A diagram showing an example of generating an inter-frame motion information list for each inter-frame prediction mode.

[0046] Figure 25 A diagram showing an example of adding the inter-frame merge candidates included in the long-term motion information list to the merge candidate list.

[0047] Figure 26 A diagram showing an example of performing a redundancy check only on some merge candidates.

[0048] Figure 27 A diagram showing an example of omitting the redundancy check for a specific merge candidate.

[0049] Figure 28 A diagram showing an offset vector according to the values of distance_idx indicating the magnitude of the offset vector and direction_idx indicating the direction of the offset vector.

[0050] Figure 29 A diagram showing an offset vector according to the values of distance_idx indicating the magnitude of the offset vector and direction_idx indicating the direction of the offset vector.

[0051] Figure 30 A diagram showing the partitioning mode of a coded block when applying the triangular partitioning technique.

[0052] Figure 31 A diagram showing an example of setting the offset vectors of each sub-unit differently.

[0053] Figure 32 It is a diagram showing motion vector candidates that a fine merge candidate can adopt.

[0054] Figure 33 It is a diagram showing the configuration of a merge refinement offset list.

[0055] Figure 34 and Figure 35 It is a diagram showing an offset vector specified by a merge offset candidate.

[0056] Figure 36 It is a diagram showing candidate blocks for deriving motion vector prediction candidates.

[0057] Figure 37 It is a diagram showing motion vector candidates that can be set as fine motion vector prediction candidates.

[0058] Figure 38 It is a diagram showing the configuration of a prediction vector refinement offset list. Detailed Description of the Invention

[0059] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0060] Video encoding and decoding are performed in units of blocks. For example, encoding / decoding processes such as transformation, quantization, prediction, loop filtering, or reconstruction can be performed on an encoding block, a transform block, or a prediction block.

[0061] Hereinafter, a block to be encoded / decoded is referred to as a "current block". For example, depending on the current encoding / decoding processing step, the current block can represent an encoding block, a transform block, or a prediction block.

[0062] In addition, the term "unit" used in this specification represents a basic unit for performing a specific encoding / decoding process, and a "block" can be understood to represent an array of samples of a predetermined size. Unless otherwise specified, "block" and "unit" can be used interchangeably. For example, in the embodiments described later, an encoding block and an encoding unit can be understood to have the same meaning.

[0063] Figure 1 It is a block diagram of a video encoder according to an embodiment of the present invention.

[0064] Referring to Figure 1 , the video encoding device 100 may include an image division unit 110, prediction units 120, 125, a transform unit 130, a quantization unit 135, a rearrangement unit 160, an entropy encoding unit 165, an inverse quantization unit 140, an inverse transform unit 145, a filter unit 150, and a memory 155.

[0065] Figure 1Each component shown is shown separately to represent different characteristic functions in a video encoding device, and does not mean that each component is composed of separate hardware or a single software component. That is, for each component, for the convenience of explanation, the components are arranged in such a way that at least two components in each component are combined into one component or one component is divided into multiple components, and thus the functions are performed. As long as the essence of the present invention is not departed from, such embodiments of integrating the components and embodiments of separating the components also belong to the scope of the claims of the present invention.

[0066] Moreover, some structural elements are not essential structural elements for performing the essential functions in the present invention, but are optional structural elements only for improving performance. The present invention can be implemented by including only the components required for the essence of the present invention except for the structural elements only for improving performance, and the structure including only the essential structural elements except for the selective structural elements only for improving performance also belongs to the scope of the claims of the present invention.

[0067] The image division unit 110 may divide the input image into at least one processing unit. In this case, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The image division unit 110 divides an image into a combination of multiple coding units, prediction units, and transform units, and may select a combination of one coding unit, prediction unit, and transform unit based on a predetermined criterion (e.g., a cost function) to encode the image.

[0068] For example, an image may be divided into multiple coding units. To divide an image into coding units, a recursive tree structure such as a quad tree structure may be used. A video or a largest coding unit may be used as the root, and the coding unit may be divided into other coding units that may have the number of child nodes equivalent to the number of divided coding units. The coding unit that is no longer divided according to certain restrictions will become a leaf node. That is, when it is assumed that a coding unit can only implement square division, a coding unit can be divided into at most 4 other coding units.

[0069] Hereinafter, in the embodiments of the present invention, the coding unit may mean a unit for performing encoding and may also mean a unit for performing decoding.

[0070] The prediction units within a coding unit may be divided into at least one shape such as a square or a rectangle of the same size, or a prediction unit within a coding unit may be divided into a shape and / or size different from another prediction unit.

[0071] When the prediction unit for performing intra prediction based on a coding unit is not the smallest coding unit, intra prediction can be performed without being divided into multiple N×N prediction units.

[0072] The prediction units 120 and 125 may include an inter prediction unit 120 for performing inter prediction and an intra prediction unit 125 for performing intra prediction. It can be determined whether to use inter prediction or intra prediction for a prediction unit, and specific information (e.g., intra prediction mode, motion vector, reference image, etc.) can be determined based on each prediction method. In this case, the processing unit performing the prediction may be different from the processing unit determining the prediction method and details. For example, the prediction method and prediction mode, etc. can be determined by the prediction unit, and the prediction can be performed by the transform unit. The residual value (residual block) between the generated prediction block and the original block can be input to the transform unit 130. Also, prediction mode information, motion vector information, etc. for prediction can be encoded together with the residual value in the entropy coding unit 165 and transmitted to the decoder. When using a specific coding mode, it is also possible to directly encode the original block and transmit it to the decoder without generating a prediction block through the prediction units 120 and 125.

[0073] The inter prediction unit 120 can predict a prediction unit based on information of at least one of the previous image or the subsequent image of the current image, and in some cases, can also predict a prediction unit based on information of a part of the region already encoded within the current image. The inter prediction unit 120 may include a reference image interpolation unit, a motion prediction unit, and a motion compensation unit.

[0074] The reference image interpolation unit receives reference image information from the memory 155, and can generate pixel information of integer pixels or fractional pixels from the reference image. For luminance pixels, in order to generate pixel information of fractional pixels in units of 1 / 4 pixels, a DCT-based 8-tap interpolation filter with different filter coefficients (DCT-based Interpolation Filter) can be used. For chrominance signals, in order to generate pixel information of fractional pixels in units of 1 / 8 pixels, a DCT-based 4-tap interpolation filter with different filter coefficients (DCT-based Interpolation Filter) can be used.

[0075] The motion prediction unit can perform motion prediction based on the reference image interpolated by the reference image interpolation unit. Methods for calculating motion vectors can use a variety of methods such as the full search block matching algorithm (FBMA), the three-step search method (TSS), and the new three-step search algorithm (NTS). The motion vector can have a motion vector value in units of 1 / 2 pixel or 1 / 4 pixel based on the interpolated pixels. Different motion prediction methods can be adopted in the motion prediction unit to predict the current prediction unit. Motion prediction methods can use a variety of methods such as the Skip method, the Merge method, the Advanced Motion Vector Prediction (AMVP) method, and the Intra Block Copy method.

[0076] The intra prediction unit 125 can generate a prediction unit based on the reference pixel information around the current block, which is the pixel information within the current image. When the adjacent block of the current prediction unit is a block for which inter prediction has been performed and the reference pixel is a pixel for which inter prediction has been performed, the reference pixel included in the block for which inter prediction has been performed can be used as the reference pixel information for the adjacent block for which intra prediction has been performed around it. That is, when the reference pixel is unavailable, at least one of the available reference pixels can be used to replace the unavailable reference pixel information.

[0077] In intra prediction, the prediction mode can have an angular prediction mode that uses reference pixel information according to the prediction direction and a non-angular mode that does not use direction information when performing prediction. The mode for predicting luminance information and the mode for predicting chrominance information can be different. To predict chrominance information, the intra prediction mode information used for predicting luminance information or the predicted luminance signal information can be applied.

[0078] When performing intra prediction, if the size of the prediction unit is the same as the size of the transform unit, intra prediction can be performed on the prediction unit based on the pixels on the left side of the prediction unit, the pixels in the upper left corner, and the pixels above. However, when performing intra prediction, if the size of the prediction unit is different from the size of the transform unit, intra prediction can be performed using the reference pixels based on the transform unit. Also, intra prediction using only N×N partitioning for the smallest coding unit can be applied.

[0079] An intra prediction method can generate a prediction block after applying an Adaptive Intra Smoothing (AIS) filter to reference pixels according to a prediction mode. The type of the adaptive intra smoothing filter for reference pixels may be different. To perform the intra prediction method, the intra prediction mode of a current prediction unit can be predicted from the intra prediction modes of prediction units existing around the current prediction unit. In a case of predicting the prediction mode of the current prediction unit by using the mode information predicted from the surrounding prediction units, if the intra prediction mode of the current prediction unit is the same as that of the surrounding prediction units, information indicating that the prediction modes of the current prediction unit and the surrounding prediction units are the same can be transmitted by using predetermined flag information. If the intra prediction mode of the current prediction unit is different from that of the surrounding prediction units, the prediction mode information of the current block can be encoded by performing entropy coding.

[0080] Also, a residual block including residual information can be generated, where the residual information is a difference between a prediction unit that performs prediction based on the prediction units generated in the prediction units 120 and 125 and an original block of the prediction unit. The generated residual block can be input to the transform unit 130.

[0081] In the transform unit 130, a transform method such as a Discrete Cosine Transform (DCT) or a Discrete Sine Transform (DST) can be used to transform the residual block, which includes residual value information between the original block and the prediction units generated by the prediction units 120 and 125. Among them, the DCT transform kernel includes at least one of DCT2 or DCT8, and the DST transform kernel includes DST7. It is possible to determine whether to apply DCT or DST to transform the residual block based on the intra prediction mode information of the prediction unit used to generate the residual block. It is also possible to skip the transformation of the residual block. A flag indicating whether the transformation of the residual block is skipped can be encoded. For a residual block, a luminance component, or a chrominance component (below 4:4:4 format) whose size is below a threshold, transformation skipping can be allowed.

[0082] The quantization unit 135 can quantize the values transformed into the frequency domain in the transform unit 130. The quantization coefficient can be changed according to the importance of the block or the video. The values calculated in the quantization unit 135 can be provided to the inverse quantization unit 140 and the rearrangement unit 160.

[0083] The rearrangement unit 160 can perform rearrangement of coefficient values on the quantized residual values.

[0084] The rearrangement unit 160 can change the two-dimensional block shape coefficients into a one-dimensional vector form through the Coefficient Scanning method. For example, the rearrangement unit 160 can scan the DC coefficients and the coefficients in the high-frequency domain using the Zig-Zag Scan method and change them into a one-dimensional vector form. Depending on the size of the transform unit and the intra prediction mode, instead of the Zig-Zag scan, a vertical scan that scans the two-dimensional block shape coefficients along the column direction and a horizontal scan that scans the two-dimensional block shape coefficients along the row direction can also be used. That is, it is possible to determine which of the Zig-Zag scan, the vertical scan, and the horizontal scan to use according to the size of the transform unit and the intra prediction mode.

[0085] The entropy encoding unit 165 can perform entropy encoding based on the values calculated by the rearrangement unit 160. For example, the entropy encoding can use various encoding methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0086] The entropy encoding unit 165 can encode various information such as the residual value coefficient information, block type information, prediction mode information, division unit information, prediction unit information, transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information of the coding units derived from the rearrangement unit 160 and the prediction units 120 and 125.

[0087] The entropy encoding unit 165 can perform entropy encoding on the coefficient values of the coding units input from the rearrangement unit 160.

[0088] The inverse quantization unit 140 and the inverse transform unit 145 perform inverse quantization on the multiple values quantized by the quantization unit 135 and perform inverse transformation on the values transformed by the transform unit 130. The residual values generated in the inverse quantization unit 140 and the inverse transform unit 145 can be merged with the prediction units predicted by the motion prediction unit, the motion compensation unit, and the intra prediction unit included in the prediction units 120 and 125 to generate a Reconstructed Block.

[0089] The filter unit 150 can include at least one of a deblocking filter, an offset correction unit, and an Adaptive Loop Filter (ALF).

[0090] The deblocking filter can remove block distortion generated in the reconstructed image due to the boundaries between blocks. To determine whether to perform deblocking, it is possible to determine whether to apply the deblocking filter to the current block based on the pixels included in several columns or rows included in the block. In the case of applying the deblocking filter to the block, strong filtering or weak filtering can be applied according to the required deblocking filter strength. Also, during the use of the deblocking filter, when performing vertical filtering and horizontal filtering, the horizontal direction filtering and the vertical direction filtering can be processed synchronously.

[0091] The offset correction unit can correct the offset between the video with deblocking performed and the original video on a pixel-by-pixel basis. To perform offset correction on a specified image, the following method can be used, that is, after dividing the pixels included in the video into a predetermined number of regions, determining the region to perform the offset, and applying the offset to the corresponding region, or applying the offset taking into account the edge information of each pixel.

[0092] Adaptive Loop Filtering (ALF) can be performed based on the value obtained by comparing the filtered reconstructed image and the original video. After dividing the pixels included in the video into predetermined groups, filtering can be performed on each group differently by determining a filter to be used for the corresponding group. Information related to whether to apply adaptive loop filtering and the luminance signal can be transmitted by Coding Unit (CU), and the shape and filter coefficients of the adaptive loop filter to be applied can be different according to each block. Also, the same type (fixed type) of adaptive loop filter can be applied regardless of the characteristics of the block to which it is applied.

[0093] The memory 155 can store the reconstructed block or image calculated by the filter unit 150, and can provide the stored reconstructed block or image to the prediction units 120 and 125 during inter-frame prediction.

[0094] Figure 2 is a block diagram of a video decoder according to an embodiment of the present invention.

[0095] Referring to Figure 2 , the video decoder 200 can include an entropy decoding unit 210, a rearrangement unit 215, an inverse quantization unit 220, an inverse transform unit 225, a prediction unit 230, a prediction unit 235, a filter unit 240, and a memory 245.

[0096] When inputting a video bitstream from a video encoder, the input bitstream can be decoded in the reverse steps of the video encoder.

[0097] The entropy decoding unit 210 can perform entropy decoding in the reverse steps of the entropy encoding performed in the entropy encoding unit of the video encoder. For example, corresponding to the methods performed in the video encoder, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) can be applied.

[0098] The entropy decoding unit 210 can decode information related to intra prediction and inter prediction performed by the encoder.

[0099] The rearrangement unit 215 can perform rearrangement based on the method of rearranging the bitstream entropy decoded by the entropy decoding unit 210 in the encoding unit. Multiple coefficients represented in the form of a one-dimensional vector can be reconstructed into a two-dimensional block-shaped coefficient for rearrangement. The rearrangement unit 215 receives information related to the coefficient scanning performed in the encoding unit and can perform rearrangement by means of reverse scanning based on the scanning order performed in the corresponding encoding unit.

[0100] The inverse quantization unit 220 can perform inverse quantization based on the quantization parameter provided by the encoder and the coefficient values of the rearranged blocks.

[0101] The inverse transformation unit 225 can perform inverse discrete cosine transform and inverse discrete sine transform on the quantization result performed by the video encoder. The inverse discrete cosine transform and inverse discrete sine transform belong to the inverse transform of the transform performed in the transformation unit, that is, the inverse transform of the discrete cosine transform and discrete sine transform. Among them, the DCT transform kernel can include at least one of DCT2 or DCT8, and the DST transform kernel can include DST7. Alternatively, if the transformation is skipped in the video encoder, the inverse transformation unit 225 may not perform the inverse transformation. The inverse transformation can be performed based on the transfer unit determined in the video encoder. In the inverse transformation unit 225 of the video decoder, the transformation method (e.g., DCT or DST) can be selectively performed according to multiple information such as the prediction method, the size of the current block, and the prediction direction.

[0102] The prediction units 230 and 235 can generate a prediction block based on the information related to the generation of the prediction block provided by the entropy decoding unit 210 and the previously decoded block or image information provided by the memory 245.

[0103] As described above, when performing intra prediction in the same manner as the operations in the video encoder, if the size of the prediction unit is the same as the size of the transform unit, intra prediction is performed on the prediction unit based on the pixels existing on the left side of the prediction unit, the pixels existing in the upper left, and the pixels existing above. If the size of the prediction unit during intra prediction is different from the size of the transform unit, intra prediction can be performed using the reference pixels based on the transform unit. Also, intra prediction using only N×N partitioning for the smallest coding unit can be applied.

[0104] The prediction units 230 and 235 may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit receives various information such as the prediction unit information input from the entropy decoding unit 210, the prediction mode information of the intra prediction method, and the motion prediction related information of the inter prediction method, classifies the prediction unit according to the current coding unit, and can determine whether the prediction unit is performing inter prediction or intra prediction. The inter prediction unit 230 can use the information required for inter prediction of the current prediction unit provided by the video encoder and perform inter prediction on the current prediction unit based on the information included in at least one of the previous image or the subsequent image of the current image to which the current prediction unit belongs. Alternatively, inter prediction can also be performed based on the information of a part of the region that has been reconstructed within the current image to which the current prediction unit belongs.

[0105] To perform inter prediction, it is possible to determine which mode among the skip mode, merge mode, advanced motion vector prediction mode (AMVP mode), and intra block copy mode is the motion prediction method of the prediction unit included in the corresponding coding unit based on the coding unit.

[0106] The intra prediction unit 235 can generate a prediction block based on the pixel information within the current image. When the prediction unit is a prediction unit that has performed intra prediction, intra prediction can be performed based on the intra prediction mode information of the prediction unit provided by the video encoder. The intra prediction unit 235 may include an adaptive intra smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The adaptive intra smoothing filter is a part that performs filtering on the reference pixels of the current block, and can determine whether to apply the filter according to the prediction mode of the current prediction unit. Adaptive intra smoothing filtering can be performed on the reference pixels of the current block using the prediction mode of the prediction unit provided by the video encoder and the adaptive intra smoothing filter information. If the prediction mode of the current block is a mode that does not perform adaptive intra smoothing filtering, the adaptive intra smoothing filter may not be applied.

[0107] For the reference pixel interpolation unit, if the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on the pixel value obtained by interpolating the reference pixels, reference pixels with integer or fractional pixel units can be generated by interpolating the reference pixels. If the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixels, the reference pixels may not be interpolated. If the prediction mode of the current block is the DC mode, the DC filter can generate a prediction block by filtering.

[0108] The reconstructed block or image can be provided to the filter unit 240. The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

[0109] Information related to whether to apply the deblocking filter to the corresponding block or image and information related to whether to apply strong filtering or weak filtering when applying the deblocking filter can be received from the video encoder. The information related to the deblocking filter provided by the video encoder is received from the deblocking filter of the video decoder, and the deblocking filtering of the corresponding block can be performed in the video decoder.

[0110] The offset correction unit can perform offset correction on the reconstructed video based on the type and offset amount information of the offset correction used for the video during encoding, etc.

[0111] The ALF can be applied to the coding unit based on the information related to whether to apply the ALF provided by the encoder, the ALF coefficient information, etc. Such ALF information can be provided by being included in a specific parameter set.

[0112] The memory 245 stores the reconstructed image or block such that the image or block can be used as a reference image or reference block, and the reconstructed image can be provided to the output unit.

[0113] Figure 3 FIG. is a diagram showing a basic coding tree unit according to an embodiment of the present invention.

[0114] The coding block with the largest size can be defined as the coding tree block. An image can be divided into a plurality of coding tree units (CTUs). The coding tree unit is the coding unit with the largest size and can also be referred to as the largest coding unit (LCU). Figure 3 FIG. shows an example of dividing an image into a plurality of coding tree units.

[0115] The size of the coding tree unit can be defined at the image level or sequence level. For this purpose, information indicating the size of the coding tree unit can be signaled through the image parameter set or the sequence parameter set.

[0116] For example, the size of the coding tree unit for the entire image within the sequence can be set to 128×128. Alternatively, either 128×128 or 256×256 at the image level can be determined as the size of the coding tree unit. For example, the size of the coding tree unit in the first image can be set to 128×128, and the size of the coding tree unit in the second image can be set to 256×256.

[0117] Coding blocks can be generated by partitioning the coding tree unit. A coding block represents the basic unit for performing encoding / decoding processing. For example, prediction or transformation can be performed on different coding blocks, or a predictive coding mode can be determined for different coding blocks. Among them, the predictive coding mode represents a method for generating a predicted image. For example, the predictive coding mode can include Intra Prediction (intra-frame prediction), Inter Prediction (inter-frame prediction), Current Picture Referencing (CPR, or Intra Block Copy (IBC)), or Combined Prediction. For a coding block, at least one of the predictive coding modes of intra-frame prediction, inter-frame prediction, current image reference, or combined prediction can be used to generate a prediction block related to the coding block.

[0118] Information indicating the predictive coding mode of the current block can be signaled through the bitstream. For example, the information can be a 1-bit flag indicating whether the predictive coding mode is an intra mode or an inter mode. Current image reference or combined prediction can be used only when the predictive coding mode of the current block is determined to be an inter mode.

[0119] Current image reference is used to set the current image as a reference image and obtain a prediction block of the current block from the encoded / decoded region within the current image. Here, the current image means the image including the current block. Information indicating whether current image reference is applied to the current block can be signaled through the bitstream. For example, the information can be a 1-bit flag. When the flag is true, the predictive coding mode of the current block can be determined as current image reference, and when the flag is false, the prediction mode of the current block can be determined as inter-frame prediction.

[0120] Alternatively, the prediction coding mode of the current block can be determined based on the reference image index. For example, when the reference image index points to the current image, the prediction coding mode of the current block can be determined as the current image reference. When the reference image index points to an image other than the current image, the prediction coding mode of the current block can be determined as inter-frame prediction. That is, the current image reference is a prediction method using the information of the encoded / decoded region within the current image, and inter-frame prediction is a prediction method using the information of other encoded / decoded images.

[0121] Combined prediction represents a coding mode formed by combining two or more of intra-frame prediction, inter-frame prediction, and current image reference. For example, in the case of applying combined prediction, a first prediction block can be generated based on one of intra-frame prediction, inter-frame prediction, or current image reference, and a second prediction block can be generated based on another. If the first prediction block and the second prediction block are generated, the final prediction block can be generated through the average operation or weighted sum operation of the first prediction block and the second prediction block. Information indicating whether combined prediction is applied can be signaled through the bitstream. The information can be a 1-bit flag.

[0122] Figure 4 It is a diagram showing various partitioning types of coding blocks.

[0123] An encoding block can be divided into multiple encoding blocks based on quadtree partitioning, binary tree partitioning, or ternary tree partitioning. The divided encoding blocks can also be further divided into multiple encoding blocks based on quadtree partitioning, binary tree partitioning, or ternary tree partitioning.

[0124] Quadtree partitioning refers to a partitioning technique that divides the current block into 4 blocks. As a result of quadtree partitioning, the current block can be divided into 4 square partitions (refer to ‘SPLIT_QT’ in part Figure 4 (a)).

[0125] Binary tree partitioning refers to a partitioning technique that divides the current block into 2 blocks. The process of dividing the current block into two blocks along the vertical direction (i.e., using a vertical line crossing the current block) can be called vertical binary tree partitioning, and the process of dividing the current block into two blocks along the horizontal direction (i.e., using a horizontal line crossing the current block) can be called horizontal binary tree partitioning. After binary tree partitioning, the current block can be divided into 2 non-square partitions. Figure 4 “SPLIT_BT_VER” in part (b) represents the result of vertical binary tree partitioning, and Figure 4 “SPLIT_BT_HOR” in part (c) represents the result of horizontal binary tree partitioning.

[0126] A ternary tree partition refers to a partitioning technique that divides the current block into three blocks. The process of dividing the current block into three blocks along the vertical direction (i.e., using two vertical lines that cross the current block) can be called a vertical ternary tree partition, and the process of dividing the current block into three blocks along the horizontal direction (i.e., using two horizontal lines that cross the current block) can be called a horizontal ternary tree partition. After the ternary tree partition, the current block can be divided into three non-square partitions. In this case, the width / height of the partition located at the center of the current block can be twice the width / height of the other partitions. Figure 4 "SPLIT_TT_VER" in part (d) represents the vertical ternary tree partition result, and Figure 4 "SPLIT_TT_HOR" in part (e) represents the horizontal ternary tree partition result.

[0127] The number of times of partitioning of a coding tree unit can be defined as the partitioning depth. The maximum partitioning depth of a coding tree unit can be determined at the sequence or picture level. Thus, the maximum partitioning depth of a coding tree unit can vary according to different sequences or pictures.

[0128] Alternatively, the maximum partitioning depth for each of multiple partitioning techniques can be determined separately. For example, the maximum partitioning depth allowed for quadtree partitioning can be different from the maximum partitioning depth allowed for binary tree partitioning and / or ternary tree partitioning.

[0129] The encoder can signal information representing at least one of the partitioning shape or partitioning depth of the current block through the bitstream. The decoder can determine the partitioning shape and partitioning depth of the coding tree unit based on the information parsed from the bitstream.

[0130] Figure 5 is a diagram showing an example of partitioning of a coding tree unit.

[0131] The process of partitioning a coding block using partitioning techniques such as quadtree partitioning, binary tree partitioning, and / or ternary tree partitioning can be called multi-tree partitioning.

[0132] The coding blocks generated by applying multi-tree partitioning to a coding block can be called multiple downstream coding blocks. When the partitioning depth of a coding block is k, the partitioning depth of multiple downstream coding blocks is set to k + 1.

[0133] On the other hand, for multiple coding blocks with a partitioning depth of k + 1, the coding block with a partitioning depth of k can be called an upstream coding block.

[0134] The partitioning type of a current coding block can be determined based on at least one of the partitioning shape of an upstream coding block or the partitioning type of an adjacent coding block. Here, the adjacent coding block is adjacent to the current coding block, and it can include at least one of the upper adjacent block, the left adjacent block, or the adjacent block adjacent to the upper left corner of the current coding block. The partitioning type can include at least one of whether to partition a quadtree, whether to partition a binary tree, the binary tree partitioning direction, whether to partition a ternary tree, or the ternary tree partitioning direction.

[0135] To determine the partitioning shape of a coding block, information indicating whether the coding block is partitioned can be signaled through a bitstream. The information is a 1-bit flag "split_cu_flag", and when the flag is true, it indicates that the coding block is partitioned by a multi-tree partitioning technique.

[0136] When "split_cu_flag" is true, information indicating whether the coding block is partitioned by a quadtree can be signaled through a bitstream. The information is a 1-bit flag "split_qt_flag", and when the flag is true, the coding block can be partitioned into 4 blocks.

[0137] For example, in Figure 5 the example shown illustrates that as a coding tree unit is partitioned by a quadtree to generate 4 coding blocks with a partitioning depth of 1. And, it is illustrated that the first coding block and the fourth coding block among the 4 coding blocks generated as a result of the quadtree partitioning are again subjected to quadtree partitioning. Finally, 4 coding blocks with a partitioning depth of 2 can be generated.

[0138] Moreover, coding blocks with a partitioning depth of 3 can be generated by applying quadtree partitioning again to the coding blocks with a partitioning depth of 2.

[0139] When no quadtree partitioning is applied to a coding block, it can be determined whether to perform binary tree partitioning or ternary tree partitioning on the coding block by considering at least one of the size of the coding block, whether the coding block is located at the image boundary, the maximum partitioning depth, or the partitioning shape of adjacent blocks. When it is determined to perform binary tree partitioning or ternary tree partitioning on the coding block, information indicating the partitioning direction can be signaled through a bitstream. The information can be a 1-bit flag "mtt_split_cu_vertical_flag". The partitioning direction can be determined to be the vertical direction or the horizontal direction based on the flag. Additionally, information indicating which of binary tree partitioning or ternary tree partitioning is applied to the coding block can be signaled through a bitstream. The information can be a 1-bit flag "mtt_split_cu_binary_flag". It can be determined whether to apply binary tree partitioning or ternary tree partitioning to the coding block based on the flag.

[0140] For example, in Figure 5In the illustrated example, vertical binary tree partitioning is applied to an encoded block with a partitioning depth of 1, vertical ternary tree partitioning is applied to the left encoded block among the encoded blocks generated as the partitioning result, and vertical binary tree partitioning is applied to the right encoded block.

[0141] When implementing a device for encoding or decoding video, there is a problem that it is difficult to process regions larger than a threshold due to hardware performance. For example, there may be a problem that when the hardware performance allows processing at most 4096 samples simultaneously, a data unit of size 64×64 should be redundantly accessed and processed, and data cannot be processed simultaneously for regions with more than 4096 samples. As described above, the basic unit of data processing can be defined as a pipeline-based basic data unit (virtual processing data unit, VPDU, hereinafter referred to as the basic data unit).

[0142] The basic data unit can be divided into square, non-square, or non-rectangular types.

[0143] Figure 6 is a diagram showing the shape of the basic data unit.

[0144] The basic data unit can include samples equal to or less than the maximum number of samples that can be processed simultaneously. For example, as in Figure 6 (a) In the illustrated example, a square block of size 64×64 can be set as the basic data unit. Alternatively, a non-square block can be set as the basic data unit. For example, as in Figure 6 (b) or Figure 6 (c) In the illustrated example, a block of size 32×128 or 64×32 can be set as the basic data unit.

[0145] Although not shown, basic data units of triangle, L-shape, or polygon can be defined.

[0146] Information for determining the basic data unit can be signaled through the bitstream. The information can be used to determine at least one of the size or shape of the basic data unit. Based on the information, it can be determined whether non-square basic data units are allowed or not.

[0147] Alternatively, at least one of the size or shape of the basic data unit can be predefined in the encoder and decoder.

[0148] It is possible to determine whether to allow a partitioning type of a coding block in consideration of the size of a basic data unit. For example, when a coding block generated as a result of partitioning a coding block is larger than the basic data unit, such partitioning may not be allowed. Alternatively, when a non-square coding block generated as a result of partitioning a coding block is larger than the basic data unit, such partitioning may not be allowed. For example, when the width or height of a coding block is greater than a threshold, or when the number of samples included in the coding block is greater than a threshold, binary tree or ternary tree partitioning may not be allowed. Thus, encoding of information related to binary tree or ternary tree partitioning may be omitted.

[0149] Alternatively, it may be set such that a coding block larger than the basic data unit must be partitioned. Alternatively, it may be set such that a binary tree partitioning or a ternary tree partitioning must be performed on a coding block larger than the basic data unit. Thus, for a coding block larger than the basic data unit, even though a flag split_flag indicating whether the coding block is partitioned is not encoded, the value of the flag may be derived as 1.

[0150] As another example, a coding block larger than the basic data unit may be partitioned into a plurality of sub-blocks. Here, the sub-blocks may be set as prediction units that are basic units for prediction, or as transform units that are basic units for transform and / or quantization. In this case, partitioning a coding block into a plurality of prediction units may be defined as VPDU prediction unit partitioning, and partitioning a coding block into a plurality of transform units may be defined as VPDU transform unit partitioning.

[0151] At least one of VPDU prediction unit partitioning and VPDU transform unit partitioning may be applied to a coding block. The partitioning type of a coding block according to the application of VPDU prediction unit partitioning may be set to be the same as the partitioning type of a coding block according to the application of VPDU transform unit partitioning.

[0152] When only VPDU prediction unit partitioning is applied to a coding block, prediction is performed on each sub-block, but transform and / or quantization may be performed on the coding block. In this case, prediction modes such as a prediction coding mode, an intra prediction mode, or an inter prediction mode may be determined for the coding block.

[0153] When only VPDU transform unit partitioning is applied to a coding block, prediction is performed on the sub-blocks, but transform and / or quantization may be performed on each sub-block.

[0154] Figure 7 and Figure 8 are diagrams showing examples of partitioning a coding block into a plurality of sub-blocks.

[0155] Figure 7 is a diagram showing a partitioning pattern when only square basic data units are allowed, Figure 8This is a diagram showing the partitioning patterns when square basic data units and non-square basic data units are allowed.

[0156] Assuming that only square basic data units are allowed, in Figure 7 (a) and Figure 7 (b), CU0 and CU3 are defined as two different VPDUs, and CU1 is defined as four different VPDUs. Therefore, CU0 and CU3 can be partitioned into two sub-blocks, and CU1 can be partitioned into four sub-blocks.

[0157] Assuming that square basic data units and non-square basic data units are allowed, in Figure 8 (a) and Figure 8 (b), CU0 and CU3 can be defined as one VPDU, and CU1 can be defined as using two different VPDUs. Therefore, CU0 and CU3 are not partitioned into sub-blocks, while CU1 can be partitioned into two sub-blocks.

[0158] In this case, CU1 can be partitioned into square sub-blocks or non-square sub-blocks. For example, based on the horizontal line that divides CU1 vertically, CU1 can be partitioned into two square sub-blocks. Alternatively, CU1 can be partitioned into two non-square sub-blocks based on the vertical line that divides CU1 horizontally.

[0159] When there are multiple partitioning type candidates applicable to a coding block, information indicating any one of the multiple partitioning type candidates can be signaled through the bitstream. For example, the information can indicate whether the coding block is partitioned into square sub-blocks or whether the coding block is partitioned into non-square sub-blocks.

[0160] Alternatively, partitioning the coding block into square sub-blocks can be set to have a higher priority than partitioning the coding block into non-square sub-blocks. For example, when the coding block cannot be partitioned into square sub-blocks, partitioning the coding block into non-square sub-blocks can be allowed.

[0161] Alternatively, the partitioning type of a coding block can be determined based on the partitioning type of the coding block as the parent node. For example, when the parent node coding block is partitioned based on a ternary tree, it can be set to partition the coding block into square sub-blocks. On the other hand, when the parent node coding block is partitioned based on a binary tree or a ternary tree, it can be set to partition the coding block into non-square sub-blocks.

[0162] Inter-frame prediction refers to predicting the prediction coding mode of the current block using the information of previous images. For example, a block at the same position as the current block in the previous image (hereinafter referred to as a collocated block) can be set as the prediction block of the current block. Hereinafter, a prediction block generated based on a block at the same position as the current block is referred to as a collocated prediction block.

[0163] On the other hand, if an object existing in the previous image has moved to another position in the current image, the movement of the object can be used to effectively predict the current block. For example, if the movement direction and size of the object can be known by comparing the previous image and the current image, the movement information of the object can be considered to generate a prediction block (or a prediction image) of the current block. Hereinafter, a prediction block generated using movement information can be referred to as a motion prediction block.

[0164] A residual block can be generated by subtracting the prediction block from the current block. In this case, when there is movement of an object, the motion prediction block can be used instead of the collocated prediction block, thereby reducing the energy of the residual block and improving the compression performance of the residual block.

[0165] As described above, the process of generating a prediction block using movement information can be referred to as motion compensation prediction. In most inter-frame predictions, a prediction block can be generated based on motion compensation prediction.

[0166] The movement information can include at least one of a motion vector, a reference image index, a prediction direction, or a bi-directional weighting value index. The motion vector represents the movement direction and size of the object. The reference image index specifies the reference image of the current block among multiple reference images included in the reference image list. The prediction direction refers to any one of unidirectional L0 prediction, unidirectional L1 prediction, or bi-directional prediction (L0 prediction and L1 prediction). At least one of the motion information in the L0 direction or the motion information in the L1 direction can be used according to the prediction direction of the current block. The bi-directional weighting value index specifies the weighting value for the L0 prediction block and the weighting value applied to the L1 prediction block.

[0167] Figure 9 is a flowchart showing an inter-frame prediction method according to an embodiment of the present invention.

[0168] Referring to Figure 9 , the inter-frame prediction method includes: determining an inter-frame prediction mode of the current block (S901); obtaining the movement information of the current block according to the determined inter-frame prediction mode (S902); and performing motion compensation prediction of the current block based on the obtained movement information (S903).

[0169] Among them, the inter-frame prediction mode represents various techniques for determining the motion information of the current block, and may include an inter-frame prediction mode using translational motion information and an inter-frame prediction mode using affine motion information. For example, the inter-frame prediction mode using translational motion information may include a merge mode and an advanced motion vector prediction mode, and the inter-frame prediction mode using affine motion information may include an affine merge mode and an affine motion vector prediction mode. According to the inter-frame prediction mode, the motion information of the current block can be determined based on adjacent blocks adjacent to the current block or information parsed from the bitstream.

[0170] Hereinafter, the inter-frame prediction method using affine motion information will be described in detail.

[0171] Figure 10 It is a diagram showing the non-linear motion of an object.

[0172] The motion of an object within a video may be non-linear motion. For example, as in the Figure 10 example shown, non-linear motion of an object may occur, such as zooming in, zooming out, rotating, affine transformation, etc. of a camera. When non-linear motion of an object occurs, the motion of the object cannot be effectively represented by a translational motion vector. Thus, in the part where non-linear motion of an object occurs, affine motion can be used instead of translational motion, thereby improving the coding efficiency.

[0173] Figure 11 It is a flowchart showing an inter-frame prediction method based on affine motion according to an embodiment of the present invention.

[0174] It can be determined whether to apply an inter-frame prediction technique based on affine motion to the current block based on information parsed from the bitstream. Specifically, it can be determined whether to apply an inter-frame prediction technique based on affine motion to the current block based on at least one of a flag indicating whether to apply an affine merge mode to the current block or a flag indicating whether to apply an affine motion vector prediction mode to the current block.

[0175] When applying an inter-frame prediction technique based on affine motion to the current block, an affine motion model of the current block can be determined (S1101). The affine motion model can be determined by at least one of a 6-parameter affine motion model or a 4-parameter affine motion model. The 6-parameter affine motion model uses 6 parameters to represent affine motion, and the 4-parameter affine motion model uses 4 parameters to represent affine motion.

[0176] Equation 1 is the case of representing affine motion using 6 parameters. Affine motion represents translational motion with respect to a predetermined region determined by an affine seed vector.

[0177]

Equation 1

[0178] vx = ax - by + e

[0179] v y = cx + dy + f

[0180] When representing an affine motion using six parameters, complex motions can be represented, but the number of bits required for encoding each parameter increases, which reduces the encoding efficiency. Thus, an affine motion can also be represented using four parameters. Equation 2 shows the case of representing an affine motion using four parameters.

[0181]

Equation 2

[0182] v x = ax - by + e

[0183] v y = bx + ay + f

[0184] Information for determining the affine motion model of the current block can be encoded and signaled through a bitstream. For example, the information can be a 1-bit flag "affine_type_flag". A value of 0 for the flag indicates that a 4-parameter affine motion model is applied, and a value of 1 for the flag indicates that a 6-parameter affine motion model is applied. The flag can be encoded in units of a slice, a tile, or a block (e.g., a coded block or a coding tree unit). When signaling the flag at the slice level, the affine motion model determined at the slice level can be applied to all blocks belonging to the slice.

[0185] Alternatively, the affine motion model of the current block can be determined based on the affine inter prediction mode of the current block. For example, when the affine merge mode is applied, the affine motion model of the current block can be determined as a 4-parameter motion model. On the other hand, when the affine motion vector prediction mode is applied, information for determining the affine motion model of the current block can be encoded and signaled through a bitstream. For example, when the affine motion vector prediction mode is applied to the current block, the affine motion model of the current block can be determined based on a 1-bit flag "affine_type_flag".

[0186] Next, the affine seed vector of the current block can be derived (S1102). When a 4-parameter affine motion model is selected, motion vectors at two control points of the current block can be derived. On the other hand, when a 6-parameter affine motion model is selected, motion vectors at three control points of the current block can be derived. The motion vectors at the control points can be referred to as affine seed vectors. The control points can include at least one of the upper left corner, the upper right corner, or the lower left corner of the current block.

[0187] Figure 12 is a diagram showing an example of the affine seed vector for each affine motion model.

[0188] In a 4-parameter affine motion model, affine seed vectors related to two of the upper left corner, upper right corner, or lower left corner can be derived. For example, in the example shown in Figure 12 (a), when the 4-parameter affine motion model is selected, the affine seed vector SV0 of the upper left corner (e.g., the upper left sample (x0, y0)) of the current block and the affine seed vector sv1 of the upper right corner (e.g., the upper right sample (x1, y1)) of the current block can be used to derive the affine vector. The affine seed vector related to the lower left corner can also be used to replace the affine seed vector related to the upper left corner, or the affine seed vector related to the lower left corner can also be used to replace the affine seed vector related to the upper right corner.

[0189] In a 6-parameter affine motion model, affine seed vectors related to the upper left corner, upper right corner, and lower left corner can be derived. For example, in the example shown in Figure 12 (b), when the 6-parameter affine motion model is selected, the affine seed vector SV0 of the upper left corner (e.g., the upper left sample (x0, y0)) of the current block, the affine seed vector sv1 of the upper right corner (e.g., the upper right sample (x1, y1)) of the current block, and the affine seed vector sv2 of the lower left corner (e.g., the lower left sample (x2, y2)) of the current block can be used to derive the affine vector.

[0190] In the embodiments described later, in the 4-parameter affine motion model, the affine seed vectors of the upper left control point and the upper right control point are respectively referred to as the first affine seed vector and the second affine seed vector. In the embodiments using the first affine seed vector and the second affine seed vector described later, at least one of the first affine seed vector and the second affine seed vector can be replaced by the affine seed vector of the lower left control point (the third affine seed vector) or the affine seed vector of the lower right control point (the fourth affine seed vector).

[0191] And, in the 6-parameter affine motion model, the affine seed vectors of the upper left control point, the upper right control point, and the lower left control point are respectively referred to as the first affine seed vector, the second affine seed vector, and the third affine seed vector. In the embodiments using the first affine seed vector, the second affine seed vector, and the third affine seed vector described later, at least one of the first affine seed vector, the second affine seed vector, and the third affine seed vector can be replaced by the affine seed vector of the lower right control point (the fourth affine seed vector).

[0192] The affine vector of each sub-block can be derived by using the affine seed vector (S1103). Among them, the affine vector represents the translational motion vector derived based on the affine seed vector. The affine vector of the sub-block can be referred to as the affine sub-block motion vector or the sub-block motion vector.

[0193] Figure 13 A diagram showing an example of the affine vector of a sub-block under a 4-parameter motion model.

[0194] The affine vector of the sub-block can be derived based on the positions of the control points, the position of the sub-block, and the affine seed vector. For example, Equation 3 shows an example of deriving the affine sub-block vector.

[0195]

Equation 3

[0196]

[0197]

[0198] In Equation 3, (x, y) represents the position of the sub-block. Among them, the position of the sub-block represents the position of the reference sample included in the sub-block. The reference sample can be the sample located at the upper left corner of the sub-block, or the sample with at least one of the x-axis or y-axis coordinates located at the central position. (x0, y0) represents the position of the first control point, and (sv 0x , sv 0y ) represents the first affine seed vector. In addition, (x1, y1) represents the position of the second control point, and (sv 1x , sv 1y ) represents the second affine seed vector.

[0199] When the first control point and the second control point correspond to the upper left corner and the upper right corner of the current block respectively, x1 - x0 can be set to the same value as the width of the current block.

[0200] Thereafter, the motion compensation prediction of each sub-block can be performed using the affine vector of each sub-block (S1104). After performing the motion compensation prediction, prediction blocks related to each sub-block can be generated. The prediction block of the sub-block can be set as the prediction block of the current block.

[0201] Next, the inter-frame prediction method using translational motion information will be described in detail.

[0202] The motion information of the current block can be derived from the motion information of other blocks of the current block. Among them, the other blocks can be the blocks that are encoded / decoded by inter-frame prediction with higher priority than the current block. Defining the situation where the motion information of the current block is set to be the same as the motion information of the other blocks as the merge mode. And defining the situation where the motion vector of the other blocks is set as the predicted value of the motion vector of the current block as the motion vector prediction mode.

[0203] Figure 14 A flowchart showing the process of deriving the motion information of the current block using the merge mode.

[0204] The merge candidates of the current block can be derived (S1401). The merge candidates of the current block can be derived from the blocks that are encoded / decoded using inter prediction before the current block.

[0205] Figure 15 FIG. is a diagram showing candidate blocks for deriving merge candidates.

[0206] The candidate blocks may include at least one of an adjacent block including samples adjacent to the current block or a non-adjacent block including samples not adjacent to the current block. Hereinafter, the samples used to determine the candidate blocks are designated as reference samples. In addition, the reference samples adjacent to the current block are called adjacent reference samples, and the reference samples not adjacent to the current block are called non-adjacent reference samples.

[0207] The adjacent reference samples may be included in the adjacent column of the leftmost column of the current block or the adjacent row of the uppermost row of the current block. For example, if the coordinates of the upper left sample of the current block are (0, 0), then at least one of the blocks including the reference sample at the position (-1, H-1), the block including the reference sample at the position (W-1, -1), the block including the reference sample at the position (W, -1), the block including the reference sample at the position (-1, H), or the block including the reference sample at the position (-1, -1) can be used as a candidate block. Referring to the accompanying drawings, the adjacent blocks with indices 0 to 4 can be used as candidate blocks.

[0208] The non-adjacent reference samples represent samples in which at least one of the x-axis distance or the y-axis distance between the reference samples adjacent to the current block has a predefined value. For example, at least one of the blocks including the reference sample with a predefined x-axis distance from the left reference sample, the block including the non-adjacent sample with a predefined y-axis distance from the upper reference sample, or the block including the non-adjacent sample with a predefined x-axis distance and y-axis distance from the upper left reference sample can be used as a candidate block. The predefined value can be an integer such as 4, 8, 12, 16, etc. Referring to the accompanying drawings, at least one of the blocks with indices 5 to 26 can be used as a candidate block.

[0209] Samples that are not on the same vertical line, horizontal line, or diagonal line as the adjacent reference samples can be set as non-adjacent reference samples.

[0210] Figure 16 FIG. is a diagram showing the positions of the reference samples.

[0211] As in Figure 16In the example shown, the x coordinate of the upper non - adjacent reference sample can be set to be different from the x coordinate of the upper adjacent reference sample. For example, when the position of the upper adjacent reference sample is (W - 1, - 1), the position of the upper non - adjacent reference sample separated from the upper adjacent reference sample by N along the y - axis can be set to ((W / 2) - 1, - 1 - N), and the position of the upper non - adjacent reference sample separated from the upper adjacent reference sample by 2N along the y - axis can be set to (0, - 1 - 2N). That is, the position of the non - adjacent reference sample can be determined based on the position of the adjacent reference sample and the distance from the adjacent reference sample.

[0212] Hereinafter, a candidate block including an adjacent reference sample in the candidate blocks is referred to as an adjacent block, and a block including a non - adjacent reference sample is referred to as a non - adjacent block.

[0213] When the distance between the current block and the candidate block is greater than or equal to a threshold, the candidate block can be set as not available for merge candidates. The threshold can be determined based on the size of the coding tree unit. For example, the threshold can be set to the height of the coding tree unit (ctu_height), or a value obtained by adding or subtracting an offset value to or from the height of the coding tree unit (e.g., ctu_height ± N). The offset value N is a predefined value in the encoder and decoder, and can be set to 4, 8, 16, 32, or ctu_height.

[0214] When the difference between the y - axis coordinate of the current block and the y - axis coordinate of the samples included in the candidate block is greater than the threshold, the candidate block can be determined as not available for merge candidates.

[0215] Alternatively, a candidate block that does not belong to the same coding tree unit as the current block can be set as not available for merge candidates. For example, when the reference sample exceeds the upper boundary of the coding tree unit to which the current block belongs, the candidate block including the reference sample can be set as not available for merge candidates.

[0216] If, when the upper boundary of the current block is adjacent to the upper boundary of the coding tree unit, multiple candidate blocks are determined as not available for merge candidates, this will reduce the encoding / decoding efficiency of the current block. To solve the above problem, candidate blocks can be set such that the number of candidate blocks located above the current block is more than the number of candidate blocks located to the left of the current block.

[0217] Figure 17 is a diagram showing an example of candidate blocks for deriving merge candidates.

[0218] As in Figure 17In the example shown, the upper block belonging to N block columns above the current block and the left block belonging to M block columns to the left of the current block can be set as candidate blocks. At this time, by setting M to be greater than N, the number of left candidate blocks can be set to be greater than the number of upper candidate blocks.

[0219] For example, the difference between the y-axis coordinate of the reference sample in the current block and the y-axis coordinate of the upper block that can be used as a candidate block can be set to not exceed N times the height of the current block. Additionally, the difference between the x-axis coordinate of the reference sample in the current block and the x-axis coordinate of the left block that can be used as a candidate block can be set to not exceed M times the width of the current block.

[0220] For example, in Figure 17 the example shown, the blocks belonging to two block columns above the current block and the blocks belonging to five block columns to the left of the current block are shown as candidate blocks.

[0221] As another example, when a candidate block does not belong to the same coding tree unit as the current block, a block belonging to the same coding tree unit as the current block, or a block containing a reference sample adjacent to the boundary of the coding tree unit can be used instead of the candidate block to derive merge candidates.

[0222] Figure 18 is a diagram showing an example of changing the position of a reference sample.

[0223] When a reference sample is included in a coding tree unit different from the current block and the reference sample is not adjacent to the boundary of the coding tree unit, a reference sample adjacent to the boundary of the coding tree unit can be used instead of the reference sample to determine the candidate block reference sample.

[0224] For example, in Figure 18 (a) and Figure 18 (b) the example shown, when the upper boundary of the current block and the upper boundary of the coding tree unit are in contact with each other, the reference sample above the current block belongs to a coding tree unit different from the current block. The reference sample that is not adjacent to the upper boundary of the coding tree unit among the reference samples belonging to a coding tree unit different from the current block can be replaced with a sample adjacent to the upper boundary of the coding tree unit.

[0225] For example, in the example shown in Figure 18 (a), the reference sample at position 6 is replaced with the sample at position 6' on the upper boundary of the coding tree unit, as shown in Figure 18As shown in the example of (b), the reference sample at position 15 is replaced by a sample at position 15' on the upper boundary of the coding tree unit. At this time, the y coordinate of the replacement sample can be changed to an adjacent position of the coding tree unit, and the x coordinate of the replacement sample can be set to be the same as that of the reference sample. For example, the sample at position 6' can have the same x coordinate as the sample at position 6, and the sample at position 15' can have the same x coordinate as the sample at position 15.

[0226] Alternatively, the value obtained by adding or subtracting an offset value to / from the x coordinate of the reference sample can be set as the x coordinate of the replacement sample. For example, when the x coordinates of adjacent and non - adjacent reference samples above the current block are the same, the value obtained by adding or subtracting an offset value to / from the x coordinate of the reference sample can be set as the x coordinate of the replacement sample. This is to prevent the replacement sample for replacing a non - adjacent reference sample from being in the same position as other non - adjacent reference samples or adjacent reference samples.

[0227] Figure 19 is a diagram showing an example of changing the position of the reference sample.

[0228] When replacing a reference sample included in a coding tree unit different from the current block and not adjacent to the boundary of the coding tree unit with a sample on the boundary of the coding tree unit, the value obtained by adding or subtracting an offset value to / from the x coordinate of the reference sample can be set as the x coordinate of the replacement sample.

[0229] For example, in Figure 19 the example shown, the reference sample at position 6 and the reference sample at position 15 can be respectively replaced by a sample at position 6' and a sample at position 15' having the same y coordinate as the row adjacent to the upper boundary of the coding tree unit. At this time, the x coordinate of the sample at position 6' can be set to a value whose difference from the x coordinate of the reference sample at position 6 is W / 2, and the x coordinate of the sample at position 15' can be set to a value whose difference from the x coordinate of the reference sample at position 15 is W - 1.

[0230] Different from Figure 18 and Figure 19 the examples shown, the y coordinate of the row above the top - most row of the current block or the y coordinate of the upper boundary of the coding tree unit can also be set as the y coordinate of the replacement sample.

[0231] Although not illustrated, samples that replace the reference samples may also be determined based on the left boundary of the coding tree unit. For example, when the reference samples are not included in the same coding tree unit as the current block and are not adjacent to the left boundary of the coding tree unit, the reference samples may be replaced with samples adjacent to the left boundary of the coding tree unit. At this time, the replacement samples may have the same y coordinate as the reference samples, or may have a y coordinate obtained by adding or subtracting an offset value to the y coordinate of the reference samples.

[0232] After that, the block containing the replacement samples may be set as a candidate block, and merge candidates for the current block may be derived based on the candidate block.

[0233] Merge candidates may also be derived from temporally adjacent blocks included in an image different from the current block. For example, merge candidates may be derived from co-located blocks included in a co-located image.

[0234] The motion information of the merge candidates may be set to be the same as the motion information of the candidate block. For example, at least one of the motion vector, reference image index, prediction direction, or bi-directional weight value index of the candidate block may be set as the motion information of the merge candidates.

[0235] A merge candidate list including the merge candidates may be generated (S1402). The merge candidates may be classified into adjacent merge candidates derived from adjacent blocks adjacent to the current block and non-adjacent merge candidates derived from non-adjacent blocks.

[0236] Indices of multiple merge candidates within the merge candidate list may be assigned in a predetermined order. For example, the index assigned to an adjacent merge candidate may have a smaller value than the index assigned to a non-adjacent merge candidate. Alternatively, based on Figure 15 or Figure 17 the index of each block shown, the index may be assigned to each merge candidate.

[0237] When multiple merge candidates are included in the merge candidate list, at least one of the multiple merge candidates may be selected (S1403). At this time, it may be signaled through the bitstream whether the motion information of the current block is information derived from adjacent merge candidates. The information may be a 1-bit flag. For example, it may be signaled through the bitstream a syntax element isAdjancentMergeFlag indicating whether the motion information of the current block is derived from adjacent merge candidates. When the value of the syntax element isAdjancentMergeFlag is 1, the motion information of the current block may be derived based on adjacent merge candidates. On the other hand, when the value of the syntax element isAdjancentMergeFlag is 0, the motion information of the current block may be derived based on non-adjacent merge candidates.

[0238] Table 1 shows a syntax table including the syntax element isAdjancentMergeFlag.

[0239]

Table 1

[0240]

[0241]

[0242] Information for specifying any one of multiple merge candidates can be signaled through the bitstream. For example, information indicating any one index among the merge candidates included in the merge candidate list can be signaled through the bitstream.

[0243] When isAdjacentMergeflag is 1, the syntax element merge_idx for determining any one of the adjacent merge candidates can be signaled. The maximum value of the syntax element merge_idx can be set to a value with a difference of 1 from the number of adjacent merge candidates.

[0244] When isAdjacentMergeflag is 0, the syntax element NA_merge_idx for determining any one of the non - adjacent merge candidates can be signaled. The syntax element NA_merge_idx indicates a value obtained by subtracting the index of the non - adjacent merge candidate from the number of adjacent merge candidates. The decoder can select a non - adjacent merge candidate by adding the index determined according to NA_merge_idx to the number of adjacent merge candidates.

[0245] When the number of merge candidates included in the merge candidate list is less than a threshold, the merge candidates included in the inter - frame motion information list can be added to the merge candidate list. Here, the threshold can be the maximum number of merge candidates that the merge candidate list can include or a value obtained by subtracting an offset from the maximum number of merge candidates. The offset can be an integer such as 1 or 2. The inter - frame motion information list can include merge candidates derived based on blocks encoded / decoded before the current block.

[0246] The inter - frame motion information list includes merge candidates derived from blocks encoded / decoded for intra - picture base inter - prediction. For example, the motion information of the merge candidates included in the inter - frame motion information list can be set to be the same as the motion information of the blocks encoded / decoded based on inter - frame prediction. Here, the motion information can include at least one of a motion vector, a reference image index, a prediction direction, or a bi - directional weighted value index.

[0247] For ease of explanation, the merge candidates included in the inter - frame motion information list are referred to as inter - frame merge candidates.

[0248] The maximum number of merge candidates that can be included in the inter - frame motion information list can be predefined in the encoder and decoder. For example, the maximum number of merge candidates that can be included in the inter - frame motion information list can be 1, 2, 3, 4, 5, 6, 7, 8 or greater (e.g., 16).

[0249] Alternatively, information representing the maximum number of merge candidates of the inter - frame motion information list can be signaled through the bitstream. This information is signaled at the sequence level, picture level, or slice level.

[0250] Alternatively, the maximum number of merge candidates of the inter - frame motion information list can be determined according to the size of the picture, the size of the slice, or the size of the coding tree unit.

[0251] The inter - frame motion information list can be initialized on a per - picture, per - slice, per - tile, per - brick, per - coding tree unit, or per - coding tree unit row (row or column) basis. For example, when a slice is initialized, the inter - frame motion information list is also initialized, and it is possible that the inter - frame motion information list does not include any merge candidates.

[0252] Alternatively, information indicating whether to initialize the inter - frame motion information list can also be signaled through the bitstream. This information can be signaled at the slice level, tile level, brick level, or block level. Before the information indicates the initialization of the inter - frame motion information list, the configured inter - frame motion information list can be used.

[0253] Alternatively, information related to inter - frame merge candidates can be signaled through the picture parameter set or slice header. Even if a slice is initialized, the inter - frame motion information list can include initial inter - frame merge candidates. Thus, the inter - frame merge candidates can be used for the first - encoded / decoded blocks within the slice.

[0254] According to the encoding / decoding order, blocks are encoded / decoded, and multiple blocks encoded / decoded based on inter - frame prediction can be sequentially set as inter - frame merge candidates according to the encoding / decoding order.

[0255] Figure 20 is a flowchart showing the process for updating the inter - frame motion information list.

[0256] When performing inter - frame prediction on the current block (S2001), inter - frame merge candidates can be derived based on the current block (S2002). The motion information of the inter - frame merge candidates can be set to be the same as the motion information of the current block.

[0257] When the inter - frame motion information list is empty (S2003), the inter - frame merge candidates derived based on the current block can be added to the inter - frame motion information list (S2004).

[0258] When the inter-frame motion information list already includes the inter-frame merge candidate (S2003), a redundancy check can be performed on the motion information of the current block (or the inter-frame merge candidate derived based on the current block) (S2005). The redundancy check is used to determine whether the motion information of the inter-frame merge candidate already stored in the inter-frame motion information list is the same as the motion information of the current block. All the inter-frame merge candidates already stored in the inter-frame motion information list can be used as objects to perform the redundancy check. Alternatively, the inter-frame merge candidates with indexes above or below a threshold among the inter-frame merge candidates already stored in the inter-frame motion information list can be used as objects to perform the redundancy check.

[0259] When there is no inter-frame merge candidate having the same motion information as the motion information of the current block, the inter-frame merge candidate derived based on the current block can be added to the inter-frame motion information list (S2008). It can be determined whether inter-frame merge candidates are the same based on whether their motion information (e.g., motion vector and / or reference image index, etc.) is the same.

[0260] In this case, when the maximum number of inter-frame merge candidates has already been stored in the inter-frame motion information list (S2006), the earliest inter-frame merge candidate is deleted (S2007), and the inter-frame merge candidate derived based on the current block can be added to the inter-frame motion information list (S2008).

[0261] Multiple inter-frame merge candidates can be identified according to respective indexes. When adding the inter-frame merge candidate derived from the current block to the inter-frame motion information list, the lowest index (e.g., 0) is assigned to the inter-frame merge candidate, and the indexes of the already stored inter-frame merge candidates can be incremented by 1 respectively. In this case, when the maximum number of inter-frame merge candidates has already been stored in the inter-frame motion information list, the inter-frame merge candidate with the largest index is removed.

[0262] Alternatively, when adding the inter-frame merge candidate derived from the current block to the inter-frame motion information list, the largest index can be assigned to the inter-frame merge candidate. For example, when the number of inter-frame merge candidates already stored in the inter-frame motion information list is less than the maximum value, an index with the same value as the number of already stored inter-frame merge candidates can be assigned to the inter-frame merge candidate. Alternatively, when the number of inter-frame merge candidates already stored in the inter-frame motion information list is equal to the maximum value, an index obtained by subtracting 1 from the maximum value can be assigned to the inter-frame merge candidate. Also, the inter-frame merge candidate with the smallest index is removed, and the indexes of the remaining multiple already stored inter-frame merge candidates are decreased by 1 respectively.

[0263] Figure 21 It is a diagram showing an embodiment of updating the inter-frame merge candidate list.

[0264] Assume that the inter-frame merge candidates derived from the current block are added to the inter-frame merge candidate list, and the maximum index is assigned to the inter-frame merge candidates. Also, assume that the maximum number of inter-frame merge candidates has been stored in the inter-frame merge candidate list.

[0265] When adding the inter-frame merge candidate HmvpCand[n + 1] derived from the current block to the inter-frame merge candidate list HmvpCandList, the inter-frame merge candidate HmvpCand[0] with the smallest index is deleted from the stored inter-frame merge candidates, and the indices of the remaining multiple inter-frame merge candidates are each decreased by 1. Additionally, the index of the inter-frame merge candidate HmvpCand[n + 1] derived from the current block can be set to the maximum value (n in the Figure 21 illustrated example).

[0266] When the same inter-frame merge candidate as the inter-frame merge candidate derived from the current block is already stored (S2005), the inter-frame merge candidate derived from the current block may not be added to the inter-frame motion information list (S2009).

[0267] Alternatively, as the inter-frame merge candidate derived from the current block is added to the inter-frame motion information list, the stored inter-frame merge candidate that is the same as the inter-frame merge candidate may also be removed. In this case, the effect of updating the indices of the stored inter-frame merge candidates will occur.

[0268] Figure 22 It is a diagram showing an example of updating the indices of the stored inter-frame merge candidates.

[0269] When the index of the stored inter-frame merge candidate that is the same as the inter-frame merge candidate mvCand derived from the current block is hIdx, deleting the stored inter-frame merge candidate, the indices of the inter-frame merge candidates with indices greater than hIdx can each be decreased by 1. For example, in the Figure 22 illustrated example, it shows deleting HmvpCand[2] that is the same as mvCand from the inter-frame motion information list HvmpCandList, and the indices of HmvpCand[3] to HmvpCand[n] are each decreased by 1.

[0270] And the inter-frame merge candidate mvCand derived from the current block can be added to the end of the inter-frame motion information list.

[0271] Alternatively, the index assigned to the stored inter-frame merge candidate that is the same as the inter-frame merge candidate derived from the current block can be updated. For example, the index of the stored inter-frame merge candidate can be changed to the minimum value or the maximum value.

[0272] The motion information of blocks included in a predetermined region can be set such that it cannot be added to the inter-frame motion information list. For example, an inter-frame merge candidate derived based on the motion information of blocks included in a merge processing region cannot be added to the inter-frame motion information list. Since the encoding / decoding order of blocks included in the merge processing region is not defined, it is not appropriate to use the motion information of any of these blocks for inter-frame prediction of other blocks. Thus, an inter-frame merge candidate derived based on blocks included in the merge processing region may not be added to the inter-frame motion information list.

[0273] When performing motion compensation prediction by means of sub-block units, an inter-frame merge candidate can be derived based on the motion information of a representative sub-block among a plurality of sub-blocks included in the current block. For example, when using a sub-block merge candidate for the current block, an inter-frame merge candidate can be derived based on the motion information of the representative sub-block in the sub-blocks.

[0274] The motion vector of a sub-block can be derived in the following order. First, any one of the merge candidates included in the merge candidate list of the current block can be selected, and an initial shift vector (shVector) can be derived based on the motion vector of the selected merge candidate. And, by adding the position (xSb, ySb) of a reference sample (e.g., the upper-left sample or the middle-position sample) of each sub-block within the coded block to the initial shift vector, a shifted sub-block with a reference sample position of (xColSb, yColSb) can be derived. The following Equation 4 shows the equation for deriving the shifted sub-block.

[0275]

Equation 4

[0276] (xColSb, ycolSb) = (xSb + shVector[O] >> 4, ySb + shVector[1] >> 4)

[0277] Next, the motion vector of the co-located block corresponding to the center position of the sub-block including (xColSb, yColSb) is set as the motion vector of the sub-block including (xSb, ySb).

[0278] The representative sub-block may mean a sub-block including the upper-left sample or the center sample of the current block.

[0279] Figure 23 is a diagram showing the position of the representative sub-block.

[0280] Figure 23 (a) shows an example of setting the sub-block located in the upper-left of the current block as the representative sub-block, Figure 23(b) shows an example in which a sub-block located at the center of the current block is set as a representative sub-block. When performing motion compensation prediction in units of sub-blocks, based on the motion vector of the sub-block including the upper left sample of the current block or the sub-block including the center sample of the current block, the inter-frame merge candidate of the current block can be derived.

[0281] Based on the inter-frame prediction mode of the current block, it can also be determined whether to use the current block as an inter-frame merge candidate. For example, a block encoded / decoded based on an affine motion model can be set as not available as an inter-frame merge candidate. Thus, even if the current block is encoded / decoded by inter-frame prediction and the inter-frame prediction mode of the current block is an affine prediction mode, the inter-frame prediction motion information list is not updated based on the current block.

[0282] Alternatively, the inter-frame merge candidate can also be derived based on at least one sub-block vector in the sub-blocks included in the block encoded / decoded based on the affine motion model. For example, the sub-block located in the upper left of the current block, the sub-block located in the center, or the sub-block located in the upper right can be used to derive the inter-frame merge candidate. Alternatively, the average value of the sub-block vectors of multiple sub-blocks can also be set as the motion vector of the inter-frame merge candidate.

[0283] Alternatively, the inter-frame merge candidate can also be derived based on the average value of the affine seed vectors of the block encoded / decoded based on the affine motion model. For example, the average value of at least one of the first affine seed vector, the second affine seed vector, or the third affine seed vector of the current block is set as the motion vector of the inter-frame merge candidate.

[0284] Alternatively, the inter-frame motion information list can be configured for different inter-frame prediction modes. For example, at least one of the inter-frame motion information list for the block encoded / decoded by intra block copy, the inter-frame motion information list for the block encoded / decoded based on the translational motion model, or the inter-frame motion information list for the block encoded / decoded based on the affine motion model can be defined. According to the inter-frame prediction mode of the current block, any one of the multiple inter-frame motion information lists can be selected.

[0285] Figure 24 A diagram showing an example of generating an inter-frame motion information list for each inter-frame prediction mode.

[0286] When encoding / decoding a block based on a non-affine motion model, the inter-frame merge candidate mvCand derived based on the block can be added to the inter-frame non-affine motion information list HmvpCandList. On the other hand, when encoding / decoding a block based on an affine motion model, the inter-frame merge candidate mvAfCand derived based on the block can be added to the inter-frame affine motion information list HmvpAfCandList.

[0287] The affine seed vectors of blocks can be stored in inter - prediction merge candidates derived from blocks encoded / decoded based on an affine motion model. Thus, the inter - prediction merge candidates can be used as merge candidates for deriving the affine seed vectors of the current block.

[0288] In addition to the described list of inter - frame motion information, another list of inter - frame motion information can be defined. In addition to the described list of inter - frame motion information (hereinafter referred to as the first list of inter - frame motion information), a long - term motion information list (hereinafter referred to as the second list of inter - frame motion information) can be defined. Among them, the long - term motion information list includes long - term merge candidates.

[0289] When both the first list of inter - frame motion information and the second list of inter - frame motion information are in an empty state, first, the inter - prediction merge candidates can be added to the second list of inter - frame motion information. After the number of available inter - prediction merge candidates in the second list of inter - frame motion information reaches the maximum number, the inter - prediction merge candidates can be added to the first list of inter - frame motion information.

[0290] Alternatively, an inter - prediction merge candidate can also be added to both the second list of inter - frame motion information and the first list of inter - frame motion information.

[0291] In this case, the second list of inter - frame motion information that has been configured may no longer be updated. Alternatively, when the decoded area is more than a predetermined ratio of the slice, the second list of inter - frame motion information can be updated. Alternatively, the second list of inter - frame motion information can be updated every N coding tree unit rows.

[0292] On the other hand, whenever a block encoded / decoded by inter - frame prediction is generated, the first list of inter - frame motion information can be updated. However, the inter - prediction merge candidates added to the second list of inter - frame motion information can also be set not to be used for updating the first list of inter - frame motion information.

[0293] Information for selecting either the first list of inter - frame motion information or the second list of inter - frame motion information can be signaled through the bitstream. When the number of merge candidates included in the merge candidate list is less than the threshold, the merge candidates included in the inter - frame motion information list indicated by the information can be added to the merge candidate list.

[0294] Alternatively, the inter - frame motion information list can also be selected based on the size, shape, inter - frame prediction mode, whether it is bi - directional prediction, whether motion vector refinement is performed, or whether triangular partitioning is performed of the current block.

[0295] Alternatively, even when the inter - prediction merge candidates included in the first list of inter - frame motion information are added, if the number of merge candidates included in the merge candidate list is still less than the maximum merge number, the inter - prediction merge candidates included in the second list of inter - frame motion information can be added to the merge candidate list.

[0296] Figure 25 It is a diagram showing an example of adding an inter - frame merge candidate included in the long - term motion information list to the merge candidate list.

[0297] When the number of merge candidates included in the merge candidate list is less than the maximum number, the inter - frame merge candidates included in the first inter - frame motion information list HmvpCandList can be added to the merge candidate list. Even when adding the inter - frame merge candidates included in the first inter - frame motion information list to the merge candidate list and the number of merge candidates included in the merge candidate list is still less than the maximum number, the inter - frame merge candidates included in the long - term motion information list HmvpLTCandList can be added to the merge candidate list.

[0298] Table 2 shows the process of adding the inter - frame merge candidates included in the long - term motion information list to the merge candidate list.

[0299]

Table 2

[0300]

[0301]

[0302] The inter - frame merge candidate can be set to include additional information in addition to the motion information. For example, the size, shape of the storage block, or block division information can be added to the inter - frame merge candidate. When constructing the merge candidate list of the current block, only the inter - frame merge candidates with the same or similar size, shape, or division information as the current block are used in the inter - frame merge candidate, or the inter - frame merge candidates with the same or similar size, shape, or division information as the current block can be preferentially added to the merge candidate list.

[0303] Alternatively, inter - frame motion information lists can be generated for different block sizes, shapes, or division information. The merge candidate list of the current block can be generated using the inter - frame motion information list corresponding to the shape, size, or division information of the current block in multiple inter - frame motion information lists.

[0304] When the number of merge candidates included in the merge candidate list of the current block is less than the threshold, the inter - frame merge candidates included in the inter - frame motion information list can be added to the merge candidate list. The addition process is performed in ascending or descending order of the index. For example, the inter - frame merge candidate with the largest index can be added to the merge candidate list.

[0305] When adding the inter - frame merge candidates included in the inter - frame motion information list to the merge candidate list, a redundancy check can be performed between the inter - frame merge candidates and the multiple merge candidates already stored in the merge candidate list.

[0306] For example, Table 3 shows the process of adding an inter-frame merge candidate to the merge candidate list.

[0307] [[Table 3]]

[0308]

[0309] Redundancy checks may also be performed only on some of the inter-frame merge candidates included in the inter-frame motion information list. For example, redundancy checks may be performed only on inter-frame merge candidates with an index above or below a threshold. Alternatively, redundancy checks may be performed only on the N merge candidates with the largest index or the N merge candidates with the smallest index.

[0310] Alternatively, redundancy checks may be performed only on some of the merge candidates already stored in the merge candidate list. For example, redundancy checks may be performed only on merge candidates with an index above or below a threshold or merge candidates derived from blocks at a specific position. Among them, the specific position may include at least one of the left adjacent block, the upper adjacent block, the upper right adjacent block, or the lower left adjacent block of the current block.

[0311] Figure 26 is a diagram showing an example of performing redundancy checks only on some merge candidates.

[0312] When adding the inter-frame merge candidate HmvpCand[j] to the merge candidate list, a redundancy check may be performed between the inter-frame merge candidate and the two merge candidates with the largest index, mergeCandList[NumMerge - 2] and mergeCandList[NumMerge - 1]. Among them, NumMerge may represent the number of available spatial merge candidates and temporal merge candidates.

[0313] Different from the example shown in the figure, when adding the inter-frame merge candidate HmvpCand[j] to the merge candidate list, a redundancy check may also be performed between the inter-frame merge candidate and the two merge candidates with the smallest index. For example, it can be confirmed whether mergeCandList[0] and mergeCandList[1] are the same as HmvpCand[j]. Alternatively, redundancy checks may be performed only on the merge candidates derived from a specific position. For example, redundancy checks may be performed on at least one of the merge candidates derived from the adjacent block on the left side of the current block or the adjacent block above the current block. When there are no merge candidates derived from a specific position in the merge candidate list, the inter-frame merge candidate may be added to the merge candidate list without performing a redundancy check.

[0314] In the case where a merge candidate identical to the first inter-frame merge candidate is found, when performing a redundancy check on the second inter-frame merge candidate, the redundancy check for the merge candidate identical to the first inter-frame merge candidate can be omitted.

[0315] Figure 27 FIG. is an example showing omission of the redundancy check for a specific merge candidate.

[0316] When adding the inter-frame merge candidate HmvpCand[i] with index i to the merge candidate list, a redundancy check can be performed between the inter-frame merge candidate and the merge candidates already stored in the merge candidate list. In this case, in the case where a merge candidate mergeCandList[j] identical to the inter-frame merge candidate HmvpCand[i] is found, the inter-frame merge candidate HmvpCand[i] will not be added to the merge candidate list, and a redundancy check can be performed between the inter-frame merge candidate HmvpCand[i-1] with index i-1 and the merge candidates. In this case, the redundancy check between the inter-frame merge candidate HmvpCand[i-1] and the merge candidate mergeCandList[j] can be omitted.

[0317] For example, in Figure 27 In the example shown, it is determined that HmvpCand[i] is identical to mergeCandList[2]. Accordingly, HmvpCand[i] is not added to the merge candidate list, and a redundancy check can be performed on HmvpCand[i-1]. In this case, the redundancy check between HvmpCand[i-1] and mergeCandList[2] can be omitted.

[0318] When the number of merge candidates included in the merge candidate list of the current block is less than the threshold, in addition to the inter-frame merge candidate, at least one of pairwise merge candidates or zero merge candidates can be included. A pairwise merge candidate refers to a merge candidate that uses the average of the motion vectors of two or more merge candidates as the motion vector, and a zero merge candidate refers to a merge candidate with a motion vector of 0.

[0319] The merge candidate list of the current block can add merge candidates in the following order.

[0320] Spatial merge candidate - Temporal merge candidate - Inter-frame merge candidate - (Inter-frame affine merge candidate) - Pairwise merge candidate - Zero merge candidate.

[0321] A spatial merge candidate refers to a merge candidate derived from at least one of adjacent blocks or non-adjacent blocks, and a temporal merge candidate refers to a merge candidate derived from a previous reference image. The inter-frame affine merge candidate list refers to inter-frame merge candidates derived from blocks encoded / decoded using an affine motion model.

[0322] The inter-frame motion information list can also be used in the advanced motion vector prediction mode. For example, when the number of motion vector prediction candidates included in the motion vector prediction candidate list of the current block is less than a threshold, the inter-frame merge candidates included in the inter-frame motion information list are set as the motion vector prediction candidates related to the current block. Specifically, the motion vectors of the inter-frame merge candidates are set as the motion vector prediction candidates.

[0323] If any one of the motion vector prediction candidates included in the motion vector prediction candidate list of the current block is selected, the selected candidate is set as the motion vector prediction value of the current block. After decoding the motion vector residual value of the current block, the motion vector of the current block can be obtained by adding the motion vector prediction value and the motion vector residual value.

[0324] The motion vector prediction candidate list of the current block can be constructed in the following order.

[0325] Spatial motion vector prediction candidate - Temporal motion vector prediction candidate - Inter-frame decoded region merge candidate - (Inter-frame decoded region affine merge candidate) - Zero motion vector prediction candidate.

[0326] The spatial motion vector prediction candidate refers to the motion vector prediction candidate derived from at least one of the adjacent blocks or non-adjacent blocks, and the temporal motion vector prediction candidate refers to the motion vector prediction candidate derived from the previous reference image. The inter-frame affine merge candidate list represents the inter-frame motion vector prediction candidates derived from the blocks encoded / decoded with the affine motion model. The zero motion vector prediction candidate represents the candidate with a motion vector value of 0.

[0327] When selecting the merge candidate of the current block, the motion vector of the selected merge candidate is set as the initial motion vector, and the motion compensation prediction of the current block can be performed using the motion vector derived by adding or subtracting the offset vector to / from the initial motion vector. Deriving a new motion vector by adding or subtracting the offset vector to / from the motion vector of the merge candidate can be defined as the merge motion difference coding method.

[0328] Information indicating whether to use the merged offset coding method can be signaled through the bitstream. The information can be a 1-bit flag, merge_offset_vector_flag. For example, a value of 1 for merge_offset_vector_flag indicates that the merged motion difference coding method is applied to the current block. When the merged motion difference coding method is applied to the current block, the motion vector of the current block can be derived by adding or subtracting an offset vector to or from the motion vector of a merge candidate. A value of 0 for merge_offset_vector_flag indicates that the merged motion difference coding method is not applied to the current block. When the merged offset coding method is not applied, the motion vector of a merge candidate can be set as the motion vector of the current block.

[0329] The flag can be signaled only when the skip flag indicating whether to apply the skip mode has a true value or the merge flag indicating whether to apply the merge mode has a true value. For example, when the value of skip_flag indicating whether to apply the skip mode to the current block is 1, or when the value of merge_flag indicating whether to apply the merge mode to the current block is 1, merge_offset_vector_flag can be encoded and signaled.

[0330] When it is determined that the merged offset coding method is applied to the current block, at least one of information specifying any one of the merge candidates included in the merge candidate list, information indicating the magnitude of the offset vector, and information indicating the direction of the offset vector can be signaled additionally.

[0331] Information for determining the maximum number of merge candidates that the merge candidate list can include can be signaled through the bitstream. For example, the maximum number of merge candidates that the merge candidate list can include can be set to an integer of 6 or less.

[0332] When it is determined that the merged offset coding method is applied to the current block, only a preset maximum number of merge candidates can be set as the initial motion vector of the current block. That is, depending on whether the merged offset coding method is applied, the number of merge candidates available for the current block can be adaptively determined. For example, when the value of merge_offset_vector_flag is set to 0, the maximum number of merge candidates available for the current block can be set to M, and when the value of merge_offset_vector_flag is set to 1, the maximum number of merge candidates available for the current block can be set to N. Here, M represents the maximum number of merge candidates that can be included in the merge candidate list, and N represents an integer equal to or less than M.

[0333] For example, when M is 6 and N is 2, two merge candidates with the smallest indices among the merge candidates included in the merge candidate list can be set to be available for the current block. Therefore, the motion vector of the merge candidate with index value 0 or the motion vector of the merge candidate with index value 1 can be set as the initial motion vector of the current block. When M and N are the same (for example, when M and N are 2), all the merge candidates included in the merge candidate list can be set to be available for the current block.

[0334] Alternatively, it can be determined whether an adjacent block can be used as a merge candidate based on whether the merged motion difference coding method is applied to the current block. For example, when the value of merge_offset_vector_flag is 1, at least one of the adjacent blocks adjacent to the upper right corner of the current block, the adjacent block adjacent to the upper left corner, and the adjacent block adjacent to the lower left corner can be set to be not available as a merge candidate. Therefore, when the merged motion difference coding method is applied to the current block, the motion vectors of at least one of the adjacent blocks adjacent to the upper right corner of the current block, the adjacent block adjacent to the upper left corner, and the adjacent block adjacent to the lower left corner cannot be set as the initial motion vector. Alternatively, when the value of merge_offset_vector_flag is 1, the temporal adjacent blocks of the current block can be set to be not available as a merge candidate.

[0335] When the merged motion difference coding method is applied to the current block, at least one of the paired merge candidate and the zero merge candidate can be set not to be used. Therefore, when the value of merge_offset_vector_flag is 1, even if the number of merge candidates included in the merge candidate list is less than the maximum number, at least one of the paired merge candidate or the zero merge candidate may not be added to the merge candidate list.

[0336] The motion vector of the merge candidate can be set as the initial motion vector of the current block. In this case, when the number of merge candidates available for the current block is multiple, information specifying any one of the multiple merge candidates can be signaled through the bitstream. For example, when the maximum number of merge candidates that the merge candidate list can include is greater than 1, information merge_idx indicating any one of the multiple merge candidates can be signaled through the bitstream. That is, under the merge offset coding method, a merge candidate can be specified by the information merge_idx for specifying any one of the multiple merge candidates. The initial motion vector of the current block can be set as the motion vector of the merge candidate indicated by merge_idx.

[0337] On the other hand, when the number of merge candidates available for the current block is 1, the information for specifying a merge candidate may be signaled omittably. For example, when the maximum number of merge candidates that the merge candidate list can include is not greater than 1, the information merge_idx for specifying a merge candidate may be signaled omittably. That is, in the merge offset coding method, when one merge candidate is included in the merge candidate list, the coding of the information merge_idx for specifying a merge candidate may be omitted, and the initial motion vector may be determined based on the merge candidate included in the merge candidate list. The motion vector of the merge candidate may be set as the initial motion vector of the current block.

[0338] As another example, after determining the merge candidates of the current block, it may be determined whether to apply the merge motion difference coding method to the current block. For example, when the maximum number of merge candidates that the merge candidate list can include is greater than 1, the information merge_idx for specifying any one of the merge candidates may be signaled. After selecting a merge candidate based on merge_idx, the merge_offset_vector_flag indicating whether to apply the merge motion difference coding method to the current block may be decoded. Table 4 is a diagram showing a syntax table according to the above-described embodiment.

[0339]

Table 4

[0340]

[0341]

[0342] As another example, after determining the merge candidates of the current block, it may be determined whether to apply the merge motion difference coding method to the current block only when the index of the determined merge candidate is less than the maximum number of merge candidates that can be used when applying the merge motion difference coding method. For example, the merge_offset_vector_flag indicating whether to apply the merge motion difference coding method to the current block may be coded and signaled only when the value of the index information merge_idx is less than N. When the value of the index information merge_idx is equal to or greater than N, the coding of merge_offset_vector_flag may be omitted. If the coding of merge_offset_vector_flag is omitted, it may be determined that the merge motion difference coding method is not applied to the current block.

[0343] Alternatively, after determining the merge candidates for the current block, it can be considered whether the determined merge candidates have bidirectional motion information or unidirectional motion information to determine whether to apply the merged motion difference coding method to the current block. For example, only when the value of the index information merge_idx is less than N and the merge candidate selected by the index information has bidirectional motion information, the merge_offset_vector_flag indicating whether to apply the merged motion difference coding method to the current block is coded and signaled. Optionally, only when the value of the index information merge_idx is less than N and the merge candidate selected by the index information has unidirectional motion information, the merge_offset_vector_flag indicating whether to apply the merged motion difference coding method to the current block is coded and signaled.

[0344] Alternatively, it can be determined whether to apply the merged motion difference coding method based on at least one of the size of the current block, the shape of the current block, and whether the current block touches the boundary of the coding tree unit. When at least one of the size of the current block, the shape of the current block, and whether the current block touches the boundary of the coding tree unit does not meet the preset conditions, the coding of the merge_offset_vector_flag indicating whether to apply the merged motion difference coding method to the current block can be omitted.

[0345] When a merge candidate is selected, the motion vector of the merge candidate can be set as the initial motion vector of the current block. Then, the information indicating the size of the offset vector and the information indicating the direction of the offset vector can be decoded to determine the offset vector. The offset vector can have a horizontal direction component or a vertical direction component.

[0346] The information indicating the size of the offset vector can be index information indicating any one of the motion offset size candidates. For example, the index information distance_idx indicating any one of the motion offset size candidates can be signaled through the bitstream. Table 5 shows the binarization of the index information distance_idx and the value of the variable DistFromMergeMV for determining the size of the offset vector according to distance_idx.

[0347]

Table 5

[0348]

[0349] The size of the offset vector can be derived by dividing the variable DistFromMergeMV by a preset value. Equation 5 shows an example of determining the size of the offset vector.

[0350]

Equation 5

[0351] abs(offsetMV) = DistFromMergeMV << 2

[0352] According to Equation 5, the value obtained by dividing the variable DistFromMegeMV by 4 or the value obtained by shifting the variable DistFromMergeMV left by 2 can be set as the magnitude of the offset vector.

[0353] More motion offset magnitude candidates or fewer motion offset magnitude candidates than the example shown in Table 5 can be used, or the numerical value of the number of motion vector offset magnitude candidates can be set differently from the example shown in Table 5. For example, the magnitude of the horizontal direction component or the vertical direction component of the offset vector can be set to not more than 2 sample distances. Table 6 shows the binarization of the index information distance_idx and the values of the variable DistFromMergeMV used to determine the magnitude of the offset vector according to distance_idx.

[0354]

Table 6

[0355]

[0356] Alternatively, the numerical value of the number of motion vector offset magnitude candidates can be set differently based on the motion vector precision. For example, when the motion vector precision of the current block is fractional-pel, the value of the variable DistFromMergeMV corresponding to the value of the index information distance_idx can be set to 1, 2, 4, 8, 16, etc. Here, fractional pixels include at least one of 1 / 16 pixel, one-eighth pixel, one-quarter pixel, or half pixel. On the other hand, when the motion vector precision of the current block is integer pixel, the value of the variable DistFromMergeMV corresponding to the value of the index information distance_idx can be set to 4, 8, 16, 32, 64, etc. That is, the table used to determine the variable DistFromMergeMV can be set differently according to the motion vector precision of the current block.

[0357] For example, when the motion vector precision of the current block or the merge candidate is one-quarter pixel, the variable DistFromMergeMV represented by distance_idx can be derived using Table 5. On the other hand, when the motion vector precision of the current block or the merge candidate is integer pixel, the value obtained by multiplying the value of the variable DistFromMergeMV indicated by distance_idx in Table 5 by N times (e.g., 4 times) can be used as the value of the variable DistFromMergeMV.

[0358] Information for determining the motion vector precision can be signaled via a bitstream. For example, the information can be signaled at the sequence level, picture level, slice level, or block level. Thus, the number value of the motion offset size candidates can be set differently according to the information related to the motion vector precision signaled via the bitstream. Alternatively, the motion vector precision can be determined based on the merge candidates of the current block. For example, the motion vector precision of the current block can be set to be the same as the motion vector precision of the merge candidates.

[0359] Alternatively, information for determining the search range of the offset vector can be signaled via a bitstream. At least one of the number of motion offset size candidates, the minimum value among the motion offset size candidates, and the maximum value among the motion offset size candidates can be determined based on the search range. For example, a flag merge_offset_vector_flag for determining the search range of the offset vector can be signaled via the bitstream. The information can be signaled via a sequence header, a picture header, or a slice header.

[0360] For example, when the value of merge_offset_extend_range_flag is 0, the size of the offset vector can be set to be no more than 2. Thus, the maximum value of DistFromMergeMV can be set to 8. On the other hand, when the value of merge_offset_extend_range_flag is 1, the size of the offset vector can be set to be no more than 32 sample distances. Thus, the maximum value of DistFromMergeMV can be set to 128.

[0361] A flag indicating whether the size of the offset vector is greater than a threshold can be used to determine the size of the offset vector. For example, a flag distance_flag indicating whether the size of the offset vector is greater than a threshold can be signaled via a bitstream. The threshold can be 1, 2, 4, 8, or 16. For example, distance_flag being 1 indicates that the size of the offset vector is greater than 4. On the other hand, distance_flag being 0 indicates that the size of the offset vector is 4 or less.

[0362] When the size of the offset vector is greater than the threshold, the index information distance_idx can be used to derive the difference between the size of the offset vector and the threshold. Alternatively, when the size of the offset vector is less than or equal to the threshold, the size of the offset vector can be determined by using the index information distance_idx. Table 7 is a syntax table showing the process of encoding distance_flag and distance_idx.

[0363]

Table 7

[0364]

[0365]

[0366] Equation 6 shows an example of using distance_flag and distance_idx to derive the variable DistFromMergeMV for determining the magnitude of the offset vector.

[0367]

Equation 6

[0368] DistFromMergeMV = N * distance_flag + (1 << distance_idx)

[0369] In Equation 6, the value of distance_flag can be set to 1 or 0. The value of distance_idx can be set to 1, 2, 4, 8, 16, 32, 64, 128, etc. N represents a coefficient determined by a threshold. For example, when the threshold is 4, N can be set to 16.

[0370] The information indicating the direction of the offset vector can be index information indicating any one of the vector direction candidates. For example, the index information direction_idx indicating any one of the vector direction candidates can be signaled through the bitstream. Table 8 shows the binarization of the index information direction_idx and the direction of the offset vector according to direction_idx.

[0371]

Table 8

[0372] direction_idx[x][y] Binarization sign[x][y][0] sign[x][y][1] 0 00 +1 0 1 01 -1 0 2 10 0 +1 3 11 0 -1

[0373] In Table 8, sign[0] indicates the horizontal direction, and sign[1] indicates the vertical direction. +1 indicates that the value of the x-component or y-component of the offset vector is positive (+), and -1 indicates that the value of the x-component or y-component of the offset vector is negative (-). Equation 7 shows an example of determining the offset vector based on the magnitude and direction of the offset vector.

[0374]

Equation 7

[0375] offsetMV[0] = abs(offsetMV) * sign[0]

[0376] offsetMV[1] abs(offsetMV) * sign[1]

[0377] In Equation 7, offsetMV[0] indicates the vertical direction component of the offset vector, and offsetMV[1] indicates the horizontal direction component of the offset vector.

[0378] Figure 28 It is a diagram showing an offset vector indicating the value of distance_idx indicating the magnitude of the offset vector and direction_idx indicating the direction of the offset vector.

[0379] As in Figure 28 the example shown, the magnitude and direction of the offset vector can be determined according to the values of distance_idx and direction_idx. The maximum magnitude of the offset vector can be set not to exceed a threshold value. Here, the threshold value can have a value predefined by the encoder and the decoder. For example, the threshold value can be 32 sample distances. Alternatively, the threshold value can be determined according to the magnitude of the initial motion vector. For example, the threshold value in the horizontal direction can be set based on the magnitude of the horizontal component of the initial motion vector, and the threshold value in the vertical direction can be set based on the magnitude of the vertical component of the initial motion vector.

[0380] When the merge candidate has bidirectional motion information, the L0 motion vector of the merge candidate can be set as the L0 initial motion vector of the current block, and the L1 motion vector of the merge candidate can be set as the L1 initial motion vector of the current block. In this case, the L0 offset vector and the L1 offset vector can be determined in consideration of the output order difference (hereinafter referred to as the L0 difference) between the L0 reference image of the merge candidate and the current image and the output order difference (hereinafter referred to as the L1 difference) between the L1 reference image of the merge candidate and the current image.

[0381] First, when the signs of the L0 difference and the L1 difference are the same, the L0 offset vector and the L1 offset vector can be set to be the same. On the other hand, when the signs of the L0 difference and the L1 difference are different, the L1 offset vector can be set in the direction opposite to the L0 offset vector.

[0382] The magnitude of the L0 offset vector and the magnitude of the L1 offset vector can be set to be the same. Alternatively, the magnitude of the L1 offset vector can be determined by scaling the L0 offset vector based on the L0 difference and the L1 difference.

[0383] For example, Equation 8 shows the L0 offset vector and the L1 offset vector when the signs of the L0 difference and the L1 difference are the same.

[0384]

Equation 8

[0385] offsetMVL0[0] = abs(offsetMV) * sign[0]

[0386] offsetMVL0[1] = abs(offsetMV) * sign[1]

[0387] offsetMVL1[0] = abs(offsetMV) * sign[0]

[0388] offsetMVL1[1] = abs(offsetMV) * sign[1]

[0389] In Equation 8, offsetMVL0[0] indicates the horizontal direction component of the L0 offset vector, and offsetMVL0[1] indicates the vertical direction component of the L0 offset vector. offsetMVL1[0] indicates the horizontal direction component of the L1 offset vector, and offsetMVL1[1] indicates the vertical direction component of the L1 offset vector.

[0390] Equation 9 shows the L0 offset vector and the L1 offset vector when the signs of the L0 difference and the L1 difference are different.

[0391]

Equation 9

[0392] offsetMVL0[0] = abs(offsetMV) * sign[0]

[0393] offsetMVL0[1] = abs(offsetMV) * sign[1]

[0394] offsetMVL1[0] = -1 * abs(offsetMV) * sign[0]

[0395] offsetMVL1[1] = -1 * abs(offsetMV) * sign[1]

[0396] More than four vector direction candidates can also be defined. Tables 9 and 10 show examples of defining eight vector direction candidates.

[0397]

Table 9

[0398] direction_idx[x][y] Binarization sign[x][y][0] sign[x][y][1] 0 000 +1 0 1 001 -1 0 2 010 0 +1 3 011 0 -1 4 100 +1 +1 5 101 +1 -1 6 110 -1 +1 7 111 -1 -1

[0399]

Table 10

[0400] direction_idx[x][y] Binarization sign[x][y][0] sign[x][y][1] 0 000 +1 0 1 001 -1 0 2 010 0 +1 3 011 0 -1 4 100 +1 / 2 +1 / 2 5 101 +1 / 2 -1 / 2 6 110 -1 / 2 +1 / 2 7 111 -1 / 2 -1 / 2

[0401] In Tables 9 and 10, the absolute values of sign[0] and sign[1] greater than 0 indicate that the offset vector is in the diagonal direction. When using Table 9, the magnitudes of the x-axis component and the y-axis component of the diagonal offset vector are set to abs(offsetMV), while when using Table 10, the magnitudes of the x-axis component and the y-axis component of the diagonal offset vector are set to abs(offsetMV / 2).

[0402] Figure 29It is a diagram showing an offset vector indicating values of distance_idx representing the magnitude of the offset vector and direction_idx representing the direction of the offset vector.

[0403] Figure 29 (a) is an example when Table 9 is applied, and Figure 29 (b) is an example when Table 10 is applied.

[0404] Information for determining at least one of the number or magnitude of vector direction candidates can be signaled through the bitstream. For example, a flag merge_offset_direction_range_flag for determining vector direction candidates can be signaled through the bitstream. The flag can be signaled at the sequence level, picture level, or slice level. For example, when the value of the flag is 0, four vector direction candidates shown in Table 8 can be used. On the other hand, when the value of the flag is 1, eight vector direction candidates shown in Table 9 or Table 10 can be used.

[0405] Alternatively, at least one of the number or magnitude of vector direction candidates can be determined based on the magnitude of the offset vector. For example, when the value of the variable DistFromMergeMV for determining the magnitude of the offset vector is equal to or less than a threshold, eight vector direction candidates shown in Table 9 or Table 10 can be used. On the other hand, when the value of the variable DistFromMergeMV is greater than the threshold, four vector direction candidates shown in Table 8 can be used.

[0406] Alternatively, at least one of the number or magnitude of vector direction candidates can be determined based on the value of the x component MVx and the value of the y component MVy of the initial motion vector. For example, when the difference or the absolute value of the difference between MVx and MVy is less than or equal to a threshold, eight vector direction candidates shown in Table 9 or Table 10 can be used. On the other hand, when the difference or the absolute value of the difference between MVx and MVy is greater than the threshold, four vector direction candidates shown in Table 8 can be used.

[0407] The motion vector of the current block can be derived by adding the offset vector to the initial motion vector. Equation 10 shows an example of determining the motion vector of the current block.

[0408]

Equation 10

[0409] mvL0[0] = mergeMVL0[0] + offsetMVL0[0]

[0410] mvL0[1] mergeMVL0[1] + offsetMVL0[1]

[0411] mvL1[0] = mergeMVZ1[0] + offsetMVL1[0]

[0412] mvL1[1] = mergeMVZ1[1] + offsetMVL1[1]

[0413] In Equation 10, mvL0 indicates the L0 motion vector of the current block, and mvL1 indicates the L1 motion vector of the current block. mergeMVL0 indicates the initial L0 motion vector of the current block (i.e., the L0 motion vector of the merge candidate), and mergeMVL1 represents the initial L1 motion vector of the current block. [0] indicates the horizontal direction component of the motion vector, and [1] indicates the vertical direction component of the motion vector.

[0414] Even when performing inter prediction on each sub-unit after dividing the coding block into multiple sub-units, the merge motion difference coding method can be applied. Here, performing inter prediction on a per-sub-unit basis can include at least one of advanced temporal motion vector prediction (ATMVP), spatio-temporal motion vector prediction (STMVP), and triangular partitioning techniques.

[0415] For example, under the ATMVP method, the initial motion vector can be derived as follows.

[0416] First, the initial shift vector can be derived using the motion vectors of the merge candidates derived from the neighboring blocks adjacent to the coding block. Additionally, the shifted block of the sub-block included in the coding block can be derived using the initial shift vector. Equation 11 shows the position of the shifted block.

[0417]

Equation 11

[0418] (xColSb, yColSb) = (xSb + shVector[0] >> 4, ySb + shVector[1] >> 4)

[0419] In Equation 11, (xColSb, yColSb) represents the position of the upper left sample of the shifted block, and (xSb, ySb) represents the position of the upper left sample of the sub-block. shVector represents the shift vector.

[0420] When the shifted block is determined, the motion vector of the co-located block at the same position as the shifted block in the co-located image can be set as the motion vector of the sub-block. That is, the motion vector of the co-located block of the sample at the (xColSb, yColSb) position included in the co-located block can be set as the motion vector of the sub-block of the sample at the (xSb, ySb) position.

[0421] When applying the triangular partitioning technique, the coding block can be divided into triangular sub-units. For example, the coding block can be divided into two sub-units by connecting the diagonal line from the upper left corner to the lower right corner of the coding block or by connecting the diagonal line from the upper right corner to the lower left corner of the coding block.

[0422] Figure 30 It is a diagram showing the partitioning pattern of coding blocks when applying the triangular partitioning technique.

[0423] The motion information of each triangular sub-unit can be specified by merge candidates. To this end, index information indicating any one of the merge candidates can be signaled for each sub-unit. For example, the index information merge_1st_idx of the first sub-unit can specify the merge candidate of the first sub-unit, and the index information merge_2nd_idx of the second sub-unit can specify the merge candidate of the second sub-unit.

[0424] The initial motion vector of each sub-unit can be determined individually. For example, when applying an affine motion model to a coding block, the affine vector of a sub-block derived from the affine seed vector of the coding block can be set as the initial motion vector of the sub-block. The motion vector of each sub-block can be derived by adding or subtracting an offset vector to / from the initial motion vector.

[0425] When applying the merged motion difference coding method to a coding block divided into multiple sub-units, multiple sub-units can be set to use the same offset vector. That is, the same offset vector can be used to change the initial motion vector of each of the multiple sub-units.

[0426] Alternatively, a coding block can be divided into multiple sub-units, and the offset vector of each sub-unit can be determined individually. Therefore, the offset vector of at least one sub-unit can be set to be different from the offset vector of another sub-unit.

[0427] Figure 31 It is a diagram showing an example of setting the offset vector of each sub-unit differently.

[0428] As in the example shown in Figure 31 information distance_idx indicating the magnitude of the offset vector and direction_idx indicating the direction of the offset vector can be coded and signaled for each sub-unit.

[0429] Alternatively, the magnitudes of the offset vectors of all sub-units can be set to be the same, and the directions of the offset vectors can be set individually for these sub-units. For example, it can be set to share the value of distance_idx signaled at the coding level among the sub-units, and direction_idx can be coded and signaled for each sub-unit.

[0430] Alternatively, the directions of the offset vectors of all the subunits can be set to be the same, and the magnitudes of the offset vectors can be set individually for these subunits. For example, the value of direction_idx signaled at the coding level can be set to be shared among the subunits, and distance_idx can be coded and signaled for each subunit.

[0431] The merged motion difference coding method can be applied only to some of the multiple subunits generated by dividing a coding block. For example, when a current block is divided into a first subunit and a second subunit, the motion vector of the first subunit can be set to be the same as the motion vector of a merge candidate, and the motion vector of the second subunit can be derived by adding an offset vector to the motion vector of the merge candidate.

[0432] Instead of signaling the information for determining the offset vector, the decoder can derive the offset vector. Specifically, the average value of the horizontal direction gradients and the average value of the vertical direction gradients of the predicted samples included in the sub-block can be used to derive the offset vector.

[0433] Here, the gradient can be derived based on the difference between the reconstructed sample corresponding to the predicted sample in the reference image and the neighboring samples adjacent to the reconstructed sample. For example, the horizontal direction gradient can indicate the difference between the reconstructed sample and the reconstructed samples neighboring on the left side and / or the right side, and the vertical direction gradient can indicate the difference between the reconstructed sample and the reconstructed samples neighboring on the upper side and / or the lower side.

[0434] Among the merge candidates included in the merge candidate list, a merge candidate having a motion vector derived by adding or subtracting an offset vector from the motion vector of a reference merge candidate among the merge candidates included in the merge candidate list can be added to the merge candidate list. A merge candidate having a motion vector derived by adding or subtracting an offset vector from the motion vector of a reference merge candidate can be referred to as a refined merge candidate.

[0435] The remaining motion information other than the motion vector of the refined merge candidate can be set to be the same as the motion information of the reference merge candidate.

[0436] Figure 32 is a diagram showing the motion vector candidates that the refined merge candidate can adopt.

[0437] When the motion vector of a reference merge candidate is (MvLX[0], MvLX[1]), the motion vector of the refined merge candidate can be derived by adding or subtracting an offset to at least one of the x - component or y - component of the motion vector of the reference merge candidate. For example, the motion vector of the refined merge candidate can be set to (MvLX[0]+M, MvLX[1]), (MvLX[0]-M, MvLX[1]), (MvLX[0], MvLX[1]+M), or (MvLX[0], MvLX[1]-M). M represents the magnitude of the offset vector.

[0438] The reference merge candidate can be a merge candidate having a predefined index value in the merge candidate list. For example, the merge candidate having the smallest index value (i.e., the merge candidate having an index value of 0) or the merge candidate having the largest index value among the merge candidates included in the merge candidate list can be set as the reference merge candidate. Alternatively, the inter - frame merge candidate having the smallest index value or the inter - frame merge candidate having the largest index value in the inter - frame motion information list can be set as the reference merge candidate.

[0439] Alternatively, the merge candidate having the smallest index value among the merge candidates having bi - directional motion information can be set as the reference merge candidate. That is, when searching for candidate blocks sequentially, the first - found bi - directional merge candidate can be set as the reference merge candidate.

[0440] The basic merge candidate can be selected based on the size of the current block, the shape of the current block, or whether the current block touches the boundary of the coding tree unit. For example, when the current block is square - shaped or the current block is non - square - shaped with a height greater than the width, the merge candidate with index 0 or the merge candidate derived from the adjacent block above the current block can be set as the reference merge candidate. When the current block is non - square - shaped with a width greater than the height, the merge candidate with index 1 or the merge candidate derived from the adjacent block to the left of the current block can be set as the reference merge candidate.

[0441] Alternatively, the information specifying the reference merge candidate can be signaled through the bitstream. The information can be the index information specifying any one of the merge candidates included in the merge candidate list.

[0442] The information indicating whether to use the refined merge candidate can be signaled through the bitstream. The information can be a 1 - bit flag. When the value of the flag is 1, the refined merge candidate generated based on the reference merge candidate can be added to the merge candidate list. On the other hand, when the value of the flag is 0, the merge candidate list may not include the refined merge candidate.

[0443] Alternatively, when the number of merge candidates added to the merge candidate list is less than the maximum number of merge candidates that the merge candidate list can include, refined merge candidates may be added to the merge candidate list. Here, the added merge candidates may include at least one of a spatial merge candidate, a temporal merge candidate, an inter-frame merge candidate, or a pairwise merge candidate. For example, when the number of at least one of the spatial merge candidate, the temporal merge candidate, and the inter-frame merge candidate included in the merge candidate list is less than or equal to a threshold, refined merge candidates may be added to the merge candidate list.

[0444] Alternatively, when the number of merge candidates added to the merge candidate list is greater than or equal to a threshold, refined merge candidates may be used.

[0445] The maximum number of merge candidates that can be included in the merge candidate list may be set differently according to whether refined merge candidates are used. For example, when it is set not to use refined merge candidates, the maximum number of merge candidates that the merge candidate list can include may be set to N, and when it is set to use refined merge candidates, the maximum number of merge candidates that the merge candidate list can include may be set to N + n.

[0446] The index of the refined merge candidate may be greater than the index of the merge candidates added to the merge candidate list. For example, Table 11 shows an example of configuring the merge candidate list.

[0447]

Table 11

[0448] mergeCand[0] mergeCand[1] mergeCand[2] mergeCand[3] mergeCand[4] mergeCand[5] mergeCand[6]: Fine merge candidate with motion vector (MvLX[0] + M, MvLX[1]) mergeCand[7]: Fine merge candidate with motion vector (MvLX[0] - M, MvLX[1]) mergeCand[8]: Fine merge candidate with motion vector (MvLX[0], MvLX[1] + M) mergeCand[9]: Fine merge candidate with motion vector (MvLX[0], MvLX[1] - M)

[0449] In Table 11, mergeCand[X] represents the merge candidate with index X. MvLX[0] represents the x-component motion vector of the reference merge candidate, and MvLX[1] represents the y-component motion vector of the reference merge candidate. For example, when the reference merge candidate is mergeCand[0], MvLX[0] and MvLX[1] may represent the motion vectors of mergeCand[0].

[0450] The magnitude M of the offset vector may be predefined in the encoder and the decoder. For example, the magnitude M of the offset vector may be set to an integer less than or equal to 4, such as 1 or 4.

[0451] Alternatively, information for determining the offset vector may be signaled via the bitstream. The information may be signaled at the sequence level, picture level, slice level, or block level. For example, at least one of the information distance_idx for determining the magnitude of the offset vector or the information direction_idx for determining the direction of the offset vector as described above may be used to determine the offset vector.

[0452] In the example shown in Table 11, at least one refined merge candidate derived based on a reference merge candidate can be added to the merge candidate list. If there is a merge candidate in the added merge candidates that has the same motion information as the motion information of the refined merge candidate, the refined merge candidate may not be added to the merge candidate list. For example, when the refined merge candidate derived based on the reference merge candidate mergeCand[0] is the same as any one of mergeCand[1] to mergeCand[5], the refined merge candidate may not be added to the merge candidate list.

[0453] Alternatively, when there is a merge candidate that has the same motion information as the motion information of the refined merge candidate, the refined merge candidate can be re-derived by changing the offset vector, or the merge candidate that has the same motion information as the motion information of the refined merge candidate can be reset as the refined merge candidate. For example, if the motion information of the refined merge candidate mergeCand[6] derived based on the reference merge candidate mergeCand[0] is the same as the motion information of the merge candidate mergeCand[2], the motion vector of the refined merge candidate mergeCand[6] can be changed to a value obtained by adding or subtracting the offset vector to or from the motion vector of the merge candidate [2]. For example, the motion vector of mergeCand[6] can be changed from (mergeCand[0]_mxLx[0]+M, mergeCand[0]_mvLx[1]) to (mergeCand[2]_mxLx[0]+M, mergeCand[2]_mvLx[1]). Here, mergeCand[X]_mvLx represents the motion vector of the merge candidate with index X.

[0454] As another example, a merge refinement offset list including at least one merge offset candidate can be used to determine the offset vector. When the merge candidate specified by the index information of the current block is a reference merge candidate, the merge refinement offset list can be used to determine the offset vector. Additionally, the motion vector of the current block can be derived by adding or subtracting the offset vector to or from the motion vector of the merge candidate. The reference merge candidate can be a merge candidate having a predefined index value in the merge candidate list. For example, the merge candidate having the smallest index value (i.e., the merge candidate having the index value 0) or the merge candidate having the largest index value among the merge candidates included in the merge candidate list can be set as the reference merge candidate. Alternatively, the inter-frame merge candidate having the smallest index value or the inter-frame merge candidate having the largest index value in the inter-frame motion information list can be set as the reference merge candidate.

[0455] Figure 33 is a diagram showing the configuration of the merge refinement offset list.

[0456] In Figure 33 it, assume that the reference merge candidate is the merge candidate with index 6.

[0457] If the index of the merge candidate specified by merge_idx indicating any one of the merge candidates is not 6, the motion vector of the merge candidate can be set to the motion vector of the current block.

[0458] On the other hand, when the index of the merge candidate specified by the index information merge_idx is 6, the merge refinement offset list can be used to derive the offset vector. The index information MrgOffset_idx specifying any one of the merge offset candidates included in the merge refinement offset list can be signaled by the bitstream.

[0459] When the offset vector is specified, the motion vector of the current block can be derived by adding or subtracting the offset vector to / from the motion vector of the reference merge candidate.

[0460] The merge refinement offset list can include at least one merge offset candidate. For example, the number of merge offset candidates included in the merge refinement offset list can be 4, 8, or 16.

[0461] Figure 34 and Figure 35 are diagrams showing the offset vectors specified by the merge offset candidates.

[0462] Figure 34 illustrates an example where the number of merge offset candidates is 8, and Figure 35 illustrates an example where the number of merge offset candidates is 16.

[0463] As in the example shown in (a) of Figure 34 , the offset vector represented by the merge offset candidate can be set such that the absolute value of the motion vector in the horizontal direction and / or the absolute value of the motion vector in the vertical direction can have a fixed value. Alternatively, as in the example shown in Figure 35 , the offset vector represented by the merge offset candidates with indices less than the threshold can be set such that the absolute value of the motion vector in the horizontal direction and / or the absolute value of the motion vector in the vertical direction can have a first value, and the offset vector indicated by the other merge offset candidates can be set such that the absolute value of the motion vector in the horizontal direction and / or the absolute value of the motion vector in the vertical direction can have a second value.

[0464] Alternatively, as shown in (b) of Figure 34 , the offset vector represented by the merge offset candidates can be set such that the sum of the absolute value of the motion vector in the horizontal direction and the absolute value of the motion vector in the vertical direction can have a fixed value.

[0465] Multiple reference merge candidates can be set. For example, among the merge candidates included in the merge candidate list, the two merge candidates with the smallest indices can be set as the reference merge candidates. Thus, when the index of the merge candidate specified by the index information merge_idx is 0 or 1, the merge refinement offset list can be used to derive the offset vector. Alternatively, the merge candidate with the smallest index among the merge candidates included in the merge candidate list and the merge candidate with the largest index among the merge candidates included in the inter-frame merge candidate list can be set as the reference merge candidates.

[0466] In the advanced motion vector prediction mode, the motion vector of the current block can be derived by adding the motion difference vector to the motion prediction vector. The motion prediction vector of the current block can be determined based on a motion vector prediction candidate list including at least one motion prediction vector candidate. For example, any one of the motion prediction vector candidates can be set as the motion prediction vector of the current block.

[0467] The motion vector prediction candidates can be derived based on at least one of the spatially adjacent blocks or temporally adjacent blocks of the current block.

[0468] Figure 36 FIG. is a diagram showing candidate blocks for deriving motion vector prediction candidates.

[0469] The spatially adjacent blocks can include an upper adjacent block located above the current block and a left adjacent block located to the left of the current block. The upper adjacent block can include one of the following blocks, i.e., block B0 including samples at the position (xCb + CbW, yCb - 1), block B1 including samples at the position (xCb + CbW - 1, yCb - 1), block B2 including samples at the position (xCb - 1, yCb - 1), or block B3 including samples at the position (xCb, yCb - 1). Herein, (xCb, yCb) represents the position of the upper left sample of the current block, and CbW represents the width of the current block. The left adjacent block can include one of the following blocks, i.e., block A0 including samples at the position (xCb - 1, yCb + CbH), block A1 including samples at the position (xCb - 1, yCb + CbH - 1), or block A2 including samples at the position (xCb - 1, yCb). Herein, CbH represents the height of the current block.

[0470] The temporally adjacent blocks can include at least one of block C0 and block C1. Block C0 includes samples at the center of a block having the same position and size as the current block in the co-located block, and block C1 includes samples adjacent to the lower right corner of the block.

[0471] The maximum number of motion vector prediction candidates that can be included in the motion vector prediction candidate list can be 2. The derivation order of the motion vector prediction candidates is as follows.

[0472] 1. When at least one of the left adjacent blocks A0 or A1 is available, set the motion vector of the available block as a motion vector prediction candidate.

[0473] 2. When at least one of the upper adjacent blocks B0, B1, or B2 is available, set the motion vector of the available block as a motion vector prediction candidate.

[0474] 3. When a temporally adjacent block is available, set the temporal motion vector as a motion vector prediction candidate.

[0475] 4. Set the zero motion vector as a motion vector prediction candidate.

[0476] Alternatively, when the number of motion vector prediction candidates derived in the order of 1 to 3 is less than 2, the motion vector included in the inter-frame motion information list can be set as a motion vector prediction candidate. When the inter-frame motion information list is available, the motion vector prediction candidates can be derived in the following order.

[0477] 1. When at least one of the left adjacent blocks A0 or A1 is available, set the motion vector of the available block as a motion vector prediction candidate.

[0478] 2. When at least one of the upper adjacent blocks B0, B1, or B2 is available, set the motion vector of the available block as a motion vector prediction candidate.

[0479] 3. When a temporally adjacent block is available, set the temporal motion vector as a motion vector prediction candidate.

[0480] 4. Set the motion vector included in the inter-frame motion information list as a motion vector prediction candidate.

[0481] 5. Set the zero motion vector as a motion vector prediction candidate.

[0482] Motion vector prediction candidates having motion vectors derived by adding or subtracting an offset vector to or from the motion vectors of reference motion vector prediction candidates can be added to the motion vector prediction candidate list. Motion vector prediction candidates having motion vectors derived by adding or subtracting an offset vector to or from the motion vectors of reference motion vector prediction candidates can be referred to as fine motion vector prediction candidates.

[0483] Figure 37 is a diagram showing motion vector candidates that can be set as fine motion vector prediction candidates.

[0484] When the motion vector of the reference motion vector prediction candidate is (MvpLX[0], MvpLX[1]), the motion vector of the refined motion vector prediction candidate can be derived by adding or subtracting an offset to at least one of the x - component or y - component of the motion vector of the reference motion vector prediction candidate. For example, the motion vector of the refined motion vector prediction candidate can be set to (MvpLX[0]+M, MvpLX[1]), (MvpLX[0] - M, MvpLX[1]), (MvpLX[0], MvpLX[1]+M) or (MvpLX[0], MvpLX[1] - M). M represents the magnitude of the offset vector.

[0485] The magnitude M of the offset vector can be predefined in the encoder and decoder. For example, the magnitude M of the offset vector can be set to an integer less than or equal to 4, such as 1 or 4.

[0486] Alternatively, the information for determining the offset vector can be signaled through the bitstream. The information can be signaled at sequence level, picture level, slice level or block level. For example, at least one of the information distance_idx for determining the magnitude of the offset vector or the information direction_idx for determining the direction of the offset vector can be used as described above to determine the offset vector.

[0487] The reference motion vector prediction candidate can be a motion vector prediction candidate having a predetermined index value in the motion vector prediction candidate list. For example, among the motion vector prediction candidates included in the motion vector prediction candidate list, the motion vector prediction candidate with index value 0 or the motion vector prediction candidate with index value 1 can be set as the reference motion vector prediction candidate.

[0488] As another example, a merge refinement offset list including at least one prediction vector offset candidate can be used to determine the offset vector. When the motion vector prediction candidate specified by the index information of the current block is the reference motion vector prediction candidate, the prediction vector refinement offset list can be used to determine the offset vector. Additionally, the motion prediction vector of the current block can be derived by adding or subtracting the offset vector to / from the motion vector of the motion vector prediction candidate. The reference motion vector prediction candidate can be a motion vector prediction candidate having a predetermined index value in the motion vector prediction candidate list. For example, among the motion vector prediction candidates included in the motion vector prediction candidate list, the motion vector prediction candidate with the minimum index value or the motion vector prediction candidate with the maximum index value can be set as the reference motion vector prediction candidate.

[0489] When using the prediction vector offset refinement list to calculate the offset vector, the maximum number of prediction vector candidates that the prediction vector candidate list can include can be set to a value greater than 2.

[0490] Figure 38 It is a diagram showing the configuration of the prediction vector refinement offset list.

[0491] In Figure 38 it, it is assumed that the reference prediction vector candidate is the prediction vector candidate with index 2.

[0492] When the index of the prediction vector candidate specified by the index information AMVPcand_idx indicating any one of the prediction vector candidates is not 2, the motion vector of the prediction vector candidate can be set as the motion prediction vector of the current block.

[0493] On the other hand, when the index of the prediction vector candidate specified by the index information AMVPcand_idx is 2, the prediction vector refinement offset list can be used to derive the offset vector. The index information AMVPOffset_idx specifying any one of the prediction vector offset candidates included in the prediction vector refinement offset list can be signaled through the bitstream.

[0494] When the offset vector is specified, the motion prediction vector of the current block can be derived by adding or subtracting the offset vector from the motion vector of the reference prediction vector candidate.

[0495] Even when encoding an encoded block based on an affine motion model, motion vector refinement techniques can be used. For example, when applying the affine advanced motion vector prediction mode, the affine seed vector of the encoded block can be derived by adding the affine seed difference vector to the affine seed prediction vector. Here, the affine seed prediction vector can be derived based on the affine seed vectors of the spatially adjacent blocks or temporally adjacent blocks of the encoded block. The affine seed difference vector can be determined based on the information signaled from the bitstream. In this case, the same affine seed difference vector can be applied to all control points. Alternatively, the information for determining the affine seed vector can be signaled for each control point.

[0496] When deriving the affine vector of a sub-block based on the affine seed vector of the encoded block, the affine vector can be set as the initial motion vector, and then the offset vector can be derived. The motion vector of each sub-block can be derived by adding or subtracting the offset vector from the initial motion vector.

[0497] Instead of signaling the information for determining the offset vector, the decoder can derive the offset vector. Specifically, the average value of the horizontal direction gradients and the average value of the vertical direction gradients of the prediction samples included in the sub-block can be used to derive the offset vector.

[0498] Intra prediction is to predict the current block using the reconstructed samples that have been encoded / decoded around the current block. In this case, the intra prediction of the current block can use the reconstructed samples before applying the loop filter.

[0499] Intra prediction techniques include matrix-based intra prediction and general intra prediction that takes into account the directionality with surrounding reconstructed samples. Information indicating the intra prediction technique for the current block can be signaled through the bitstream. The information can be a 1-bit flag. Alternatively, based on at least one of the position, size, shape of the current block, or the intra prediction technique of adjacent blocks, the intra prediction technique for the current block can be determined. For example, when the current block straddles the image boundary, the current block is set not to apply matrix-based intra prediction.

[0500] Matrix-based intra prediction is a method of obtaining a predicted block for the current block based on matrix multiplication between a matrix stored in the encoder and decoder and the reconstructed samples around the current block. Information for specifying any one of the multiple stored matrices can be signaled through the bitstream. The decoder can determine the matrix for intra prediction of the current block based on the information and the size of the current block.

[0501] General intra prediction is a method of obtaining a predicted block related to the current block based on a non-angular intra prediction mode or an angular intra prediction mode.

[0502] The derived residual image can be obtained by subtracting the predicted image from the original image. In this case, when the residual image is transformed into the frequency domain, even if the high-frequency components in the frequency components are removed, the subjective picture quality of the video will not be significantly reduced. Thus, if the value of the high-frequency component is made smaller or the value of the high-frequency component is set to 0, there is an effect of improving the compression efficiency without causing obvious visual distortion. Reflecting the above characteristics, the current block can be transformed to decompose the residual image into two-dimensional frequency components. Transformation techniques such as discrete cosine transform (DCT, Discrete Cosine Transform) or discrete sine transform (DST, Discrete Sine Transform) can be used to perform the transformation.

[0503] After transforming the current block using DCT or DST, the transformed current block can be transformed again. In this case, the transformation based on DCT or DST can be defined as the main transformation, and the process of transforming the block to which the main transformation is applied again can be called the secondary transformation.

[0504] The main transformation can be performed using any one of multiple transformation kernel candidates. For example, the main transformation can be performed using any one of DCT2, DCT8, or DCT7.

[0505] Different transformation kernels can also be used for the horizontal and vertical directions. Information indicating the combination of the transformation kernel for the horizontal direction and the transformation kernel for the vertical direction can also be signaled through the bitstream.

[0506] The execution units for the primary transform and the secondary transform are different. For example, the primary transform can be performed on an 8×8 block, and the secondary transform can be performed on a 4×4 sub-block within the transformed 8×8 block. In this case, the transform coefficients of the remaining regions where the secondary transform is not performed can also be set to 0.

[0507] Alternatively, the primary transform can be performed on a 4×4 block, and the secondary transform can be performed on an 8×8 region including the transformed 4×4 block.

[0508] The information indicating whether the secondary transform is performed can be signaled through the bitstream.

[0509] In the decoder, the inverse transform (second inverse transform) of the secondary transform can be performed, and the inverse transform (first inverse transform) of the primary transform can be performed on the result. As a result of the execution of the second inverse transform and the first inverse transform, the residual signal of the current block can be obtained.

[0510] Quantization is used to reduce the energy of a block, and the quantization process includes a process of dividing the transform coefficients by a specific constant. The constant can be derived from a quantization parameter, and the quantization parameter can be defined as a value between 1 and 63.

[0511] If the transform and quantization are performed in the encoder, the decoder can obtain the residual block through inverse quantization and inverse transform. By adding the prediction block and the residual block in the decoder, the reconstructed block of the current block can be obtained.

[0512] If the reconstructed block of the current block is obtained, the information loss generated during the quantization and encoding processes can be reduced through in-loop filtering. The in-loop filter can include at least one of a deblocking filter, a sample adaptive offset filter (SAO), or an adaptive loop filter (ALF).

[0513] Using the embodiments described centering on the decoding process or the encoding process for the encoding process or the decoding process is also included within the scope of the present invention. Changing the multiple embodiments described in a predetermined order to a different order is also included within the scope of the present invention.

[0514] The embodiments have been described based on a series of steps or flowcharts, but this does not limit the time-sequence order of the invention, and they can be executed simultaneously or in another order as needed. Also, in the above embodiments, the structural elements (e.g., units, modules, etc.) constituting the block diagrams can be implemented as hardware devices or software respectively, and multiple structural elements can be combined and implemented as a single hardware device or software. The embodiments can be implemented in the form of program instructions, which can be executed by various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium can include program instructions, data files, data structures, etc. alone or in combination. Examples of the computer-readable recording medium can include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs, DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store program instructions and execute the instructions such as ROMs, RAMs, flash memories, etc. The hardware devices can be configured to operate as one or more software modules to execute the processing according to the present invention, and vice versa.

[0515]

Industrial Applicability

[0516] The present invention can be applied to electronic devices for encoding / decoding video.

Claims

1. A video decoding method, comprising the following steps: Determine whether to apply a merged motion difference coding method to a current block; Generate a merge candidate list for the current block; Determine a merge candidate for the current block based on the merge candidate list; And Derive a motion vector for the current block based on the merge candidate, wherein, When applying the merged motion difference coding method to the current block, derive the motion vector of the current block by adding an offset vector to the motion vector derived based on the merge candidate, and when the maximum number of merge candidates included in the merge candidate list is 1, determine the merge candidate without decoding index information indicating one of the merge candidates in the merge candidate list decoded from a bitstream; Determine the magnitude of the offset vector based on first index information indicating one of a plurality of motion offset magnitude candidates; wherein at least one of the maximum value or the minimum value of the motion offset magnitude candidates is set differently according to the value of a flag indicating the range of the motion offset magnitude candidates, or at least one of the maximum value or the minimum value of the motion offset magnitude candidates is set differently according to the motion vector precision of the current block.

2. The video decoding method according to claim 1, wherein, The flag is signaled at the picture level.

3. The video decoding method according to claim 1, wherein, The magnitude of the offset vector is obtained by performing a shift operation on a value represented by the motion offset magnitude candidate specified by the first index information.

4. The video decoding method according to claim 1, wherein, Determine the direction of the offset vector based on second index information for indicating one of a plurality of vector direction candidates.

5. A video encoding method, comprising the following steps: Determine whether to apply a merged motion difference coding method to a current block; Generate a merge candidate list for the current block; Determine a merge candidate for the current block based on the merge candidate list; And Derive a motion vector for the current block based on the merge candidate, wherein, When applying the merged motion difference coding method to the current block, derive the motion vector of the current block by adding an offset vector to the motion vector derived based on the merge candidate, and when the maximum number of merge candidates included in the merge candidate list is 1, omit encoding index information indicating the merge candidate of the current block in the merge candidate list; Encode first index information for specifying a motion offset magnitude candidate indicating the magnitude of the offset vector among a plurality of motion offset magnitude candidates; wherein encode a flag indicating the range of the motion offset magnitude candidates, wherein at least one of the maximum value or the minimum value of the motion offset magnitude candidates is set differently according to the value of the flag, or at least one of the maximum value or the minimum value of the motion offset magnitude candidates is set differently according to the motion vector precision of the current block.

6. The video encoding method according to claim 5, wherein, The flag is encoded at the picture level.

7. The video encoding method according to claim 5, wherein, The motion offset size candidate has a value derived by performing a shift operation on the magnitude of the offset vector.

8. The video encoding method according to claim 5, wherein, Encode second index information for specifying a vector direction candidate indicating the direction of the offset vector among a plurality of vector direction candidates.

9. A video decoding device includes an inter prediction unit configured to: Determine whether to apply a merged motion difference encoding method to a current block; Generate a merged candidate list for the current block; determine a merged candidate for the current block based on the merged candidate list; And derive a motion vector of the current block based on the merged candidate, wherein, When the merged motion difference encoding method is applied to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector derived based on the merged candidate, and when the maximum number of merged candidates included in the merged candidate list is 1, the merged candidate is determined without decoding index information indicating one of the merged candidates in the merged candidate list decoded from the bitstream; The inter prediction unit is further configured to determine the magnitude of the offset vector based on first index information indicating one of a plurality of motion offset size candidates; the inter prediction unit is further configured to differently set at least one of the maximum value or the minimum value of the motion offset size candidate according to the value of a flag indicating the range of the motion offset size candidate, or differently set at least one of the maximum value or the minimum value of the motion offset size candidate according to the motion vector precision of the current block.

10. The video decoding device according to claim 9, wherein, The flag is signaled at the picture level.

11. The video decoding device according to claim 9, wherein, The magnitude of the offset vector is obtained by performing a shift operation on the value represented by the motion offset size candidate specified by the first index information.

12. The video decoding device according to claim 9, wherein, The inter prediction unit is configured to determine the direction of the offset vector based on second index information indicating one of a plurality of vector direction candidates.

13. A video encoding device includes an inter prediction unit configured to: Determine whether to apply a merged motion difference encoding method to a current block; Generate a merged candidate list for the current block; Determine a merged candidate for the current block based on the merged candidate list; And Derive a motion vector of the current block based on the merged candidate, wherein, When the merged motion difference encoding method is applied to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector derived based on the merged candidate, and when the maximum number of merged candidates included in the merged candidate list is 1, encoding of index information indicating the merged candidate of the current block in the merged candidate list is omitted; The inter-frame prediction unit is further configured to encode first index information for specifying a motion offset size candidate indicating the size of the offset vector among a plurality of motion offset size candidates, and the inter-frame prediction unit is further configured to encode a flag indicating a range of the motion offset size candidates, wherein at least one of a maximum value or a minimum value of the motion offset size candidate is set differently according to a value of the flag, or the inter-frame prediction unit is further configured to set at least one of the maximum value or the minimum value of the motion offset size candidate differently according to a motion vector precision of the current block.

14. The video coding device according to claim 13, wherein, the flag is encoded at an image level.

15. The video coding device according to claim 13, wherein, the motion offset size candidate has a value derived by performing a shift operation on the size of the offset vector.

16. The video coding device according to claim 13, wherein, the inter-frame prediction unit is configured to encode second index information for specifying a vector direction candidate indicating the direction of the offset vector among a plurality of vector direction candidates.

17. A video decoder, comprising: a processor; and a memory for storing a computer program; wherein, the processor is configured to execute the computer program to perform the video decoding method according to any one of claims 1 to 4.

18. A video encoder, comprising: a processor; and a memory for storing a computer program; wherein, the processor is configured to execute the computer program to perform the video encoding method according to any one of claims 5 to 8.

19. A computer-readable storage medium stores an executable program, which when executed by a processor, implements the video decoding method according to any one of claims 1 to 4, or implements the video encoding method according to any one of claims 5 to 8.

Images