Video signal encoding / decoding method and apparatus for the method

Abstract

The present invention provides a video signal encoding / decoding method and an apparatus for the method. The video decoding method of the present invention includes the following steps: determining whether to apply a merged motion difference encoding method to a current block; generating a merged candidate list for the current block; determining a merged candidate for the current block based on the merged candidate list; and deriving a motion vector for the current block based on the merged candidate.

Description

Technical Field

[0001] The present invention relates to a method for encoding / decoding a video signal and an apparatus for the method. Background Art

[0002] With the trend of larger display panels, there is an increasing need for video services with higher image quality. The biggest problem with high-definition video services is the significant increase in data volume. To solve this problem, research on improving video compression ratios is actively underway. As a representative example, in 2009, the Motion Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) under the International Telecommunication Union - Telecommunication (ITU-T) established the Joint Collaborative Team on Video Coding (JCT-VC). JCT-VC proposed the video compression standard High Efficiency Video Coding (HEVC), which was approved on January 25, 2013, and its compression performance is approximately twice that of H.264 / AVC. With the rapid development of high-definition video services, the limitations of HEVC's performance have gradually emerged. Summary of the Invention

[0003] Technical Problem to be Solved

[0004] An object of the present invention is to provide a method for refining a motion vector derived from merge candidates based on an offset vector when encoding / decoding a video signal and an apparatus for performing the method.

[0005] An object of the present invention is to provide a method for signaling an offset vector when encoding / decoding a video signal and an apparatus for performing the method.

[0006] The technical problems to be achieved by the present invention are not limited to the above-mentioned technical problems, and other technical problems not mentioned will be clearly understood by those of ordinary skill in the technical field to which the present invention pertains through the following description.

[0007] Technical Solution

[0008] The video signal decoding method according to the present invention includes the following steps: determining whether to apply a merged motion difference coding method to a current block; generating a merged candidate list for the current block; determining a merged candidate for the current block based on the merged candidate list; and deriving a motion vector for the current block based on the merged candidate. Among them, when the merged motion difference coding method is applied to the current block, the motion vector for the current block is derived by adding an offset vector to the motion vector derived based on the merged candidate. When the maximum number of merged candidates included in the merged candidate list is more than 1, a merged candidate for the current block is selected based on index information decoded from a bitstream indicating one of the merged candidates in the merged candidate list. The magnitude of the offset vector is determined based on first index information indicating one of a plurality of motion offset magnitude candidates; wherein, at least one of the maximum value or the minimum value of the motion offset magnitude candidates is set differently according to the value of a flag indicating the range of the motion offset magnitude candidates, or at least one of the maximum value or the minimum value of the motion offset magnitude candidates is set differently according to the motion vector precision of the current block.

[0009] In the video signal decoding method according to the present invention, the flag may be signaled at the picture level.

[0010] In the video signal decoding method according to the present invention, the magnitude of the offset vector may be obtained by performing a shift operation on the value represented by the motion offset magnitude candidate specified by the first index information.

[0011] In the video signal decoding method according to the present invention, the direction of the offset vector may be determined based on second index information for indicating one of a plurality of vector direction candidates.

[0012] The video signal encoding method of the present invention includes the following steps: determining whether to apply the merged motion difference encoding method to a current block; generating a merged candidate list for the current block; determining a merged candidate for the current block based on the merged candidate list; and deriving a motion vector for the current block based on the merged candidate, wherein when the merged motion difference encoding method is applied to the current block, the motion vector for the current block is derived by adding an offset vector to the motion vector derived based on the merged candidate, and when the maximum number of merged candidates included in the merged candidate list is more than 1, encoding index information indicating the merged candidate for the current block in the merged candidate list. Encoding first index information for specifying a motion offset size candidate indicating the size of the offset vector among a plurality of motion offset size candidates; wherein encoding a flag indicating the range of the motion offset size candidate, and at least one of the maximum value or the minimum value of the motion offset size candidate is set differently according to the value of the flag, or at least one of the maximum value or the minimum value of the motion offset size candidate is set differently according to the motion vector precision of the current block.

[0013] The video decoding device of the present invention includes an inter prediction unit, and the inter prediction unit is configured to: determine whether to apply the merged motion difference encoding method to a current block; generate a merged candidate list for the current block; determine a merged candidate for the current block based on the merged candidate list; and derive a motion vector for the current block based on the merged candidate, wherein when the merged motion difference encoding method is applied to the current block, the motion vector for the current block is derived by adding an offset vector to the motion vector derived based on the merged candidate, and when the maximum number of merged candidates included in the merged candidate list is more than 1, selecting the merged candidate for the current block based on index information decoded from a bitstream indicating one merged candidate in the merged candidate list. The inter prediction unit is further configured to determine the size of the offset vector based on first index information indicating one of a plurality of motion offset size candidates; the inter prediction unit is further configured to set at least one of the maximum value or the minimum value of the motion offset size candidate differently according to the value of a flag indicating the range of the motion offset size candidate, or set at least one of the maximum value or the minimum value of the motion offset size candidate differently according to the motion vector precision of the current block.

[0014] The video encoding device of the present invention includes an inter prediction unit, and the inter prediction unit is configured to: determine whether to apply a merged motion difference encoding method to a current block; generate a merged candidate list for the current block; determine a merged candidate for the current block based on the merged candidate list; and derive a motion vector for the current block based on the merged candidate, wherein when the merged motion difference encoding method is applied to the current block, the motion vector for the current block is derived by adding an offset vector to the motion vector derived based on the merged candidate, and when the maximum number of merged candidates included in the merged candidate list is more than 1, index information indicating the merged candidate for the current block in the merged candidate list is encoded. The inter prediction unit is further configured to encode first index information for specifying a motion offset size candidate indicating the size of the offset vector among a plurality of motion offset size candidates, and the inter prediction unit is further configured to encode a flag indicating a range of the motion offset size candidate, wherein at least one of a maximum value or a minimum value of the motion offset size candidate is set differently according to a value of the flag, or the inter prediction unit is further configured to set at least one of the maximum value or the minimum value of the motion offset size candidate differently according to a motion vector precision of the current block.

[0015] The video decoder of the present invention includes a processor and a memory storing a computer program. The processor is configured to execute the computer program to perform the above video decoding method.

[0016] The video encoder of the present invention includes a processor and a memory storing a computer program. The processor is configured to execute the computer program to perform the above video encoding method.

[0017] The computer-readable storage medium of the present invention stores an executable program, and when the executable program is executed by a processor, the above video decoding method or video encoding method is implemented.

[0018] The features briefly outlined above for the present invention are merely exemplary embodiments of the detailed description of the present invention to be described later and do not limit the scope of the present invention.

[0019] Advantages of the Invention

[0020] According to the present invention, the inter prediction efficiency can be improved by refining the motion vector of the merged candidate based on the offset vector.

[0021] According to the present invention, the inter prediction efficiency can be improved by adaptively determining the size and direction of the offset vector.

[0022] The effects achievable in the present invention are not limited to the above effects, and other effects not mentioned will be clearly understood by those of ordinary skill in the technical field to which the present invention pertains through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a block diagram of a video encoder according to an embodiment of the present invention.

[0024] Figure 2 is a block diagram of a video decoder according to an embodiment of the present invention.

[0025] Figure 3 is a diagram showing a basic coding tree unit according to an embodiment of the present invention.

[0026] Figure 4 is a diagram showing various partitioning types of coding blocks.

[0027] Figure 5 is a diagram showing an example of partitioning of a coding tree unit.

[0028] Figure 6 is a diagram showing the shape of a basic data unit.

[0029] Figure 7 and Figure 8 is a diagram showing an example of partitioning a coding block into multiple sub-blocks.

[0030] Figure 9 is a flowchart showing an inter-frame prediction method according to an embodiment of the present invention.

[0031] Figure 10 is a diagram showing non-linear motion of an object.

[0032] Figure 11 is a flowchart showing an inter-frame prediction method based on affine motion according to an embodiment of the present invention.

[0033] Figure 12 is a diagram showing an example of an affine seed vector for each affine motion model.

[0034] Figure 13 is a diagram showing an example of an affine vector of a sub-block under a 4-parameter motion model.

[0035] Figure 14 is a flowchart showing a process of deriving motion information of a current block using a merge mode.

[0036] Figure 15 is a diagram showing candidate blocks for deriving merge candidates.

[0037] Figure 16 is a diagram showing the position of a reference sample.

[0038] Figure 17 A diagram showing an example of candidate blocks for deriving merge candidates.

[0039] Figure 18 A diagram showing an example of changing the position of a reference sample.

[0040] Figure 19 A diagram showing an example of changing the position of a reference sample.

[0041] Figure 20 A flowchart showing the process for updating the inter - frame motion information list.

[0042] Figure 21 A diagram showing an embodiment of updating the inter - frame merge candidate list.

[0043] Figure 22 A diagram showing an example of updating the index of stored inter - frame merge candidates.

[0044] Figure 23 A diagram showing the position representing sub - blocks.

[0045] Figure 24 A diagram showing an example of generating an inter - frame motion information list for each inter - frame prediction mode.

[0046] Figure 25 A diagram showing an example of adding inter - frame merge candidates included in the long - term motion information list to the merge candidate list.

[0047] Figure 26 A diagram showing an example of performing a redundancy check only on some merge candidates.

[0048] Figure 27 A diagram showing an example of omitting the redundancy check for a specific merge candidate.

[0049] Figure 28 A diagram showing an offset vector according to the values of distance_idx indicating the magnitude of the offset vector and direction_idx indicating the direction of the offset vector.

[0050] Figure 29 A diagram showing an offset vector according to the values of distance_idx indicating the magnitude of the offset vector and direction_idx indicating the direction of the offset vector.

[0051] Figure 30 A diagram showing the partitioning pattern of coded blocks when applying triangular partitioning technology.

[0052] Figure 31 A diagram showing an example of setting the offset vectors of each sub - unit differently.

[0053] Figure 32 It is a diagram showing motion vector candidates that a fine merge candidate can adopt.

[0054] Figure 33 It is a diagram showing the configuration of a merge refinement offset list.

[0055] Figure 34 and Figure 35 It is a diagram showing an offset vector specified by a merge offset candidate.

[0056] Figure 36 It is a diagram showing candidate blocks for deriving motion vector prediction candidates.

[0057] Figure 37 It is a diagram showing motion vector candidates that can be set as fine motion vector prediction candidates.

[0058] Figure 38 It is a diagram showing the configuration of a prediction vector refinement offset list. Detailed Description of the Invention

[0059] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0060] Video encoding and decoding are performed in units of blocks. For example, encoding / decoding processes such as transformation, quantization, prediction, loop filtering, or reconstruction can be performed on an encoding block, a transform block, or a prediction block.

[0061] Hereinafter, a block to be encoded / decoded is referred to as a "current block". For example, according to the current encoding / decoding processing step, the current block can represent an encoding block, a transform block, or a prediction block.

[0062] In addition, the term "unit" used in this specification represents a basic unit for performing a specific encoding / decoding process, and a "block" can be understood to represent an array of samples of a predetermined size. Unless otherwise specified, "block" and "unit" can be used interchangeably. For example, in the embodiments described later, an encoding block and an encoding unit can be understood to have the same meaning.

[0063] Figure 1 It is a block diagram of a video encoder according to an embodiment of the present invention.

[0064] Referring to Figure 1 , the video encoding device 100 may include an image partitioning unit 110, prediction units 120, 125, a transform unit 130, a quantization unit 135, a rearrangement unit 160, an entropy encoding unit 165, an inverse quantization unit 140, an inverse transform unit 145, a filter unit 150, and a memory 155.

[0065] Figure 1The components shown are individually shown to represent different characteristic functions in a video encoding device, and do not mean that each component is composed of separate hardware or a single software component. That is, for each component, for the sake of convenience of explanation, at least two components in each component are combined into one component or one component is divided into multiple components in such a way that functions are performed thereby. As long as the essence of the present invention is not deviated from, embodiments integrating each component and embodiments separating each component also fall within the scope of the claims of the present invention.

[0066] Moreover, some structural elements are not necessary structural elements for performing the essential functions in the present invention, but are optional structural elements only for improving performance. The present invention can be implemented by including only components required for the essence of the present invention except for the structural elements only for improving performance, and a structure including only essential structural elements except for the optional structural elements only for improving performance also falls within the scope of the claims of the present invention.

[0067] The image partitioning unit 110 may partition an input image into at least one processing unit. In this case, the processing unit may be a Prediction Unit (PU), a Transform Unit (TU), or a Coding Unit (CU). The image partitioning unit 110 partitions an image into a combination of multiple coding units, prediction units, and transform units, and may select a combination of one coding unit, prediction unit, and transform unit based on a predetermined criterion (e.g., a cost function) to encode the image.

[0068] For example, an image may be partitioned into multiple coding units. To partition an image into coding units, a recursive tree structure such as a Quad Tree Structure may be used. A video or a largest coding unit may be used as a root, and coding units may be partitioned into other coding units that may have the number of child nodes equivalent to the number of the partitioned coding units. Coding units that are no longer partitioned according to certain restrictions will become leaf nodes. That is, when assuming that a coding unit can only implement square partitioning, a coding unit can be partitioned into at most 4 other coding units.

[0069] Hereinafter, in an embodiment of the present invention, a coding unit may mean a unit for performing encoding and may also mean a unit for performing decoding.

[0070] A prediction unit within a coding unit may be partitioned into at least one shape such as a square or a rectangle of the same size, or a prediction unit within a coding unit may be partitioned into a shape and / or size different from those of another prediction unit.

[0071] When the prediction unit for performing intra prediction based on a coding unit is not the smallest coding unit, intra prediction can be performed without being divided into multiple N×N prediction units.

[0072] The prediction units 120 and 125 may include an inter prediction unit 120 for performing inter prediction and an intra prediction unit 125 for performing intra prediction. It can be determined whether to use inter prediction or intra prediction for a prediction unit, and specific information (e.g., intra prediction mode, motion vector, reference image, etc.) can be determined based on each prediction method. In this case, the processing unit for performing prediction may be different from the processing unit for determining the prediction method and details. For example, the prediction method and prediction mode, etc. can be determined by the prediction unit, and the prediction can be performed by the transform unit. The residual value (residual block) between the generated prediction block and the original block can be input to the transform unit 130. Also, prediction mode information, motion vector information, etc. for prediction can be encoded together with the residual value in the entropy encoding unit 165 and transmitted to the decoder. When using a specific coding mode, it is also possible to directly encode the original block and transmit it to the decoder without generating a prediction block through the prediction units 120 and 125.

[0073] The inter prediction unit 120 can predict a prediction unit based on information of at least one of the previous image or the subsequent image of the current image, and in some cases, can also predict a prediction unit based on information of a part of the region already encoded within the current image. The inter prediction unit 120 may include a reference image interpolation unit, a motion prediction unit, and a motion compensation unit.

[0074] The reference image interpolation unit receives reference image information from the memory 155 and can generate pixel information of integer pixels or fractional pixels from the reference image. For luminance pixels, in order to generate pixel information of fractional pixels in units of 1 / 4 pixels, a DCT-based 8-tap interpolation filter with different filter coefficients (DCT-based Interpolation Filter) can be used. For chrominance signals, in order to generate pixel information of fractional pixels in units of 1 / 8 pixels, a DCT-based 4-tap interpolation filter with different filter coefficients (DCT-based Interpolation Filter) can be used.

[0075] The motion prediction unit can perform motion prediction based on the reference image interpolated by the reference image interpolation unit. Methods for calculating the motion vector can use a variety of methods such as the full search block matching algorithm (FBMA), the three-step search method (TSS), the new three-step search algorithm (NTS), etc. The motion vector can have a motion vector value in units of 1 / 2 pixel or 1 / 4 pixel based on the interpolated pixels. Different motion prediction methods can be adopted in the motion prediction unit to predict the current prediction unit. The motion prediction methods can use a variety of methods such as the Skip method, the Merge method, the Advanced Motion Vector Prediction (AMVP) method, the Intra Block Copy method, etc.

[0076] The intra prediction unit 125 can generate a prediction unit based on the reference pixel information around the current block, which is the pixel information within the current image. In the case where the adjacent block of the current prediction unit is a block for which inter prediction has been performed and the reference pixel is a pixel for which inter prediction has been performed, the reference pixel included in the block for which inter prediction has been performed can be used as the reference pixel information for the adjacent block for which intra prediction has been performed around it. That is, in the case where the reference pixel is not available, at least one of the available reference pixels can be used to replace the unavailable reference pixel information.

[0077] In intra prediction, the prediction mode can have an angular prediction mode that uses reference pixel information according to the prediction direction and a non-angular mode that does not use direction information during prediction. The mode for predicting luminance information and the mode for predicting chrominance information can be different. To predict chrominance information, the intra prediction mode information used for predicting luminance information or the predicted luminance signal information can be applied.

[0078] When performing intra prediction, if the size of the prediction unit is the same as the size of the transform unit, intra prediction can be performed on the prediction unit based on the pixels on the left side of the prediction unit, the pixels in the upper left corner, and the pixels above. However, when performing intra prediction, if the size of the prediction unit is different from the size of the transform unit, intra prediction can be performed using the reference pixels based on the transform unit. Also, intra prediction using only N×N partitioning for the smallest coding unit can be applied.

[0079] An intra prediction method can generate a prediction block after applying an Adaptive Intra Smoothing (AIS) filter to reference pixels according to a prediction mode. The type of the adaptive intra smoothing filter for reference pixels may be different. To perform the intra prediction method, the intra prediction mode of a current prediction unit can be predicted from the intra prediction modes of prediction units existing around the current prediction unit. In a case where the prediction mode of the current prediction unit is predicted using the mode information predicted from surrounding prediction units, if the intra prediction modes of the current prediction unit and the surrounding prediction units are the same, predetermined flag information can be used to transmit information indicating that the prediction modes of the current prediction unit and the surrounding prediction units are the same. If the intra prediction modes of the current prediction unit and the surrounding prediction units are different, the prediction mode information of the current block can be encoded by performing entropy coding.

[0080] Also, a residual block including residual information can be generated, where the residual information is the difference between a prediction unit that performs prediction based on the prediction units generated in the prediction units 120 and 125 and the original block of the prediction unit. The generated residual block can be input to the transform unit 130.

[0081] In the transform unit 130, a transform method such as a Discrete Cosine Transform (DCT) or a Discrete Sine Transform (DST) can be used to transform the residual block, which includes residual value information between the original block and the prediction units generated by the prediction units 120 and 125. Among them, the DCT transform kernel includes at least one of DCT2 or DCT8, and the DST transform kernel includes DST7. It is possible to determine whether to apply DCT or DST to transform the residual block based on the intra prediction mode information of the prediction unit used to generate the residual block. It is also possible to skip the transformation of the residual block. A flag indicating whether the transformation of the residual block is skipped can be encoded. For a residual block, a luminance component, or a chrominance component (below the 4:4:4 format) whose size is below a threshold, transformation skipping can be allowed.

[0082] The quantization unit 135 can quantize the values transformed into the frequency domain in the transform unit 130. The quantization coefficient can be changed according to the importance of the block or the video. The values calculated in the quantization unit 135 can be provided to the inverse quantization unit 140 and the rearrangement unit 160.

[0083] The rearrangement unit 160 can perform rearrangement of coefficient values on the quantized residual values.

[0084] The rearrangement unit 160 can change the two-dimensional block shape coefficients into a one-dimensional vector form through the Coefficient Scanning method. For example, the rearrangement unit 160 can scan the DC coefficient and the coefficients in the high-frequency domain using the Zig-Zag Scan method and change them into a one-dimensional vector form. Depending on the size of the transform unit and the intra prediction mode, instead of the Zig-Zag scan, a vertical scan that scans the two-dimensional block shape coefficients along the column direction and a horizontal scan that scans the two-dimensional block shape coefficients along the row direction can also be used. That is, it is possible to determine which one of the Zig-Zag scan, the vertical scan, and the horizontal scan to use according to the size of the transform unit and the intra prediction mode.

[0085] The entropy encoding unit 165 can perform entropy encoding based on the values calculated by the rearrangement unit 160. For example, the entropy encoding can use various encoding methods such as the Exponential Golomb code, the Context-Adaptive Variable Length Coding (CAVLC), and the Context-Adaptive Binary Arithmetic Coding (CABAC).

[0086] The entropy encoding unit 165 can encode various information such as the residual value coefficient information, the block type information, the prediction mode information, the partitioning unit information, the prediction unit information, the transmission unit information, the motion vector information, the reference frame information, the interpolation information of the block, and the filtering information of the coding units derived from the rearrangement unit 160 and the prediction units 120, 125.

[0087] Entropy encoding can be performed on the coefficient values of the coding units input to the entropy encoding unit 165 from the rearrangement unit 160.

[0088] The inverse quantization unit 140 and the inverse transform unit 145 perform inverse quantization on the multiple values quantized by the quantization unit 135 and perform inverse transformation on the values transformed by the transform unit 130. The residual values generated in the inverse quantization unit 140 and the inverse transform unit 145 can be combined with the prediction units predicted by the motion prediction unit, the motion compensation unit, and the intra prediction unit included in the prediction units 120, 125 to generate a Reconstructed Block.

[0089] The filter unit 150 can include at least one of a deblocking filter, an offset correction unit, and an Adaptive Loop Filter (ALF).

[0090] The deblocking filter can remove the block distortion generated in the reconstructed image due to the boundary between blocks. To determine whether to perform deblocking, it is possible to judge whether to apply the deblocking filter to the current block based on the pixels included in several columns or rows included in the block. In the case of applying the deblocking filter to the block, strong filtering or weak filtering can be applied according to the required deblocking filter strength. Also, during the process of using the deblocking filter, when performing vertical filtering and horizontal filtering, the horizontal direction filtering and the vertical direction filtering can be processed synchronously.

[0091] The offset correction unit can correct the offset between the video on which deblocking is performed and the original video in pixel units. To perform offset correction on a specified image, the following method can be used, that is, after dividing the pixels included in the video into a predetermined number of regions, determining the region to perform the offset, and applying the offset to the corresponding region, or applying the offset taking into account the edge information of each pixel.

[0092] Adaptive Loop Filtering (ALF) can be performed based on the value obtained by comparing the filtered reconstructed image and the original video. After dividing the pixels included in the video into predetermined groups, filtering can be performed differentially for each group by determining a filter to be used for the corresponding group. Information related to whether to apply adaptive loop filtering and the luminance signal can be transmitted in Coding Units (CUs), and the shape and filter coefficients of the adaptive loop filter to be applied can vary according to each block. Also, an adaptive loop filter of the same type (fixed type) can be applied regardless of the characteristics of the block to which it is applied.

[0093] The memory 155 can store the reconstructed blocks or images calculated by the filter unit 150, and can provide the stored reconstructed blocks or images to the prediction units 120 and 125 during inter-frame prediction.

[0094] Figure 2 is a block diagram of a video decoder according to an embodiment of the present invention.

[0095] Refer to Figure 2 , the video decoder 200 can include an entropy decoding unit 210, a rearrangement unit 215, an inverse quantization unit 220, an inverse transformation unit 225, a prediction unit 230, a prediction unit 235, a filter unit 240, and a memory 245.

[0096] When inputting a video bitstream from a video encoder, the input bitstream can be decoded in the reverse steps of the video encoder.

[0097] The entropy decoding unit 210 can perform entropy decoding in steps opposite to those of the entropy encoding performed in the entropy encoding unit of the video encoder. For example, corresponding to the method performed in the video encoder, various methods such as Exponential Golomb code, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) can be applied.

[0098] The entropy decoding unit 210 can decode information related to intra prediction and inter prediction performed by the encoder.

[0099] The rearrangement unit 215 can perform rearrangement based on the method of rearranging the bitstream entropy decoded by the entropy decoding unit 210 in the encoding unit. Multiple coefficients represented in the form of a one-dimensional vector can be reconstructed into a two-dimensional block-shaped coefficient for rearrangement. The rearrangement unit 215 receives information related to the coefficient scanning performed in the encoding unit and can perform rearrangement by means of reverse scanning based on the scanning order performed in the corresponding encoding unit.

[0100] The inverse quantization unit 220 can perform inverse quantization based on the quantization parameter provided by the encoder and the coefficient values of the rearranged blocks.

[0101] The inverse transformation unit 225 can perform inverse discrete cosine transform and inverse discrete sine transform on the quantization result performed by the video encoder. The inverse discrete cosine transform and inverse discrete sine transform belong to the inverse transform of the transform performed in the transformation unit, that is, the inverse transform of the discrete cosine transform and discrete sine transform. Among them, the DCT transform kernel can include at least one of DCT2 or DCT8, and the DST transform kernel can include DST7. Alternatively, if the transform is skipped in the video encoder, the inverse transform may not be performed in the inverse transformation unit 225. The inverse transform can be performed based on the transfer unit determined in the video encoder. In the inverse transformation unit 225 of the video decoder, the transformation method (e.g., DCT or DST) can be selectively performed according to multiple information such as the prediction method, the size of the current block, and the prediction direction.

[0102] The prediction units 230 and 235 can generate a prediction block based on the information related to the generation of the prediction block provided by the entropy decoding unit 210 and the previously decoded block or image information provided by the memory 245.

[0103] As described above, when performing intra prediction in the same manner as the operations in the video encoder, if the size of the prediction unit is the same as the size of the transform unit, intra prediction is performed on the prediction unit based on the pixels existing on the left side of the prediction unit, the pixels existing in the upper left, and the pixels existing above. If the size of the prediction unit during intra prediction is different from the size of the transform unit, intra prediction can be performed using the reference pixels based on the transform unit. Also, intra prediction using only N×N partitioning for the smallest coding unit can be applied.

[0104] The prediction units 230 and 235 may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit receives various information such as the prediction unit information input from the entropy decoding unit 210, the prediction mode information of the intra prediction method, and the motion prediction related information of the inter prediction method, classifies the prediction unit according to the current coding unit, and can determine whether the prediction unit is performing inter prediction or intra prediction. The inter prediction unit 230 can use the information required for inter prediction of the current prediction unit provided by the video encoder and perform inter prediction on the current prediction unit based on the information included in at least one of the previous image or the subsequent image of the current image to which the current prediction unit belongs. Alternatively, inter prediction can also be performed based on the information of a part of the region that has been reconstructed within the current image to which the current prediction unit belongs.

[0105] To perform inter prediction, it is possible to determine, based on the coding unit, which mode among the skip mode, merge mode, advanced motion vector prediction mode (AMVP mode), and intra block copy mode is the motion prediction method of the prediction unit included in the corresponding coding unit.

[0106] The intra prediction unit 235 can generate a prediction block based on the pixel information within the current image. When the prediction unit is a prediction unit for which intra prediction has been performed, intra prediction can be performed based on the intra prediction mode information of the prediction unit provided by the video encoder. The intra prediction unit 235 may include an adaptive intra smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The adaptive intra smoothing filter is a part that performs filtering on the reference pixels of the current block and can determine whether to apply the filter according to the prediction mode of the current prediction unit. Adaptive intra smoothing filtering can be performed on the reference pixels of the current block using the prediction mode of the prediction unit and the adaptive intra smoothing filter information provided by the video encoder. If the prediction mode of the current block is a mode that does not perform adaptive intra smoothing filtering, the adaptive intra smoothing filter may not be applied.

[0107] For the reference pixel interpolation unit, if the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on the pixel value obtained by interpolating the reference pixels, reference pixels with integer or fractional pixel units can be generated by interpolating the reference pixels. If the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixels, the reference pixels may not be interpolated. If the prediction mode of the current block is the DC mode, the DC filter can generate a prediction block by filtering.

[0108] A reconstructed block or image can be provided to the filter unit 240. The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

[0109] Information related to whether to apply a deblocking filter to a corresponding block or image and information related to whether to apply strong filtering or weak filtering when applying the deblocking filter can be received from the video encoder. The information related to the deblocking filter provided by the video encoder is received from the deblocking filter of the video decoder, and the deblocking filter can be performed on the corresponding block in the video decoder.

[0110] The offset correction unit can perform offset correction on the reconstructed video based on the type and offset information used for offset correction of the video during encoding and the like.

[0111] The ALF can be applied to the coding unit based on the information related to whether to apply the ALF provided by the encoder, the ALF coefficient information, and the like. Such ALF information can be provided by being included in a specific parameter set.

[0112] The memory 245 stores the reconstructed image or block such that the image or block can be used as a reference image or reference block, and can provide the reconstructed image to the output unit.

[0113] ​ FIG. is a diagram showing a basic coding tree unit according to an embodiment of the present invention.

[0114] The coding block with the largest size can be defined as a coding tree block. An image can be divided into a plurality of coding tree units (CTUs). A coding tree unit is the coding unit with the largest size and can also be referred to as the largest coding unit (LCU). ​ FIG. shows an example of dividing an image into a plurality of coding tree units.

[0115] The size of the coding tree unit can be defined at the image level or sequence level. To this end, information indicating the size of the coding tree unit can be signaled through an image parameter set or a sequence parameter set.

[0116] For example, the size of the coding tree unit for the entire image within the sequence can be set to 128×128. Alternatively, either 128×128 or 256×256 at the image level can be determined as the size of the coding tree unit. For example, the size of the coding tree unit in the first image can be set to 128×128, and the size of the coding tree unit in the second image can be set to 256×256.

[0117] Coding blocks can be generated by partitioning the coding tree unit. The coding block represents the basic unit for performing encoding / decoding processing. For example, prediction or transformation can be performed on different coding blocks, or the prediction coding mode can be determined for different coding blocks. Among them, the prediction coding mode represents the method for generating the predicted image. For example, the prediction coding mode can include Intra Prediction (frame - in prediction), Inter Prediction (frame - out prediction), Current Picture Referencing (CPR, or Intra Block Copy (IBC)), or Combined Prediction. For a coding block, at least one of the prediction coding modes of Intra Prediction, Inter Prediction, Current Picture Referencing, or Combined Prediction can be used to generate a prediction block related to the coding block.

[0118] The information indicating the prediction coding mode of the current block can be signaled through the bitstream. For example, the information can be a 1 - bit flag indicating whether the prediction coding mode is an intra mode or an inter mode. Current Picture Referencing or Combined Prediction can be used only when the prediction coding mode of the current block is determined to be an inter mode.

[0119] Current Picture Referencing is used to set the current image as the reference image and obtain the prediction block of the current block from the encoded / decoded region within the current image. Here, the current image means the image including the current block. The information indicating whether Current Picture Referencing is applied to the current block can be signaled through the bitstream. For example, the information can be a 1 - bit flag. When the flag is true, the prediction coding mode of the current block can be determined as Current Picture Referencing, and when the flag is false, the prediction mode of the current block can be determined as Inter Prediction.

[0120] Alternatively, the prediction coding mode of the current block can be determined based on the reference image index. For example, when the reference image index points to the current image, the prediction coding mode of the current block can be determined as the current image reference. When the reference image index points to an image other than the current image, the prediction coding mode of the current block can be determined as inter-frame prediction. That is, the current image reference is a prediction method using the information of the encoded / decoded regions within the current image, and inter-frame prediction is a prediction method using the information of other encoded / decoded images.

[0121] Combined prediction represents a coding mode formed by combining two or more of intra-frame prediction, inter-frame prediction, and current image reference. For example, in the case of applying combined prediction, a first prediction block can be generated based on one of intra-frame prediction, inter-frame prediction, or current image reference, and a second prediction block can be generated based on another one. If the first prediction block and the second prediction block are generated, the final prediction block can be generated through the average operation or weighted sum operation of the first prediction block and the second prediction block. Information indicating whether combined prediction is applied can be signaled through the bitstream. The information can be a 1-bit flag.

[0122] ​ It is a diagram showing various partitioning types of coding blocks.

[0123] The coding block can be divided into multiple coding blocks based on quadtree partitioning, binary tree partitioning, or ternary tree partitioning. The divided coding blocks can also be further divided into multiple coding blocks based on quadtree partitioning, binary tree partitioning, or ternary tree partitioning.

[0124] Quadtree partitioning refers to a partitioning technique that divides the current block into 4 blocks. As a result of quadtree partitioning, the current block can be divided into 4 square partitions (refer to ​ ‘SPLIT_QT’ in part (a)).

[0125] Binary tree partitioning refers to a partitioning technique that divides the current block into 2 blocks. The process of dividing the current block into two blocks along the vertical direction (i.e., using a vertical line crossing the current block) can be called vertical direction binary tree partitioning, and the process of dividing the current block into two blocks along the horizontal direction (i.e., using a horizontal line crossing the current block) can be called horizontal direction binary tree partitioning. After binary tree partitioning, the current block can be divided into 2 non-square partitions. ​ “SPLIT_BT_VER” in part (b) represents the result of vertical direction binary tree partitioning, and ​ “SPLIT_BT_HOR” in part (c) represents the result of horizontal direction binary tree partitioning.

[0126] Trinary tree partitioning refers to a partitioning technique that divides the current block into three blocks. The process of dividing the current block into three blocks along the vertical direction (i.e., using two vertical lines that cross the current block) can be called vertical trinary tree partitioning, and the process of dividing the current block into three blocks along the horizontal direction (i.e., using two horizontal lines that cross the current block) can be called horizontal trinary tree partitioning. After trinary tree partitioning, the current block can be divided into three non-square partitions. In this case, the width / height of the partition located at the center of the current block can be twice the width / height of the other partitions. ​ "SPLIT_TT_VER" in part (d) represents the vertical trinary tree partitioning result, and ​ "SPLIT_TT_HOR" in part (e) represents the horizontal trinary tree partitioning result.

[0127] The number of times of partitioning of a coding tree unit can be defined as the partitioning depth. The maximum partitioning depth of a coding tree unit can be determined at the sequence or picture level. Thus, the maximum partitioning depth of a coding tree unit can vary according to different sequences or pictures.

[0128] Alternatively, the maximum partitioning depth for each of multiple partitioning techniques can be determined separately. For example, the maximum partitioning depth allowed for quadtree partitioning can be different from the maximum partitioning depth allowed for binary tree partitioning and / or trinary tree partitioning.

[0129] The encoder can signal information representing at least one of the partitioning shape or partitioning depth of the current block through the bitstream. The decoder can determine the partitioning shape and partitioning depth of the coding tree unit based on the information parsed from the bitstream.

[0130] ​ is a diagram showing an example of partitioning of a coding tree unit.

[0131] The process of partitioning a coding block using partitioning techniques such as quadtree partitioning, binary tree partitioning, and / or trinary tree partitioning can be called multi-tree partitioning.

[0132] The coding blocks generated by applying multi-tree partitioning to a coding block can be called multiple downstream coding blocks. When the partitioning depth of a coding block is k, the partitioning depth of the multiple downstream coding blocks is set to k + 1.

[0133] On the other hand, for multiple coding blocks with a partitioning depth of k + 1, the coding block with a partitioning depth of k can be called an upstream coding block.

[0134] The partitioning type of the current coding block can be determined based on at least one of the partitioning shape of the upstream coding block or the partitioning type of the neighboring coding blocks. Here, the neighboring coding blocks are adjacent to the current coding block, and can include at least one of the upper neighboring block, the left neighboring block, or the neighboring block adjacent to the upper left corner of the current coding block. The partitioning type can include at least one of whether to partition a quadtree, whether to partition a binary tree, the binary tree partitioning direction, whether to partition a ternary tree, or the ternary tree partitioning direction.

[0135] To determine the partitioning shape of a coding block, information indicating whether the coding block is partitioned can be signaled through the bitstream. The information is a 1-bit flag "split_cu_flag", and when the flag is true, it indicates that the coding block is partitioned by a multi-tree partitioning technique.

[0136] When "split_cu_flag" is true, information indicating whether the coding block is partitioned by a quadtree can be signaled through the bitstream. The information is a 1-bit flag "split_qt_flag", and when the flag is true, the coding block can be partitioned into 4 blocks.

[0137] For example, in ​ the illustrated example shows that as the coding tree unit is partitioned by a quadtree, 4 coding blocks with a partitioning depth of 1 are generated. Also, it is illustrated that the first coding block and the fourth coding block among the 4 coding blocks generated as a result of the quadtree partitioning are again subjected to quadtree partitioning. Finally, 4 coding blocks with a partitioning depth of 2 can be generated.

[0138] Moreover, coding blocks with a partitioning depth of 3 can be generated by applying quadtree partitioning again to the coding blocks with a partitioning depth of 2.

[0139] When no quadtree partitioning is applied to the coding block, it can be determined whether to perform binary tree partitioning or ternary tree partitioning on the coding block by considering at least one of the size of the coding block, whether the coding block is located at the image boundary, the maximum partitioning depth, or the partitioning shape of the neighboring blocks. When it is determined to perform binary tree partitioning or ternary tree partitioning on the coding block, information indicating the partitioning direction can be signaled through the bitstream. The information can be a 1-bit flag "mtt_split_cu_vertical_flag". The partitioning direction can be determined to be the vertical direction or the horizontal direction based on the flag. Additionally, information indicating which of binary tree partitioning or ternary tree partitioning is applied to the coding block can be signaled through the bitstream. The information can be a 1-bit flag "mtt_split_cu_binary_flag". It can be determined whether to apply binary tree partitioning or ternary tree partitioning to the coding block based on the flag.

[0140] For example, in ​In the illustrated example, vertical binary tree partitioning is applied to an encoded block with a partitioning depth of 1, vertical ternary tree partitioning is applied to the left encoded block among the encoded blocks generated as the partitioning result, and vertical binary tree partitioning is applied to the right encoded block.

[0141] When implementing a device for encoding or decoding video, there is a problem that it is difficult to process regions larger than a threshold due to hardware performance. For example, there may be a problem that when the hardware performance allows processing at most 4096 samples simultaneously, a data unit of size 64×64 should be redundantly accessed and processed, and data cannot be processed simultaneously for regions with more than 4096 samples. As described above, the basic unit of data processing can be defined as a pipeline-based basic data unit (virtual processing data unit, VPDU, hereinafter referred to as the basic data unit).

[0142] The basic data unit can be divided into square, non-square, or non-rectangular types.

[0143] ​ is a diagram showing the shape of the basic data unit.

[0144] The basic data unit can include samples equal to or less than the maximum number of samples that can be processed simultaneously. For example, as in ​ (a) In the illustrated example, a square block of size 64×64 can be set as the basic data unit. Alternatively, a non-square block can be set as the basic data unit. For example, as in ​ (b) or ​ (c) In the illustrated example, a block of size 32×128 or a block of size 64×32 can be set as the basic data unit.

[0145] Although not shown, basic data units in the shape of triangles, L-shapes, or polygons can be defined.

[0146] Information for determining the basic data unit can be signaled through the bitstream. The information can be used to determine at least one of the size or shape of the basic data unit. Based on the information, it can be determined whether non-square basic data units are allowed or not.

[0147] Alternatively, at least one of the size or shape of the basic data unit can be predefined in the encoder and decoder.

[0148] It is possible to determine whether to allow a partitioning type of a coding block in consideration of the size of a basic data unit. For example, when a coding block generated as a result of partitioning a coding block is larger than the basic data unit, such partitioning may not be allowed. Alternatively, when a non-square coding block generated as a result of partitioning a coding block is larger than the basic data unit, such partitioning may not be allowed. For example, when the width or height of a coding block is greater than a threshold, or when the number of samples included in a coding block is greater than a threshold, binary tree or ternary tree partitioning may not be allowed. Accordingly, coding of information related to binary tree or ternary tree partitioning may be omitted.

[0149] Alternatively, it may be set that a coding block larger than the basic data unit must be partitioned. Alternatively, it may be set that a binary tree partitioning or a ternary tree partitioning must be performed on a coding block larger than the basic data unit. Accordingly, for a coding block larger than the basic data unit, although a flag split_flag indicating whether the coding block is partitioned is not coded, the value of the flag may be derived as 1.

[0150] As another example, a coding block larger than the basic data unit may be partitioned into a plurality of sub-blocks. Here, the sub-blocks may be set as prediction units that are basic units for prediction, or as transform units that are basic units for transform and / or quantization. In this case, partitioning a coding block into a plurality of prediction units may be defined as VPDU prediction unit partitioning, and partitioning a coding block into a plurality of transform units may be defined as VPDU transform unit partitioning.

[0151] At least one of VPDU prediction unit partitioning and VPDU transform unit partitioning may be applied to a coding block. The partitioning type of a coding block according to the application of VPDU prediction unit partitioning may be set to be the same as the partitioning type of a coding block according to the application of VPDU transform unit partitioning.

[0152] When only VPDU prediction unit partitioning is applied to a coding block, prediction is performed on each sub-block, but transform and / or quantization may be performed on the coding block. In this case, a prediction mode such as a predictive coding mode, an intra prediction mode, or an inter prediction mode may be determined for the coding block.

[0153] When only VPDU transform unit partitioning is applied to a coding block, prediction is performed on the sub-blocks, but transform and / or quantization may be performed on each sub-block.

[0154] ​ and ​ are diagrams showing examples of partitioning a coding block into a plurality of sub-blocks.

[0155] ​ is a diagram showing a partitioning pattern when only square basic data units are allowed, ​It is a diagram showing the partitioning patterns when square basic data units and non-square basic data units are allowed.

[0156] Assume that when only square basic data units are allowed, in ​ (a) and ​ (b), CU0 and CU3 are defined as two different VPDUs, and CU1 is defined as four different VPDUs. Therefore, CU0 and CU3 can be partitioned into two sub-blocks, and CU1 can be partitioned into four sub-blocks.

[0157] Assume that when square basic data units and non-square basic data units are allowed, in ​ (a) and ​ (b), CU0 and CU3 can be defined as one VPDU, and CU1 can be defined as using two different VPDUs. Therefore, CU0 and CU3 are not partitioned into sub-blocks, while CU1 can be partitioned into two sub-blocks.

[0158] In this case, CU1 can be partitioned into square sub-blocks or non-square sub-blocks. For example, based on the horizontal line that divides CU1 vertically, CU1 can be partitioned into two square sub-blocks. Alternatively, CU1 can be partitioned into two non-square sub-blocks based on the vertical line that divides CU1 horizontally.

[0159] When there are multiple partitioning type candidates applicable to an encoding block, information indicating any one of the multiple partitioning type candidates can be signaled through the bitstream. For example, the information can indicate whether the encoding block is partitioned into square sub-blocks or whether the encoding block is partitioned into non-square sub-blocks.

[0160] Alternatively, partitioning the encoding block into square sub-blocks can be set to have a higher priority than partitioning the encoding block into non-square sub-blocks. For example, when the encoding block cannot be partitioned into square sub-blocks, partitioning the encoding block into non-square sub-blocks can be allowed.

[0161] Alternatively, the partitioning type of an encoding block can be determined based on the partitioning type of the encoding block that is the parent node. For example, when the parent node encoding block is partitioned based on a ternary tree, it can be set to partition the encoding block into square sub-blocks. On the other hand, when the parent node encoding block is partitioned based on a binary tree or a ternary tree, it can be set to partition the encoding block into non-square sub-blocks.

[0162] Inter-frame prediction refers to predicting the prediction coding mode of the current block using information from previous images. For example, a block at the same position as the current block in the previous image (hereinafter referred to as a collocated block) can be set as the prediction block of the current block. Hereinafter, a prediction block generated based on a block at the same position as the current block is referred to as a collocated prediction block.

[0163] On the other hand, if an object existing in the previous image has moved to another position in the current image, the movement of the object can be used to effectively predict the current block. For example, if the movement direction and size of the object can be known by comparing the previous image and the current image, the movement information of the object can be considered to generate a prediction block (or a prediction image) of the current block. Hereinafter, a prediction block generated using movement information can be referred to as a motion prediction block.

[0164] A residual block can be generated by subtracting the prediction block from the current block. In this case, when there is movement of an object, the motion prediction block can be used instead of the collocated prediction block, thereby reducing the energy of the residual block and thus improving the compression performance of the residual block.

[0165] As described above, the process of generating a prediction block using movement information can be referred to as motion compensation prediction. In most inter-frame predictions, a prediction block can be generated based on motion compensation prediction.

[0166] The movement information can include at least one of a motion vector, a reference image index, a prediction direction, or a bi-directional weighting value index. The motion vector represents the movement direction and size of the object. The reference image index specifies the reference image of the current block among multiple reference images included in the reference image list. The prediction direction refers to any one of unidirectional L0 prediction, unidirectional L1 prediction, or bi-directional prediction (L0 prediction and L1 prediction). At least one of the motion information in the L0 direction or the motion information in the L1 direction can be used according to the prediction direction of the current block. The bi-directional weighting value index specifies the weighting value applied to the L0 prediction block and the weighting value applied to the L1 prediction block.

[0167] ​ is a flowchart showing an inter-frame prediction method according to an embodiment of the present invention.

[0168] Refer to ​ , the inter-frame prediction method includes: determining an inter-frame prediction mode of the current block (S901); obtaining movement information of the current block according to the determined inter-frame prediction mode (S902); and performing motion compensation prediction of the current block based on the obtained movement information (S903).

[0169] Among them, the inter-frame prediction mode represents various techniques for determining the motion information of a current block, which may include an inter-frame prediction mode using translational motion information and an inter-frame prediction mode using affine motion information. For example, the inter-frame prediction mode using translational motion information may include a merge mode and an advanced motion vector prediction mode, and the inter-frame prediction mode using affine motion information may include an affine merge mode and an affine motion vector prediction mode. According to the inter-frame prediction mode, the motion information of the current block can be determined based on adjacent blocks adjacent to the current block or information parsed from the bitstream.

[0170] Hereinafter, the inter-frame prediction method using affine motion information will be described in detail.

[0171] ​ It is a diagram showing the non-linear motion of an object.

[0172] The motion of an object within a video may be non-linear motion. For example, as in the ​ example shown, non-linear motion of an object may occur, such as zooming in, zooming out, rotating, and affine transformation of a camera. When non-linear motion of an object occurs, the motion of the object cannot be effectively represented by a translational motion vector. Thus, in a portion where non-linear motion of an object occurs, affine motion can be used instead of translational motion, thereby improving the coding efficiency.

[0173] ​ It is a flowchart showing an inter-frame prediction method based on affine motion according to an embodiment of the present invention.

[0174] It can be determined whether to apply an inter-frame prediction technique based on affine motion to a current block based on information parsed from the bitstream. Specifically, it can be determined whether to apply an inter-frame prediction technique based on affine motion to the current block based on at least one of a flag indicating whether to apply an affine merge mode to the current block or a flag indicating whether to apply an affine motion vector prediction mode to the current block.

[0175] When applying an inter-frame prediction technique based on affine motion to a current block, an affine motion model of the current block can be determined (S1101). The affine motion model can be determined by at least one of a 6-parameter affine motion model or a 4-parameter affine motion model. The 6-parameter affine motion model uses 6 parameters to represent affine motion, and the 4-parameter affine motion model uses 4 parameters to represent affine motion.

[0176] Equation 1 is the case of representing affine motion using 6 parameters. Affine motion represents translational motion with respect to a predetermined region determined by an affine seed vector.

[0177]

Equation 1

[0178] vx =ax-by+e

[0179] v y =cx+dy+f

[0180] While using six parameters to represent affine motion can express complex motion, the number of bits required to encode each parameter increases, which reduces encoding efficiency. Therefore, affine motion can also be represented using four parameters. Equation 2 shows the case of using four parameters to represent affine motion.

[0181] Equation 2

[0182] v x =ax-by+e

[0183] v y =bx+ay+f

[0184] Information used to determine the affine motion model for the current block can be encoded and signaled via the codestream. For example, the information can be a 1-bit flag, "affine_type_flag." A value of 0 for the flag indicates that a 4-parameter affine motion model is applied, while a value of 1 for the flag indicates that a 6-parameter affine motion model is applied. The flag can be encoded per slice, tile, or block (e.g., coding block or coding tree unit). When the flag is signaled at the slice level, the affine motion model determined at the slice level can be applied to all blocks in the slice.

[0185] Alternatively, the affine motion model of the current block can be determined based on the affine inter prediction mode of the current block. For example, when the affine merge mode is applied, the affine motion model of the current block can be determined as a 4-parameter motion model. On the other hand, when the affine motion vector prediction mode is applied, information used to determine the affine motion model of the current block can be encoded and signaled via the codestream. For example, when the affine motion vector prediction mode is applied to the current block, the affine motion model of the current block can be determined based on the 1-bit flag "affine_type_flag".

[0186] Next, an affine seed vector for the current block may be derived (S1102). When a 4-parameter affine motion model is selected, motion vectors at two control points of the current block may be derived. On the other hand, when a 6-parameter affine motion model is selected, motion vectors at three control points of the current block may be derived. The motion vectors at the control points may be referred to as affine seed vectors. The control points may include at least one of the upper left corner, upper right corner, or lower left corner of the current block.

[0187] ​ is a diagram showing an example of an affine seed vector for each affine motion model.

[0188] In a four-parameter affine motion model, affine seed vectors related to two of the upper left corner, upper right corner, or lower left corner can be derived. For example, in the example shown in ​ (a), when a four-parameter affine motion model is selected, the affine seed vector SV0 of the upper left corner of the current block (e.g., the upper left sample (x0, y0)) and the affine seed vector sv1 of the upper right corner of the current block (e.g., the upper right sample (x1, y1)) can be used to derive the affine vector. The affine seed vector related to the lower left corner can also be used instead of the affine seed vector related to the upper left corner, or the affine seed vector related to the lower left corner can also be used instead of the affine seed vector related to the upper right corner.

[0189] In a six-parameter affine motion model, affine seed vectors related to the upper left corner, upper right corner, and lower left corner can be derived. For example, in the example shown in ​ (b), when a six-parameter affine motion model is selected, the affine seed vector SV0 of the upper left corner of the current block (e.g., the upper left sample (x0, y0)), the affine seed vector sv1 of the upper right corner of the current block (e.g., the upper right sample (x1, y1)), and the affine seed vector sv2 of the lower left corner of the current block (e.g., the lower left sample (x2, y2)) can be used to derive the affine vector.

[0190] In the embodiments described later, in the four-parameter affine motion model, the affine seed vectors of the upper left control point and the upper right control point are respectively referred to as the first affine seed vector and the second affine seed vector. In the embodiments using the first affine seed vector and the second affine seed vector described later, at least one of the first affine seed vector and the second affine seed vector can be replaced by the affine seed vector of the lower left control point (the third affine seed vector) or the affine seed vector of the lower right control point (the fourth affine seed vector).

[0191] Moreover, in the six-parameter affine motion model, the affine seed vectors of the upper left control point, the upper right control point, and the lower left control point are respectively referred to as the first affine seed vector, the second affine seed vector, and the third affine seed vector. In the embodiments using the first affine seed vector, the second affine seed vector, and the third affine seed vector described later, at least one of the first affine seed vector, the second affine seed vector, and the third affine seed vector can be replaced by the affine seed vector of the lower right control point (the fourth affine seed vector).

[0192] The affine vector of each sub-block can be derived by using the affine seed vector (S1103). Among them, the affine vector represents the translational motion vector derived based on the affine seed vector. The affine vector of the sub-block can be referred to as the affine sub-block motion vector or the sub-block motion vector.

[0193] ​ A diagram showing an example of an affine vector of a sub-block under a 4-parameter motion model.

[0194] The affine vector of the sub-block can be derived based on the positions of the control points, the position of the sub-block, and the affine seed vector. For example, Equation 3 shows an example of deriving the affine sub-block vector.

[0195]

Equation 3

[0196]

[0197]

[0198] In Equation 3, (x, y) represents the position of the sub-block. Among them, the position of the sub-block represents the position of the reference sample included in the sub-block. The reference sample can be the sample located at the upper left corner of the sub-block, or the sample with at least one of the x-axis or y-axis coordinates located at the central position. (x0, y0) represents the position of the first control point, and (sv 0x , sv 0y ) represents the first affine seed vector. In addition, (x1, y1) represents the position of the second control point, and (sv 1x , sv 1y ) represents the second affine seed vector.

[0199] When the first control point and the second control point correspond to the upper left corner and the upper right corner of the current block respectively, x1 - x0 can be set to the same value as the width of the current block.

[0200] Thereafter, the motion compensation prediction of each sub-block can be performed using the affine vector of each sub-block (S1104). After performing the motion compensation prediction, a prediction block related to each sub-block can be generated. The prediction block of the sub-block can be set as the prediction block of the current block.

[0201] Next, the inter-frame prediction method using translational motion information will be described in detail.

[0202] The motion information of the current block can be derived from the motion information of other blocks of the current block. Among them, the other blocks can be the blocks that are preferentially encoded / decoded by inter-frame prediction compared to the current block. Defining the situation where the motion information of the current block is set to be the same as the motion information of other blocks as the merge mode. And defining the situation where the motion vector of other blocks is set as the predicted value of the motion vector of the current block as the motion vector prediction mode.

[0203] ​ A flowchart showing the process of deriving the motion information of the current block using the merge mode.

[0204] The merge candidates of the current block can be derived (S1401). The merge candidates of the current block can be derived from the blocks that have been encoded / decoded using inter prediction before the current block.

[0205] ​ FIG. is a diagram showing candidate blocks for deriving merge candidates.

[0206] The candidate blocks can include at least one of an adjacent block containing samples adjacent to the current block or a non - adjacent block containing samples not adjacent to the current block. Hereinafter, the samples used to determine the candidate blocks are designated as reference samples. In addition, the reference samples adjacent to the current block are called adjacent reference samples, and the reference samples not adjacent to the current block are called non - adjacent reference samples.

[0207] The adjacent reference samples can be included in the adjacent column of the left - most column of the current block or the adjacent row of the top - most row of the current block. For example, if the coordinates of the upper - left sample of the current block are (0, 0), then at least one of the blocks including the reference sample at the position (-1, H - 1), the block including the reference sample at the position (W - 1, -1), the block including the reference sample at the position (W, -1), the block including the reference sample at the position (-1, H), or the block including the reference sample at the position (-1, -1) can be used as a candidate block. Referring to the accompanying drawings, the adjacent blocks with indices 0 to 4 can be used as candidate blocks.

[0208] The non - adjacent reference samples represent samples for which at least one of the x - axis distance or y - axis distance between the adjacent reference samples of the current block has a predefined value. For example, at least one of the blocks including the reference sample with a predefined x - axis distance from the left - hand reference sample, the block including the non - adjacent sample with a predefined y - axis distance from the upper reference sample, or the block including the non - adjacent sample with a predefined x - axis distance and y - axis distance from the upper - left reference sample can be used as a candidate block. The predefined value can be an integer such as 4, 8, 12, 16, etc. Referring to the accompanying drawings, at least one of the blocks with indices 5 to 26 can be used as a candidate block.

[0209] Samples that are not on the same vertical line, horizontal line, or diagonal line as the adjacent reference samples can be set as non - adjacent reference samples.

[0210] ​ FIG. is a diagram showing the positions of the reference samples.

[0211] As in ​In the example shown, the x coordinate of the upper non - adjacent reference sample can be set to be different from the x coordinate of the upper adjacent reference sample. For example, when the position of the upper adjacent reference sample is (W - 1, - 1), the position of the upper non - adjacent reference sample separated from the upper adjacent reference sample by N along the y - axis can be set to ((W / 2) - 1, - 1 - N), and the position of the upper non - adjacent reference sample separated from the upper adjacent reference sample by 2N along the y - axis can be set to (0, - 1 - 2N). That is, the position of the non - adjacent reference sample can be determined based on the position of the adjacent reference sample and the distance from the adjacent reference sample.

[0212] Hereinafter, the candidate block containing the adjacent reference sample in the candidate block is called an adjacent block, and the block containing the non - adjacent reference sample is called a non - adjacent block.

[0213] When the distance between the current block and the candidate block is greater than or equal to a threshold value, the candidate block can be set as not available for merge candidates. The threshold value can be determined based on the size of the coding tree unit. For example, the threshold value can be set to the height of the coding tree unit (ctu_height), or a value obtained by adding or subtracting an offset value to the height of the coding tree unit (e.g., ctu_height ± N). The offset value N is a predefined value in the encoder and decoder, and can be set to 4, 8, 16, 32, or ctu_height.

[0214] When the difference between the y - axis coordinate of the current block and the y - axis coordinate of the samples included in the candidate block is greater than the threshold value, the candidate block can be determined as not available for merge candidates.

[0215] Alternatively, a candidate block that does not belong to the same coding tree unit as the current block can be set as not available for merge candidates. For example, when the reference sample exceeds the upper boundary of the coding tree unit to which the current block belongs, the candidate block including the reference sample can be set as not available for merge candidates.

[0216] If, when the upper boundary of the current block is adjacent to the upper boundary of the coding tree unit, multiple candidate blocks are determined as not available for merge candidates, this will reduce the coding / decoding efficiency of the current block. To solve the above problem, candidate blocks can be set so that the number of candidate blocks located above the current block is more than the number of candidate blocks located to the left of the current block.

[0217] ​ It is a diagram showing an example of candidate blocks for deriving merge candidates.

[0218] As in ​In the example shown, the upper block belonging to N block columns above the current block and the left block belonging to M block columns to the left of the current block can be set as candidate blocks. At this time, by setting M to be greater than N, the number of left candidate blocks can be set to be greater than the number of upper candidate blocks.

[0219] For example, the difference between the y-axis coordinate of the reference sample in the current block and the y-axis coordinate of the upper block that can be used as a candidate block can be set to not exceed N times the height of the current block. Additionally, the difference between the x-axis coordinate of the reference sample in the current block and the x-axis coordinate of the left block that can be used as a candidate block can be set to not exceed M times the width of the current block.

[0220] For example, in ​ the example shown, the blocks belonging to two block columns above the current block and the blocks belonging to five block columns to the left of the current block are shown as candidate blocks.

[0221] As another example, when a candidate block does not belong to the same coding tree unit as the current block, a block belonging to the same coding tree unit as the current block, or a block containing a reference sample adjacent to the boundary of the coding tree unit can be used instead of the candidate block to derive merge candidates.

[0222] ​ is a diagram showing an example of changing the position of a reference sample.

[0223] When a reference sample is included in a coding tree unit different from the current block and the reference sample is not adjacent to the boundary of the coding tree unit, a reference sample adjacent to the boundary of the coding tree unit can be used instead of the reference sample to determine the candidate block reference sample.

[0224] For example, in ​ (a) and ​ (b) the example shown, when the upper boundary of the current block and the upper boundary of the coding tree unit are in contact with each other, the reference sample above the current block belongs to a coding tree unit different from the current block. The reference samples in the reference samples belonging to a coding tree unit different from the current block that are not adjacent to the upper boundary of the coding tree unit can be replaced with samples adjacent to the upper boundary of the coding tree unit.

[0225] For example, in the example shown in ​ (a), the reference sample at position 6 is replaced with the sample at position 6' on the upper boundary of the coding tree unit, as shown in ​As shown in the example of (b), the reference sample at position 15 is replaced by a sample at position 15' on the upper boundary of the coding tree unit. At this time, the y - coordinate of the replacement sample can be changed to an adjacent position of the coding tree unit, and the x - coordinate of the replacement sample can be set to be the same as that of the reference sample. For example, the sample at position 6' can have the same x - coordinate as the sample at position 6, and the sample at position 15' can have the same x - coordinate as the sample at position 15.

[0226] Alternatively, the value obtained by adding or subtracting an offset value to / from the x - coordinate of the reference sample can be set as the x - coordinate of the replacement sample. For example, when the x - coordinates of adjacent and non - adjacent reference samples above the current block are the same, the value obtained by adding or subtracting an offset value to / from the x - coordinate of the reference sample can be set as the x - coordinate of the replacement sample. This is to prevent the replacement sample for replacing a non - adjacent reference sample from being in the same position as other non - adjacent reference samples or adjacent reference samples.

[0227] ​ is a diagram showing an example of changing the position of the reference sample.

[0228] When replacing a reference sample included in a coding tree unit different from the current block and not adjacent to the boundary of the coding tree unit with a sample on the boundary of the coding tree unit, the value obtained by adding or subtracting an offset value to / from the x - coordinate of the reference sample can be set as the x - coordinate of the replacement sample.

[0229] For example, in ​ the example shown, the reference sample at position 6 and the reference sample at position 15 can be respectively replaced by a sample at position 6' and a sample at position 15' having the same y - coordinate as the row adjacent to the upper boundary of the coding tree unit. At this time, the x - coordinate of the sample at position 6' can be set to a value whose difference from the x - coordinate of the reference sample at position 6 is W / 2, and the x - coordinate of the sample at position 15' can be set to a value whose difference from the x - coordinate of the reference sample at position 15 is W - 1.

[0230] Different from ​ and ​ the examples shown, the y - coordinate of the row above the top - most row of the current block or the y - coordinate of the upper boundary of the coding tree unit can also be set as the y - coordinate of the replacement sample.

[0231] Although not illustrated, samples substituting the reference sample can also be determined based on the left boundary of the coding tree unit. For example, when the reference sample is not included in the same coding tree unit as the current block and is not adjacent to the left boundary of the coding tree unit, the reference sample can be replaced with a sample adjacent to the left boundary of the coding tree unit. At this time, the replacement sample can have the same y coordinate as the reference sample, or can have a y coordinate obtained by adding or subtracting an offset value to the y coordinate of the reference sample.

[0232] After that, the block including the replacement sample can be set as a candidate block, and merge candidates for the current block can be derived based on the candidate block.

[0233] Merge candidates can also be derived from temporally adjacent blocks included in an image different from the current block. For example, merge candidates can be derived from co-located blocks included in a co-located image.

[0234] The motion information of the merge candidate can be set to be the same as the motion information of the candidate block. For example, at least one of the motion vector, reference image index, prediction direction, or bi-directional weighted value index of the candidate block can be set as the motion information of the merge candidate.

[0235] A merge candidate list including the merge candidates can be generated (S1402). The merge candidates can be classified into adjacent merge candidates derived from adjacent blocks adjacent to the current block, and non-adjacent merge candidates derived from non-adjacent blocks.

[0236] Indices of multiple merge candidates within the merge candidate list can be assigned in a predetermined order. For example, the index assigned to an adjacent merge candidate can have a smaller value than the index assigned to a non-adjacent merge candidate. Alternatively, based on ​ or ​ the index of each block shown, the index can be assigned to each merge candidate.

[0237] When multiple merge candidates are included in the merge candidate list, at least one of the multiple merge candidates can be selected (S1403). At this time, it can be signaled in the bitstream whether the motion information of the current block is information derived from an adjacent merge candidate. The information can be a 1-bit flag. For example, it can be signaled in the bitstream a syntax element isAdjancentMergeFlag indicating whether the motion information of the current block is derived from an adjacent merge candidate. When the value of the syntax element isAdjancentMergeFlag is 1, the motion information of the current block can be derived based on the adjacent merge candidate. On the other hand, when the value of the syntax element isAdjancentMergeFlag is 0, the motion information of the current block can be derived based on the non-adjacent merge candidate.

[0238] Table 1 shows a syntax table including the syntax element isAdjancentMergeFlag.

[0239]

Table 1

[0240]

[0241]

[0242] Information for specifying any one of a plurality of merge candidates can be signaled through the bitstream. For example, information indicating an index of any one of the merge candidates included in the merge candidate list can be signaled through the bitstream.

[0243] When isAdjacentMergeflag is 1, the syntax element merge_idx for determining any one of the adjacent merge candidates can be signaled. The maximum value of the syntax element merge_idx can be set to a value with a difference of 1 from the number of adjacent merge candidates.

[0244] When isAdjacentMergeflag is 0, the syntax element NA_merge_idx for determining any one of the non - adjacent merge candidates can be signaled. The syntax element NA_merge_idx indicates a value obtained by subtracting the index of the non - adjacent merge candidate from the number of adjacent merge candidates. The decoder can select a non - adjacent merge candidate by adding the number of adjacent merge candidates to the index determined according to NA_merge_idx.

[0245] When the number of merge candidates included in the merge candidate list is less than a threshold, the merge candidates included in the inter - frame motion information list can be added to the merge candidate list. Here, the threshold can be the maximum number of merge candidates that the merge candidate list can include or a value obtained by subtracting an offset from the maximum number of merge candidates. The offset can be an integer such as 1 or 2. The inter - frame motion information list can include merge candidates derived based on blocks encoded / decoded before the current block.

[0246] The inter - frame motion information list includes merge candidates derived from blocks encoded / decoded for intra - picture base inter - prediction. For example, the motion information of the merge candidates included in the inter - frame motion information list can be set to be the same as the motion information of the blocks encoded / decoded based on inter - frame prediction. Here, the motion information can include at least one of a motion vector, a reference image index, a prediction direction, or a bi - directional weighted value index.

[0247] For ease of explanation, the merge candidates included in the inter - frame motion information list are referred to as inter - frame merge candidates.

[0248] The maximum number of merge candidates that can be included in the inter - frame motion information list can be predefined in the encoder and decoder. For example, the maximum number of merge candidates that can be included in the inter - frame motion information list can be 1, 2, 3, 4, 5, 6, 7, 8 or greater (e.g., 16).

[0249] Alternatively, information representing the maximum number of merge candidates of the inter - frame motion information list can be signaled via a bitstream. The information is signaled at the sequence level, picture level, or slice level.

[0250] Alternatively, the maximum number of merge candidates of the inter - frame motion information list can be determined according to the size of the picture, the size of the slice, or the size of the coding tree unit.

[0251] The inter - frame motion information list can be initialized on a per - picture, per - slice, per - tile, per - patch, per - coding tree unit, or per - coding tree unit row (row or column) basis. For example, when a slice is initialized, the inter - frame motion information list is also initialized, and it is possible that the inter - frame motion information list does not include any merge candidates.

[0252] Alternatively, information indicating whether to initialize the inter - frame motion information list can also be signaled via a bitstream. The information can be signaled at the slice level, tile level, patch level, or block level. Before the information indicates to initialize the inter - frame motion information list, the configured inter - frame motion information list can be used.

[0253] Alternatively, information related to inter - frame merge candidates can be signaled via the picture parameter set or slice header. Even when a slice is initialized, the inter - frame motion information list can include initial inter - frame merge candidates. Thus, the inter - frame merge candidates can be used for the blocks that are first encoded / decoded within the slice.

[0254] According to the encoding / decoding order, blocks are encoded / decoded, and multiple blocks encoded / decoded based on inter - frame prediction can be sequentially set as inter - frame merge candidates according to the encoding / decoding order.

[0255] ​ is a flowchart showing the process for updating the inter - frame motion information list.

[0256] When performing inter - frame prediction on the current block (S2001), inter - frame merge candidates can be derived based on the current block (S2002). The motion information of the inter - frame merge candidates can be set to be the same as the motion information of the current block.

[0257] When the inter - frame motion information list is empty (S2003), the inter - frame merge candidates derived based on the current block can be added to the inter - frame motion information list (S2004).

[0258] When the inter-frame motion information list already includes the inter-frame merge candidate (S2003), a redundancy check can be performed on the motion information of the current block (or the inter-frame merge candidate derived based on the current block) (S2005). The redundancy check is used to determine whether the motion information of the inter-frame merge candidate already stored in the inter-frame motion information list is the same as the motion information of the current block. All the inter-frame merge candidates already stored in the inter-frame motion information list can be used as objects to perform the redundancy check. Alternatively, the inter-frame merge candidates with indexes above or below a threshold among the inter-frame merge candidates already stored in the inter-frame motion information list can be used as objects to perform the redundancy check.

[0259] When there is no inter-frame merge candidate having the same motion information as the motion information of the current block, the inter-frame merge candidate derived based on the current block can be added to the inter-frame motion information list (S2008). It can be determined whether the inter-frame merge candidates are the same based on whether their motion information (e.g., motion vector and / or reference image index, etc.) is the same.

[0260] In this case, when the maximum number of inter-frame merge candidates has already been stored in the inter-frame motion information list (S2006), the earliest inter-frame merge candidate is deleted (S2007), and the inter-frame merge candidate derived based on the current block can be added to the inter-frame motion information list (S2008).

[0261] Multiple inter-frame merge candidates can be identified according to their respective indexes. When adding the inter-frame merge candidate derived from the current block to the inter-frame motion information list, the lowest index (e.g., 0) is assigned to the inter-frame merge candidate, and the indexes of the already stored inter-frame merge candidates are each incremented by 1. In this case, when the maximum number of inter-frame merge candidates has already been stored in the inter-frame motion information list, the inter-frame merge candidate with the largest index is removed.

[0262] Alternatively, when adding the inter-frame merge candidate derived from the current block to the inter-frame motion information list, the largest index can be assigned to the inter-frame merge candidate. For example, when the number of inter-frame merge candidates already stored in the inter-frame motion information list is less than the maximum value, an index having the same value as the number of already stored inter-frame merge candidates can be assigned to the inter-frame merge candidate. Alternatively, when the number of inter-frame merge candidates already stored in the inter-frame motion information list is equal to the maximum value, an index obtained by subtracting 1 from the maximum value can be assigned to the inter-frame merge candidate. Also, the inter-frame merge candidate with the smallest index is removed, and the indexes of the remaining multiple already stored inter-frame merge candidates are each decremented by 1.

[0263] ​ It is a diagram showing an embodiment of updating the inter-frame merge candidate list.

[0264] Assume that the inter - prediction merge candidate derived from the current block is added to the inter - prediction merge candidate list, and the maximum index is assigned to the said inter - prediction merge candidate. Also, assume that the maximum number of inter - prediction merge candidates has been stored in the inter - prediction merge candidate list.

[0265] When adding the inter - prediction merge candidate HmvpCand[n + 1] derived from the current block to the inter - prediction merge candidate list HmvpCandList, the inter - prediction merge candidate HmvpCand[0] with the smallest index is deleted from the stored inter - prediction merge candidates, and the indices of the remaining multiple inter - prediction merge candidates are each decreased by 1. Additionally, the index of the inter - prediction merge candidate HmvpCand[n + 1] derived from the current block can be set to the maximum value (n in the ​ shown example).

[0266] When the same inter - prediction merge candidate as the one derived from the current block is already stored (S2005), the inter - prediction merge candidate derived from the current block may not be added to the inter - prediction motion information list (S2009).

[0267] Alternatively, as the inter - prediction merge candidate derived from the current block is added to the inter - prediction motion information list, the stored inter - prediction merge candidate that is the same as the said inter - prediction merge candidate can also be removed. In this case, the effect that the indices of the stored inter - prediction merge candidates are updated will occur.

[0268] ​ It is a diagram showing an example of updating the indices of the stored inter - prediction merge candidates.

[0269] When the index of the stored inter - prediction merge candidate that is the same as the inter - prediction merge candidate mvCand derived from the current block is hIdx, deleting the said stored inter - prediction merge candidate, the indices of the inter - prediction merge candidates with indices greater than hIdx can each be decreased by 1. For example, in the ​ shown example, it shows that HmvpCand[2] which is the same as mvCand is deleted from the inter - prediction motion information list HvmpCandList, and the indices of HmvpCand[3] to HmvpCand[n] are each decreased by 1.

[0270] And the inter - prediction merge candidate mvCand derived from the current block can be added to the end of the inter - prediction motion information list.

[0271] Alternatively, the index assigned to the stored inter - prediction merge candidate that is the same as the inter - prediction merge candidate derived from the current block can be updated. For example, the index of the stored inter - prediction merge candidate can be changed to the minimum value or the maximum value.

[0272] The motion information of blocks included in a predetermined region can be set such that it cannot be added to the inter-frame motion information list. For example, an inter-frame merge candidate derived based on the motion information of blocks included in a merge processing region cannot be added to the inter-frame motion information list. Since the encoding / decoding order of blocks included in the merge processing region is not defined, it is inappropriate to use the motion information of any of these blocks for inter-frame prediction of other blocks. Thus, an inter-frame merge candidate derived based on blocks included in the merge processing region may not be added to the inter-frame motion information list.

[0273] When performing motion compensation prediction by means of a sub-block unit, an inter-frame merge candidate can be derived based on the motion information of a representative sub-block among multiple sub-blocks included in the current block. For example, when using a sub-block merge candidate for the current block, an inter-frame merge candidate can be derived based on the motion information of the representative sub-block in the sub-blocks.

[0274] The motion vector of a sub-block can be derived in the following order. First, any one of the merge candidates included in the merge candidate list of the current block can be selected, and an initial shift vector (shVector) can be derived based on the motion vector of the selected merge candidate. And, by adding the position (xSb, ySb) of each sub-block's reference sample (e.g., the upper-left sample or the middle-position sample) within the coded block to the initial shift vector, a shifted sub-block with the reference sample's position of (xColSb, yColSb) can be derived. The following Equation 4 shows the equation for deriving the shifted sub-block.

[0275]

Equation 4

[0276] (xColSb, yColSb) = (xSb + shVector[0] >> 4, ySb + shVector[1] >> 4)

[0277] Next, the motion vector of the co-located block corresponding to the center position of the sub-block including (xColSb, yColSb) is set as the motion vector of the sub-block including (xSb, ySb).

[0278] The representative sub-block may mean a sub-block including the upper-left sample or the center sample of the current block.

[0279] ​ is a diagram showing the position of the representative sub-block.

[0280] ​ (a) shows an example of setting the sub-block located in the upper-left of the current block as the representative sub-block, ​(b) shows an example in which a sub-block located at the center of the current block is set as a representative sub-block. When performing motion compensation prediction in units of sub-blocks, an inter-frame merge candidate for the current block can be derived based on the motion vector of a sub-block including the upper left sample of the current block or a sub-block including the center sample of the current block.

[0281] Based on the inter-frame prediction mode of the current block, it can also be determined whether the current block is used as an inter-frame merge candidate. For example, a block encoded / decoded based on an affine motion model can be set as not available for use as an inter-frame merge candidate. Thus, even if the current block is encoded / decoded by inter-frame prediction and the inter-frame prediction mode of the current block is an affine prediction mode, the inter-frame prediction motion information list is not updated based on the current block.

[0282] Alternatively, an inter-frame merge candidate can also be derived based on at least one sub-block vector among the sub-blocks included in a block encoded / decoded based on an affine motion model. For example, a sub-block located in the upper left of the current block, a sub-block located at the center, or a sub-block located in the upper right can be used to derive an inter-frame merge candidate. Alternatively, the average value of the sub-block vectors of multiple sub-blocks can also be set as the motion vector of the inter-frame merge candidate.

[0283] Alternatively, an inter-frame merge candidate can also be derived based on the average value of the affine seed vectors of a block encoded / decoded based on an affine motion model. For example, the average value of at least one of the first affine seed vector, the second affine seed vector, or the third affine seed vector of the current block is set as the motion vector of the inter-frame merge candidate.

[0284] Alternatively, the inter-frame motion information list can be configured for different inter-frame prediction modes. For example, at least one of an inter-frame motion information list for a block encoded / decoded by intra-block copy, an inter-frame motion information list for a block encoded / decoded based on a translational motion model, or an inter-frame motion information list for a block encoded / decoded based on an affine motion model can be defined. According to the inter-frame prediction mode of the current block, any one of multiple inter-frame motion information lists can be selected.

[0285] ​ A diagram showing an example of generating an inter-frame motion information list for each inter-frame prediction mode.

[0286] When a block is encoded / decoded based on a non-affine motion model, the inter-frame merge candidate mvCand derived based on the block can be added to the inter-frame non-affine motion information list HmvpCandList. On the other hand, when a block is encoded / decoded based on an affine motion model, the inter-frame merge candidate mvAfCand derived based on the block can be added to the inter-frame affine motion information list HmvpAfCandList.

[0287] The affine seed vectors of blocks can be stored in inter - prediction merge candidates derived from blocks encoded / decoded based on an affine motion model. Thus, the inter - prediction merge candidates can be used as merge candidates for deriving the affine seed vectors of the current block.

[0288] In addition to the described list of inter - frame motion information, another list of inter - frame motion information can be defined. In addition to the described list of inter - frame motion information (hereinafter referred to as the first list of inter - frame motion information), a long - term motion information list (hereinafter referred to as the second list of inter - frame motion information) can be defined. Among them, the long - term motion information list includes long - term merge candidates.

[0289] When both the first list of inter - frame motion information and the second list of inter - frame motion information are in an empty state, first, the inter - prediction merge candidates can be added to the second list of inter - frame motion information. After the number of available inter - prediction merge candidates in the second list of inter - frame motion information reaches the maximum number, the inter - prediction merge candidates can be added to the first list of inter - frame motion information.

[0290] Alternatively, an inter - prediction merge candidate can also be added to both the second list of inter - frame motion information and the first list of inter - frame motion information.

[0291] In this case, the second list of inter - frame motion information with the completed configuration may no longer be updated. Alternatively, when the decoded area is above a predetermined ratio of a stripe, the second list of inter - frame motion information can be updated. Alternatively, the second list of inter - frame motion information can be updated every N coding tree unit rows.

[0292] On the other hand, whenever a block encoded / decoded by inter - frame prediction is generated, the first list of inter - frame motion information can be updated. However, the inter - prediction merge candidates added to the second list of inter - frame motion information can also be set not to be used for updating the first list of inter - frame motion information.

[0293] Information for selecting either the first list of inter - frame motion information or the second list of inter - frame motion information can be signaled through the bitstream. When the number of merge candidates included in the merge candidate list is less than a threshold, the merge candidates included in the inter - frame motion information list indicated by the information can be added to the merge candidate list.

[0294] Alternatively, the inter - frame motion information list can also be selected based on the size, shape, inter - frame prediction mode, whether it is bi - directional prediction, whether motion vector refinement is performed, or whether triangular partitioning is performed of the current block.

[0295] Alternatively, even when the inter - prediction merge candidates included in the first list of inter - frame motion information are added, if the number of merge candidates included in the merge candidate list is still less than the maximum merge number, the inter - prediction merge candidates included in the second list of inter - frame motion information can be added to the merge candidate list.

[0296] ​ FIG. is an example showing adding an inter-frame merge candidate included in the long-term motion information list to the merge candidate list.

[0297] When the number of merge candidates included in the merge candidate list is less than the maximum number, an inter-frame merge candidate included in the first inter-frame motion information list HmvpCandList can be added to the merge candidate list. Even when adding the inter-frame merge candidate included in the first inter-frame motion information list to the merge candidate list, if the number of merge candidates included in the merge candidate list is still less than the maximum number, an inter-frame merge candidate included in the long-term motion information list HmvpLTCandList can be added to the merge candidate list.

[0298] Table 2 shows the process of adding an inter-frame merge candidate included in the long-term motion information list to the merge candidate list.

[0299] [Table 2]

[0300]

[0301]

[0302] The inter-frame merge candidate can be set to include additional information in addition to the motion information. For example, the size, shape, or block division information of the storage block can be added to the inter-frame merge candidate. When constructing the merge candidate list of the current block, only the inter-frame merge candidates with the same or similar size, shape, or division information as the current block are used in the inter-frame merge candidate, or the inter-frame merge candidates with the same or similar size, shape, or division information as the current block can be preferentially added to the merge candidate list.

[0303] Alternatively, inter-frame motion information lists can be generated for different block sizes, shapes, or division information. The merge candidate list of the current block can be generated using the inter-frame motion information list corresponding to the shape, size, or division information of the current block in multiple inter-frame motion information lists.

[0304] When the number of merge candidates included in the merge candidate list of the current block is less than the threshold, an inter-frame merge candidate included in the inter-frame motion information list can be added to the merge candidate list. The addition process is performed in ascending or descending order of the index. For example, the inter-frame merge candidate with the largest index can be added to the merge candidate list.

[0305] When adding an inter-frame merge candidate included in the inter-frame motion information list to the merge candidate list, a redundancy check can be performed between the inter-frame merge candidate and multiple merge candidates already stored in the merge candidate list.

[0306] For example, Table 3 shows the process of adding an inter-frame merge candidate to the merge candidate list.

[0307] [Table 3]

[0308]

[0309] It is also possible to perform a redundancy check only on some of the inter-frame merge candidates included in the inter-frame motion information list. For example, a redundancy check can be performed only on the inter-frame merge candidates whose indices are above or below a threshold. Alternatively, a redundancy check can be performed only on the N merge candidates with the largest indices or the N merge candidates with the smallest indices.

[0310] Alternatively, a redundancy check can be performed only on some of the merge candidates already stored in the merge candidate list. For example, a redundancy check can be performed only on the merge candidates whose indices are above or below a threshold or the merge candidates derived from blocks at a specific position. Among them, the specific position can include at least one of the left adjacent block, the upper adjacent block, the upper right adjacent block, or the lower left adjacent block of the current block.

[0311] ​ is a diagram showing an example of performing a redundancy check only on some merge candidates.

[0312] When adding the inter-frame merge candidate HmvpCand[j] to the merge candidate list, a redundancy check can be performed between the inter-frame merge candidate and the two merge candidates with the largest indices mergeCandList[NumMerge - 2] and mergeCandList[NumMerge - 1]. Among them, NumMerge can represent the number of available spatial merge candidates and temporal merge candidates.

[0313] Different from the example shown in the figure, when adding the inter-frame merge candidate HmvpCand[j] to the merge candidate list, a redundancy check can also be performed between the inter-frame merge candidate and the two merge candidates with the smallest indices. For example, it can be confirmed whether mergeCandList[0] and mergeCandList[1] are the same as HmvpCand[j]. Alternatively, a redundancy check can be performed only on the merge candidates derived from a specific position. For example, a redundancy check can be performed only on at least one of the merge candidates derived from the adjacent block on the left side of the current block or the adjacent block above the current block. When there are no merge candidates derived from a specific position in the merge candidate list, the inter-frame merge candidate can be added to the merge candidate list without performing a redundancy check.

[0314] In the case where a merge candidate identical to the merge candidate between the first frames is found, when performing a redundancy check on the merge candidate between the second frames, the redundancy check on the merge candidate identical to the merge candidate between the first frames can be omitted.

[0315] ​ It is a diagram showing an example of omitting the redundancy check on a specific merge candidate.

[0316] When adding the inter-frame merge candidate HmvpCand[i] with index i to the merge candidate list, a redundancy check can be performed between the inter-frame merge candidate and the merge candidates already stored in the merge candidate list. In this case, in the case where a merge candidate mergeCandList[j] identical to the inter-frame merge candidate HmvpCand[i] is found, the inter-frame merge candidate HmvpCand[i] will not be added to the merge candidate list, and a redundancy check can be performed between the inter-frame merge candidate HmvpCand[i - 1] with index i - 1 and the merge candidates. In this case, the redundancy check between the inter-frame merge candidate HmvpCand[i - 1] and the merge candidate mergeCandList[j] can be omitted.

[0317] For example, in ​ In the example shown, it is determined that HmvpCand[i] is identical to mergeCandList[2]. Thus, HmvpCand[i] is not added to the merge candidate list, and a redundancy check can be performed on HmvpCand[i - 1]. In this case, the redundancy check between HvmpCand[i - 1] and mergeCandList[2] can be omitted.

[0318] When the number of merge candidates included in the merge candidate list of the current block is less than the threshold, in addition to the inter-frame merge candidates, it may also include at least one of pairwise merge candidates or zero merge candidates. A pairwise merge candidate refers to a merge candidate that uses the average of the motion vectors of two or more merge candidates as the motion vector, and a zero merge candidate refers to a merge candidate with a motion vector of 0.

[0319] The merge candidate list of the current block can add merge candidates in the following order.

[0320] Spatial merge candidate - Temporal merge candidate - Inter-frame merge candidate - (Inter-frame affine merge candidate) - Pairwise merge candidate - Zero merge candidate.

[0321] A spatial merge candidate refers to a merge candidate derived from at least one of adjacent blocks or non-adjacent blocks, and a temporal merge candidate refers to a merge candidate derived from a previous reference image. The inter-frame affine merge candidate list represents inter-frame merge candidates derived from blocks encoded / decoded using an affine motion model.

[0322] The inter-frame motion information list can also be used in the advanced motion vector prediction mode. For example, when the number of motion vector prediction candidates included in the motion vector prediction candidate list of the current block is less than a threshold, the inter-frame merge candidates included in the inter-frame motion information list are set as the motion vector prediction candidates related to the current block. Specifically, the motion vector of the inter-frame merge candidate is set as the motion vector prediction candidate.

[0323] If any one of the motion vector prediction candidates included in the motion vector prediction candidate list of the current block is selected, the selected candidate is set as the motion vector prediction value of the current block. After decoding the motion vector residual value of the current block, the motion vector of the current block can be obtained by adding the motion vector prediction value and the motion vector residual value.

[0324] The motion vector prediction candidate list of the current block can be constructed in the following order.

[0325] Spatial motion vector prediction candidate - Temporal motion vector prediction candidate - Inter-frame decoded region merge candidate - (Inter-frame decoded region affine merge candidate) - Zero motion vector prediction candidate.

[0326] The spatial motion vector prediction candidate refers to the motion vector prediction candidate derived from at least one of adjacent blocks or non-adjacent blocks, and the temporal motion vector prediction candidate refers to the motion vector prediction candidate derived from the previous reference image. The inter-frame affine merge candidate list represents the inter-frame motion vector prediction candidates derived from the blocks encoded / decoded with an affine motion model. The zero motion vector prediction candidate represents the candidate whose motion vector value is 0.

[0327] When selecting the merge candidate of the current block, the motion vector of the selected merge candidate is set as the initial motion vector, and the motion compensation prediction of the current block can be performed using the motion vector derived by adding or subtracting the offset vector to / from the initial motion vector. Deriving a new motion vector by adding or subtracting the offset vector to / from the motion vector of the merge candidate can be defined as the merge motion difference coding method.

[0328] The information indicating whether to use the merged offset coding method can be signaled through the bitstream. The information can be a 1-bit flag, merge_offset_vector_flag. For example, a value of 1 for merge_offset_vector_flag indicates that the merged motion difference coding method is applied to the current block. When the merged motion difference coding method is applied to the current block, the motion vector of the current block can be derived by adding or subtracting an offset vector to or from the motion vector of the merge candidate. A value of 0 for merge_offset_vector_flag indicates that the merged motion difference coding method is not applied to the current block. When the merged offset coding method is not applied, the motion vector of the merge candidate can be set as the motion vector of the current block.

[0329] The flag can be signaled only when the skip flag indicating whether to apply the skip mode has a true value or the merge flag indicating whether to apply the merge mode has a true value. For example, when the value of skip_flag indicating whether to apply the skip mode to the current block is 1, or when the value of merge_flag indicating whether to apply the merge mode to the current block is 1, merge_offset_vector_flag can be encoded and signaled.

[0330] When it is determined that the merged offset coding method is applied to the current block, at least one of the information specifying any one of the merge candidates included in the merge candidate list, the information indicating the size of the offset vector, and the information indicating the direction of the offset vector can be additionally signaled.

[0331] The information for determining the maximum number of merge candidates that the merge candidate list can include can be signaled through the bitstream. For example, the maximum number of merge candidates that the merge candidate list can include can be set to an integer of 6 or less.

[0332] When it is determined that the merged offset coding method is applied to the current block, only a preset maximum number of merge candidates can be set as the initial motion vector of the current block. That is, depending on whether the merged offset coding method is applied, the number of merge candidates available for the current block can be adaptively determined. For example, when the value of merge_offset_vector_flag is set to 0, the maximum number of merge candidates available for the current block can be set to M, and when the value of merge_offset_vector_flag is set to 1, the maximum number of merge candidates available for the current block can be set to N. Here, M represents the maximum number of merge candidates that can be included in the merge candidate list, and N represents an integer equal to or less than M.

[0333] For example, when M is 6 and N is 2, two merge candidates with the smallest indices among the merge candidates included in the merge candidate list can be set as available for the current block. Thus, the motion vector of the merge candidate with index value 0 or the motion vector of the merge candidate with index value 1 can be set as the initial motion vector of the current block. When M and N are the same (e.g., when M and N are 2), all the merge candidates included in the merge candidate list can be set as available for the current block.

[0334] Alternatively, it can be determined whether adjacent blocks can be used as merge candidates based on whether the merge motion difference coding method is applied to the current block. For example, when the value of merge_offset_vector_flag is 1, at least one of the adjacent blocks adjacent to the upper right corner of the current block, the adjacent block adjacent to the upper left corner, and the adjacent block adjacent to the lower left corner can be set as not available as a merge candidate. Thus, when the merge motion difference coding method is applied to the current block, the motion vectors of at least one of the adjacent blocks adjacent to the upper right corner of the current block, the adjacent block adjacent to the upper left corner, and the adjacent block adjacent to the lower left corner cannot be set as the initial motion vector. Alternatively, when the value of merge_offset_vector_flag is 1, the temporal adjacent blocks of the current block can be set as not available as merge candidates.

[0335] When the merge motion difference coding method is applied to the current block, at least one of the paired merge candidate and the zero merge candidate can be set as not used. Thus, when the value of merge_offset_vector_flag is 1, although the number of merge candidates included in the merge candidate list is less than the maximum number, at least one of the paired merge candidate or the zero merge candidate may not be added to the merge candidate list.

[0336] The motion vector of the merge candidate can be set as the initial motion vector of the current block. In this case, when the number of merge candidates available for the current block is multiple, information specifying any one of the multiple merge candidates can be signaled by the bitstream. For example, when the maximum number of merge candidates that the merge candidate list can include is greater than 1, information merge_idx indicating any one of the multiple merge candidates can be signaled by the bitstream. That is, under the merge offset coding method, a merge candidate can be specified by the information merge_idx for specifying any one of the multiple merge candidates. The initial motion vector of the current block can be set as the motion vector of the merge candidate indicated by merge_idx.

[0337] On the other hand, when the number of merge candidates available for the current block is 1, signaling the information for specifying a merge candidate may be omitted. For example, when the maximum number of merge candidates that the merge candidate list can include is not greater than 1, signaling the information merge_idx for specifying a merge candidate may be omitted. That is, in the merge offset coding method, when one merge candidate is included in the merge candidate list, coding of the information merge_idx for specifying a merge candidate may be omitted, and the initial motion vector may be determined based on the merge candidate included in the merge candidate list. The motion vector of the merge candidate may be set as the initial motion vector of the current block.

[0338] As another example, after determining the merge candidates of the current block, it may be determined whether to apply the merge motion difference coding method to the current block. For example, when the maximum number of merge candidates that the merge candidate list can include is greater than 1, the information merge_idx for specifying any one of the merge candidates may be signaled. After selecting a merge candidate based on merge_idx, the merge_offset_vector_flag indicating whether to apply the merge motion difference coding method to the current block may be decoded. Table 4 is a diagram showing the syntax table according to the above embodiment.

[0339]

Table 4

[0340]

[0341]

[0342] As another example, after determining the merge candidates of the current block, whether to apply the merge motion difference coding method to the current block may be determined only when the index of the determined merge candidate is less than the maximum number of merge candidates that can be used when applying the merge motion difference coding method. For example, only when the value of the index information merge_idx is less than N, the merge_offset_vector_flag indicating whether to apply the merge motion difference coding method to the current block may be coded and signaled. When the value of the index information merge_idx is equal to or greater than N, coding of the merge_offset_vector_flag may be omitted. If coding of the merge_offset_vector_flag is omitted, it may be determined that the merge motion difference coding method is not applied to the current block.

[0343] Alternatively, after determining the merge candidates for the current block, it can be considered whether the determined merge candidates have bidirectional motion information or unidirectional motion information to determine whether to apply the merged motion difference coding method to the current block. For example, only when the value of the index information merge_idx is less than N and the merge candidate selected by the index information has bidirectional motion information, the merge_offset_vector_flag indicating whether to apply the merged motion difference coding method to the current block is coded and signaled. Optionally, only when the value of the index information merge_idx is less than N and the merge candidate selected by the index information has unidirectional motion information, the merge_offset_vector_flag indicating whether to apply the merged motion difference coding method to the current block is coded and signaled.

[0344] Alternatively, it can be determined whether to apply the merged motion difference coding method based on at least one of the size of the current block, the shape of the current block, and whether the current block touches the boundary of the coding tree unit. When at least one of the size of the current block, the shape of the current block, and whether the current block touches the boundary of the coding tree unit does not meet the preset conditions, the coding of the merge_offset_vector_flag indicating whether to apply the merged motion difference coding method to the current block can be omitted.

[0345] When a merge candidate is selected, the motion vector of the merge candidate can be set as the initial motion vector of the current block. Then, the information indicating the size of the offset vector and the information indicating the direction of the offset vector can be decoded to determine the offset vector. The offset vector can have a horizontal direction component or a vertical direction component.

[0346] The information indicating the size of the offset vector can be index information indicating any one of the motion offset size candidates. For example, the index information distance_idx indicating any one of the motion offset size candidates can be signaled through the bitstream. Table 5 shows the binarization of the index information distance_idx and the value of the variable DistFromMergeMV used to determine the size of the offset vector according to distance_idx.

[0347]

Table 5

[0348]

[0349] The size of the offset vector can be derived by dividing the variable DistFromMergeMV by a preset value. Equation 5 shows an example of determining the size of the offset vector.

[0350]

Equation 5

[0351] abs(offsetMV) = DistFromMergeMV << 2

[0352] According to Equation 5, the value obtained by dividing the variable DistFromMegeMV by 4 or the value obtained by shifting the variable DistFromMergeMV to the left by 2 can be set as the magnitude of the offset vector.

[0353] More motion offset magnitude candidates or fewer motion offset magnitude candidates than the examples shown in Table 5 can be used, or the numerical value of the number of motion vector offset magnitude candidates can be set to be different from the examples shown in Table 5. For example, the magnitude of the horizontal direction component or the vertical direction component of the offset vector can be set to not more than 2 sample distances. Table 6 shows the binarization of the index information distance_idx and the values of the variable DistFromMergeMV for determining the magnitude of the offset vector according to distance_idx.

[0354]

Table 6

[0355]

[0356] Alternatively, the numerical value of the number of motion vector offset magnitude candidates can be set differently based on the motion vector precision. For example, when the motion vector precision of the current block is fractional-pel, the value of the variable DistFromMergeMV corresponding to the value of the index information distance_idx can be set to 1, 2, 4, 8, 16, etc. Here, fractional pixels include at least one of 1 / 16 pixel, one-eighth pixel, one-quarter pixel, or half pixel. On the other hand, when the motion vector precision of the current block is integer pixel, the value of the variable DistFromMergeMV corresponding to the value of the index information distance_idx can be set to 4, 8, 16, 32, 64, etc. That is, the table used to determine the variable DistFromMergeMV can be set differently according to the motion vector precision of the current block.

[0357] For example, when the motion vector precision of the current block or the merge candidate is one-quarter pixel, the variable DistFromMergeMV represented by distance_idx can be derived using Table 5. On the other hand, when the motion vector precision of the current block or the merge candidate is integer pixel, the value obtained by multiplying the value of the variable DistFromMergeMV indicated by distance_idx in Table 5 by N times (e.g., 4 times) can be used as the value of the variable DistFromMergeMV.

[0358] Information for determining the motion vector precision can be signaled through the bitstream. For example, the information can be signaled at the sequence level, picture level, slice level, or block level. Therefore, the number value of the motion offset size candidates can be set differently according to the information related to the motion vector precision signaled through the bitstream. Alternatively, the motion vector precision can be determined based on the merge candidates of the current block. For example, the motion vector precision of the current block can be set to be the same as that of the merge candidates.

[0359] Alternatively, information for determining the search range of the offset vector can be signaled through the bitstream. At least one of the number of motion offset size candidates, the minimum value among the motion offset size candidates, and the maximum value among the motion offset size candidates can be determined based on the search range. For example, a flag merge_offset_vector_flag for determining the search range of the offset vector can be signaled through the bitstream. The information can be signaled through the sequence header, picture header, or slice header.

[0360] For example, when the value of merge_offset_extend_range_flag is 0, the size of the offset vector can be set to not exceed 2. Therefore, the maximum value of DistFromMergeMV can be set to 8. On the other hand, when the value of merge_offset_extend_range_flag is 1, the size of the offset vector can be set to not exceed 32 sample distances. Therefore, the maximum value of DistFromMergeMV can be set to 128.

[0361] A flag indicating whether the size of the offset vector is greater than a threshold can be used to determine the size of the offset vector. For example, a flag distance_flag indicating whether the size of the offset vector is greater than a threshold can be signaled through the bitstream. The threshold can be 1, 2, 4, 8, or 16. For example, distance_flag being 1 indicates that the size of the offset vector is greater than 4. On the other hand, distance_flag being 0 indicates that the size of the offset vector is 4 or less.

[0362] When the size of the offset vector is greater than the threshold, the index information distance_idx can be used to derive the difference between the size of the offset vector and the threshold. Alternatively, when the size of the offset vector is less than or equal to the threshold, the size of the offset vector can be determined by using the index information distance_idx. Table 7 is a syntax table showing the process of encoding distance_flag and distance_idx.

[0363]

Table 7

[0364]

[0365]

[0366] Equation 6 shows an example of using distance_flag and distance_idx to derive the variable DistFromMergeMV for determining the magnitude of the offset vector.

[0367]

Equation 6

[0368] DistFromMergeMV = N * distance_flag + (1 << distance_idx)

[0369] In Equation 6, the value of distance_flag can be set to 1 or 0. The value of distance_idx can be set to 1, 2, 4, 8, 16, 32, 64, 128, etc. N represents a coefficient determined by a threshold. For example, when the threshold is 4, N can be set to 16.

[0370] The information indicating the direction of the offset vector can be index information indicating any one of the vector direction candidates. For example, the index information direction_idx indicating any one of the vector direction candidates can be signaled through the bitstream. Table 8 shows the binarization of the index information direction_idx and the direction of the offset vector according to direction_idx.

[0371]

Table 8

[0372] ​ ​ ​ ​ 0 00 +1 0 1 01 -1 0 2 10 0 +1 3 11 0 -1

[0373] In Table 8, sign[0] indicates the horizontal direction, and sign[1] indicates the vertical direction. +1 indicates that the value of the x - component or y - component of the offset vector is positive (+), and -1 indicates that the value of the x - component or y - component of the offset vector is negative (-). Equation 7 shows an example of determining the offset vector based on the magnitude and direction of the offset vector.

[0374]

Equation 7

[0375] offsetMV[0] = abs(offsetMV) * sign[0]

[0376] offsetMV[1] = abs(offsetMV) * sign[1]

[0377] In Equation 7, offsetMV[0] indicates the vertical - direction component of the offset vector, and offsetMV[1] indicates the horizontal - direction component of the offset vector.

[0378] ​ It is a diagram showing an offset vector indicating values of distance_idx indicating the magnitude of an offset vector and direction_idx indicating the direction of the offset vector.

[0379] As in the ​ example shown, the magnitude and direction of the offset vector can be determined based on the values of distance_idx and direction_idx. The maximum magnitude of the offset vector can be set not to exceed a threshold. Here, the threshold can have a value predefined by the encoder and the decoder. For example, the threshold can be 32 sample distances. Alternatively, the threshold can be determined based on the magnitude of the initial motion vector. For example, the threshold in the horizontal direction can be set based on the magnitude of the horizontal component of the initial motion vector, and the threshold in the vertical direction can be set based on the magnitude of the vertical component of the initial motion vector.

[0380] When the merge candidate has bidirectional motion information, the L0 motion vector of the merge candidate can be set as the L0 initial motion vector of the current block, and the L1 motion vector of the merge candidate can be set as the L1 initial motion vector of the current block. In this case, the L0 offset vector and the L1 offset vector can be determined considering the output order difference (hereinafter referred to as the L0 difference) between the L0 reference image of the merge candidate and the current image and the output order difference (hereinafter referred to as the L1 difference) between the L1 reference image of the merge candidate and the current image.

[0381] First, when the signs of the L0 difference and the L1 difference are the same, the L0 offset vector and the L1 offset vector can be set to be the same. On the other hand, when the signs of the L0 difference and the L1 difference are different, the L1 offset vector can be set in the direction opposite to the L0 offset vector.

[0382] The magnitude of the L0 offset vector and the magnitude of the L1 offset vector can be set to be the same. Alternatively, the magnitude of the L1 offset vector can be determined by scaling the L0 offset vector based on the L0 difference and the L1 difference.

[0383] For example, Equation 8 shows the L0 offset vector and the L1 offset vector when the signs of the L0 difference and the L1 difference are the same.

[0384]

Equation 8

[0385] offsetMVL0[0] = abs(offsetMV) * sign[0]

[0386] offsetMVL0[1] = abs(offsetMV) * sign[1]

[0387] offsetMVL1[0] = abs(offsetMV) * sign[0]

[0388] offsetMVL1[1] = abs(offsetMV) * sign[1]

[0389] In Equation 8, offsetMVL0[0] indicates the horizontal direction component of the L0 offset vector, and offsetMVL0[1] indicates the vertical direction component of the L0 offset vector. offsetMVL1[0] indicates the horizontal direction component of the L1 offset vector, and offsetMVL1[1] indicates the vertical direction component of the L1 offset vector.

[0390] Equation 9 shows the L0 offset vector and the L1 offset vector when the signs of the L0 difference and the L1 difference are different.

[0391]

Equation 9

[0392] offsetMVL0[0] = abs(offsetMV) * sign[0]

[0393] offsetMVL0[1] = abs(offsetMV) * sign[1]

[0394] offsetMVL1[0] = -1 * abs(offsetMV) * sign[0]

[0395] offsetMVL1[1] = -1 * abs(offsetMV) * sign[1]

[0396] More than four vector direction candidates can also be defined. Tables 9 and 10 show examples of defining eight vector direction candidates.

[0397]

Table 9

[0398] ​ ​ ​ ​ 0 000 +1 0 1 001 -1 0 2 010 0 +1 3 011 0 -1 4 100 +1 +1 5 101 +1 -1 6 110 -1 +1 7 111 -1 -1

[0399]

Table 10

[0400] ​ ​ ​ ​ 0 000 +1 0 1 001 -1 0 2 010 0 +1 3 011 0 -1 4 100 +1 / 2 +1 / 2 5 101 +1 / 2 -1 / 2 6 110 -1 / 2 +1 / 2 7 111 -1 / 2 -1 / 2

[0401] In Tables 9 and 10, the absolute values of sign[0] and sign[1] greater than 0 indicate that the offset vector is in the diagonal direction. When using Table 9, the magnitudes of the x-axis component and the y-axis component of the diagonal offset vector are set to abs(offsetMV), while when using Table 10, the magnitudes of the x-axis component and the y-axis component of the diagonal offset vector are set to abs(offsetMV / 2).

[0402] ​It is a diagram showing an offset vector indicating values of distance_idx representing the magnitude of an offset vector and direction_idx representing the direction of the offset vector.

[0403] ​ (a) is an example when applying Table 9, and ​ (b) is an example when applying Table 10.

[0404] Information for determining at least one of the number or magnitude of vector direction candidates can be signaled through a bitstream. For example, a flag merge_offset_direction_range_flag for determining vector direction candidates can be signaled through a bitstream. The flag can be signaled at the sequence level, picture level, or slice level. For example, when the value of the flag is 0, four vector direction candidates shown in Table 8 can be used. On the other hand, when the value of the flag is 1, eight vector direction candidates shown in Table 9 or Table 10 can be used.

[0405] Alternatively, at least one of the number or magnitude of vector direction candidates can be determined based on the magnitude of the offset vector. For example, when the value of the variable DistFromMergeMV for determining the magnitude of the offset vector is equal to or less than a threshold, eight vector direction candidates shown in Table 9 or Table 10 can be used. On the other hand, when the value of the variable DistFromMergeMV is greater than the threshold, four vector direction candidates shown in Table 8 can be used.

[0406] Alternatively, at least one of the number or magnitude of vector direction candidates can be determined based on the value of the x-component MVx and the value of the y-component MVy of the initial motion vector. For example, when the difference or the absolute value of the difference between MVx and MVy is less than or equal to a threshold, eight vector direction candidates shown in Table 9 or Table 10 can be used. On the other hand, when the difference or the absolute value of the difference between MVx and MVy is greater than the threshold, four vector direction candidates shown in Table 8 can be used.

[0407] The motion vector of the current block can be derived by adding the offset vector to the initial motion vector. Equation 10 shows an example of determining the motion vector of the current block.

[0408]

Equation 10

[0409] mvL0[0] = mergeMVL0[0] + offsetMVL0[0]

[0410] mvL0[1] = mergeMVL0[1] + offsetMVL0[1]

[0411] mvL1[0] = mergeMVL1[0] + offsetMVL1[0]

[0412] mvL1[1] = mergeMVL1[1] + offsetMVL1[1]

[0413] In Equation 10, mvL0 indicates the L0 motion vector of the current block, and mvL1 indicates the L1 motion vector of the current block. mergeMVL0 indicates the initial L0 motion vector of the current block (i.e., the L0 motion vector of the merge candidate), and mergeMVL1 represents the initial L1 motion vector of the current block. [0] indicates the horizontal direction component of the motion vector, and [1] indicates the vertical direction component of the motion vector.

[0414] Even when performing inter prediction on each sub-unit after dividing the coding block into multiple sub-units, the merge motion difference coding method can be applied. Here, performing inter prediction on a sub-unit basis may include at least one of advanced temporal motion vector prediction (ATMVP), spatio-temporal motion vector prediction (STMVP), and triangular partitioning techniques.

[0415] For example, under the ATMVP method, the initial motion vector can be derived as follows.

[0416] First, the initial shift vector can be derived using the motion vector of the merge candidate derived from an adjacent block adjacent to the coding block. Additionally, the shifted block of the sub-block included in the coding block can be derived using the initial shift vector. Equation 11 shows the position of the shifted block.

[0417]

Equation 11

[0418] (xColSb, yColSb) = (xSb + shVector[0] >> 4, ySb + shVector[1] >> 4)

[0419] In Equation 11, (xColSb, yColSb) represents the position of the upper left sample of the shifted block, and (xSb, ySb) represents the position of the upper left sample of the sub-block. shVector represents the shift vector.

[0420] When the shifted block is determined, the motion vector of the co-located block at the same position as the shifted block in the co-located picture can be set as the motion vector of the sub-block. That is, the motion vector of the co-located block of the sample at the (xColSb, yColSb) position included in the co-located block can be set as the motion vector of the sub-block of the sample at the (xSb, ySb) position.

[0421] When applying triangular partitioning techniques, coding blocks can be partitioned into triangular sub-units. For example, a coding block can be divided into two sub-units by a diagonal line connecting the upper left corner and the lower right corner of the coding block or by a diagonal line connecting the upper right corner and the lower left corner of the coding block.

[0422] ​ is a diagram showing the partitioning pattern of a coding block when applying triangular partitioning techniques.

[0423] The motion information of each triangular sub-unit can be specified by merge candidates. To this end, index information indicating any one of the merge candidates can be signaled for each sub-unit. For example, the index information merge_1st_idx of the first sub-unit can specify the merge candidate of the first sub-unit, and the index information merge_2nd_idx of the second sub-unit can specify the merge candidate of the second sub-unit.

[0424] The initial motion vector of each sub-unit can be determined individually. For example, when applying an affine motion model to a coding block, the affine vector of a sub-block derived from the affine seed vector of the coding block can be set as the initial motion vector of the sub-block. The motion vector of each sub-block can be derived by adding or subtracting an offset vector to / from the initial motion vector.

[0425] When applying the merged motion difference coding method to a coding block divided into multiple sub-units, multiple sub-units can be set to use the same offset vector. That is, the same offset vector can be used to change the initial motion vector of each of the multiple sub-units.

[0426] Alternatively, a coding block can be partitioned into multiple sub-units, and the offset vector of each sub-unit can be determined individually. Therefore, the offset vector of at least one sub-unit can be set to be different from the offset vector of another sub-unit.

[0427] ​ is a diagram showing an example of setting the offset vector of each sub-unit differently.

[0428] As in the example shown in ​ information distance_idx indicating the magnitude of the offset vector and direction_idx indicating the direction of the offset vector can be coded and signaled for each sub-unit.

[0429] Alternatively, the magnitudes of the offset vectors of all sub-units can be set to be the same, and the directions of the offset vectors can be set individually for these sub-units. For example, it can be set to share the value of distance_idx signaled at the coding level among the sub-units, and direction_idx can be coded and signaled for each sub-unit.

[0430] Alternatively, the directions of the offset vectors of all sub-units can be set to be the same, and the magnitudes of the offset vectors can be set individually for these sub-units. For example, the value of direction_idx signaled at the coding level can be set to be shared among the sub-units, and distance_idx can be coded and signaled for each sub-unit.

[0431] The combined motion difference coding method can be applied only to some of the multiple sub-units generated by dividing a coding block. For example, when a current block is divided into a first sub-unit and a second sub-unit, the motion vector of the first sub-unit can be set to be the same as the motion vector of a merge candidate, and the motion vector of the second sub-unit can be derived by adding an offset vector to the motion vector of the merge candidate.

[0432] Instead of signaling information for determining an offset vector, the decoder can derive the offset vector. Specifically, the offset vector can be derived using the average of the horizontal direction gradients and the average of the vertical direction gradients of the predicted samples included in a sub-block.

[0433] Here, the gradient can be derived based on the difference between a reconstructed sample corresponding to a predicted sample in a reference image and an adjacent sample adjacent to the reconstructed sample. For example, the horizontal direction gradient can indicate the difference between the reconstructed sample and the reconstructed samples adjacent to the left and / or right, and the vertical direction gradient can indicate the difference between the reconstructed sample and the reconstructed samples adjacent to the top and / or bottom.

[0434] Among the merge candidates included in a merge candidate list, a merge candidate having a motion vector derived by adding or subtracting an offset vector from the motion vector of a reference merge candidate included in the merge candidates included in the merge candidate list can be added to the merge candidate list. A merge candidate having a motion vector derived by adding or subtracting an offset vector from the motion vector of a reference merge candidate can be referred to as a refined merge candidate.

[0435] The remaining motion information other than the motion vector of the refined merge candidate can be set to be the same as the motion information of the reference merge candidate.

[0436] ​ is a diagram showing motion vector candidates that a refined merge candidate can adopt.

[0437] When the motion vector of the reference merge candidate is (MvLX[0], MvLX[1]), the motion vector of the refined merge candidate can be derived by adding or subtracting an offset to at least one of the x - component or y - component of the motion vector of the reference merge candidate. For example, the motion vector of the refined merge candidate can be set to (MvLX[0]+M, MvLX[1]), (MvLX[0]-M, MvLX[1]), (MvLX[0], MvLX[1]+M), or (MvLX[0], MvLX[1]-M). M represents the magnitude of the offset vector.

[0438] The reference merge candidate can be a merge candidate having a predefined index value in the merge candidate list. For example, the merge candidate having the smallest index value (i.e., the merge candidate having the index value 0) or the merge candidate having the largest index value among the merge candidates included in the merge candidate list can be set as the reference merge candidate. Alternatively, the inter - frame merge candidate having the smallest index value or the inter - frame merge candidate having the largest index value in the inter - frame motion information list can be set as the reference merge candidate.

[0439] Alternatively, the merge candidate having the smallest index value among the merge candidates having bi - directional motion information can be set as the reference merge candidate. That is, when searching for candidate blocks sequentially, the first - found bi - directional merge candidate can be set as the reference merge candidate.

[0440] The basic merge candidate can be selected based on the size of the current block, the shape of the current block, or whether the current block touches the boundary of the coding tree unit. For example, when the current block is square - shaped or the current block is non - square - shaped with a height greater than the width, the merge candidate with index 0 or the merge candidate derived from the adjacent block above the current block can be set as the reference merge candidate. When the current block is non - square - shaped with a width greater than the height, the merge candidate with index 1 or the merge candidate derived from the adjacent block to the left of the current block can be set as the reference merge candidate.

[0441] Alternatively, the information specifying the reference merge candidate can be signaled in the bitstream. The information can be the index information specifying any one of the merge candidates included in the merge candidate list.

[0442] The information indicating whether to use the refined merge candidate can be signaled in the bitstream. The information can be a 1 - bit flag. When the value of the flag is 1, the refined merge candidate generated based on the reference merge candidate can be added to the merge candidate list. On the other hand, when the value of the flag is 0, the merge candidate list may not include the refined merge candidate.

[0443] Alternatively, when the number of merge candidates added to the merge candidate list is less than the maximum number of merge candidates that the merge candidate list can include, a refined merge candidate may be added to the merge candidate list. Here, the added merge candidates may include at least one of a spatial merge candidate, a temporal merge candidate, an inter-frame merge candidate, or a pairwise merge candidate. For example, when the number of at least one of the spatial merge candidate, the temporal merge candidate, and the inter-frame merge candidate included in the merge candidate list is less than or equal to a threshold, a refined merge candidate may be added to the merge candidate list.

[0444] Alternatively, when the number of merge candidates added to the merge candidate list is greater than or equal to a threshold, a refined merge candidate may be used.

[0445] The maximum number of merge candidates that can be included in the merge candidate list may be set differently according to whether a refined merge candidate is used. For example, when it is set not to use a refined merge candidate, the maximum number of merge candidates that the merge candidate list can include may be set to N, and when it is set to use a refined merge candidate, the maximum number of merge candidates that the merge candidate list can include may be set to N + n.

[0446] The index of the refined merge candidate may be greater than the index of the merge candidates added to the merge candidate list. For example, Table 11 shows an example of configuring the merge candidate list.

[0447]

Table 11

[0448] ​ ​ ​ ​ ​ ​ ​ ​ ​ ​

[0449] In Table 11, mergeCand[X] represents the merge candidate with index X. MvLX[0] represents the x-component motion vector of the reference merge candidate, and MvLX[1] represents the y-component motion vector of the reference merge candidate. For example, when the reference merge candidate is mergeCand[0], MvLX[0] and MvLX[1] may represent the motion vectors of mergeCand[0].

[0450] The magnitude M of the offset vector may be predefined in the encoder and the decoder. For example, the magnitude M of the offset vector may be set to an integer less than or equal to 4, such as 1 or 4.

[0451] Alternatively, information for determining the offset vector may be signaled via a bitstream. The information may be signaled at the sequence level, picture level, slice level, or block level. For example, at least one of the information distance_idx for determining the magnitude of the offset vector or the information direction_idx for determining the direction of the offset vector may be used as described above to determine the offset vector.

[0452] In the example shown in Table 11, at least one refined merge candidate derived based on a reference merge candidate can be added to the merge candidate list. If there is a merge candidate among the added merge candidates that has the same motion information as the motion information of the refined merge candidate, the refined merge candidate may not be added to the merge candidate list. For example, when the refined merge candidate derived based on the reference merge candidate mergeCand[0] is the same as any one of mergeCand[1] to mergeCand[5], the refined merge candidate may not be added to the merge candidate list.

[0453] Alternatively, when there is a merge candidate that has the same motion information as the motion information of the refined merge candidate, the refined merge candidate can be re-derived by changing the offset vector, or the merge candidate that has the same motion information as the motion information of the refined merge candidate can be reset as the refined merge candidate. For example, if the motion information of the refined merge candidate mergeCand[6] derived based on the reference merge candidate mergeCand[0] is the same as the motion information of the merge candidate mergeCand[2], the motion vector of the refined merge candidate mergeCand[6] can be changed to a value obtained by adding or subtracting the offset vector to or from the motion vector of the merge candidate [2]. For example, the motion vector of mergeCand[6] can be changed from (mergeCand[0]_mxLx[0]+M, mergeCand[0]_mvLx[1]) to (mergeCand[2]_mxLx[0]+M, mergeCand[2]_mvLx[1]). Here, mergeCand[X]_mvLx represents the motion vector of the merge candidate with index X.

[0454] As another example, a merge refinement offset list including at least one merge offset candidate can be used to determine the offset vector. When the merge candidate specified by the index information of the current block is a reference merge candidate, the merge refinement offset list can be used to determine the offset vector. Additionally, the motion vector of the current block can be derived by adding or subtracting the offset vector to or from the motion vector of the merge candidate. The reference merge candidate can be a merge candidate having a predefined index value in the merge candidate list. For example, the merge candidate having the smallest index value (i.e., the merge candidate having the index value 0) or the merge candidate having the largest index value among the merge candidates included in the merge candidate list can be set as the reference merge candidate. Alternatively, the inter-frame merge candidate having the smallest index value or the inter-frame merge candidate having the largest index value in the inter-frame motion information list can be set as the reference merge candidate.

[0455] ​ is a diagram showing the configuration of the merge refinement offset list.

[0456] In ​ , it is assumed that the reference merge candidate is the merge candidate with index 6.

[0457] If the index of the merge candidate specified by merge_idx indicating any one of the merge candidates is not 6, the motion vector of the merge candidate can be set to the motion vector of the current block.

[0458] On the other hand, when the index of the merge candidate specified by the index information merge_idx is 6, a merge refinement offset list can be used to derive an offset vector. Index information MrgOffset_idx specifying any one of the merge offset candidates included in the merge refinement offset list can be signaled through the bitstream.

[0459] When the offset vector is specified, the motion vector of the current block can be derived by adding or subtracting the offset vector to / from the motion vector of the reference merge candidate.

[0460] The merge refinement offset list can include at least one merge offset candidate. For example, the number of merge offset candidates included in the merge refinement offset list can be 4, 8, or 16.

[0461] ​ And ​ are diagrams showing the offset vectors specified by the merge offset candidates.

[0462] ​ illustrates an example where the number of merge offset candidates is 8, and ​ illustrates an example where the number of merge offset candidates is 16.

[0463] As in the example shown in ​ (a), the offset vector represented by the merge offset candidate can be set such that the absolute value of the motion vector in the horizontal direction and / or the absolute value of the motion vector in the vertical direction can have a fixed value. Alternatively, as in the example shown in ​ , the offset vector represented by the merge offset candidates with indices less than the threshold can be set such that the absolute value of the motion vector in the horizontal direction and / or the absolute value of the motion vector in the vertical direction can have a first value, and the offset vector indicated by the other merge offset candidates can be set such that the absolute value of the motion vector in the horizontal direction and / or the absolute value of the motion vector in the vertical direction can have a second value.

[0464] Alternatively, as shown in ​ (b), the offset vector represented by the merge offset candidates can be set such that the sum of the absolute value of the motion vector in the horizontal direction and the absolute value of the motion vector in the vertical direction can have a fixed value.

[0465] Multiple reference merge candidates can be set. For example, among the merge candidates included in the merge candidate list, the two merge candidates with the smallest indices can be set as the reference merge candidates. Therefore, when the index of the merge candidate specified by the index information merge_idx is 0 or 1, the merge refinement offset list can be used to derive the offset vector. Alternatively, the merge candidate with the smallest index among the merge candidates included in the merge candidate list and the merge candidate with the largest index among the merge candidates included in the inter-frame merge candidate list can be set as the reference merge candidates.

[0466] In the advanced motion vector prediction mode, the motion vector of the current block can be derived by adding the motion difference vector to the motion prediction vector. The motion prediction vector of the current block can be determined based on a motion vector prediction candidate list including at least one motion prediction vector candidate. For example, any one of the motion prediction vector candidates can be set as the motion prediction vector of the current block.

[0467] The motion vector prediction candidates can be derived based on at least one of the spatial neighboring blocks or the temporal neighboring blocks of the current block.

[0468] ​ FIG. is a diagram showing candidate blocks for deriving motion vector prediction candidates.

[0469] The spatial neighboring blocks may include an upper neighboring block located above the current block and a left neighboring block located to the left of the current block. The upper neighboring block may include one of the following blocks, i.e., block B0 including samples at the position (xCb+CbW, yCb-1), block B1 including samples at the position (xCb+CbW-1, yCb-1), block B2 including samples at the position (xCb-1, yCb-1), or block B3 including samples at the position (xCb, yCb-1). Herein, (xCb, yCb) represents the position of the upper left sample of the current block, and CbW represents the width of the current block. The left neighboring block may include one of the following blocks, i.e., block A0 including samples at the position (xCb-1, yCb+CbH), block A1 including samples at the position (xCb-1, yCb+CbH-1), or block A2 including samples at the position (xCb-1, yCb). Herein, CbH represents the height of the current block.

[0470] The temporal neighboring blocks may include at least one of block C0 and block C1, where block C0 includes samples at the center of a block having the same position and size as the current block in the co-located block, and block C1 includes samples adjacent to the lower right corner of the block.

[0471] The maximum number of motion vector prediction candidates that can be included in the motion vector prediction candidate list can be 2. The derivation order of the motion vector prediction candidates is as follows.

[0472] 1. When at least one of the left adjacent blocks A0 or A1 is available, set the motion vector of the available block as a motion vector prediction candidate.

[0473] 2. When at least one of the upper adjacent blocks B0, B1 or B2 is available, set the motion vector of the available block as a motion vector prediction candidate.

[0474] 3. When a temporally adjacent block is available, set the temporal motion vector as a motion vector prediction candidate.

[0475] 4. Set the zero motion vector as a motion vector prediction candidate.

[0476] Alternatively, when the number of motion vector prediction candidates derived in the order of 1 to 3 is less than 2, the motion vectors included in the inter-frame motion information list may be set as motion vector prediction candidates. When the inter-frame motion information list is available, motion vector prediction candidates may be derived in the following order.

[0477] 1. When at least one of the left adjacent blocks A0 or A1 is available, set the motion vector of the available block as a motion vector prediction candidate.

[0478] 2. When at least one of the upper adjacent blocks B0, B1 or B2 is available, set the motion vector of the available block as a motion vector prediction candidate.

[0479] 3. When a temporally adjacent block is available, set the temporal motion vector as a motion vector prediction candidate.

[0480] 4. Set the motion vectors included in the inter-frame motion information list as motion vector prediction candidates.

[0481] 5. Set the zero motion vector as a motion vector prediction candidate.

[0482] Motion vector prediction candidates having motion vectors derived by adding or subtracting an offset vector to or from the motion vectors of reference motion vector prediction candidates may be added to the motion vector prediction candidate list. Motion vector prediction candidates having motion vectors derived by adding or subtracting an offset vector to or from the motion vectors of reference motion vector prediction candidates may be referred to as fine motion vector prediction candidates.

[0483] ​ is a diagram showing motion vector candidates that can be set as fine motion vector prediction candidates.

[0484] When the motion vector of the reference motion vector prediction candidate is (MvpLX[0], MvpLX[1]), the motion vector of the refined motion vector prediction candidate can be derived by adding or subtracting an offset to at least one of the x - component or y - component of the motion vector of the reference motion vector prediction candidate. For example, the motion vector of the refined motion vector prediction candidate can be set to (MvpLX[0]+M, MvpLX[1]), (MvpLX[0] - M, MvpLX[1]), (MvpLX[0], MvpLX[1]+M) or (MvpLX[0], MvpLX[1] - M). M represents the magnitude of the offset vector.

[0485] The magnitude M of the offset vector can be predefined in the encoder and decoder. For example, the magnitude M of the offset vector can be set to an integer less than or equal to 4, such as 1 or 4.

[0486] Alternatively, the information for determining the offset vector can be signaled through the bitstream. The information can be signaled at sequence level, picture level, slice level or block level. For example, at least one of the information distance_idx for determining the magnitude of the offset vector or the information direction_idx for determining the direction of the offset vector can be used as described above to determine the offset vector.

[0487] The reference motion vector prediction candidate can be a motion vector prediction candidate with a predetermined index value in the motion vector prediction candidate list. For example, among the motion vector prediction candidates included in the motion vector prediction candidate list, the motion vector prediction candidate with index value 0 or the motion vector prediction candidate with index value 1 can be set as the reference motion vector prediction candidate.

[0488] As another example, a merge refinement offset list including at least one prediction vector offset candidate can be used to determine the offset vector. When the motion vector prediction candidate specified by the index information of the current block is the reference motion vector prediction candidate, the prediction vector refinement offset list can be used to determine the offset vector. Additionally, the motion prediction vector of the current block can be derived by adding or subtracting the offset vector to the motion vector of the motion vector prediction candidate. The reference motion vector prediction candidate can be a motion vector prediction candidate with a predetermined index value in the motion vector prediction candidate list. For example, among the motion vector prediction candidates included in the motion vector prediction candidate list, the motion vector prediction candidate with the minimum index value or the motion vector prediction candidate with the maximum index value can be set as the reference motion vector prediction candidate.

[0489] When using the prediction vector offset refinement list to calculate the offset vector, the maximum number of prediction vector candidates that the prediction vector candidate list can include can be set to a value greater than 2.

[0490] ​ It is a diagram showing the configuration of a prediction vector refinement offset list.

[0491] In ​ it is assumed that the reference prediction vector candidate is the prediction vector candidate with index 2.

[0492] When the index of the prediction vector candidate specified by the index information AMVPcand_idx indicating any one of the prediction vector candidates is not 2, the motion vector of the prediction vector candidate can be set as the motion prediction vector of the current block.

[0493] On the other hand, when the index of the prediction vector candidate specified by the index information AMVPcand_idx is 2, the prediction vector refinement offset list can be used to derive the offset vector. The index information AMVPOffset_idx specifying any one of the prediction vector offset candidates included in the prediction vector refinement offset list can be signaled through the bitstream.

[0494] When the offset vector is specified, the motion prediction vector of the current block can be derived by adding or subtracting the offset vector from the motion vector of the reference prediction vector candidate.

[0495] Even when encoding a coding block based on an affine motion model, motion vector refinement techniques can be used. For example, when applying the affine advanced motion vector prediction mode, the affine seed vector of the coding block can be derived by adding the affine seed difference vector to the affine seed prediction vector. Here, the affine seed prediction vector can be derived based on the affine seed vectors of the spatially adjacent blocks or temporally adjacent blocks of the coding block. The affine seed difference vector can be determined based on the information signaled from the bitstream. In this case, the same affine seed difference vector can be applied to all control points. Alternatively, information for determining the affine seed vector can be signaled for each control point.

[0496] When deriving the affine vector of a sub-block based on the affine seed vector of the coding block, the affine vector can be set as the initial motion vector, and then the offset vector can be derived. The motion vector of each sub-block can be derived by adding or subtracting the offset vector from the initial motion vector.

[0497] Instead of signaling the information for determining the offset vector, the decoder can derive the offset vector. Specifically, the average value of the horizontal direction gradient and the average value of the vertical direction gradient of the prediction samples included in the sub-block can be used to derive the offset vector.

[0498] Intra prediction uses the reconstructed samples that have been encoded / decoded around the current block to predict the current block. In this case, the intra prediction of the current block can use the reconstructed samples before applying the loop filter.

[0499] Intra prediction techniques include matrix-based intra prediction and general intra prediction that takes into account the directionality with surrounding reconstructed samples. Information indicating the intra prediction technique for the current block can be signaled through the bitstream. The information can be a 1-bit flag. Alternatively, based on at least one of the position, size, shape of the current block, or the intra prediction technique of adjacent blocks, the intra prediction technique for the current block can be determined. For example, when the current block straddles the image boundary, the current block is set not to apply matrix-based intra prediction.

[0500] Matrix-based intra prediction is a method of obtaining a predicted block for the current block based on matrix multiplication between a matrix stored in the encoder and decoder and the reconstructed samples around the current block. Information for specifying any one of the stored multiple matrices can be signaled through the bitstream. The decoder can determine the matrix for intra prediction of the current block based on the information and the size of the current block.

[0501] General intra prediction is a method of obtaining a predicted block related to the current block based on a non-angle intra prediction mode or an angle intra prediction mode.

[0502] The derived residual image can be obtained by subtracting the predicted image from the original image. In this case, when the residual image is transformed into the frequency domain, even if the high-frequency components in the frequency components are removed, the subjective picture quality of the video will not be significantly reduced. Thus, if the value of the high-frequency component is made smaller or the value of the high-frequency component is set to 0, there is an effect of improving the compression efficiency without causing obvious visual distortion. Reflecting the above characteristics, the current block can be transformed to decompose the residual image into two-dimensional frequency components. Transformation techniques such as the Discrete Cosine Transform (DCT) or the Discrete Sine Transform (DST) can be used to perform the transformation.

[0503] After transforming the current block using DCT or DST, the transformed current block can be transformed again. In this case, the transformation based on DCT or DST can be defined as the primary transformation, and the process of transforming the block to which the primary transformation is applied again can be called the secondary transformation.

[0504] The primary transformation can be performed using any one of multiple transformation kernel candidates. For example, the primary transformation can be performed using any one of DCT2, DCT8, or DCT7.

[0505] Different transformation kernels can also be used for the horizontal and vertical directions. Information representing the combination of the transformation kernel for the horizontal direction and the transformation kernel for the vertical direction can also be signaled through the bitstream.

[0506] The execution units for the primary transform and the secondary transform are different. For example, the primary transform may be performed on an 8×8 block, and the secondary transform may be performed on a 4×4 sub-block within the transformed 8×8 block. In this case, the transform coefficients of the remaining region where the secondary transform is not performed may also be set to 0.

[0507] Alternatively, the primary transform may be performed on a 4×4 block, and the secondary transform may be performed on an 8×8 region including the transformed 4×4 block.

[0508] Information indicating whether the secondary transform is performed may be signaled through a bitstream.

[0509] In the decoder, the inverse transform of the secondary transform (the second inverse transform) may be performed, and the inverse transform of the primary transform (the first inverse transform) may be performed on the result. As a result of the execution of the second inverse transform and the first inverse transform, a residual signal of the current block may be obtained.

[0510] Quantization is used to reduce the energy of a block, and the quantization process includes a process of dividing the transform coefficients by a specific constant. The constant may be derived from a quantization parameter, and the quantization parameter may be defined as a value between 1 and 63.

[0511] If the transform and quantization are performed in the encoder, the decoder may obtain a residual block through inverse quantization and inverse transform. By adding the prediction block and the residual block in the decoder, a reconstructed block of the current block may be obtained.

[0512] If a reconstructed block of the current block is obtained, information loss generated during the quantization and encoding processes may be reduced through in-loop filtering. The in-loop filter may include at least one of a deblocking filter, a sample adaptive offset (SAO) filter, or an adaptive loop filter (ALF).

[0513] The embodiments described centering on the decoding process or the encoding process are also included in the scope of the present invention when used for the encoding process or the decoding process. Changing the order of a plurality of embodiments described in a predetermined order to a different order is also included in the scope of the present invention.

[0514] The embodiments have been described based on a series of steps or flowcharts, but this does not limit the chronological order of the invention, and they can be executed simultaneously or in a different order as needed. Also, in the above embodiments, the structural elements (e.g., units, modules, etc.) constituting the block diagrams can be implemented as hardware devices or software respectively, and multiple structural elements can be combined and implemented as a single hardware device or software. The embodiments can be implemented in the form of program instructions, which can be executed by various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium can include program instructions, data files, data structures, etc. alone or in combination. Examples of the computer-readable recording medium can include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs, DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store program instructions and execute the instructions such as ROMs, RAMs, flash memories, etc. The hardware devices can be configured to operate as one or more software modules to execute the processing according to the present invention, and vice versa.

[0515]

Industrial Applicability

[0516] The present invention can be applied to electronic devices for encoding / decoding videos.

Claims

1. A video decoding method, comprising the following steps: Determine whether to apply a merged motion differential encoding method to a current block; Generate a merged candidate list for the current block; Determine a merged candidate for the current block based on the merged candidate list; And Derive a motion vector for the current block based on the merged candidate, wherein, When applying the merged motion differential encoding method to the current block, derive the motion vector for the current block by adding an offset vector to the motion vector derived based on the merged candidate, and when the maximum number of merged candidates included in the merged candidate list is more than 1, select the merged candidate for the current block based on index information decoded from a bitstream indicating one of the merged candidates in the merged candidate list; Determine the magnitude of the offset vector based on first index information, the first index information indicating one of a plurality of motion offset magnitude candidates; wherein, at least one of the maximum value or the minimum value of the motion offset magnitude candidates is set differently according to the value of a flag indicating the range of the motion offset magnitude candidates, or at least one of the maximum value or the minimum value of the motion offset magnitude candidates is set differently according to the motion vector precision of the current block.

2. The video decoding method according to claim 1, wherein, The flag is signaled at the picture level.

3. The video decoding method according to claim 1, wherein, The magnitude of the offset vector is obtained by performing a shift operation on the value represented by the motion offset magnitude candidate specified by the first index information.

4. The video decoding method according to claim 1, wherein, Determine the direction of the offset vector based on second index information, the second index information being used to indicate one of a plurality of vector direction candidates.

5. A video encoding method, comprising the following steps: Determine whether to apply a merged motion differential encoding method to a current block; Generate a merged candidate list for the current block; Determine a merged candidate for the current block based on the merged candidate list; And Derive a motion vector for the current block based on the merged candidate, wherein, When applying the merged motion differential encoding method to the current block, derive the motion vector for the current block by adding an offset vector to the motion vector derived based on the merged candidate, and when the maximum number of merged candidates included in the merged candidate list is more than 1, encode index information indicating the merged candidate for the current block in the merged candidate list; Encode first index information, the first index information being used to specify a motion offset magnitude candidate indicating the magnitude of the offset vector among a plurality of motion offset magnitude candidates; wherein, encode a flag indicating the range of the motion offset magnitude candidates, wherein at least one of the maximum value or the minimum value of the motion offset magnitude candidates is set differently according to the value of the flag, or at least one of the maximum value or the minimum value of the motion offset magnitude candidates is set differently according to the motion vector precision of the current block.

6. The video encoding method according to claim 5, wherein, The flag is encoded at the picture level.

7. The video encoding method according to claim 5, wherein, The motion offset size candidate has a value derived by performing a shift operation on the magnitude of the offset vector.

8. The video encoding method according to claim 5, wherein, Encode second index information for specifying a vector direction candidate that indicates the direction of the offset vector among a plurality of vector direction candidates.

9. A video decoding device includes an inter prediction unit configured to: Determine whether to apply a merged motion difference encoding method to a current block; Generate a merge candidate list for the current block; determine a merge candidate for the current block based on the merge candidate list; And derive a motion vector for the current block based on the merge candidate, wherein, When the merged motion difference encoding method is applied to the current block, the motion vector for the current block is derived by adding an offset vector to the motion vector derived based on the merge candidate. When the maximum number of merge candidates included in the merge candidate list is more than 1, select the merge candidate for the current block based on index information decoded from a bitstream that indicates one of the merge candidates in the merge candidate list; The inter prediction unit is further configured to determine the magnitude of the offset vector based on first index information that indicates one of a plurality of motion offset size candidates; the inter prediction unit is further configured to differently set at least one of a maximum value or a minimum value of the motion offset size candidate according to a value of a flag indicating a range of the motion offset size candidate, or differently set at least one of a maximum value or a minimum value of the motion offset size candidate according to a motion vector precision of the current block.

10. The video decoding device according to claim 9, wherein, The flag is signaled at the picture level.

11. The video decoding device according to claim 9, wherein, The magnitude of the offset vector is obtained by performing a shift operation on a value represented by the motion offset size candidate specified by the first index information.

12. The video decoding device according to claim 9, wherein, The inter prediction unit is configured to determine the direction of the offset vector based on second index information that indicates one of a plurality of vector direction candidates.

13. A video encoding device includes an inter prediction unit configured to: Determine whether to apply a merged motion difference encoding method to a current block; Generate a merge candidate list for the current block; Determine a merge candidate for the current block based on the merge candidate list; And Derive a motion vector for the current block based on the merge candidate, wherein, When the merged motion difference encoding method is applied to the current block, the motion vector for the current block is derived by adding an offset vector to the motion vector derived based on the merge candidate. When the maximum number of merge candidates included in the merge candidate list is more than 1, encode index information indicating the merge candidate for the current block in the merge candidate list; The inter-frame prediction unit is further configured to encode first index information for specifying a motion offset size candidate indicating the size of the offset vector among a plurality of motion offset size candidates, and the inter-frame prediction unit is further configured to encode a flag indicating a range of the motion offset size candidate, wherein at least one of a maximum value or a minimum value of the motion offset size candidate is set differently according to the value of the flag, or the inter-frame prediction unit is further configured to set at least one of the maximum value or the minimum value of the motion offset size candidate differently according to the motion vector precision of the current block.

14. The video encoding device according to claim 13, wherein the flag is encoded at the picture level.

15. The video encoding device according to claim 13, wherein the motion offset size candidate has a value derived by performing a shift operation on the size of the offset vector.

16. The video encoding device according to claim 13, wherein the inter-frame prediction unit is configured to encode second index information for specifying a vector direction candidate indicating the direction of the offset vector among a plurality of vector direction candidates.

17. A video decoder, comprising: a processor; and a memory for storing a computer program; wherein the processor is configured to execute the computer program to perform the video decoding method according to any one of claims 1 to 4.

18. A video encoder, comprising: a processor; and a memory for storing a computer program; wherein the processor is configured to execute the computer program to perform the video encoding method according to any one of claims 5 to 8.

19. A computer-readable storage medium storing an executable program, which when executed by a processor, implements the video decoding method according to any one of claims 1 to 4, or implements the video encoding method according to any one of claims 5 to 8.

Images