Video signal encoding / decoding method and apparatus therefor

Abstract

A video signal encoding / decoding method and apparatus therefor are provided. The video decoding method of the present invention includes the steps of determining whether to apply a merge motion difference value encoding method to a current block, generating a merge candidate list for the current block, determining a merge candidate for the current block based on the merge candidate list, and deriving a motion vector for the current block based on the merge candidate.

Description

TECHNICAL FIELD

[0001] The present application relates to a video signal encoding / decoding method and an apparatus for the same. BACKGROUND

[0002] With the trend of larger display panels, there is a growing need for video services with higher quality. The biggest problem with high-definition video services is the significant increase in data volume, and to address this problem, research is actively being conducted to improve video compression rates. As a representative example, in 2009, the Motion Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) under the International Telecommunication Union-Telecommunication (ITU-T) established the Joint Collaborative Team on Video Coding (JCT-VC). The JCT-VC proposed the video compression standard HEVC (High Efficiency Video Coding) and was approved on January 25, 2013, and its compression performance is about twice that of H.264 / AVC. With the rapid development of high-definition video services, the performance of HEVC has gradually revealed its limitations. SUMMARY

[0003] Technical problems to be solved

[0004] The object of the present application is to provide a method for refining a motion vector derived from a merge candidate based on an offset vector when encoding / decoding a video signal and an apparatus for performing the same.

[0005] The object of the present application is to provide a method for signaling an offset vector when encoding / decoding a video signal and an apparatus for performing the same.

[0006] The technical problems to be solved by the present application are not limited to the above-mentioned technical problems, and other technical problems not mentioned will be clearly understood by those skilled in the art to which the present application pertains from the following description.

[0007] Technical solutions

[0008] The video signal decoding method according to the present application comprises the steps of: determining whether a merge motion difference value coding method is applied to a current block; generating a merge candidate list of the current block; determining a merge candidate of the current block based on the merge candidate list; and deriving a motion vector of the current block based on the merge candidate. When the merge motion difference value coding method is applied to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector derived based on the merge candidate. The size of the offset vector is determined based on first index information, which indicates one of a plurality of motion offset size candidates. At least one of a maximum value or a minimum value of the motion offset size candidates is set differently according to a value of a flag indicating a range of the motion offset size candidates, or at least one of a maximum value or a minimum value of the motion offset size candidates is set differently according to a motion vector precision of the current block.

[0009] In the video signal decoding method according to the present application, the flag can be signaled at a picture level.

[0010] In the video signal decoding method according to the present application, the size of the offset vector can be obtained by a shift operation on a value represented by the motion offset size candidate specified by the first index information.

[0011] In the video signal decoding method according to the present application, the direction of the offset vector can be determined based on second index information, which is used to indicate one of a plurality of vector direction candidates.

[0012] The video signal encoding method according to the present application comprises the steps of: determining whether a merge motion difference value coding method is applied to a current block; generating a merge candidate list of the current block; determining a merge candidate of the current block based on the merge candidate list; and deriving a motion vector of the current block based on the merge candidate, wherein when the merge motion difference value coding method is applied to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector derived based on the merge candidate. First index information is encoded, which is used to specify a motion offset size candidate of a plurality of motion offset size candidates, which indicates the size of the offset vector. A flag indicating a range of the motion offset size candidates is encoded, wherein at least one of a maximum value or a minimum value of the motion offset size candidates is set differently according to a value of the flag, or at least one of a maximum value or a minimum value of the motion offset size candidates is set differently according to a motion vector precision of the current block.

[0013] The video decoding device of the present application comprises an inter prediction unit configured to determine whether a merge motion difference coding method is applied to a current block, generate a merge candidate list of the current block, determine a merge candidate of the current block based on the merge candidate list, and derive a motion vector of the current block based on the merge candidate, wherein when the merge motion difference coding method is applied to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector derived based on the merge candidate. The inter prediction unit is further configured to determine a size of the offset vector based on first index information indicating one of a plurality of motion offset size candidates, and differently set at least one of a maximum value or a minimum value of the motion offset size candidates according to a value of a flag indicating a range of the motion offset size candidates, or differently set at least one of the maximum value or the minimum value of the motion offset size candidates according to a motion vector precision of the current block.

[0014] The video encoding device of the present application comprises an inter prediction unit configured to determine whether a merge motion difference coding method is applied to a current block, generate a merge candidate list of the current block, determine a merge candidate of the current block based on the merge candidate list, and derive a motion vector of the current block based on the merge candidate, wherein when the merge motion difference coding method is applied to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector derived based on the merge candidate. The inter prediction unit is further configured to encode first index information for specifying a motion offset size candidate of a plurality of motion offset size candidates indicating a size of the offset vector, and encode a flag indicating a range of the motion offset size candidates, wherein at least one of a maximum value or a minimum value of the motion offset size candidates is differently set according to a value of the flag, or at least one of the maximum value or the minimum value of the motion offset size candidates is differently set according to a motion vector precision of the current block.

[0015] The video decoder of the present application comprises a processor and a memory storing a computer program. The processor is configured to execute the computer program to perform the video decoding method described above.

[0016] The video encoder of the present application comprises a processor and a memory storing a computer program. The processor is configured to execute the computer program to perform the video encoding method described above.

[0017] The computer readable storage medium of the present application stores an executable program, which is executed by a processor to implement the video decoding method or the video encoding method described above.

[0018] The features briefly described above for the present application are merely exemplary implementations of the detailed description of the present application that will be described later, and do not limit the scope of the present application.

[0019] Effects of Invention

[0020] According to the present application, inter prediction efficiency can be improved by refining the motion vector of the merge candidate based on the offset vector.

[0021] According to the present application, inter prediction efficiency can be improved by adaptively determining the size and direction of the offset vector.

[0022] The effects obtainable in the present application are not limited to those described above and other effects that are not mentioned will become clear to those having ordinary skill in the art to which the present application pertains from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 is a block diagram of a video encoder (encoder) according to an embodiment of the present application.

[0024] Figure 2 is a block diagram of a video decoder (decoder) according to an embodiment of the present application.

[0025] Figure 3 is a diagram illustrating a basic coding tree unit according to an embodiment of the present application.

[0026] Figure 4 is a diagram illustrating various partition types of a coding block.

[0027] Figure 5 is a diagram illustrating a partition example of a coding tree unit.

[0028] Figure 6 is a diagram illustrating shapes of a basic data unit.

[0029] Figure 7 and Figure 8 is a diagram illustrating an example of partitioning a coding block into a plurality of sub-blocks.

[0030] Figure 9 is a flowchart illustrating an inter prediction method according to an embodiment of the present application.

[0031] Figure 10 is a diagram illustrating a non-linear motion of an object.

[0032] Figure 11 is a flowchart illustrating an inter prediction method based on affine motion according to an embodiment of the present application.

[0033] Figure 12is a diagram showing an example of an affine seed vector of each affine motion model.

[0034] Figure 13 is a diagram showing an example of an affine vector of a sub-block under a 4-parameter motion model.

[0035] Figure 14 is a flowchart showing a process of deriving motion information of a current block using a merge mode.

[0036] Figure 15 is a diagram showing a candidate block for deriving a merge candidate.

[0037] Figure 16 is a diagram showing a position of a reference sample.

[0038] Figure 17 is a diagram showing an example of a candidate block for deriving a merge candidate.

[0039] Figure 18 is a diagram showing an example of changing a position of a reference sample.

[0040] Figure 19 is a diagram showing an example of changing a position of a reference sample.

[0041] Figure 20 is a flowchart showing a process for updating an inter motion information list.

[0042] Figure 21 is a diagram showing an embodiment of updating an inter merge candidate list.

[0043] Figure 22 is a diagram showing an example of updating an index of a stored inter merge candidate.

[0044] Figure 23 is a diagram showing a position of a representative sub-block.

[0045] Figure 24 is a diagram showing an example of generating an inter motion information list for each inter prediction mode.

[0046] Figure 25 is a diagram showing an example of adding an inter merge candidate included in a long-term motion information list to a merge candidate list.

[0047] Figure 26 is a diagram showing an example of performing a redundancy check only for some merge candidates.

[0048] Figure 27 is a diagram showing an example of omitting a redundancy check for a specific merge candidate.

[0049] Figure 28is a diagram illustrating an offset vector according to values of distance_idx indicating a size of the offset vector and direction_idx indicating a direction of the offset vector.

[0050] Figure 29 is a diagram illustrating an offset vector according to values of distance_idx indicating a size of the offset vector and direction_idx indicating a direction of the offset vector.

[0051] Figure 30 is a diagram illustrating a partition mode of a coding block when a triangle partitioning technique is applied.

[0052] Figure 31 is a diagram illustrating an example in which offset vectors of each sub-unit are set differently.

[0053] Figure 32 is a diagram illustrating motion vector candidates that a fine merge candidate can take.

[0054] Figure 33 is a diagram illustrating a configuration of a merge refinement offset list.

[0055] Figure 34 and Figure 35 is a diagram illustrating an offset vector specified by a merge offset candidate.

[0056] Figure 36 is a diagram illustrating a candidate block used to derive a motion vector prediction candidate.

[0057] Figure 37 is a diagram illustrating a motion vector candidate that can be set as a fine motion vector prediction candidate.

[0058] Figure 38 is a diagram illustrating a configuration of a prediction vector refinement offset list. DETAILED DESCRIPTION

[0059] Hereinafter, embodiments of the present application will be explained in detail with reference to the accompanying drawings.

[0060] Encoding and decoding of a video are performed in units of blocks. For example, encoding / decoding processes such as transform, quantization, prediction, loop filtering, or reconstruction can be performed on a coding block, a transform block, or a prediction block.

[0061] Hereinafter, a block to be encoded / decoded will be referred to as a "current block". For example, according to a current encoding / decoding process step, the current block can represent a coding block, a transform block, or a prediction block.

[0062] Also, the term "unit" used in the present specification refers to a basic unit for performing a specific encoding / decoding process, and a "block" can be understood to refer to an array of samples of a predetermined size. Unless otherwise specified, "block" and "unit" are used interchangeably. For example, in the embodiments described later, an encoding block and an encoding unit can be understood to have the same meaning.

[0063] Figure 1 is a block diagram of a video encoder (encoder) according to an embodiment of the present application.

[0064] Referring to Figure 1 The video encoding apparatus 100 can include an image partitioning section 110, prediction sections 120, 125, a transform section 130, a quantization section 135, a rearrangement section 160, an entropy encoding section 165, an inverse quantization section 140, an inverse transform section 145, a filter section 150, and a memory 155.

[0065] Figure 1 The components shown in the drawing are shown separately in order to indicate mutually different characteristic functions in the video encoding apparatus, and do not indicate that the components are constituted by separate hardware or a single software component. That is, for each component, in order to facilitate explanation, at least two of the components are combined into one component or one component is divided into a plurality of components in a manner of arranging the components, and the functions are performed thereby, and such embodiments of combining the components and embodiments of separating the components do not depart from the essence of the present application and are within the scope of the present application.

[0066] Also, some of the structural elements are not essential structural elements for performing the essence of the present application in the present application, but are optional structural elements for improving performance. The present application can be implemented by including only the components necessary for implementing the essence of the present application, excluding the structural elements for improving performance, and a structure including only the essential structural elements, excluding the optional structural elements for improving performance, also belongs to the scope of the present application.

[0067] The image partitioning section 110 can partition an input image into at least one processing unit. In this case, the processing unit can be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The image partitioning section 110 partitions one image into a combination of a plurality of coding units, prediction units, and transform units, and can select one coding unit, prediction unit, and transform unit combination to encode the image based on a predetermined criterion (e.g., a cost function).

[0068] For example, one picture can be divided into a plurality of coding units. In order to divide a picture into coding units, a recursive tree structure such as a quad tree structure can be used, and a video or a largest coding unit can be divided into other coding units having sub-nodes corresponding to the number of divided coding units, with a coding unit as a root. A coding unit that is no longer divided according to certain restrictions will become a leaf node. That is, when it is assumed that a coding unit can be divided only in a square shape, a coding unit can be divided into a maximum of 4 other coding units.

[0069] Hereinafter, in the embodiments of the present application, a coding unit can mean a unit that performs encoding, and can also mean a unit that performs decoding.

[0070] A prediction unit within a coding unit can be divided into at least one of the same shapes of squares or rectangles, or one prediction unit within a coding unit can be divided into a shape different from another prediction unit.

[0071] When a prediction unit performing intra prediction based on a coding unit is not a minimum coding unit, intra prediction can be performed without being divided into a plurality of prediction units NxN.

[0072] The prediction units 120, 125 can include an inter prediction unit 120 that performs inter prediction and an intra prediction unit 125 that performs intra prediction. It can be determined whether to use inter prediction or to perform intra prediction for a prediction unit, and specific information (e.g., an intra prediction mode, a motion vector, a reference picture, etc.) is determined based on each prediction method. In this case, the processing unit that performs prediction can be different from the processing unit that determines the prediction method and the specific content. For example, the prediction method and the prediction mode, etc. can be determined by the prediction unit, and the prediction can be performed by the transform unit. The residual value (residual block) between the generated prediction block and the original block can be input to the transform unit 130. Also, the prediction mode information, the motion vector information, etc. used for prediction can be encoded together with the residual value in the entropy encoding unit 165 and transmitted to the decoder. When a specific encoding mode is used, the original block can also be encoded directly and transmitted to the decoder without generating a prediction block by the prediction unit 120, 125.

[0073] The inter prediction unit 120 can predict a prediction unit based on information of at least one of a previous picture or a subsequent picture of a current picture, and in some cases, based on information of a part of an already encoded region within the current picture. The inter prediction unit 120 can include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0074] The reference picture interpolation section receives reference picture information from the memory 155 and can generate pixel information of integer pixels or fractional pixels from the reference picture. For a luma pixel, in order to generate pixel information of fractional pixels in units of 1 / 4 pixels, a DCT-based 8-tap interpolation filter (DCT-based Interpolation Filter) having different filter coefficients can be used. For a chroma signal, in order to generate pixel information of fractional pixels in units of 1 / 8 pixels, a DCT-based 4-tap interpolation filter (DCT-based Interpolation Filter) having different filter coefficients can be used.

[0075] The motion prediction section can perform motion prediction based on a reference picture interpolated by the reference picture interpolation section. Methods for calculating a motion vector can use a full search-based block matching algorithm (FBMA), a three step search method (TSS), a new three-step search algorithm (NTS), or the like. The motion vector can have a motion vector value in units of 1 / 2 pixels or 1 / 4 pixels based on the interpolated pixels. The current prediction unit can be predicted in the motion prediction section by taking different motion prediction methods. The motion prediction method can use a skip method, a merge method, an advanced motion vector prediction (AMVP) method, an intra block copy method, or the like.

[0076] The intra prediction section 125 can generate a prediction unit based on reference pixel information of a current block periphery as pixel information within a current picture. In a case where a neighboring block of the current prediction unit is a block in which inter prediction has been performed and in a case where a reference pixel is a pixel in which inter prediction has been performed, the reference pixel included in the block in which inter prediction has been performed can be used as reference pixel information of the periphery of the block in which intra prediction has been performed. That is, in a case where the reference pixel is not available, at least one of the available reference pixels can be used instead of the unavailable reference pixel information.

[0077] In intra prediction, the prediction mode can have an angular prediction mode using reference pixel information in a prediction direction and a non-angular mode not using direction information when performing prediction. The mode for predicting luma information and the mode for predicting chroma information can be different, and in order to predict chroma information, intra prediction mode information used to predict luma information or predicted luma signal information can be applied.

[0078] When performing intra prediction, if the size of the prediction unit is the same as the size of the transform unit, intra prediction can be performed on the prediction unit based on pixels present to the left of the prediction unit, pixels present above and to the left, and pixels present above. However, when performing intra prediction, if the size of the prediction unit is different from the size of the transform unit, intra prediction can be performed using reference pixels based on the transform unit. Also, intra prediction using N x N partitioning can be applied only to the smallest coding unit.

[0079] The intra prediction method can generate a prediction block after applying an adaptive intra smoothing (AIS) filter to reference pixels according to a prediction mode. The type of adaptive intra smoothing filter for reference pixels can be different. To perform the intra prediction method, the intra prediction mode of the current prediction unit can be predicted from the intra prediction modes of the prediction units present in the periphery of the current prediction unit. In the case of predicting the prediction mode of the current prediction unit using the mode information predicted from the peripheral prediction units, if the intra prediction mode of the current prediction unit is the same as that of the peripheral prediction units, information indicating that the prediction mode of the current prediction unit is the same as that of the peripheral prediction units can be transmitted using predetermined flag information, and if the prediction mode of the current prediction unit is different from that of the peripheral prediction units, the prediction mode information of the current block can be encoded by performing entropy encoding.

[0080] Also, a residual block including residual information that is the difference between the prediction unit predicted based on the prediction unit generated in the prediction section 120, 125 and the original block of the prediction unit can be generated. The generated residual block can be input to the transform section 130.

[0081] In the transform section 130, the residual block including residual information between the original block and the prediction unit generated by the prediction section 120, 125 can be transformed using a transform method such as a discrete cosine transform (DCT) or a discrete sine transform (DST). Among them, the DCT transform kernel includes at least one of DCT2 or DCT8, and the DST transform kernel includes DST7. Whether to apply DCT or DST to transform the residual block can be determined based on the intra prediction mode information of the prediction unit used to generate the residual block. It is also possible to skip the transformation of the residual block. A flag indicating whether the transformation of the residual block is skipped can be encoded. For residual blocks whose size is below a threshold, luminance components, or chrominance components (below 4:4:4 format), transform skipping can be allowed.

[0082] The quantization unit 135 can quantize values transformed into a frequency domain in the transform unit 130. A quantization coefficient can vary according to importance of a block or a video. Values calculated in the quantization unit 135 can be provided to the inverse quantization unit 140 and the rearrangement unit 160.

[0083] The rearrangement unit 160 can perform rearrangement of coefficient values on the quantized residual values.

[0084] The rearrangement unit 160 can change 2-dimensional block-shaped coefficients into a 1-dimensional vector form through a coefficient scanning (Coefficient Scanning) method. For example, the rearrangement unit 160 can scan a DC coefficient or a coefficient of a high frequency domain using a Zig-Zag Scan method and change it into a 1-dimensional vector form. According to a size of a transform unit and an intra prediction mode, instead of the Zig-Zag Scan, a vertical scan of scanning 2-dimensional block-shaped coefficients in a column direction and a horizontal scan of scanning 2-dimensional block-shaped coefficients in a row direction can be used. That is, which one of the Zig-Zag Scan, the vertical scan, and the horizontal scan is used can be determined according to the size of the transform unit and the intra prediction mode.

[0085] The entropy encoding unit 165 can perform entropy encoding based on values calculated through the rearrangement unit 160. For example, the entropy encoding can use various encoding methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), Context-Adaptive Binary Arithmetic Coding (CABAC), etc.

[0086] The entropy encoding unit 165 can encode various information such as residual value coefficient information and block type information of an encoding unit originating from the rearrangement unit 160 and the prediction units 120, 125, prediction mode information, division unit information, prediction unit information, and transmission unit information, motion vector information, reference frame information, interpolation information of a block, filtering information, etc.

[0087] The entropy encoding unit 165 can entropy-encode coefficient values of an encoding unit input from the rearrangement unit 160.

[0088] The inverse quantization unit 140 and the inverse transform unit 145 inverse-quantize a plurality of values quantized in the quantization unit 135 and inverse-transform values transformed in the transform unit 130. Residual values generated in the inverse quantization unit 140 and the inverse transform unit 145 can generate a reconstructed block by merging a prediction unit predicted by a motion prediction unit, a motion compensation unit, and an intra prediction unit included in the prediction units 120, 125.

[0089] The filter unit 150 can include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0090] The deblocking filter can remove blocking distortion generated in a reconstructed image due to a boundary between blocks. In order to determine whether to perform deblocking, it is possible to determine whether to apply a deblocking filter to a current block based on pixels included in several columns or rows included in the block. In the case of applying a deblocking filter to a block, a strong filter or a weak filter can be applied according to a required deblocking filter strength. Also, in a process of using a deblocking filter, when vertical filtering and horizontal filtering are performed, horizontal direction filtering and vertical direction filtering can be processed in synchronization.

[0091] The offset correction unit can correct an offset between an original video and a video in which deblocking is performed, on a pixel unit basis. In order to perform offset correction on a designated image, a method of determining a region in which offset is to be performed after dividing pixels included in a video into a predetermined number of regions and applying offset to the corresponding region, or a method of applying offset while considering edge information of each pixel can be used.

[0092] The adaptive loop filtering (ALF) can be performed based on a value obtained by comparing a filtered reconstructed image and an original video. After dividing pixels included in a video into predetermined groups, filtering can be performed differently for each group by determining one filter to be used for the corresponding group. Information related to whether to apply adaptive loop filtering and a luma signal can be transmitted on a coding unit (CU) basis, and a shape of an adaptive loop filter to be applied and a filter coefficient can be different for each block. Also, it is possible to apply an adaptive loop filter of the same type (fixed type) regardless of characteristics of a block to which it is applied.

[0093] The memory 155 can store a reconstructed block or image calculated through the filter unit 150, and can provide the stored reconstructed block or image to the prediction units 120 and 125 when inter prediction is performed.

[0094] Figure 2 is a block diagram of a video decoder (decoder) according to an embodiment of the disclosure.

[0095] Referring to Figure 2The video decoder 200 can include an entropy decoding unit 210, a rearranging unit 215, an inverse quantization unit 220, an inverse transform unit 225, a prediction unit 230, a prediction unit 235, a filter unit 240, and a memory 245.

[0096] When inputting a video bitstream from the video encoder, the inputted bitstream can be decoded in a step opposite to that of the video encoder.

[0097] The entropy decoding unit 210 can perform entropy decoding in a step opposite to that of the entropy encoding unit of the video encoder. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), Context-Adaptive Binary Arithmetic Coding (CABAC), etc. can be applied in correspondence with the method performed in the video encoder.

[0098] The entropy decoding unit 210 can decode information related to intra prediction and inter prediction performed by the encoder.

[0099] The rearranging unit 215 can perform rearranging based on a method of rearranging a bitstream entropy-decoded by the entropy decoding unit 210 in the encoding unit. The rearranging can be performed by reconstructing a plurality of coefficients represented in a 1-dimensional vector form into a 2-dimensional block-shaped coefficient. The rearranging unit 215 receives information related to coefficient scanning performed in the encoding unit, and can perform rearranging by a method of performing inverse scanning based on a scanning order performed in the corresponding encoding unit.

[0100] The inverse quantization unit 220 can perform inverse quantization based on a quantization parameter provided by the encoder and a coefficient value of a rearranged block.

[0101] The inverse transform unit 225 can perform inverse discrete cosine transform, inverse discrete sine transform, which belong to inverse transform of transform performed in the transform unit, i.e., inverse transform of discrete cosine transform, discrete sine transform, on a quantization result performed by the video encoder. Herein, the DCT transform kernel can include at least one of DCT2 or DCT8, and the DST transform kernel can include DST7. Alternatively, if transform is skipped in the video encoder, the inverse transform unit 225 can not perform inverse transform. The inverse transform can be performed based on a transport unit determined in the video encoder. In the inverse transform unit 225 of the video decoder, a transform method (e.g., DCT or DST) can be selectively performed according to a prediction method, a size of a current block, a prediction direction, etc.

[0102] The prediction units 230, 235 can generate a prediction block based on information related to the generation of the prediction block provided by the entropy decoding unit 210 and previously decoded block or image information provided by the memory 245.

[0103] As described above, when performing intra prediction in the same manner as in the operation of the video encoder, if the size of the prediction unit is the same as the size of the transform unit, the intra prediction of the prediction unit is performed based on the pixels present at the left side of the prediction unit, the pixels present at the upper left, and the pixels present at the upper side, and if the size of the prediction unit is different from the size of the transform unit when performing the intra prediction, the intra prediction can be performed using the reference pixels based on the transform unit. Also, it is possible to apply the intra prediction using only N x N partitioning for the minimum coding unit.

[0104] The prediction units 230, 235 can include a prediction unit decision unit, an inter prediction unit, and an intra prediction unit. The prediction unit decision unit receives various information such as prediction unit information input from the entropy decoding unit 210, prediction mode information of an intra prediction method, motion prediction related information of an inter prediction method, and the like, and classifies the prediction unit according to the current coding unit, and can determine whether the prediction unit is performing inter prediction or intra prediction. The inter prediction unit 230 can use information required for inter prediction of the current prediction unit provided by the video encoder, and perform inter prediction of the current prediction unit based on information included in at least one of a previous image or a subsequent image of a current image to which the current prediction unit belongs. Alternatively, it is also possible to perform inter prediction based on information of a part of an area that has been reconstructed within the current image to which the current prediction unit belongs.

[0105] In order to perform inter prediction, it is possible to determine which of the skip mode, the merge mode, the advanced motion vector prediction mode (AMVP mode), and the intra block copy mode is the motion prediction method of the prediction unit included in the corresponding coding unit based on the coding unit.

[0106] The intra prediction unit 235 can generate a prediction block based on pixel information within the current picture. When the prediction unit is a prediction unit for which intra prediction has been performed, the intra prediction can be performed based on intra prediction mode information of the prediction unit provided by the video encoder. The intra prediction unit 235 can include an adaptive intra smoothing (AIS) filter, a reference pixel interpolation unit, a DC filter. The adaptive intra smoothing filter is a portion that performs filtering on reference pixels of the current block, and can determine whether to apply the filter according to the prediction mode of the current prediction unit. The adaptive intra smoothing filter can be applied to the reference pixels of the current block using the prediction mode of the prediction unit and the adaptive intra smoothing filter information provided by the video encoder. If the prediction mode of the current block is a mode for which the adaptive intra smoothing filter is not applied, the adaptive intra smoothing filter can not be applied.

[0107] For the reference pixel interpolation unit, if the prediction mode of the prediction unit is a prediction unit for which intra prediction is performed based on pixel values obtained by interpolating reference pixels, the reference pixels in integer or fractional pixel units can be generated by interpolating the reference pixels. If the prediction mode of the current prediction unit is a prediction mode in which the prediction block is generated without interpolating the reference pixels, the reference pixels can not be interpolated. If the prediction mode of the current block is a DC mode, the DC filter can generate the prediction block by filtering.

[0108] The reconstructed block or picture can be provided to the filter unit 240. The filter unit 240 can include a deblocking filter, an offset correction unit, an ALF.

[0109] Information related to whether to apply the deblocking filter to the corresponding block or picture and information related to whether to apply strong filtering or weak filtering when the deblocking filter is applied can be received from the video encoder. The information related to the deblocking filter provided by the video encoder is received from the deblocking filter of the video decoder, and the deblocking filter can be applied to the corresponding block by the video decoder.

[0110] The offset correction unit can perform offset correction on the reconstructed video based on the type of offset correction for the video at the time of encoding and offset amount information, etc.

[0111] The ALF can be applied to the coding unit based on information related to whether to apply the ALF, ALF coefficient information, etc. provided by the encoder. Such ALF information can be provided by being included in a specific parameter set.

[0112] The memory 245 stores the reconstructed picture or block so that the picture or block can be used as a reference picture or a reference block, and can provide the reconstructed picture to the output unit.

[0113] Figure 3 FIG. 1 is a diagram illustrating a basic coding tree unit according to an embodiment of the present application.

[0114] A coding block having the largest size can be defined as a coding tree block. One picture can be divided into a plurality of coding tree units (CTUs). A coding tree unit is a coding unit having the largest size, and can also be referred to as a largest coding unit (LCU). Figure 3 An example of dividing one picture into a plurality of coding tree units is illustrated.

[0115] The size of a coding tree unit can be defined at a picture level or a sequence level. To this end, information indicating the size of a coding tree unit can be signaled through a picture parameter set or a sequence parameter set.

[0116] For example, the size of a coding tree unit for the entire picture within a sequence can be set to 128x128. Alternatively, either of 128x128 or 256x256 at a picture level can be determined as the size of a coding tree unit. For example, the size of a coding tree unit in a first picture can be set to 128x128, and the size of a coding tree unit in a second picture can be set to 256x256.

[0117] A coding block can be generated by dividing a coding tree unit. A coding block indicates a basic unit for encoding / decoding processing. For example, prediction or transformation can be performed per different coding block, or a prediction encoding mode can be determined per different coding block. The prediction encoding mode indicates a method of generating a prediction picture. For example, the prediction encoding mode can include intra prediction, inter prediction, current picture referencing (CPR), or combined prediction. For a coding block, at least one of intra prediction, inter prediction, current picture referencing, or combined prediction can be used to generate a prediction block related to the coding block.

[0118] Information indicating a prediction encoding mode of a current block can be signaled through a bitstream. For example, the information can be a 1-bit flag indicating whether the prediction encoding mode is an intra mode or an inter mode. Current picture referencing or combined prediction can be used only in the case where the prediction encoding mode of the current block is determined to be an inter mode.

[0119] The current picture reference is used to set the current picture as a reference picture and obtain a prediction block of the current block from a coded / decoded area within the current picture. Here, the current picture means a picture including the current block. Information indicating whether the current picture reference is applied to the current block can be signaled through a bitstream. For example, the information can be a 1-bit flag. When the flag is true, a prediction coding mode of the current block can be determined as the current picture reference, and when the flag is false, the prediction mode of the current block can be determined as inter prediction.

[0120] Alternatively, the prediction coding mode of the current block can be determined based on a reference picture index. For example, when the reference picture index points to the current picture, the prediction coding mode of the current block can be determined as the current picture reference. When the reference picture index points to another picture other than the current picture, the prediction coding mode of the current block can be determined as inter prediction. That is, the current picture reference is a prediction method using information of a coded / decoded area within the current picture, and the inter prediction is a prediction method using information of a coded / decoded other picture.

[0121] The combined prediction means a coding mode combined by two or more of the intra prediction, the inter prediction, and the current picture reference. For example, in the case where the combined prediction is applied, a first prediction block can be generated based on one of the intra prediction, the inter prediction, or the current picture reference, and a second prediction block can be generated based on another. If the first prediction block and the second prediction block are generated, a final prediction block can be generated through an average operation or a weighted sum operation of the first prediction block and the second prediction block. Information indicating whether the combined prediction is applied can be signaled through a bitstream. The information can be a 1-bit flag.

[0122] Figure 4 is a diagram showing various partition types of a coding block.

[0123] The coding block can be partitioned into a plurality of coding blocks based on a quad tree partition, a binary tree partition, or a ternary tree partition. The partitioned coding block can also be partitioned into a plurality of coding blocks again based on the quad tree partition, the binary tree partition, or the ternary tree partition.

[0124] The quad tree partition refers to a partitioning technique of partitioning a current block into 4 blocks. As a result of the quad tree partition, the current block can be partitioned into 4 square sub-areas (refer to Figure 4 in the (a) part) 'SPLIT_QT').

[0125] A binary tree split refers to a partitioning technique that splits a current block into 2 blocks. A process of splitting a current block into two blocks along a vertical direction (i.e., using a vertical line that crosses the current block) can be referred to as a vertical binary tree split, and a process of splitting a current block into two blocks along a horizontal direction (i.e., using a horizontal line that crosses the current block) can be referred to as a horizontal binary tree split. A current block can be split into 2 non-square sub-blocks after a binary tree split. Figure 4 (b) in the "SPLIT_BT_VER" indicates a vertical binary tree split result, and Figure 4 (c) in the "SPLIT_BT_HOR" indicates a horizontal binary tree split result.

[0126] A ternary tree split refers to a partitioning technique that splits a current block into 3 blocks. A process of splitting a current block into three blocks along a vertical direction (i.e., using two vertical lines that cross the current block) can be referred to as a vertical ternary tree split, and a process of splitting a current block into three blocks along a horizontal direction (i.e., using two horizontal lines that cross the current block) can be referred to as a horizontal ternary tree split. A current block can be split into 3 non-square sub-blocks after a ternary tree split. In this case, a width / height of a sub-block located at the center of the current block can be 2 times a width / height of other sub-blocks. Figure 4 (d) in the "SPLIT_TT_VER" indicates a vertical ternary tree split result, and Figure 4 (e) in the "SPLIT_TT_HOR" indicates a horizontal ternary tree split result.

[0127] A number of splits of a coding tree unit can be defined as a partitioning depth. A maximum partitioning depth of a coding tree unit can be determined at a sequence or picture level. Thus, the maximum partitioning depth of a coding tree unit can become different according to different sequences or pictures.

[0128] Alternatively, a maximum partitioning depth can be separately determined for each of a plurality of partitioning techniques. For example, a maximum partitioning depth that allows a quad tree split can be different from a maximum partitioning depth that allows a binary tree split and / or a ternary tree split.

[0129] An encoder can signal information indicating at least one of a split shape or a partitioning depth of a current block through a bitstream. A decoder can determine a split shape and a partitioning depth of a coding tree unit based on information parsed from a bitstream.

[0130] Figure 5 is a diagram illustrating examples of partitioning of a coding tree unit.

[0131] The process of dividing coding blocks using partitioning techniques such as quadtree partitioning, binary tree partitioning, and / or ternary tree partitioning is called multitree partitioning.

[0132] The coded blocks generated by applying a multi-way tree partitioning to the coded block can be called multiple downstream coded blocks. When the partitioning depth of the coded block is k, the partitioning depth of the multiple downstream coded blocks is set to k+1.

[0133] On the other hand, for multiple coding blocks with a partitioning depth of k+1, the coding block with a partitioning depth of k can be called the upstream coding block.

[0134] The partition type of the current coding block can be determined based on at least one of the partition shape of the upstream coding block or the partition type of the adjacent coding blocks. The adjacent coding blocks are adjacent to the current coding block and can include at least one of the current coding block's upper adjacent block, left adjacent block, or adjacent block to its upper left corner. The partition type can include at least one of whether to partition into a quadtree, whether to partition into a binary tree, the binary tree partition direction, whether to partition into a ternary tree, or the ternary tree partition direction.

[0135] To determine the shape of the coded block partition, information indicating whether the coded block has been partitioned can be sent via a signal in the bitstream. This information is a 1-bit flag "split_cu_flag," and when the flag is true, it indicates that the coded block has been partitioned using a multi-way tree partitioning technique.

[0136] When "split_cu_flag" is true, information indicating whether the coded block has been partitioned by a quadtree can be sent via a signal in the bitstream. This information is a 1-bit flag "split_qt_flag". When this flag is true, the coded block can be divided into 4 blocks.

[0137] For example, in Figure 5 The example shown illustrates how the coding tree unit is partitioned by a quadtree to generate four coding blocks with a partition depth of 1. Furthermore, the example illustrates applying quadtree partitioning again to the first and fourth coding blocks generated as a result of the quadtree partitioning. Ultimately, four coding blocks with a partition depth of 2 can be generated.

[0138] Furthermore, a coded block with a partition depth of 3 can be generated by applying a quadtree partition to the coded block with a partition depth of 2 again.

[0139] When a quadtree partitioning is not applied to the coded block, it can be determined whether to perform a binary tree partitioning or a ternary tree partitioning on the coded block by considering at least one of the following: the size of the coded block, whether the coded block is located at an image boundary, the maximum partitioning depth, or the partitioning shape of adjacent blocks. When it is determined whether to perform a binary tree partitioning or a ternary tree partitioning on the coded block, information indicating the partitioning direction can be transmitted via a signal in the bitstream. This information can be a 1-bit flag "mtt_split_cu_vertical_flag". The partitioning direction (vertical or horizontal) can be determined based on this flag. Additionally, information indicating whether a binary tree partitioning or a ternary tree partitioning is applied to the coded block can be transmitted via a signal in the bitstream. This information can be a 1-bit flag "mtt_split_cu_binary_flag". The binary tree partitioning or ternary tree partitioning can be determined based on this flag.

[0140] For example, in Figure 5 The example shown illustrates the application of a vertical binary tree partitioning to a coded block with a partitioning depth of 1, the application of a vertical ternary tree partitioning to the left coded block in the resulting coded block, and the application of a vertical binary tree partitioning to the right coded block.

[0141] When implementing devices for encoding or decoding video, there is a problem that hardware performance may be insufficient to handle regions larger than a threshold. For example, there may be a problem where, when hardware performance allows for processing up to 4096 samples simultaneously, 64×64 data units would be redundantly accessed and processed, and data could not be processed simultaneously for regions with more than 4096 samples. As mentioned above, the basic unit of data processing can be defined as a pipelined basic data unit (virtual processing data unit, VPDU, hereinafter referred to as basic data unit).

[0142] Basic data units can be categorized into square, non-square, or non-rectangular types.

[0143] Figure 6 It is a diagram showing the shape of the basic data unit.

[0144] The basic data unit can include samples that are equal to or less than the maximum number of samples that can be processed simultaneously. For example, as in Figure 6 In the example shown in (a), a 64×64 square block can be used as the basic data unit. Alternatively, a non-square block can be used as the basic data unit. For example, as in... Figure 6 (b) or Figure 6 In the example shown in (c), a 32×128 block or a 64×32 block can be set as the basic data unit.

[0145] Although not shown, a basic data unit of a triangle, an L shape, or a polygon can be defined.

[0146] Information for determining the basic data unit can be signaled by a bitstream. The information can be used to determine at least one of a size or a shape of the basic data unit. Based on the information, it can be determined whether a non-square basic data unit is allowed or not.

[0147] Alternatively, at least one of a size or a shape of the basic data unit can be predefined in an encoder and a decoder.

[0148] Whether a type of division of a coding block is allowed can be determined in consideration of a size of the basic data unit. For example, when a coding block resulting from division of a coding block is larger than the basic data unit, such division can not be allowed. Alternatively, when a non-square coding block resulting from division of a coding block is larger than the basic data unit, such division can not be allowed. For example, when a width or a height of a coding block is larger than a threshold, or when a number of samples included in a coding block is larger than a threshold, binary tree or ternary tree division can not be allowed. Accordingly, encoding of information related to binary tree or ternary tree division can be omitted.

[0149] Alternatively, it can be set that division of a coding block larger than the basic data unit must be performed. Alternatively, it can be set that binary tree division or ternary tree division of a coding block larger than the basic data unit must be performed. Accordingly, for a coding block larger than the basic data unit, although a flag split_flag indicating whether a coding block is divided is not encoded, a value of the flag can be derived as 1.

[0150] As another example, a coding block larger than the basic data unit can be divided into a plurality of sub-blocks. Here, the sub-blocks can be set as prediction units which are basic units for prediction, or as transform units which are basic units for transform and / or quantization. In this case, division of a coding block into a plurality of prediction units can be defined as VPDU prediction unit division, and division of a coding block into a plurality of transform units can be defined as VPDU transform unit division.

[0151] At least one of VPDU prediction unit division and VPDU transform unit division can be applied to a coding block. A type of division of a coding block to which VPDU prediction unit division is applied can be set to be the same as a type of division of a coding block to which VPDU transform unit division is applied.

[0152] When only the VPDU prediction unit partition is applied to the coding block, prediction is performed for each sub-block, but transform and / or quantization can be performed for the coding block. In this case, a prediction mode such as a predictive coding mode, an intra prediction mode, or an inter prediction mode can be determined for the coding block.

[0153] When only the VPDU transform unit partition is applied to the coding block, prediction is performed for the sub-blocks, but transform and / or quantization can be performed for each sub-block.

[0154] Figure 7 and Figure 8 is a diagram illustrating an example of partitioning a coding block into a plurality of sub-blocks.

[0155] Figure 7 is a diagram illustrating partition modes when only square basic data units are allowed, Figure 8 is a diagram illustrating partition modes when square basic data units and non-square basic data units are allowed.

[0156] Assuming that only square basic data units are allowed, in Figure 7 (a) and Figure 7 (b), CU0 and CU3 are defined as two different VPDU, and CU1 is defined as four different VPDU. Accordingly, CU0 and CU3 can be partitioned into two sub-blocks, and CU1 can be partitioned into four sub-blocks.

[0157] Assuming that square basic data units and non-square basic data units are allowed, in Figure 8 (a) and Figure 8 (b), CU0 and CU3 can be defined as one VPDU, and CU1 can be defined as using two different VPDU. Accordingly, CU0 and CU3 are not partitioned into sub-blocks, and CU1 can be partitioned into two sub-blocks.

[0158] In this case, CU1 can be partitioned into square sub-blocks or non-square sub-blocks. For example, CU1 can be partitioned into two square sub-blocks based on a horizontal line partitioning CU1 upward and downward. Alternatively, CU1 can be partitioned into two non-square sub-blocks based on a vertical line partitioning CU1 left and right.

[0159] When there are a plurality of partition type candidates applicable to the coding block, information indicating any one of the plurality of partition type candidates can be signaled through a bitstream. For example, the information can indicate whether the coding block is partitioned into square sub-blocks or whether the coding block is partitioned into non-square sub-blocks.

[0160] Alternatively, the division of the coding block into a square sub-block can be set to have a higher priority than the division of the coding block into a non-square sub-block. For example, when the coding block cannot be divided into a square sub-block, the division of the coding block into a non-square sub-block can be allowed.

[0161] Alternatively, the division type of the coding block can be determined based on the division type of a parent node coding block. For example, when the parent node coding block is divided based on a ternary tree, the coding block can be set to be divided into a square sub-block. On the other hand, when the parent node coding block is divided based on a binary tree or a ternary tree, the coding block can be set to be divided into a non-square sub-block.

[0162] Inter prediction refers to a prediction encoding mode in which information of a previous image is used to predict a current block. For example, a block in the previous image having the same position as the current block (hereinafter referred to as a collocated block) can be set as a prediction block of the current block. Hereinafter, a prediction block generated based on a block having the same position as the current block will be referred to as a collocated prediction block.

[0163] On the other hand, if an object existing in the previous image has moved to another position in the current image, the motion of the object can be used to effectively predict the current block. For example, if the moving direction and size of the object can be known by comparing the previous image and the current image, a prediction block (or a predicted image) of the current block can be generated considering the motion information of the object. Hereinafter, a prediction block generated using the motion information can be referred to as a motion prediction block.

[0164] A residual block can be generated by subtracting the prediction block from the current block. In this case, when there is motion of the object, the energy of the residual block can be reduced by using the motion prediction block instead of the collocated prediction block, and thus the compression performance of the residual block can be improved.

[0165] As described above, a process of generating a prediction block using motion information can be referred to as motion-compensated prediction. In most inter prediction, a prediction block can be generated based on motion-compensated prediction.

[0166] The motion information can include at least one of a motion vector, a reference picture index, a prediction direction, or a bi-directional weighting value index. The motion vector indicates a moving direction and a size of the object. The reference picture index specifies a reference picture of the current block from among a plurality of reference pictures included in a reference picture list. The prediction direction refers to any one of a uni-prediction L0, a uni-prediction L1, or a bi-prediction (L0 and L1). At least one of the motion information of the L0 direction or the motion information of the L1 direction can be used according to the prediction direction of the current block. The bi-directional weighting value index specifies a weighting value for the L0 prediction block and a weighting value applied to the L1 prediction block.

[0167] Figure 9 is a flowchart illustrating an inter prediction method according to an embodiment of the present application.

[0168] Referring to Figure 9 , the inter prediction method includes determining an inter prediction mode of a current block (S901), obtaining motion information of the current block according to the determined inter prediction mode (S902), and performing a motion compensation prediction of the current block based on the obtained motion information (S903).

[0169] The inter prediction mode indicates a plurality of techniques for determining the motion information of the current block, and can include an inter prediction mode using translation motion information and an inter prediction mode using affine motion information. For example, the inter prediction mode using translation motion information can include a merge mode and an advanced motion vector prediction mode, and the inter prediction mode using affine motion information can include an affine merge mode and an affine motion vector prediction mode. According to the inter prediction mode, the motion information of the current block can be determined based on a neighboring block adjacent to the current block or information parsed from a bitstream.

[0170] Hereinafter, an inter prediction method using affine motion information will be described in detail.

[0171] Figure 10 is a diagram illustrating a non-linear motion of an object.

[0172] The motion of an object within a video can be a non-linear motion. For example, as in an example shown in Figure 10 , a non-linear motion of an object, such as zoom-in, zoom-out, rotation, affine transformation, etc. of a camera, can occur. When the non-linear motion of the object occurs, the motion of the object cannot be effectively represented by a translation motion vector. Thus, in a portion where the non-linear motion of the object occurs, the translation motion can be replaced by using affine motion, thereby improving coding efficiency.

[0173] Figure 11 is a flowchart illustrating an inter prediction method based on affine motion according to an embodiment of the present application.

[0174] It can be determined whether to apply the affine motion based inter prediction technique to the current block based on information parsed from the bitstream. Specifically, it can be determined whether to apply the affine motion based inter prediction technique to the current block based on at least one of a flag indicating whether to apply an affine merge mode to the current block or a flag indicating whether to apply an affine motion vector prediction mode to the current block.

[0175] When the affine motion based inter prediction technique is applied to the current block, an affine motion model of the current block can be determined (S1101). The affine motion model can be determined by at least one of a 6-parameter affine motion model or a 4-parameter affine motion model. The 6-parameter affine motion model uses 6 parameters to represent the affine motion, and the 4-parameter affine motion model uses 4 parameters to represent the affine motion.

[0176] Equation 1 is a case where the affine motion is represented using 6 parameters. The affine motion represents a translational motion of a predetermined region determined by an affine seed vector.

[0177]

Equation 1

[0178] v x = ax - by + e

[0179] v y = cx + dy + f

[0180] When the affine motion is represented using 6 parameters, a complex motion can be represented, but the number of bits required to encode each parameter increases, which can reduce the coding efficiency. Thus, the affine motion can also be represented using 4 parameters. Equation 2 is a case where the affine motion is represented using 4 parameters.

[0181]

Equation 2

[0182] v x = ax - by + e

[0183] v y = bx + ay + f

[0184] Information for determining the affine motion model of the current block can be encoded and can be signaled through the bitstream. For example, the information can be a 1-bit flag "affine_type_flag". A value of 0 of the flag indicates that the 4-parameter affine motion model is applied, and a value of 1 of the flag indicates that the 6-parameter affine motion model is applied. The flag can be encoded in units of a slice, a tile, or a block (e.g., a coding block or a coding tree unit). When the flag is signaled at a slice level, the affine motion model determined at the slice level can be applied to all blocks belonging to the slice.

[0185] Alternatively, the affine motion model of the current block can be determined based on the affine inter-frame prediction mode of the current block. For example, when applying the affine merging mode, the affine motion model of the current block can be determined as a 4-parameter motion model. On the other hand, when applying the affine motion vector prediction mode, the information used to determine the affine motion model of the current block can be encoded and transmitted via a signal through the bitstream. For example, when applying the affine motion vector prediction mode to the current block, the affine motion model of the current block can be determined based on a 1-bit flag "affine_type_flag".

[0186] Next, the affine seed vector of the current block can be exported (S1102). When a 4-parameter affine motion model is selected, motion vectors at two control points of the current block can be exported. On the other hand, when a 6-parameter affine motion model is selected, motion vectors at three control points of the current block can be exported. The motion vectors at the control points can be called affine seed vectors. Control points can include at least one of the upper left, upper right, or lower left corners of the current block.

[0187] Figure 12 This is a diagram showing an example of the affine seed vector for each affine motion model.

[0188] In a 4-parameter affine motion model, two related affine seed vectors can be derived from the top left, top right, or bottom left corners. For example, as in... Figure 12 In the example shown in (a), when the 4-parameter affine motion model is selected, the affine vectors can be derived using the affine seed vector SV0 of the top-left corner of the current block (e.g., the top-left sample (x0, y0)) and the affine seed vector sv1 of the top-right corner of the current block (e.g., the top-right sample (x1, y1)). Alternatively, the affine seed vector associated with the bottom-left corner can be used instead of the affine seed vector associated with the top-left corner, or vice versa.

[0189] In a 6-parameter affine motion model, affine seed vectors related to the top-left, top-right, and bottom-left corners can be derived. For example, as in... Figure 12 In the example shown in (b), when the 6-parameter affine motion model is selected, the affine vectors can be derived using the affine seed vector SV0 of the top left corner of the current block (e.g., the top left sample (x0, y0)), the affine seed vector sv1 of the top right corner of the current block (e.g., the top right sample (x1, y1)), and the affine seed vector sv2 of the bottom left corner of the current block (e.g., the bottom left sample (x2, y2)).

[0190] In the embodiments described later, the affine seed vectors of the top-left control point and the top-right control point are referred to as a first affine seed vector and a second affine seed vector, respectively, under the 4-parameter affine motion model. In the embodiments described later using the first affine seed vector and the second affine seed vector, at least one of the first affine seed vector and the second affine seed vector can be replaced with an affine seed vector of the bottom-left control point (a third affine seed vector) or an affine seed vector of the bottom-right control point (a fourth affine seed vector).

[0191] Also, under the 6-parameter affine motion model, the affine seed vectors of the top-left control point, the top-right control point, and the bottom-left control point are referred to as a first affine seed vector, a second affine seed vector, and a third affine seed vector, respectively. In the embodiments described later using the first affine seed vector, the second affine seed vector, and the third affine seed vector, at least one of the first affine seed vector, the second affine seed vector, and the third affine seed vector can be replaced with an affine seed vector of the bottom-right control point (a fourth affine seed vector).

[0192] The affine vector of each sub-block can be derived by using the affine seed vectors (S1103). Here, the affine vector represents a translational motion vector derived based on the affine seed vectors. The affine vector of the sub-block can be referred to as an affine sub-block motion vector or a sub-block motion vector.

[0193] Figure 13 FIG. 4 is a diagram illustrating an example of an affine vector of a sub-block under a 4-parameter motion model.

[0194] The affine vector of the sub-block can be derived based on the positions of the control points, the position of the sub-block, and the affine seed vectors. For example, Equation 3 represents an example of deriving the affine sub-block vector.

[0195] [Equation 3]

[0196]

[0197]

[0198] In the Equation 3, (x, y) represents the position of the sub-block. Here, the position of the sub-block represents the position of a reference sample included in the sub-block. The reference sample can be a sample located at the top-left corner of the sub-block, or a sample whose at least one of the x-axis or y-axis coordinates is located at the center position. (x0, y0) represents the position of the first control point, and (sv 0x , sv 0y ) represents the first affine seed vector. In addition, (x1, y1) represents the position of the second control point, and (sv 1x , sv 1y ) represents the second affine seed vector.

[0199] When the first control point and the second control point correspond to the top-left corner and the top-right corner of the current block, respectively, x1-x0 can be set to a value identical to the width of the current block.

[0200] Thereafter, motion-compensated prediction of each sub-block can be performed using the affine vector of each sub-block (S1104). After performing the motion-compensated prediction, a prediction block related to each sub-block can be generated. The prediction block of the sub-block can be set as the prediction block of the current block.

[0201] Next, an inter prediction method using translational motion information will be described in detail.

[0202] Motion information of the current block can be derived from motion information of other blocks of the current block. The other blocks can be blocks that are more preferentially encoded / decoded with inter prediction than the current block. A case in which the motion information of the current block is set to be identical to the motion information of the other blocks can be defined as a merge mode. Also, a case in which the motion vector of the other blocks is set as a predicted value of the motion vector of the current block can be defined as a motion vector prediction mode.

[0203] Figure 14 is a flowchart illustrating a process of deriving motion information of a current block using a merge mode.

[0204] A merge candidate of the current block can be derived (S1401). The merge candidate of the current block can be derived from a block that is encoded / decoded with inter prediction before the current block.

[0205] Figure 15 is a diagram illustrating a candidate block used to derive a merge candidate.

[0206] The candidate block can include at least one of a neighboring block including samples neighboring the current block or a non-neighboring block including samples not neighboring the current block. Hereinafter, a sample used to determine the candidate block will be designated as a reference sample. In addition, a reference sample neighboring the current block will be referred to as a neighboring reference sample, and a reference sample not neighboring the current block will be referred to as a non-neighboring reference sample.

[0207] The neighboring reference samples can include samples in a neighboring column of the leftmost column of the current block or samples in a neighboring row of the topmost row of the current block. For example, if the coordinates of the top-left sample of the current block are (0, 0), at least one of a block including a reference sample at position (-1, H-1), a block including a reference sample at position (W-1, -1), a block including a reference sample at position (W, -1), a block including a reference sample at position (-1, H), or a block including a reference sample at position (-1, -1) can be used as a candidate block. Referring to the drawings, the neighboring blocks indexed 0 to 4 can be used as the candidate blocks.

[0208] A non-adjacent reference sample denotes a sample having at least one of an x-axis distance or a y-axis distance from a neighboring reference sample of a current block of a predefined value. For example, at least one of a block including a reference sample having an x-axis distance of a predefined value from a left neighboring reference sample, a block including a non-adjacent sample having a y-axis distance of a predefined value from an above neighboring reference sample, or a block including a non-adjacent sample having x-axis and y-axis distances of predefined values from a top-left neighboring reference sample can be used as a candidate block. The predefined value can be an integer of 4, 8, 12, 16, etc. Referring to the drawings, at least one of the blocks having indices of 5 to 26 can be used as a candidate block.

[0209] A sample not on the same vertical line, horizontal line, or diagonal line as a neighboring reference sample can be set as a non-adjacent reference sample.

[0210] Figure 16 is a diagram showing positions of reference samples.

[0211] As in the example shown in Figure 16 , an x-coordinate of an above non-adjacent reference sample can be set to be different from an x-coordinate of an above neighboring reference sample. For example, when a position of an above neighboring reference sample is (W-1, -1), a position of an above non-adjacent reference sample spaced by N along a y-axis from the above neighboring reference sample can be set to be ((W / 2)-1, -1-N), and a position of an above non-adjacent reference sample spaced by 2N along the y-axis from the above neighboring reference sample can be set to be (0, -1-2N). That is, a position of a non-adjacent reference sample can be determined based on a position of a neighboring reference sample and a distance between the neighboring reference sample.

[0212] Hereinafter, a candidate block including a neighboring reference sample among candidate blocks is referred to as a neighboring block, and a block including a non-adjacent reference sample is referred to as a non-adjacent block.

[0213] When a distance between a current block and a candidate block is greater than or equal to a threshold value, the candidate block can be set to be unusable as a merge candidate. The threshold value can be determined based on a size of a coding tree unit. For example, the threshold value can be set to a height of the coding tree unit (ctu_height), or a value obtained by adding or subtracting an offset value to or from the height of the coding tree unit (e.g., ctu_height±N). The offset value N is a value predefined in an encoder and a decoder, and can be set to 4, 8, 16, 32, or ctu_height.

[0214] When a difference between a y-axis coordinate of a current block and a y-axis coordinate of a sample included in a candidate block is greater than a threshold value, the candidate block can be determined to be unusable as a merge candidate.

[0215] Alternatively, a candidate block not belonging to the same coding tree unit as the current block can be set as unusable as a merge candidate. For example, when a reference sample is outside the upper boundary of the coding tree unit to which the current block belongs, a candidate block including the reference sample can be set as unusable as a merge candidate.

[0216] If, when the upper boundary of the current block is adjacent to the upper boundary of the coding tree unit, a plurality of candidate blocks are determined as unusable as a merge candidate, the coding / decoding efficiency of the current block can be reduced. To resolve the above problem, the candidate blocks can be set so that the number of candidate blocks located above the current block is greater than the number of candidate blocks located to the left of the current block.

[0217] Figure 17 FIG. 1 is a diagram illustrating an example of candidate blocks used to derive a merge candidate.

[0218] As in the example shown in Figure 17 , the upper block belonging to the N block columns above the current block and the left block belonging to the M block columns to the left of the current block can be set as candidate blocks. At this time, by setting M to be greater than N, the number of left candidate blocks can be set to be greater than the number of upper candidate blocks.

[0219] For example, the difference between the y-axis coordinate of the reference sample within the current block and the y-axis coordinate of the upper block usable as a candidate block can be set to be not more than N times the height of the current block. In addition, the difference between the x-axis coordinate of the reference sample within the current block and the x-axis coordinate of the left block usable as a candidate block can be set to be not more than M times the width of the current block.

[0220] For example, in the example shown in Figure 17 , it is shown that blocks belonging to two block columns above the current block and blocks belonging to five block columns to the left of the current block are set as candidate blocks.

[0221] As another example, when a candidate block does not belong to the same coding tree unit as the current block, a block belonging to the same coding tree unit as the current block, or a block including a reference sample adjacent to the boundary of the coding tree unit can be used instead of the candidate block to derive a merge candidate.

[0222] Figure 18 FIG. 1 is a diagram illustrating an example of candidate blocks used to derive a merge candidate.

[0223] When a reference sample is included in a coding tree unit different from the current block, and the reference sample is not adjacent to the boundary of the coding tree unit, a reference sample adjacent to the boundary of the coding tree unit can be used instead of the reference sample to determine a candidate block reference sample.

[0224] For example, in Figure 18 (a) and Figure 18(b) In the example shown, when the upper boundary of the current block and the upper boundary of the coding tree unit contact each other, the reference sample above the current block belongs to a coding tree unit different from the current block. A reference sample not adjacent to the upper boundary of the coding tree unit among the reference samples belonging to the coding tree unit different from the current block can be replaced with a sample adjacent to the upper boundary of the coding tree unit.

[0225] For example, as Figure 18 (a) In the example shown, the reference sample at position 6 is replaced with a sample at position 6' located at the upper boundary of the coding tree unit, as Figure 18 (b) In the example shown, the reference sample at position 15 is replaced with a sample at position 15' located at the upper boundary of the coding tree unit. At this time, the y coordinate of the replacement sample can be changed to the adjacent position of the coding tree unit, and the x coordinate of the replacement sample can be set to be the same as that of the reference sample. For example, the sample at position 6' can have the same x coordinate as the sample at position 6, and the sample at position 15' can have the same x coordinate as the sample at position 15.

[0226] Alternatively, a value obtained by adding or subtracting an offset value to the x coordinate of the reference sample can be set as the x coordinate of the replacement sample. For example, when the x coordinates of the adjacent reference sample and the non-adjacent reference sample located above the current block are the same, a value obtained by adding or subtracting an offset value to the x coordinate of the reference sample can be set as the x coordinate of the replacement sample. This is to prevent the replacement sample for replacing the non-adjacent reference sample from being located at the same position as other non-adjacent reference samples or adjacent reference samples.

[0227] Figure 19 is a diagram showing an example of changing the position of a reference sample.

[0228] In replacing a reference sample included in a coding tree unit different from the current block and not adjacent to the boundary of the coding tree unit with a sample located at the boundary of the coding tree unit, a value obtained by adding or subtracting an offset value to the x coordinate of the reference sample can be set as the x coordinate of the replacement sample.

[0229] For example, in Figure 19 the example shown, the reference sample at position 6 and the reference sample at position 15 can be replaced with a sample at position 6' and a sample at position 15', respectively, having the same y coordinate as the row adjacent to the upper boundary of the coding tree unit. At this time, the x coordinate of the sample at position 6' can be set to a value having a difference of W / 2 from the x coordinate of the reference sample at position 6, and the x coordinate of the sample at position 15' can be set to a value having a difference of W-1 from the x coordinate of the reference sample at position 15.

[0230] Unlike Figure 18 and Figure 19In the illustrated example, the y-coordinate of the line above the uppermost line of the current block or the y-coordinate of the upper boundary of the coding tree unit can also be set as the y-coordinate of the replacement sample.

[0231] Although not illustrated, a sample that replaces a reference sample can also be determined based on the left boundary of the coding tree unit. For example, when the reference sample is not included in the same coding tree unit as the current block and is not adjacent to the left boundary of the coding tree unit, the reference sample can be replaced with a sample adjacent to the left boundary of the coding tree unit. At this time, the replacement sample can have the same y-coordinate as the reference sample, or can have a y-coordinate obtained by adding or subtracting an offset value from the y-coordinate of the reference sample.

[0232] Thereafter, the block including the replacement sample can be set as a candidate block, and a merge candidate of the current block can be derived based on the candidate block.

[0233] A merge candidate can also be derived from a temporal neighboring block included in a different picture from the current block. For example, a merge candidate can be derived from a collocated block included in a collocated picture.

[0234] Motion information of the merge candidate can be set to be the same as motion information of the candidate block. For example, at least one of a motion vector, a reference picture index, a prediction direction, or a bi-directional weighted value index of the candidate block can be set as the motion information of the merge candidate.

[0235] A merge candidate list including the merge candidate can be generated (S1402). The merge candidate can be classified into a neighboring merge candidate derived from a neighboring block adjacent to the current block, and a non-neighboring merge candidate derived from a non-neighboring block.

[0236] Indices of a plurality of merge candidates within the merge candidate list can be assigned in a predetermined order. For example, an index assigned to a neighboring merge candidate can have a smaller value than an index assigned to a non-neighboring merge candidate. Alternatively, the indices can be assigned based on a distance between the current block and the candidate block. Figure 15 or Figure 17 An index can be assigned to each merge candidate based on an index of each block illustrated.

[0237] When a plurality of merge candidates are included in the merge candidate list, at least one of the plurality of merge candidates can be selected (S1403). At this time, information indicating whether or not the motion information of the current block is derived from the adjacent merge candidate can be signaled through a bitstream. The information can be a 1-bit flag. For example, a syntax element isAdjancentMergeFlag indicating whether or not the motion information of the current block is derived from the adjacent merge candidate can be signaled through a bitstream. When the value of the syntax element isAdjancentMergeFlag is 1, the motion information of the current block can be derived based on the adjacent merge candidate. On the other hand, when the value of the syntax element isAdjancentMergeFlag is 0, the motion information of the current block can be derived based on the non-adjacent merge candidate.

[0238] Table 1 shows a syntax table including the syntax element isAdjancentMergeFlag.

[0239]

Table 1

[0240]

[0241] Information for specifying any one of the plurality of merge candidates can be signaled through a bitstream. For example, information indicating an index of any one of the merge candidates included in the merge candidate list can be signaled through a bitstream.

[0242] When isAdjacentMergeflag is 1, a syntax element merge_idx for determining any one of the adjacent merge candidates can be signaled. The maximum value of the syntax element merge_idx can be set to a value which is 1 less than the number of the adjacent merge candidates.

[0243] When isAdjacentMergeflag is 0, a syntax element NA_merge_idx for determining any one of the non-adjacent merge candidates can be signaled. The syntax element NA_merge_idx indicates a value obtained by subtracting the number of the adjacent merge candidates from the index of the non-adjacent merge candidate. The decoder can select the non-adjacent merge candidate by adding the number of the adjacent merge candidates to the index determined according to NA_merge_idx.

[0244] When the number of the merge candidates included in the merge candidate list is less than a threshold value, a merge candidate included in an inter motion information list can be added to the merge candidate list. The threshold value can be a value obtained by subtracting an offset from the maximum number of the merge candidates that the merge candidate list can include or the maximum number of the merge candidates. The offset can be an integer such as 1 or 2. The inter motion information list can include a merge candidate derived based on a block encoded / decoded before the current block.

[0245] The inter motion information list includes merge candidates derived from blocks encoded / decoded with inter prediction within the current picture. For example, the motion information of the merge candidates included in the inter motion information list can be set to be the same as the motion information of the blocks encoded / decoded based on inter prediction. The motion information can include at least one of a motion vector, a reference picture index, a prediction direction, or a bi-directional weighting value index.

[0246] For ease of illustration, the merge candidates included in the inter motion information list are referred to as inter merge candidates.

[0247] The maximum number of merge candidates that can be included in the inter motion information list can be predefined in the encoder and the decoder. For example, the maximum number of merge candidates that can be included in the inter motion information list can be 1, 2, 3, 4, 5, 6, 7, 8, or greater (e.g., 16).

[0248] Alternatively, information indicating the maximum number of merge candidates of the inter motion information list can be signaled through a bitstream. The information is signaled at a sequence level, a picture level, or a slice level.

[0249] Alternatively, the maximum number of merge candidates of the inter motion information list can be determined according to a size of a picture, a size of a slice, or a size of a coding tree unit.

[0250] The inter motion information list can be initialized in units of a picture, a slice, a tile, a brick, a coding tree unit, or a coding tree unit line (row or column). For example, when a slice is initialized, the inter motion information list is also initialized, and the inter motion information list can not include any merge candidate.

[0251] Alternatively, information indicating whether to initialize the inter motion information list can also be signaled through a bitstream. The information can be signaled at a slice level, a tile level, a brick level, or a block level. Before the information indicates that the inter motion information list is initialized, a configured inter motion information list can be used.

[0252] Alternatively, information related to the inter merge candidate can be signaled through a picture parameter set or a slice header. The inter motion information list can include an initial inter merge candidate even if the slice is initialized. Thus, the inter merge candidate can be used for a block first encoded / decoded within the slice.

[0253] The blocks are encoded / decoded according to an encoding / decoding order, and the blocks encoded / decoded based on inter prediction can be sequentially set as the inter merge candidates according to the encoding / decoding order.

[0254] Figure 20 is a flowchart illustrating a process for updating an inter- motion information list.

[0255] When inter prediction is performed on the current block (S2001), an inter merge candidate can be derived based on the current block (S2002). The motion information of the inter merge candidate can be set to be the same as the motion information of the current block.

[0256] When the inter motion information list is empty (S2003), the inter merge candidate derived based on the current block can be added to the inter motion information list (S2004).

[0257] When the inter motion information list already includes the inter merge candidate (S2003), a redundancy check can be performed on the motion information of the current block (or the inter merge candidate derived based on the current block) (S2005). The redundancy check is used to determine whether the motion information of the inter merge candidate already stored in the inter motion information list is the same as the motion information of the current block. The redundancy check can be performed on all inter merge candidates already stored in the inter motion information list. Alternatively, the redundancy check can be performed on inter merge candidates whose indices are above or below a threshold among the inter merge candidates already stored in the inter motion information list.

[0258] When the inter merge candidate with the same motion information as the current block is not included (S2006), the inter merge candidate derived based on the current block can be added to the inter motion information list (S2008). Whether the inter merge candidates are the same can be determined based on whether the motion information (e.g., motion vector and / or reference picture index, etc.) of the inter merge candidates is the same.

[0259] In this case, when the maximum number of inter merge candidates has been stored in the inter motion information list (S2006), the oldest inter merge candidate is removed (S2007), and the inter merge candidate derived based on the current block can be added to the inter motion information list (S2008).

[0260] The plurality of inter merge candidates can be identified according to respective indices. When the inter merge candidate derived from the current block is added to the inter motion information list, the inter merge candidate is assigned the lowest index (e.g., 0), and the indices of the inter merge candidates already stored can be increased by 1, respectively. In this case, when the maximum number of inter merge candidates has been stored in the inter motion information list, the inter merge candidate with the largest index is removed.

[0261] Alternatively, when adding the inter merge candidate derived from the current block to the inter motion information list, the inter merge candidate can be assigned the largest index. For example, when the number of inter merge candidates already stored in the inter motion information list is less than the maximum value, the inter merge candidate can be assigned an index having the same value as the number of inter merge candidates already stored. Alternatively, when the number of inter merge candidates already stored in the inter motion information list is equal to the maximum value, the inter merge candidate can be assigned an index of the maximum value minus 1. Also, the inter merge candidate having the smallest index is removed, and the indices of the remaining stored inter merge candidates are each decreased by 1.

[0262] Figure 21 FIG. 1 is a diagram illustrating an embodiment of updating an inter merge candidate list.

[0263] It is assumed that an inter merge candidate derived from a current block is added to an inter merge candidate list, and the inter merge candidate is assigned the largest index. Also, it is assumed that the maximum number of inter merge candidates is already stored in the inter merge candidate list.

[0264] When adding the inter merge candidate HmvpCand[n+1] derived from the current block to the inter merge candidate list HmvpCandList, the inter merge candidate HmvpCand[0] having the smallest index is deleted from the stored inter merge candidates, and the indices of the remaining inter merge candidates are each decreased by 1. Also, the index of the inter merge candidate HmvpCand[n+1] derived from the current block can be set to the maximum value (n in the example shown in FIG. 1). Figure 21

[0265] When the same inter merge candidate as the inter merge candidate derived based on the current block is already stored (S2005), the inter merge candidate derived based on the current block can not be added to the inter motion information list (S2009).

[0266] Alternatively, as the inter merge candidate derived based on the current block is added to the inter motion information list, the stored inter merge candidate identical to the inter merge candidate can also be removed. In this case, an effect that the indices of the stored inter merge candidates are updated can occur.

[0267] Figure 22 FIG. 1 is a diagram illustrating an embodiment of updating an inter merge candidate list.

[0268] When the index of the stored inter merge candidate identical to the inter merge candidate derived based on the current block is hIdx, the stored inter merge candidate is deleted, and the indices of the inter merge candidates having indices greater than hIdx are each decreased by 1. For example, when the index of the stored inter merge candidate identical to the inter merge candidate derived based on the current block is hIdx, the stored inter merge candidate is deleted, and the indices of the inter merge candidates having indices greater than hIdx are each decreased by 1. Figure 22 ​In the illustrated example, HmvpCand[2] which is the same as mvCand is deleted from the inter motion information list HvmpCandList, and the indices of HmvpCand[3] to HmvpCand[n] are each decreased by 1.

[0269] Also, the inter merge candidate mvCand derived based on the current block can be added to the end of the inter motion information list.

[0270] Alternatively, the index assigned to the stored inter merge candidate which is the same as the inter merge candidate derived based on the current block can be updated. For example, the index of the stored inter merge candidate can be changed to the minimum value or the maximum value.

[0271] The motion information of the blocks included in a predetermined region can be set so as not to be added to the inter motion information list. For example, the inter merge candidate derived based on the motion information of the blocks included in the merge processing region can not be added to the inter motion information list. Since the encoding / decoding order of the blocks included in the merge processing region is not defined, it is not appropriate to use the motion information of any of these blocks for the inter prediction of other blocks. Thus, the inter merge candidate derived based on the blocks included in the merge processing region can not be added to the inter motion information list.

[0272] When motion compensation prediction is performed by sub-block units, the inter merge candidate can be derived based on the motion information of a representative sub-block among the plurality of sub-blocks included in the current block. For example, when a sub-block merge candidate is used for the current block, the inter merge candidate can be derived based on the motion information of a representative sub-block among the sub-blocks.

[0273] The motion vector of the sub-blocks can be derived in the following order. First, any one of the merge candidates included in the merge candidate list of the current block can be selected, and an initial shift vector (shVector) can be derived based on the motion vector of the selected merge candidate. Also, by adding the position (xSb, ySb) of the reference sample (for example, the top-left sample or the middle position sample) of each sub-block within the coded block to the initial shift vector, a shifted sub-block whose position of the reference sample is (xColSb, yColSb) can be derived. Equation 4 below shows an equation for deriving the shifted sub-block.

[0274]

Equation 4

[0275] (xColSb, yColSb) = (xSb + shVector[0] » 4, ySb + shVector[1] » 4)

[0276] Next, the motion vector of the collocated block corresponding to the sub-block center position including (xColSb, yColSb) is set as the motion vector of the sub-block including (xSb, ySb).

[0277] A representative sub-block can mean a sub-block including the top-left sample or the center sample of the current block.

[0278] Figure 23 is a diagram showing the positions of representative sub-blocks.

[0279] Figure 23 (a) shows an example of setting a sub-block located at the top-left of the current block as a representative sub-block, Figure 23 (b) shows an example of setting a sub-block located at the center of the current block as a representative sub-block. When performing motion compensation prediction in sub-block units, an inter-frame merge candidate of the current block can be derived based on the motion vector of a sub-block including the top-left sample of the current block or a sub-block including the center sample of the current block.

[0280] Based on the inter-frame prediction mode of the current block, it can also be determined whether to use the current block as an inter-frame merge candidate. For example, a block encoded / decoded based on an affine motion model can be set as unavailable to be used as an inter-frame merge candidate. Thus, even if the current block is encoded / decoded by inter-frame prediction, when the inter-frame prediction mode of the current block is an affine prediction mode, the inter-frame prediction motion information list is not updated based on the current block.

[0281] Alternatively, an inter-frame merge candidate can also be derived based on at least one of the sub-block vectors included in the sub-blocks of the block encoded / decoded based on an affine motion model. For example, an inter-frame merge candidate can be derived using a sub-block located at the top-left of the current block, a sub-block located at the center, or a sub-block located at the top-right of the current block. Alternatively, an average of the sub-block vectors of multiple sub-blocks can also be set as the motion vector of the inter-frame merge candidate.

[0282] Alternatively, an inter-frame merge candidate can also be derived based on an average of the affine seed vectors of the block encoded / decoded based on an affine motion model. For example, an average of at least one of the first affine seed vector, the second affine seed vector, or the third affine seed vector of the current block is set as the motion vector of the inter-frame merge candidate.

[0283] Alternatively, the inter-frame motion information list can be configured for different inter-frame prediction modes. For example, at least one of an inter-frame motion information list for a block encoded / decoded by intra-block copy, an inter-frame motion information list for a block encoded / decoded based on a translation motion model, or an inter-frame motion information list for a block encoded / decoded based on an affine motion model can be defined. According to the inter-frame prediction mode of the current block, any one of the multiple inter-frame motion information lists can be selected.

[0284] Figure 24 A diagram showing an example of generating a list of inter motion information for each inter prediction mode is shown.

[0285] When a block is encoded / decoded based on a non-affine motion model, an inter merge candidate mvCand derived based on the block can be added to a list of inter non-affine motion information HmvpCandList. On the other hand, when a block is encoded / decoded based on an affine motion model, an inter merge candidate mvAfCand derived based on the block can be added to a list of inter affine motion information HmvpAfCandList.

[0286] An affine seed vector of a block can be stored in an inter merge candidate derived from a block encoded / decoded based on an affine motion model. Thereby, the inter merge candidate can be used as a merge candidate for deriving an affine seed vector of a current block.

[0287] In addition to the described list of inter motion information, another list of inter motion information can be defined. In addition to the described list of inter motion information (hereinafter referred to as a first list of inter motion information), a long-term motion information list (hereinafter referred to as a second list of inter motion information) can be defined. Among them, the long-term motion information list includes a long-term merge candidate.

[0288] When both the first list of inter motion information and the second list of inter motion information are in an empty state, inter merge candidates can be first added to the second list of inter motion information. After the number of inter merge candidates available in the second list of inter motion information reaches a maximum number, inter merge candidates can be added to the first list of inter motion information.

[0289] Alternatively, one inter merge candidate can be added to both the second list of inter motion information and the first list of inter motion information.

[0290] In this case, the second list of inter motion information that has been configured can no longer be updated. Alternatively, the second list of inter motion information can be updated when the decoded area is above a predetermined ratio of a slice. Alternatively, the second list of inter motion information can be updated every N coding tree unit row.

[0291] On the other hand, the first list of inter motion information can be updated every time a block encoded / decoded with inter prediction is generated. However, the inter merge candidate added to the second list of inter motion information can be set not to be used to update the first list of inter motion information.

[0292] The information for selecting either the first inter motion information list or the second inter motion information list can be signaled through a bitstream. When the number of merge candidates included in the merge candidate list is less than a threshold, the merge candidate included in the inter motion information list indicated by the information can be added to the merge candidate list.

[0293] Alternatively, the inter motion information list can be selected based on the size, shape, inter prediction mode, bi-prediction or not, motion vector refinement or not, or triangle partition of the current block.

[0294] Alternatively, even when the inter merge candidate included in the first inter motion information list is added, the number of merge candidates included in the merge candidate list is less than the maximum number of merges, the inter merge candidate included in the second inter motion information list can be added to the merge candidate list.

[0295] Figure 25 FIG. 1 is a diagram illustrating an example of adding the inter merge candidate included in the long-term motion information list to the merge candidate list.

[0296] When the number of merge candidates included in the merge candidate list is less than the maximum number, the inter merge candidate included in the first inter motion information list HmvpCandList can be added to the merge candidate list. Even when the inter merge candidate included in the first inter motion information list is added to the merge candidate list, the number of merge candidates included in the merge candidate list is less than the maximum number, the inter merge candidate included in the long-term motion information list HmvpLTCandList can be added to the merge candidate list.

[0297] Table 2 illustrates a process of adding the inter merge candidate included in the long-term motion information list to the merge candidate list.

[0298]

Table 2

[0299]

[0300]

[0301] The inter merge candidate can be set to include additional information in addition to the motion information. For example, the size, shape, or partition information of the block storing the block can be added in the inter merge candidate. When the merge candidate list of the current block is constructed, only the inter merge candidate having the same or similar size, shape, or partition information as the current block is used in the inter merge candidate, or the inter merge candidate having the same or similar size, shape, or partition information as the current block can be preferentially added to the merge candidate list.

[0302] Alternatively, the inter motion information list can be generated for different block sizes, shapes, or partition information. The inter motion information list corresponding to the shape, size, or partition information of the current block can be used to generate the merge candidate list of the current block.

[0303] When the number of the merge candidates included in the merge candidate list of the current block is less than a threshold, the inter merge candidates included in the inter motion information list can be added to the merge candidate list. The adding process is performed in ascending order or descending order of the index. For example, the inter merge candidate with the largest index can be added to the merge candidate list.

[0304] When the inter merge candidates included in the inter motion information list are to be added to the merge candidate list, a redundancy check between the inter merge candidates and the plurality of merge candidates already stored in the merge candidate list can be performed.

[0305] For example, Table 3 shows the process of adding the inter merge candidates to the merge candidate list.

[0306]

Table 3

[0307]

[0308] The redundancy check can also be performed only on some of the inter merge candidates included in the inter motion information list. For example, the redundancy check can be performed only on the inter merge candidates with an index above or below a threshold. Alternatively, the redundancy check can be performed only on the N merge candidates with the largest index or the N merge candidates with the smallest index.

[0309] Alternatively, the redundancy check can be performed only on some of the merge candidates already stored in the merge candidate list. For example, the redundancy check can be performed only on the merge candidates with an index above or below a threshold or the merge candidates derived from blocks at specific positions. The specific positions can include at least one of a left neighboring block, an above neighboring block, a top-right neighboring block, or a bottom-left neighboring block of the current block.

[0310] Figure 26 FIG. 4 is a diagram showing an example of performing the redundancy check only on some of the merge candidates.

[0311] When the inter merge candidate HmvpCand[j] is to be added to the merge candidate list, a redundancy check between the inter merge candidate and the two merge candidates with the largest index, mergeCandList[NumMerge-2] and mergeCandList[NumMerge-1], can be performed. NumMerge can represent the number of the spatial merge candidates and the temporal merge candidates available.

[0312] Unlike the example shown in the figure, when an inter merge candidate HmvpCand[j] is to be added to the merge candidate list, a redundancy check between the inter merge candidate and the 2 merge candidates with the smallest index can also be performed. For example, it can be confirmed whether mergeCandList[0] and mergeCandList[1] are the same as HmvpCand[j]. Alternatively, the redundancy check can be performed only on the merge candidates derived from a specific location. For example, the redundancy check can be performed only on at least one of the merge candidates derived from the neighboring block located on the left side of the current block or the merge candidates derived from the neighboring block located above the current block. When there is no merge candidate derived from a specific location in the merge candidate list, the inter merge candidate can be added to the merge candidate list without the redundancy check.

[0313] In the case where the same merge candidate as the first inter merge candidate is found, the redundancy check of the same merge candidate as the first inter merge candidate can be omitted when the redundancy check is performed on the second inter merge candidate.

[0314] Figure 27 is a figure showing an example of omitting the redundancy check of a specific merge candidate.

[0315] When an inter merge candidate HmvpCand[i] with index i is to be added to the merge candidate list, a redundancy check between the inter merge candidate and the merge candidates already stored in the merge candidate list can be performed. In this case, in the case where the same merge candidate mergeCandList[j] as the inter merge candidate HmvpCand[i] is found, the inter merge candidate HmvpCand[i] will not be added to the merge candidate list, and a redundancy check between the inter merge candidate HmvpCand[i-1] with index i-1 and the merge candidate can be performed. In this case, the redundancy check between the inter merge candidate HmvpCand[i-1] and the merge candidate mergeCandList[j] can be omitted.

[0316] For example, in the example shown in Figure 27 In the example shown in FIG. 6, it is determined that HmvpCand[i] is the same as mergeCandList[2]. Thus, HmvpCand[i] is not added to the merge candidate list, and the redundancy check can be performed on HmvpCand[i-1]. In this case, the redundancy check between HmvpCand[i-1] and mergeCandList[2] can be omitted.

[0317] When the number of merge candidates included in the merge candidate list of the current block is less than a threshold, at least one of a pairwise merge candidate or a zero merge candidate can be included in addition to the inter merge candidate. The pairwise merge candidate refers to a merge candidate having an average value of motion vectors of two or more merge candidates as a motion vector, and the zero merge candidate refers to a merge candidate having a motion vector of 0.

[0318] The merge candidate list of the current block can add merge candidates in the following order.

[0319] Spatial merge candidate - temporal merge candidate - inter merge candidate - (inter affine merge candidate) - pairwise merge candidate - zero merge candidate.

[0320] The spatial merge candidate refers to a merge candidate derived from at least one of a neighboring block or a non-neighboring block, and the temporal merge candidate refers to a merge candidate derived from a previous reference picture. The inter affine merge candidate list indicates an inter merge candidate derived from a block encoded / decoded in an affine motion model.

[0321] The inter motion information list can also be used in the advanced motion vector prediction mode. For example, when the number of motion vector prediction candidates included in the motion vector prediction candidate list of the current block is less than a threshold, an inter merge candidate included in the inter motion information list is set as a motion vector prediction candidate related to the current block. Specifically, the motion vector of the inter merge candidate is set as the motion vector prediction candidate.

[0322] If any one of the motion vector prediction candidates included in the motion vector prediction candidate list of the current block is selected, the selected candidate is set as a motion vector prediction value of the current block. Thereafter, after the motion vector residual value of the current block is decoded, the motion vector of the current block can be obtained by adding the motion vector prediction value and the motion vector residual value.

[0323] The motion vector prediction candidate list of the current block can be constructed in the following order.

[0324] Spatial motion vector prediction candidate - temporal motion vector prediction candidate - inter decoded region merge candidate - (inter decoded region affine merge candidate) - zero motion vector prediction candidate.

[0325] The spatial motion vector prediction candidate refers to a motion vector prediction candidate derived from at least one of a neighboring block or a non-neighboring block, and the temporal motion vector prediction candidate refers to a motion vector prediction candidate derived from a previous reference picture. The inter affine merge candidate list indicates an inter motion vector prediction candidate derived from a block encoded / decoded in an affine motion model. The zero motion vector prediction candidate indicates a candidate having a value of 0 for a motion vector.

[0326] When a merge candidate of the current block is selected, a motion vector of the selected merge candidate is set as an initial motion vector, and motion-compensated prediction of the current block can be performed using a motion vector derived by adding or subtracting an offset vector to the initial motion vector. Deriving a new motion vector by adding or subtracting the offset vector to the motion vector of the merge candidate can be defined as a merge motion difference coding method.

[0327] Information indicating whether the merge offset coding method is used can be signaled through a bitstream. The information can be a 1-bit flag merge_offset_vector_flag. For example, a value of 1 of the merge_offset_vector_flag indicates that the merge motion difference coding method is applied to the current block. When the merge motion difference coding method is applied to the current block, a motion vector of the current block can be derived by adding or subtracting an offset vector to a motion vector of a merge candidate. A value of 0 of the merge_offset_vector_flag indicates that the merge motion difference coding method is not applied to the current block. When the merge offset coding method is not applied, the motion vector of the merge candidate can be set as the motion vector of the current block.

[0328] The flag can be signaled only when a value of a skip flag indicating whether a skip mode is applied is true or a value of a merge flag indicating whether a merge mode is applied is true. For example, when a value of a skip_flag indicating whether the skip mode is applied to the current block is 1, or when a value of a merge_flag indicating whether the merge mode is applied to the current block is 1, the merge_offset_vector_flag can be encoded and signaled.

[0329] When it is determined that the merge offset coding method is applied to the current block, at least one of information specifying any one of the merge candidates included in the merge candidate list, information indicating a size of the offset vector, and information indicating a direction of the offset vector can be additionally signaled.

[0330] Information for determining a maximum number of the merge candidates that the merge candidate list can include can be signaled through a bitstream. For example, the maximum number of the merge candidates that the merge candidate list can include can be set as 6 or less.

[0331] When it is determined to apply the merge offset coding method to the current block, only a preset maximum number of merge candidates can be set as initial motion vectors of the current block. That is, depending on whether the merge offset coding method is applied, the number of merge candidates that can be used by the current block can be adaptively determined. For example, when the value of the merge_offset_vector_flag is set to 0, the maximum number of merge candidates that can be used by the current block can be set to M, and when the value of the merge_offset_vector_flag is set to 1, the maximum number of merge candidates that can be used by the current block can be set to N. Here, M denotes the maximum number of merge candidates that can be included in the merge candidate list, and N denotes an integer equal to or smaller than M.

[0332] For example, when M is 6 and N is 2, two merge candidates having the smallest indices among the merge candidates included in the merge candidate list can be set as available for the current block. Accordingly, the motion vector of the merge candidate having the index value of 0 or the motion vector of the merge candidate having the index value of 1 can be set as the initial motion vector of the current block. When M and N are the same (for example, when M and N are 2), all of the merge candidates included in the merge candidate list can be set as available for the current block.

[0333] Alternatively, whether neighboring blocks can be used as merge candidates can be determined based on whether the merge motion difference coding method is applied to the current block. For example, when the value of the merge_offset_vector_flag is 1, at least one of a neighboring block adjacent to the upper right corner of the current block, a neighboring block adjacent to the upper left corner, and a neighboring block adjacent to the lower left corner can be set as unavailable to be used as a merge candidate. Accordingly, when the merge motion difference coding method is applied to the current block, the motion vector of at least one of the neighboring block adjacent to the upper right corner of the current block, the neighboring block adjacent to the upper left corner, and the neighboring block adjacent to the lower left corner cannot be set as the initial motion vector. Alternatively, when the value of the merge_offset_vector_flag is 1, a temporal neighboring block of the current block can be set as unavailable to be used as a merge candidate.

[0334] When the merge motion difference coding method is applied to the current block, at least one of a pair-wise merge candidate and a zero merge candidate can be set not to be used. Accordingly, when the value of the merge_offset_vector_flag is 1, at least one of the pair-wise merge candidate or the zero merge candidate can not be added to the merge candidate list although the number of merge candidates included in the merge candidate list is smaller than the maximum number.

[0335] The motion vector of the merge candidate can be set as the initial motion vector of the current block. In this case, when the number of the merge candidates available for the current block is plural, information for specifying any one of the plural merge candidates can be signaled through a bitstream. For example, when the maximum number of the merge candidates that can be included in the merge candidate list is greater than 1, information indicating any one of the plural merge candidates, merge_idx, can be signaled through the bitstream. That is, in the merge offset coding method, the merge candidate can be specified by the information for specifying any one of the plural merge candidates, merge_idx. The initial motion vector of the current block can be set as the motion vector of the merge candidate indicated by merge_idx.

[0336] On the other hand, when the number of the merge candidates available for the current block is 1, the information for specifying the merge candidate can be omitted to be signaled. For example, when the maximum number of the merge candidates that can be included in the merge candidate list is not greater than 1, the information for specifying the merge candidate, merge_idx, can be omitted to be signaled. That is, in the merge offset coding method, when one merge candidate is included in the merge candidate list, the coding of the information for specifying the merge candidate, merge_idx, can be omitted, and the initial motion vector can be determined based on the merge candidate included in the merge candidate list. The motion vector of the merge candidate can be set as the initial motion vector of the current block.

[0337] As another example, after determining the merge candidate of the current block, it can be determined whether to apply the merge motion difference value coding method to the current block. For example, when the maximum number of the merge candidates that can be included in the merge candidate list is greater than 1, the information for specifying any one of the merge candidates, merge_idx, can be signaled. After selecting the merge candidate based on merge_idx, merge_offset_vector_flag indicating whether to apply the merge motion difference value coding method to the current block can be decoded. Table 4 is a diagram illustrating a syntax table according to the above-described embodiment.

[0338]

Table 4

[0339]

[0340]

[0341] As another example, after determining the merge candidates for the current block, it can be determined whether to apply the merge motion interpolation (MIO) method to the current block only if the index of the determined merge candidate is less than the maximum number of merge candidates that can be used when applying the merge motion interpolation method. For example, the merge_offset_vector_flag, which indicates whether to apply the merge motion interpolation method to the current block, can only be encoded and signaled if the value of the index information merge_idx is less than N. When the value of the index information merge_idx is equal to or greater than N, the encoding of merge_offset_vector_flag can be omitted. If the encoding of merge_offset_vector_flag is omitted, it can be determined that the merge motion interpolation method has not been applied to the current block.

[0342] Alternatively, after determining the merge candidates for the current block, it is possible to consider whether the determined merge candidates have bidirectional or unidirectional motion information to determine whether to apply the merge motion difference encoding method to the current block. For example, the merge_offset_vector_flag indicating whether to apply the merge motion difference encoding method to the current block is encoded and signaled only if the value of the index information merge_idx is less than N and the merge candidate selected by the index information has bidirectional motion information. Alternatively, the merge_offset_vector_flag indicating whether to apply the merge motion difference encoding method to the current block is encoded and signaled only if the value of the index information merge_idx is less than N and the merge candidate selected by the index information has unidirectional motion information.

[0343] Alternatively, the decision to apply the merge motion interpolation coding method can be based on at least one of the following: the size of the current block, the shape of the current block, and whether the current block is in contact with the boundary of a coding tree unit. When at least one of the following conditions is not met, the encoding of the merge_offset_vector_flag indicating whether to apply the merge motion interpolation coding method to the current block can be omitted.

[0344] When a merge candidate is selected, its motion vector can be set as the initial motion vector of the current block. Then, information indicating the magnitude and direction of the offset vector can be decoded to determine the offset vector. The offset vector can have a horizontal or vertical component.

[0345] The information indicating the size of the offset vector can be index information indicating any one of the motion offset size candidates. For example, index information distance_idx indicating any one of the motion offset size candidates can be signaled through a bitstream. Table 5 shows binarization of the index information distance_idx and values of a variable DistFromMergeMV used to determine the size of the offset vector from the distance_idx.

[0346] [Table 5]

[0347]

[0348] The size of the offset vector can be derived by dividing the variable DistFromMergeMV by a preset value. Equation 5 shows an example of determining the size of the offset vector.

[0349] [Equation 5]

[0350] abs(offsetMV) = DistFromMergeMV « 2

[0351] According to Equation 5, a value obtained by dividing the variable DistFromMegeMV by 4 or a value obtained by left-shifting the variable DistFromMergeMV by 2 can be set as the size of the offset vector.

[0352] More motion offset size candidates than the example shown in Table 5 or fewer motion offset size candidates can be used, or the number of motion vector offset size candidates can be set to be different from the example shown in Table 5. For example, the size of the horizontal direction component or the vertical direction component of the offset vector can be set to be not more than 2 sample distances. Table 6 shows binarization of the index information distance_idx and values of a variable DistFromMergeMV used to determine the size of the offset vector from the distance_idx.

[0353] [Table 6]

[0354]

[0355] Alternatively, the number of motion vector offset size candidates can be set differently based on the motion vector precision. For example, when the motion vector precision of the current block is fractional-pel, the value of the variable DistFromMergeMV corresponding to the value of the index information distance_idx can be set to 1, 2, 4, 8, 16, etc. Here, the fractional-pel includes at least one of 1 / 16-pel, 1 / 8-pel, 1 / 4-pel, or half-pel. On the other hand, when the motion vector precision of the current block is integer-pel, the value of the variable DistFromMergeMV corresponding to the value of the index information distance_idx can be set to 4, 8, 16, 32, 64, etc. That is, the table used to determine the variable DistFromMergeMV can be set differently according to the motion vector precision of the current block.

[0356] For example, when the motion vector precision of the current block or the merge candidate is 1 / 4-pel, the variable DistFromMergeMV represented by distance_idx can be derived using Table 5. On the other hand, when the motion vector precision of the current block or the merge candidate is integer-pel, the value of the variable DistFromMergeMV corresponding to the value of distance_idx in Table 5 can be set as the value of the variable DistFromMergeMV by taking N times (e.g., 4 times) the value of the variable DistFromMergeMV indicated by distance_idx in Table 5.

[0357] Information for determining the motion vector precision can be signaled through a bitstream. For example, the information can be signaled at a sequence level, a picture level, a slice level, or a block level. Accordingly, the number of motion offset size candidates can be set differently according to the information related to the motion vector precision signaled through the bitstream. Alternatively, the motion vector precision can be determined based on the merge candidate of the current block. For example, the motion vector precision of the current block can be set to be the same as the motion vector precision of the merge candidate.

[0358] Alternatively, information for determining the search range of the offset vector can be signaled through a bitstream. At least one of the number of motion offset size candidates, the minimum value among the motion offset size candidates, and the maximum value among the motion offset size candidates can be determined based on the search range. For example, a flag merge_offset_vector_flag for determining the search range of the offset vector can be signaled through a bitstream. The information can be signaled through a sequence header, a picture header, or a slice header.

[0359] For example, when the value of merge offset extend range flag is 0, the size of the offset vector can be set to no more than 2. Thus, the maximum value of DistFromMergeMV can be set to 8. On the other hand, when the value of merge offset extend range flag is 1, the size of the offset vector can be set to no more than 32 sample distances. Thus, the maximum value of DistFromMergeMV can be set to 128.

[0360] A flag indicating whether the size of the offset vector is greater than a threshold value can be used to determine the size of the offset vector. For example, a flag distance_flag indicating whether the size of the offset vector is greater than a threshold value can be signaled through the bitstream. The threshold value can be 1, 2, 4, 8, or 16. For example, distance_flag of 1 indicates that the size of the offset vector is greater than 4. On the other hand, distance_flag of 0 indicates that the size of the offset vector is 4 or less.

[0361] When the size of the offset vector is greater than the threshold value, an index information distance_idx can be used to derive the difference between the size of the offset vector and the threshold value. Alternatively, when the size of the offset vector is less than or equal to the threshold value, the size of the offset vector can be determined by using the index information distance_idx. Table 7 is a syntax table showing the process of encoding distance_flag and distance_idx.

[0362]

Table 7

[0363]

[0364]

[0365] Equation 6 shows an example of using distance_flag and distance_idx to derive a variable DistFromMergeMV used to determine the size of the offset vector.

[0366]

Equation 6

[0367] DistFromMergeMV = N * distance_flag + (1 « distance_idx)

[0368] In Equation 6, the value of distance_flag can be set to 1 or 0. The value of distance_idx can be set to 1, 2, 4, 8, 16, 32, 64, 128, etc. N represents a coefficient determined by the threshold value. For example, when the threshold value is 4, N can be set to 16.

[0369] The information indicating the direction of the offset vector can be index information indicating any one of the vector direction candidates. For example, the index information indicating any one of the vector direction candidates, direction idx, can be signaled through a bitstream. Table 8 shows binarization of the index information direction idx and the direction of the offset vector according to the direction idx.

[0370]

Table 8

[0371] direction_idx[x][y] binarization sign[x][y][0] sign[x][y][1] 0 00 +1 0 1 01 -1 0 2 10 0 +1 3 11 0 -1

[0372] In Table 8, sign[0] indicates the horizontal direction, and sign[1] indicates the vertical direction. +1 indicates that the value of the x component or the y component of the offset vector is positive (+), and -1 indicates that the value of the x component or the y component of the offset vector is negative (-). Equation 7 shows an example of determining the offset vector based on the size and the direction of the offset vector.

[0373]

Equation 7

[0374] offsetMV[0] = abs(offsetMV) * sign[0]

[0375] offsetMV[1] = abs(offsetMV) * sign[1]

[0376] In Equation 7, offsetMV[0] indicates the vertical direction component of the offset vector, and offsetMV[1] indicates the horizontal direction component of the offset vector.

[0377] Figure 28 is a graph showing the offset vector according to the value of distance_idx indicating the size of the offset vector and direction_idx indicating the direction of the offset vector.

[0378] As shown in the example in Figure 28 , the size and the direction of the offset vector can be determined according to the value of distance_idx and direction_idx. The maximum size of the offset vector can be set not to exceed a threshold value. Here, the threshold value can have a value predefined by the encoder and the decoder. For example, the threshold value can be 32 sample distance. Alternatively, the threshold value can be determined according to the size of the initial motion vector. For example, the threshold value of the horizontal direction can be set based on the size of the horizontal component of the initial motion vector, and the threshold value of the vertical direction can be set based on the size of the vertical component of the initial motion vector.

[0379] When the merge candidate has bi-directional motion information, the L0 motion vector of the merge candidate can be set as the L0 initial motion vector of the current block, and the L1 motion vector of the merge candidate can be set as the L1 initial motion vector of the current block. In this case, the L0 offset vector and the L1 offset vector can be determined in consideration of an output order difference between the L0 reference picture of the merge candidate and the current picture (hereinafter referred to as an L0 difference) and an output order difference between the L1 reference picture of the merge candidate and the current picture (hereinafter referred to as an L1 difference).

[0380] First, when the signs of the L0 difference and the L1 difference are the same, the L0 offset vector and the L1 offset vector can be set to be the same. On the other hand, when the signs of the L0 difference and the L1 difference are different, the L1 offset vector can be set in a direction opposite to the L0 offset vector.

[0381] The size of the L0 offset vector and the size of the L1 offset vector can be set to be the same. Alternatively, the size of the L1 offset vector can be determined by scaling the L0 offset vector based on the L0 difference and the L1 difference.

[0382] For example, Equation 8 shows the L0 offset vector and the L1 offset vector when the signs of the L0 difference and the L1 difference are the same.

[0383] [Equation 8]

[0384] offsetMVL0[0] = abs(offsetMV) * sign[0]

[0385] offsetMVL0[1] = abs(offsetMV) * sign[1]

[0386] offsetMVL1[0] = abs(offsetMV) * sign[0]

[0387] offsetMVL1[1] = abs(offsetMV) * sign[1]

[0388] In Equation 8, offsetMVL0[0] indicates a horizontal direction component of the L0 offset vector, and offsetMVL0[1] indicates a vertical direction component of the L0 offset vector. offsetMVL1[0] indicates a horizontal direction component of the L1 offset vector, and offsetMVL1[1] indicates a vertical direction component of the L1 offset vector.

[0389] Equation 9 shows the L0 offset vector and the L1 offset vector when the signs of the L0 difference and the L1 difference are different.

[0390] [Equation 9]

[0391] offsetMVL0[0] = abs(offsetMV) * sign[0]

[0392] offsetMVL0[1] = abs(offsetMV) * sign[1]

[0393] offsetMVL1[0] = -1 * abs(offsetMV) * sign[0]

[0394] offsetMVL1[1] = -1 * abs(offsetMV) * sign[1]

[0395] More than four vector direction candidates can also be defined. Tables 9 and 10 show an example of defining eight vector direction candidates.

[0396]

Table 9

[0397] direction_idx[x][y] binarization sign[x][y][0] sign[x][y][1] 0 000 +1 0 1 001 -1 0 2 010 0 +1 3 011 0 -1 4 100 +1 +1 5 101 +1 -1 6 110 -1 +1 7 111 -1 -1

[0398]

Table 10

[0399]

[0400]

[0401] In Tables 9 and 10, the absolute values of sign[0] and sign[1] greater than 0 indicate that the offset vector is in the diagonal direction. When Table 9 is used, the sizes of the x-axis component and the y-axis component of the diagonal offset vector are set to abs(offsetMV), while when Table 10 is used, the sizes of the x-axis component and the y-axis component of the diagonal offset vector are set to abs(offsetMV / 2).

[0402] Figure 29 is a diagram showing an offset vector according to the value of distance_idx indicating the size of the offset vector and direction_idx indicating the direction of the offset vector.

[0403] Figure 29 (a) is an example when Table 9 is applied, and Figure 29 (b) is an example when Table 10 is applied.

[0404] Information for determining at least one of the number or size of vector direction candidates can be transmitted via signals in the bitstream. For example, the flag `merge_offset_direction_range_flag` for determining vector direction candidates can be transmitted via signals in the bitstream. The flag can be transmitted via signals at the sequence level, image level, or strip level. For example, when the flag value is 0, the four vector direction candidates exemplified in Table 8 can be used. On the other hand, when the flag value is 1, the eight vector direction candidates exemplified in Table 9 or Table 10 can be used.

[0405] Alternatively, at least one of the number or size of vector direction candidates can be determined based on the magnitude of the offset vector. For example, when the value of the variable DistFromMergeMV, used to determine the magnitude of the offset vector, is equal to or less than a threshold, the eight vector direction candidates exemplified in Table 9 or Table 10 can be used. On the other hand, when the value of the variable DistFromMergeMV is greater than the threshold, the four vector direction candidates exemplified in Table 8 can be used.

[0406] Alternatively, at least one of the number or size of vector direction candidates can be determined based on the values ​​of the x-component MVx and the y-component MVy of the initial motion vector. For example, when the difference or absolute value of the difference between MVx and MVy is less than or equal to a threshold, the eight vector direction candidates exemplified in Table 9 or Table 10 can be used. On the other hand, when the difference or absolute value of the difference between MVx and MVy is greater than a threshold, the four vector direction candidates exemplified in Table 8 can be used.

[0407] The motion vector of the current block can be derived by adding the offset vector to the initial motion vector. Equation 10 shows an example of determining the motion vector of the current block.

[0408] Equation 10

[0409] mvL0[0]=mergeMVL0[0]+offsetMVL0[0]

[0410] mvL0[1]=mergeMVL0[1]+offsetMVL0[1]

[0411] mvL1[0]=mergeMVL1[0]+offsetMVL1[0]

[0412] mvL1[1]=mergeMVL1[1]+offsetMVL1[1]

[0413] In Equation 10, mvL0 indicates an L0 motion vector of the current block, and mvL1 indicates an L1 motion vector of the current block. mergeMVL0 indicates an L0 initial motion vector of the current block (i.e., an L0 motion vector of a merge candidate), and mergeMVL1 indicates an L1 initial motion vector of the current block. [0] indicates a horizontal direction component of a motion vector, and [1] indicates a vertical direction component of a motion vector.

[0414] The merge motion difference coding method can be applied even when inter prediction is performed on each sub-unit after the coding block is divided into the sub-units. Here, performing inter prediction in units of sub-units can include at least one of an advanced temporal motion vector prediction (ATMVP), a spatial-temporal motion vector prediction (STMVP), and a triangle partitioning technique.

[0415] For example, under the ATMVP method, an initial motion vector can be derived as follows.

[0416] First, an initial shift vector can be derived using a motion vector of a merge candidate derived from a neighboring block adjacent to the coding block. In addition, a shifted block of a sub-block included in the coding block can be derived using the initial shift vector. Equation 11 shows a position of the shifted block.

[0417] [Equation 11]

[0418] (xColSb, yColSb) = (xSb + shVector[0] » 4, ySb + shVector[1] » 4)

[0419] In Equation 11, (xColSb, yColSb) indicates a position of a top-left sample of the shifted block, and (xSb, ySb) indicates a position of a top-left sample of the sub-block. shVector indicates a shift vector.

[0420] When the shifted block is determined, a motion vector of a collocated block located at the same position as the shifted block in the collocated picture can be set as a motion vector of the sub-block. That is, a motion vector of a collocated block including a sample at the (xColSb, yColSb) position can be set as a motion vector of a sub-block including a sample at the (xSb, ySb) position.

[0421] When the triangle partitioning technique is applied, the coding block can be divided into triangular sub-units. For example, the coding block can be divided into two sub-units by a diagonal line connecting a top-left corner and a bottom-right corner of the coding block or a diagonal line connecting a top-right corner and a bottom-left corner of the coding block.

[0422] Figure 30 is a diagram showing a division pattern of a coding block when the triangle partitioning technique is applied.

[0423] Motion information of each triangular sub-unit can be designated by a merge candidate. To this end, index information indicating any one of the merge candidates can be signaled for each sub-unit. For example, index information merge_1st_idx of a first sub-unit can designate a merge candidate of the first sub-unit, and index information merge_2nd_idx of a second sub-unit can designate a merge candidate of the second sub-unit.

[0424] An initial motion vector of each sub-unit can be determined individually. For example, when an affine motion model is applied to a coding block, an affine vector of a sub-block derived from an affine seed vector of the coding block can be set as an initial motion vector of the sub-block. A motion vector of each sub-block can be derived by adding or subtracting an offset vector to the initial motion vector.

[0425] When the merge motion difference encoding method is applied to a coding block divided into a plurality of sub-units, the plurality of sub-units can be set to use the same offset vector. That is, the same offset vector can be used to change the initial motion vector of each of the plurality of sub-units.

[0426] Alternatively, the coding block can be divided into a plurality of sub-units, and an offset vector of each sub-unit can be determined individually. Accordingly, the offset vector of at least one sub-unit can be set to be different from the offset vector of another sub-unit.

[0427] Figure 31 is a diagram illustrating an example of differently setting offset vectors of each sub-unit.

[0428] As in the example shown in Figure 31 , information distance_idx indicating a size of an offset vector and direction_idx indicating a direction of the offset vector can be encoded and signaled for each sub-unit.

[0429] Alternatively, the size of the offset vector of all sub-units can be set to be the same, and the direction of the offset vector can be set individually for the sub-units. For example, a value of distance_idx signaled at an encoding level can be set to be shared among the sub-units, and direction_idx can be encoded and signaled for each sub-unit.

[0430] Alternatively, the direction of the offset vector of all sub-units can be set to be the same, and the size of the offset vector can be set individually for the sub-units. For example, a value of direction_idx signaled at an encoding level can be set to be shared among the sub-units, and distance_idx can be encoded and signaled for each sub-unit.

[0431] The merge motion difference coding method can be applied only to some of the plurality of sub-units generated by partitioning the coding block. For example, when the current block is partitioned into a first sub-unit and a second sub-unit, the motion vector of the first sub-unit can be set to be the same as the motion vector of the merge candidate, and the motion vector of the second sub-unit can be derived by adding or subtracting the offset vector to the motion vector of the merge candidate.

[0432] Instead of signaling information for determining the offset vector, the decoder can derive the offset vector. Specifically, the offset vector can be derived using an average of horizontal direction gradients and an average of vertical direction gradients of the prediction samples included in the sub-block.

[0433] Here, the gradients can be derived based on a difference between a reconstructed sample corresponding to a prediction sample in the reference picture and a neighboring sample adjacent to the reconstructed sample. For example, the horizontal direction gradient can indicate a difference between the reconstructed sample and a reconstructed sample adjacent to the left and / or right side, and the vertical direction gradient can indicate a difference between the reconstructed sample and a reconstructed sample adjacent to the upper and / or lower side.

[0434] Among the merge candidates included in the merge candidate list, a merge candidate having a motion vector derived by adding or subtracting the offset vector to a motion vector of a reference merge candidate among the merge candidates included in the merge candidate list can be added to the merge candidate list. The merge candidate having the motion vector derived by adding or subtracting the offset vector to the motion vector of the reference merge candidate can be referred to as a fine merge candidate.

[0435] The remaining motion information except for the motion vector of the fine merge candidate can be set to be the same as the motion information of the reference merge candidate.

[0436] Figure 32 is a diagram illustrating motion vector candidates that the fine merge candidate can take.

[0437] When the motion vector of the reference merge candidate is (MvLX[0], MvLX[1]), the motion vector of the fine merge candidate can be derived by adding or subtracting the offset to at least one of the x component or the y component of the motion vector of the reference merge candidate. For example, the motion vector of the fine merge candidate can be set to (MvLX[0]+M, MvLX[1]), (MvLX[0]-M, MvLX[1]), (MvLX[0], MvLX[1]+M), or (MvLX[0], MvLX[1]-M). M denotes the size of the offset vector.

[0438] The reference merge candidate can be a merge candidate having a predefined index value among the merge candidate list. For example, a merge candidate having a smallest index value among the merge candidates included in the merge candidate list, i.e., a merge candidate having an index value of 0, or a merge candidate having a largest index value can be set as the reference merge candidate. Alternatively, an inter merge candidate having a smallest index value among the inter merge candidates in the inter motion information list or an inter merge candidate having a largest index value can be set as the reference merge candidate.

[0439] Alternatively, a merge candidate having a smallest index value among the merge candidates having bi-directional motion information can be set as the reference merge candidate. That is, when the candidate blocks are sequentially searched, a bi-directional merge candidate found first can be set as the reference merge candidate.

[0440] The base merge candidate can be selected based on a size of the current block, a shape of the current block, or whether the current block is in contact with a boundary of the coding tree unit. For example, when the current block is a square shape or the current block is a non-square shape having a height greater than a width, a merge candidate having an index of 0 or a merge candidate derived from a neighboring block located above the current block can be set as the reference merge candidate. When the current block is a non-square shape having a width greater than a height, a merge candidate having an index of 1 or a merge candidate derived from a neighboring block located to the left of the current block can be set as the reference merge candidate.

[0441] Alternatively, information specifying the reference merge candidate can be signaled through a bitstream. The information can be index information specifying any one of the merge candidates included in the merge candidate list.

[0442] Information indicating whether to use the refined merge candidate can be signaled through a bitstream. The information can be a 1-bit flag. When a value of the flag is 1, the refined merge candidate generated based on the reference merge candidate can be added to the merge candidate list. On the other hand, when the value of the flag is 0, the refined merge candidate can not be included in the merge candidate list.

[0443] Alternatively, the refined merge candidate can be added to the merge candidate list when a number of the merge candidates already added to the merge candidate list is less than a maximum number of the merge candidates that the merge candidate list can include. Here, the added merge candidates can include at least one of the spatial merge candidate, the temporal merge candidate, the inter merge candidate, or the paired merge candidate. For example, the refined merge candidate can be added to the merge candidate list when a number of at least one of the spatial merge candidate, the temporal merge candidate, and the inter merge candidate included in the merge candidate list is less than or equal to a threshold value.

[0444] Alternatively, the refined merge candidate can be used when a number of the merge candidates already added to the merge candidate list is greater than or equal to a threshold value.

[0445] The maximum number of merge candidates that can be included in the merge candidate list can be set differently according to whether the fine merge candidate is used or not. For example, when set not to use the fine merge candidate, the maximum number of merge candidates that can be included in the merge candidate list can be set to N, and when set to use the fine merge candidate, the maximum number of merge candidates that can be included in the merge candidate list can be set to N+n.

[0446] The index of the fine merge candidate can be greater than the index of the merge candidate that has been added to the merge candidate list. For example, Table 11 shows an example of configuring the merge candidate list.

[0447]

Table 11

[0448] mergeCand[0] mergeCand[1] mergeCand[2] mergeCand[3] mergeCand[4] mergeCand[5] mergeCand[6]: fine merge candidate with motion vector (MvLX[0] + M, MvLX[1]) mergeCand[7]: fine merge candidate with motion vector (MvLX[0] - M, MvLX[1]) mergeCand[8]: fine merge candidate with motion vector (MvLX[0], MvLX[1] + M) mergeCand[9]: fine merge candidate with motion vector (MvLX[0], MvLX[1] - M)

[0449] In Table 11, mergeCand[X] denotes a merge candidate having an index X. MvLX[0] denotes an x-component motion vector referring to the merge candidate, and MvLX[1] denotes a y-component motion vector referring to the merge candidate. For example, when the referring merge candidate is mergeCand[0], MvLX[0] and MvLX[1] can denote the motion vector of mergeCand[0].

[0450] The size M of the offset vector can be predefined in the encoder and the decoder. For example, the size M of the offset vector can be set to an integer less than or equal to 4, such as 1 or 4.

[0451] Alternatively, information for determining the offset vector can be signaled through a bitstream. The information can be signaled at a sequence level, a picture level, a slice level, or a block level. For example, the offset vector can be determined using at least one of the information distance_idx for determining the size of the offset vector or the information direction_idx for determining the direction of the offset vector as described above.

[0452] As in the example shown in Table 11, at least one fine merge candidate derived based on the referring merge candidate can be added to the merge candidate list. If there is a merge candidate having the same motion information as the fine merge candidate among the added merge candidates, the fine merge candidate can not be added to the merge candidate list. For example, when the fine merge candidate derived based on the referring merge candidate mergeCand[0] is the same as any one of mergeCand[1] to mergeCand[5], the fine merge candidate can not be added to the merge candidate list.

[0453] Alternatively, when there is a merge candidate having the same motion information as that of the refined merge candidate, the refined merge candidate can be re-derived by changing the offset vector, or the merge candidate having the same motion information as that of the refined merge candidate can be re-set as the refined merge candidate. For example, if the motion information of the refined merge candidate mergeCand[6] derived based on the reference merge candidate mergeCand[0] is the same as that of the merge candidate mergeCand[2], the motion vector of the refined merge candidate mergeCand[6] can be changed to a value obtained by adding or subtracting the offset vector to the motion vector of the merge candidate [2]. For example, the motion vector of mergeCand[6] can be changed from (mergeCand[0]_mxLx[0]+M, mergeCand[0]_mvLx[1]) to (mergeCand[2]_mxLx[0]+M, mergeCand[2]_mvLx[1]). Here, mergeCand[X]_mvLx denotes the motion vector of the merge candidate having the index X.

[0454] As another example, an offset vector can be determined using a merge refinement offset list including at least one merge offset candidate. When a merge candidate specified by index information of a current block is a reference merge candidate, the offset vector can be determined using the merge refinement offset list. In addition, a motion vector of the current block can be derived by adding or subtracting the offset vector to a motion vector of the merge candidate. The reference merge candidate can be a merge candidate having a predefined index value among merge candidate lists. For example, a merge candidate having a smallest index value (i.e., a merge candidate having an index value of 0) or a merge candidate having a largest index value among the merge candidates included in the merge candidate list can be set as the reference merge candidate. Alternatively, an inter motion information list having a smallest index value or an inter motion information list having a largest index value among inter merge candidates can be set as the reference merge candidate.

[0455] Figure 33 is a diagram illustrating a configuration of a merge refinement offset list.

[0456] In Figure 33 In the above-described embodiment, it is assumed that the reference merge candidate is the merge candidate having the index of 6.

[0457] If the index of the merge candidate specified by merge_idx indicating any one of the merge candidates is not 6, the motion vector of the merge candidate can be set as the motion vector of the current block.

[0458] On the other hand, when the index of the merge candidate specified by the index information merge_idx is 6, the offset vector can be derived using a merge refinement offset list. The index information MrgOffset_idx that specifies any one of the merge offset candidates included in the merge refinement offset list can be signaled through the bitstream.

[0459] When the offset vector is specified, the motion vector of the current block can be derived by adding or subtracting the offset vector to the motion vector of the reference merge candidate.

[0460] The merge refinement offset list can include at least one merge offset candidate. For example, the number of the merge offset candidates included in the merge refinement offset list can be 4, 8, or 16.

[0461] Figure 34 and Figure 35 is a diagram showing the offset vector specified by the merge offset candidate.

[0462] Figure 34 illustrates an example in which the number of the merge offset candidates is 8, and Figure 35 illustrates an example in which the number of the merge offset candidates is 16.

[0463] As in the example shown in (a) of FIG. 10, Figure 34 As in the example shown in (a) of FIG. 10, Figure 35 the offset vector indicated by the merge offset candidate having an index less than the threshold value can be set such that the absolute value of the motion vector in the horizontal direction and / or the absolute value of the motion vector in the vertical direction can have a first value, and the offset vectors indicated by the other merge offset candidates can be set such that the absolute value of the motion vector in the horizontal direction and / or the absolute value of the motion vector in the vertical direction can have a second value.

[0464] As in the example shown in (a) of FIG. 10, Figure 34 As in the example shown in (b) of FIG. 10,

[0465] A plurality of reference merge candidates can be set. For example, among the merge candidates included in the merge candidate list, two merge candidates having the smallest indices can be set as the reference merge candidates. Thus, when the index of the merge candidate specified by the index information merge_idx is 0 or 1, the merge refinement offset list can be used to derive the offset vector. Alternatively, among the merge candidates included in the merge candidate list, the merge candidate having the smallest index and the merge candidate having the largest index among the merge candidates included in the inter merge candidate list can be set as the reference merge candidates.

[0466] In the advanced motion vector prediction mode, the motion vector of the current block can be derived by adding a motion difference vector to a motion prediction vector. The motion prediction vector of the current block can be determined based on a motion vector prediction candidate list including at least one motion prediction vector candidate. For example, any one of the motion prediction vector candidates can be set as the motion prediction vector of the current block.

[0467] The motion vector prediction candidate can be derived based on at least one of a spatial neighboring block of the current block or a temporal neighboring block of the current block.

[0468] Figure 36 is a diagram illustrating a candidate block for deriving a motion vector prediction candidate.

[0469] The spatial neighboring block can include an above neighboring block located above the current block and a left neighboring block located left of the current block. The above neighboring block can include one of a block B0 including a sample at a position (xCb+CbW, yCb-1), a block B1 including a sample at a position (xCb+CbW-1, yCb-1), a block B2 including a sample at a position (xCb-1, yCb-1), or a block B3 including a sample at a position (xCb, yCb-1). Wherein (xCb, yCb) denotes a position of a top-left sample of the current block, and CbW denotes a width of the current block. The left neighboring block can include one of a block A0 including a sample at a position (xCb-1, yCb+CbH), a block A1 including a sample at a position (xCb-1, yCb+CbH-1), or a block A2 including a sample at a position (xCb-1, yCb). Wherein CbH denotes a height of the current block.

[0470] The temporal neighboring block can include at least one of a block C0 including a sample located at a center of a block having the same position and size as the current block in the co-located block and a block C1 including a sample adjacent to a lower-right corner of the block.

[0471] The maximum number of motion vector prediction candidates that can be included in the motion vector prediction candidate list can be 2. The derivation order of the motion vector prediction candidates is as follows.

[0472] 1. When at least one of the left neighboring block A0 or the left neighboring block A1 is available, set the motion vector of the available block as a motion vector prediction candidate.

[0473] 2. When at least one of the above neighboring block B0, the above neighboring block B1 or the above neighboring block B2 is available, set the motion vector of the available block as a motion vector prediction candidate.

[0474] 3. When the temporal neighboring block is available, set the temporal motion vector as a motion vector prediction candidate.

[0475] 4. Set a zero motion vector as a motion vector prediction candidate.

[0476] Alternatively, when the number of motion vector prediction candidates derived in the order of 1 to 3 is less than 2, a motion vector included in the inter motion information list can be set as a motion vector prediction candidate. When the inter motion information list is available, the motion vector prediction candidates can be derived in the following order.

[0477] 1. When at least one of the left neighboring block A0 or the left neighboring block A1 is available, set the motion vector of the available block as a motion vector prediction candidate.

[0478] 2. When at least one of the above neighboring block B0, the above neighboring block B1 or the above neighboring block B2 is available, set the motion vector of the available block as a motion vector prediction candidate.

[0479] 3. When the temporal neighboring block is available, set the temporal motion vector as a motion vector prediction candidate.

[0480] 4. Set a motion vector included in the inter motion information list as a motion vector prediction candidate.

[0481] 5. Set a zero motion vector as a motion vector prediction candidate.

[0482] A motion vector prediction candidate having a motion vector derived by adding or subtracting an offset vector to or from a motion vector of a reference motion vector prediction candidate can be added to the motion vector prediction candidate list. The motion vector prediction candidate having a motion vector derived by adding or subtracting an offset vector to or from a motion vector of a reference motion vector prediction candidate can be referred to as a fine motion vector prediction candidate.

[0483] Figure 37 is a diagram illustrating a motion vector candidate that can be set as a fine motion vector prediction candidate.

[0484] When the motion vector of the reference motion vector prediction candidate is (MvPLX[0], MvPLX[1]), the motion vector of the refined motion vector prediction candidate can be derived by adding or subtracting the offset vector to or from at least one of the x component or the y component of the motion vector of the reference motion vector prediction candidate. For example, the motion vector of the refined motion vector prediction candidate can be set to (MvPLX[0]+M, MvPLX[1]), (MvPLX[0]-M, MvPLX[1]), (MvPLX[0], MvPLX[1]+M), or (MvPLX[0], MvPLX[1]-M). M denotes the size of the offset vector.

[0485] The size M of the offset vector can be predefined in the encoder and the decoder. For example, the size M of the offset vector can be set to an integer less than or equal to 4, such as 1 or 4.

[0486] Alternatively, information for determining the offset vector can be signaled through a bitstream. The information can be signaled at a sequence level, a picture level, a slice level, or a block level. For example, at least one of the information distance_idx for determining the size of the offset vector or the information direction_idx for determining the direction of the offset vector can be used to determine the offset vector as described above.

[0487] The reference motion vector prediction candidate can be a motion vector prediction candidate having a predetermined index value in a motion vector prediction candidate list. For example, among the motion vector prediction candidates included in the motion vector prediction candidate list, a motion vector prediction candidate having an index value of 0 or a motion vector prediction candidate having an index value of 1 can be set as the reference motion vector prediction candidate.

[0488] As another example, the offset vector can be determined using a merge refinement offset list including at least one prediction vector offset candidate. When a motion vector prediction candidate specified by index information of a current block is a reference motion vector prediction candidate, the offset vector can be determined using the prediction vector refinement offset list. In addition, a motion prediction vector of the current block can be derived by adding or subtracting the offset vector to or from a motion vector of the motion vector prediction candidate. The reference motion vector prediction candidate can be a motion vector prediction candidate having a predetermined index value in a motion vector prediction candidate list. For example, among the motion vector prediction candidates included in the motion vector prediction candidate list, a motion vector prediction candidate having a minimum index value or a motion vector prediction candidate having a maximum index value can be set as the reference motion vector prediction candidate.

[0489] When the offset vector is calculated using the prediction vector offset refinement list, the maximum number of prediction vector candidates that the prediction vector candidate list can include can be set to a value greater than 2.

[0490] Figure 38 is a diagram illustrating a configuration of a prediction vector refinement offset list.

[0491] In Figure 38 , it is assumed that the reference prediction vector candidate is the prediction vector candidate with index 2.

[0492] When the index of the prediction vector candidate specified by the index information AMVPcand_idx indicating any one of the prediction vector candidates is not 2, the motion vector of the prediction vector candidate can be set as the motion prediction vector of the current block.

[0493] On the other hand, when the index of the prediction vector candidate specified by the index information AMVPcand_idx is 2, an offset vector can be derived using a prediction vector refinement offset list. Index information AMVPOffset_idx specifying any one of the prediction vector offset candidates included in the prediction vector refinement offset list can be signaled through a bitstream.

[0494] When the offset vector is specified, the motion prediction vector of the current block can be derived by adding or subtracting the offset vector to the motion vector of the reference prediction vector candidate.

[0495] The motion vector refinement technique can be used even when an affine motion model is used for encoding a block. For example, when an affine advanced motion vector prediction mode is applied, an affine seed vector of the block to be encoded can be derived by adding an affine seed difference vector to an affine seed prediction vector. Here, the affine seed prediction vector can be derived based on affine seed vectors of spatial neighboring blocks or temporal neighboring blocks of the block to be encoded. The affine seed difference vector can be determined based on information signaled from a bitstream. In this case, the same affine seed difference vector can be applied to all control points. Alternatively, information for determining the affine seed vector can be signaled for each control point.

[0496] When affine vectors of sub-blocks are derived based on the affine seed vector of the block to be encoded, the affine vectors can be set as initial motion vectors, and then offset vectors can be derived. The motion vector of each sub-block can be derived by adding or subtracting the offset vector to the initial motion vector.

[0497] Instead of signaling information for determining the offset vector, the decoder can derive the offset vector. Specifically, the offset vector can be derived using an average of horizontal direction gradients and an average of vertical direction gradients of prediction samples included in the sub-block.

[0498] Intra prediction is to predict a current block using reconstructed samples of a periphery of the current block that have been encoded / decoded. In this case, the intra prediction of the current block can use reconstructed samples before an in-loop filter is applied.

[0499] Intra prediction techniques include matrix-based intra prediction and general intra prediction considering directionality with neighboring reconstructed samples. Information indicating the intra prediction technique of the current block can be signaled through a bitstream. The information can be a 1-bit flag. Alternatively, the intra prediction technique of the current block can be determined based on at least one of a position, a size, a shape of the current block, or an intra prediction technique of a neighboring block. For example, when the current block exists across an image boundary, the current block is set not to apply matrix-based intra prediction.

[0500] Matrix-based intra prediction is a method of obtaining a prediction block of a current block based on matrix multiplication between a matrix stored in an encoder and a decoder and reconstructed samples around the current block. Information for specifying any one of a plurality of stored matrices can be signaled through a bitstream. The decoder can determine a matrix for intra prediction of the current block based on the information and a size of the current block.

[0501] General intra prediction is a method of obtaining a prediction block related to a current block based on a non-angular intra prediction mode or an angular intra prediction mode.

[0502] A derived residual image can be derived by subtracting an original image from a prediction image. In this case, when the residual image is changed into a frequency domain, even if a high frequency component among frequency components is removed, a subjective quality of a video is not greatly reduced. Thus, if a value of the high frequency component is made smaller or set to 0, there is an effect of improving compression efficiency without causing a significant visual distortion. Reflecting the above characteristics, a transform can be performed on the current block to decompose the residual image into 2-dimensional frequency components. The transform can be performed using a transform technique such as a discrete cosine transform (DCT), a discrete sine transform (DST), or the like.

[0503] After the current block is transformed using the DCT or the DST, the transformed current block can be transformed again. In this case, the DCT or the DST-based transform can be defined as a primary transform, and a process of transforming a block to which the primary transform is applied again can be referred to as a secondary transform.

[0504] The primary transform can be performed using any one of a plurality of transform kernel candidates. For example, the primary transform can be performed using any one of DCT2, DCT8, or DCT7.

[0505] Different transform kernels can be used for a horizontal direction and a vertical direction. Information indicating a combination of a transform kernel for the horizontal direction and a transform kernel for the vertical direction can also be signaled through a bitstream.

[0506] The execution unit of the primary transform and the secondary transform will be different. For example, the primary transform can be performed on an 8x8 block, and the secondary transform can be performed on a sub-block of size 4x4 in the transformed 8x8 block. In this case, the transform coefficients of the remaining area on which the secondary transform is not performed can also be set to 0.

[0507] Alternatively, the primary transform can be performed on a 4x4 block, and the secondary transform can be performed on an area of size 8x8 including the transformed 4x4 block.

[0508] Information indicating whether the secondary transform is performed can be signaled through a bitstream.

[0509] The inverse transform of the secondary transform (second inverse transform) can be performed in the decoder, and the inverse transform of the primary transform (first inverse transform) can be performed on the result thereof. As a result of the execution of the second inverse transform and the first inverse transform, the residual signal of the current block can be obtained.

[0510] Quantization is used to reduce the energy of a block, and the quantization process includes a process of dividing the transform coefficient by a certain constant. The constant can be derived by a quantization parameter, and the quantization parameter can be defined as a value between 1 and 63.

[0511] If the transform and the quantization are performed in the encoder, the decoder can obtain the residual block through the inverse quantization and the inverse transform. If the prediction block and the residual block are added in the decoder, the reconstructed block of the current block can be obtained.

[0512] If the reconstructed block of the current block is obtained, the loss of information generated in the quantization and the encoding process can be reduced through in-loop filtering. The in-loop filter can include at least one of a deblocking filter, a sample adaptive offset filter (SAO), or an adaptive loop filter (ALF).

[0513] The embodiments described centering on the decoding process or the encoding process are also included in the scope of the present application. The multiple embodiments described in a predetermined order are also included in the scope of the present application in a different order from that described.

[0514] The embodiments have been described based on a series of steps or flowcharts, but this does not limit the time sequence of the invention, and can be performed simultaneously or in another order as needed. Also, in the above-described embodiments, the structural elements (for example, units, modules, etc.) constituting the block diagrams can also be respectively implemented as hardware devices or software, and a plurality of structural elements can be implemented in combination as a single hardware device or software. The embodiments can be implemented in the form of program instructions executable by various computer components and recorded in computer-readable recording media. The computer-readable recording media can individually or in combination include program instructions, data files, data structures, etc. Examples of the computer-readable recording media can include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs, DVDs, and the like, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions such as ROMs, RAMs, flash memories, and the like. The hardware devices can be configured to operate as one or more software modules to perform processes according to the present invention, and vice versa.

[0515]

Industrial Applicability

[0516] The present application can be applied to an electronic device that encodes / decodes a video.

Claims

1. A video decoding method, comprising the following steps: Determine whether to apply the merged motion difference coding method to the current block; Generate a list of candidate blocks to be merged; The merge candidates for the current block are determined based on the merge candidate list; as well as The motion vector of the current block is derived based on the merged candidate, wherein, When the merging motion difference encoding method is applied to the current block, the motion vector of the current block is derived by adding the offset vector to the motion vector derived based on the merging candidate. The magnitude of the offset vector is determined based on first index information, which indicates one of a plurality of motion offset size candidates; wherein at least one of the maximum or minimum values ​​of the motion offset size candidates is set differently according to the value of a flag indicating the range of the motion offset size candidates, or at least one of the maximum or minimum values ​​of the motion offset size candidates is set differently according to the motion vector precision of the current block.

2. The video decoding method according to claim 1, wherein, The flag is transmitted as a signal at the image level.

3. The video decoding method according to claim 1, wherein, The magnitude of the offset vector is obtained by shifting the value represented by the motion offset magnitude candidate specified by the first index information.

4. The video decoding method according to claim 1, wherein, The direction of the offset vector is determined based on the second index information, which is used to indicate one of a plurality of vector direction candidates.

5. A video encoding method, comprising the following steps: Determine whether to apply the merged motion difference coding method to the current block; Generate a list of candidate blocks to be merged; The merge candidates for the current block are determined based on the merge candidate list; as well as The motion vector of the current block is derived based on the merged candidate, wherein, When the merging motion difference encoding method is applied to the current block, the motion vector of the current block is derived by adding the offset vector to the motion vector derived based on the merging candidate. The first index information is encoded to specify a motion offset size candidate among a plurality of motion offset size candidates that indicates the size of the offset vector; wherein, a flag indicating the range of the motion offset size candidates is encoded, wherein at least one of the maximum or minimum values ​​of the motion offset size candidates is set differently according to the value of the flag, or at least one of the maximum or minimum values ​​of the motion offset size candidates is set differently according to the motion vector precision of the current block.

6. The video encoding method according to claim 5, wherein, The flag is encoded at the image level.

7. The video encoding method according to claim 5, wherein, The motion offset size candidate has a value derived by shifting the size of the offset vector.

8. The video encoding method according to claim 5, wherein, The second index information is encoded, which is used to specify a vector direction candidate that indicates the direction of the offset vector among a plurality of vector direction candidates.

9. A video decoding apparatus, comprising an inter-frame prediction unit, the inter-frame prediction unit being used for: Determine whether to apply the merged motion difference coding method to the current block; Generate a list of candidate mergers for the current block; determine the candidate mergers for the current block based on the list of candidate mergers; And derive the motion vector of the current block based on the merge candidate, wherein, When the merging motion difference encoding method is applied to the current block, the motion vector of the current block is derived by adding the offset vector to the motion vector derived based on the merging candidate. The inter-frame prediction unit is further configured to determine the size of the offset vector based on first index information, the first index information indicating one of a plurality of motion offset size candidates; the inter-frame prediction unit is further configured to set at least one of the maximum or minimum values ​​of the motion offset size candidates differently according to the value of a flag indicating the range of the motion offset size candidates, or to set at least one of the maximum or minimum values ​​of the motion offset size candidates differently according to the motion vector precision of the current block.

10. The video decoding device according to claim 9, wherein, The flag is transmitted as a signal at the image level.

11. The video decoding device according to claim 9, wherein, The magnitude of the offset vector is obtained by shifting the value represented by the motion offset magnitude candidate specified by the first index information.

12. The video decoding device according to claim 9, wherein, The inter-frame prediction unit is used to determine the direction of the offset vector based on second index information, which is used to indicate one of a plurality of vector direction candidates.

13. A video encoding apparatus, comprising an inter-frame prediction unit, the inter-frame prediction unit being used to: Determine whether to apply the merged motion difference coding method to the current block; Generate a list of candidate blocks to be merged; The merge candidates for the current block are determined based on the merge candidate list; as well as The motion vector of the current block is derived based on the merged candidate, wherein, When the merging motion difference encoding method is applied to the current block, the motion vector of the current block is derived by adding the offset vector to the motion vector derived based on the merging candidate. The inter-frame prediction unit is further configured to encode first index information, which is used to specify a motion offset size candidate among a plurality of motion offset size candidates that indicates the size of the offset vector. The inter-frame prediction unit is further configured to encode a flag indicating the range of the motion offset size candidates, wherein at least one of the maximum or minimum values ​​of the motion offset size candidates is set differently according to the value of the flag, or the inter-frame prediction unit is further configured to set at least one of the maximum or minimum values ​​of the motion offset size candidates differently according to the motion vector precision of the current block.

14. The video encoding device according to claim 13, wherein, The flag is encoded at the image level.

15. The video encoding device according to claim 13, wherein, The motion offset size candidate has a value derived by shifting the size of the offset vector.

16. The video encoding device according to claim 13, wherein, The inter-frame prediction unit is used to encode the second index information, which is used to specify a vector direction candidate that indicates the direction of the offset vector among a plurality of vector direction candidates.

17. A video decoder, comprising: processor; as well as Memory, used to store computer programs; The processor is used to execute the computer program to perform the video decoding method according to any one of claims 1 to 4.

18. A video encoder, comprising: processor; as well as Memory, used to store computer programs; The processor is configured to execute the computer program to perform the video encoding method according to any one of claims 5 to 8.

19. A computer-readable storage medium storing an executable program, which, when executed by a processor, implements the video decoding method of any one of claims 1 to 4, or the video encoding method of any one of claims 5 to 8.

Images