Video encoding / decoding method, video encoder, video decoder, video encoding device, video decoding device, and computer-readable storage medium

Abstract

To provide a method and a device for improving the prediction efficiency of a merge mode.SOLUTION: A video decoding method includes the steps for determining whether a merge offset vector encoding method is applied to the current block, generating a merge candidate list for the current block, identifying a merge candidate for the current block on the basis of the merge candidate list, and deriving a motion vector for the current block on the basis of the merge candidate.SELECTED DRAWING: None

Description

Technical Field

[0001] The present invention relates to a video encoding / decoding method, a video encoder, a video decoder, a video encoding apparatus, a video decoding apparatus, and a computer-readable storage medium.

Background Art

[0002] With the increasing size of display panels, higher-quality video services are increasingly demanded. The biggest problem with high-definition video services is that the amount of data increases significantly. To solve such problems, research on improving the video compression ratio has been actively conducted. As a typical example, in 2009, MPEG (Motion Picture Experts Group) and VCEG (Video Coding Experts Group) under ITU-T (International Telecommunication Union-Telecommunication) formed JCT-VC (Joint Collaborative Team on Video Coding). Compared with H.264 / AVC, JCT-VC proposed HEVC (High Efficiency Video Coding), a video compression standard with approximately twice the compression performance, and the standard was approved on January 25, 2013. With the rapid development of high-definition video services, the performance of HEVC is also increasingly showing its limitations.

Summary of the Invention

[0003] The present invention is defined by the independent claims. Further embodiments are defined by the dependent claims.

[0004] The video decoding method according to the present invention includes a step of determining whether a merge offset vector encoding method is applied to a current block, a step of generating a merge candidate list for the current block, a step of identifying a merge candidate for the current block based on the merge candidate list, and a step of deriving a motion vector for the current block based on the merge candidate. At this time, when the merge offset vector encoding method is applied to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector of the merge candidate. When the maximum number of merge candidates that the merge candidate list can include is plural, the merge candidate of the current block is selected based on index information decoded from a bitstream indicating any one of the merge candidates. When the maximum number is 1, the merge candidate can be determined without decoding the index information.

[0005] The video encoding method according to the present invention includes a step of determining whether a merge offset vector encoding method is applied to a current block, a step of generating a merge candidate list for the current block, a step of identifying a merge candidate for the current block based on the merge candidate list, and a step of deriving a motion vector for the current block based on the merge candidate. When the merge offset vector encoding method is applied to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector of the merge candidate. When the maximum number of merge candidates that the merge candidate list can include is plural, the index information indicating the merge candidate of the current block among the merge candidates is encoded. When the maximum number is 1, the encoding of the index information is omitted.

[0006] The coder in the video according to the present invention includes a processor and a memory. The memory is used to store a computer program. The processor, by executing the computer program stored in the memory, determines whether the merge offset vector encoding method is applicable to the current block, generates a merge candidate list for the current block, identifies a merge candidate for the current block based on the merge candidate list, and derives a motion vector for the current block based on the merge candidate. When the merge offset vector encoding method is applicable to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector of the merge candidate. When the maximum number of merge candidates that the merge candidate list can include is plural, the merge candidate for the current block is selected based on index information decoded from a bit stream indicating any one of the merge candidates. When the maximum number is 1, the merge candidate is determined without decoding the index information.

[0007] The video encoder according to the present invention includes a processor and a memory. The memory is used to store a computer program. By executing the computer program stored in the memory, the processor determines whether the merge offset vector encoding method is applicable to the current block, generates a merge candidate list for the current block, identifies a merge candidate for the current block based on the merge candidate list, and derives a motion vector for the current block based on the merge candidate. When the merge offset vector encoding method is applicable to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector of the merge candidate. When the maximum number of merge candidates that the merge candidate list can include is plural, index information indicating the merge candidate of the current block among the merge candidates is encoded. When the maximum number is 1, the encoding of the index information is omitted.

[0008] The video decoding apparatus according to the present invention includes an inter prediction unit that determines whether the merge offset vector encoding method is applicable to the current block, generates a merge candidate list for the current block, identifies a merge candidate for the current block based on the merge candidate list, and derives a motion vector for the current block based on the merge candidate. When the merge offset vector encoding method is applicable to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector of the merge candidate. When the maximum number of merge candidates that the merge candidate list can include is plural, the merge candidate of the current block is selected based on index information decoded from a bit stream indicating any one of the merge candidates. When the maximum number is 1, the merge candidate is determined without decoding the index information.

[0009] The video encoding apparatus according to the present invention determines whether a merge offset vector encoding method is applied to a current block, generates a merge candidate list for the current block, identifies a merge candidate for the current block based on the merge candidate list, and includes an inter prediction unit that derives a motion vector for the current block based on the merge candidate. When the merge offset vector encoding method is applied to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector of the merge candidate. When the maximum number of merge candidates that the merge candidate list can include is plural, index information indicating the merge candidate of the current block among the merge candidates is encoded. When the maximum number is 1, the encoding of the index information is omitted.

[0010] The computer-readable storage medium according to the present invention is used for storing a computer program, and when the computer program is executed by a processor, it executes a video decoding method or a video encoding method.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

Figure 36

Figure 37

Figure 38

[0012] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0013] The video decoding method according to the present invention includes a step of determining whether a merge offset vector encoding method is applied to a current block, a step of generating a merge candidate list for the current block, a step of specifying a merge candidate for the current block based on the merge candidate list, and a step of deriving a motion vector for the current block based on the merge candidate. At this time, when the merge offset vector encoding method is applied to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector of the merge candidate. When the maximum number of merge candidates that the merge candidate list can include is plural, the merge candidate of the current block is selected based on index information decoded from a bit stream indicating any one of the merge candidates. When the maximum number is 1, the merge candidate can be determined without decoding the index information.

[0014] In one embodiment, the magnitude of the offset vector is determined based on first index information specifying one of vector magnitude candidates.

[0015] In one embodiment, at least one of the maximum value or the minimum value of the vector magnitude candidates is set to be different based on the value of a flag indicating the numerical value of the vector magnitude candidates.

[0016] In one embodiment, the flag is signaled at the picture level.

[0017] In one embodiment, at least one of the maximum value or the minimum value of the vector magnitude candidates is set to be different according to the accuracy of the motion vector for the current block.

[0018] In one embodiment, the magnitude of the offset vector is obtained by applying a bit shift operation to the value indicated by the vector magnitude candidate specified by the first index information.

[0019] In one embodiment, the direction of the offset vector is determined based on second index information specifying any one of the vector direction candidates.

[0020] The video encoding method according to the present invention includes a step of determining whether a merge offset vector encoding method is applied to a current block, a step of generating a merge candidate list for the current block, a step of specifying a merge candidate for the current block based on the merge candidate list, and a step of deriving a motion vector for the current block based on the merge candidate. When the merge offset vector encoding method is applied to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector of the merge candidate. When the maximum number of merge candidates that the merge candidate list can include is plural, the index information indicating the merge candidate of the current block among the merge candidates is encoded, and when the maximum number is 1, the encoding of the index information is omitted.

[0021] In one embodiment, the video encoding method further includes encoding first index information for identifying a vector magnitude candidate indicating the magnitude of the offset vector among a plurality of vector magnitude candidates.

[0022] In one embodiment, the video encoding method further includes encoding a flag indicating the numerical value of the vector magnitude candidate, and at least one of the maximum value or the minimum value of the vector magnitude candidate is set to be different according to the value of the flag.

[0023] In one embodiment, the flag is encoded at the picture level.

[0024] In one embodiment, at least one of the maximum value or the minimum value of the vector magnitude candidate is set to be different according to the accuracy of the motion vector for the current block.

[0025] In one embodiment, the vector magnitude candidate has a value induced by applying a bit shift operation to the magnitude of the offset vector.

[0026] In one embodiment, the video encoding method further includes encoding second index information for identifying a vector direction candidate indicating the direction of the offset vector among a number of vector direction candidates.

[0027] The video decoder according to the present invention includes a processor and a memory. The memory is used to store a computer program. The processor, by executing the computer program stored in the memory, determines whether the merge offset vector encoding method is applicable to the current block, generates a merge candidate list for the current block, identifies a merge candidate for the current block based on the merge candidate list, and derives a motion vector for the current block based on the merge candidate. When the merge offset vector encoding method is applicable to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector of the merge candidate. When the maximum number of merge candidates that the merge candidate list can include is plural, the merge candidate of the current block is selected based on index information decoded from a bit stream indicating any one of the merge candidates. When the maximum number is 1, the merge candidate is determined without decoding the index information.

[0028] The video encoder according to the present invention includes a processor and a memory. The memory is used to store a computer program. By executing the computer program stored in the memory, the processor determines whether the merge offset vector encoding method is applicable to the current block, generates a merge candidate list for the current block, identifies a merge candidate for the current block based on the merge candidate list, and derives a motion vector for the current block based on the merge candidate. When the merge offset vector encoding method is applicable to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector of the merge candidate. When the maximum number of merge candidates that the merge candidate list can include is plural, the index information indicating the merge candidate of the current block among the merge candidates is encoded. When the maximum number is 1, the encoding of the index information is omitted.

[0029] The video decoding apparatus according to the present invention includes an inter prediction unit that determines whether the merge offset vector encoding method is applicable to the current block, generates a merge candidate list for the current block, identifies a merge candidate for the current block based on the merge candidate list, and derives a motion vector for the current block based on the merge candidate. When the merge offset vector encoding method is applicable to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector of the merge candidate. When the maximum number of merge candidates that the merge candidate list can include is plural, the merge candidate of the current block is selected based on index information decoded from a bit stream indicating any one of the merge candidates. When the maximum number is 1, the merge candidate is determined without decoding the index information.

[0030] The video encoding apparatus according to the present invention determines whether a merge offset vector encoding method is applied to a current block, generates a merge candidate list for the current block, specifies a merge candidate for the current block based on the merge candidate list, and includes an inter prediction unit that derives a motion vector for the current block based on the merge candidate. When the merge offset vector encoding method is applied to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector of the merge candidate. When the maximum number of merge candidates that the merge candidate list can include is plural, index information indicating the merge candidate of the current block among the merge candidates is encoded, and when the maximum number is 1, encoding of the index information is omitted.

[0031] The computer-readable storage medium according to the present invention is used to store a computer program, and when the computer program is executed by a processor, a video decoding method or a video encoding method is executed.

[0032] The present invention can further provide a method for refining a motion vector derived from a merge candidate based on an offset vector when encoding / decoding a video signal, and an apparatus for executing this method.

[0033] The present invention can further provide a method for signaling an offset vector and an apparatus for executing this method when encoding / decoding a video signal.

[0034] According to the present invention, the inter prediction efficiency can be improved by refining the motion vector of the merge candidate based on the offset vector.

[0035] According to the present invention, the inter-prediction efficiency can be improved by adaptively determining the magnitude and direction of the offset vector.

[0036] Video encoding and decoding are performed in block units. As an example, encoding / decoding processes such as transformation, quantization, prediction, in-loop filtering, or restoration can be executed on a coding block, a transform block, or a prediction block.

[0037] Hereinafter, a block that is the target of encoding / decoding is referred to as the "current block". As an example, based on the current encoding / decoding process step, the current block can indicate a coding block, a transform block, or a prediction block.

[0038] Furthermore, the term "unit" used in this specification represents a basic unit for executing a specific encoding / decoding process, and it can be understood that the term "block" indicates an array of samples of a predetermined size. Unless otherwise explained, "block" and "unit" can be used interchangeably. As an example, in the embodiments described later, it can be understood that a coding block and a coding unit have the same meaning.

[0039] FIG. 1 is a block diagram showing a video encoder (encoder) according to an embodiment of the present invention.

[0040] Referring to FIG. 1, the video encoding apparatus 100 can include a picture division unit 110, prediction units 120, 125, a transform unit 130, a quantization unit 135, a reordering unit 160, an entropy coding unit 165, an inverse quantization unit 140, an inverse transform unit 145, a filter unit 150, and a memory 155.

[0041] In a video encoding device, each component shown in FIG. 1 is illustrated independently to show different characteristic functions, but this does not mean that each component is composed of separate hardware or one software configuration unit. That is, for each component, for the convenience of explanation, each component included is listed, and at least two of the components are combined to form one component, or one component can be divided into multiple components to execute functions. As long as the embodiments in which these components are integrated and the separated embodiments do not deviate from the essence of the present invention, they are included in the scope of rights of the present invention.

[0042] Also, some components may not be essential components for executing the essential functions of the present invention, but can be optional components for merely improving performance. The present invention can include only the essential components necessary for realizing the essence of the present invention excluding the components used merely for improving performance, and a structure including only the necessary components excluding the optional components used merely for improving performance is also included in the scope of rights of the present invention.

[0043] The picture splitting unit 110 can split the input picture into at least one processing unit. At this time, the processing unit can be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The picture splitting unit 110 can split one picture into a combination of a plurality of coding units, prediction units, and transform units, and select a combination of one coding unit, prediction unit, and transform unit based on a predetermined criterion (for example, a cost function) to encode the picture.

[0044] For example, one picture can be divided into a plurality of coding units. To divide a picture into coding units, a recursive tree structure such as a quad tree structure can be used. Taking one video or the largest coding unit as the root, a coding unit to be divided into other coding units can be divided with the number of child nodes equal to the number of divided coding units. A coding unit that is not further divided according to certain restrictions becomes a leaf node. That is, assuming that only square division is possible for one coding unit, one coding unit can be divided into at most four other coding units.

[0045] Hereinafter, in the embodiments of the present invention, a coding unit can be used in the sense of a unit for performing encoding, and can also be used in the sense of a unit for performing decoding.

[0046] A prediction unit can be divided within one coding unit and have a shape such as at least one square or rectangle of the same size, and can also be divided such that any one of the prediction units divided within one coding unit has a different shape and / or size from other prediction units.

[0047] When generating a prediction unit for performing intra prediction based on a coding unit, if there is no minimum coding unit, intra prediction can be performed without dividing into a plurality of prediction units of N×N.

[0048] The prediction units 120 and 125 can include an inter-prediction unit 120 that performs inter-prediction and an intra-prediction unit 125 that performs intra-prediction. It is possible to determine whether to perform inter-prediction or intra-prediction on the prediction unit, and to determine specific information (such as an intra-prediction mode, a motion vector, a reference picture, etc.) according to each prediction method. At this time, the processing unit on which the prediction is performed can be different from the processing unit for which the prediction method and the specific content are determined. For example, the prediction method and the prediction mode, etc. can be determined by the prediction unit, and the execution of the prediction can also be performed by the conversion unit. The residual value (residual block) between the generated prediction block and the original block can be input to the conversion unit 130. Also, the prediction mode information, motion vector information, etc. used for prediction can be encoded by the entropy coding unit 165 together with the residual value and transferred to the decoder. When using a specific encoding mode, it is also possible to directly encode the original block without generating a prediction block by the prediction units 120 and 125 and transmit it to the decoder.

[0049] The inter-prediction unit 120 can predict the prediction unit based on the information of at least one picture among the pictures before or after the current picture, and in some cases, can also predict the prediction unit based on the information of the encoded partial region within the current picture. The inter-prediction unit 120 can include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0050] The reference picture interpolation unit can receive reference picture information from the memory 155 and generate pixel information below integer pixels in the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter with varying filter coefficients (DCT-based Interpolation Filter) can be used to generate pixel information below integer pixels in units of 1 / 4 pixels. In the case of chrominance signals, a DCT-based 4-tap interpolation filter with varying filter coefficients (DCT-based Interpolation Filter) can be used to generate pixel information below integer pixels in units of 1 / 8 pixels.

[0051] The motion prediction unit can perform motion prediction based on the reference picture interpolated by the reference picture interpolation unit. As methods for calculating motion vectors, various methods such as FBMA (Full search-based Block Matching Algorithm), TSS (Three Step Search), and NTS (New Three-Step Search Algorithm) can be used. The motion vector can have a motion vector value in units of 1 / 2 or 1 / 4 pixels based on the interpolated pixels. The motion prediction unit can change the motion prediction method to predict the current prediction unit. As motion prediction methods, various methods such as the Skip method, the Merge method, the AMVP (Advanced Motion Vector Prediction) method, and the Intra Block Copy method can be used.

[0052] The intra prediction unit 125 can generate a prediction unit based on the reference pixel information around the current block, which is the pixel information of the current picture. When the peripheral block of the current prediction unit is a block that has performed inter prediction and the reference pixel is a pixel that has performed inter prediction, the reference pixels included in the block that has performed inter prediction can be replaced with the reference pixel information of the peripheral block that has performed intra prediction and used. That is, when the reference pixel is not available, the unavailable reference pixel information can be replaced with at least one of the available reference pixels and used.

[0053] In intra prediction, the prediction mode includes a directional prediction mode that uses reference pixel information according to the prediction direction and a non-directional mode that does not use directional information when making a prediction. The mode for predicting luminance information and the mode for predicting chrominance information can be different, and in order to predict chrominance information, the intra prediction mode information used to predict luminance information or the predicted luminance signal information can be utilized.

[0054] When performing intra prediction, if the size of the prediction unit is the same as the size of the transform unit, intra prediction can be performed on the prediction unit based on the pixels existing on the left side, upper left side, and upper end of the prediction unit. However, when performing intra prediction, if the size of the prediction unit is different from the size of the transform unit, intra prediction can be performed using the reference pixels based on the transform unit. Also, intra prediction using only N×N splitting for the minimum coding unit can be used.

[0055] The intra prediction method can generate a prediction block after applying an AIS (Adaptive Intra Smoothing) filter to reference pixels according to a prediction mode. The types of AIS filters applied to the reference pixels can be different. To execute the intra prediction method, the intra prediction mode of the current prediction unit can be predicted from the intra prediction modes of prediction units existing around the current prediction unit. When predicting the prediction mode of the current prediction unit by using the mode information predicted from the surrounding prediction units, if the intra prediction modes of the current prediction unit and the surrounding prediction units are the same, information indicating that the prediction modes of the current prediction unit and the surrounding prediction units are the same can be transmitted by using predetermined flag information. If the prediction modes of the current prediction unit and the surrounding prediction units are different, entropy coding can be executed to encode the prediction mode information of the current block.

[0056] Also, a residual block including residual information, which is a difference value between a prediction unit for which prediction is executed based on the prediction unit generated by the prediction units 120 and 125 and the original block of the prediction unit, can be generated. The generated residual block can be input to the conversion unit 130.

[0057] The conversion unit 130 can convert a residual block including residual information between the original block and the prediction units generated by the prediction units 120 and 125 using a conversion method such as DCT (Discrete Cosine Transform) or DST (Discrete Sine Transform). Here, the DCT conversion core includes at least one of DCT2 or DCT8, and the DST conversion core includes DST7. Whether to apply DCT or DST to convert the residual block can be determined based on the intra prediction mode information of the prediction unit used to generate the residual block. It is also possible to skip the conversion of the residual block. A flag indicating whether to skip the conversion of the residual block can be encoded. The conversion skip can be allowed for a residual block whose size is below a threshold, a luma component, or a chroma component under the 4:4:4 format.

[0058] The quantization unit 135 can quantize the values converted into the frequency domain by the conversion unit 130. The quantization coefficient can vary according to the importance of the block or video. The values calculated by the quantization unit 135 can be provided to the inverse quantization unit 140 and the reordering unit 160.

[0059] The reordering unit 160 can perform reordering of the coefficient values for the quantized residual values.

[0060] The reordering unit 160 can change two-dimensional block form coefficients into a one-dimensional vector form by a coefficient scanning method. For example, the reordering unit 160 can scan from the DC coefficient to the coefficients in the high-frequency region using a zig-zag scan method and change them into the form of a one-dimensional vector. Based on the size of the conversion unit and the intra prediction mode, instead of the zig-zag scan, a vertical scan that scans the two-dimensional block form coefficients in the column direction or a horizontal scan that scans the two-dimensional block form coefficients in the row direction can also be used. That is, based on the size of the conversion unit and the intra prediction mode, it is possible to determine which scan method among the zig-zag scan, the vertical scan, and the horizontal scan to use.

[0061] The entropy coding unit 165 can perform entropy coding based on the value calculated by the reordering unit 160. Entropy coding can use various coding methods such as exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding).

[0062] The entropy coding unit 165 can encode various information such as the residual value coefficient information, block type information, prediction mode information, division unit information, prediction unit information, transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information of the coding unit from the reordering unit 160 and the prediction units 120, 125.

[0063] The entropy coding unit 165 can entropy-encode the coefficient values of the coding unit input from the reordering unit 160.

[0064] The inverse quantization unit 140 inverse quantizes the value quantized by the quantization unit 135, and the inverse transform unit 145 inverse transforms the value transformed by the transform unit 130. The residual values (Residual) generated from the inverse quantization unit 140 and the inverse transform unit 145 can be combined with the predicted units predicted by the motion estimation unit, the motion compensation unit, and the intra prediction unit included in the prediction units 120 and 125 to generate a reconstructed block (Reconstructed Block).

[0065] The filter unit 150 can include at least one of a deblocking filter, an offset correction unit, and an ALF (Adaptive Loop Filter).

[0066] The deblocking filter can remove the block distortion generated by the boundary between blocks in the restored picture. In order to determine whether to perform deblocking, it is possible to determine whether to apply the deblocking filter to the current block based on the pixels included in several columns or rows included in the block. When applying the deblocking filter to the block, a strong filter or a weak filter can be applied according to the required deblocking filter strength. Also, when applying the deblocking filter, when performing vertical filtering and horizontal filtering, the horizontal direction filtering and the vertical direction filtering can be made to be processed in parallel.

[0067] The offset correction unit can correct the offset with the original video in pixel units for the video that has undergone deblocking. In order to perform offset correction for a specific picture, the pixels included in the video are divided into a certain number of regions, then the region to perform the offset is determined, and a method of applying the offset to the corresponding region or a method of applying the offset considering the edge information of each pixel can be used.

[0068] Adaptive Loop Filtering (ALF) can be executed based on a value obtained by comparing a filtered reconstructed video with an original video. After dividing pixels included in a video into predetermined groups, one filter applied to a corresponding group can be determined, and filtering can be performed differentially for each group. A luminance signal, which is information related to whether or not to apply ALF, can be transmitted for each coding unit (CU), and the shape and filter coefficients of an ALF filter applied according to each block can be different. Also, an ALF filter of the same form (fixed form) can be applied regardless of the characteristics of a block to which it is applied.

[0069] Memory 155 can store a reconstructed block or picture calculated via filter unit 150, and the stored reconstructed block or picture can be provided to prediction units 120 and 125 when performing inter prediction.

[0070] FIG. 2 is a block diagram showing a video decoder (decoder) according to an embodiment of the present invention.

[0071] Referring to FIG. 2, video decoder 200 can include entropy decoding unit 210, reordering unit 215, inverse quantization unit 220, inverse transform unit 225, prediction units 230 and 235, filter unit 240, and memory 245.

[0072] When a video bitstream is input to the video decoder, the input bitstream can be decoded in a procedure reverse to that of the video encoder.

[0073] The entropy decoding unit 210 can perform entropy decoding in a procedure reverse to that of performing entropy coding in the entropy coding unit of the video encoder. For example, corresponding to the method executed by the video encoder, various methods such as Exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding) can be applied.

[0074] The entropy decoding unit 210 can decode information regarding intra prediction and inter prediction executed by the encoder.

[0075] The rearrangement unit 215 can perform rearrangement on the entropy-decoded bitstream from the entropy decoding unit 210 based on the rearrangement method executed by the encoding unit. Coefficients represented in the form of a one-dimensional vector can be restored and rearranged into coefficients in the form of a two-dimensional block. The rearrangement unit 215 receives information related to the coefficient scan executed by the encoding unit and can perform rearrangement using a method of reverse scanning based on the scan order executed by the corresponding encoding unit.

[0076] The inverse quantization unit 220 can perform inverse quantization based on the quantization parameter provided by the encoder and the coefficient values of the rearranged blocks.

[0077] The inverse transform unit 225 can perform an inverse transform on the quantization result executed by the video encoder with respect to the transform executed by the transform unit, i.e., DCT or DST, that is, it can perform an inverse DCT or inverse DST. Here, the DCT transform core can include at least one of DCT2 or DCT8, and the DST transform core can include DST7. Or when the transform is skipped in the video encoder, the inverse transform unit 225 may also not perform the inverse transform. The inverse transform can be executed based on the transmission unit determined by the video encoder. The inverse transform unit 225 of the video decoder can selectively execute a transform technique (e.g., DCT or DST) based on a plurality of information such as the prediction method, the size of the current block, and the prediction direction.

[0078] The prediction units 230, 235 can generate a prediction block based on the information related to the generation of the prediction block provided by the entropy decoding unit 210 and the previously decoded block or picture information provided by the memory 245.

[0079] As described above, similar to the operation of the video encoder, when performing intra prediction, if the size of the prediction unit is the same as the size of the transform unit, intra prediction can be performed on the prediction unit based on the pixels existing on the left side, upper left, and upper end of the prediction unit. However, when performing intra prediction, if the size of the prediction unit is different from the size of the transform unit, intra prediction can be performed using the reference pixels based on the transform unit. Also, intra prediction using only N×N division for the minimum coding unit can be used.

[0080] The prediction units 230 and 235 can include a prediction unit discrimination unit, an inter-prediction unit, and an intra-prediction unit. The prediction unit discrimination unit receives various information such as prediction unit information input from the entropy decoding unit 210, prediction mode information of the intra-prediction method, and information related to motion prediction of the inter-prediction method, divides the prediction unit in the current coding unit, and can determine whether the prediction unit executes inter-prediction or intra-prediction. The inter-prediction unit 230 uses the information necessary for inter-prediction of the current prediction unit provided by the video encoder, and based on the information included in at least one of the previous picture or the subsequent picture of the current picture including the current prediction unit, can perform inter-prediction on the current prediction unit. Or, inter-prediction can also be performed based on the information of several regions previously restored within the current picture including the current prediction unit.

[0081] To execute inter-prediction, based on the coding unit, it is possible to determine which method among the skip mode, merge mode, motion vector prediction mode (AMVP Mode), and intra-block copy mode the motion prediction method of the prediction unit included in the corresponding coding unit is.

[0082] The intra prediction unit 235 can generate a prediction block based on the pixel information within the current picture. When the prediction unit is a prediction unit that has performed intra prediction, intra prediction can be performed based on the intra prediction mode information of the prediction unit provided from the video encoder. The intra prediction unit 235 can include an AIS (Adaptive Intra Smoothing) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter, as a part that performs filtering on the reference pixels of the current block, can determine whether to apply the filter based on the prediction mode of the current prediction unit and can apply the AIS filter. Using the prediction mode of the prediction unit provided by the video encoder and the AIS filter information, AIS filtering can be performed on the reference pixels of the current block. When the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

[0083] When the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on the pixel value obtained by interpolating the reference pixels, the reference pixel interpolation unit can interpolate the reference pixels to generate reference pixels of pixel units below the integer value. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixels, the reference pixels may not be interpolated. When the prediction mode of the current block is the DC mode, the DC filter can generate a prediction block by filtering.

[0084] The restored block or picture can be provided to the filter unit 240. The filter unit 240 can include a deblocking filter, an offset correction unit, and an ALF.

[0085] Information on whether a deblocking filter has been applied to a corresponding block or picture from a video encoder can be received, and information on whether a strong filter or a weak filter has been applied when the deblocking filter is applied can also be received. The deblocking filter of a video decoder receives information related to the deblocking filter provided by the video encoder, and the video encoder can perform deblocking filter processing on the corresponding block.

[0086] When encoding, the offset correction unit can perform offset correction on the restored video based on the type of offset correction applied to the video and offset value information, etc.

[0087] ALF can be applied to a coding unit based on information on whether to apply the ALF provided by the encoder, ALF coefficient information, etc. Such ALF information can be provided included in a specific parameter set.

[0088] The memory 245 can store the restored picture or block so that it can be used as a reference picture or reference block, and can further provide the restored picture to the output unit.

[0089] FIG. 3 is a diagram showing a basic coding tree unit according to an embodiment of the present invention.

[0090] A coding block of the maximum size can be defined as a coding tree block. One picture is divided into a plurality of coding tree units (Coding Tree Unit, CTU). The coding tree unit is a coding unit of the maximum size and can also be called an LCU (Largest Coding Unit). FIG. 3 shows an example in which one picture is divided into a plurality of coding tree units.

[0091] The size of a coding tree unit can be defined at the picture level or the sequence level. For this purpose, information indicating the size of the coding tree unit can be signaled by a picture parameter set or a sequence parameter set.

[0092] As an example, the size of the coding tree unit for the entire picture in a sequence can be set to 128×128. Alternatively, either 128×128 or 256×256 at the picture level can be determined as the size of the coding tree unit. As an example, in the first picture, the size of the coding tree unit can be set to 128×128, and in the second picture, the size of the coding tree unit can be set to 256×256.

[0093] The coding tree unit can be divided to generate coding blocks. A coding block represents a basic unit for performing encoding / decoding. As an example, prediction or transformation can be performed for each coding block, or a prediction encoding mode can be determined for each coding block. Here, the prediction encoding mode indicates a method for generating a predicted video. As an example, the prediction encoding mode can include intra prediction, inter prediction, current picture referencing (CPR, or intra block copy (IBC)), or combined prediction. For a coding block, a predicted block of the coding block can be generated using at least one of the prediction encoding modes of intra prediction, inter prediction, current picture referencing, or combined prediction.

[0094] Information indicating the current block's prediction encoding mode can be signaled via a bitstream. As an example, the information can be a 1-bit flag indicating whether the prediction encoding mode is an intra mode or an inter mode. Only when the current block's prediction encoding mode is determined to be an inter mode can a current picture reference or composite prediction be used.

[0095] The current picture reference is for setting the current picture as a reference picture and obtaining a predicted block of the current block from an area within the current picture where encoding / decoding has already been completed. Here, the current picture means the picture containing the current block. Information indicating whether the current picture reference is applied to the current block can be signaled via a bitstream. As an example, the information can be a 1-bit flag. When the flag is true, the prediction encoding of the current block is determined as the current picture reference, and when the flag is false, the prediction mode of the current block can be determined as an inter prediction.

[0096] Alternatively, the current block's prediction encoding mode can be determined based on a reference picture index. As an example, when the reference picture index points to the current picture, the current block's prediction encoding mode can be determined as the current picture reference. When the reference picture index points to a picture other than the current picture, the current block's prediction encoding mode can be determined as an inter prediction. That is, the current picture reference is a prediction method that utilizes information from an area within the current picture where encoding / decoding has been completed, and inter prediction is a prediction method that utilizes information from other pictures where encoding / decoding has been completed.

[0097] Compound prediction indicates an encoding mode that combines two or more of intra prediction, inter prediction, and current picture reference. As an example, when compound prediction is applied, a first prediction block can be generated based on any one of intra prediction, inter prediction, or current picture reference, and a second prediction block can be generated based on another one of them. When the first prediction block and the second prediction block are generated, a final prediction block can be generated through an average operation or a weighted sum operation of the first prediction block and the second prediction block. Information indicating whether compound prediction is applied can be signaled through a bitstream. The information can be a 1-bit flag.

[0098] Figure 4 is a diagram showing various division forms of a coding block.

[0099] A coding block can be divided into a plurality of coding blocks based on quadtree division, binary tree division, or triple tree division. The divided coding blocks can also be divided into a plurality of coding blocks again based on quadtree division, binary tree division, or triple tree division.

[0100] Quadtree division indicates a division method that divides the current block into four blocks. As a result of quadtree division, the current block can be divided into four square partitions (see "SPLIT_QT" in (a) of Figure 4).

[0101] Binary tree splitting refers to a splitting method that divides the current block into two blocks. Splitting the current block into two blocks along the vertical direction (i.e., using a vertical line that crosses the current block) can be called vertical binary tree splitting, and splitting the current block into two blocks along the horizontal direction (i.e., using a horizontal line that crosses the current block) can be called horizontal binary tree splitting. As a result of the binary tree splitting, the current block can be divided into two non-square partitions. (b) "SPLIT_BT_VER" in Figure 4 shows the vertical binary tree splitting result, and (c) "SPLIT_BT_HOR" in Figure 4 shows the horizontal binary tree splitting result.

[0102] Triple tree splitting refers to a splitting method that divides the current block into three blocks. Splitting the current block into three blocks along the vertical direction (i.e., using two vertical lines that cross the current block) can be called vertical triple tree splitting, and splitting the current block into three blocks along the horizontal direction (i.e., using two horizontal lines that cross the current block) can be called horizontal triple tree splitting. As a result of the triple tree splitting, the current block can be divided into three non-square partitions. At this time, the width / height of the partition located in the center of the current block can be twice the width / height of the other partitions. (d) "SPLIT_TT_VER" in Figure 4 shows the vertical triple tree splitting result, and (e) "SPLIT_TT_HOR" in Figure 4 shows the horizontal triple tree splitting result.

[0103] The number of times a coding tree unit is split can be defined as the partitioning depth. The maximum partitioning depth of a coding tree unit can be determined at the sequence or picture level. Thus, the maximum partitioning depth of a coding tree unit may vary for each sequence or picture.

[0104] Alternatively, the maximum split depth of each splitting method can be determined individually. As an example, the maximum split depth permitted for quad tree splitting may be different from the maximum split depth permitted for binary tree splitting and / or triple tree splitting.

[0105] The encoder can signal, via a bitstream, information indicating at least one of the split type or split depth of the current block. The decoder can determine the split type and split depth of the coding tree unit based on the information parsed from the bitstream.

[0106] FIG. 5 is a diagram showing a split pattern of a coding tree unit.

[0107] The use of splitting methods such as quad tree splitting, binary tree splitting and / or triple tree splitting to split a coding block can be referred to as multi-tree partitioning.

[0108] The coding block generated by applying multi-tree partitioning to a coding block can be referred to as a sub-coding block. When the split depth of the coding block is k, the split depth of the sub-coding block is set to k + 1.

[0109] Conversely, for a coding block with a split depth of k + 1, the code block with a split depth of k can be referred to as a super-coding block.

[0110] The current coding block splitting type can be determined based on at least one of the splitting type of the upper coding block or the splitting type of the adjacent coding block. Here, the adjacent coding block is adjacent to the current coding block and can include at least one of the upper adjacent block, the left adjacent block, or the adjacent block adjacent to the upper left corner of the current coding block. Here, the splitting type can include at least one of whether to perform quadtree splitting, whether to perform binary tree splitting, the binary tree splitting direction, whether to perform triple tree splitting, or the triple tree splitting direction.

[0111] To determine the splitting form of the coding block, the information indicating whether the coding block is split can be signaled via a bitstream. The said information is a 1-bit flag "split_cu_flag", and when the said flag is true, it indicates that the coding block is split by the multi-tree splitting technique.

[0112] When split_cu_flag is true, the information indicating whether the coding block is quadtree split can be signaled via a bitstream. The said information is a 1-bit flag split_qt_flag, and when the said flag is true, the coding block can be split into 4 blocks.

[0113] As an example, in the example shown in FIG. 5, it is shown that as the coding tree unit is quadtree split, 4 coded blocks with a split depth of 1 are generated. Also, as a quadtree split result, it is shown that quadtree split is applied again to the first coding block and the fourth coding block among the 4 generated coding blocks. As a result, 4 coding blocks with a split depth of 2 can be generated.

[0114] Also, by applying quad-tree splitting to a coding block with a splitting depth of 2 again, a coding block with a splitting depth of 3 can be generated.

[0115] When quad-tree splitting is not applied to a coding block, it is possible to determine whether to perform binary-tree splitting or triple-tree splitting on the coding block in consideration of at least one of the size of the coding block, whether the coding block is located at the picture boundary, the maximum splitting depth, or the splitting form of adjacent blocks. When it is determined that binary-tree splitting or triple-tree splitting is to be performed on the coding block, information indicating the splitting direction can be signaled via a bitstream. The information can be a 1-bit flag mtt_split_cu_vertical_flag. Based on the flag, it is possible to determine whether the splitting direction is vertical or horizontal. In addition, information indicating which of binary-tree splitting or triple-tree splitting is applied to the coding block can be signaled via a bitstream. The information can be a 1-bit flag mtt_split_cu_binary_flag. Based on the flag, it is possible to determine whether binary-tree splitting or triple-tree splitting is applied to the coding block.

[0116] As an example, in the example illustrated in FIG. 5, vertical binary-tree splitting is applied to a coding block with a splitting depth of 1, and in the coding block generated by the splitting result, it is illustrated that vertical triple-tree splitting is applied to the left coding block and vertical binary-tree splitting is applied to the right coding block.

[0117] In an apparatus for encoding or decoding video, problems that are difficult to process occur in regions larger than the threshold of hardware performance. As an example, when the hardware can process a maximum of 4096 samples simultaneously, in a region having more than 4096 samples, problems such as the need to repeatedly access and process 64x64 size data units and the inability to process data simultaneously can occur. Thus, the basic unit of data processing can be defined as a pipeline-based basic data unit (Virtual Processing Data Unit, VPDU, hereinafter referred to as the basic data unit).

[0118] The basic data unit can be classified into square, non-square or rectangular types.

[0119] FIG. 6 is a diagram showing the form of the basic unit of data.

[0120] The basic data unit can contain the same number or a smaller number of samples as the maximum number of samples that can be processed simultaneously. As an example, as shown in the example illustrated in FIG. 6(a), a 64x64 size square block can be set as the basic data unit. Or, a non-square block can be set as the basic data unit. As an example, as shown in the example illustrated in FIG. 6(b) or FIG. 6(c), a 32x128 size block or a 64x32 size block can be set as the basic data unit.

[0121] Although not shown, triangular, L-shaped (L-shaped) or polygonal basic data units can also be defined.

[0122] Information for determining a data basic unit can be signaled via a bitstream. The information can be for determining at least one of the size or shape of the data basic unit. Based on the information, it can be determined whether to allow a non-square data basic unit or whether to allow a non-rectangular data basic unit.

[0123] Alternatively, or, the encoder and decoder can pre-define at least one of the size or shape of the data basic unit.

[0124] Whether to permit the division form of the coding block can be determined in consideration of the size of the data basic unit. As an example, if the coding block generated as a division result of the coding block is larger than the data basic unit, the division cannot be permitted. Or, if the non-square coding block generated as a division result of the coding block is larger than the data basic unit, the division cannot be permitted. As an example, if the width or height of the coding block is larger than a threshold value, or if the number of samples included in the coding block is larger than a threshold value, binary tree division or triple tree division cannot be permitted. Thereby, the encoding of information regarding binary tree division or triple tree division can be omitted.

[0125] Alternatively, it can be set to always divide a coding block larger than the data basic unit. Or, for a coding block larger than the data basic unit, it can be set to always perform binary tree division or triple tree division. Thereby, for a coding block larger than the data basic unit, even if a flag split_flag indicating whether to divide the coding block is not encoded, the value of the flag can be induced to 1.

[0126] As another example, a coding block larger than a data basic unit can be divided into a plurality of sub-blocks. Here, the sub-blocks can be set as prediction units that are basic units for prediction or conversion units that are basic units for conversion and / or quantization. At this time, dividing a coding block into a plurality of prediction units can be defined as VPDU prediction unit partitioning, and dividing a coding block into a plurality of conversion units can be defined as VPDU conversion unit partitioning.

[0127] At least one of VPDU prediction unit partitioning or VPDU conversion unit partitioning can be applied to a coding block. The division type of the coding block by the application of VPDU prediction unit partitioning can be set to be the same as the division type of the coding block by the application of VPDU conversion unit partitioning.

[0128] When only VPDU prediction unit partitioning is applied to a coding block, prediction is performed for each sub-block, but conversion and / or quantization can be executed on the coding block. At this time, a prediction mode such as a prediction encoding mode, an intra prediction mode, or an inter prediction mode can be determined for the coding block.

[0129] When only VPDU conversion unit partitioning is applied to a coding block, prediction is performed on the sub-blocks, but conversion and / or quantization can be executed for each sub-block.

[0130] FIG. 7 and FIG. 8 are diagrams showing examples in which a coding block is divided into a plurality of sub-blocks.

[0131] FIG. 7 is a diagram showing a splitting aspect when only square data basic units are permitted, and FIG. 8 is a diagram showing a splitting aspect when both square data basic units and non-square data basic units are permitted.

[0132] Assuming that only square data basic units are permitted, in FIGS. 7(a) and 7(b), CU0 and CU3 are defined by two different VPDUs, and CU1 is defined by four different VPDUs. Thus, CU0 and CU3 can be split into two sub-blocks, and CU1 can be split into four sub-blocks.

[0133] Assuming that both square data basic units and non-square data basic units are permitted, in FIGS. 8(a) and 8(b), CU0 and CU3 can be defined by one VPDU, while CU1 can be defined using two different VPDUs. Thus, CU0 and CU3 are not split into sub-blocks, but CU1 can be split into two sub-blocks.

[0134] At this time, CU1 can be split into square sub-blocks or non-square sub-blocks. As an example, based on the horizontal line that splits CU1 vertically, CU1 can be split into two square sub-blocks. Or based on the vertical line that splits CU1 horizontally, CU1 can be split into two non-square sub-blocks.

[0135] When there are multiple candidate splitting types applicable to a coding block, information indicating any one of the multiple candidate splitting types can be signaled via a bitstream. As an example, the information can indicate whether the coding block is split into square sub-blocks or whether the coding block is split into non-square sub-blocks.

[0136] Alternatively, it can be set such that dividing a coding block into square sub-blocks has a higher priority than dividing the coding block into non-square sub-blocks. For example, dividing a coding block into non-square sub-blocks can be permitted when the coding block cannot be divided into square sub-blocks.

[0137] Alternatively, based on the division form of the coding block which is the parent node, the division form of the coding block can be determined. As an example, when the coding block which is the parent node is divided based on a triple tree, it can be set such that the coding block is divided into square sub-blocks. On the other hand, when the coding block which is the parent node is divided based on a binary tree or a triple tree, it can be set such that the coding block is divided into non-square sub-blocks.

[0138] Inter prediction is a predictive encoding mode that uses information from a previous picture to predict the current block. As an example, a block at the same position as the current block in the previous picture (hereinafter, a collocated block) can be set as the prediction block of the current block. Hereinafter, the prediction block generated based on the block at the same position as the current block will be referred to as a collocated prediction block.

[0139] On the other hand, when an object existing in the previous picture has moved to another position in the current picture, the movement of the object can be used to effectively predict the current block. For example, if the movement direction and size of the object can be known by comparing the previous picture and the current picture, a prediction block (or, predicted video) of the current block can be generated in consideration of the movement information of the object. Hereinafter, the prediction block generated using the movement information can be referred to as a motion prediction block.

[0140] The predicted block can be differentiated from the current block to generate a residual block. At this time, if there is movement of the object, by using a motion prediction block instead of a collocated prediction block, the energy of the residual block can be reduced, and thus the compression performance of the residual block can be improved.

[0141] As described above, generating a prediction block using motion information can be referred to as motion compensation prediction. In most inter-predictions, a prediction block can be generated based on motion compensation prediction.

[0142] The motion information can include at least one of a motion vector, a reference picture index, a prediction direction, or a bi-directional weight index. The motion vector indicates the moving direction and size of the object. The reference picture index identifies the reference picture of the current block among the reference pictures included in the reference picture list. The prediction direction refers to any one of unidirectional L0 prediction, unidirectional L1 prediction, or bi-directional prediction (L0 prediction and L1 prediction). Depending on the prediction direction of the current block, at least one of the motion information in the L0 direction or the motion information in the L1 direction can be used. The bi-directional weight index identifies the weight applied to the L0 prediction block and the weight applied to the L1 prediction block.

[0143] FIG. 9 is a flowchart of an inter-prediction method according to an embodiment of the present invention.

[0144] Referring to FIG. 9, the inter-prediction method includes a step of determining an inter-prediction mode of the current block (S901), a step of obtaining motion information of the current block according to the determined inter-prediction mode (S902), and a step of performing motion compensation prediction of the current block based on the obtained motion information (S903).

[0145] Here, the inter-prediction mode indicates various methods for determining the motion information of the current block, and can include an inter-prediction mode that utilizes translational motion information and an inter-prediction mode that utilizes affine motion information. As an example, the inter-prediction mode that utilizes translational motion information can include a merge mode and a motion vector prediction mode, and the inter-prediction mode that utilizes affine motion information can include an affine merge mode and an affine motion vector prediction mode. The motion information of the current block can be determined based on information analyzed from adjacent blocks adjacent to the current block or from the bitstream according to the inter-prediction mode.

[0146] Hereinafter, the inter-prediction method that utilizes affine motion information will be described in detail.

[0147] FIG. 10 is a diagram showing the non-linear motion of an object.

[0148] Non-linear motion of an object can occur in a video. As an example, as shown in FIG. 10, non-linear motion of an object such as camera zoom-in, zoom-out, rotation, or affine transformation can occur. When non-linear motion of an object occurs, the translational motion vector cannot effectively represent the motion of the object. Therefore, for the part where non-linear motion of the object occurs, the affine motion can be utilized instead of the translational motion to improve the encoding efficiency.

[0149] FIG. 11 is a flowchart of an inter-prediction method based on affine motion according to an embodiment of the present invention.

[0150] Whether an inter prediction method based on affine motion is applied to the current block can be determined based on the information analyzed from the bitstream. Specifically, based on at least one of a flag indicating whether an affine merge mode is applied to the current block or a flag indicating whether an affine motion vector prediction mode is applied to the current block, it can be determined whether an inter prediction method based on affine motion is applied to the current block.

[0151] When an inter prediction method based on affine motion is applied to the current block, an affine motion model of the current block can be determined (S1101). The affine motion model can be determined as at least one of a six-parameter affine motion model or a four-parameter affine motion model. The six-parameter affine motion model represents affine motion using six parameters, and the four-parameter affine motion model represents affine motion using four parameters.

[0152] Equation (1) represents affine motion using six parameters. Affine motion indicates the translational motion of a predetermined region determined by an affine seed vector.

[0153] [Equation 1]

[0154] [Number]

[0155] When representing affine motion using six parameters, complex motions can be represented, but the number of bits required for encoding each parameter increases, and the encoding efficiency can decrease. Thus, it is also possible to represent affine motion using four parameters. Mathematical formula 2 represents affine motion using four parameters.

[0156] [Formula 2]

[0157] [Number]

[0158] Information for determining the affine motion model of the current block can be encoded and signaled via a bitstream. As an example, the information can be a 1-bit flag "affine_type_flag". A value of 0 for the flag indicates that a 4-parameter affine motion model is applied, and a value of 1 for the flag indicates that a 6-parameter affine motion model is applied. The flag can be encoded on a per-slice, per-tile, or per-block (e.g., coding block or coding tree unit) basis. When the flag is signaled at the slice level, the affine motion model determined at the slice level can be applied to all blocks belonging to the slice.

[0159] Alternatively, based on the affine inter-prediction mode of the current block, the affine motion model of the current block can be determined. As an example, when the affine merge mode is applied, the affine motion model of the current block can be determined as a 4-parameter motion model. On the other hand, when the affine motion vector prediction mode is applied, information for determining the affine motion model of the current block can be encoded and signaled via a bitstream. As an example, when the affine motion vector prediction mode of the current block is applied, the affine motion model of the current block can be determined based on a 1-bit flag "affine_type_flag".

[0160] Next, an affine seed vector of the current block can be derived (S1102). When a 4-parameter affine motion model is selected, motion vectors at two control points of the current block can be derived. On the other hand, when a 6-parameter affine motion model is selected, motion vectors at three control points of the current block can be derived. The motion vectors at the control points can be referred to as affine seed vectors. The control points can include at least one of the upper left corner, upper right corner, or lower left corner of the current block.

[0161] FIG. 12 is a diagram illustrating the affine seed vectors of each affine motion model.

[0162] In the parameter affine motion model, two affine seed vectors among the upper left corner, upper right corner, or lower left corner can be derived. As an example, as shown in FIG. 12(a), when a 4-parameter affine motion model is selected, an affine vector can be derived using the affine seed vector sv0 at the upper left corner of the current block (e.g., the upper left sample (x1, y1)) and the affine seed vector sv1 at the upper right corner of the current block (e.g., the upper right sample (x1, y1)). Instead of the affine seed vector at the upper left corner, the affine seed vector at the lower left corner can be used, or instead of the affine seed vector at the upper right corner, the affine seed vector at the lower left corner can be used.

[0163] In the parameter affine motion model, the affine seed vectors of the upper left corner, upper right corner, and lower left corner can be derived. As an example, as shown in FIG. 12(b), when a 6-parameter affine motion model is selected, the affine seed vector sv0 of the upper left corner of the current block (for example, the upper left sample (x1, y1)), the affine seed vector sv1 of the upper right corner of the current block (for example, the upper right sample (x1, y1)), and the affine seed vector sv2 of the upper left corner of the current block (for example, the upper left sample (x2, y2)) can be used to derive the affine vector.

[0164] In the embodiments described below, in the 4-parameter affine motion model, the affine seed vectors of the upper left control point and the upper right control point are respectively referred to as the first affine seed vector and the second affine seed vector. In the embodiments using the first affine seed vector and the second affine seed vector described below, at least one of the first affine seed vector and the second affine seed vector can be replaced with the affine seed vector of the lower left control point (the third affine seed vector) or the affine seed vector of the lower right control point (the fourth affine seed vector).

[0165] Also, in the 6-parameter affine motion model, the affine seed vectors of the upper left control point, the upper right control point, and the lower left control point are respectively referred to as the first affine seed vector, the second affine seed vector, and the third affine seed vector. In the embodiments using the first affine seed vector, the second affine seed vector, and the third affine seed vector described below, at least one of the first affine seed vector, the second affine seed vector, and the third affine seed vector can be replaced with the affine seed vector of the lower right control point (the fourth affine seed vector).

[0166] An affinity vector of each sub-block can be derived using an affinity seed vector (S1103). Here, the affinity vector indicates a translational motion vector derived based on the affinity seed vector. The affinity vector of the sub-block can be referred to as an affinity sub-block motion vector or a sub-block motion vector.

[0167] FIG. 13 is a diagram illustrating the affinity vector of a sub-block in a four-parameter motion model.

[0168] The affinity vector of the sub-block can be derived based on the position of the control point, the position of the sub-block, and the affinity seed vector. As an example, Equation 3 shows an example of deriving an affinity sub-block vector.

[0169] [Equation 3]

[0170] [Number]

[0171] In Equation 3 above, (x, y) indicates the position of the sub-block. Here, the position of the sub-block indicates the position of the reference sample included in the sub-block. The reference sample can be a sample located at the upper left corner of the sub-block, or a sample in which at least one of the x-axis coordinate or the y-axis coordinate is at the central position. (x0, y0) indicates the position of the first control point, and (sv0x, sv0y) indicates the first affinity seed vector. Note that (x1, y1) indicates the position of the second control point, and (sv1x, sv1y) indicates the second affinity seed vector.

[0172] When the first control point and the second control point respectively correspond to the upper left corner and the upper right corner of the current block, x1 - x0 can be set to the same value as the width of the current block.

[0173] After that, motion compensation prediction for each sub-block can be performed using the affine vector of each sub-block (S1104). As a result of the execution of the motion compensation prediction, a prediction block for each sub-block can be generated. The prediction block of the sub-block can be set as the prediction block of the current block.

[0174] Next, an inter prediction method using translational motion information will be described in detail.

[0175] The motion information of the current block can be derived from the motion information of other blocks other than the current block. Here, the other block can be a block encoded / decoded by inter prediction before the current block. Defining the setting of the motion information of the current block to be the same as the motion information of the other block as the merge mode. Also, defining the setting of the motion vector of the other block to the predicted value of the motion vector of the current block as the motion vector prediction mode.

[0176] FIG. 14 is a flowchart of a process of deriving the motion information of the current block in the merge mode.

[0177] A merge candidate for the current block can be derived (S1401). The merge candidate for the current block can be derived from a block encoded / decoded by inter prediction before the current block.

[0178] FIG. 15 is a diagram illustrating candidate blocks used to derive merge candidates.

[0179] The candidate block can include at least one of an adjacent block including samples adjacent to the current block or a non-adjacent block including samples not adjacent to the current block. Hereinafter, a sample for determining the candidate block is defined as a reference sample. Note that a reference sample adjacent to the current block is referred to as an adjacent reference sample, and a reference sample not adjacent to the current block is referred to as a non-adjacent reference sample.

[0180] The adjacent reference sample can be included in an adjacent column of the leftmost column of the current block or an adjacent row of the uppermost row of the current block. As an example, when the coordinates of the upper left sample of the current block are (0, 0), at least one of a block including a reference sample at the position of (-1, H-1), a block including a reference sample at the position of (W-1, -1), a block including a reference sample at the position of (W, -1), a block including a reference sample at the position of (-1, H), or a block including a reference sample at the position of (-1, -1) can be used as the candidate block. Referring to the drawings, adjacent blocks with indexes 0 to 4 can be used as candidate blocks.

[0181] The non-adjacent reference sample indicates a sample in which at least one of the distances from the reference sample adjacent to the current block to the x-axis or the y-axis has a defined value. As an example, at least one of a block including a reference sample in which the x-axis distance from the left reference sample has a defined value, a block including a non-adjacent sample in which the y-axis distance from the upper reference sample has a defined value, or a block including a non-adjacent sample in which the x-axis distance and the y-axis distance from the upper left reference sample have defined values can be used as the candidate block. The defined value can be a natural number such as 4, 8, 12, 16, etc. Referring to the drawings, at least one of the blocks with indexes 5 to 26 can be used as the candidate block.

[0182] Samples that are not located on the same vertical line, horizontal line, or diagonal line as the adjacent reference sample can also be set as non-adjacent reference samples.

[0183] Figure 16 is a diagram showing the positions of the reference samples.

[0184] As in the example shown in Figure 16, the x coordinate of the upper non-adjacent reference sample can be set to be different from the x coordinate of the upper adjacent reference sample. As an example, when the position of the upper adjacent reference sample is (W - 1, -1), the position of the upper non-adjacent reference sample that is N away from the upper adjacent reference sample toward the y-axis can be set to ((W / 2) - 1, -1 - N), and the position of the upper non-adjacent reference sample that is 2N away from the upper adjacent reference sample toward the y-axis can be set to (0, -1 - 2N). That is, the position of the non-adjacent reference sample can be determined based on the position of the adjacent reference sample and the distance from the adjacent reference sample.

[0185] Hereinafter, a candidate block that includes an adjacent reference sample among the candidate blocks will be referred to as an adjacent block, and a block that includes a non-adjacent reference sample will be referred to as a non-adjacent block.

[0186] When the distance between the current block and the candidate block is equal to or greater than a threshold value, the candidate block can be set so that it cannot be used as a merge candidate. The threshold value can be determined based on the size of the coding tree unit. As an example, the threshold value can be set to the height of the coding tree unit (ctu_height) or a value obtained by adding or subtracting an offset from the height of the coding tree unit (for example, ctu_height ± N). The offset N is a value already defined in the encoder and the decoder, and can be set to 4, 8, 16, 32, or ctu_height.

[0187] When the difference between the y-axis coordinate of the current block and the y-axis coordinate of the sample included in the candidate block is greater than the threshold value, the candidate block can be determined so that it cannot be used as a merge candidate.

[0188] Alternatively, candidate blocks that do not belong to the same coding tree unit as the current block can be set not to be available as merge candidates. As an example, when the reference sample exceeds the upper boundary of the coding tree unit to which the current block belongs, candidate blocks including the reference sample can be set not to be available as merge candidates.

[0189] If the upper boundary of the current block is adjacent to the upper boundary of the coding tree unit, it can be determined that a large number of candidate blocks are not available as merge candidates, and the encoding / decoding efficiency of the current block can be reduced. To solve such problems, candidate blocks can be set such that the number of candidate blocks located on the left side of the current block is larger than the number of candidate blocks located at the upper end of the current block.

[0190] FIG. 17 is a diagram illustrating candidate blocks used to induce merge candidates.

[0191] As shown in the example shown in FIG. 17, the upper end blocks belonging to the upper end N block columns of the current block and the left side blocks belonging to the left side M block columns of the current block can be set as candidate blocks. At this time, M can be set to be even larger than N, and the number of left side candidate blocks can be set to be even larger than the number of upper end candidate blocks.

[0192] As an example, it can be set such that the difference between the y-axis coordinate of the reference sample in the current block and the y-axis coordinate of the upper end block available as a candidate block does not exceed N times the height of the current block. Also, it can be set such that the difference between the x-axis coordinate of the reference sample in the current block and the x-axis coordinate of the left side block available as a candidate block does not exceed M times the width of the current block.

[0193] As an example, in the example shown in FIG. 17, it is illustrated that the blocks belonging to the two block columns at the upper end of the current block and the blocks belonging to the five block columns on the left side of the current block are set as candidate blocks.

[0194] As another example, when the candidate block and the current block do not belong to the same coding tree unit, instead of the candidate block, a block belonging to the same coding tree unit as the current block or a block including a reference sample adjacent to the boundary of the coding tree unit can be used to derive a merge candidate.

[0195] FIG. 18 is a diagram showing an example in which the position of the reference sample is changed.

[0196] When the reference sample is included in a coding tree unit different from the current block and the reference sample is not adjacent to the boundary of the coding tree unit, instead of the reference sample, a reference sample adjacent to the boundary of the coding tree unit can be used to determine a candidate block.

[0197] As an example, in the examples shown in FIGS. 18(a) and (b), when the upper end boundary of the current block is in contact with the upper end boundary of the coding tree unit, the reference sample at the upper end of the current block will belong to a coding tree unit different from the current block. Among the reference samples belonging to a coding tree unit different from the current block, a reference sample not adjacent to the upper end boundary of the coding tree unit can be replaced with a sample adjacent to the upper end boundary of the coding tree unit.

[0198] As an example, as shown in the example of FIG. 18(a), the reference sample at the 6 position can be replaced with the sample at the 6' position located at the upper boundary of the coding tree unit, and as shown in the example of FIG. 18(b), the reference sample at the 15 position can be replaced with the sample at the 15' position located at the upper boundary of the coding tree unit. At this time, the y coordinate of the alternative sample can be changed to an adjacent position of the coding tree unit, and the x coordinate of the alternative sample can be set in the same manner as the reference sample. As an example, the sample at the 6' position can have the same x coordinate as the sample at the 6 position, and the sample at the 15' position can have the same x coordinate as the sample at the 15 position.

[0199] Alternatively, a value obtained by adding or subtracting an offset from the x coordinate of the reference sample can be set as the x coordinate of the alternative sample. As an example, when the x coordinates of the adjacent reference sample and the non-adjacent reference sample located at the upper end of the current block are the same, a value obtained by adding or subtracting an offset from the x coordinate of the reference sample can be set as the x coordinate of the alternative sample. This is to prevent the alternative sample replacing the non-adjacent reference sample from being in the same position as another non-adjacent reference sample or the adjacent reference sample.

[0200] FIG. 19 is a diagram showing an example in which the position of the reference sample is changed.

[0201] When replacing a reference sample included in a coding tree unit different from the current block and not adjacent to the boundary of the coding tree unit with a sample located at the boundary of the coding tree unit, a value obtained by adding or subtracting an offset from the x coordinate of the reference sample can be set as the x coordinate of the alternative sample.

[0202] As an example, in the example shown in FIG. 19, the reference sample at position 6 and the reference sample at position 15 can be replaced with the sample at position 6' and the sample at position 15' respectively, where the y - coordinates of these samples are the same as those of the rows adjacent to the upper - end boundary of the coding tree unit. At this time, the x - coordinate of the sample at position 6' can be set to the value obtained by subtracting W / 2 from the x - coordinate of the reference sample at position 6, and the x - coordinate of the sample at position 15' can be set to the value obtained by subtracting W - 1 from the x - coordinate of the reference sample at position 15.

[0203] Different from the examples shown in FIGS. 18 and 19, the y - coordinate of the row located at the upper end of the uppermost row of the current block or the y - coordinate of the upper - end boundary of the coding tree unit can also be set as the y - coordinate of the alternative sample.

[0204] Although not shown, it is also possible to determine a sample for replacing the reference sample with reference to the left - hand boundary of the coding tree unit. As an example, when the reference sample is not included in the same coding tree unit as the current block and is not adjacent to the left - hand boundary of the coding tree unit, the reference sample can be replaced with a sample adjacent to the left - hand boundary of the coding tree unit. At this time, the alternative sample can have the same y - coordinate as the reference sample, or a y - coordinate obtained by adding or subtracting an offset from the y - coordinate of the reference sample.

[0205] Thereafter, the block including the alternative sample is set as a candidate block, and based on the candidate block, the merge candidate of the current block can be derived.

[0206] It is also possible to derive a merge candidate from temporally adjacent blocks included in a picture different from the current block. As an example, a merge candidate can be derived from co - located blocks included in a co - located picture.

[0207] The motion information of the merge candidate can be set in the same way as the motion information of the candidate block. As an example, at least one of the motion vector, reference picture index, prediction direction, or bi-directional weight index of the candidate block can be set as the motion information of the merge candidate.

[0208] A merge candidate list including the merge candidate can be generated (S1402). The merge candidate can be classified into an adjacent merge candidate derived from an adjacent block adjacent to the current block and a non-adjacent merge candidate derived from a non-adjacent block.

[0209] The index of the merge candidate in the merge candidate list can be assigned in a predetermined order. As an example, the index assigned to the adjacent merge candidate can have a smaller value than the index assigned to the non-adjacent merge candidate. Alternatively, based on the index of each block illustrated in FIG. 15 or FIG. 17, an index can be assigned to each merge candidate.

[0210] When a plurality of merge candidates are included in the merge candidate, at least one of the plurality of merge candidates can be selected (S1403). At this time, information indicating whether the motion information of the current block is derived from an adjacent merge candidate can be signaled via a bit stream. The information can be a 1-bit flag. As an example, a syntax element isAdjancentMergeFlag indicating whether the motion information of the current block is derived from an adjacent merge candidate can be signaled via a bit stream. When the value of the syntax element isAdjancentMergeFlag is 1, the motion information of the current block can be derived based on the adjacent merge candidate. On the other hand, when the value of the syntax element isAdjancentMergeFlag is 0, the motion information of the current block can be derived based on the non-adjacent merge candidate.

[0211] Table 1 shows a syntax table including the syntax element isAdjancentMergeFlag.

[0212]

Table 1

[0213] Information for identifying any one of a plurality of merge candidates can be signaled via a bit stream. As an example, information indicating the index of any one of the merge candidates included in the merge candidate list can be signaled via the bit stream.

[0214] When isAdjacentMergeflag is 1, the syntax element merge_idx for identifying any one of the adjacent merge candidates can be signaled. The maximum value of the syntax element merge_idx can be set as a value obtained by subtracting 1 from the number of adjacent merge candidates.

[0215] When isAdjacentMergeflag is 0, the syntax element NA_merge_idx for identifying any one of the non-adjacent merge candidates can be signaled. The syntax element NA_merge_idx indicates a value obtained by subtracting the number of adjacent merge candidates from the index of the non-adjacent merge candidate. The decoder can select a non-adjacent merge candidate by adding the number of adjacent merge candidates to the index specified by NA_merge_idx.

[0216] If the number of merge candidates included in the merge candidate list is smaller than the threshold value, the merge candidates included in the inter-region motion information table can be added to the merge candidate list. Here, the threshold value can be the maximum number of merge candidates that the merge candidate list can include or a value obtained by subtracting an offset from the maximum number of merge candidates. The offset can be a natural number such as 1 or 2. The inter-region motion information table can include merge candidates derived based on blocks encoded / decoded before the current block.

[0217] The inter-region motion information table includes merge candidates derived from blocks encoded / decoded based on inter-prediction within the current picture. As an example, the motion information of the merge candidates included in the inter-region motion information table can be set in the same way as the motion information of the blocks encoded / decoded based on inter-prediction. Here, the motion information can include at least one of a motion vector, a reference picture index, a prediction direction, or a bi-directional weight index.

[0218] For convenience of explanation, the merge candidates included in the inter-region motion information table are referred to as inter-region merge candidates.

[0219] The maximum number of merge candidates included in the inter-region motion information table can be predefined in the encoder and decoder. As an example, the maximum number of merge candidates included in the inter-region motion information table can be 1, 2, 3, 4, 5, 6, 7, 8, or more (e.g., 16).

[0220] Alternatively, information indicating the maximum number of merge candidates in the inter-region motion information table can be signaled via a bitstream. The said information can be signaled at the sequence, picture, or slice level.

[0221] Alternatively, the number of maximum merge candidates in the inter-region motion information table can be determined based on the size of the picture, the size of the slice, or the size of the coding tree unit.

[0222] The inter-region motion information table can be initialized on a picture, slice, tile, block, coding tree unit, or coding tree unit line (row or column) basis. As an example, when a slice is initialized, the inter-region motion information table is also initialized, and the inter-region motion information table may not contain any merge candidates.

[0223] Alternatively, information indicating whether to initialize the inter-region motion information table can also be signaled via the bitstream. The said information can be signaled at the slice, tile, block, or block level. The already configured inter-region motion information table can be used before the said information instructs to initialize the inter-region motion information table.

[0224] Alternatively, information regarding initial inter-region merge candidates can be signaled via the picture parameter set or the slice header. Even when a slice is initialized, the inter-region motion information table can include the initial inter-region merge candidates. Therefore, the inter-region merge candidates can be utilized for the block that is the first encoding / decoding target within the slice.

[0225] Encode / decode the blocks in the encoding / decoding order, and sequentially set the blocks encoded / decoded based on inter prediction as inter-region merge candidates in the encoding / decoding order.

[0226] FIG. 20 is a diagram showing a mode of updating the inter-region motion information table.

[0227] When inter prediction is performed on the current block (S2001), an inter-region merge candidate can be derived based on the current block (S2002). The motion information of the inter-region merge candidate can be set to be the same as the motion information of the current block.

[0228] When the inter-region motion information table is empty (S2003), the inter-region merge candidate derived based on the current block can be added to the inter-region motion information table (S2004).

[0229] When the inter-region motion information table already contains an inter-region merge candidate (S2003), a redundancy check can be performed on the motion information of the current block (or the inter-region merge candidate derived based thereon) (S2005). The redundancy check is for determining whether the motion information of the inter-region merge candidate already stored in the inter-region motion information table is the same as the motion information of the current block. The redundancy check can be executed for all the inter-region merge candidates already stored in the inter-region motion information table. Alternatively, the redundancy check can be executed for the inter-region merge candidates whose indexes in the inter-region merge candidates already stored in the inter-region motion information table are greater than or equal to a threshold or less than or equal to the threshold.

[0230] When the inter prediction merge candidate having the same motion information as the motion information of the current block is not included, the inter-region merge candidate derived based on the current block can be added to the inter-region motion information table (S2008). Whether the inter prediction merge candidates are the same can be determined based on whether the motion information (e.g., motion vector and / or reference picture index, etc.) of the inter prediction merge candidates is the same.

[0231] At this time, if the maximum number of inter-region merge candidates has already been stored in the inter-region motion information table (S2006), the oldest inter-region merge candidate is deleted (S2007), and the inter-region merge candidate derived based on the current block can be added to the inter-region motion information table (S2008).

[0232] The inter-region merge candidates can be identified by indexes respectively. When the inter-region merge candidate derived from the current block is added to the inter-region motion information table, the lowest index (for example, 0) is assigned to the inter-region merge candidate, and the indexes of the already stored inter-region merge candidates can be incremented by 1. At this time, if the maximum number of inter-prediction merge candidates has already been stored in the inter-region motion information table, the inter-region merge candidate with the maximum index is removed.

[0233] Alternatively, when the inter-region merge candidate derived from the current block is added to the inter-region motion information table, the largest index can be assigned to the inter-region merge candidate. As an example, if the number of the already stored inter-prediction merge candidates in the inter-region motion information table is smaller than the maximum value, an index with the same value as the number of the already stored inter-prediction merge candidates can be assigned to the inter-region merge candidate. Or, if the number of the already stored inter-prediction merge candidates in the inter-region motion information table is the same as the maximum value, an index obtained by subtracting 1 from the maximum value can be assigned to the inter-region merge candidate. Note that the inter-region merge candidate with the smallest index is removed, and the indexes of the remaining already stored inter-region merge candidates are decremented by 1.

[0234] FIG. 21 is a diagram showing a mode of updating the inter-region merge candidate table.

[0235] Assume that the inter-region merge candidate derived from the current block is added to the inter-region merge candidate table, and the largest index is assigned to the inter-region merge candidate. It is assumed that the inter-region merge candidate table already stores the maximum number of inter-region merge candidates.

[0236] When adding the inter-region merge candidate HmvpCand[n + 1] derived from the current block to the inter-region merge candidate table HmvpCandList, the inter-region merge candidate HmvpCand[0] with the smallest index among the already stored inter-region merge candidates can be deleted, and the indexes of the remaining inter-region merge candidates can be decreased by 1. Note that the index of the inter-region merge candidate HmvpCand[n+1] derived from the current block can be set to the maximum value (n in the example illustrated in FIG. 21).

[0237] If the same inter-region merge candidate as the inter-region merge candidate derived based on the current block is already stored (S2005), it may not be necessary to add the inter-region merge candidate derived based on the current block to the inter-region motion information table (S2009).

[0238] Alternatively, while adding the inter-region merge candidate derived based on the current block to the inter-region motion information table, the already stored inter-region merge candidate that is the same as the inter-region merge candidate can be deleted. In this case, the same effect as newly updating the indexes of the already stored inter-region merge candidates can be obtained.

[0239] FIG. 22 is a diagram showing an example in which the indexes of the already stored inter-region merge candidates are updated.

[0240] If there is an index hIdx of a previously stored inter-prediction merge candidate that is the same as the inter-region merge candidate mvCand derived based on the current block, the previously stored inter-prediction merge candidate can be deleted, and the index of the inter-prediction merge candidate whose index is greater than hIdx can be decreased by 1. As an example, in the example shown in FIG. 22, the same HmvpCand[2] as mvCand is deleted from the inter-region motion information table HvmpCandList, and it is illustrated that the indices from HmvpCand[3] to HmvpCand[n] decrease by 1 one by one.

[0241] And the inter-region merge candidate mvCand derived based on the current block can be added to the end of the inter-region motion information table.

[0242] Alternatively, the index assigned to a previously stored inter-region merge candidate that is the same as the inter-region merge candidate derived based on the current block can be updated. For example, the index of the previously stored inter-region merge candidate can be changed to the minimum value or the maximum value.

[0243] The motion information of the blocks included in a predetermined region can be set not to be added to the inter-region motion information table. As an example, the inter-region merge candidate derived based on the motion information of the blocks included in the merge processing region can be not added to the inter-region motion information table. The encoding / decoding order is not defined for the blocks included in the merge processing region, and it is inappropriate to use any one of these motion information during the inter-prediction of other blocks. Therefore, the inter-region merge candidate derived based on the blocks included in the merge processing region can be not added to the inter-region motion information table.

[0244] When motion compensation prediction is performed in the sub-block unit, an inter-region merge candidate can be derived based on the motion information of the representative sub-block among the plurality of sub-blocks included in the current block. As an example, when a sub-block merge candidate is used for the current block, an inter-region merge candidate can be derived based on the motion information of the representative sub-block among the sub-blocks.

[0245] The motion vectors of the sub-blocks can be derived in the following order. First, any one of the merge candidates included in the merge candidate list of the current block is selected, and an initial shift vector (shVector) can be derived based on the motion vector of the selected merge candidate. Then, the initial shift vector is added to the positions (xSb, ySb) of the reference samples (for example, the upper left sample or the middle position sample) of each sub-block in the coding block to derive a shifted sub-block whose position of the reference sample is (xColSb, yColSb). The following Equation 4 shows the mathematical formula for deriving the shifted sub-block.

[0246] [Equation 4]

[0247] [Number]

[0248] Then, the motion vector of the collocated block corresponding to the center position of the sub-block including (xColSb, yColSb) can be set as the motion vector of the sub-block including (xSb, ySb).

[0249] The representative sub-block can mean a sub-block including the upper left sample or the central sample of the current block.

[0250] FIG. 23 is a diagram showing the position of the representative sub-block.

[0251] Fig. 23(a) shows an example where a sub-block located at the upper left of the current block is set as the representative sub-block, and Fig. 23(b) shows an example where a sub-block located at the center of the current block is set as the representative sub-block. When motion compensation prediction is performed in the sub-block unit, the inter-region merge candidate of the current block can be derived based on the motion vector of the sub-block including the sample at the upper left of the current block or the sub-block including the sample at the center of the current block.

[0252] Based on the inter prediction mode of the current block, it is also possible to determine whether to use the current block as an inter-region merge candidate. As an example, a block encoded / decoded based on the affine motion model can be set so that it cannot be used as an inter-region merge candidate. Therefore, even if the current block is encoded / decoded by inter prediction, when the inter prediction mode of the current block is the affine prediction mode, the inter prediction motion information table may not be updated based on the current block.

[0253] Alternatively, the inter-region merge candidate can also be derived based on at least one sub-block vector among the sub-blocks included in the block encoded / decoded based on the affine motion model. As an example, the inter-region merge candidate can be derived using the sub-block located at the upper left, center, or upper right of the current block. Alternatively, the average value of the sub-block vectors of a plurality of sub-blocks can also be set as the motion vector of the inter-region merge candidate.

[0254] Alternatively, an inter-region merge candidate can also be derived based on the average value of the affine seed vectors of the blocks encoded / decoded based on the affine motion model. As an example, the average of at least one of the first affine seed vector, the second affine seed vector, or the third affine seed vector of the current block can be set as the motion vector of the inter-region merge candidate.

[0255] Alternatively, an inter-region motion information table can be constructed for each inter-prediction mode. As an example, at least one of an inter-region motion information table for a block encoded / decoded by intra-block copy, an inter-region motion information table for a block encoded / decoded based on a translational motion model, or an inter-region motion information table for a block encoded / decoded based on an affine motion model can be defined. Depending on the inter-prediction mode of the current block, any one of the plurality of inter-region motion information tables can be selected.

[0256] FIG. 24 is a diagram showing an example in which an inter-region motion information table is generated for each inter-prediction mode.

[0257] When a block is encoded / decoded based on a non-fine motion model, the inter-region merge candidate mvCand derived based on the block can be added to the inter-region non-fine motion information table HmvpCandList. On the other hand, when a block is encoded / decoded based on an affine motion model, the inter-region merge candidate mvAfCand derived based on the block can be added to the inter-region affine motion information table HmvpAfCandList.

[0258] In the inter-region merge candidates derived from the blocks encoded / decoded based on the affine motion model, the affine seed vector of the block can be stored. Therefore, the inter-region merge candidates can be used as merge candidates for deriving the affine seed vector of the current block.

[0259] In addition to the described inter-region motion information table, an additional inter-region motion information table can also be defined. In addition to the above-described inter-region motion information table (hereinafter referred to as the first inter-region motion information table), a long-term motion information table (hereinafter referred to as the second inter-region motion information table) can be defined. Here, the long-term motion information table includes long-term merge candidates.

[0260] When both the first inter-region motion information table and the second inter-region motion information table are empty, first, inter-region merge candidates can be added to the second inter-region motion information table. Only after the number of available inter-region merge candidates in the second inter-region motion information table reaches the maximum number, inter-region merge candidates can be added to the first inter-region motion information table.

[0261] Alternatively, one inter-prediction merge candidate can be added to both the second inter-region motion information table and the first inter-region motion information table.

[0262] At this time, the completed second inter-region motion information table can no longer be updated. Alternatively, when the decoded region is more than a predetermined ratio of the slice, the second inter-region motion information table can be updated. Alternatively, the second inter-region motion information table can be updated every N coding tree unit lines.

[0263] On the one hand, the first inter-region motion information table can be updated every time a block encoded / decoded by inter-prediction occurs. However, the inter-region merge candidates added to the second inter-region motion information table can also be set so as not to be used to update the first inter-region motion information table.

[0264] Information for selecting any one of the first inter-region motion information table or the second inter-region motion information table can be signaled via a bitstream. When the number of merge candidates included in the merge candidate list is smaller than a threshold value, the merge candidates included in the inter-region motion information table indicated by the information can be added to the merge candidate list.

[0265] Alternatively, the inter-region motion information table can also be selected based on the size, shape, inter-prediction mode, whether to perform bidirectional prediction, whether to refine the motion vector, or whether to perform triangular partitioning of the current block.

[0266] Alternatively, even if the inter-region merge candidates included in the first inter-region motion information table are added, when the number of merge candidates included in the merge candidate list is smaller than the maximum merge number, the inter-region merge candidates included in the second inter-region motion information table can be added to the merge candidate list.

[0267] FIG. 25 is a diagram showing an example in which the inter-region merge candidates included in the long-term motion information table are added to the merge candidate list.

[0268] If the number of merge candidates included in the merge candidate list is smaller than the maximum number, the inter-region merge candidates included in the first inter-region motion information table HmvpCandList can be added to the merge candidate list. Even if the inter-region merge candidates included in the first inter-region motion information table are added to the merge candidate list and the number of merge candidates included in the merge candidate list is smaller than the maximum number, the inter-region merge candidates included in the long-term motion information table HmvpLTCandList can be added to the merge candidate list.

[0269] Table 2 shows the process of adding the inter-region merge candidates included in the long-term motion information table to the merge candidate list.

[0270]

Table 2

[0271] The inter-region merge candidates can be set to include additional information in addition to the motion information. As an example, the inter-region merge candidates can additionally store the block size, shape, or block partition information. When constructing the merge candidate list of the current block, only the inter-prediction merge candidates having the same or similar size, shape, or partition information as the current block among the inter-prediction merge candidates can be used, or the inter-prediction merge candidates having the same or similar size, shape, or partition information as the current block can be first added to the merge candidate list.

[0272] Alternatively, an inter-region motion information table can be generated for each block size, shape, or partition information. Among the multiple inter-region motion information tables, the inter-region motion information table that matches the shape, size, or partition information of the current block can be used to generate the merge candidate list of the current block.

[0273] If the number of merge candidates included in the current block's merge candidate list is smaller than the threshold, the inter-region merge candidates included in the inter-region motion information table can be added to the merge candidate list. The addition process is executed in ascending or descending order of the index criterion. As an example, the merge candidate with the largest index can be added to the merge candidate list from the inter-region merge candidates.

[0274] When attempting to add the inter-region merge candidates included in the inter-region motion information table to the merge candidate list, a redundancy check can be performed between the inter-region merge candidates and the merge candidates already stored in the merge candidate list.

[0275] As an example, Table 3 shows the process in which the inter-region merge candidates are added to the merge candidate list.

[0276]

Table 3

[0277] The redundancy check can be performed only on a part of the inter-region merge candidates included in the inter-region motion information table. As an example, the redundancy check can be performed only on the inter-region merge candidates whose index is greater than or equal to the threshold or less than or equal to the threshold. Or the redundancy check can be performed only on the N merge candidates with the largest index or the N merge candidates with the smallest index.

[0278] Alternatively, the redundancy check can be performed only on a part of the merge candidates already stored in the merge candidate list. As an example, the redundancy check can be performed only on the merge candidates whose index is greater than or equal to the threshold or less than or equal to the threshold or the merge candidates derived from the blocks at specific positions. Here, the specific positions can include at least one of the left adjacent block, upper adjacent block, upper right adjacent block, or lower left adjacent block of the current block.

[0279] FIG. 26 is a diagram showing an example in which redundancy checking is performed only on a part of merge candidates.

[0280] When trying to add an inter-region merge candidate HmvpCand[j] to the merge candidate list, redundancy checking can be performed on the inter-region merge candidate with the two merge candidates mergeCandList[NumMerge - 2] and mergeCandList[NumMerge - 1] having the largest indices. Here, NumMerge can indicate the number of available spatial merge candidates and temporal merge candidates.

[0281] Unlike the illustrated example, when trying to add an inter-region merge candidate HmvpCand[j] to the merge candidate list, redundancy checking can also be performed on the inter-region merge candidate with the two merge candidates having the smallest indices. For example, it can be confirmed whether it is the same as HmvpCand[j] for mergeCandList[0] and mergeCandList[1]. Or redundancy checking can be performed only on merge candidates derived from a specific position. As an example, redundancy checking can be performed on at least one of the merge candidates derived from the peripheral block located on the left side of the current block or the merge candidates derived from the peripheral block located at the upper end of the current block. If there are no merge candidates derived from a specific position in the merge candidate list, the inter-region merge candidate can be added to the merge candidate list without redundancy checking.

[0282] When a merge candidate same as the first inter-region merge candidate is found, when performing redundancy checking on the second inter-region merge candidate, the redundancy checking with the merge candidate same as the first inter-region merge candidate can be omitted.

[0283] FIG. 27 is a diagram showing an example in which redundancy checking with a specific merge candidate is omitted.

[0284] When attempting to add an inter-region merge candidate HmvpCand[i] whose index is i to the merge candidate list, a redundancy check is performed between the inter-region merge candidate and the merge candidates already stored in the merge candidate list. At this time, if the same merge candidate mergeCandList[j] as the inter-region merge candidate HmvpCand[i] is found, instead of adding the inter-region merge candidate HmvpCand[i] to the merge candidate list, a redundancy check can be performed between the inter-region merge candidate HmvpCand[i-1] whose index is i-1 and the merge candidate. At this time, the redundancy check between the inter-region merge candidate HmvpCand[i-1] and the merge candidate mergeCandList[j] can be omitted.

[0285] As an example, in the example shown in FIG. 27, it was determined that HmvpCand[i] and mergeCandList[2] are the same. Therefore, HmvpCand[i] is not added to the merge candidate list, and a redundancy check of HmvpCand[i-1] can be performed. At this time, the redundancy check between HvmpCand[i-1] and mergeCandList[2] can be omitted.

[0286] When the number of merge candidates included in the merge candidate list of the current block is smaller than the threshold, in addition to the inter-region merge candidates, it may further include at least one of pairwise merge candidates or zero merge candidates. Pairwise merge candidates mean merge candidates whose motion vector is the average value of the motion vectors of two or more merge candidates, and zero merge candidates mean merge candidates whose motion vector is 0.

[0287] The merge candidate list of the current block can have merge candidates added in the following procedure.

[0288] Spatial merge candidate - Temporal merge candidate - Inter-region merge candidate - (Inter-region affine merge candidate) - Pairwise merge candidate - Zero merge candidate

[0289] A spatial merge candidate means a merge candidate derived from at least one of adjacent blocks or non - adjacent blocks, and a temporal merge candidate means a merge candidate derived from a previous reference picture. An inter - region affine merge candidate indicates an inter - region merge candidate derived from a block encoded / decoded by an affine motion model.

[0290] The inter - region motion information table can also be used in the motion vector prediction mode. As an example, when the number of motion vector prediction candidates included in the motion vector prediction candidate list of the current block is smaller than a threshold value, the inter - region merge candidates included in the inter - region motion information table can be set as the motion vector prediction candidates of the current block. Specifically, the motion vector of the inter - region merge candidate can be set as the motion vector prediction candidate.

[0291] When any one of the motion vector prediction candidates included in the motion vector prediction candidate list of the current block is selected, the selected candidate can be set as the motion vector predictor of the current block. Thereafter, after decoding the motion vector residual value of the current block, the motion vector of the current block can be obtained by adding the motion vector predictor and the motion vector residual value.

[0292] The motion vector prediction candidate list of the current block can be configured in the following order.

[0293] Spatial motion vector prediction candidates - Temporal motion vector prediction candidates - Inter - region merge candidates - (Inter - region affine merge candidates)-Zero motion vector prediction candidates

[0294] The spatial motion vector prediction candidate means a motion vector prediction candidate derived from at least one of adjacent blocks or non - adjacent blocks, and the temporal motion vector prediction candidate means a motion vector prediction candidate derived from the previous reference picture. The inter - region affine merge candidate indicates an inter - region motion vector prediction candidate derived from a block encoded / decoded by an affine motion model. The zero - motion vector prediction candidate indicates a candidate whose motion vector value is 0.

[0295] When a merge candidate of the current block is selected, the motion vector of the selected merge candidate is set as the initial motion vector, and a motion - compensated prediction of the current block can be performed using the motion vector derived by adding or subtracting an offset vector to the initial motion vector. Defining adding or subtracting an offset vector to the motion vector of a merge candidate to derive a new motion vector as a merge offset vector encoding method.

[0296] Information indicating whether to use the merge offset encoding method can be signaled via a bitstream. The information can be a 1 - bit flag merge_offset_vector_flag. As an example, a value of 1 for merge_offset_vector_flag indicates that the merge offset vector encoding method is applied to the current block. When the merge offset vector encoding method is applied to the current block, the motion vector of the current block can be derived by adding or subtracting an offset vector to the motion vector of the merge candidate. A value of 0 for merge_offset_vector_flag indicates that the merge offset vector encoding method is not applied to the current block. When the merge offset vector encoding method is not applied, the motion vector of the merge candidate can be set as the motion vector of the current block.

[0297] The flag can be signaled only if the value of the skip flag indicating whether skip mode is applied is true, or if the value of the merge flag indicating whether merge mode is applied is true. As an example, if the value of skip_flag indicating that skip mode is applied to the current block is 1, or if the value of merge_flag indicating that merge mode is applied to the current block is 1, merge_offset_vector_flag can be encoded and signaled.

[0298] If it is determined that the merge offset encoding method is applied to the current block, at least one of the information identifying any one of the merge candidates included in the merge candidate list, the information indicating the size of the offset vector, or the information indicating the direction of the offset vector can be additionally signaled.

[0299] Information for determining the maximum number of merge candidates that the merge candidate list can include can be signaled via the bitstream. As an example, the maximum number of merge candidates that the merge candidate list can include can be set as a natural number of 6 or less.

[0300] If it is determined that the merge offset encoding method is applied to the current block, only the maximum number of merge candidates already set can be set as the initial motion vector of the current block. That is, depending on whether the merge offset encoding method is applied, the number of available merge candidates for the current block can be adaptively determined. As an example, when the value of merge_offset_vector_flag is set to 0, the maximum number of available merge candidates for the current block is set to M, but when the value of merge_offset_vector_flag is set to 1, the maximum number of available merge candidates for the current block can be set to N. Here, M indicates the maximum number of merge candidates that the merge candidate list can include, and N indicates a natural number that is the same as M or smaller than M.

[0301] For example, when M is 6 and N is 2, among the merge candidates included in the merge candidate list, it can be set that the two merge candidates with the smallest indexes are available for the current block. Therefore, the motion vector of the merge candidate with an index value of 0 or the motion vector of the merge candidate with an index value of 1 can be set as the initial motion vector of the current block. If M and N are the same (for example, when M and N are 2), it can be set that all the merge candidates included in the merge candidate list are available for the current block.

[0302] Alternatively, whether an adjacent block can be used as a merge candidate can be determined based on whether a merge offset vector encoding method is applied to the current block. For example, when the value of merge_offset_vector_flag is 1, it can be set that at least one of the adjacent blocks adjacent to the upper right corner or the lower left corner of the current block cannot be used as a merge candidate. Therefore, when a merge offset vector encoding method is applied to the current block, the motion vector of at least one of the adjacent blocks adjacent to the upper right corner or the lower left corner of the current block cannot be set as the initial motion vector. Or, when the value of merge_offset_vector_flag is 1, the temporal adjacent blocks of the current block can be set not to be available as merge candidates.

[0303] When the merge offset vector encoding method is applied to the current block, it can be set not to use at least one of the pairwise merge candidates or zero merge candidates. Therefore, when the value of merge_offset_vector_flag is 1, even if the number of merge candidates included in the merge candidate list is smaller than the maximum number, at least one of the pairwise merge candidates or zero merge candidates can be set not to be added to the merge candidate list.

[0304] The motion vector of the merge candidate can be set as the initial motion vector of the current block. At this time, when the number of available merge candidates for the current block is plural, information specifying any one of the plural merge candidates can be signaled via the bit stream. As an example, when the maximum number of merge candidates that the merge candidate list can include is greater than 1, information merge_idx indicating any one of the plural merge candidates can be signaled via the bit stream. That is, by the merge offset encoding method, the merge candidate can be specified via the information merge_idx for specifying any one of the plural merge candidates. The initial motion vector of the current block can be set as the motion vector of the merge candidate indicated by merge_idx.

[0305] On one hand, when the number of merge candidates available for the current block is 1, the signaling of information for identifying the merge candidate can be omitted. As an example, when the maximum number of merge candidates that the merge candidate list can contain is not greater than 1, the signaling of the information merge_idx for identifying the merge candidate can be omitted. That is, under the merge offset encoding method, when the merge candidate list contains one merge candidate, the encoding of the information merge_idx for identifying the merge candidate is omitted, and the initial motion vector can be determined based on the merge candidate contained in the merge candidate list. The motion vector of the merge candidate can be set as the initial motion vector of the current block.

[0306] As another example, after determining the merge candidates for the current block, it is also possible to determine whether to apply the merge offset vector encoding method to the current block. As an example, when the maximum number of merge candidates that the merge candidate can contain is greater than 1, the information merge_idx for identifying any one of the merge candidates can be signaled. After selecting a merge candidate based on merge_idx, the merge_offset_vector_flag indicating whether the merge offset vector encoding method is applied to the current block can be decoded. Table 4 is a diagram showing the syntax table according to the above-described embodiment.

[0307]

Table 4

[0308] As another example, after determining the merge candidates for the current block, it is also possible to determine whether to apply the merge offset vector encoding method to the current block, provided that the index of the determined merge candidate is less than the maximum number of merge candidates that can be used when applying the merge offset vector encoding method. As an example, it is possible to encode and signal a merge_offset_vector_flag indicating whether to apply the merge offset vector encoding method to the current block, provided that the value of the index information merge_idx is less than N. If the value of the index information merge_idx is the same as or greater than N, the encoding of the merge_offset_vector_flag can be omitted. When the encoding of the merge_offset_vector_flag is omitted, it can be determined that the merge offset vector encoding method is not applied to the current block.

[0309] Alternatively, after determining the merge candidates for the current block, it is possible to determine whether to apply the merge offset vector encoding method to the current block by considering whether the determined merge candidate has bidirectional motion information or unidirectional motion information. As an example, it is possible to encode and signal a merge_offset_vector_flag indicating whether to apply the merge offset vector encoding method to the current block, provided that the value of the index information merge_idx is less than N and the merge candidate selected by the index information has bidirectional motion information. Alternatively, it is possible to encode and signal a merge_offset_vector_flag indicating whether to apply the merge offset vector encoding method to the current block, provided that the value of the index information merge_idx is less than N and the merge candidate selected by the index information has unidirectional motion information.

[0310] Alternatively, it is possible to determine whether to apply the merge offset vector encoding method based on at least one of the size, shape of the current block, or whether the current block touches the boundary of the coding tree unit. If at least one of the size, shape of the current block, or whether the current block touches the boundary of the coding tree unit has already been set and does not meet the conditions, the encoding of the merge_offset_vector_flag indicating whether to apply the merge offset vector encoding method to the current block can be omitted.

[0311] When a merge candidate is selected, the motion vector of the merge candidate can be set as the initial motion vector of the current block. Then, information indicating the magnitude of the offset vector and information indicating the direction of the offset vector can be decoded to determine the offset vector. The offset vector can have a horizontal component or a vertical component.

[0312] The information indicating the magnitude of the offset vector can be index information indicating any one of the vector magnitude candidates. As an example, index information distance_idx indicating any one of the vector magnitude candidates can be signaled via the bitstream. Table 5 shows the binarization of the index information distance_idx and the values of the variable DistFromMergeMV for determining the magnitude of the offset vector based on distance_idx.

[0313]

Table 5

[0314] The magnitude of the offset vector can be derived by dividing the variable DistFromMergeMV by a preset value. Equation 5 shows an example of determining the magnitude of the offset vector.

[0315] [Formula 5]

[0316] [Number]

[0317] According to the above Formula 5, the value obtained by dividing the variable DistFromMegeMV by 4 or the value obtained by shifting the variable DistFromMergeMV 2 bits to the left can be set as the magnitude of the offset vector.

[0318] A larger or smaller number of vector magnitude candidates than those shown in Table 5 can be used, or the range of motion vector offset size candidates can be set to be different from the example shown in Table 5. As an example, it can be set such that the magnitude of the horizontal or vertical component of the offset vector is not greater than the distance of 2 samples. Table 6 shows the binarization of the index information distance_idx and the values of the variable DistFromMergeMV for determining the magnitude of the offset vector by distance_idx.

[0319] [Table 6]

[0320] Alternatively, based on the motion vector accuracy, the range of motion vector offset size candidates can be set differently. As an example, when the accuracy of the motion vector of the current block is fractional-pel, the value of the variable DistFromMergeMV corresponding to the value of the index information distance_idx can be set to 1, 2, 4, 8, 16, etc. Here, fractional-pel includes at least one of 1 / 16 pel, Octo-pel, Quarter-pel, or Half-pel. On the other hand, when the accuracy of the motion vector of the current block is integer-pel, the value of the variable DistFromMergeMV corresponding to the value of the index information distance_idx can be set to 4, 8, 16, 32, 64, etc. That is, depending on the accuracy of the motion vector of the current block, the table referred to for determining the variable DistFromMergeMV can be set differently.

[0321] As an example, when the accuracy of the motion vector of the current block or merge candidate is Quarter-pel, Table 5 can be used to derive the variable DistFromMergeMV pointed to by distance_idx. On the other hand, when the accuracy of the motion vector of the current block or merge candidate is integer-pel, a value obtained by multiplying the value of the variable DistFromMergeMV pointed to by distance_idx in Table 5 by N times (for example, 4 times) can be derived as the value of the variable DistFromMergeMV.

[0322] Information for determining the accuracy of motion vectors can be signaled via a bitstream. As an example, the information can be signaled at the sequence, picture, slice, or block level. Thus, the range of vector magnitude candidates can be set to vary according to information regarding the accuracy of motion vectors signaled via the bitstream. Alternatively, based on the merge candidates of the current block, the accuracy of the motion vector can be determined. As an example, the accuracy of the motion vector of the current block can be set the same as the accuracy of the motion vectors of the merge candidates.

[0323] Alternatively, information for determining the search range of the offset vector can be signaled via a bitstream. Based on the search range, at least one of the number of vector magnitude candidates, the minimum value or the maximum value among the vector magnitude candidates can be determined. As an example, a flag merge_offset_vector_flag for determining the search range of the offset vector can be signaled via the bitstream. The information can be signaled via a sequence header, a picture header, or a slice header.

[0324] As an example, when the value of merge_offset_extend_range_flag is 0, the magnitude of the offset vector can be set not to exceed 2. Thus, the maximum value of DistFromMergeMV can be set to 8. On the other hand, when the merge_offset_extend_range_flag value is 1, the magnitude of the offset vector can be set not to exceed a distance of 32 samples. Thus, the maximum value of DistFromMergeMV can be set to 128.

[0325] The magnitude of the offset vector can be determined using a flag indicating whether the magnitude of the offset vector is greater than a threshold. As an example, a flag distance_flag indicating whether the magnitude of the offset vector is greater than a threshold can be signaled via a bit stream. The threshold can be 1, 2, 4, 8, or 16. As an example, distance_flag being 1 indicates that the magnitude of the offset vector is greater than 4. On the other hand, distance_flag being 0 indicates that the magnitude of the offset vector is 4 or less.

[0326] If the magnitude of the offset vector is greater than the threshold, the difference value between the magnitude of the offset vector and the threshold can be derived using the index information distance_idx. Alternatively, if the magnitude of the offset vector is less than or equal to the threshold, the magnitude of the offset vector can be determined using the index information distance_idx. Table 7 is a syntax table showing the aspects of the encoding of distance_flag and distance_idx.

[0327] [Table 7]

[0328] Equation 6 shows an example of deriving the variable DistFromMergeMV for determining the magnitude of the offset vector using distance_flag and distance_idx.

[0329] [Equation 6]

[0330] [Number]

[0331] In Equation 6, the value of distance_flag can be set to 1 or 0. The value of distance_idx can be set to 1, 2, 4, 8, 16, 32, 64, 128, etc. N represents a coefficient determined by a threshold value. As an example, when the threshold value is 4, N can be set to 16.

[0332] The information indicating the direction of the offset vector can be index information indicating any one of the vector direction candidates. As an example, index information direction_idx indicating any one of the vector direction candidates can be signaled via a bit stream. Table 8 shows the binarization of the index information direction_idx and the direction of the offset vector based on direction_idx.

[0333]

Table 8

[0334] In Table 8, sign[0] indicates the horizontal direction, and sign[1] indicates the vertical direction. +1 indicates that the value of the x component or y component of the offset vector is +, and -1 indicates that the value of the x component or y component of the offset vector is -. Equation 7 shows an example of determining the offset vector based on the magnitude and direction of the offset vector.

[0335] [Equation 7]

[0336]

Number

[0337] In Equation 7, offsetMV[0] indicates the vertical direction component of the offset vector, and offsetMV[1] indicates the horizontal direction component of the offset vector.

[0338] FIG. 28 is a diagram showing an offset vector based on the value of distance_idx indicating the magnitude of the offset vector and direction_idx indicating the direction of the offset vector.

[0339] As shown in the example illustrated in FIG. 28, the magnitude and direction of the offset vector can be determined based on the values of distance_idx and direction_idx. The maximum magnitude of the offset vector can be set so as not to exceed a threshold value. Here, the threshold value can have a value already defined in the encoder and the decoder. As an example, the threshold value can be a distance of 32 samples. Alternatively, the threshold value can be determined according to the magnitude of the initial motion vector. As an example, the threshold value in the horizontal direction can be set based on the magnitude of the horizontal direction component of the initial motion vector, and the threshold value in the vertical direction can be set based on the magnitude of the vertical direction component of the initial motion vector.

[0340] When the merge candidate has bidirectional motion information, the L0 motion vector of the merge candidate can be set as the L0 initial motion vector of the current block, and the L1 motion vector of the merge candidate can be set as the L1 initial motion vector of the current block. At this time, considering the output order difference value between the L0 reference picture of the merge candidate and the current picture (hereinafter referred to as the L0 difference value) and the output order difference value between the L1 reference picture of the merge candidate and the current picture (hereinafter referred to as the L1 difference value), the L0 offset vector and the L1 offset vector can be determined.

[0341] First, when the signs of the L0 difference value and the L1 difference value are the same, the L0 offset vector and the L1 offset vector can be set in the same manner. On the other hand, when the signs of the L0 difference value and the L1 difference value are different, the L1 offset vector can be set in the opposite direction to the L0 offset vector.

[0342] The magnitudes of the L0 offset vector and the L1 offset vector can be set to be the same. Alternatively, based on the L0 difference value and the L1 difference value, the magnitude of the L1 offset vector can be determined by scaling the L0 offset vector.

[0343] As an example, Equation 8 shows the L0 offset vector and the L1 offset vector when the signs of the L0 difference value and the L1 difference value are the same.

[0344] [Equation 8]

[0345] [Number]

[0346] In Equation 8, offsetMVL0[0] represents the horizontal component of the L0 offset vector, and offsetMVL0[1] represents the vertical component of the L0 offset vector. offsetMVL1[0] represents the horizontal component of the L1 offset vector, and offsetMVL1[1] represents the vertical component of the L1 offset vector.

[0347] Equation 9 shows the L0 offset vector and the L1 offset vector when the signs of the L0 difference value and the L1 difference value are different.

[0348] [Equation 9]

[0349] [Number]

[0350] More than four vector direction candidates can also be defined. Tables 9 and 10 show examples where eight vector direction candidates are defined.

[0351] [Table 9]

[0352]

Table 10

[0353] In Tables 9 and 10, if the absolute values of sign[0] and sign[1] are greater than 0, it indicates that the offset vector is in the diagonal direction. When Table 8 is used, the magnitudes of the x-axis and y-axis components of the offset vector in the diagonal direction are set to abs(offsetMV). However, when Table 9 is used, the magnitudes of the x-axis and y-axis components of the offset vector in the diagonal direction can be set to abs(offsetMV / 2).

[0354] FIG. 29 is a diagram showing an offset vector according to the value of distance_idx indicating the magnitude of the offset vector and direction_idx indicating the direction of the offset vector.

[0355] FIG. 29(a) is an illustration when Table 8 is applied, and FIG. 29(b) is an illustration when Table 9 is applied.

[0356] Information for determining at least one of the number or magnitude of vector direction candidates can be signaled via a bitstream. As an example, a flag merge_offset_direction_range_flag for determining vector direction candidates can be signaled via a bitstream. The flag can be signaled at the sequence, picture, or slice level. As an example, when the value of the flag is 0, the four vector direction candidates illustrated in Table 8 can be used. On the other hand, when the value of the flag is 1, the eight vector direction candidates illustrated in Table 9 or Table 10 can be used.

[0357] Alternatively, based on the magnitude of the offset vector, at least one of the number or magnitude of the vector direction candidates can be determined. As an example, when the value of the variable DistFromMergeMV for determining the magnitude of the offset vector is the same as or smaller than the threshold value, the eight vector direction candidates illustrated in Table 9 or Table 10 can be used. On the other hand, when the value of the variable DistFromMergeMV is larger than the threshold value, the four vector direction candidates illustrated in Table 8 can be used.

[0358] Alternatively, based on the x-component value MVx and the y-component value MVy of the initial motion vector, at least one of the number or magnitude of the vector direction candidates can be determined. As an example, when the MVx and MVy differences or the absolute values of the differences are less than or equal to the threshold value, the eight vector direction candidates illustrated in Table 9 or Table 10 can be used. On the other hand, when the MVx and MVy differences or the absolute values of the differences are larger than the threshold value, the four vector direction candidates illustrated in Table 8 can be used.

[0359] The motion vector of the current block can be derived by adding the offset vector to the initial motion vector. Equation 10 shows an example of determining the motion vector of the current block.

[0360] [Equation 10]

[0361] [Number]

[0362] In Equation 10, mvL0 represents the L0 motion vector of the current block, and mvL1 represents the L1 motion vector of the current block. mergeMVL0 represents the L0 initial motion vector of the current block (i.e., the L0 motion vector of the merge candidate), and mergeMVL1 represents the L1 initial motion vector of the current block. [0] represents the horizontal component of the motion vector, and [1] represents the vertical component of the motion vector.

[0363] Even when a coding block is divided into a plurality of sub-units and then inter prediction is performed on each sub-unit, the merge offset vector encoding method can be applied. Here, performing inter prediction in units of sub-units can include at least one of ATMVP (Advanced Temporal Motion Vector Prediction), STMVP (Spatial Temporal Motion Vector Predition), or triangular partitioning techniques.

[0364] As an example, in the ATMVP method, the initial motion vector can be derived as follows.

[0365] First, an initial shift vector can be derived using the motion vectors of merge candidates derived from adjacent blocks adjacent to the coding block. Then, using the initial shift vector, a shifted block of the sub-blocks included in the coding block can be derived. Equation 11 is a diagram showing the position of the shifted block.

[0366] [Equation 11]

[0367] [Number]

[0368] In Equation 11, (xColSb, yColSb) indicates the position of the sample at the upper left end of the shifted block, and (xSb, ySb) indicates the position of the sample at the upper left end of the sub-block. shVector indicates the shift vector.

[0369] Once the shift block is determined, the motion vector of the collocated block at the same position as the shift block in the collocated picture can be set as the motion vector of the sub-block. That is, the motion vector of the collocated block including the sample at the (xColSb, yColSb) position in the collocated block can be set as the motion vector of the sub-block including the sample at the (xSb, ySb) position.

[0370] When the triangular partitioning technique is applied, the coding block can be divided into triangular sub-units. As an example, the coding block can be divided into two sub-units by a diagonal line connecting the upper left end and the lower right end of the coding block or a diagonal line connecting the upper right end and the lower left end of the coding block.

[0371] FIG. 30 is a diagram showing a division state of a coding block when the triangular division technique is applied.

[0372] The motion information of each triangular sub-unit can be specified by a merge candidate. For each sub-unit, index information indicating any one of the merge candidates can be signaled. As an example, the index information merge_1st_idx of the first sub-unit can specify the merge candidate of the first sub-unit, and the index information merge_2nd_idx of the second sub-unit can specify the merge candidate of the second sub-unit.

[0373] The initial motion vector of each sub-unit can be determined individually. As an example, when an affine motion model is applied to the coding block, the affine vector of the sub-block derived from the affine seed vector of the coding block can be set as the initial motion vector of the sub-block. The motion vector of each sub-block can be derived by adding or subtracting an offset vector to the initial motion vector.

[0374] When a merge offset vector encoding method is applied to a coding block divided into a plurality of sub-units, the plurality of sub-units can be set to use the same offset vector. That is, the initial motion vector of each of the plurality of sub-units can be corrected using the same offset vector.

[0375] Alternatively, the coding block can be divided into a plurality of sub-units, and the offset vector of each sub-unit can be determined individually. Therefore, at least one offset vector of the sub-units can be set to be different from the offset vectors of other sub-units.

[0376] FIG. 31 is a diagram showing an example in which the offset vectors of each sub-unit are set to be different.

[0377] As in the example shown in FIG. 31, information distance_idx indicating the magnitude of the offset vector and information direction_idx indicating the direction of the offset vector can be encoded and signaled for each sub-unit.

[0378] Alternatively, the magnitudes of the offset vectors of all sub-units can be set to be the same, but the directions of the offset vectors of the sub-units can be set individually. As an example, the values of distance_idx signaled at the coding level of the sub-units can be set to be shared, and direction_idx can be encoded and signaled for each sub-unit.

[0379] Alternatively, the directions of the offset vectors of all sub-units can be set to be the same, but the magnitudes of the offset vectors of the sub-units can be set individually. As an example, the values of direction_idx signaled at the coding level of the sub-units can be set to be shared, and distance_idx can be encoded and signaled for each sub-unit.

[0380] It is also possible to apply the merge offset vector encoding method only to some of the plurality of sub-units generated by splitting the coding block. As an example, when the current block is split into a first sub-unit and a second sub-unit, the motion vector of the first sub-unit can be set to be the same as the motion vector of the merge candidate, and the motion vector of the second sub-unit can be derived by adding an offset vector to the motion vector of the merge candidate.

[0381] Instead of signaling information for determining the offset vector, the decoder can also derive the offset vector. Specifically, the offset vector can be derived using the average value of the horizontal gradient of the predicted samples included in the sub-block and the average value of the vertical gradient.

[0382] Here, the gradient can be derived based on the difference between the restored sample corresponding to the predicted sample in the reference picture and the adjacent samples adjacent to the restored sample. As an example, the horizontal gradient can indicate the difference between the restored sample and the restored samples adjacent to the left and / or right, and the vertical gradient can indicate the difference between the restored sample and the restored samples adjacent to the top and / or bottom.

[0383] Among the merge candidates included in the merge candidate list, a merge candidate having a motion vector derived by adding or subtracting an offset vector to the motion vector of the reference merge candidate can be added to the merge candidate list. A merge candidate having a motion vector derived by adding or subtracting an offset vector to the motion vector of the reference merge candidate can be referred to as a refined merge candidate.

[0384] The remaining motion information excluding the motion vector of the refined merge candidate can be set to be the same as that of the reference merge candidate.

[0385] FIG. 32 is a diagram showing motion vector candidates that a refine merge candidate can take.

[0386] When the motion vector of the reference merge candidate is (MvLX[0], MvLX[1]), the motion vector of the refine merge candidate can be derived by adding or subtracting an offset to at least one of the x component or the y component of the motion vector of the reference merge candidate. As an example, the motion vector of the refine merge candidate can be set to (MvLX[0] + M, MvLX[1]), (MvLX[0] - M, MvLX[1]), (MvLX[0], MvLX[1] + M), or (MvLX[0], MvLX[1] - M). M indicates the magnitude of the offset vector.

[0387] The reference merge candidate can be a merge candidate having an already defined index value in the merge candidate list. As an example, in the merge candidates included in the merge candidate list, the merge candidate with the smallest index value (i.e., the merge candidate with an index value of 0), or the merge candidate with the largest index value can be set as the reference merge candidate. Alternatively, the inter-region motion information table's inter-region merge candidate with the smallest index value or the inter-region merge candidate with the largest index value can be set as the reference merge candidate.

[0388] Alternatively, in a merge candidate having bidirectional motion information, the merge candidate with the smallest index value can be set as the reference merge candidate. That is, when searching for candidate blocks in order, the first discovered bidirectional merge candidate can be set as the reference merge candidate.

[0389] Based on the size, shape of the current block, or whether it touches the boundary of the coding tree unit, basic merge candidates can also be selected. As an example, when the current block is square, or when the current block is non-square with a height greater than the width, a merge candidate with an index of 0 or a merge candidate derived from an adjacent block located at the upper end of the current block can be set as the reference merge candidate. When the current block is non-square with a width greater than the height, a merge candidate with an index of 1 or a merge candidate derived from an adjacent block located on the left side of the current block can be set as the reference merge candidate.

[0390] Alternatively, information for specifying the reference merge candidate can be signaled via a bitstream. The information can be index information that specifies any one of the merge candidates included in the merge candidate list.

[0391] Information indicating whether to use the refined merge candidate can be signaled via a bitstream. The information can be a 1-bit flag. When the value of the flag is 1, the refined merge candidate generated based on the reference merge candidate can be added to the merge candidate list. On the other hand, when the value of the above flag is 0, the merge candidate list does not include the refined merge candidate.

[0392] Alternatively, when the number of merge candidates already added to the merge candidate list is less than the maximum number of merge candidates that the merge candidate list can include, the refined merge candidate can be added to the merge candidate list. Here, the merge candidates already added can include at least one of a spatial merge candidate, a temporal merge candidate, an inter-region merge candidate, or a pairwise merge candidate. As an example, when the number of at least one of the spatial merge candidate, the temporal merge candidate, or the inter-region merge candidate included in the merge candidate list is below a threshold, the refined merge candidate can be added to the merge candidate list.

[0393] Alternatively, when the number of merge candidates already added to the merge candidate list is equal to or greater than a threshold value, refined merge candidates can also be used.

[0394] Depending on whether refined merge candidates are used or not, the maximum number of merge candidates that the merge candidate list can contain can be set to be different. As an example, when set not to use refined merge candidates, the maximum number of merge candidates that the merge candidate list can contain is set to N, but when set to use refined merge candidates, the maximum number of merge candidates that the merge candidate list can contain can be set to N + n.

[0395] Refined merge candidates can have an index larger than that of the merge candidates already added to the merge candidate list. As an example, Table 11 shows a configuration example of the merge candidate list.

[0396]

Table 11

[0397] In Table 11, mergeCand[X] indicates a merge candidate with an index of X. mvLx[0] indicates the x-component motion vector of the reference merge candidate, and mvLx[1] indicates the y-component motion vector of the reference merge candidate. As an example, when the reference merge candidate is mergeCand[0], mvLx[0] and mvLx[1] can indicate the motion vector of mergeCand[0].

[0398] The magnitude M of the offset vector can already be defined in the encoder and the decoder. As an example, the magnitude M of the offset vector can be set to an integer of 4 or less, such as 1 or 4.

[0399] Alternatively, information for determining an offset vector can be signaled via a bitstream. The information can be signaled at the sequence, picture, slice, or block level. As an example, the offset vector can be determined by using at least one of the information distance_idx for determining the magnitude of the offset vector described above or the information direction_idx for determining the direction of the offset vector.

[0400] As shown in the example of Table 11, at least one refined merge candidate derived based on a reference merge candidate can be added to the merge candidate list. If there is a merge candidate in the already added merge candidates whose motion information is the same as that of the refined merge candidate, the refined merge candidate cannot be added to the merge candidate list. As an example, if the refined merge candidate derived based on the reference merge candidate mergeCand[0] is the same as any one of mergeCand[1] to mergeCand[5], the refined merge candidate cannot be added to the merge candidate list.

[0401] Alternatively, if there is a merge candidate whose motion information is the same as the refine merge candidate, the offset vector can be changed, or the merge candidate whose motion information is the same as the refine merge candidate can be reset to the refine merge candidate to re-induce the refine merge candidate. As an example, if the motion information of the refine merge candidate mergeCand[6] induced based on the reference merge candidate mergeCand[0] is the same as that of the merge candidate mergeCand[2], the motion vector of the refine merge candidate mergeCand[6] can be changed to a value obtained by adding or subtracting the offset vector from the motion vector of the merge candidate mergeCand[2]. As an example, the motion vector of mergeCand[6] can be changed from (mergeCand[0]_mxLx[0]+M, mergeCand[0]_mvLx[1]) to (mergeCand[2]_mxLx[0]+ M, mergeCand[2]_mvLx[1]). Here, mergeCand[X]_mvLx indicates the motion vector of the merge candidate with index X.

[0402] As another example, an offset vector can be determined by using a merge refinement offset list including at least one or more merge offset candidates. When a merge candidate specified by the index information of the current block is a reference merge candidate, an offset vector can be determined by using the merge refinement offset list. Then, the offset vector can be added to or subtracted from the motion vector of the merge candidate to derive the motion vector of the current block. The reference merge candidate can be a merge candidate having a predefined index value within the merge candidate list. As an example, among the merge candidates included in the merge candidate list, the merge candidate with the smallest index value (i.e., the merge candidate with an index value of 0) or the merge candidate with the largest index value can be set as the reference merge candidate. Alternatively, the inter-region merge candidate with the smallest index value or the inter-region merge candidate with the largest index value within the inter-region motion information table can be set as the reference merge candidate.

[0403] FIG. 33 is a diagram showing the configuration of a merge refinement offset list.

[0404] In FIG. 33, assume that the reference merge candidate is a merge candidate with an index of 6.

[0405] When the index of the merge candidate specified by the index information merge_idx indicating any one of the merge candidates is not 6, the motion vector of the merge candidate can be set as the motion vector of the current block.

[0406] On the one hand, when the index of the merge candidate specified by the index information merge_idx is 6, an offset vector can be derived using the merge refinement offset list. Index information MrgOffset_idx that specifies any one of the merge offset candidates included in the merge refinement offset list can be signaled via a bitstream.

[0407] Once the offset vector is specified, the offset vector can be added to or subtracted from the motion vector of the reference merge candidate to derive the motion vector of the current block.

[0408] The merge refinement offset list can include at least one or more merge offset candidates. As an example, the number of merge offset candidates included in the merge refinement offset list can be 4, 8, or 16.

[0409] FIG. 34 and FIG. 35 are diagrams showing the offset vectors specified by the merge offset candidates.

[0410] FIG. 34 shows an example when the number of merge offset candidates is 8, and FIG. 35 shows an example when the number of merge offset candidates is 16.

[0411] As shown in the example shown in FIG. 34(a), the offset vectors indicated by the merge offset candidates can be set such that the absolute value of the horizontal motion vector and / or the absolute value of the vertical motion vector have fixed values. Alternatively, as shown in the example shown in FIG. 35, among the merge offset candidates, the offset vectors indicated by the merge offset candidates whose index is smaller than the threshold value can be set such that the absolute value of the horizontal motion vector and / or the absolute value of the vertical motion vector have a first value, and the offset vectors indicated by the other merge offset candidates can be set such that the absolute value of the horizontal motion vector and / or the absolute value of the vertical motion vector have a second value.

[0412] Alternatively, as in the example shown in FIG. 34(b), the offset vector indicated by the merge offset candidate can be set such that the sum of the absolute value of the horizontal motion vector and the absolute value of the vertical motion vector has a fixed value.

[0413] Multiple reference merge candidates can also be set. As an example, among the merge candidates included in the merge candidate list, the two merge candidates with the smallest indices can be set as the reference merge candidates. Therefore, when the index of the merge candidate specified by the index information merge_idx is 0 or 1, the offset vector can be derived using the merge refinement offset list. Alternatively, among the merge candidates included in the merge candidate list, the merge candidate with the smallest index and the merge candidate with the largest index among the merge candidates included in the inter-region merge candidate list can be set as the reference merge candidates.

[0414] In the motion vector prediction mode, the motion vector of the current block can be derived by adding the motion difference vector to the motion prediction vector. The motion prediction vector of the current block can be determined based on a motion vector prediction candidate list including at least one motion prediction vector candidate. As an example, any one of the motion prediction vector candidates can be set as the motion prediction vector of the current block.

[0415] The motion vector prediction candidate can be derived based on at least one of the spatial adjacent blocks of the current block or the temporal adjacent blocks of the current block.

[0416] FIG. 36 is a diagram showing candidate blocks used to derive motion vector prediction candidates.

[0417] Spatial adjacent blocks can include an upper adjacent block located at the upper end of the current block and a left adjacent block located on the left side of the current block. The upper adjacent block can include at least one of block B0 including samples at the position (xCb + CbW, yCb - 1), block B1 including samples at the position (xCb + CbW - 1, yCb - 1), block B2 including samples at the position (xCb - 1, yCb - 1), or block B3 including samples at the position (xCb, yCb - 1). Here, (xCb, yCb) indicates the position of the upper left sample of the current block, and CbW indicates the width of the current block. The left adjacent block can include at least one of block A0 including samples at the position (xCb - 1, yCb + CbH), block A1 including samples at the position (xCb - 1, yCb + CbH - 1), or block A2 including samples at the position (xCb - 1, yCb). Here, CbH indicates the height of the current block.

[0418] Temporal adjacent blocks can include at least one of block C0 including the central sample of a block having the same position and size as the current block within the collocated block or block C1 including samples adjacent to the upper right corner of the block.

[0419] The maximum number of motion vector prediction candidates that the motion vector prediction candidate list can include can be two. The derivation order of the motion vector prediction candidates is as follows.

[0420] 1. When at least one of left adjacent block A0 or left adjacent block A1 is available, set the motion vector of the available block as a motion vector prediction candidate.

[0421] 2. When at least one of upper adjacent block B0, upper adjacent block B1, or upper adjacent block B2 is available, set the motion vector of the available block as a motion vector prediction candidate.

[0422] 3. If the temporal adjacent block is available, set the temporal motion vector as a motion vector prediction candidate.

[0423] 4. Set the zero motion vector as a motion vector prediction candidate.

[0424] Alternatively, if the number of motion vector prediction candidates derived according to the order of 1 to 3 is less than 2, the motion vectors included in the inter-region motion information table can also be set as motion vector prediction candidates. When the inter-region motion information table is available, the motion vector prediction candidates can be derived according to the following order.

[0425] 1. If at least one of the left adjacent blocks A0 or A1 is available, set the motion vector of the available block as a motion vector prediction candidate.

[0426] 2. If at least one of the upper adjacent blocks B0, B1, or B2 is available, set the motion vector of the available block as a motion vector prediction candidate.

[0427] 3. If the temporal adjacent block is available, set the temporal motion vector as a motion vector prediction candidate.

[0428] 4. Set the motion vectors included in the inter-region motion information table as motion vector prediction candidates.

[0429] 5. Set the zero motion vector as a motion vector prediction candidate.

[0430] It is also possible to add a motion vector prediction candidate having a motion vector obtained by adding or subtracting an offset vector to or from the motion vector of the reference motion vector prediction candidate to the motion vector prediction candidate list. A motion vector prediction candidate having a motion vector obtained by adding or subtracting an offset vector to or from the motion vector of the reference motion vector prediction candidate can be referred to as a refined motion vector prediction candidate.

[0431] FIG. 37 is a diagram showing a motion vector candidate that can be set as a refined motion vector prediction candidate.

[0432] When the motion vector of the reference motion vector prediction candidate is (MvpLX[0], MvpLX[1]), the motion vector of the refined motion vector prediction candidate can be derived by adding or subtracting an offset to at least one of the x-component or y-component of the motion vector of the reference motion vector prediction candidate. As an example, the motion vector of the refined motion vector prediction candidate can be set to (MvpLX[0]+M, MvpLX[1]), (MvpLX[0]-M, MvpLX[1]), (MvpLX[0], MvpLX[1]+M) or (MvpLX[0], MvpLX[1]-M). M represents the magnitude of the offset vector.

[0433] The magnitude M of the offset vector can already be defined in the encoder and decoder. As an example, the magnitude M of the offset vector can be set to an integer of 4 or less, such as 1 or 4.

[0434] Alternatively, information for determining an offset vector can be signaled via a bitstream. The information can be signaled at the sequence, picture, slice, or block level. As an example, the offset vector can be determined using at least one of the information distance_idx for determining the size of the offset vector described above or the information direction_idx for determining the direction of the offset vector.

[0435] A reference motion vector prediction candidate can be a motion vector prediction candidate having a predefined index value within a motion vector prediction candidate list. As an example, among the motion vector prediction candidates included in the motion vector prediction candidate list, a motion vector prediction candidate with an index value of 0 or a motion vector prediction candidate with an index value of 1 can be set as the reference motion vector prediction candidate.

[0436] As another example, the offset vector can be determined using a merge refinement offset list that includes at least one or more prediction vector offset candidates. When the motion vector prediction candidate specified by the index information of the current block is the reference motion vector prediction candidate, the offset vector can be determined using the prediction vector refinement offset list. Then, the offset vector can be added to or subtracted from the motion vector of the motion vector prediction candidate to derive the motion prediction vector of the current block. A reference motion vector prediction candidate can be a motion vector prediction candidate having a predefined index value within a motion vector prediction candidate list. As an example, among the motion vector prediction candidates included in the motion vector prediction candidate list, a motion vector prediction candidate with the smallest index value or a motion vector prediction candidate with the largest index value can be set as the reference motion vector prediction candidate.

[0437] When calculating the offset vector using the prediction vector offset refinement list, the maximum number of prediction vector candidates that the prediction vector candidate list can contain can also be set to a value greater than 2.

[0438] FIG. 38 is a diagram showing the configuration of the prediction vector refinement offset list.

[0439] In FIG. 38, it is assumed that the reference prediction vector candidate is the prediction vector candidate with index 2.

[0440] If the index of the prediction vector candidate specified by the index information AMVPcand_idx indicating any one of the prediction vector candidates is not 2, the motion vector of the prediction vector candidate can be set as the motion prediction vector of the current block.

[0441] On the other hand, if the index of the prediction vector candidate specified by the index information AMVPcand_idx is 2, the offset vector can be derived using the prediction vector refinement offset list. The index information AMVPOffset_idx specifying any one of the prediction vector offset candidates included in the prediction vector refinement offset list can be signaled via the bit stream.

[0442] Once the offset vector is specified, the offset vector can be added to or subtracted from the motion vector of the reference prediction vector candidate to derive the motion prediction vector of the current block.

[0443] Even when the coding block is encoded based on an affine motion model, a motion vector refinement technique can be utilized. As an example, when an affine motion vector prediction mode is applied, an affine seed vector of a coding block can be derived by adding an affine seed difference vector to an affine seed prediction vector. Here, the affine seed prediction vector can be derived based on the affine seed vectors of spatially adjacent blocks or temporally adjacent blocks of the coding block. The affine seed difference vector can be determined based on information signaled from the bitstream. At this time, the same affine seed difference vector can be applied to all control points. Alternatively, information for determining the affine seed vector can be signaled to each control point.

[0444] When an affine vector of a sub-block is derived based on the affine seed vector of a coding block, after setting the affine vector as an initial motion vector, an offset vector can be derived. The motion vector of each sub-block can be derived by adding or subtracting an offset vector to / from the initial motion vector.

[0445] Instead of signaling information for determining the offset vector, the decoder can also derive the offset vector. Specifically, the offset vector can be derived using the average value of the horizontal gradient and the average value of the vertical gradient of the predicted samples included in the sub-block.

[0446] Intra prediction predicts the current block using the restored samples encoded / decoded around the current block. At this time, the restored samples before the in-loop filter is applied can be used for the intra prediction of the current block.

[0447] The intra prediction technique includes intra prediction based on a matrix and general intra prediction considering the directionality with surrounding restored samples. Information indicating the intra prediction technique of the current block can be signaled via a bitstream. The information can be a 1-bit flag. Alternatively, the intra prediction technique of the current block can be determined based on at least one of the position, size, shape of the current block, or the intra prediction technique of adjacent blocks. As an example, when the current block exists beyond the picture boundary, the intra prediction based on the matrix can be set not to be applied to the current block.

[0448] The intra prediction based on a matrix is a method of obtaining a predicted block of the current block based on the matrix product between the matrix already stored in the encoder / decoder and the restored samples around the current block. Information for specifying any one of the plurality of already stored matrices can be signaled via a bitstream. The decoder can determine the matrix for the intra prediction of the current block based on the information and the size of the current block.

[0449] The general intra prediction is a method of obtaining a predicted block of the current block based on a non-directional intra prediction mode or a directional intra prediction mode.

[0450] It is possible to subtract the predicted video from the original video to derive the induced residual video. At this time, when the residual video is changed to the frequency domain, even if the high-frequency components among the frequency components are removed, the subjective image quality of the video does not significantly decrease. Therefore, if the value of the high-frequency component is slightly changed or the value of the high-frequency component is set to 0, visual distortion does not occur significantly, and there is an effect of improving the compression efficiency. Reflecting the above characteristics, the current block can be transformed to decompose the residual video into two-dimensional frequency components. The transformation can be performed using a transformation technique such as DCT (Discrete Cosine Transform) or DST (Discrete Sine Transform).

[0451] After converting the current block using DCT or DST, the converted current block can be converted again. At this time, the conversion based on DCT or DST can be defined as the first conversion, and the conversion of the block to which the first conversion is applied again can be defined as the second conversion.

[0452] The first conversion can be executed using any one of a plurality of conversion core candidates. As an example, the first conversion can be executed using any one of DCT2, DCT8, or DCT7.

[0453] Different conversion cores may be used for the horizontal and vertical directions. Information indicating the combination of the horizontal conversion core and the vertical conversion core can also be signaled via the bitstream.

[0454] The execution units of the first conversion and the second conversion can be different. As an example, an 8×8 block can execute the first conversion, and a 4×4 size sub-block of the converted 8×8 block can execute the second conversion. At this time, the conversion coefficients of the remaining area where the second conversion is not executed can also be set to 0.

[0455] Alternatively, the 4×4 block can perform a first transformation, and an 8×8 sized region including the transformed 4×4 block can perform a second transformation.

[0456] Information indicating whether to perform the second transformation can be signaled via a bitstream.

[0457] The decoder can perform an inverse transformation of the second transformation (second inverse transformation) and perform an inverse transformation of the first transformation (first inverse transformation) on the execution result. As a result of performing the second inverse transformation and the first inverse transformation, the residual signal of the current block can be obtained.

[0458] Quantization is for reducing the energy of a block, and the quantization process includes a process of dividing the transform coefficient by a specific constant value. The constant value can be induced by a quantization parameter, and the quantization parameter can be defined as a value between 1 and 63.

[0459] When the encoder performs transformation and quantization, the decoder can obtain a residual block through inverse quantization and inverse transformation. The decoder can add the prediction block and the residual block to obtain the restored block of the current block.

[0460] When the restored block of the current block is obtained, the information loss generated in the quantization and encoding process can be reduced through in-loop filtering. The in-loop filter can include at least one of a deblocking filter, a sample adaptive offset filter (SAO), or an adaptive loop filter (ALF).

[0461] Applying the embodiments described centering on the decoding process or the encoding process to the encoding process or the decoding process is included in the scope of the present invention. Changing the embodiments described in a predetermined order to an order different from the described order is also included in the scope of the present invention.

[0462] Although the above-described embodiments are described based on a series of steps or flowcharts, this does not limit the chronological order of the present invention, and they can be executed simultaneously or in other orders as necessary. Also, in the above-described embodiments, each of the components (e.g., units, modules, etc.) constituting the block diagram can be embodied in a hardware device or software, and a plurality of components can be combined and embodied in one hardware device or software. The above-described embodiments can be embodied in the form of program commands executed via various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium can include program commands, data files, data structures, etc. alone or in combination. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs, DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands such as ROMs, RAMs, and flash memories. The hardware device can be configured to operate using one or more software modules to execute the processing according to the present invention, and vice versa.

[0463] The present invention can be applied to an electronic device that encodes / decodes video.

Claims

1. 1. A video decoding method, comprising: determining whether a merge offset vector encoding method is applied to the current block; generating a merge candidate list for the current block; identifying a merge candidate for the current block based on the merge candidate list; deriving a motion vector for the current block based on the merging candidates; Including, When the merge offset vector encoding method is applied to the current block, a motion vector of the current block is derived by adding an offset vector to the motion vector of the merge candidate; when the maximum number of merge candidates that the merge candidate list can include is plural, the merge candidate of the current block is selected based on index information decoded from a bitstream indicating any one of the merge candidates; when the maximum number is 1, the merge candidate is determined without decoding the index information; At least one of the maximum value and the minimum value of the vector magnitude candidates is set to be different based on the value of a flag indicating the numerical value of the vector magnitude candidate.

1. A video decoding method comprising:

2. the magnitude of the offset vector is determined based on first index information that identifies one of the vector magnitude candidates.

2. The video decoding method of claim 1.

3. The flag is signaled at the picture level.

2. The video decoding method of claim 1.

4. the direction of the offset vector is determined based on second index information that identifies one of vector direction candidates; Video decoding method according to any one of claims 1 to 3.

5. 1. A video encoding method, comprising: determining whether a merge offset vector encoding method is applied to the current block; generating a merge candidate list for the current block; identifying a merge candidate for the current block based on the merge candidate list; deriving a motion vector for the current block based on the merging candidates; Including, When the merge offset vector encoding method is applied to the current block, a motion vector of the current block is derived by adding an offset vector to the motion vector of the merge candidate; when the maximum number of merge candidates that the merge candidate list can include is plural, index information indicating the merge candidate of the current block among the merge candidates is encoded; when the maximum number is 1, encoding of the index information is omitted; At least one of the maximum value and the minimum value of the vector magnitude candidates is set to be different based on the value of a flag indicating the numerical value of the vector magnitude candidate. A video encoding method comprising:

6. encoding first index information for identifying a vector magnitude candidate indicating a magnitude of the offset vector among a plurality of vector magnitude candidates; 6. The video encoding method of claim 5.

7. further comprising encoding a flag indicating a value of the vector magnitude candidate, the flag being encoded at a picture level.

6. The video encoding method of claim 5.

8. further comprising encoding second index information for identifying a vector direction candidate indicating the direction of the offset vector among a plurality of vector direction candidates; Video encoding method according to any one of claims 5 to 7.

9. 1. A video decoder comprising: a processor and a memory, The memory is used to store a computer program, and the processor is used to perform the method according to any one of claims 1 to 4 by executing the computer program stored in the memory.

1. A video decoder comprising:

10. 1. A video encoder comprising: a memory and a processor, The memory is used to store a computer program, and the processor is used to execute the method according to any one of claims 5 to 8 by executing the computer program stored in the memory.

1. A video encoder comprising:

11. A video decoding device, comprising: an inter prediction unit that determines whether a merge offset vector encoding method is applied to a current block, generates a merge candidate list for the current block, identifies merge candidates for the current block based on the merge candidate list, and derives a motion vector for the current block based on the merge candidates; When the merge offset vector encoding method is applied to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector of the merge candidate; if the maximum number of merge candidates that the merge candidate list can include is multiple, the merge candidate of the current block is selected based on index information decoded from a bitstream indicating any one of the merge candidates; if the maximum number is 1, the merge candidate is determined without decoding the index information; and at least one of the maximum or minimum values ​​of the vector magnitude candidates is set to be different based on the value of a flag indicating the numerical value of the vector magnitude candidate. A video decoding device comprising:

12. 1. A video encoding device, comprising: an inter prediction unit that determines whether a merge offset vector encoding method is applied to a current block, generates a merge candidate list for the current block, identifies merge candidates for the current block based on the merge candidate list, and derives a motion vector for the current block based on the merge candidates; When the merge offset vector encoding method is applied to the current block, the motion vector of the current block is derived by adding an offset vector to the motion vector of the merge candidate; if the maximum number of merge candidates that the merge candidate list can include is plural, index information indicating the merge candidate of the current block among the merge candidates is encoded; if the maximum number is 1, encoding of the index information is omitted; and at least one of maximum or minimum values ​​of vector magnitude candidates is set to be different based on the value of a flag indicating the numerical value of the vector magnitude candidate. A video encoding device comprising:

13. A computer-readable storage medium storing computer programs / instructions and bitstreams, comprising: When the computer program is executed by a processor, it performs the video encoding method according to any one of claims 5 to 8 to generate the bitstream. A computer-readable storage medium comprising:

Images